2024-09-10T20:43:44.263 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T20:43:44.288 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T20:43:44.385 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899646' kernel: client: sha1: distro flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBEUoUz3i6PF0gAAabJwG8jwJikCVV3giS/YH+d4yGyMpy/DXqoCBZJ7DHm7gBv7DfcuIUtesx+vzZ67MsILLUo= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfOF6s7nzNdfiW4xYf/FCa7c9RDGB0pLey7fpHbIGmQsfdybuiKL0NCNPRnf+y22m4f2vOjz/h6h5JvMfvJnWg= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ3Z5hChwSgwOsmu6IEUeU6SL7DoRwiaUt+SfJtIN5U6TyZUOml25gCU1S/JHN+vofckpp4NioFeMD0vY03T1bw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T20:43:44.385 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T20:43:44.386 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T20:43:44.386 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T20:43:44.388 INFO:teuthology.task.internal:Checking packages... 2024-09-10T20:43:44.412 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T20:43:44.412 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T20:43:44.412 INFO:teuthology.packaging:ref: None 2024-09-10T20:43:44.412 INFO:teuthology.packaging:tag: None 2024-09-10T20:43:44.412 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T20:43:44.413 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:43:44.413 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T20:43:44.569 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T20:43:44.571 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T20:43:44.577 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T20:43:44.577 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T20:43:44.583 INFO:teuthology.task.internal:Saving configuration 2024-09-10T20:43:44.603 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T20:43:44.610 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T20:43:44.643 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:38:48.964513', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBEUoUz3i6PF0gAAabJwG8jwJikCVV3giS/YH+d4yGyMpy/DXqoCBZJ7DHm7gBv7DfcuIUtesx+vzZ67MsILLUo='} 2024-09-10T20:43:44.675 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:38:48.963126', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfOF6s7nzNdfiW4xYf/FCa7c9RDGB0pLey7fpHbIGmQsfdybuiKL0NCNPRnf+y22m4f2vOjz/h6h5JvMfvJnWg='} 2024-09-10T20:43:44.705 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 20:38:48.961431', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ3Z5hChwSgwOsmu6IEUeU6SL7DoRwiaUt+SfJtIN5U6TyZUOml25gCU1S/JHN+vofckpp4NioFeMD0vY03T1bw='} 2024-09-10T20:43:44.705 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T20:43:44.711 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-10T20:43:44.712 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-10T20:43:44.712 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-10T20:43:44.712 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T20:43:44.820 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8c5390fbe0>, signals=[15]) 2024-09-10T20:43:44.821 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T20:43:44.827 INFO:teuthology.task.internal:Opening connections... 2024-09-10T20:43:44.827 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-09-10T20:43:44.829 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:43:44.911 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-09-10T20:43:44.912 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:43:44.994 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-09-10T20:43:44.995 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:43:45.079 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T20:43:45.086 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-09-10T20:43:45.105 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-09-10T20:43:45.105 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-09-10T20:43:45.162 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-09-10T20:43:45.163 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T20:43:45.164 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T20:43:45.164 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T20:43:45.165 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-09-10T20:43:45.200 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-09-10T20:43:45.219 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-09-10T20:43:45.219 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T20:43:45.277 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-09-10T20:43:45.278 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T20:43:45.278 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T20:43:45.278 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T20:43:45.278 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-09-10T20:43:45.308 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-09-10T20:43:45.327 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-09-10T20:43:45.327 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-09-10T20:43:45.385 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T20:43:45.386 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T20:43:45.387 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-09-10T20:43:45.416 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T20:43:45.426 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T20:43:45.432 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T20:43:45.432 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:43:45.435 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:43:45.437 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-09-10T20:43:45.454 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T20:43:45.461 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T20:43:45.462 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:43:45.492 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:43:45.494 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-09-10T20:43:45.519 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T20:43:45.624 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T20:43:45.638 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T20:43:45.638 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-10T20:43:45.638 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-10T20:43:45.638 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:43:45.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:43:45.656 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-10T20:43:45.713 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-503.el9.x86_64 2024-09-10T20:43:45.713 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-503.el9.x86_64 2024-09-10T20:43:45.713 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-09-10T20:43:49.586 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.2 MB 00:02 2024-09-10T20:43:51.687 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-10T20:43:56.846 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.5 MB 00:01 2024-09-10T20:43:58.646 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 19 kB 00:00 2024-09-10T20:43:59.775 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T20:44:05.364 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T20:44:07.504 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T20:44:07.505 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T20:44:07.588 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T20:44:07.595 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:44:07.596 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-09-10T20:44:07.596 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-09-10T20:44:07.596 INFO:teuthology.orchestra.run.smithi033.stdout:Install 4 Packages 2024-09-10T20:44:07.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:44:07.596 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 79 M 2024-09-10T20:44:07.597 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 126 M 2024-09-10T20:44:07.597 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-09-10T20:44:08.007 INFO:teuthology.orchestra.run.smithi033.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 78 kB/s | 23 kB 00:00 2024-09-10T20:44:17.948 INFO:teuthology.orchestra.run.smithi033.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 2.9 MB/s | 29 MB 00:09 2024-09-10T20:44:22.856 INFO:teuthology.orchestra.run.smithi033.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 1.0 MB/s | 16 MB 00:15 2024-09-10T20:44:25.439 INFO:teuthology.orchestra.run.smithi033.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 1.9 MB/s | 35 MB 00:17 2024-09-10T20:44:25.439 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:44:25.439 INFO:teuthology.orchestra.run.smithi033.stdout:Total 4.4 MB/s | 79 MB 00:17 2024-09-10T20:44:26.151 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-09-10T20:44:26.557 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-09-10T20:44:26.557 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-09-10T20:44:27.551 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-09-10T20:44:27.552 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-09-10T20:44:30.160 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-09-10T20:44:30.832 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:44:30.914 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:44:33.729 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:44:33.869 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:44:38.993 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:44:39.037 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:44:44.171 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:45:23.147 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:45:23.188 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:45:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:45:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T20:45:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T20:45:23.552 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T20:45:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T20:45:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:45:24.440 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-09-10T20:45:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T20:45:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T20:45:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T20:45:24.440 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T20:45:24.441 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:45:24.441 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-09-10T20:45:24.767 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:45:24.851 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:45:24.852 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:45:24.852 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T20:45:24.852 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T20:45:24.852 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T20:45:24.852 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-10T20:45:24.852 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi033.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T20:45:24.853 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T20:45:24.911 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T20:45:24.911 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T20:45:24.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo rpm -qi grub2-tools 2024-09-10T20:45:24.943 INFO:teuthology.orchestra.run.smithi033.stdout:Name : grub2-tools 2024-09-10T20:45:24.943 INFO:teuthology.orchestra.run.smithi033.stdout:Epoch : 1 2024-09-10T20:45:24.943 INFO:teuthology.orchestra.run.smithi033.stdout:Version : 2.06 2024-09-10T20:45:24.943 INFO:teuthology.orchestra.run.smithi033.stdout:Release : 82.el9 2024-09-10T20:45:24.943 INFO:teuthology.orchestra.run.smithi033.stdout:Architecture: x86_64 2024-09-10T20:45:24.943 INFO:teuthology.orchestra.run.smithi033.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T20:45:24.943 INFO:teuthology.orchestra.run.smithi033.stdout:Group : Unspecified 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Size : 8274670 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:License : GPLv3+ 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Packager : builder@centos.org 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Vendor : CentOS 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Summary : Support tools for GRUB. 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout:Description : 2024-09-10T20:45:24.944 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:45:24.945 INFO:teuthology.orchestra.run.smithi033.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T20:45:24.945 INFO:teuthology.orchestra.run.smithi033.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T20:45:24.945 INFO:teuthology.orchestra.run.smithi033.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T20:45:24.945 INFO:teuthology.orchestra.run.smithi033.stdout:hardware devices. 2024-09-10T20:45:24.945 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:45:24.945 INFO:teuthology.orchestra.run.smithi033.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T20:45:24.946 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T20:45:24.946 INFO:teuthology.task.kernel:Updating grub on smithi033 to boot 5.14.0-505.el9.x86_64 2024-09-10T20:45:24.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T20:45:25.525 INFO:teuthology.orchestra.run.smithi033.stderr:Generating grub configuration file ... 2024-09-10T20:45:26.843 INFO:teuthology.orchestra.run.smithi033.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T20:45:26.874 INFO:teuthology.orchestra.run.smithi033.stderr:done 2024-09-10T20:45:26.876 DEBUG:teuthology.orchestra.run.smithi033:> mktemp 2024-09-10T20:45:26.891 INFO:teuthology.orchestra.run.smithi033.stdout:/tmp/tmp.PHzuzD2KHk 2024-09-10T20:45:26.892 DEBUG:teuthology.orchestra.run.smithi033:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PHzuzD2KHk 2024-09-10T20:45:26.967 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0666 /tmp/tmp.PHzuzD2KHk 2024-09-10T20:45:27.111 DEBUG:teuthology.orchestra.remote:smithi033:/tmp/tmp.PHzuzD2KHk is 6KB 2024-09-10T20:45:27.133 DEBUG:teuthology.orchestra.run.smithi033:> rm -fr /tmp/tmp.PHzuzD2KHk 2024-09-10T20:45:27.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T20:45:27.210 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T20:45:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T20:45:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T20:45:27.211 INFO:teuthology.orchestra.run.smithi033.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T20:45:27.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T20:45:27.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo shutdown -r now 2024-09-10T20:45:57.359 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T20:45:57.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-10T20:45:57.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:46:15.873 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-10T20:46:24.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-10T20:46:24.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:46:43.261 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.33 2024-09-10T20:46:55.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-10T20:46:55.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:46:58.679 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-10T20:46:59.150 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-10T20:46:59.150 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-10T20:46:59.151 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-09-10T20:46:59.166 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-505.el9.x86_64 2024-09-10T20:46:59.166 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T20:46:59.166 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T20:46:59.166 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T20:47:00.167 DEBUG:teuthology.parallel:result is None 2024-09-10T20:47:00.167 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T20:47:00.178 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T20:47:00.179 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:47:00.183 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:47:00.185 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T20:47:00.204 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T20:47:00.210 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T20:47:00.216 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T20:47:00.217 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:47:00.238 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:47:00.243 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T20:47:00.345 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T20:47:00.354 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T20:47:00.354 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:47:00.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:47:00.371 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T20:47:00.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:47:00.389 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:47:00.391 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:47:00.414 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T20:47:00.473 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:47:00.477 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:47:00.487 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:47:00.490 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:47:00.502 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:47:00.507 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T20:47:00.509 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T20:47:00.516 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T20:47:00.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:47:00.519 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:47:00.521 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T20:47:00.548 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T20:47:00.557 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T20:47:00.557 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:47:00.585 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:47:00.588 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T20:47:00.604 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:47:00.693 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:47:00.732 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:47:00.824 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:47:00.857 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:47:00.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:47:00.944 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:47:00.997 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:47:01.043 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:47:01.138 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:47:01.171 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:47:01.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:47:01.238 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:47:01.291 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T20:47:01.344 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:47:01.400 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T20:47:01.432 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:47:01.432 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T20:47:01.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-09-10T20:47:01.503 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-09-10T20:47:01.507 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-09-10T20:47:01.542 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T20:47:01.544 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T20:47:01.580 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T20:47:01.884 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T20:47:01.892 INFO:teuthology.task.internal:Starting timer... 2024-09-10T20:47:01.892 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T20:47:01.938 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T20:47:01.946 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T20:47:01.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-09-10T20:47:01.991 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-09-10T20:47:01.994 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-09-10T20:47:02.050 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-09-10T20:47:02.052 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-09-10T20:47:02.091 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-09-10T20:47:02.093 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T20:47:02.093 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-09-10T20:47:02.119 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-09-10T20:47:02.120 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-09-10T20:47:02.140 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-09-10T20:47:02.140 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-09-10T20:47:02.160 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-09-10T20:47:02.160 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-09-10T20:47:02.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T20:47:02.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:47:02.187 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T20:47:02.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:47:02.216 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T20:47:02.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T20:47:02.241 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T20:47:02.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-09-10T20:47:02.283 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-09-10T20:47:02.320 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-09-10T20:47:02.352 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T20:47:02.361 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T20:47:02.375 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T20:47:02.376 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi104.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T20:50:58.568 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-09-10T20:50:58.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-10T20:50:58.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:50:58.655 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-09-10T20:50:58.737 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-09-10T20:50:58.737 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-10T20:50:58.738 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:50:58.822 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-09-10T20:50:58.902 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-09-10T20:50:58.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-10T20:50:58.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T20:50:58.988 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-09-10T20:50:59.077 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-09-10T20:50:59.078 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T20:50:59.089 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T20:50:59.090 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T20:50:59.090 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T20:50:59.092 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T20:50:59.093 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T20:50:59.095 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T20:50:59.095 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T20:50:59.123 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T20:50:59.127 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T20:50:59.139 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T20:50:59.143 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T20:50:59.148 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T20:50:59.163 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T20:50:59.166 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-09-10T20:50:59.169 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-09-10T20:50:59.178 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-09-10T20:50:59.180 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-09-10T20:50:59.189 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-09-10T20:50:59.193 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T20:50:59.195 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T20:50:59.200 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-09-10T20:50:59.208 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T20:50:59.209 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T20:50:59.215 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T20:50:59.229 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T20:50:59.268 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-10T20:50:59.270 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-09-10T20:50:59.272 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T20:50:59.272 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.273 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.274 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.274 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.291 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-10T20:50:59.294 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T20:50:59.294 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-10T20:50:59.294 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.294 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.295 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.295 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T20:50:59.296 INFO:teuthology.run_tasks:Running task install... 2024-09-10T20:50:59.303 DEBUG:teuthology.task.install:project ceph 2024-09-10T20:50:59.303 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T20:50:59.304 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T20:50:59.304 INFO:teuthology.task.install:Using flavor: default 2024-09-10T20:50:59.309 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T20:50:59.309 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T20:50:59.309 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T20:50:59.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:50:59.312 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T20:50:59.312 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:50:59.313 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T20:50:59.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:50:59.474 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T20:50:59.474 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T20:50:59.487 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T20:50:59.487 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T20:50:59.488 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T20:50:59.488 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T20:50:59.595 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T20:50:59.595 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:50:59.595 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T20:50:59.608 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T20:50:59.608 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:50:59.608 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T20:50:59.616 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T20:50:59.616 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:50:59.616 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T20:50:59.622 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T20:50:59.622 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T20:50:59.634 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T20:50:59.634 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T20:50:59.644 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T20:50:59.644 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T20:50:59.689 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T20:50:59.701 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T20:50:59.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T20:50:59.764 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T20:50:59.775 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T20:50:59.788 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T20:50:59.828 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-09-10T20:50:59.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-09-10T20:50:59.839 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-09-10T20:50:59.841 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-09-10T20:50:59.855 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-09-10T20:50:59.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-09-10T20:51:00.053 INFO:teuthology.orchestra.run.smithi104.stdout:45 files removed 2024-09-10T20:51:00.067 INFO:teuthology.orchestra.run.smithi174.stdout:45 files removed 2024-09-10T20:51:00.080 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T20:51:00.092 INFO:teuthology.orchestra.run.smithi033.stdout:45 files removed 2024-09-10T20:51:00.094 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T20:51:00.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T20:51:00.627 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-10T20:51:00.650 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 302 kB/s | 84 kB 00:00 2024-09-10T20:51:00.699 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 301 kB/s | 84 kB 00:00 2024-09-10T20:51:00.911 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-10T20:51:00.933 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 100 kB/s | 24 kB 00:00 2024-09-10T20:51:01.008 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 92 kB/s | 24 kB 00:00 2024-09-10T20:51:01.186 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-10T20:51:01.201 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-10T20:51:01.283 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-10T20:51:01.795 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-10T20:51:02.050 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-10T20:51:02.676 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 5.7 MB/s | 8.2 MB 00:01 2024-09-10T20:51:03.935 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 28 MB/s | 20 MB 00:00 2024-09-10T20:51:04.444 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 30 MB/s | 20 MB 00:00 2024-09-10T20:51:05.354 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 20 MB 00:02 2024-09-10T20:51:08.979 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-09-10T20:51:09.494 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-10T20:51:10.637 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-10T20:51:11.420 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 19 kB 00:00 2024-09-10T20:51:12.125 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 17 MB/s | 23 MB 00:01 2024-09-10T20:51:12.549 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T20:51:12.570 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.5 MB 00:03 2024-09-10T20:51:14.229 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-10T20:51:15.934 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-10T20:51:17.614 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T20:51:18.188 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T20:51:19.955 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:51:19.956 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:51:19.968 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T20:51:19.969 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T20:51:19.970 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T20:51:19.970 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T20:51:19.972 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T20:51:19.975 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T20:51:19.976 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T20:51:20.039 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-09-10T20:51:20.060 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-09-10T20:51:20.060 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-09-10T20:51:20.060 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-09-10T20:51:20.060 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-09-10T20:51:20.060 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T20:51:20.061 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T20:51:20.062 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T20:51:20.063 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T20:51:20.064 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T20:51:20.065 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T20:51:20.066 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T20:51:20.067 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T20:51:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T20:51:20.069 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T20:51:20.070 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T20:51:20.071 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T20:51:20.072 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T20:51:20.073 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout:Install 149 Packages 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 8 Packages 2024-09-10T20:51:20.074 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:51:20.075 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 240 M 2024-09-10T20:51:20.075 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-09-10T20:51:20.633 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:51:20.633 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:51:20.645 INFO:teuthology.orchestra.run.smithi033.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T20:51:20.646 INFO:teuthology.orchestra.run.smithi033.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T20:51:20.647 INFO:teuthology.orchestra.run.smithi033.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T20:51:20.648 INFO:teuthology.orchestra.run.smithi033.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T20:51:20.649 INFO:teuthology.orchestra.run.smithi033.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T20:51:20.652 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T20:51:20.653 INFO:teuthology.orchestra.run.smithi033.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T20:51:20.720 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-09-10T20:51:20.736 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-09-10T20:51:20.736 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-09-10T20:51:20.736 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-09-10T20:51:20.736 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-09-10T20:51:20.736 INFO:teuthology.orchestra.run.smithi033.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T20:51:20.737 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T20:51:20.738 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T20:51:20.739 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T20:51:20.740 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T20:51:20.741 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T20:51:20.742 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T20:51:20.743 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T20:51:20.744 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T20:51:20.745 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T20:51:20.746 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T20:51:20.747 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T20:51:20.748 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout:======================================================================================= 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout:Install 149 Packages 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 8 Packages 2024-09-10T20:51:20.749 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:51:20.750 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 240 M 2024-09-10T20:51:20.750 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-09-10T20:51:20.765 INFO:teuthology.orchestra.run.smithi174.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 49 kB/s | 6.6 kB 00:00 2024-09-10T20:51:20.999 INFO:teuthology.orchestra.run.smithi174.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 3.8 MB/s | 900 kB 00:00 2024-09-10T20:51:21.107 INFO:teuthology.orchestra.run.smithi174.stdout:(3/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 12 MB/s | 5.5 MB 00:00 2024-09-10T20:51:21.132 INFO:teuthology.orchestra.run.smithi174.stdout:(4/157): ceph-immutable-object-cache-19.3.0-485 1.1 MB/s | 145 kB 00:00 2024-09-10T20:51:21.249 INFO:teuthology.orchestra.run.smithi174.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 17 MB/s | 2.4 MB 00:00 2024-09-10T20:51:21.308 INFO:teuthology.orchestra.run.smithi174.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-09-10T20:51:21.364 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T20:51:21.500 INFO:teuthology.orchestra.run.smithi174.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 19 MB/s | 4.8 MB 00:00 2024-09-10T20:51:21.523 INFO:teuthology.orchestra.run.smithi033.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-09-10T20:51:21.909 INFO:teuthology.orchestra.run.smithi174.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 27 MB/s | 11 MB 00:00 2024-09-10T20:51:21.959 INFO:teuthology.orchestra.run.smithi174.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 507 kB/s | 25 kB 00:00 2024-09-10T20:51:21.981 INFO:teuthology.orchestra.run.smithi033.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.9 MB/s | 900 kB 00:00 2024-09-10T20:51:22.140 INFO:teuthology.orchestra.run.smithi033.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 916 kB/s | 145 kB 00:00 2024-09-10T20:51:22.759 INFO:teuthology.orchestra.run.smithi174.stdout:(10/157): ceph-common-19.3.0-4854.g8f219c47.el9 10 MB/s | 22 MB 00:02 2024-09-10T20:51:22.907 INFO:teuthology.orchestra.run.smithi033.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 3.1 MB/s | 2.4 MB 00:00 2024-09-10T20:51:22.982 INFO:teuthology.orchestra.run.smithi033.stdout:(5/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 3.2 MB/s | 5.5 MB 00:01 2024-09-10T20:51:23.059 INFO:teuthology.orchestra.run.smithi174.stdout:(11/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 9.8 MB/s | 17 MB 00:01 2024-09-10T20:51:23.093 INFO:teuthology.orchestra.run.smithi174.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 674 kB/s | 225 kB 00:00 2024-09-10T20:51:23.118 INFO:teuthology.orchestra.run.smithi174.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 461 kB/s | 27 kB 00:00 2024-09-10T20:51:23.124 INFO:teuthology.orchestra.run.smithi033.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 7.0 MB/s | 1.5 MB 00:00 2024-09-10T20:51:23.143 INFO:teuthology.orchestra.run.smithi174.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 635 kB/s | 32 kB 00:00 2024-09-10T20:51:23.194 INFO:teuthology.orchestra.run.smithi174.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 9.8 MB/s | 755 kB 00:00 2024-09-10T20:51:23.277 INFO:teuthology.orchestra.run.smithi174.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.2 MB/s | 162 kB 00:00 2024-09-10T20:51:23.303 INFO:teuthology.orchestra.run.smithi174.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 1.1 MB/s | 127 kB 00:00 2024-09-10T20:51:23.553 INFO:teuthology.orchestra.run.smithi174.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 1.8 MB/s | 507 kB 00:00 2024-09-10T20:51:23.557 INFO:teuthology.orchestra.run.smithi033.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 8.3 MB/s | 4.8 MB 00:00 2024-09-10T20:51:23.611 INFO:teuthology.orchestra.run.smithi174.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 777 kB/s | 45 kB 00:00 2024-09-10T20:51:23.658 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:51:23.658 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T20:51:23.670 INFO:teuthology.orchestra.run.smithi104.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T20:51:23.671 INFO:teuthology.orchestra.run.smithi104.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T20:51:23.672 INFO:teuthology.orchestra.run.smithi104.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T20:51:23.673 INFO:teuthology.orchestra.run.smithi104.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T20:51:23.674 INFO:teuthology.orchestra.run.smithi104.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T20:51:23.677 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T20:51:23.678 INFO:teuthology.orchestra.run.smithi104.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T20:51:23.741 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-09-10T20:51:23.751 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-09-10T20:51:23.751 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-09-10T20:51:23.751 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T20:51:23.752 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T20:51:23.753 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T20:51:23.754 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T20:51:23.755 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T20:51:23.756 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T20:51:23.757 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T20:51:23.758 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T20:51:23.759 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T20:51:23.760 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T20:51:23.761 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T20:51:23.762 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T20:51:23.763 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T20:51:23.764 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T20:51:23.765 INFO:teuthology.orchestra.run.smithi104.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout:======================================================================================= 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout:Install 149 Packages 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 8 Packages 2024-09-10T20:51:23.766 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:51:23.767 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 240 M 2024-09-10T20:51:23.767 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-09-10T20:51:24.045 INFO:teuthology.orchestra.run.smithi174.stdout:(20/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 24 MB/s | 50 MB 00:02 2024-09-10T20:51:24.079 INFO:teuthology.orchestra.run.smithi174.stdout:(21/157): python3-ceph-common-19.3.0-4854.g8f21 306 kB/s | 143 kB 00:00 2024-09-10T20:51:24.104 INFO:teuthology.orchestra.run.smithi174.stdout:(22/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.7 MB/s | 161 kB 00:00 2024-09-10T20:51:24.141 INFO:teuthology.orchestra.run.smithi033.stdout:(8/157): ceph-radosgw-19.3.0-4854.g8f219c47.el9 19 MB/s | 11 MB 00:00 2024-09-10T20:51:24.188 INFO:teuthology.orchestra.run.smithi174.stdout:(23/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 6.2 MB/s | 5.5 MB 00:00 2024-09-10T20:51:24.213 INFO:teuthology.orchestra.run.smithi174.stdout:(24/157): python3-rados-19.3.0-4854.g8f219c47.e 2.3 MB/s | 321 kB 00:00 2024-09-10T20:51:24.238 INFO:teuthology.orchestra.run.smithi174.stdout:(25/157): python3-rbd-19.3.0-4854.g8f219c47.el9 2.2 MB/s | 306 kB 00:00 2024-09-10T20:51:24.263 INFO:teuthology.orchestra.run.smithi174.stdout:(26/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.3 MB/s | 99 kB 00:00 2024-09-10T20:51:24.289 INFO:teuthology.orchestra.run.smithi174.stdout:(27/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.1 MB/s | 85 kB 00:00 2024-09-10T20:51:24.322 INFO:teuthology.orchestra.run.smithi174.stdout:(28/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 2.9 MB/s | 171 kB 00:00 2024-09-10T20:51:24.347 INFO:teuthology.orchestra.run.smithi174.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 587 kB/s | 34 kB 00:00 2024-09-10T20:51:24.358 INFO:teuthology.orchestra.run.smithi033.stdout:(9/157): ceph-common-19.3.0-4854.g8f219c47.el9. 7.0 MB/s | 22 MB 00:03 2024-09-10T20:51:24.373 INFO:teuthology.orchestra.run.smithi174.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 3.0 MB/s | 154 kB 00:00 2024-09-10T20:51:24.375 INFO:teuthology.orchestra.run.smithi033.stdout:(10/157): ceph-selinux-19.3.0-4854.g8f219c47.el 108 kB/s | 25 kB 00:00 2024-09-10T20:51:24.432 INFO:teuthology.orchestra.run.smithi174.stdout:(31/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 16 MB/s | 3.2 MB 00:00 2024-09-10T20:51:24.433 INFO:teuthology.orchestra.run.smithi033.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 3.8 MB/s | 225 kB 00:00 2024-09-10T20:51:24.449 INFO:teuthology.orchestra.run.smithi104.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 45 kB/s | 6.6 kB 00:00 2024-09-10T20:51:24.490 INFO:teuthology.orchestra.run.smithi174.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.9 MB/s | 291 kB 00:00 2024-09-10T20:51:24.492 INFO:teuthology.orchestra.run.smithi033.stdout:(12/157): cephfs-shell-19.3.0-4854.g8f219c47.el 464 kB/s | 27 kB 00:00 2024-09-10T20:51:24.542 INFO:teuthology.orchestra.run.smithi033.stdout:(13/157): libcephfs-devel-19.3.0-4854.g8f219c47 636 kB/s | 32 kB 00:00 2024-09-10T20:51:24.557 INFO:teuthology.orchestra.run.smithi174.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 741 kB/s | 49 kB 00:00 2024-09-10T20:51:24.627 INFO:teuthology.orchestra.run.smithi033.stdout:(14/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 8.9 MB/s | 755 kB 00:00 2024-09-10T20:51:24.657 INFO:teuthology.orchestra.run.smithi174.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 164 kB/s | 16 kB 00:00 2024-09-10T20:51:24.685 INFO:teuthology.orchestra.run.smithi033.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.7 MB/s | 162 kB 00:00 2024-09-10T20:51:24.741 INFO:teuthology.orchestra.run.smithi174.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 3.4 MB/s | 285 kB 00:00 2024-09-10T20:51:24.760 INFO:teuthology.orchestra.run.smithi033.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 1.7 MB/s | 127 kB 00:00 2024-09-10T20:51:24.824 INFO:teuthology.orchestra.run.smithi174.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 9.2 MB/s | 783 kB 00:00 2024-09-10T20:51:24.851 INFO:teuthology.orchestra.run.smithi033.stdout:(17/157): libradosstriper1-19.3.0-4854.g8f219c4 5.4 MB/s | 507 kB 00:00 2024-09-10T20:51:24.891 INFO:teuthology.orchestra.run.smithi174.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 274 kB/s | 18 kB 00:00 2024-09-10T20:51:24.974 INFO:teuthology.orchestra.run.smithi104.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 900 kB 00:00 2024-09-10T20:51:24.975 INFO:teuthology.orchestra.run.smithi174.stdout:(38/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-10T20:51:25.092 INFO:teuthology.orchestra.run.smithi174.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 399 kB/s | 80 kB 00:00 2024-09-10T20:51:25.099 INFO:teuthology.orchestra.run.smithi104.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.1 MB/s | 145 kB 00:00 2024-09-10T20:51:25.159 INFO:teuthology.orchestra.run.smithi174.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-10T20:51:25.192 INFO:teuthology.orchestra.run.smithi174.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-10T20:51:25.193 INFO:teuthology.orchestra.run.smithi033.stdout:(18/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 16 MB/s | 5.5 MB 00:00 2024-09-10T20:51:25.243 INFO:teuthology.orchestra.run.smithi174.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-10T20:51:25.244 INFO:teuthology.orchestra.run.smithi033.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 906 kB/s | 45 kB 00:00 2024-09-10T20:51:25.301 INFO:teuthology.orchestra.run.smithi174.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-10T20:51:25.311 INFO:teuthology.orchestra.run.smithi033.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.1 MB/s | 143 kB 00:00 2024-09-10T20:51:25.328 INFO:teuthology.orchestra.run.smithi174.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-09-10T20:51:25.378 INFO:teuthology.orchestra.run.smithi033.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.4 MB/s | 161 kB 00:00 2024-09-10T20:51:25.394 INFO:teuthology.orchestra.run.smithi174.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-09-10T20:51:25.419 INFO:teuthology.orchestra.run.smithi174.stdout:(46/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-10T20:51:25.445 INFO:teuthology.orchestra.run.smithi033.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 4.7 MB/s | 321 kB 00:00 2024-09-10T20:51:25.469 INFO:teuthology.orchestra.run.smithi174.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-10T20:51:25.520 INFO:teuthology.orchestra.run.smithi033.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 4.0 MB/s | 306 kB 00:00 2024-09-10T20:51:25.591 INFO:teuthology.orchestra.run.smithi104.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 4.3 MB/s | 5.5 MB 00:01 2024-09-10T20:51:25.695 INFO:teuthology.orchestra.run.smithi033.stdout:(24/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 6.7 MB/s | 17 MB 00:02 2024-09-10T20:51:25.720 INFO:teuthology.orchestra.run.smithi033.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 497 kB/s | 99 kB 00:00 2024-09-10T20:51:25.733 INFO:teuthology.orchestra.run.smithi104.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 3.8 MB/s | 2.4 MB 00:00 2024-09-10T20:51:25.754 INFO:teuthology.orchestra.run.smithi033.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.4 MB/s | 85 kB 00:00 2024-09-10T20:51:25.833 INFO:teuthology.orchestra.run.smithi104.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 6.3 MB/s | 1.5 MB 00:00 2024-09-10T20:51:25.846 INFO:teuthology.orchestra.run.smithi033.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 1.8 MB/s | 171 kB 00:00 2024-09-10T20:51:25.913 INFO:teuthology.orchestra.run.smithi033.stdout:(28/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 17 MB/s | 3.2 MB 00:00 2024-09-10T20:51:25.928 INFO:teuthology.orchestra.run.smithi174.stdout:(48/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 9.0 MB/s | 14 MB 00:01 2024-09-10T20:51:25.947 INFO:teuthology.orchestra.run.smithi033.stdout:(29/157): ceph-grafana-dashboards-19.3.0-4854.g 342 kB/s | 34 kB 00:00 2024-09-10T20:51:25.981 INFO:teuthology.orchestra.run.smithi033.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.3 MB/s | 154 kB 00:00 2024-09-10T20:51:26.153 INFO:teuthology.orchestra.run.smithi174.stdout:(49/157): automake-1.16.2-8.el9.noarch.rpm 939 kB/s | 689 kB 00:00 2024-09-10T20:51:26.262 INFO:teuthology.orchestra.run.smithi174.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 200 kB/s | 22 kB 00:00 2024-09-10T20:51:26.287 INFO:teuthology.orchestra.run.smithi174.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-10T20:51:26.379 INFO:teuthology.orchestra.run.smithi174.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 2.6 MB/s | 313 kB 00:00 2024-09-10T20:51:26.404 INFO:teuthology.orchestra.run.smithi174.stdout:(53/157): boost-program-options-1.75.0-8.el9.x8 225 kB/s | 107 kB 00:00 2024-09-10T20:51:26.429 INFO:teuthology.orchestra.run.smithi174.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 230 kB/s | 33 kB 00:00 2024-09-10T20:51:26.455 INFO:teuthology.orchestra.run.smithi033.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 16 MB/s | 7.4 MB 00:00 2024-09-10T20:51:26.504 INFO:teuthology.orchestra.run.smithi174.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 182 kB/s | 18 kB 00:00 2024-09-10T20:51:26.523 INFO:teuthology.orchestra.run.smithi033.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 4.3 MB/s | 291 kB 00:00 2024-09-10T20:51:26.546 INFO:teuthology.orchestra.run.smithi174.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 95 kB/s | 11 kB 00:00 2024-09-10T20:51:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 986 kB/s | 49 kB 00:00 2024-09-10T20:51:26.613 INFO:teuthology.orchestra.run.smithi174.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.5 MB/s | 171 kB 00:00 2024-09-10T20:51:26.623 INFO:teuthology.orchestra.run.smithi033.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 329 kB/s | 16 kB 00:00 2024-09-10T20:51:26.682 INFO:teuthology.orchestra.run.smithi033.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 4.8 MB/s | 285 kB 00:00 2024-09-10T20:51:26.722 INFO:teuthology.orchestra.run.smithi174.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2024-09-10T20:51:26.741 INFO:teuthology.orchestra.run.smithi104.stdout:(7/157): ceph-common-19.3.0-4854.g8f219c47.el9. 8.9 MB/s | 22 MB 00:02 2024-09-10T20:51:26.757 INFO:teuthology.orchestra.run.smithi033.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 10 MB/s | 783 kB 00:00 2024-09-10T20:51:26.807 INFO:teuthology.orchestra.run.smithi033.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 365 kB/s | 18 kB 00:00 2024-09-10T20:51:26.808 INFO:teuthology.orchestra.run.smithi104.stdout:(8/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 4.5 MB/s | 4.8 MB 00:01 2024-09-10T20:51:26.830 INFO:teuthology.orchestra.run.smithi174.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 565 kB/s | 160 kB 00:00 2024-09-10T20:51:26.856 INFO:teuthology.orchestra.run.smithi174.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-09-10T20:51:26.858 INFO:teuthology.orchestra.run.smithi104.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 505 kB/s | 25 kB 00:00 2024-09-10T20:51:26.941 INFO:teuthology.orchestra.run.smithi033.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 598 kB/s | 80 kB 00:00 2024-09-10T20:51:26.964 INFO:teuthology.orchestra.run.smithi174.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-10T20:51:27.031 INFO:teuthology.orchestra.run.smithi174.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 2.8 MB/s | 581 kB 00:00 2024-09-10T20:51:27.073 INFO:teuthology.orchestra.run.smithi174.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-09-10T20:51:27.074 INFO:teuthology.orchestra.run.smithi033.stdout:(39/157): libgfortran-11.5.0-2.el9.x86_64.rpm 5.9 MB/s | 801 kB 00:00 2024-09-10T20:51:27.125 INFO:teuthology.orchestra.run.smithi033.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T20:51:27.175 INFO:teuthology.orchestra.run.smithi033.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-10T20:51:27.190 INFO:teuthology.orchestra.run.smithi174.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-10T20:51:27.215 INFO:teuthology.orchestra.run.smithi174.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.6 MB/s | 292 kB 00:00 2024-09-10T20:51:27.259 INFO:teuthology.orchestra.run.smithi033.stdout:(42/157): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-10T20:51:27.299 INFO:teuthology.orchestra.run.smithi174.stdout:(66/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.3 MB/s | 3.0 MB 00:00 2024-09-10T20:51:27.301 INFO:teuthology.orchestra.run.smithi033.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-10T20:51:27.324 INFO:teuthology.orchestra.run.smithi174.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 287 kB/s | 38 kB 00:00 2024-09-10T20:51:27.352 INFO:teuthology.orchestra.run.smithi033.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T20:51:27.401 INFO:teuthology.orchestra.run.smithi033.stdout:(45/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T20:51:27.433 INFO:teuthology.orchestra.run.smithi104.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 16 MB/s | 11 MB 00:00 2024-09-10T20:51:27.452 INFO:teuthology.orchestra.run.smithi033.stdout:(46/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T20:51:27.484 INFO:teuthology.orchestra.run.smithi104.stdout:(11/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 4.4 MB/s | 225 kB 00:00 2024-09-10T20:51:27.558 INFO:teuthology.orchestra.run.smithi174.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-10T20:51:27.643 INFO:teuthology.orchestra.run.smithi104.stdout:(12/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 9.5 MB/s | 17 MB 00:01 2024-09-10T20:51:27.668 INFO:teuthology.orchestra.run.smithi104.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 147 kB/s | 27 kB 00:00 2024-09-10T20:51:27.683 INFO:teuthology.orchestra.run.smithi174.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 7.1 MB/s | 910 kB 00:00 2024-09-10T20:51:27.693 INFO:teuthology.orchestra.run.smithi104.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 637 kB/s | 32 kB 00:00 2024-09-10T20:51:27.710 INFO:teuthology.orchestra.run.smithi033.stdout:(47/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 8.0 MB/s | 14 MB 00:01 2024-09-10T20:51:27.735 INFO:teuthology.orchestra.run.smithi104.stdout:(15/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 11 MB/s | 755 kB 00:00 2024-09-10T20:51:27.760 INFO:teuthology.orchestra.run.smithi104.stdout:(16/157): libcephsqlite-19.3.0-4854.g8f219c47.e 2.4 MB/s | 162 kB 00:00 2024-09-10T20:51:27.785 INFO:teuthology.orchestra.run.smithi104.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 2.5 MB/s | 127 kB 00:00 2024-09-10T20:51:27.800 INFO:teuthology.orchestra.run.smithi174.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-10T20:51:27.802 INFO:teuthology.orchestra.run.smithi033.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 689 kB 00:00 2024-09-10T20:51:27.810 INFO:teuthology.orchestra.run.smithi104.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 9.9 MB/s | 507 kB 00:00 2024-09-10T20:51:27.869 INFO:teuthology.orchestra.run.smithi104.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 771 kB/s | 45 kB 00:00 2024-09-10T20:51:27.928 INFO:teuthology.orchestra.run.smithi104.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 2.4 MB/s | 143 kB 00:00 2024-09-10T20:51:27.986 INFO:teuthology.orchestra.run.smithi033.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 583 kB/s | 107 kB 00:00 2024-09-10T20:51:27.987 INFO:teuthology.orchestra.run.smithi104.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 2.7 MB/s | 161 kB 00:00 2024-09-10T20:51:28.053 INFO:teuthology.orchestra.run.smithi033.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 2.7 MB/s | 943 kB 00:00 2024-09-10T20:51:28.070 INFO:teuthology.orchestra.run.smithi033.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-10T20:51:28.087 INFO:teuthology.orchestra.run.smithi104.stdout:(22/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 18 MB/s | 5.5 MB 00:00 2024-09-10T20:51:28.120 INFO:teuthology.orchestra.run.smithi033.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 659 kB/s | 33 kB 00:00 2024-09-10T20:51:28.121 INFO:teuthology.orchestra.run.smithi104.stdout:(23/157): python3-rados-19.3.0-4854.g8f219c47.e 2.3 MB/s | 321 kB 00:00 2024-09-10T20:51:28.125 INFO:teuthology.orchestra.run.smithi174.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 5.4 MB/s | 4.9 MB 00:00 2024-09-10T20:51:28.145 INFO:teuthology.orchestra.run.smithi104.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 5.1 MB/s | 306 kB 00:00 2024-09-10T20:51:28.154 INFO:teuthology.orchestra.run.smithi033.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-10T20:51:28.171 INFO:teuthology.orchestra.run.smithi104.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.9 MB/s | 99 kB 00:00 2024-09-10T20:51:28.196 INFO:teuthology.orchestra.run.smithi104.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 85 kB 00:00 2024-09-10T20:51:28.247 INFO:teuthology.orchestra.run.smithi104.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 3.3 MB/s | 171 kB 00:00 2024-09-10T20:51:28.297 INFO:teuthology.orchestra.run.smithi104.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 685 kB/s | 34 kB 00:00 2024-09-10T20:51:28.355 INFO:teuthology.orchestra.run.smithi104.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 2.6 MB/s | 154 kB 00:00 2024-09-10T20:51:28.406 INFO:teuthology.orchestra.run.smithi104.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 14 MB/s | 3.2 MB 00:00 2024-09-10T20:51:28.613 INFO:teuthology.orchestra.run.smithi033.stdout:(54/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 12 MB/s | 50 MB 00:04 2024-09-10T20:51:28.629 INFO:teuthology.orchestra.run.smithi033.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 38 kB/s | 18 kB 00:00 2024-09-10T20:51:28.688 INFO:teuthology.orchestra.run.smithi033.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 149 kB/s | 11 kB 00:00 2024-09-10T20:51:28.713 INFO:teuthology.orchestra.run.smithi033.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-10T20:51:28.763 INFO:teuthology.orchestra.run.smithi033.stdout:(58/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 45 kB 00:00 2024-09-10T20:51:28.814 INFO:teuthology.orchestra.run.smithi033.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-10T20:51:28.923 INFO:teuthology.orchestra.run.smithi104.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 14 MB/s | 7.4 MB 00:00 2024-09-10T20:51:28.930 INFO:teuthology.orchestra.run.smithi033.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-10T20:51:28.965 INFO:teuthology.orchestra.run.smithi033.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 3.8 MB/s | 581 kB 00:00 2024-09-10T20:51:28.990 INFO:teuthology.orchestra.run.smithi033.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 415 kB/s | 24 kB 00:00 2024-09-10T20:51:29.015 INFO:teuthology.orchestra.run.smithi033.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 949 kB/s | 47 kB 00:00 2024-09-10T20:51:29.065 INFO:teuthology.orchestra.run.smithi104.stdout:(32/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 20 MB/s | 14 MB 00:00 2024-09-10T20:51:29.074 INFO:teuthology.orchestra.run.smithi033.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-10T20:51:29.090 INFO:teuthology.orchestra.run.smithi104.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 1.7 MB/s | 291 kB 00:00 2024-09-10T20:51:29.099 INFO:teuthology.orchestra.run.smithi033.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T20:51:29.115 INFO:teuthology.orchestra.run.smithi104.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 987 kB/s | 49 kB 00:00 2024-09-10T20:51:29.140 INFO:teuthology.orchestra.run.smithi104.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 328 kB/s | 16 kB 00:00 2024-09-10T20:51:29.149 INFO:teuthology.orchestra.run.smithi033.stdout:(66/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-10T20:51:29.165 INFO:teuthology.orchestra.run.smithi104.stdout:(36/157): ceph-volume-19.3.0-4854.g8f219c47.el9 5.6 MB/s | 285 kB 00:00 2024-09-10T20:51:29.207 INFO:teuthology.orchestra.run.smithi104.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 437 kB/s | 18 kB 00:00 2024-09-10T20:51:29.233 INFO:teuthology.orchestra.run.smithi033.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 242 kB/s | 38 kB 00:00 2024-09-10T20:51:29.241 INFO:teuthology.orchestra.run.smithi104.stdout:(38/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 7.6 MB/s | 783 kB 00:00 2024-09-10T20:51:29.583 INFO:teuthology.orchestra.run.smithi104.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 213 kB/s | 80 kB 00:00 2024-09-10T20:51:29.633 INFO:teuthology.orchestra.run.smithi033.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-09-10T20:51:29.642 INFO:teuthology.orchestra.run.smithi174.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 681 kB/s | 1.0 MB 00:01 2024-09-10T20:51:29.783 INFO:teuthology.orchestra.run.smithi104.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-10T20:51:29.842 INFO:teuthology.orchestra.run.smithi174.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 438 kB/s | 87 kB 00:00 2024-09-10T20:51:29.875 INFO:teuthology.orchestra.run.smithi033.stdout:(69/157): postgresql-contrib-13.16-1.el9.x86_64 3.7 MB/s | 910 kB 00:00 2024-09-10T20:51:29.925 INFO:teuthology.orchestra.run.smithi033.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 2.7 MB/s | 137 kB 00:00 2024-09-10T20:51:29.992 INFO:teuthology.orchestra.run.smithi104.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-10T20:51:30.117 INFO:teuthology.orchestra.run.smithi033.stdout:(71/157): openblas-openmp-0.3.26-2.el9.x86_64.r 4.9 MB/s | 4.9 MB 00:01 2024-09-10T20:51:30.142 INFO:teuthology.orchestra.run.smithi104.stdout:(42/157): libgfortran-11.5.0-2.el9.x86_64.rpm 890 kB/s | 801 kB 00:00 2024-09-10T20:51:30.250 INFO:teuthology.orchestra.run.smithi104.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 983 kB/s | 106 kB 00:00 2024-09-10T20:51:30.326 INFO:teuthology.orchestra.run.smithi033.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.8 MB/s | 4.4 MB 00:01 2024-09-10T20:51:30.359 INFO:teuthology.orchestra.run.smithi033.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-09-10T20:51:30.367 INFO:teuthology.orchestra.run.smithi104.stdout:(44/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-10T20:51:30.393 INFO:teuthology.orchestra.run.smithi033.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-10T20:51:30.401 INFO:teuthology.orchestra.run.smithi104.stdout:(45/157): python3-cryptography-36.0.1-4.el9.x86 3.0 MB/s | 1.2 MB 00:00 2024-09-10T20:51:30.432 INFO:teuthology.orchestra.run.smithi174.stdout:(74/157): postgresql-server-13.16-1.el9.x86_64. 2.2 MB/s | 5.8 MB 00:02 2024-09-10T20:51:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-10T20:51:30.476 INFO:teuthology.orchestra.run.smithi104.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-09-10T20:51:30.518 INFO:teuthology.orchestra.run.smithi104.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-10T20:51:30.549 INFO:teuthology.orchestra.run.smithi174.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 526 kB/s | 61 kB 00:00 2024-09-10T20:51:30.560 INFO:teuthology.orchestra.run.smithi033.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-10T20:51:30.669 INFO:teuthology.orchestra.run.smithi033.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-09-10T20:51:30.711 INFO:teuthology.orchestra.run.smithi033.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-09-10T20:51:30.752 INFO:teuthology.orchestra.run.smithi104.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 2.4 MB/s | 689 kB 00:00 2024-09-10T20:51:30.753 INFO:teuthology.orchestra.run.smithi033.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 464 kB/s | 19 kB 00:00 2024-09-10T20:51:30.811 INFO:teuthology.orchestra.run.smithi104.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-09-10T20:51:30.836 INFO:teuthology.orchestra.run.smithi104.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-09-10T20:51:30.836 INFO:teuthology.orchestra.run.smithi033.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-10T20:51:30.861 INFO:teuthology.orchestra.run.smithi104.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-10T20:51:30.886 INFO:teuthology.orchestra.run.smithi033.stdout:(81/157): python3-markupsafe-1.1.1-12.el9.x86_6 698 kB/s | 35 kB 00:00 2024-09-10T20:51:30.895 INFO:teuthology.orchestra.run.smithi104.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-10T20:51:30.920 INFO:teuthology.orchestra.run.smithi104.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 557 kB/s | 33 kB 00:00 2024-09-10T20:51:30.933 INFO:teuthology.orchestra.run.smithi174.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 638 kB/s | 245 kB 00:00 2024-09-10T20:51:30.970 INFO:teuthology.orchestra.run.smithi104.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-09-10T20:51:31.021 INFO:teuthology.orchestra.run.smithi104.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-10T20:51:31.070 INFO:teuthology.orchestra.run.smithi033.stdout:(82/157): postgresql-server-13.16-1.el9.x86_64. 5.0 MB/s | 5.8 MB 00:01 2024-09-10T20:51:31.079 INFO:teuthology.orchestra.run.smithi104.stdout:(56/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-10T20:51:31.154 INFO:teuthology.orchestra.run.smithi033.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.0 MB/s | 250 kB 00:00 2024-09-10T20:51:31.237 INFO:teuthology.orchestra.run.smithi033.stdout:(84/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-10T20:51:31.308 INFO:teuthology.orchestra.run.smithi174.stdout:(77/157): python3-jinja2-2.11.3-6.el9.noarch.rp 662 kB/s | 248 kB 00:00 2024-09-10T20:51:31.321 INFO:teuthology.orchestra.run.smithi033.stdout:(85/157): python3-babel-2.9.1-2.el9.noarch.rpm 6.2 MB/s | 6.0 MB 00:00 2024-09-10T20:51:31.346 INFO:teuthology.orchestra.run.smithi033.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 711 kB/s | 77 kB 00:00 2024-09-10T20:51:31.363 INFO:teuthology.orchestra.run.smithi033.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-10T20:51:31.413 INFO:teuthology.orchestra.run.smithi033.stdout:(88/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T20:51:31.433 INFO:teuthology.orchestra.run.smithi174.stdout:(78/157): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-09-10T20:51:31.439 INFO:teuthology.orchestra.run.smithi033.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-09-10T20:51:31.481 INFO:teuthology.orchestra.run.smithi033.stdout:(90/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-09-10T20:51:31.515 INFO:teuthology.orchestra.run.smithi033.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-09-10T20:51:31.538 INFO:teuthology.orchestra.run.smithi104.stdout:(57/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 11 MB/s | 50 MB 00:04 2024-09-10T20:51:31.540 INFO:teuthology.orchestra.run.smithi033.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 910 kB/s | 54 kB 00:00 2024-09-10T20:51:31.550 INFO:teuthology.orchestra.run.smithi174.stdout:(79/157): python3-jsonpointer-2.0-4.el9.noarch. 164 kB/s | 19 kB 00:00 2024-09-10T20:51:31.563 INFO:teuthology.orchestra.run.smithi104.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 331 kB/s | 160 kB 00:00 2024-09-10T20:51:31.598 INFO:teuthology.orchestra.run.smithi033.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-09-10T20:51:31.613 INFO:teuthology.orchestra.run.smithi104.stdout:(59/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 4.2 MB/s | 3.0 MB 00:00 2024-09-10T20:51:31.674 INFO:teuthology.orchestra.run.smithi033.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 589 kB/s | 44 kB 00:00 2024-09-10T20:51:31.680 INFO:teuthology.orchestra.run.smithi104.stdout:(60/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-10T20:51:31.705 INFO:teuthology.orchestra.run.smithi104.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-09-10T20:51:31.731 INFO:teuthology.orchestra.run.smithi104.stdout:(62/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 235 kB/s | 45 kB 00:00 2024-09-10T20:51:31.750 INFO:teuthology.orchestra.run.smithi174.stdout:(80/157): python3-mako-1.1.4-6.el9.noarch.rpm 861 kB/s | 172 kB 00:00 2024-09-10T20:51:31.756 INFO:teuthology.orchestra.run.smithi104.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 324 kB/s | 24 kB 00:00 2024-09-10T20:51:31.765 INFO:teuthology.orchestra.run.smithi033.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-09-10T20:51:31.784 INFO:teuthology.orchestra.run.smithi104.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-09-10T20:51:31.815 INFO:teuthology.orchestra.run.smithi104.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-10T20:51:31.816 INFO:teuthology.orchestra.run.smithi033.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-10T20:51:31.817 INFO:teuthology.orchestra.run.smithi174.stdout:(81/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 1.0 MB/s | 4.4 MB 00:04 2024-09-10T20:51:31.840 INFO:teuthology.orchestra.run.smithi104.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-10T20:51:31.859 INFO:teuthology.orchestra.run.smithi174.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 320 kB/s | 35 kB 00:00 2024-09-10T20:51:31.873 INFO:teuthology.orchestra.run.smithi104.stdout:(67/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-09-10T20:51:31.874 INFO:teuthology.orchestra.run.smithi033.stdout:(97/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-09-10T20:51:32.008 INFO:teuthology.orchestra.run.smithi033.stdout:(98/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-10T20:51:32.009 INFO:teuthology.orchestra.run.smithi174.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-09-10T20:51:32.092 INFO:teuthology.orchestra.run.smithi033.stdout:(99/157): uuid-1.6.2-55.el9.x86_64.rpm 695 kB/s | 58 kB 00:00 2024-09-10T20:51:32.115 INFO:teuthology.orchestra.run.smithi104.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-09-10T20:51:32.118 INFO:teuthology.orchestra.run.smithi174.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-10T20:51:32.152 INFO:teuthology.orchestra.run.smithi174.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.5 MB/s | 222 kB 00:00 2024-09-10T20:51:32.166 INFO:teuthology.orchestra.run.smithi104.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 5.4 MB/s | 1.6 MB 00:00 2024-09-10T20:51:32.167 INFO:teuthology.orchestra.run.smithi033.stdout:(100/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-10T20:51:32.199 INFO:teuthology.orchestra.run.smithi104.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-10T20:51:32.227 INFO:teuthology.orchestra.run.smithi174.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 710 kB/s | 77 kB 00:00 2024-09-10T20:51:32.234 INFO:teuthology.orchestra.run.smithi033.stdout:(101/157): python3-numpy-1.20.1-5.el9.x86_64.rp 3.8 MB/s | 5.1 MB 00:01 2024-09-10T20:51:32.259 INFO:teuthology.orchestra.run.smithi033.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 693 kB/s | 64 kB 00:00 2024-09-10T20:51:32.277 INFO:teuthology.orchestra.run.smithi174.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 337 kB/s | 42 kB 00:00 2024-09-10T20:51:32.293 INFO:teuthology.orchestra.run.smithi033.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-09-10T20:51:32.299 INFO:teuthology.orchestra.run.smithi104.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.0 MB/s | 137 kB 00:00 2024-09-10T20:51:32.344 INFO:teuthology.orchestra.run.smithi174.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-09-10T20:51:32.375 INFO:teuthology.orchestra.run.smithi104.stdout:(72/157): postgresql-server-13.16-1.el9.x86_64. 33 MB/s | 5.8 MB 00:00 2024-09-10T20:51:32.386 INFO:teuthology.orchestra.run.smithi174.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-10T20:51:32.425 INFO:teuthology.orchestra.run.smithi104.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-10T20:51:32.461 INFO:teuthology.orchestra.run.smithi174.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-10T20:51:32.492 INFO:teuthology.orchestra.run.smithi104.stdout:(74/157): openblas-openmp-0.3.26-2.el9.x86_64.r 7.3 MB/s | 4.9 MB 00:00 2024-09-10T20:51:32.512 INFO:teuthology.orchestra.run.smithi174.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 407 kB/s | 51 kB 00:00 2024-09-10T20:51:32.534 INFO:teuthology.orchestra.run.smithi104.stdout:(75/157): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-09-10T20:51:32.551 INFO:teuthology.orchestra.run.smithi033.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 76 kB/s | 22 kB 00:00 2024-09-10T20:51:32.559 INFO:teuthology.orchestra.run.smithi104.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 917 kB/s | 61 kB 00:00 2024-09-10T20:51:32.570 INFO:teuthology.orchestra.run.smithi174.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 494 kB/s | 54 kB 00:00 2024-09-10T20:51:32.593 INFO:teuthology.orchestra.run.smithi104.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-10T20:51:32.618 INFO:teuthology.orchestra.run.smithi104.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-10T20:51:32.635 INFO:teuthology.orchestra.run.smithi033.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-10T20:51:32.677 INFO:teuthology.orchestra.run.smithi033.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-09-10T20:51:32.679 INFO:teuthology.orchestra.run.smithi174.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-09-10T20:51:32.701 INFO:teuthology.orchestra.run.smithi104.stdout:(79/157): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-09-10T20:51:32.702 INFO:teuthology.orchestra.run.smithi033.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 776 kB/s | 19 kB 00:00 2024-09-10T20:51:32.726 INFO:teuthology.orchestra.run.smithi104.stdout:(80/157): python3-jsonpatch-1.21-16.el9.noarch. 192 kB/s | 26 kB 00:00 2024-09-10T20:51:32.742 INFO:teuthology.orchestra.run.smithi104.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-10T20:51:32.768 INFO:teuthology.orchestra.run.smithi104.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-10T20:51:32.787 INFO:teuthology.orchestra.run.smithi174.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 408 kB/s | 44 kB 00:00 2024-09-10T20:51:32.793 INFO:teuthology.orchestra.run.smithi104.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 692 kB/s | 35 kB 00:00 2024-09-10T20:51:32.852 INFO:teuthology.orchestra.run.smithi104.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-09-10T20:51:32.877 INFO:teuthology.orchestra.run.smithi104.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-10T20:51:32.902 INFO:teuthology.orchestra.run.smithi104.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-10T20:51:32.904 INFO:teuthology.orchestra.run.smithi174.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-10T20:51:32.936 INFO:teuthology.orchestra.run.smithi033.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-10T20:51:32.961 INFO:teuthology.orchestra.run.smithi033.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T20:51:32.969 INFO:teuthology.orchestra.run.smithi104.stdout:(87/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-09-10T20:51:32.986 INFO:teuthology.orchestra.run.smithi033.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T20:51:32.987 INFO:teuthology.orchestra.run.smithi104.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 388 kB/s | 42 kB 00:00 2024-09-10T20:51:33.011 INFO:teuthology.orchestra.run.smithi033.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T20:51:33.012 INFO:teuthology.orchestra.run.smithi104.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 2.4 MB/s | 269 kB 00:00 2024-09-10T20:51:33.021 INFO:teuthology.orchestra.run.smithi174.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 307 kB/s | 36 kB 00:00 2024-09-10T20:51:33.037 INFO:teuthology.orchestra.run.smithi104.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-10T20:51:33.045 INFO:teuthology.orchestra.run.smithi033.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-10T20:51:33.062 INFO:teuthology.orchestra.run.smithi104.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-10T20:51:33.087 INFO:teuthology.orchestra.run.smithi104.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-10T20:51:33.104 INFO:teuthology.orchestra.run.smithi033.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-10T20:51:33.113 INFO:teuthology.orchestra.run.smithi104.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 712 kB/s | 54 kB 00:00 2024-09-10T20:51:33.138 INFO:teuthology.orchestra.run.smithi033.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 16 MB/s | 548 kB 00:00 2024-09-10T20:51:33.138 INFO:teuthology.orchestra.run.smithi104.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-09-10T20:51:33.155 INFO:teuthology.orchestra.run.smithi033.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-10T20:51:33.171 INFO:teuthology.orchestra.run.smithi033.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-10T20:51:33.172 INFO:teuthology.orchestra.run.smithi104.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 753 kB/s | 44 kB 00:00 2024-09-10T20:51:33.180 INFO:teuthology.orchestra.run.smithi174.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-10T20:51:33.189 INFO:teuthology.orchestra.run.smithi033.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 843 kB/s | 14 kB 00:00 2024-09-10T20:51:33.197 INFO:teuthology.orchestra.run.smithi104.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-10T20:51:33.205 INFO:teuthology.orchestra.run.smithi174.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 258 kB/s | 47 kB 00:00 2024-09-10T20:51:33.213 INFO:teuthology.orchestra.run.smithi033.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-10T20:51:33.222 INFO:teuthology.orchestra.run.smithi104.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-09-10T20:51:33.248 INFO:teuthology.orchestra.run.smithi033.stdout:(119/157): protobuf-compiler-3.14.0-13.el9.x86_ 904 kB/s | 863 kB 00:00 2024-09-10T20:51:33.256 INFO:teuthology.orchestra.run.smithi104.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-09-10T20:51:33.281 INFO:teuthology.orchestra.run.smithi033.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-10T20:51:33.289 INFO:teuthology.orchestra.run.smithi104.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.5 MB/s | 304 kB 00:00 2024-09-10T20:51:33.305 INFO:teuthology.orchestra.run.smithi174.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-10T20:51:33.314 INFO:teuthology.orchestra.run.smithi104.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-09-10T20:51:33.323 INFO:teuthology.orchestra.run.smithi033.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 5.5 MB/s | 220 kB 00:00 2024-09-10T20:51:33.330 INFO:teuthology.orchestra.run.smithi174.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 461 kB/s | 58 kB 00:00 2024-09-10T20:51:33.348 INFO:teuthology.orchestra.run.smithi104.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-10T20:51:33.357 INFO:teuthology.orchestra.run.smithi033.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-10T20:51:33.373 INFO:teuthology.orchestra.run.smithi104.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-10T20:51:33.399 INFO:teuthology.orchestra.run.smithi104.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-09-10T20:51:33.399 INFO:teuthology.orchestra.run.smithi033.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-10T20:51:33.415 INFO:teuthology.orchestra.run.smithi033.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 650 kB/s | 11 kB 00:00 2024-09-10T20:51:33.422 INFO:teuthology.orchestra.run.smithi174.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 821 kB/s | 96 kB 00:00 2024-09-10T20:51:33.432 INFO:teuthology.orchestra.run.smithi033.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-10T20:51:33.449 INFO:teuthology.orchestra.run.smithi033.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-10T20:51:33.456 INFO:teuthology.orchestra.run.smithi174.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 508 kB/s | 64 kB 00:00 2024-09-10T20:51:33.467 INFO:teuthology.orchestra.run.smithi033.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-10T20:51:33.517 INFO:teuthology.orchestra.run.smithi033.stdout:(128/157): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-09-10T20:51:33.531 INFO:teuthology.orchestra.run.smithi174.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 412 kB/s | 45 kB 00:00 2024-09-10T20:51:33.533 INFO:teuthology.orchestra.run.smithi033.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 294 kB/s | 20 kB 00:00 2024-09-10T20:51:33.565 INFO:teuthology.orchestra.run.smithi174.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-09-10T20:51:33.615 INFO:teuthology.orchestra.run.smithi104.stdout:(104/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 29 MB/s | 16 MB 00:00 2024-09-10T20:51:33.641 INFO:teuthology.orchestra.run.smithi104.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-09-10T20:51:33.648 INFO:teuthology.orchestra.run.smithi174.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-10T20:51:33.682 INFO:teuthology.orchestra.run.smithi174.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-09-10T20:51:33.701 INFO:teuthology.orchestra.run.smithi033.stdout:(130/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.4 MB/s | 16 MB 00:02 2024-09-10T20:51:33.716 INFO:teuthology.orchestra.run.smithi104.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-09-10T20:51:33.717 INFO:teuthology.orchestra.run.smithi033.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 252 kB/s | 46 kB 00:00 2024-09-10T20:51:33.741 INFO:teuthology.orchestra.run.smithi104.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 774 kB/s | 19 kB 00:00 2024-09-10T20:51:33.743 INFO:teuthology.orchestra.run.smithi174.stdout:(107/157): gperftools-libs-2.9.1-2.el9.x86_64.r 3.2 MB/s | 309 kB 00:00 2024-09-10T20:51:33.760 INFO:teuthology.orchestra.run.smithi033.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.7 MB/s | 58 kB 00:00 2024-09-10T20:51:33.766 INFO:teuthology.orchestra.run.smithi174.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-09-10T20:51:33.775 INFO:teuthology.orchestra.run.smithi104.stdout:(108/157): abseil-cpp-20211102.0-3.el9.x86_64.r 3.4 MB/s | 548 kB 00:00 2024-09-10T20:51:33.783 INFO:teuthology.orchestra.run.smithi174.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-10T20:51:33.800 INFO:teuthology.orchestra.run.smithi174.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-10T20:51:33.801 INFO:teuthology.orchestra.run.smithi033.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 3.6 MB/s | 1.0 MB 00:00 2024-09-10T20:51:33.808 INFO:teuthology.orchestra.run.smithi104.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-09-10T20:51:33.827 INFO:teuthology.orchestra.run.smithi033.stdout:(134/157): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-09-10T20:51:33.834 INFO:teuthology.orchestra.run.smithi104.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 442 kB/s | 26 kB 00:00 2024-09-10T20:51:33.842 INFO:teuthology.orchestra.run.smithi174.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-09-10T20:51:33.852 INFO:teuthology.orchestra.run.smithi033.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 327 kB/s | 16 kB 00:00 2024-09-10T20:51:33.867 INFO:teuthology.orchestra.run.smithi104.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-10T20:51:33.875 INFO:teuthology.orchestra.run.smithi174.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-10T20:51:33.877 INFO:teuthology.orchestra.run.smithi033.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T20:51:33.901 INFO:teuthology.orchestra.run.smithi104.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-09-10T20:51:33.903 INFO:teuthology.orchestra.run.smithi033.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-10T20:51:33.928 INFO:teuthology.orchestra.run.smithi033.stdout:(138/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-10T20:51:33.934 INFO:teuthology.orchestra.run.smithi104.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.3 MB/s | 151 kB 00:00 2024-09-10T20:51:33.953 INFO:teuthology.orchestra.run.smithi033.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-10T20:51:33.979 INFO:teuthology.orchestra.run.smithi033.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-10T20:51:33.993 INFO:teuthology.orchestra.run.smithi174.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-10T20:51:34.004 INFO:teuthology.orchestra.run.smithi033.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-10T20:51:34.026 INFO:teuthology.orchestra.run.smithi104.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 5.9 MB/s | 548 kB 00:00 2024-09-10T20:51:34.029 INFO:teuthology.orchestra.run.smithi033.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-10T20:51:34.054 INFO:teuthology.orchestra.run.smithi033.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-10T20:51:34.059 INFO:teuthology.orchestra.run.smithi174.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-09-10T20:51:34.059 INFO:teuthology.orchestra.run.smithi104.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-10T20:51:34.079 INFO:teuthology.orchestra.run.smithi033.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-10T20:51:34.093 INFO:teuthology.orchestra.run.smithi174.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-10T20:51:34.093 INFO:teuthology.orchestra.run.smithi104.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 961 kB/s | 32 kB 00:00 2024-09-10T20:51:34.105 INFO:teuthology.orchestra.run.smithi033.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T20:51:34.119 INFO:teuthology.orchestra.run.smithi104.stdout:(117/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-09-10T20:51:34.130 INFO:teuthology.orchestra.run.smithi033.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-10T20:51:34.135 INFO:teuthology.orchestra.run.smithi174.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 770 kB/s | 32 kB 00:00 2024-09-10T20:51:34.144 INFO:teuthology.orchestra.run.smithi104.stdout:(118/157): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-10T20:51:34.160 INFO:teuthology.orchestra.run.smithi174.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-10T20:51:34.164 INFO:teuthology.orchestra.run.smithi033.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-10T20:51:34.177 INFO:teuthology.orchestra.run.smithi104.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-10T20:51:34.197 INFO:teuthology.orchestra.run.smithi033.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.9 MB/s | 427 kB 00:00 2024-09-10T20:51:34.202 INFO:teuthology.orchestra.run.smithi174.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-10T20:51:34.228 INFO:teuthology.orchestra.run.smithi104.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-10T20:51:34.253 INFO:teuthology.orchestra.run.smithi104.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-10T20:51:34.269 INFO:teuthology.orchestra.run.smithi174.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.4 MB/s | 4.4 MB 00:00 2024-09-10T20:51:34.289 INFO:teuthology.orchestra.run.smithi033.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 9.9 MB/s | 1.6 MB 00:00 2024-09-10T20:51:34.294 INFO:teuthology.orchestra.run.smithi174.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2024-09-10T20:51:34.295 INFO:teuthology.orchestra.run.smithi104.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-10T20:51:34.320 INFO:teuthology.orchestra.run.smithi174.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-10T20:51:34.337 INFO:teuthology.orchestra.run.smithi104.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-09-10T20:51:34.344 INFO:teuthology.orchestra.run.smithi174.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T20:51:34.378 INFO:teuthology.orchestra.run.smithi174.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T20:51:34.396 INFO:teuthology.orchestra.run.smithi104.stdout:(124/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.7 MB/s | 4.4 MB 00:00 2024-09-10T20:51:34.404 INFO:teuthology.orchestra.run.smithi174.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-10T20:51:34.412 INFO:teuthology.orchestra.run.smithi104.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 143 kB/s | 11 kB 00:00 2024-09-10T20:51:34.429 INFO:teuthology.orchestra.run.smithi174.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-10T20:51:34.429 INFO:teuthology.orchestra.run.smithi104.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 530 kB/s | 18 kB 00:00 2024-09-10T20:51:34.446 INFO:teuthology.orchestra.run.smithi104.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 688 kB/s | 23 kB 00:00 2024-09-10T20:51:34.454 INFO:teuthology.orchestra.run.smithi174.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-10T20:51:34.463 INFO:teuthology.orchestra.run.smithi104.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 579 kB/s | 19 kB 00:00 2024-09-10T20:51:34.473 INFO:teuthology.orchestra.run.smithi033.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 12 MB/s | 3.2 MB 00:00 2024-09-10T20:51:34.479 INFO:teuthology.orchestra.run.smithi174.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 783 kB/s | 19 kB 00:00 2024-09-10T20:51:34.480 INFO:teuthology.orchestra.run.smithi104.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 587 kB/s | 20 kB 00:00 2024-09-10T20:51:34.497 INFO:teuthology.orchestra.run.smithi104.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 2.8 MB/s | 46 kB 00:00 2024-09-10T20:51:34.504 INFO:teuthology.orchestra.run.smithi174.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-10T20:51:34.531 INFO:teuthology.orchestra.run.smithi104.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T20:51:34.555 INFO:teuthology.orchestra.run.smithi174.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 8.7 MB/s | 2.0 MB 00:00 2024-09-10T20:51:34.564 INFO:teuthology.orchestra.run.smithi104.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-09-10T20:51:34.573 INFO:teuthology.orchestra.run.smithi033.stdout:(151/157): librados2-19.3.0-4854.g8f219c47.el9. 8.5 MB/s | 3.5 MB 00:00 2024-09-10T20:51:34.580 INFO:teuthology.orchestra.run.smithi174.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T20:51:34.590 INFO:teuthology.orchestra.run.smithi104.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-09-10T20:51:34.597 INFO:teuthology.orchestra.run.smithi174.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 4.7 MB/s | 79 kB 00:00 2024-09-10T20:51:34.615 INFO:teuthology.orchestra.run.smithi104.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-09-10T20:51:34.622 INFO:teuthology.orchestra.run.smithi174.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-10T20:51:34.624 INFO:teuthology.orchestra.run.smithi033.stdout:(152/157): NetworkManager-1.49.90-1.el9.x86_64. 6.9 MB/s | 2.3 MB 00:00 2024-09-10T20:51:34.640 INFO:teuthology.orchestra.run.smithi104.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T20:51:34.656 INFO:teuthology.orchestra.run.smithi174.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-10T20:51:34.665 INFO:teuthology.orchestra.run.smithi104.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T20:51:34.674 INFO:teuthology.orchestra.run.smithi033.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-10T20:51:34.681 INFO:teuthology.orchestra.run.smithi174.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 658 kB/s | 16 kB 00:00 2024-09-10T20:51:34.691 INFO:teuthology.orchestra.run.smithi104.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-10T20:51:34.707 INFO:teuthology.orchestra.run.smithi174.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-10T20:51:34.708 INFO:teuthology.orchestra.run.smithi033.stdout:(154/157): NetworkManager-team-1.49.90-1.el9.x8 305 kB/s | 41 kB 00:00 2024-09-10T20:51:34.716 INFO:teuthology.orchestra.run.smithi104.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-10T20:51:34.732 INFO:teuthology.orchestra.run.smithi174.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-10T20:51:34.749 INFO:teuthology.orchestra.run.smithi033.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 6.5 MB/s | 1.8 MB 00:00 2024-09-10T20:51:34.757 INFO:teuthology.orchestra.run.smithi174.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-10T20:51:34.758 INFO:teuthology.orchestra.run.smithi104.stdout:(139/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.0 MB/s | 2.0 MB 00:00 2024-09-10T20:51:34.782 INFO:teuthology.orchestra.run.smithi174.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 619 kB/s | 31 kB 00:00 2024-09-10T20:51:34.783 INFO:teuthology.orchestra.run.smithi104.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 886 kB/s | 59 kB 00:00 2024-09-10T20:51:34.791 INFO:teuthology.orchestra.run.smithi033.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-10T20:51:34.807 INFO:teuthology.orchestra.run.smithi174.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T20:51:34.808 INFO:teuthology.orchestra.run.smithi104.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-10T20:51:34.833 INFO:teuthology.orchestra.run.smithi174.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 1.4 MB/s | 36 kB 00:00 2024-09-10T20:51:34.834 INFO:teuthology.orchestra.run.smithi104.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-10T20:51:34.858 INFO:teuthology.orchestra.run.smithi174.stdout:(141/157): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-10T20:51:34.859 INFO:teuthology.orchestra.run.smithi104.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 988 kB/s | 74 kB 00:00 2024-09-10T20:51:34.883 INFO:teuthology.orchestra.run.smithi174.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-10T20:51:34.884 INFO:teuthology.orchestra.run.smithi104.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-10T20:51:34.909 INFO:teuthology.orchestra.run.smithi174.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T20:51:34.909 INFO:teuthology.orchestra.run.smithi104.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T20:51:34.934 INFO:teuthology.orchestra.run.smithi174.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T20:51:34.935 INFO:teuthology.orchestra.run.smithi104.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 398 kB/s | 20 kB 00:00 2024-09-10T20:51:34.941 INFO:teuthology.orchestra.run.smithi033.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-09-10T20:51:34.948 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:51:34.948 INFO:teuthology.orchestra.run.smithi033.stdout:Total 17 MB/s | 240 MB 00:14 2024-09-10T20:51:34.960 INFO:teuthology.orchestra.run.smithi104.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T20:51:34.967 INFO:teuthology.orchestra.run.smithi174.stdout:(145/157): python3-werkzeug-2.0.3-3.el9.1.noarc 7.2 MB/s | 427 kB 00:00 2024-09-10T20:51:34.985 INFO:teuthology.orchestra.run.smithi104.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-10T20:51:34.992 INFO:teuthology.orchestra.run.smithi174.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-10T20:51:35.017 INFO:teuthology.orchestra.run.smithi174.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-10T20:51:35.052 INFO:teuthology.orchestra.run.smithi104.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T20:51:35.085 INFO:teuthology.orchestra.run.smithi174.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-10T20:51:35.344 INFO:teuthology.orchestra.run.smithi104.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 9.0 MB/s | 3.2 MB 00:00 2024-09-10T20:51:35.368 INFO:teuthology.orchestra.run.smithi174.stdout:(149/157): librados2-19.3.0-4854.g8f219c47.el9. 9.9 MB/s | 3.5 MB 00:00 2024-09-10T20:51:35.395 INFO:teuthology.orchestra.run.smithi104.stdout:(151/157): librados2-19.3.0-4854.g8f219c47.el9. 8.0 MB/s | 3.5 MB 00:00 2024-09-10T20:51:35.419 INFO:teuthology.orchestra.run.smithi174.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 9.7 MB/s | 3.2 MB 00:00 2024-09-10T20:51:35.485 INFO:teuthology.orchestra.run.smithi174.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-10T20:51:35.527 INFO:teuthology.orchestra.run.smithi174.stdout:(152/157): NetworkManager-libnm-1.49.90-1.el9.x 17 MB/s | 1.8 MB 00:00 2024-09-10T20:51:35.553 INFO:teuthology.orchestra.run.smithi174.stdout:(153/157): NetworkManager-team-1.49.90-1.el9.x8 609 kB/s | 41 kB 00:00 2024-09-10T20:51:35.611 INFO:teuthology.orchestra.run.smithi174.stdout:(154/157): NetworkManager-tui-1.49.90-1.el9.x86 2.9 MB/s | 249 kB 00:00 2024-09-10T20:51:35.686 INFO:teuthology.orchestra.run.smithi104.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 140 kB/s | 41 kB 00:00 2024-09-10T20:51:35.703 INFO:teuthology.orchestra.run.smithi174.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-10T20:51:35.762 INFO:teuthology.orchestra.run.smithi174.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-09-10T20:51:36.162 INFO:teuthology.orchestra.run.smithi104.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 523 kB/s | 249 kB 00:00 2024-09-10T20:51:36.212 INFO:teuthology.orchestra.run.smithi104.stdout:(154/157): NetworkManager-1.49.90-1.el9.x86_64. 2.0 MB/s | 2.3 MB 00:01 2024-09-10T20:51:36.222 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-09-10T20:51:36.312 INFO:teuthology.orchestra.run.smithi104.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 1.9 MB/s | 1.8 MB 00:00 2024-09-10T20:51:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-09-10T20:51:36.394 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-09-10T20:51:36.396 INFO:teuthology.orchestra.run.smithi104.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 5.8 MB/s | 1.4 MB 00:00 2024-09-10T20:51:36.796 INFO:teuthology.orchestra.run.smithi104.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.1 MB/s | 2.4 MB 00:00 2024-09-10T20:51:36.802 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:51:36.802 INFO:teuthology.orchestra.run.smithi104.stdout:Total 18 MB/s | 240 MB 00:13 2024-09-10T20:51:38.037 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-09-10T20:51:38.188 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-09-10T20:51:38.189 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-09-10T20:51:38.790 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-09-10T20:51:38.791 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-09-10T20:51:39.028 INFO:teuthology.orchestra.run.smithi174.stdout:(157/157): python3-babel-2.9.1-2.el9.noarch.rpm 665 kB/s | 6.0 MB 00:09 2024-09-10T20:51:39.034 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:51:39.034 INFO:teuthology.orchestra.run.smithi174.stdout:Total 13 MB/s | 240 MB 00:18 2024-09-10T20:51:40.088 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-09-10T20:51:40.088 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-09-10T20:51:40.272 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-09-10T20:51:40.425 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-09-10T20:51:40.425 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-09-10T20:51:42.329 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-09-10T20:51:42.330 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-09-10T20:51:45.591 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-09-10T20:51:45.729 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T20:51:45.781 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T20:51:45.839 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T20:51:45.880 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T20:51:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T20:51:45.966 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T20:51:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T20:51:46.439 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:51:46.573 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:51:46.607 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:51:46.615 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-09-10T20:51:46.707 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T20:51:46.714 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:51:46.745 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T20:51:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:51:46.792 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T20:51:46.798 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T20:51:46.832 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T20:51:46.858 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T20:51:46.901 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T20:51:46.976 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T20:51:47.035 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T20:51:47.062 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:51:47.152 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:51:47.180 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:51:47.243 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:51:47.245 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T20:51:47.281 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:51:47.305 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:51:47.359 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:51:47.488 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:51:47.501 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:51:47.518 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:51:47.608 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:51:47.639 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:51:47.668 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T20:51:47.673 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:51:47.813 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T20:51:47.824 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T20:51:47.878 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T20:51:47.906 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:51:47.937 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:48.009 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:51:48.030 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:48.032 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:51:48.094 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:51:48.116 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:51:48.139 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:51:48.267 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:48.326 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:51:48.382 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T20:51:48.410 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T20:51:48.446 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T20:51:48.465 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:51:48.522 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T20:51:48.573 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T20:51:48.580 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T20:51:48.620 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T20:51:48.663 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T20:51:48.671 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:48.685 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-09-10T20:51:48.696 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T20:51:48.724 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T20:51:48.758 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:48.797 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T20:51:48.815 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T20:51:48.848 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T20:51:48.853 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T20:51:48.885 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T20:51:48.906 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T20:51:48.938 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T20:51:48.940 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T20:51:48.966 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T20:51:48.983 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:49.008 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T20:51:49.025 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T20:51:49.056 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T20:51:49.100 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T20:51:49.103 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T20:51:49.128 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T20:51:49.153 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T20:51:49.189 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T20:51:49.223 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T20:51:49.270 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T20:51:49.277 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T20:51:49.314 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T20:51:49.354 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T20:51:49.356 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T20:51:49.398 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T20:51:49.412 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T20:51:49.430 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T20:51:49.458 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T20:51:49.501 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T20:51:49.525 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:51:49.534 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T20:51:49.564 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T20:51:49.597 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T20:51:49.627 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T20:51:49.631 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:51:49.663 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T20:51:49.668 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:51:49.742 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T20:51:49.759 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:51:49.774 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T20:51:49.789 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:51:49.817 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T20:51:49.818 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T20:51:49.845 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T20:51:49.916 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T20:51:49.969 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T20:51:49.971 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T20:51:50.019 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T20:51:50.044 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T20:51:50.047 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:51:50.127 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T20:51:50.152 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:51:50.173 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:51:50.174 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T20:51:50.219 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T20:51:50.219 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T20:51:50.229 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T20:51:50.241 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:51:50.262 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T20:51:50.266 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T20:51:50.314 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T20:51:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T20:51:50.394 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T20:51:50.434 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:51:50.569 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T20:51:50.584 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T20:51:50.646 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T20:51:50.680 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T20:51:50.741 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T20:51:50.752 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T20:51:50.776 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:50.804 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T20:51:50.844 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T20:51:50.865 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:50.874 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T20:51:50.884 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T20:51:50.924 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T20:51:50.931 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T20:51:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T20:51:50.971 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T20:51:51.012 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T20:51:51.015 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T20:51:51.050 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T20:51:51.092 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T20:51:51.172 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T20:51:51.189 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T20:51:51.203 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T20:51:51.235 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T20:51:51.240 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T20:51:51.270 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T20:51:51.331 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T20:51:51.335 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T20:51:51.379 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T20:51:51.395 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T20:51:51.420 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T20:51:51.439 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T20:51:51.463 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T20:51:51.468 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T20:51:51.496 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T20:51:51.509 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T20:51:51.524 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T20:51:51.545 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T20:51:51.591 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T20:51:51.592 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T20:51:51.595 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T20:51:51.627 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T20:51:51.637 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T20:51:51.646 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T20:51:51.670 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T20:51:51.676 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T20:51:51.687 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T20:51:51.711 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T20:51:51.712 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T20:51:51.796 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T20:51:51.803 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T20:51:51.830 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T20:51:51.837 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T20:51:51.849 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T20:51:51.873 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T20:51:51.878 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T20:51:51.889 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T20:51:51.902 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T20:51:51.910 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T20:51:51.930 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T20:51:51.935 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T20:51:51.971 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T20:51:52.017 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T20:51:52.038 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T20:51:52.041 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T20:51:52.080 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T20:51:52.091 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T20:51:52.120 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T20:51:52.146 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T20:51:52.168 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T20:51:52.170 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T20:51:52.217 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T20:51:52.226 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T20:51:52.248 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T20:51:52.277 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T20:51:52.284 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T20:51:52.319 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T20:51:52.322 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T20:51:52.359 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T20:51:52.389 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T20:51:52.418 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T20:51:52.451 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T20:51:52.502 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T20:51:52.536 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T20:51:52.617 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T20:51:52.627 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T20:51:52.660 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T20:51:52.688 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T20:51:52.798 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T20:51:52.809 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T20:51:52.893 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T20:51:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T20:51:52.922 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T20:51:52.964 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T20:51:53.002 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T20:51:53.031 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T20:51:53.044 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T20:51:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T20:51:53.144 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T20:51:53.157 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T20:51:53.173 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T20:51:53.200 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T20:51:53.207 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T20:51:53.216 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T20:51:53.241 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T20:51:53.263 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T20:51:53.283 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T20:51:53.317 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T20:51:53.355 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T20:51:53.406 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T20:51:53.419 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T20:51:53.464 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T20:51:53.492 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T20:51:53.495 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T20:51:53.523 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T20:51:53.559 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T20:51:53.599 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T20:51:53.606 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T20:51:53.642 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T20:51:53.654 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T20:51:53.685 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T20:51:53.727 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T20:51:53.863 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T20:51:53.904 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T20:51:53.945 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T20:51:53.994 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T20:51:54.037 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T20:51:54.048 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T20:51:54.071 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T20:51:54.078 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T20:51:54.118 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T20:51:54.152 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T20:51:54.173 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T20:51:54.176 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T20:51:54.235 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T20:51:54.251 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T20:51:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T20:51:54.378 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T20:51:54.379 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T20:51:54.381 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T20:51:54.415 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T20:51:54.417 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T20:51:54.435 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T20:51:54.445 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T20:51:54.469 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T20:51:54.479 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T20:51:54.518 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T20:51:54.550 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T20:51:54.570 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T20:51:54.579 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T20:51:54.609 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T20:51:54.614 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T20:51:54.623 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T20:51:54.637 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T20:51:54.647 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T20:51:54.669 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T20:51:54.680 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T20:51:54.718 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T20:51:54.724 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T20:51:54.758 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T20:51:54.804 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T20:51:54.810 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T20:51:54.862 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T20:51:54.893 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T20:51:54.988 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T20:51:54.997 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T20:51:55.018 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T20:51:55.043 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T20:51:55.096 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T20:51:55.420 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T20:51:55.541 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T20:51:55.743 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T20:51:55.765 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T20:51:55.807 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T20:51:55.841 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T20:51:55.868 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T20:51:55.940 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T20:51:55.995 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T20:51:56.040 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T20:51:56.128 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T20:51:56.160 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T20:51:56.345 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T20:51:56.374 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T20:51:56.452 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T20:51:56.867 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T20:51:56.950 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T20:51:56.981 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T20:51:57.011 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T20:51:57.096 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T20:51:57.099 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T20:51:57.188 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T20:51:57.213 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T20:51:57.216 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T20:51:57.249 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T20:51:57.349 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T20:51:57.398 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:51:57.499 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:51:57.531 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T20:51:57.567 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T20:51:57.570 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T20:51:57.620 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:51:57.702 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:51:57.733 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T20:51:57.778 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T20:51:58.089 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T20:51:58.128 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:51:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:51:58.238 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T20:51:58.274 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T20:51:58.307 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:51:58.411 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:51:58.442 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T20:51:58.446 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T20:51:58.543 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T20:51:58.572 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T20:51:58.604 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T20:51:58.916 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T20:51:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:51:59.050 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T20:51:59.080 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T20:51:59.115 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T20:51:59.610 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T20:51:59.636 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:51:59.690 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T20:51:59.721 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T20:52:00.690 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:00.726 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:00.789 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:00.823 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:01.331 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:01.371 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:52:01.376 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:01.417 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:52:01.464 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:52:01.464 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T20:52:01.465 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T20:52:01.465 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:01.506 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:52:01.506 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T20:52:01.506 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T20:52:01.506 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:01.507 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:52:01.535 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:52:02.091 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:02.125 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:02.680 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T20:52:02.711 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:52:02.803 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T20:52:02.803 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T20:52:02.803 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T20:52:02.803 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:02.838 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:52:13.387 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:52:13.387 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-09-10T20:52:13.388 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-09-10T20:52:13.388 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-09-10T20:52:13.388 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-09-10T20:52:13.388 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-09-10T20:52:13.388 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-09-10T20:52:13.388 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-09-10T20:52:13.388 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:13.553 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:52:13.570 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:52:13.570 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:13.570 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T20:52:13.570 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:13.847 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-09-10T20:52:13.848 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:14.049 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:52:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:52:14.066 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:14.066 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T20:52:14.066 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-09-10T20:52:14.475 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:14.647 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:52:14.664 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T20:52:14.664 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:14.664 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T20:52:14.664 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:15.056 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:52:15.083 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:52:15.083 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:15.083 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T20:52:15.083 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:52:15.083 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:52:15.083 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:15.307 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:52:15.330 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:52:15.330 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:15.330 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T20:52:15.330 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:52:15.330 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:52:15.331 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:15.644 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:52:15.672 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:52:15.672 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:15.672 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T20:52:15.672 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:52:15.672 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:52:15.672 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:15.782 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:52:15.806 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:52:15.806 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:15.806 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T20:52:15.806 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:52:15.806 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:52:15.807 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:15.911 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:52:15.913 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T20:52:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:52:15.936 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:15.936 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T20:52:15.936 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:52:15.936 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:52:15.936 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:15.969 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T20:52:16.025 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T20:52:16.183 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:52:16.210 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T20:52:16.210 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:16.210 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T20:52:16.210 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:52:16.210 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T20:52:16.211 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:16.222 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T20:52:16.267 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T20:52:16.345 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T20:52:16.412 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:52:16.433 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:52:16.433 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T20:52:16.436 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:52:16.436 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:16.436 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T20:52:16.437 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:52:16.437 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:52:16.437 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:16.457 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T20:52:16.457 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:16.457 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T20:52:16.457 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:52:16.457 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T20:52:16.457 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:16.551 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T20:52:16.583 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T20:52:16.610 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T20:52:16.625 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T20:52:16.657 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T20:52:16.673 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T20:52:16.684 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T20:52:16.877 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T20:52:16.909 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:52:16.922 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T20:52:16.933 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T20:52:16.933 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:16.933 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T20:52:16.933 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:52:16.933 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T20:52:16.933 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:17.002 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T20:52:17.039 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T20:52:17.091 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T20:52:17.095 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T20:52:17.113 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:52:17.141 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:52:17.151 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T20:52:17.173 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T20:52:17.240 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T20:52:17.244 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T20:52:17.264 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:52:17.295 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T20:52:17.329 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T20:52:17.338 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T20:52:17.356 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T20:52:17.385 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T20:52:17.463 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T20:52:17.551 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T20:52:17.699 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T20:52:17.734 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T20:52:17.774 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T20:52:17.802 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T20:52:17.814 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:52:17.846 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:52:17.879 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T20:52:17.948 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T20:52:17.978 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:52:18.189 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:52:18.217 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T20:52:18.249 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T20:52:18.316 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T20:52:18.340 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:52:20.469 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:52:20.536 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:52:21.357 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:52:21.426 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:52:21.494 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:52:21.528 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:52:21.538 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T20:52:21.604 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:52:21.644 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:52:21.806 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T20:52:21.838 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:52:21.863 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:52:21.863 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:21.863 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T20:52:21.863 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:52:21.863 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:52:21.863 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:21.923 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:52:21.935 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:52:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T20:52:22.087 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T20:52:22.445 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:52:22.476 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:52:22.574 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T20:52:22.597 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:52:22.612 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:52:22.728 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T20:52:22.771 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T20:52:22.811 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:52:22.836 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:52:22.836 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:22.837 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T20:52:22.837 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:52:22.837 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:52:22.837 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:22.886 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:52:22.893 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T20:52:22.898 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:52:22.914 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:52:22.939 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T20:52:22.939 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:22.939 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T20:52:22.939 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:52:22.939 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T20:52:22.939 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:22.990 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:52:23.002 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T20:52:23.006 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T20:52:23.035 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T20:52:23.066 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:52:23.076 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T20:52:23.090 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:52:23.090 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:23.090 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T20:52:23.090 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:52:23.090 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:52:23.090 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:23.092 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T20:52:23.133 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:52:23.138 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T20:52:23.156 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:52:23.157 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:23.157 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T20:52:23.157 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:23.201 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:52:23.224 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:52:23.224 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:23.224 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T20:52:23.224 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:52:23.224 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:52:23.224 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:23.535 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:52:23.558 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:52:23.559 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:23.559 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T20:52:23.559 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:52:23.559 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:52:23.559 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T20:52:24.098 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T20:52:24.114 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:52:24.125 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:52:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:52:24.139 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.139 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T20:52:24.139 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:52:24.139 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:52:24.139 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:24.150 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T20:52:24.150 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.150 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T20:52:24.150 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:52:24.150 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T20:52:24.150 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:24.181 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:52:24.193 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:52:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:52:24.205 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.205 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T20:52:24.205 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:24.217 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T20:52:24.217 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.217 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T20:52:24.217 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:24.260 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:52:24.265 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:52:24.284 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:52:24.284 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.284 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T20:52:24.284 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:52:24.284 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:52:24.284 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:24.289 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T20:52:24.290 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.290 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T20:52:24.290 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:52:24.290 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T20:52:24.290 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:24.594 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:52:24.618 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:52:24.618 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.618 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T20:52:24.618 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:52:24.618 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:52:24.618 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:24.624 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:52:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T20:52:24.649 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T20:52:24.649 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T20:52:24.649 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:52:24.649 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T20:52:24.649 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:28.246 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T20:52:28.287 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T20:52:28.328 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T20:52:28.360 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T20:52:28.387 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T20:52:28.576 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T20:52:28.626 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T20:52:28.656 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T20:52:28.686 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T20:52:28.732 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T20:52:29.298 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T20:52:29.339 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T20:52:29.355 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:29.388 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T20:52:29.396 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:29.416 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:29.420 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T20:52:29.447 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T20:52:29.628 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T20:52:29.645 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T20:52:29.668 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T20:52:29.687 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T20:52:29.699 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T20:52:29.728 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T20:52:29.729 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T20:52:29.759 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T20:52:29.766 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T20:52:29.786 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T20:52:29.996 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T20:52:30.041 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T20:52:30.072 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T20:52:30.102 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T20:52:30.147 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T20:52:30.246 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T20:52:30.334 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T20:52:30.338 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:30.374 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T20:52:30.380 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:30.401 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:30.456 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T20:52:30.495 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T20:52:30.529 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T20:52:30.563 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T20:52:30.563 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:52:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:52:30.647 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:52:30.731 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:52:30.756 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T20:52:30.761 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:30.761 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:30.795 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:30.836 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:30.858 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T20:52:30.942 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:30.942 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:52:31.017 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:52:31.088 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:31.240 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T20:52:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T20:52:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T20:52:31.465 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T20:52:31.495 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T20:52:31.521 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T20:52:31.547 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T20:52:31.547 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:52:31.639 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:52:31.639 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:52:31.732 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:52:31.756 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T20:52:31.761 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T20:52:31.762 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:31.762 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:31.849 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T20:52:31.902 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T20:52:31.950 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:31.950 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:52:32.000 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T20:52:32.034 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:52:32.044 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T20:52:32.069 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T20:52:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T20:52:32.095 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:52:32.098 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:32.179 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T20:52:32.180 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:52:32.264 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T20:52:32.289 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T20:52:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:32.294 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T20:52:32.482 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:52:32.566 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T20:52:32.635 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:37.326 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:37.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T20:52:37.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T20:52:37.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T20:52:37.326 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T20:52:37.327 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T20:52:37.328 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T20:52:37.340 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T20:52:37.341 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T20:52:37.342 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T20:52:37.343 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T20:52:37.344 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T20:52:37.345 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T20:52:37.346 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T20:52:37.347 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T20:52:37.348 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T20:52:37.349 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T20:52:37.350 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T20:52:38.171 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T20:52:38.172 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T20:52:38.173 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T20:52:38.173 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T20:52:38.173 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T20:52:38.173 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T20:52:38.173 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T20:52:38.173 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T20:52:38.185 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T20:52:38.186 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T20:52:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T20:52:38.188 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T20:52:38.189 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T20:52:38.190 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T20:52:38.191 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T20:52:38.192 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T20:52:38.193 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T20:52:38.194 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T20:52:38.195 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T20:52:38.195 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T20:52:38.195 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T20:52:38.195 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T20:52:38.195 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T20:52:38.195 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T20:52:38.195 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T20:52:39.026 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:39.026 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T20:52:39.026 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T20:52:39.026 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T20:52:39.027 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T20:52:39.028 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T20:52:39.029 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T20:52:39.029 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T20:52:39.040 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T20:52:39.040 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T20:52:39.040 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T20:52:39.040 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T20:52:39.040 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T20:52:39.041 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T20:52:39.042 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T20:52:39.043 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T20:52:39.044 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T20:52:39.045 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T20:52:39.046 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T20:52:39.047 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T20:52:39.048 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T20:52:39.049 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T20:52:39.050 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T20:52:39.051 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.814 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.815 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.816 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T20:52:39.817 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T20:52:39.818 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T20:52:39.819 INFO:teuthology.orchestra.run.smithi104.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T20:52:39.820 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T20:52:39.821 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T20:52:39.822 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T20:52:39.823 INFO:teuthology.orchestra.run.smithi104.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T20:52:39.824 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T20:52:39.825 INFO:teuthology.orchestra.run.smithi104.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T20:52:39.826 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T20:52:39.827 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T20:52:39.827 INFO:teuthology.orchestra.run.smithi104.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T20:52:39.827 INFO:teuthology.orchestra.run.smithi104.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T20:52:39.827 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T20:52:39.827 INFO:teuthology.orchestra.run.smithi104.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T20:52:39.827 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:52:39.827 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-09-10T20:52:40.241 DEBUG:teuthology.parallel:result is None 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:40.595 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.596 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.597 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T20:52:40.598 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T20:52:40.599 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T20:52:40.600 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T20:52:40.601 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T20:52:40.602 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T20:52:40.603 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T20:52:40.604 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.605 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T20:52:40.606 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T20:52:40.607 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T20:52:40.608 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T20:52:40.608 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:52:40.608 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-09-10T20:52:40.993 DEBUG:teuthology.parallel:result is None 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T20:52:41.685 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.686 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.687 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T20:52:41.688 INFO:teuthology.orchestra.run.smithi033.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.689 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T20:52:41.690 INFO:teuthology.orchestra.run.smithi033.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T20:52:41.691 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T20:52:41.692 INFO:teuthology.orchestra.run.smithi033.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T20:52:41.693 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T20:52:41.694 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T20:52:41.695 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T20:52:41.696 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T20:52:41.697 INFO:teuthology.orchestra.run.smithi033.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T20:52:41.698 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T20:52:41.698 INFO:teuthology.orchestra.run.smithi033.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T20:52:41.698 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:52:41.698 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-09-10T20:52:42.263 DEBUG:teuthology.parallel:result is None 2024-09-10T20:52:42.264 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:52:42.420 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T20:52:42.445 INFO:teuthology.orchestra.run.smithi033.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T20:52:42.445 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T20:52:42.446 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T20:52:42.447 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:52:42.595 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T20:52:42.622 INFO:teuthology.orchestra.run.smithi104.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T20:52:42.622 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T20:52:42.622 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T20:52:42.624 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:52:42.775 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T20:52:42.802 INFO:teuthology.orchestra.run.smithi174.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T20:52:42.803 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T20:52:42.803 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T20:52:42.804 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T20:52:42.805 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:52:42.805 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T20:52:42.839 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:52:42.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T20:52:42.872 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:52:42.872 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T20:52:42.904 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T20:52:42.904 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:52:42.904 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T20:52:42.934 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T20:52:43.003 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:52:43.004 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T20:52:43.033 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T20:52:43.102 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:52:43.102 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T20:52:43.132 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T20:52:43.201 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T20:52:43.201 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:52:43.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T20:52:43.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T20:52:43.299 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:52:43.299 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T20:52:43.329 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T20:52:43.399 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:52:43.399 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T20:52:43.428 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T20:52:43.496 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T20:52:43.497 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:52:43.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T20:52:43.528 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T20:52:43.599 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:52:43.600 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T20:52:43.628 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T20:52:43.695 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:52:43.695 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T20:52:43.726 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T20:52:43.794 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T20:52:43.892 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T20:52:43.893 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:52:43.893 INFO:tasks.cephadm:Cluster fsid is a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:52:43.893 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T20:52:43.894 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.33', 'mon.b': '172.21.15.104', 'mon.c': '172.21.15.174'} 2024-09-10T20:52:43.894 INFO:tasks.cephadm:First mon is mon.a on smithi033 2024-09-10T20:52:43.894 INFO:tasks.cephadm:First mgr is x 2024-09-10T20:52:43.895 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T20:52:43.895 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-09-10T20:52:43.927 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-09-10T20:52:43.956 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-09-10T20:52:43.985 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T20:52:43.985 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:52:44.136 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82938/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4854-g8f219c47', 'node_name': '172.21.2.8+braggi08', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4854.g8f219c47'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 09:27:13.215997', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-hemanth-testing-2024-09-10-0723', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T20:52:44.284 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-hemanth-testing-2024-09-10-0723, sha1 8f219c476cb4f26fc5719c661c0323719a45cb30 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:52:44.285 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T20:52:44.285 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm 2024-09-10T20:52:44.286 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:45.436 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 20:52 /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:45.436 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:46.428 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 20:52 /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:46.428 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:47.605 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 803561 Sep 10 20:52 /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:47.605 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:47.626 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:47.645 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T20:52:47.681 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 on all hosts... 2024-09-10T20:52:47.681 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T20:52:47.684 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T20:52:47.688 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 pull 2024-09-10T20:52:49.258 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T20:52:49.291 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T20:52:49.402 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T20:53:20.790 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T20:53:20.791 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T20:53:20.791 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T20:53:20.791 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-09-10T20:53:20.791 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T20:53:20.791 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-09-10T20:53:20.791 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T20:53:37.558 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T20:53:37.559 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T20:53:37.559 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T20:53:37.559 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-09-10T20:53:37.559 INFO:teuthology.orchestra.run.smithi033.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T20:53:37.559 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T20:53:37.559 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T20:53:39.154 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-10T20:53:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T20:53:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480", 2024-09-10T20:53:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-09-10T20:53:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e" 2024-09-10T20:53:39.155 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-09-10T20:53:39.155 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-10T20:53:39.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-09-10T20:53:39.214 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-09-10T20:53:39.241 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-09-10T20:53:39.275 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-09-10T20:53:39.304 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-09-10T20:53:39.329 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-09-10T20:53:39.357 INFO:tasks.cephadm:Writing seed config... 2024-09-10T20:53:39.358 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-10T20:53:39.358 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-10T20:53:39.358 INFO:tasks.cephadm: override: [mgr] client mount timeout = 30 2024-09-10T20:53:39.358 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-10T20:53:39.359 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T20:53:39.360 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-10T20:53:39.361 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:53:39.361 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T20:53:39.379 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a07b33bc-6fb6-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 client mount timeout = 30 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T20:53:39.379 DEBUG:teuthology.orchestra.run.smithi033:mon.a> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T20:53:39.421 DEBUG:teuthology.orchestra.run.smithi033:mgr.x> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T20:53:39.464 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T20:53:39.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 -v bootstrap --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T20:53:39.679 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-09-10T20:53:39.679 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', '-v', 'bootstrap', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-09-10T20:53:39.679 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T20:53:39.679 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-09-10T20:53:39.703 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 5.2.2 2024-09-10T20:53:39.703 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-09-10T20:53:39.703 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-09-10T20:53:39.711 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T20:53:39.711 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T20:53:39.718 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T20:53:39.718 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-09-10T20:53:39.724 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-09-10T20:53:39.731 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-09-10T20:53:39.731 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-09-10T20:53:39.731 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-09-10T20:53:39.754 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 5.2.2 2024-09-10T20:53:39.755 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T20:53:39.755 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-09-10T20:53:39.755 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-09-10T20:53:39.761 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T20:53:39.761 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T20:53:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T20:53:39.768 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-09-10T20:53:39.774 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-09-10T20:53:39.781 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-09-10T20:53:39.781 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-09-10T20:53:39.781 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-09-10T20:53:39.781 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:39.781 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 140182354436976 on /run/cephadm/a07b33bc-6fb6-11ef-bcea-c7b262605968.lock 2024-09-10T20:53:39.781 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 140182354436976 acquired on /run/cephadm/a07b33bc-6fb6-11ef-bcea-c7b262605968.lock 2024-09-10T20:53:39.782 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-09-10T20:53:39.782 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-09-10T20:53:39.782 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-09-10T20:53:39.785 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-09-10T20:53:39.785 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-09-10T20:53:39.788 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T20:53:39.788 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T20:53:39.788 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2024-09-10T20:53:39.790 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T20:53:39.790 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T20:53:39.790 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T20:53:39.791 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T20:53:39.791 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-09-10T20:53:39.791 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T20:53:39.791 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-09-10T20:53:39.791 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-09-10T20:53:39.791 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T20:53:39.792 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T20:53:39.792 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T20:53:40.268 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T20:53:40.269 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30... 2024-09-10T20:53:40.269 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T20:53:40.269 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:edd8cbe4deb828f809e8d76f308301dd93e971948e909d25cc03015fed1fd796 2024-09-10T20:53:40.269 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T20:53:40.269 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 2024-09-10T20:53:40.269 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T20:53:41.182 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T20:53:41.182 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev) 2024-09-10T20:53:41.182 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T20:53:42.180 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-09-10T20:53:42.181 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-09-10T20:53:43.070 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDWseBmtKV7HRAAP1XntTzvLXRr6yeAjgQY9g== 2024-09-10T20:53:44.017 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDXseBmy4jxGRAA/sr+f9HDfr2CNMvOk7n+xQ== 2024-09-10T20:53:44.899 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQDYseBmS509FBAAQ1154u1kutrm6Kv1fMTYmw== 2024-09-10T20:53:44.899 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-09-10T20:53:45.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T20:53:45.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T20:53:45.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:45.739 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T20:53:45.740 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for a [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T20:53:45.740 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = quincy 2024-09-10T20:53:45.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:45.740 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T20:53:45.740 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:53:45.740 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-09-10T20:53:46.720 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.a on 2024-09-10T20:53:46.949 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T20:53:47.096 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T20:53:47.282 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968.target. 2024-09-10T20:53:47.282 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968.target. 2024-09-10T20:53:47.546 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a 2024-09-10T20:53:47.546 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T20:53:47.716 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968.target.wants/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@.service. 2024-09-10T20:53:48.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:48 smithi033 podman[30475]: 2024-09-10 20:53:48.171607163 +0000 UTC m=+0.319041780 container start afbd475c46a803d239c9169d06b7ef84d6066e565108b0fed889b24f3a1c3ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T20:53:48.216 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:48 smithi033 ceph-mon[30494]: mkfs a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:48.269 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T20:53:48.270 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-09-10T20:53:48.277 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T20:53:48.277 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-09-10T20:53:48.277 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-09-10T20:53:48.277 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T20:53:48.278 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-09-10T20:53:48.278 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-09-10T20:53:48.526 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:48 smithi033 ceph-mon[30494]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T20:53:48.526 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:48 smithi033 bash[30475]: afbd475c46a803d239c9169d06b7ef84d6066e565108b0fed889b24f3a1c3ba9 2024-09-10T20:53:48.526 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:48 smithi033 systemd[1]: Started Ceph mon.a for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:53:49.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T20:53:49.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:49.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T20:53:49.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:53:49.325 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.573464s) 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-09-10T20:53:49.326 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: monmap epoch 1 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: last_changed 2024-09-10T20:53:45.201883+0000 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: min_mon_release 19 (squid) 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: election_strategy: 1 2024-09-10T20:53:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:53:49.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: fsmap 2024-09-10T20:53:49.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T20:53:49.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: mgrmap e1: no daemons active 2024-09-10T20:53:49.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:49 smithi033 ceph-mon[30494]: from='client.? 172.21.15.33:0/3532704080' entity='client.admin' cmd={"prefix": "status"} : dispatch 2024-09-10T20:53:50.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:50 smithi033 ceph-mon[30494]: from='client.? 172.21.15.33:0/1333585616' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T20:53:50.386 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:50 smithi033 ceph-mon[30494]: from='client.? 172.21.15.33:0/1333585616' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T20:53:50.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:53:50.390 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:53:50.391 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T20:53:50.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T20:53:50.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T20:53:50.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T20:53:50.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T20:53:50.392 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T20:53:50.392 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-09-10T20:53:51.422 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-09-10T20:53:51.440 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:51 smithi033 ceph-mon[30494]: from='client.? 172.21.15.33:0/1463401679' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:53:51.739 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:51 smithi033 systemd[1]: Stopping Ceph mon.a for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T20:53:51.739 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a[30490]: 2024-09-10T20:53:51.684+0000 7f0292791640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T20:53:51.740 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a[30490]: 2024-09-10T20:53:51.684+0000 7f0292791640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T20:53:52.005 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:51 smithi033 podman[30736]: 2024-09-10 20:53:51.736534912 +0000 UTC m=+0.150632667 container died afbd475c46a803d239c9169d06b7ef84d6066e565108b0fed889b24f3a1c3ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2) 2024-09-10T20:53:52.339 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 podman[30736]: 2024-09-10 20:53:52.085128137 +0000 UTC m=+0.499225891 container cleanup afbd475c46a803d239c9169d06b7ef84d6066e565108b0fed889b24f3a1c3ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T20:53:52.339 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 bash[30736]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a 2024-09-10T20:53:52.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 podman[30749]: 2024-09-10 20:53:52.335942118 +0000 UTC m=+0.601703444 container remove afbd475c46a803d239c9169d06b7ef84d6066e565108b0fed889b24f3a1c3ba9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T20:53:52.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T20:53:52.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 systemd[1]: Stopped Ceph mon.a for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:53:52.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 systemd[1]: Starting Ceph mon.a for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T20:53:52.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 podman[30841]: 2024-09-10 20:53:52.670362021 +0000 UTC m=+0.098262424 container create 9658efd159e1a861bb6475bb112ffea3de2ab5612ab4332c5bca9173064496d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T20:53:52.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 podman[30841]: 2024-09-10 20:53:52.588396264 +0000 UTC m=+0.016296637 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 podman[30841]: 2024-09-10 20:53:52.864399866 +0000 UTC m=+0.292300226 container init 9658efd159e1a861bb6475bb112ffea3de2ab5612ab4332c5bca9173064496d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 podman[30841]: 2024-09-10 20:53:52.867588714 +0000 UTC m=+0.295489075 container start 9658efd159e1a861bb6475bb112ffea3de2ab5612ab4332c5bca9173064496d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: pidfile_write: ignore empty --pid-file 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: load: jerasure load: lrc 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: RocksDB version: 7.9.2 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Git sha 0 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T20:53:52.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: DB SUMMARY 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: DB Session ID: RM3VL2V9K9QJ7K39XZY4 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: CURRENT file: CURRENT 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 95126 ; 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.error_if_exists: 0 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.create_if_missing: 0 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.paranoid_checks: 1 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T20:53:52.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.env: 0x5647c1ed6180 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.info_log: 0x5647c3e9f700 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.statistics: (nil) 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.use_fsync: 0 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_log_file_size: 0 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.allow_fallocate: 1 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T20:53:52.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.use_direct_reads: 0 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.db_log_dir: 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.wal_dir: 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T20:53:52.931 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.write_buffer_manager: 0x5647c3ea3b80 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T20:53:52.932 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T20:53:52.933 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.unordered_write: 0 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.row_cache: None 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.wal_filter: None 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T20:53:52.934 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.two_write_queues: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.wal_compression: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.atomic_flush: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.log_readahead_size: 0 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T20:53:52.935 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_background_jobs: 2 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_background_compactions: -1 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_subcompactions: 1 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T20:53:52.936 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_open_files: -1 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_background_flushes: -1 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Compression algorithms supported: 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kZSTD supported: 0 2024-09-10T20:53:52.937 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kXpressCompression supported: 0 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kBZip2Compression supported: 0 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kLZ4Compression supported: 1 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kZlibCompression supported: 1 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: kSnappyCompression supported: 1 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.merge_operator: 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_filter: None 2024-09-10T20:53:52.938 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5647c3e9f320) 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_type: 0 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_index_type: 0 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_shortening: 1 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T20:53:52.939 INFO:journalctl@ceph.mon.a.smithi033.stdout: checksum: 4 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: no_block_cache: 0 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache: 0x5647c3e93610 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_options: 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: capacity : 536870912 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_shard_bits : 4 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: strict_capacity_limit : 0 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_cache_compressed: (nil) 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: persistent_cache: (nil) 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size: 4096 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_size_deviation: 10 2024-09-10T20:53:52.940 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_restart_interval: 16 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: index_block_restart_interval: 1 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: metadata_block_size: 4096 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: partition_filters: 0 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: use_delta_encoding: 1 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: filter_policy: bloomfilter 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: whole_key_filtering: 1 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: verify_compression: 0 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: format_version: 5 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_index_compression: 1 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: block_align: 0 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: max_auto_readahead_size: 262144 2024-09-10T20:53:52.941 INFO:journalctl@ceph.mon.a.smithi033.stdout: prepopulate_block_cache: 0 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout: initial_auto_readahead_size: 8192 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression: NoCompression 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.num_levels: 7 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T20:53:52.942 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T20:53:52.943 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T20:53:52.944 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T20:53:52.945 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.inplace_update_support: 0 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T20:53:52.946 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.bloom_locality: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.max_successive_merges: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.ttl: 2592000 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T20:53:52.947 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.enable_blob_files: false 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.min_blob_size: 0 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T20:53:52.948 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 3ddd5c8b-ca35-4cf5-b11c-03d5fbd70b1e 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001632894298, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001632895267, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 91279, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 285, "table_properties": {"data_size": 89327, "index_size": 272, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11686, "raw_average_key_size": 47, "raw_value_size": 82747, "raw_average_value_size": 339, "num_data_blocks": 12, "num_entries": 244, "num_filter_entries": 244, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726001632, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "3ddd5c8b-ca35-4cf5-b11c-03d5fbd70b1e", "db_session_id": "RM3VL2V9K9QJ7K39XZY4", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001632895366, "job": 1, "event": "recovery_finished"} 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5647c3ec2e00 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: DB pointer 0x5647c3fc4000 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** DB Stats ** 2024-09-10T20:53:52.949 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: L0 2/0 91.00 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Sum 2/0 91.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T20:53:52.950 INFO:journalctl@ceph.mon.a.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** Compaction Stats [default] ** 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 143.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T20:53:52.951 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: Cumulative compaction: 0.00 GB write, 27.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: Interval compaction: 0.00 GB write, 27.35 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache BinnedLRUCache@0x5647c3e93610#2 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: starting mon.a rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???) e1 preinit fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???).mds e1 new map 2024-09-10T20:53:52.952 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???).mds e1 print_map 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout: e1 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout: btime 2024-09-10T20:53:48:210161+0000 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout: legacy client fscid: -1 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout: 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout: No filesystems configured 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:53:52.953 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T20:53:52.955 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T20:53:53.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 bash[30841]: 9658efd159e1a861bb6475bb112ffea3de2ab5612ab4332c5bca9173064496d3 2024-09-10T20:53:53.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 systemd[1]: Started Ceph mon.a for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:53:53.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T20:53:53.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: monmap epoch 1 2024-09-10T20:53:53.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:53:53.201 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: last_changed 2024-09-10T20:53:45.201883+0000 2024-09-10T20:53:53.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:53:53.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: min_mon_release 19 (squid) 2024-09-10T20:53:53.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: election_strategy: 1 2024-09-10T20:53:53.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:53:53.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: fsmap 2024-09-10T20:53:53.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T20:53:53.202 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:52 smithi033 ceph-mon[30860]: mgrmap e1: no daemons active 2024-09-10T20:53:54.011 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T20:53:54.032 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T20:53:54.032 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-09-10T20:53:54.032 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T20:53:54.033 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T20:53:54.295 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.x 2024-09-10T20:53:54.295 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-10T20:53:54.447 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968.target.wants/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@.service. 2024-09-10T20:53:54.668 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:54 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3544338417' entity='client.admin' 2024-09-10T20:53:54.668 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:53:54 smithi033 systemd[1]: Starting Ceph mgr.x for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T20:53:54.921 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:53:54 smithi033 podman[31074]: 2024-09-10 20:53:54.664817327 +0000 UTC m=+0.101810608 container create 72b7c416be32aed30b5a1ef8e4f7874183cfb96fa503e87f07e9930a9563a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T20:53:54.921 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:53:54 smithi033 podman[31074]: 2024-09-10 20:53:54.57903905 +0000 UTC m=+0.016032302 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T20:53:54.921 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:53:54 smithi033 podman[31074]: 2024-09-10 20:53:54.850689656 +0000 UTC m=+0.287682902 container init 72b7c416be32aed30b5a1ef8e4f7874183cfb96fa503e87f07e9930a9563a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T20:53:54.921 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:53:54 smithi033 podman[31074]: 2024-09-10 20:53:54.853422648 +0000 UTC m=+0.290415888 container start 72b7c416be32aed30b5a1ef8e4f7874183cfb96fa503e87f07e9930a9563a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T20:53:54.921 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:53:54 smithi033 bash[31074]: 72b7c416be32aed30b5a1ef8e4f7874183cfb96fa503e87f07e9930a9563a924 2024-09-10T20:53:54.936 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T20:53:54.936 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-09-10T20:53:54.943 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T20:53:54.944 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-09-10T20:53:54.944 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-09-10T20:53:54.944 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T20:53:54.953 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T20:53:54.953 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-09-10T20:53:54.960 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T20:53:54.960 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-09-10T20:53:54.961 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-09-10T20:53:54.961 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T20:53:54.961 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-09-10T20:53:54.961 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-09-10T20:53:55.177 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:53:54 smithi033 systemd[1]: Started Ceph mgr.x for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:53:58.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:53:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3215035865' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T20:54:00.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: Activating manager daemon x 2024-09-10T20:54:00.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: mgrmap e2: x(active, starting, since 0.00326418s) 2024-09-10T20:54:00.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T20:54:00.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T20:54:00.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T20:54:00.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:54:00.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T20:54:00.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: Manager daemon x is now available 2024-09-10T20:54:00.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T20:54:00.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T20:54:00.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' 2024-09-10T20:54:00.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' 2024-09-10T20:54:00.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:00 smithi033 ceph-mon[30860]: from='mgr.14100 172.21.15.33:0/3365464388' entity='mgr.x' 2024-09-10T20:54:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:02 smithi033 ceph-mon[30860]: mgrmap e3: x(active, since 1.0077s) 2024-09-10T20:54:03.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:03 smithi033 ceph-mon[30860]: mgrmap e4: x(active, since 2s) 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T20:54:04.490 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:04.491 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T20:54:04.492 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T20:54:04.492 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T20:54:04.492 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T20:54:04.492 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T20:54:04.492 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T20:54:04.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:04.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T20:54:04.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T20:54:04.495 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T20:54:04.496 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T20:53:48:210161+0000", 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T20:54:04.497 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T20:53:48.210866+0000", 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-09-10T20:54:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-09-10T20:54:07.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:07 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4120892884' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T20:54:07.745 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-09-10T20:54:07.748 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T20:54:07.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T20:54:07.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T20:54:07.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T20:54:07.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T20:54:07.750 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T20:54:07.753 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:07.753 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T20:54:07.754 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:07.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T20:54:07.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:07.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T20:53:48:210161+0000", 2024-09-10T20:54:07.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T20:54:07.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T20:54:07.756 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T20:54:07.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T20:53:48.210866+0000", 2024-09-10T20:54:07.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T20:54:07.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-09-10T20:54:07.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T20:54:07.757 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-09-10T20:54:07.757 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-09-10T20:54:08.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:08 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3694699138' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T20:54:08.903 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T20:54:08.904 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T20:54:08.905 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-09-10T20:54:09.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:09 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/25072993' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-10T20:54:09.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:09 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: ignoring --setuser ceph since I am not root 2024-09-10T20:54:09.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:09 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: ignoring --setgroup ceph since I am not root 2024-09-10T20:54:10.913 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:10 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/25072993' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T20:54:10.914 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:10 smithi033 ceph-mon[30860]: mgrmap e5: x(active, since 9s) 2024-09-10T20:54:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:11 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/100711612' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T20:54:13.636 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-09-10T20:54:13.636 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T20:54:13.636 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T20:54:13.636 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T20:54:13.636 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T20:54:13.636 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-09-10T20:54:13.637 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-09-10T20:54:13.637 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: Active manager daemon x restarted 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: Activating manager daemon x 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: mgrmap e6: x(active, starting, since 0.00365332s) 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T20:54:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:14 smithi033 ceph-mon[30860]: Manager daemon x is now available 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: Found migration_current of "None". Setting to last migration. 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T20:54:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: mgrmap e7: x(active, since 1.0093s) 2024-09-10T20:54:16.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:16 smithi033 ceph-mon[30860]: from='client.14120 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T20:54:16.969 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-09-10T20:54:16.969 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T20:54:16.969 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T20:54:16.969 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-09-10T20:54:16.969 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-09-10T20:54:16.969 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T20:54:17.029+0000 7f7f56a06640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:20:54:17] ENGINE Error in HTTPServer.serve 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: Traceback (most recent call last): 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self._connections.run(self.expiration_interval) 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self._run(expiration_interval) 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T20:54:17.302 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: s = self.context.wrap_socket( 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: return self.sslsocket_class._create( 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self.do_handshake() 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self._sslobj.do_handshake() 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T20:54:17.303 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T20:54:17.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:16] ENGINE Bus STARTING 2024-09-10T20:54:17.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:16] ENGINE Serving on http://172.21.15.33:8765 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:17] ENGINE Serving on https://172.21.15.33:7150 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:17] ENGINE Bus STARTED 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:17] ENGINE Error in HTTPServer.serve 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: Traceback (most recent call last): 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: self._connections.run(self.expiration_interval) 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: self._run(expiration_interval) 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T20:54:17.880 INFO:journalctl@ceph.mon.a.smithi033.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: s = self.context.wrap_socket( 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: return self.sslsocket_class._create( 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: self.do_handshake() 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: self._sslobj.do_handshake() 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:17.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:17 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:18.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:18 smithi033 ceph-mon[30860]: mgrmap e8: x(active, since 2s) 2024-09-10T20:54:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T20:54:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-09-10T20:54:19.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:19 smithi033 ceph-mon[30860]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: Generating public/private rsa key pair. 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: Your identification has been saved in /tmp/tmptpcr_914/key 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: Your public key has been saved in /tmp/tmptpcr_914/key.pub 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: The key fingerprint is: 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: SHA256:PoAhvnuS+qhw496bNHeLIlBZ0yeI1vhtPZ5V1kmAib8 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: The key's randomart image is: 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: +---[RSA 3072]----+ 2024-09-10T20:54:20.345 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: | + o . o.+.. | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: | + = o o o o o | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: | ..+.o + . o | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: | .o..oo o o | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: | .. ....S+ . | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: |. . oo E | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: |..+.o . + | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: |.++=o+ o o | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: |=+=++o. . | 2024-09-10T20:54:20.346 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: +----[SHA256]-----+ 2024-09-10T20:54:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:21 smithi033 ceph-mon[30860]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:21 smithi033 ceph-mon[30860]: Generating ssh key... 2024-09-10T20:54:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:21 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:21 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:21.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEUIy25/yLchglMHh1S+HUGYyLtG/2GWqADqi7dM5yJiN69YNmCdtV3tdDzzKDkktlONMHAv1SLTm0oHuxBUzjZ+jTLhvOJfEuyMSgoUq9vT9PIpP5QgVRtjcckiVKt3FmbdH6FnZZwHsqlOCZZ3ZLpBadI4Mq9lxWGrVlTzgjwZRxvmWAjSSq+H93Q+Bavq/FbeMBy8LRr4Ce/UOODURVmWRX0hKobExCui94hk+n9NeQRUNRNAtagvsAj/S6AOLfBxCLmUN8ldx0gAGf4zStaT4SOgjXs4nJFpYj0GALHhLTnYibihleQINr3FK/zBLmwm///Spb/QB0621cH3MffuPrMCWijM54mTeM/DQQh9oGKvU5G9WdeVEhnafU9VtQnJHnKU+KWzkZhsFYLxN8DOS2ot3mijohkhYwdGWNnWjnrDPkX2TcUlhEZIguz4RkHzsptbSvRjf9N2x+DhDkUThFv8RGlWbARTdzKmMIGHyS6KOHrqVViBb8h9ol8/0= ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:54:21.888 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T20:54:21.888 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T20:54:21.892 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-09-10T20:54:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:22 smithi033 ceph-mon[30860]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:23.206 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:23 smithi033 ceph-mon[30860]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:24.119 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:24 smithi033 ceph-mon[30860]: Deploying cephadm binary to smithi033 2024-09-10T20:54:26.937 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-09-10T20:54:26.937 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mon service... 2024-09-10T20:54:27.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:27 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:27.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:27 smithi033 ceph-mon[30860]: Added host smithi033 2024-09-10T20:54:27.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:27 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:28.846 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:28 smithi033 ceph-mon[30860]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:28.847 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:28 smithi033 ceph-mon[30860]: Saving service mon spec with placement count:5 2024-09-10T20:54:28.847 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:28 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:29.313 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T20:54:29.313 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying unmanaged mgr service... 2024-09-10T20:54:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:29 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:29 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:30.466 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T20:54:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:30 smithi033 ceph-mon[30860]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:30 smithi033 ceph-mon[30860]: Saving service mgr spec with placement count:2 2024-09-10T20:54:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:30 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:32.248 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:32 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2439991513' entity='client.admin' 2024-09-10T20:54:32.248 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:32 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:33.458 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-09-10T20:54:34.108 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:33 smithi033 ceph-mon[30860]: from='mgr.14116 172.21.15.33:0/2247198926' entity='mgr.x' 2024-09-10T20:54:34.108 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:33 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3482823482' entity='client.admin' 2024-09-10T20:54:35.072 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:34 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1908954679' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-10T20:54:35.073 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:34 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: ignoring --setuser ceph since I am not root 2024-09-10T20:54:35.073 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:34 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: ignoring --setgroup ceph since I am not root 2024-09-10T20:54:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:35 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1908954679' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T20:54:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:35 smithi033 ceph-mon[30860]: mgrmap e9: x(active, since 19s) 2024-09-10T20:54:36.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:36 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1975688815' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T20:54:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-09-10T20:54:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T20:54:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T20:54:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-10T20:54:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T20:54:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-09-10T20:54:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-09-10T20:54:36.936 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-09-10T20:54:40.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: Active manager daemon x restarted 2024-09-10T20:54:40.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: Activating manager daemon x 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: mgrmap e10: x(active, starting, since 0.00589229s) 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T20:54:40.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:40 smithi033 ceph-mon[30860]: Manager daemon x is now available 2024-09-10T20:54:41.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:41.320 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-10T20:54:41.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-10T20:54:41.321 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:41 smithi033 ceph-mon[30860]: mgrmap e11: x(active, since 1.00622s) 2024-09-10T20:54:41.624 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-09-10T20:54:41.624 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T20:54:41.624 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T20:54:41.624 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-09-10T20:54:41.624 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-09-10T20:54:41.624 INFO:teuthology.orchestra.run.smithi033.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-10T20:54:42.270 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-mon[30860]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T20:54:42.270 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-mon[30860]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T20:54:42.270 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T20:54:42.071+0000 7f0dd6b1c640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:20:54:42] ENGINE Error in HTTPServer.serve 2024-09-10T20:54:42.270 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: Traceback (most recent call last): 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self._connections.run(self.expiration_interval) 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self._run(expiration_interval) 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: s = self.context.wrap_socket( 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: return self.sslsocket_class._create( 2024-09-10T20:54:42.271 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T20:54:42.272 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self.do_handshake() 2024-09-10T20:54:42.272 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T20:54:42.272 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: self._sslobj.do_handshake() 2024-09-10T20:54:42.272 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T20:54:42.272 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:54:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:41] ENGINE Bus STARTING 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:41] ENGINE Serving on http://172.21.15.33:8765 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: mgrmap e12: x(active, since 2s) 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:42] ENGINE Serving on https://172.21.15.33:7150 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:42] ENGINE Bus STARTED 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: [10/Sep/2024:20:54:42] ENGINE Error in HTTPServer.serve 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout: Traceback (most recent call last): 2024-09-10T20:54:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: self._connections.run(self.expiration_interval) 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: self._run(expiration_interval) 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: s = self.context.wrap_socket( 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: return self.sslsocket_class._create( 2024-09-10T20:54:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T20:54:43.429 INFO:journalctl@ceph.mon.a.smithi033.stdout: self.do_handshake() 2024-09-10T20:54:43.429 INFO:journalctl@ceph.mon.a.smithi033.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T20:54:43.429 INFO:journalctl@ceph.mon.a.smithi033.stdout: self._sslobj.do_handshake() 2024-09-10T20:54:43.429 INFO:journalctl@ceph.mon.a.smithi033.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T20:54:43.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:43.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:44.674 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:44 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:45.265 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIULfUGYPPn7/dMFl7Cm0gceZvHEeowDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMDIwNTQ0NFoXDTM0\nMDkxMTIwNTQ0NFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjMzMIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEA1b0rOWm/CcAyrB3ImjgFToF5AFxjPfSCeDiZ\n+AaKuWtIaGpXaMYVFTtwATziaKdssTxaQ0CNPnw+E+cnEzn7DYfvWNLGnTQvhFW4\netIBUEv3fFYNMaS2bjlbK48Plu9eudmAtOnLPg7WM3KsOo0ktfAbGXZN8kBx07Gs\n6wIIxFR2knwV67b9fdv34ETFWkGvAVMkG+AdsOUEAq1ubIbFVEEfBggjR8JtReAe\n2IxS9fzYUCYd1y346qi93X1mpLwukk7F4GfLCo3ctOY/q6C03zgMROAA6pPHnAaq\n+GulIn9XKYQ6yu+0xdrcNhJXNhdj1qVKhjg/oGTMM8ak9PzUwtGDrexfGn6OpK3d\n4Kml1xid96TZ6WDE6Sy6GgDFbYR9t+KZytDrELYoyYP+kBXTkxp7TWvR2cEVwZYq\nd+GU4827l0ZaOl4wWExF4LmfTd/PFmb2057LwdWooNqJ2vWzr7MxHQEpX/s7IlMc\nRp9Lr755bmx4FkqEeAWlqWo9zQerx9Dtam2W4VW25dW0wI5d2BUHzlPPmxifF28l\nHfKGCOHnHl8qYpxBNZQ4zq01KHvAEgFmoy9SE1tjEqUHY2N+tbjavDvqVRYYyalO\nO/ld0fzp8eIt4bAlA8fjFON4g4/6Fxjaconw/Ykfh6kONhiKBAW6mKEXK64xf9W5\nfFXR3E8CAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDMzghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPITAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAy\nNx1Tt7ytmrErwhUtIP91JizV9VVvcmEWaLb06zTHLg8wCO1ybV0hy90/gBGS1u4M\n35CLo3Q3N/1y0Pfc86jgWOxQxCX5vuJdS6LqT3/Wgo4j/4JR1coJCar0G1AR+RKl\ncMs9bSR+rUhmfzFpgdF3JyeQlht8cxh/PzeeNL2WVqhV871BKYmA+3HDntxI1SzZ\nlrOUI2uEaJWmtL2kJLTduNQWc66QF0xFN/MLYRnskwnxSKRYd6OEw9fzNwP83Qs2\nqJrJDo1eoPXmQozLeSAalNHbIdiSQIpJ9A3rJJx9Bmtw0UeD4B+dD1kLvV94RQqI\n8NZrJsyNUb9fGZ7WoMxeP7hRHkzhOd40ND3rDNWqbctCpRp0rM/WK/Iww2/kdtNd\nYCniP9DlXN+NhlcikYTzD3UsN+WA9aVpY55+qkW6pIT0cduZ8hqsvnmCjCbukwCO\n7mrCqWt30sKRW06sO4M+Ur/I4gNgP1ZibFLgi2eJXV0kkVfm6PvIEXDGntNYVZED\nWLN4hQn5nad582Qf1HbbLjQrSM16fdRRGCMV18jbtbpbG+Fpa7eKVddQiAvnfj40\nJZY0u+IO4bX+IXmDOmTrMd18igt2ktXLbQFQf8i4ww7TT2hRSNHtjt7st4kE0qxH\n1nO+hkHjtfxyeIeTPVwCwaNSP2hDEnjtUYPbYpSIfA==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEA1b0rOWm/CcAyrB3ImjgFToF5AFxjPfSCeDiZ+AaKuWtIaGpX\naMYVFTtwATziaKdssTxaQ0CNPnw+E+cnEzn7DYfvWNLGnTQvhFW4etIBUEv3fFYN\nMaS2bjlbK48Plu9eudmAtOnLPg7WM3KsOo0ktfAbGXZN8kBx07Gs6wIIxFR2knwV\n67b9fdv34ETFWkGvAVMkG+AdsOUEAq1ubIbFVEEfBggjR8JtReAe2IxS9fzYUCYd\n1y346qi93X1mpLwukk7F4GfLCo3ctOY/q6C03zgMROAA6pPHnAaq+GulIn9XKYQ6\nyu+0xdrcNhJXNhdj1qVKhjg/oGTMM8ak9PzUwtGDrexfGn6OpK3d4Kml1xid96TZ\n6WDE6Sy6GgDFbYR9t+KZytDrELYoyYP+kBXTkxp7TWvR2cEVwZYqd+GU4827l0Za\nOl4wWExF4LmfTd/PFmb2057LwdWooNqJ2vWzr7MxHQEpX/s7IlMcRp9Lr755bmx4\nFkqEeAWlqWo9zQerx9Dtam2W4VW25dW0wI5d2BUHzlPPmxifF28lHfKGCOHnHl8q\nYpxBNZQ4zq01KHvAEgFmoy9SE1tjEqUHY2N+tbjavDvqVRYYyalOO/ld0fzp8eIt\n4bAlA8fjFON4g4/6Fxjaconw/Ykfh6kONhiKBAW6mKEXK64xf9W5fFXR3E8CAwEA\nAQKCAgARyJkJlbOoRIkSyrGaVJyVwRkHv310YLKK/S6hCoOiqiX5UMEavcHFpU82\n6HwauAoc9F1lk2qNGeTyXqe2JlCt+ghJ7Cj2rdUMSN4+Ey4PzRpM3+izdOBxla0b\nwoifXCWWB15O7yroFsTvZ+mjlNh1Q4KFtlMFm84SDopJeBRcUtNrcblDMP5baUtw\n/1fU9Xpxp5ttWaguw9yhumYMepQoOU4zAByrXPvJi2cVnmtNU0R45WBTOzHRmHOf\ns6OFg7DI9JaV4Bzuhl99ilt0KhK9MIUjepMo8V3I12yEmd7rWrauWiBS1T9FvRAt\nzDiHDsrRnP7GjHSNdwWJ2KEjTyFb403I0sC0iV1UBif3sQGxti/R+zaaIHuSDvhY\nD/1CwVhLHtgEsQcIznBqg+MEC8RtpWdDrjdVWzDGOwoYptflPgRI13HZZN8EcvQz\nrIHuTe/N8UePW/wSELVhlu8werZ0X07/hKwOZEEp7kATaE1I/xBaYp+AfQA4k6Sc\n0HeKC65fHBHcCIOgeQ/Se2B8UAQGfGLfsoXSqXCUVazCEvJrQrtyEqA/bCzcyXNN\nUJpm9HSgGYDbvZxgnPSbYkenanTSzdoRd97aLahKqIfHynwAmJz407PbsILM5WZ+\nbFIM/vaJv1GdL79mXcSh4xpyqJA3ZAbL/9X/i4OGJlNi64gCkQKCAQEA+E1uuxOM\nLanOjH6UL/mF2oyWBcWbksgIqZ3JdVd3xtdZI/Np0+MATSc8xzosC0ajx0KhomAa\nuh74VWlBqyKjFc1jguKZHLrx5ssMCkudQ43zpoBqWVnuGhT6jarIckqsAAOjgfs6\n7aBS5XjoaVS1z7oVzc2VggjR55XI5CbkXUOVjSqPeSQaw4FVWVJ/ma2u9wpCvkna\nyERzg7Lo8t1HVf9Vr+xywM0XJaEWLUPMbwg67OiWWJVt0qAg7XL8jebXD2CCrpPm\nyuBLtdRxy34UOgCIkhzGepH6PifrALmwoI+4TWD2fB4xsywHk1BO99lwwN2zi54L\nbSH+zatTd71W3wKCAQEA3F1vSAJDpE2xPt0mctT20Xe6Ygr8AmFiED23dm+3gZXa\nlFfbRzToRVSqZvPTv6lntTTJO2j8qzysC1oCzK+MHQYpAm5thjPacA88SBdMzhOu\nFvjZk6fUppfubdLarlbpd5FHVydNkAQbKNfhPaO5LsKojMy9Fno2nURJ6yDsgUrP\nEPD6GF8q6ez5CabuC6M/9CaO7BgAXrl4aQ8kiqQ3PvypV+VJ9EaWvsaZ/u1ZwYLp\nVDfx+mdw5ctfAHr+XKTOsQLCHHuehlJmfEMGgLs8P+xnOSO/7sbzVnoQEK9xAjt1\nlEUnI6Z0zUINhURRivRq2dgA0u6kgZxlNMWK/ClYkQKCAQEAmHtEwtGwReJ/mzuo\nWQXMs3eeMZE2hp8c4rypGMxMG+nnLfKvTJWce8TayCXrIkmRDi7W/9Yw05dtPVZU\nI3oz2JG2yIvLJx+OqgKLNE9XFkM4G0nFBXX6iD3V1+XHDpjYo3GYmZvkhkTsY0sq\nvnRs6lIGTcowRBiSfrs3ERTpLHTyhCxcyzHm1aS4+w90mLVWOxMsBZtpJY8TuQ7H\ni01kH2WcA1sa5VeNHWidK9L02kKdnRsAKQoq1SvMzMTOX34hF6pW1kqxcmjnVZRP\nBX45gZ0OQVw2btLyb/UnldxyoUdhEPO4qmQIdVVWmZq5w+7eFTKQre6yaaR0y2yo\n52F41QKCAQAXECRUagBAUu1JAgZDIGzQkXJNTo7/hMepoXvxBlIzWsasneFSO4RP\ntUO8UKqJtaYn/eVNtXg0R75N1dbTK1BDvWBPY9VMpWKoNb/+X4nnS4H933+OB5Om\n9+G5VscSVgBLRkYsJe3lM7jP9oAZXmhEL5VKXycvEg1jOkFuPVzjTzwKrt3N7S/l\nmaqaywDA8Q4BdIOizARF5a1neXE+nV821WS30USDL1wZjVmSp66sJRJcgRpvNaJI\nOpevyi+S7KDBP+Om7X0y267kNHOax33PxuMqxMYSrHnNRCsoQ+RU8nok0XP+NC6n\nSNF3cCYEn/4K3KOBb/6BnietdTtNltoxAoIBAQDUXn6tSU2zkKyDA6jsz5OiclyL\nf519QkNAv36fJPmib7DwlFHbGT5RyYWGLSiSvO6vlCaf18cUr+5wpQlaSdKtjE3y\nPqWn8N5qYlAZXnKfA4l9zxPEZsQJVWXZD+LqOTdDq6Ux4dLsVBwSQSv0WE39qo6G\nfdMlebxgy+t68FsP7w/TYomPa0KXSfOI7N3BdfxHrzxbfmO49RvgmKlfThC/p0h7\nP+aOzhWMk1srrHqUY3OD2Y9BaRm6hpRxrJFGcC32sKQGxeDvFczNLM0i4rD18I++\nIHz8nDcRcEtRdJxcmzz3AbG6SFOx2hjoDMNT2K8Cpn6VKQWbED61bWzYl8NW\n-----END RSA PRIVATE KEY-----\n"} 2024-09-10T20:54:46.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-10T20:54:47.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:46 smithi033 ceph-mon[30860]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:47.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:47.745 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-10T20:54:47.745 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-09-10T20:54:48.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:48 smithi033 ceph-mon[30860]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:48.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:48 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:49.486 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$BSzEBRIWIys.ddTh5rxi3ukNICrfBjDDawL2sCavnlpq4HiG29feW", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726001688, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T20:54:49.486 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-09-10T20:54:49.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:49 smithi033 ceph-mon[30860]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:49.742 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:50.721 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3371320123' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-10T20:54:50.723 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T20:54:50.733 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T20:54:50.733 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-09-10T20:54:50.740 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T20:54:50.740 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-09-10T20:54:50.740 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-09-10T20:54:50.740 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T20:54:50.743 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-09-10T20:54:50.743 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:50.743 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033.front.sepia.ceph.com:8443/ 2024-09-10T20:54:50.743 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-09-10T20:54:50.743 INFO:teuthology.orchestra.run.smithi033.stdout: Password: i0wo6brg34 2024-09-10T20:54:50.743 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:50.743 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config directory 2024-09-10T20:54:52.172 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T20:54:52.172 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T20:54:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T20:54:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.172 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T20:54:52.172 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:54:52.173 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-09-10T20:54:52.220 INFO:tasks.cephadm:Fetching config... 2024-09-10T20:54:52.220 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:54:52.220 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T20:54:52.237 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T20:54:52.237 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:54:52.237 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T20:54:52.293 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T20:54:52.293 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:54:52.293 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T20:54:52.360 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T20:54:52.360 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:54:52.360 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T20:54:52.416 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T20:54:52.417 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEUIy25/yLchglMHh1S+HUGYyLtG/2GWqADqi7dM5yJiN69YNmCdtV3tdDzzKDkktlONMHAv1SLTm0oHuxBUzjZ+jTLhvOJfEuyMSgoUq9vT9PIpP5QgVRtjcckiVKt3FmbdH6FnZZwHsqlOCZZ3ZLpBadI4Mq9lxWGrVlTzgjwZRxvmWAjSSq+H93Q+Bavq/FbeMBy8LRr4Ce/UOODURVmWRX0hKobExCui94hk+n9NeQRUNRNAtagvsAj/S6AOLfBxCLmUN8ldx0gAGf4zStaT4SOgjXs4nJFpYj0GALHhLTnYibihleQINr3FK/zBLmwm///Spb/QB0621cH3MffuPrMCWijM54mTeM/DQQh9oGKvU5G9WdeVEhnafU9VtQnJHnKU+KWzkZhsFYLxN8DOS2ot3mijohkhYwdGWNnWjnrDPkX2TcUlhEZIguz4RkHzsptbSvRjf9N2x+DhDkUThFv8RGlWbARTdzKmMIGHyS6KOHrqVViBb8h9ol8/0= ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T20:54:52.493 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEUIy25/yLchglMHh1S+HUGYyLtG/2GWqADqi7dM5yJiN69YNmCdtV3tdDzzKDkktlONMHAv1SLTm0oHuxBUzjZ+jTLhvOJfEuyMSgoUq9vT9PIpP5QgVRtjcckiVKt3FmbdH6FnZZwHsqlOCZZ3ZLpBadI4Mq9lxWGrVlTzgjwZRxvmWAjSSq+H93Q+Bavq/FbeMBy8LRr4Ce/UOODURVmWRX0hKobExCui94hk+n9NeQRUNRNAtagvsAj/S6AOLfBxCLmUN8ldx0gAGf4zStaT4SOgjXs4nJFpYj0GALHhLTnYibihleQINr3FK/zBLmwm///Spb/QB0621cH3MffuPrMCWijM54mTeM/DQQh9oGKvU5G9WdeVEhnafU9VtQnJHnKU+KWzkZhsFYLxN8DOS2ot3mijohkhYwdGWNnWjnrDPkX2TcUlhEZIguz4RkHzsptbSvRjf9N2x+DhDkUThFv8RGlWbARTdzKmMIGHyS6KOHrqVViBb8h9ol8/0= ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:54:52.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEUIy25/yLchglMHh1S+HUGYyLtG/2GWqADqi7dM5yJiN69YNmCdtV3tdDzzKDkktlONMHAv1SLTm0oHuxBUzjZ+jTLhvOJfEuyMSgoUq9vT9PIpP5QgVRtjcckiVKt3FmbdH6FnZZwHsqlOCZZ3ZLpBadI4Mq9lxWGrVlTzgjwZRxvmWAjSSq+H93Q+Bavq/FbeMBy8LRr4Ce/UOODURVmWRX0hKobExCui94hk+n9NeQRUNRNAtagvsAj/S6AOLfBxCLmUN8ldx0gAGf4zStaT4SOgjXs4nJFpYj0GALHhLTnYibihleQINr3FK/zBLmwm///Spb/QB0621cH3MffuPrMCWijM54mTeM/DQQh9oGKvU5G9WdeVEhnafU9VtQnJHnKU+KWzkZhsFYLxN8DOS2ot3mijohkhYwdGWNnWjnrDPkX2TcUlhEZIguz4RkHzsptbSvRjf9N2x+DhDkUThFv8RGlWbARTdzKmMIGHyS6KOHrqVViBb8h9ol8/0= ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T20:54:52.563 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEUIy25/yLchglMHh1S+HUGYyLtG/2GWqADqi7dM5yJiN69YNmCdtV3tdDzzKDkktlONMHAv1SLTm0oHuxBUzjZ+jTLhvOJfEuyMSgoUq9vT9PIpP5QgVRtjcckiVKt3FmbdH6FnZZwHsqlOCZZ3ZLpBadI4Mq9lxWGrVlTzgjwZRxvmWAjSSq+H93Q+Bavq/FbeMBy8LRr4Ce/UOODURVmWRX0hKobExCui94hk+n9NeQRUNRNAtagvsAj/S6AOLfBxCLmUN8ldx0gAGf4zStaT4SOgjXs4nJFpYj0GALHhLTnYibihleQINr3FK/zBLmwm///Spb/QB0621cH3MffuPrMCWijM54mTeM/DQQh9oGKvU5G9WdeVEhnafU9VtQnJHnKU+KWzkZhsFYLxN8DOS2ot3mijohkhYwdGWNnWjnrDPkX2TcUlhEZIguz4RkHzsptbSvRjf9N2x+DhDkUThFv8RGlWbARTdzKmMIGHyS6KOHrqVViBb8h9ol8/0= ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:54:52.576 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEUIy25/yLchglMHh1S+HUGYyLtG/2GWqADqi7dM5yJiN69YNmCdtV3tdDzzKDkktlONMHAv1SLTm0oHuxBUzjZ+jTLhvOJfEuyMSgoUq9vT9PIpP5QgVRtjcckiVKt3FmbdH6FnZZwHsqlOCZZ3ZLpBadI4Mq9lxWGrVlTzgjwZRxvmWAjSSq+H93Q+Bavq/FbeMBy8LRr4Ce/UOODURVmWRX0hKobExCui94hk+n9NeQRUNRNAtagvsAj/S6AOLfBxCLmUN8ldx0gAGf4zStaT4SOgjXs4nJFpYj0GALHhLTnYibihleQINr3FK/zBLmwm///Spb/QB0621cH3MffuPrMCWijM54mTeM/DQQh9oGKvU5G9WdeVEhnafU9VtQnJHnKU+KWzkZhsFYLxN8DOS2ot3mijohkhYwdGWNnWjnrDPkX2TcUlhEZIguz4RkHzsptbSvRjf9N2x+DhDkUThFv8RGlWbARTdzKmMIGHyS6KOHrqVViBb8h9ol8/0= ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T20:54:52.619 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEUIy25/yLchglMHh1S+HUGYyLtG/2GWqADqi7dM5yJiN69YNmCdtV3tdDzzKDkktlONMHAv1SLTm0oHuxBUzjZ+jTLhvOJfEuyMSgoUq9vT9PIpP5QgVRtjcckiVKt3FmbdH6FnZZwHsqlOCZZ3ZLpBadI4Mq9lxWGrVlTzgjwZRxvmWAjSSq+H93Q+Bavq/FbeMBy8LRr4Ce/UOODURVmWRX0hKobExCui94hk+n9NeQRUNRNAtagvsAj/S6AOLfBxCLmUN8ldx0gAGf4zStaT4SOgjXs4nJFpYj0GALHhLTnYibihleQINr3FK/zBLmwm///Spb/QB0621cH3MffuPrMCWijM54mTeM/DQQh9oGKvU5G9WdeVEhnafU9VtQnJHnKU+KWzkZhsFYLxN8DOS2ot3mijohkhYwdGWNnWjnrDPkX2TcUlhEZIguz4RkHzsptbSvRjf9N2x+DhDkUThFv8RGlWbARTdzKmMIGHyS6KOHrqVViBb8h9ol8/0= ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:54:52.632 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T20:54:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:54:52.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:52 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2569185486' entity='client.admin' 2024-09-10T20:54:53.846 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:53 smithi033 ceph-mon[30860]: mgrmap e13: x(active, since 12s) 2024-09-10T20:54:55.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:54 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/504781523' entity='client.admin' 2024-09-10T20:54:55.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:55.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:55.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"} : dispatch 2024-09-10T20:54:55.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:55.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:55.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:55.376 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T20:54:55.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T20:54:56.295 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:54:57.716 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-09-10T20:54:57.717 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:54:57.717 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-09-10T20:54:57.737 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:54:57.737 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:54:57.796 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-09-10T20:54:57.797 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch host add smithi104 2024-09-10T20:54:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: Updating smithi033:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:54:58.169 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:54:58.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: Updating smithi033:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T20:54:58.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:58.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:58.170 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:54:59.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:54:58 smithi033 ceph-mon[30860]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:01.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:01 smithi033 ceph-mon[30860]: Deploying cephadm binary to smithi104 2024-09-10T20:55:01.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:01 smithi033 ceph-mon[30860]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:02.527 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-09-10T20:55:03.192 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T20:55:03.446 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:03.785 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:03 smithi033 ceph-mon[30860]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:03.785 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:03.785 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:03 smithi033 ceph-mon[30860]: Added host smithi104 2024-09-10T20:55:03.785 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:03.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:03.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:04.068 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:55:04.069 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-09-10T20:55:04.645 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-09-10T20:55:04.645 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:55:04.646 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-09-10T20:55:04.665 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:55:04.665 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:04.724 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-09-10T20:55:04.724 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch host add smithi174 2024-09-10T20:55:04.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:04 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:04.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:04 smithi033 ceph-mon[30860]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:04.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:04 smithi033 ceph-mon[30860]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T20:55:04.976 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:05 smithi033 ceph-mon[30860]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"} : dispatch 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:08.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:08.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:08 smithi033 ceph-mon[30860]: Deploying cephadm binary to smithi174 2024-09-10T20:55:08.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:08 smithi033 ceph-mon[30860]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:08.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:08 smithi033 ceph-mon[30860]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:08.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:08 smithi033 ceph-mon[30860]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T20:55:09.564 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-09-10T20:55:09.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:09 smithi033 ceph-mon[30860]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:09.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:09.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:10.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T20:55:10.430 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:10.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:10 smithi033 ceph-mon[30860]: Added host smithi174 2024-09-10T20:55:10.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:10 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:10.882 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:10 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:11.074 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:55:11.074 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-09-10T20:55:11.666 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T20:55:11.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T20:55:11.919 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:11.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:11 smithi033 ceph-mon[30860]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:11.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-09-10T20:55:13.038 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:12 smithi033 ceph-mon[30860]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T20:55:13.038 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:12 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/506609299' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T20:55:13.253 INFO:tasks.cephadm:Adding mon.a on smithi033 2024-09-10T20:55:13.253 INFO:tasks.cephadm:Adding mon.b on smithi104 2024-09-10T20:55:13.253 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-09-10T20:55:13.254 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c' 2024-09-10T20:55:13.923 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T20:55:13.980 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T20:55:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:13 smithi033 ceph-mon[30860]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:13 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/506609299' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T20:55:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:13 smithi033 ceph-mon[30860]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:14.724 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-09-10T20:55:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"} : dispatch 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c;count:3 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:15.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:15.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:15.179 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:14 smithi033 ceph-mon[30860]: Deploying daemon mon.c on smithi174 2024-09-10T20:55:15.229 DEBUG:teuthology.orchestra.run.smithi104:mon.b> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T20:55:15.231 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T20:55:15.234 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T20:55:15.234 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T20:55:16.838 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:17 smithi033 ceph-mon[30860]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:17.875 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:55:17.875 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","modified":"2024-09-10T20:53:45.201883Z","created":"2024-09-10T20:53:45.201883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T20:55:17.876 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-09-10T20:55:18.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:18 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/3797978659' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:18.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:18 smithi174 systemd[1]: Starting Ceph mon.c for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T20:55:18.953 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:18 smithi174 podman[35616]: 2024-09-10 20:55:18.746516415 +0000 UTC m=+0.016013634 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T20:55:18.954 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:18 smithi174 podman[35616]: 2024-09-10 20:55:18.843654109 +0000 UTC m=+0.113151323 container create 9dea2b2e5d3467a1ab497c246ebfdcf9adc88afe813bb26a5f598588b010337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T20:55:19.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 podman[35616]: 2024-09-10 20:55:19.054154829 +0000 UTC m=+0.323652045 container init 9dea2b2e5d3467a1ab497c246ebfdcf9adc88afe813bb26a5f598588b010337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 podman[35616]: 2024-09-10 20:55:19.057329838 +0000 UTC m=+0.326827054 container start 9dea2b2e5d3467a1ab497c246ebfdcf9adc88afe813bb26a5f598588b010337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2) 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev), process ceph-mon, pid 2 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pidfile_write: ignore empty --pid-file 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: load: jerasure load: lrc 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: RocksDB version: 7.9.2 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Git sha 0 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: DB SUMMARY 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: DB Session ID: 9E80IZEY4L1W1MF9BUHB 2024-09-10T20:55:19.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: CURRENT file: CURRENT 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.error_if_exists: 0 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.create_if_missing: 0 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.paranoid_checks: 1 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.env: 0x55b8e6305180 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T20:55:19.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.info_log: 0x55b8e81e9700 2024-09-10T20:55:19.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T20:55:19.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.statistics: (nil) 2024-09-10T20:55:19.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.use_fsync: 0 2024-09-10T20:55:19.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_log_file_size: 0 2024-09-10T20:55:19.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T20:55:19.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T20:55:19.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.allow_fallocate: 1 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.use_direct_reads: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.db_log_dir: 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.wal_dir: 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T20:55:19.399 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.write_buffer_manager: 0x55b8e81edb80 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T20:55:19.400 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.unordered_write: 0 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.row_cache: None 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.wal_filter: None 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.two_write_queues: 0 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T20:55:19.401 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.wal_compression: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.atomic_flush: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.log_readahead_size: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_background_jobs: 2 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_background_compactions: -1 2024-09-10T20:55:19.402 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_subcompactions: 1 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_open_files: -1 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_background_flushes: -1 2024-09-10T20:55:19.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Compression algorithms supported: 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kZSTD supported: 0 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kXpressCompression supported: 0 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kBZip2Compression supported: 0 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kLZ4Compression supported: 1 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kZlibCompression supported: 1 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: kSnappyCompression supported: 1 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T20:55:19.404 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.merge_operator: 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_filter: None 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b8e81e9320) 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_type: 0 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_index_type: 0 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_shortening: 1 2024-09-10T20:55:19.405 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: checksum: 4 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: no_block_cache: 0 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache: 0x55b8e81dd610 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_options: 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: capacity : 536870912 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_shard_bits : 4 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: strict_capacity_limit : 0 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_compressed: (nil) 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: persistent_cache: (nil) 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size: 4096 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size_deviation: 10 2024-09-10T20:55:19.406 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_restart_interval: 16 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_block_restart_interval: 1 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: metadata_block_size: 4096 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: partition_filters: 0 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: use_delta_encoding: 1 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: filter_policy: bloomfilter 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: whole_key_filtering: 1 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: verify_compression: 0 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: format_version: 5 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_index_compression: 1 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_align: 0 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: max_auto_readahead_size: 262144 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: prepopulate_block_cache: 0 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: initial_auto_readahead_size: 8192 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T20:55:19.407 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression: NoCompression 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.num_levels: 7 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T20:55:19.408 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T20:55:19.409 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T20:55:19.410 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T20:55:19.411 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.inplace_update_support: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.bloom_locality: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.max_successive_merges: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.ttl: 2592000 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T20:55:19.412 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.enable_blob_files: false 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.min_blob_size: 0 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T20:55:19.413 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ed42664e-edac-4db8-80cb-a48b8fcb55ea 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001719084461, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001719084869, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726001719, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ed42664e-edac-4db8-80cb-a48b8fcb55ea", "db_session_id": "9E80IZEY4L1W1MF9BUHB", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001719084962, "job": 1, "event": "recovery_finished"} 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b8e820ce00 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: DB pointer 0x55b8e830e000 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** DB Stats ** 2024-09-10T20:55:19.414 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:19.415 INFO:journalctl@ceph.mon.c.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T20:55:19.416 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache BinnedLRUCache@0x55b8e81dd610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: starting mon.c rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(???) e0 preinit fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: e1 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: btime 2024-09-10T20:53:48:210161+0000 2024-09-10T20:55:19.417 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout: legacy client fscid: -1 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout: No filesystems configured 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:55:19.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"} : dispatch 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:19.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Deploying cephadm binary to smithi174 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Added host smithi174 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.420 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/506609299' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/506609299' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"} : dispatch 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:19.421 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c;count:3 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:19.422 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: Deploying daemon mon.c on smithi174 2024-09-10T20:55:19.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/3797978659' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:19.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 ceph-mon[35635]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T20:55:19.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 bash[35616]: 9dea2b2e5d3467a1ab497c246ebfdcf9adc88afe813bb26a5f598588b010337e 2024-09-10T20:55:19.423 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:19 smithi174 systemd[1]: Started Ceph mon.c for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:55:19.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:19 smithi033 ceph-mon[30860]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:19.696 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T20:55:19.696 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T20:55:19.945 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:55:24.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:55:24.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: mon.a calling monitor election 2024-09-10T20:55:24.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: mon.c calling monitor election 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: monmap epoch 2 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: last_changed 2024-09-10T20:55:19.109579+0000 2024-09-10T20:55:24.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: min_mon_release 19 (squid) 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: election_strategy: 1 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: fsmap 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: mgrmap e13: x(active, since 44s) 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: overall HEALTH_OK 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:24.647 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:24.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:55:24.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: mon.a calling monitor election 2024-09-10T20:55:24.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: mon.c calling monitor election 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: monmap epoch 2 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: last_changed 2024-09-10T20:55:19.109579+0000 2024-09-10T20:55:24.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: min_mon_release 19 (squid) 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: election_strategy: 1 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: fsmap 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: mgrmap e13: x(active, since 44s) 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: overall HEALTH_OK 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:24.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:25.027 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:55:25.027 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","modified":"2024-09-10T20:55:19.109579Z","created":"2024-09-10T20:53:45.201883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T20:55:25.027 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-10T20:55:25.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:25 smithi174 ceph-mon[35635]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:25.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:25 smithi174 ceph-mon[35635]: Deploying daemon mon.b on smithi104 2024-09-10T20:55:25.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:25 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/3620568425' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:25.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:25 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:25 smithi033 ceph-mon[30860]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:25 smithi033 ceph-mon[30860]: Deploying daemon mon.b on smithi104 2024-09-10T20:55:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:25 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/3620568425' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:25.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:25 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:26.602 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T20:55:26.602 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T20:55:26.856 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:55:27.274 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:27 smithi174 ceph-mon[35635]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:27.531 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:55:27.531 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","modified":"2024-09-10T20:55:19.109579Z","created":"2024-09-10T20:53:45.201883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T20:55:27.531 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-09-10T20:55:27.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:27 smithi033 ceph-mon[30860]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:27.843 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: load: jerasure load: lrc 2024-09-10T20:55:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: RocksDB version: 7.9.2 2024-09-10T20:55:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Git sha 0 2024-09-10T20:55:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Compile date 2024-09-10 08:02:51 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: DB SUMMARY 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: DB Session ID: 83XD1MP2BYE6PK08SSUS 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: CURRENT file: CURRENT 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-09-10T20:55:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.error_if_exists: 0 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.create_if_missing: 0 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.paranoid_checks: 1 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.env: 0x55ddfcbac180 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.info_log: 0x55ddfe385700 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T20:55:28.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.statistics: (nil) 2024-09-10T20:55:28.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.use_fsync: 0 2024-09-10T20:55:28.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_log_file_size: 0 2024-09-10T20:55:28.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T20:55:28.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T20:55:28.316 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T20:55:28.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T20:55:28.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.allow_fallocate: 1 2024-09-10T20:55:28.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T20:55:28.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T20:55:28.317 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.use_direct_reads: 0 2024-09-10T20:55:28.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T20:55:28.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T20:55:28.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.db_log_dir: 2024-09-10T20:55:28.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.wal_dir: 2024-09-10T20:55:28.318 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.write_buffer_manager: 0x55ddfe389b80 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T20:55:28.319 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T20:55:28.320 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T20:55:28.320 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T20:55:28.320 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T20:55:28.320 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T20:55:28.320 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T20:55:28.320 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.unordered_write: 0 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.row_cache: None 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.wal_filter: None 2024-09-10T20:55:28.321 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.two_write_queues: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.wal_compression: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.atomic_flush: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.log_readahead_size: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T20:55:28.322 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_background_jobs: 2 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_background_compactions: -1 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_subcompactions: 1 2024-09-10T20:55:28.323 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_open_files: -1 2024-09-10T20:55:28.325 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_background_flushes: -1 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Compression algorithms supported: 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kZSTD supported: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kXpressCompression supported: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kBZip2Compression supported: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kLZ4Compression supported: 1 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kZlibCompression supported: 1 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: kSnappyCompression supported: 1 2024-09-10T20:55:28.326 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.merge_operator: 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_filter: None 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55ddfe385320) 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks: 1 2024-09-10T20:55:28.327 INFO:journalctl@ceph.mon.b.smithi104.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: pin_top_level_index_and_filter: 1 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_type: 0 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_index_type: 0 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_shortening: 1 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: checksum: 4 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: no_block_cache: 0 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache: 0x55ddfe379610 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_name: BinnedLRUCache 2024-09-10T20:55:28.328 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_options: 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: capacity : 536870912 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_shard_bits : 4 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: strict_capacity_limit : 0 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: high_pri_pool_ratio: 0.000 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_cache_compressed: (nil) 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: persistent_cache: (nil) 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size: 4096 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_size_deviation: 10 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_restart_interval: 16 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: index_block_restart_interval: 1 2024-09-10T20:55:28.329 INFO:journalctl@ceph.mon.b.smithi104.stdout: metadata_block_size: 4096 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: partition_filters: 0 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: use_delta_encoding: 1 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: filter_policy: bloomfilter 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: whole_key_filtering: 1 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: verify_compression: 0 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: read_amp_bytes_per_bit: 0 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: format_version: 5 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_index_compression: 1 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: block_align: 0 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: max_auto_readahead_size: 262144 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: prepopulate_block_cache: 0 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: initial_auto_readahead_size: 8192 2024-09-10T20:55:28.330 INFO:journalctl@ceph.mon.b.smithi104.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression: NoCompression 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.num_levels: 7 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T20:55:28.331 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T20:55:28.332 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T20:55:28.333 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T20:55:28.334 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T20:55:28.335 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.inplace_update_support: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.bloom_locality: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.max_successive_merges: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.ttl: 2592000 2024-09-10T20:55:28.336 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.enable_blob_files: false 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.min_blob_size: 0 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T20:55:28.337 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e00261e3-e8f0-40e2-a2a4-0756379a764d 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001727841966, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001727842355, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726001727, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e00261e3-e8f0-40e2-a2a4-0756379a764d", "db_session_id": "83XD1MP2BYE6PK08SSUS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726001727842446, "job": 1, "event": "recovery_finished"} 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55ddfe3a8e00 2024-09-10T20:55:28.338 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: DB pointer 0x55ddfe4aa000 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** DB Stats ** 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T20:55:28.339 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** Compaction Stats [default] ** 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-10T20:55:28.340 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: Cumulative compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: Interval compaction: 0.00 GB write, 0.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T20:55:28.341 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache BinnedLRUCache@0x55ddfe379610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: using public_addr v2:172.21.15.104:0/0 -> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: starting mon.b rank -1 at public addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] at bind addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(???) e0 preinit fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout: e1 2024-09-10T20:55:28.342 INFO:journalctl@ceph.mon.b.smithi104.stdout: btime 2024-09-10T20:53:48:210161+0000 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout: legacy client fscid: -1 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout: 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout: No filesystems configured 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T20:55:28.343 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"} : dispatch 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:28.344 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Deploying cephadm binary to smithi174 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Added host smithi174 2024-09-10T20:55:28.345 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/506609299' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/506609299' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.346 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"} : dispatch 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:28.347 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Saving service mon spec with placement smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c;count:3 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Deploying daemon mon.c on smithi174 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/3797978659' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.a calling monitor election 2024-09-10T20:55:28.348 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.c calling monitor election 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: monmap epoch 2 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: last_changed 2024-09-10T20:55:19.109579+0000 2024-09-10T20:55:28.349 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: min_mon_release 19 (squid) 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: election_strategy: 1 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: fsmap 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mgrmap e13: x(active, since 44s) 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: overall HEALTH_OK 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:28.350 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: Deploying daemon mon.b on smithi104 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/3620568425' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 ceph-mon[35325]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 bash[35306]: a799043be3baa5b8a17bf9b2ad4d6faa3de92c514c00580917f65fc820fba47f 2024-09-10T20:55:28.351 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:27 smithi104 systemd[1]: Started Ceph mon.b for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:55:29.062 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T20:55:29.063 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T20:55:29.304 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:55:33.193 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:55:33.193 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","modified":"2024-09-10T20:55:27.873092Z","created":"2024-09-10T20:53:45.201883Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-10T20:55:33.193 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: mon.a calling monitor election 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: mon.c calling monitor election 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: mon.b calling monitor election 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.205 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: monmap epoch 3 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: last_changed 2024-09-10T20:55:27.873092+0000 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: min_mon_release 19 (squid) 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: election_strategy: 1 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: fsmap 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:33.206 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: mgrmap e13: x(active, since 52s) 2024-09-10T20:55:33.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: overall HEALTH_OK 2024-09-10T20:55:33.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.207 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:55:33.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: mon.a calling monitor election 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: mon.c calling monitor election 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: mon.b calling monitor election 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: monmap epoch 3 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: last_changed 2024-09-10T20:55:27.873092+0000 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: min_mon_release 19 (squid) 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: election_strategy: 1 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: fsmap 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: mgrmap e13: x(active, since 52s) 2024-09-10T20:55:33.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: overall HEALTH_OK 2024-09-10T20:55:33.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:55:33.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: mon.a calling monitor election 2024-09-10T20:55:33.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: mon.c calling monitor election 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: mon.b calling monitor election 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: monmap epoch 3 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: last_changed 2024-09-10T20:55:27.873092+0000 2024-09-10T20:55:33.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: created 2024-09-10T20:53:45.201883+0000 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: min_mon_release 19 (squid) 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: election_strategy: 1 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.a 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: 2: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.b 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: fsmap 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: mgrmap e13: x(active, since 52s) 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: overall HEALTH_OK 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:33.803 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T20:55:33.803 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T20:55:34.071 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:33 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/1950406141' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:33 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/1950406141' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:34.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:34.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:33 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/1950406141' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T20:55:34.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:55:34.742 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:34.742 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-09-10T20:55:34.742 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T20:55:34.742 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-09-10T20:55:34.997 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 20:55:34 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T20:55:34.873+0000 7f0e0fd5c640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T20:55:35.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-10T20:55:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-10T20:55:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T20:55:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:35 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2839373634' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T20:55:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:35 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2839373634' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:35.406 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:35.406 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.406 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: Updating smithi033:/etc/ceph/ceph.conf 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: Updating smithi104:/etc/ceph/ceph.conf 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: Updating smithi174:/etc/ceph/ceph.conf 2024-09-10T20:55:35.407 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:35 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2839373634' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:35.447 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T20:55:35.448 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:55:35.448 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T20:55:35.477 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:55:35.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:35.547 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:55:35.547 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T20:55:35.575 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:55:35.575 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:35.640 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:55:35.641 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T20:55:35.667 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:55:35.667 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T20:55:35.733 INFO:tasks.cephadm:Adding mgr.x on smithi033 2024-09-10T20:55:35.733 INFO:tasks.cephadm:Adding mgr.y on smithi104 2024-09-10T20:55:35.733 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-09-10T20:55:35.733 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi033=x;smithi104=y;smithi174=z' 2024-09-10T20:55:36.008 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:55:36.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: Updating smithi033:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.373 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: Reconfiguring daemon mon.a on smithi033 2024-09-10T20:55:36.374 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: Updating smithi033:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T20:55:36.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:36.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:36.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:36.430 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: Reconfiguring daemon mon.a on smithi033 2024-09-10T20:55:36.431 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: Updating smithi033:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: Updating smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: Updating smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T20:55:36.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:36.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:36.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:36.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: Reconfiguring daemon mon.a on smithi033 2024-09-10T20:55:36.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:36.661 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-09-10T20:55:37.289 DEBUG:teuthology.orchestra.run.smithi104:mgr.y> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T20:55:37.292 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.z.service 2024-09-10T20:55:37.295 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-10T20:55:37.295 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set client client_mount_timeout 600 2024-09-10T20:55:37.298 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-10T20:55:37.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set client debug_client 20 2024-09-10T20:55:37.341 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-10T20:55:37.342 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set client debug_ms 1 2024-09-10T20:55:37.384 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-10T20:55:37.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set client rados_mon_op_timeout 900 2024-09-10T20:55:37.427 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-10T20:55:37.428 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set client rados_osd_op_timeout 900 2024-09-10T20:55:37.471 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-10T20:55:37.471 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds 20 2024-09-10T20:55:37.515 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-10T20:55:37.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds debug_mds_balancer 20 2024-09-10T20:55:37.558 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-10T20:55:37.558 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds debug_ms 2 2024-09-10T20:55:37.601 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-10T20:55:37.601 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-10T20:55:37.645 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-10T20:55:37.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_merge_size 5 2024-09-10T20:55:37.695 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-10T20:55:37.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_bits 3 2024-09-10T20:55:37.739 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-10T20:55:37.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_bal_split_size 100 2024-09-10T20:55:37.783 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-10T20:55:37.783 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_frag True 2024-09-10T20:55:37.826 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-10T20:55:37.826 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_debug_scatterstat True 2024-09-10T20:55:37.870 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-10T20:55:37.870 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_op_complaint_time 180 2024-09-10T20:55:37.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:37.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi104=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:37.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: Saving service mgr spec with placement smithi033=x;smithi104=y;smithi174=z;count:3 2024-09-10T20:55:37.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:37.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:37.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:37.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T20:55:37.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:37.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:37.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:37.899 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:37 smithi033 ceph-mon[30860]: Reconfiguring daemon mgr.x on smithi033 2024-09-10T20:55:37.900 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-10T20:55:37.900 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds mds_verify_scatter True 2024-09-10T20:55:37.943 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-10T20:55:37.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds osd_op_complaint_time 180 2024-09-10T20:55:37.988 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T20:55:37.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds rados_mon_op_timeout 900 2024-09-10T20:55:38.033 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T20:55:38.034 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mds rados_osd_op_timeout 900 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi104=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: Saving service mgr spec with placement smithi033=x;smithi104=y;smithi174=z;count:3 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T20:55:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:37 smithi104 ceph-mon[35325]: Reconfiguring daemon mgr.x on smithi033 2024-09-10T20:55:38.077 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-10T20:55:38.077 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mon mon_op_complaint_time 120 2024-09-10T20:55:38.121 INFO:tasks.cephadm:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T20:55:38.121 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi033=x;smithi104=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: Saving service mgr spec with placement smithi033=x;smithi104=y;smithi174=z;count:3 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: Reconfiguring mgr.x (unknown last config time)... 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:38.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:37 smithi174 ceph-mon[35635]: Reconfiguring daemon mgr.x on smithi033 2024-09-10T20:55:38.171 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-10T20:55:38.171 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config set osd osd_op_complaint_time 180 2024-09-10T20:55:38.219 DEBUG:tasks.cephadm:set 22 configs 2024-09-10T20:55:38.219 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi033') 2024-09-10T20:55:38.233 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:38.700 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:38.724 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:38.735 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:38.743 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 -- 172.21.15.33:0/2246750613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f352403f510 msgr2=0x7f35240419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 --2- 172.21.15.33:0/2246750613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f352403f510 0x7f35240419d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f352c0058c0 tx=0x7f352c005850 comp rx=0 tx=0).stop 2024-09-10T20:55:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 -- 172.21.15.33:0/2246750613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f353c105bf0 msgr2=0x7f353c198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 --2- 172.21.15.33:0/2246750613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f353c105bf0 0x7f353c198880 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f353000b560 tx=0x7f353000ba20 comp rx=0 tx=0).stop 2024-09-10T20:55:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 -- 172.21.15.33:0/2246750613 shutdown_connections 2024-09-10T20:55:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 --2- 172.21.15.33:0/2246750613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f352403f510 0x7f35240419d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:39.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 --2- 172.21.15.33:0/2246750613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f353c108740 0x7f353c1994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 --2- 172.21.15.33:0/2246750613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f353c105bf0 0x7f353c198880 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 --2- 172.21.15.33:0/2246750613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f353c06c060 0x7f353c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 -- 172.21.15.33:0/2246750613 >> 172.21.15.33:0/2246750613 conn(0x7f353c0fd200 msgr2=0x7f353c103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 -- 172.21.15.33:0/2246750613 shutdown_connections 2024-09-10T20:55:39.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:39.178+0000 7f35420ef640 1 -- 172.21.15.33:0/2246750613 wait complete. 2024-09-10T20:55:39.435 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: Reconfiguring mon.b (monmap changed)... 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: Reconfiguring daemon mon.b on smithi104 2024-09-10T20:55:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:39 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2246750613' entity='client.admin' 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: Reconfiguring mon.b (monmap changed)... 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:39.649 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: Reconfiguring daemon mon.b on smithi104 2024-09-10T20:55:39.650 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:39 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2246750613' entity='client.admin' 2024-09-10T20:55:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: Reconfiguring mon.b (monmap changed)... 2024-09-10T20:55:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:39.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:39.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: Reconfiguring daemon mon.b on smithi104 2024-09-10T20:55:39.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:39 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2246750613' entity='client.admin' 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 -- 172.21.15.33:0/2020607968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c105c30 msgr2=0x7f936c108020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 --2- 172.21.15.33:0/2020607968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c105c30 0x7f936c108020 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9360009900 tx=0x7f936002a410 comp rx=0 tx=0).stop 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 -- 172.21.15.33:0/2020607968 shutdown_connections 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 --2- 172.21.15.33:0/2020607968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f936c108760 0x7f936c10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 --2- 172.21.15.33:0/2020607968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c105c30 0x7f936c108020 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 --2- 172.21.15.33:0/2020607968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f936c06c000 0x7f936c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 -- 172.21.15.33:0/2020607968 >> 172.21.15.33:0/2020607968 conn(0x7f936c0fd200 msgr2=0x7f936c0ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 -- 172.21.15.33:0/2020607968 shutdown_connections 2024-09-10T20:55:40.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.214+0000 7f9371cb2640 1 -- 172.21.15.33:0/2020607968 wait complete. 2024-09-10T20:55:40.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 Processor -- start 2024-09-10T20:55:40.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 -- start start 2024-09-10T20:55:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c06c000 0x7f936c1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f936c105c30 0x7f936c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f936c108760 0x7f936c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f936c10da50 con 0x7f936c06c000 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f936c10d8d0 con 0x7f936c105c30 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f9371cb2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f936c10dbd0 con 0x7f936c108760 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f936b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c06c000 0x7f936c1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f936b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c06c000 0x7f936c1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48704/0 (socket says 172.21.15.33:48704) 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.215+0000 7f936b7fe640 1 -- 172.21.15.33:0/288703775 learned_addr learned my addr 172.21.15.33:0/288703775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936bfff640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f936c108760 0x7f936c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936affd640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f936c105c30 0x7f936c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936b7fe640 1 -- 172.21.15.33:0/288703775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f936c108760 msgr2=0x7f936c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936b7fe640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f936c108760 0x7f936c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936b7fe640 1 -- 172.21.15.33:0/288703775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f936c105c30 msgr2=0x7f936c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936b7fe640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f936c105c30 0x7f936c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936b7fe640 1 -- 172.21.15.33:0/288703775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f936c199e10 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936affd640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f936c105c30 0x7f936c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f936b7fe640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c06c000 0x7f936c1982e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f935c00d770 tx=0x7f935c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c017070 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(22 keys) ==== 983+0+0 (secure 0 0 0) 0x7f935c005460 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f936c19a100 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f935c004aa0 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.216+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f936c1a07e0 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.217+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f935c004c40 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.217+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f936c106010 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.218+0000 7f9368ff9640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f935403f510 0x7f93540419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.218+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f935c053810 con 0x7f936c06c000 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.218+0000 7f936affd640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f935403f510 0x7f93540419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:40.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.218+0000 7f936affd640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f935403f510 0x7f93540419d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:40.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.222+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f935c019310 con 0x7f936c06c000 2024-09-10T20:55:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.414+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f936c063d50 con 0x7f936c06c000 2024-09-10T20:55:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.418+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v11)=0 v11) ==== 123+0+0 (secure 0 0 0) 0x7f935c005020 con 0x7f936c06c000 2024-09-10T20:55:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.419+0000 7f9368ff9640 1 -- 172.21.15.33:0/288703775 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f935c009c30 con 0x7f936c06c000 2024-09-10T20:55:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f935403f510 msgr2=0x7f93540419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f935403f510 0x7f93540419d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9360000c00 tx=0x7f9360005a20 comp rx=0 tx=0).stop 2024-09-10T20:55:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c06c000 msgr2=0x7f936c1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c06c000 0x7f936c1982e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f935c00d770 tx=0x7f935c00dc30 comp rx=0 tx=0).stop 2024-09-10T20:55:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 shutdown_connections 2024-09-10T20:55:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f935403f510 0x7f93540419d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f936c108760 0x7f936c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f936c105c30 0x7f936c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 --2- 172.21.15.33:0/288703775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f936c06c000 0x7f936c1982e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 >> 172.21.15.33:0/288703775 conn(0x7f936c0fd200 msgr2=0x7f936c103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 shutdown_connections 2024-09-10T20:55:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:40.420+0000 7f9371cb2640 1 -- 172.21.15.33:0/288703775 wait complete. 2024-09-10T20:55:41.030 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: Reconfiguring mon.c (monmap changed)... 2024-09-10T20:55:41.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:41.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:41.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:41.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: Reconfiguring daemon mon.c on smithi174 2024-09-10T20:55:41.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:40 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/288703775' entity='client.admin' 2024-09-10T20:55:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: Reconfiguring mon.c (monmap changed)... 2024-09-10T20:55:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: Reconfiguring daemon mon.c on smithi174 2024-09-10T20:55:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:40 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/288703775' entity='client.admin' 2024-09-10T20:55:41.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.140 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: Reconfiguring mon.c (monmap changed)... 2024-09-10T20:55:41.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T20:55:41.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T20:55:41.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:41.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: Reconfiguring daemon mon.c on smithi174 2024-09-10T20:55:41.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:40 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/288703775' entity='client.admin' 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 -- 172.21.15.33:0/3135446844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 msgr2=0x7f39a0109db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 --2- 172.21.15.33:0/3135446844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 0x7f39a0109db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3998009cd0 tx=0x7f399802a8e0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 -- 172.21.15.33:0/3135446844 shutdown_connections 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 --2- 172.21.15.33:0/3135446844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 0x7f39a0109db0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 --2- 172.21.15.33:0/3135446844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39a0102a70 0x7f39a0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 --2- 172.21.15.33:0/3135446844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 0x7f39a0101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 -- 172.21.15.33:0/3135446844 >> 172.21.15.33:0/3135446844 conn(0x7f39a00fd1e0 msgr2=0x7f39a00ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 -- 172.21.15.33:0/3135446844 shutdown_connections 2024-09-10T20:55:41.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.247+0000 7f39a4e25640 1 -- 172.21.15.33:0/3135446844 wait complete. 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 Processor -- start 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 -- start start 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 0x7f39a01056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39a0102a70 0x7f39a0103f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 0x7f39a0104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010d250 con 0x7f39a0103430 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010d0d0 con 0x7f39a0102a70 2024-09-10T20:55:41.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f39a4e25640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39a010d3d0 con 0x7f39a0101910 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 0x7f39a0104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 0x7f39a01056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 0x7f39a0104480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48726/0 (socket says 172.21.15.33:48726) 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 0x7f39a01056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:43946/0 (socket says 172.21.15.33:43946) 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399ed9d640 1 -- 172.21.15.33:0/1018683653 learned_addr learned my addr 172.21.15.33:0/1018683653 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399dd9b640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39a0102a70 0x7f39a0103f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399e59c640 1 -- 172.21.15.33:0/1018683653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39a0102a70 msgr2=0x7f39a0103f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399e59c640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39a0102a70 0x7f39a0103f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399e59c640 1 -- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 msgr2=0x7f39a0104480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399e59c640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 0x7f39a0104480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.248+0000 7f399e59c640 1 -- 172.21.15.33:0/1018683653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a0104d40 con 0x7f39a0101910 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.249+0000 7f399e59c640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 0x7f39a01056d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f399000e9f0 tx=0x7f399000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.249+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399000ccc0 con 0x7f39a0101910 2024-09-10T20:55:41.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.249+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a01a2b60 con 0x7f39a0101910 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.249+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(23 keys) ==== 1014+0+0 (secure 0 0 0) 0x7f3990004480 con 0x7f39a0101910 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.249+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39900047a0 con 0x7f39a0101910 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.249+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a01a3020 con 0x7f39a0101910 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.250+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f3990004960 con 0x7f39a0101910 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.250+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a01a31f0 con 0x7f39a0101910 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.250+0000 7f398f7fe640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f398803f510 0x7f39880419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.250+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3990053d80 con 0x7f39a0101910 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.250+0000 7f399dd9b640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f398803f510 0x7f39880419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:41.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.251+0000 7f399dd9b640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f398803f510 0x7f39880419d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3994005e10 tx=0x7f3994005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.255+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3990016330 con 0x7f39a0101910 2024-09-10T20:55:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.444+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f39a01a33c0 con 0x7f39a0101910 2024-09-10T20:55:41.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.449+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 7 ==== config(24 keys) ==== 1045+0+0 (secure 0 0 0) 0x7f3990005020 con 0x7f39a0101910 2024-09-10T20:55:41.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.449+0000 7f398f7fe640 1 -- 172.21.15.33:0/1018683653 <== mon.1 v2:172.21.15.174:3300/0 8 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7f3990004160 con 0x7f39a0101910 2024-09-10T20:55:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f398803f510 msgr2=0x7f39880419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f398803f510 0x7f39880419d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3994005e10 tx=0x7f3994005da0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 msgr2=0x7f39a01056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 0x7f39a01056d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f399000e9f0 tx=0x7f399000eeb0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 shutdown_connections 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f398803f510 0x7f39880419d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39a0103430 0x7f39a0104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39a0102a70 0x7f39a0103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 --2- 172.21.15.33:0/1018683653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39a0101910 0x7f39a01056d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 >> 172.21.15.33:0/1018683653 conn(0x7f39a00fd1e0 msgr2=0x7f39a00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 shutdown_connections 2024-09-10T20:55:41.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:41.451+0000 7f39a4e25640 1 -- 172.21.15.33:0/1018683653 wait complete. 2024-09-10T20:55:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:41.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:41.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:41.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T20:55:41.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:41.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:41.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:41 smithi174 ceph-mon[35635]: from='client.? ' entity='client.admin' 2024-09-10T20:55:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:41 smithi104 ceph-mon[35325]: from='client.? ' entity='client.admin' 2024-09-10T20:55:42.087 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:42.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:42.089 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:41 smithi033 ceph-mon[30860]: from='client.? ' entity='client.admin' 2024-09-10T20:55:42.118 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi033') 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 -- 172.21.15.33:0/3515957138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc102a50 msgr2=0x7f5fdc102e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 --2- 172.21.15.33:0/3515957138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc102a50 0x7f5fdc102e50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8009930 tx=0x7f5fd802a420 comp rx=0 tx=0).stop 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 -- 172.21.15.33:0/3515957138 shutdown_connections 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 --2- 172.21.15.33:0/3515957138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fdc104570 0x7f5fdc10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 --2- 172.21.15.33:0/3515957138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc103bb0 0x7f5fdc104030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 --2- 172.21.15.33:0/3515957138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc102a50 0x7f5fdc102e50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 -- 172.21.15.33:0/3515957138 >> 172.21.15.33:0/3515957138 conn(0x7f5fdc0fe320 msgr2=0x7f5fdc100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 -- 172.21.15.33:0/3515957138 shutdown_connections 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 -- 172.21.15.33:0/3515957138 wait complete. 2024-09-10T20:55:42.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 Processor -- start 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.260+0000 7f5fe438c640 1 -- start start 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe438c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc102a50 0x7f5fdc198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe438c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fdc103bb0 0x7f5fdc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe438c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc104570 0x7f5fdc199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe438c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10e4a0 con 0x7f5fdc104570 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe438c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10e320 con 0x7f5fdc102a50 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe438c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fdc10e620 con 0x7f5fdc103bb0 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc104570 0x7f5fdc199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc104570 0x7f5fdc199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48744/0 (socket says 172.21.15.33:48744) 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 -- 172.21.15.33:0/2131875209 learned_addr learned my addr 172.21.15.33:0/2131875209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2128640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc102a50 0x7f5fdc198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe1927640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fdc103bb0 0x7f5fdc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 -- 172.21.15.33:0/2131875209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fdc103bb0 msgr2=0x7f5fdc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fdc103bb0 0x7f5fdc1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 -- 172.21.15.33:0/2131875209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc102a50 msgr2=0x7f5fdc198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc102a50 0x7f5fdc198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 -- 172.21.15.33:0/2131875209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fdc19a780 con 0x7f5fdc104570 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe2929640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc104570 0x7f5fdc199ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400eb10 tx=0x7f5fd400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fcf7fe640 1 -- 172.21.15.33:0/2131875209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4004b20 con 0x7f5fdc104570 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.261+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fdc19aa70 con 0x7f5fdc104570 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.262+0000 7f5fcf7fe640 1 -- 172.21.15.33:0/2131875209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(24 keys) ==== 1045+0+0 (secure 0 0 0) 0x7f5fd4004cc0 con 0x7f5fdc104570 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.262+0000 7f5fcf7fe640 1 -- 172.21.15.33:0/2131875209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4010690 con 0x7f5fdc104570 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.262+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fdc1a1150 con 0x7f5fdc104570 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.262+0000 7f5fcf7fe640 1 -- 172.21.15.33:0/2131875209 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f5fd4010830 con 0x7f5fdc104570 2024-09-10T20:55:42.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.263+0000 7f5fcf7fe640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fc803f510 0x7f5fc80419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.263+0000 7f5fcf7fe640 1 -- 172.21.15.33:0/2131875209 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5fd4059660 con 0x7f5fdc104570 2024-09-10T20:55:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.263+0000 7f5fe2128640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fc803f510 0x7f5fc80419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.263+0000 7f5fe2128640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fc803f510 0x7f5fc80419d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8004770 tx=0x7f5fd80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.264+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fdc109e50 con 0x7f5fdc104570 2024-09-10T20:55:42.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.268+0000 7f5fcf7fe640 1 -- 172.21.15.33:0/2131875209 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5fd4017e50 con 0x7f5fdc104570 2024-09-10T20:55:42.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.458+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f5fdc063d50 con 0x7f5fdc104570 2024-09-10T20:55:42.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.461+0000 7f5fcf7fe640 1 -- 172.21.15.33:0/2131875209 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v13)=0 v13) ==== 106+0+0 (secure 0 0 0) 0x7f5fd400baf0 con 0x7f5fdc104570 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.463+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fc803f510 msgr2=0x7f5fc80419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.463+0000 7f5fe438c640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fc803f510 0x7f5fc80419d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8004770 tx=0x7f5fd80058d0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.463+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc104570 msgr2=0x7f5fdc199ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.463+0000 7f5fe438c640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc104570 0x7f5fdc199ee0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400eb10 tx=0x7f5fd400c2d0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 shutdown_connections 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fc803f510 0x7f5fc80419d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fdc104570 0x7f5fdc199ee0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fdc103bb0 0x7f5fdc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 --2- 172.21.15.33:0/2131875209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5fdc102a50 0x7f5fdc198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 >> 172.21.15.33:0/2131875209 conn(0x7f5fdc0fe320 msgr2=0x7f5fdc107020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:42.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 shutdown_connections 2024-09-10T20:55:42.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.464+0000 7f5fe438c640 1 -- 172.21.15.33:0/2131875209 wait complete. 2024-09-10T20:55:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 -- 172.21.15.33:0/3094951262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 msgr2=0x7f4220101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 --2- 172.21.15.33:0/3094951262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f4220101d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4218009cd0 tx=0x7f421802a8e0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 -- 172.21.15.33:0/3094951262 shutdown_connections 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 --2- 172.21.15.33:0/3094951262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4220103430 0x7f4220109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 --2- 172.21.15.33:0/3094951262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4220102a70 0x7f4220102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 --2- 172.21.15.33:0/3094951262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f4220101d10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 -- 172.21.15.33:0/3094951262 >> 172.21.15.33:0/3094951262 conn(0x7f42200fd1e0 msgr2=0x7f42200ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.339+0000 7f4226e3a640 1 -- 172.21.15.33:0/3094951262 shutdown_connections 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.340+0000 7f4226e3a640 1 -- 172.21.15.33:0/3094951262 wait complete. 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.340+0000 7f4226e3a640 1 Processor -- start 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.340+0000 7f4226e3a640 1 -- start start 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.340+0000 7f4226e3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f42201056d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.340+0000 7f4226e3a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4220102a70 0x7f4220103f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.340+0000 7f4226e3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4220103430 0x7f4220104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4226e3a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f422010d360 con 0x7f4220101910 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4226e3a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f422010d1e0 con 0x7f4220102a70 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4226e3a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f422010d4e0 con 0x7f4220103430 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f42201056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f42201056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48752/0 (socket says 172.21.15.33:48752) 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 -- 172.21.15.33:0/2839629280 learned_addr learned my addr 172.21.15.33:0/2839629280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:42.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f42253d7640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4220103430 0x7f4220104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4217fff640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4220102a70 0x7f4220103f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 -- 172.21.15.33:0/2839629280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4220103430 msgr2=0x7f4220104480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4220103430 0x7f4220104480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 -- 172.21.15.33:0/2839629280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4220102a70 msgr2=0x7f4220103f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4220102a70 0x7f4220103f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 -- 172.21.15.33:0/2839629280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4220104d40 con 0x7f4220101910 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f42253d7640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4220103430 0x7f4220104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4224bd6640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f42201056d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4218006010 tx=0x7f4218003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218004450 con 0x7f4220101910 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42201a2d10 con 0x7f4220101910 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(24 keys) ==== 1045+0+0 (secure 0 0 0) 0x7f4218003e50 con 0x7f4220101910 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.341+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421803b400 con 0x7f4220101910 2024-09-10T20:55:42.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.342+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42201a3170 con 0x7f4220101910 2024-09-10T20:55:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.343+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42201a33a0 con 0x7f4220101910 2024-09-10T20:55:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.343+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f4218030070 con 0x7f4220101910 2024-09-10T20:55:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.343+0000 7f4215ffb640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f420803f510 0x7f42080419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.343+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f421802e030 con 0x7f4220101910 2024-09-10T20:55:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.343+0000 7f4217fff640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f420803f510 0x7f42080419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.344+0000 7f4217fff640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f420803f510 0x7f42080419d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f421c005d00 tx=0x7f421c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:42.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.348+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f42180453e0 con 0x7f4220101910 2024-09-10T20:55:42.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.541+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f42201a3c20 con 0x7f4220101910 2024-09-10T20:55:42.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.547+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v14)=0 v14) ==== 114+0+0 (secure 0 0 0) 0x7f4218039c80 con 0x7f4220101910 2024-09-10T20:55:42.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.547+0000 7f4215ffb640 1 -- 172.21.15.33:0/2839629280 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4218036030 con 0x7f4220101910 2024-09-10T20:55:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f420803f510 msgr2=0x7f42080419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f420803f510 0x7f42080419d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f421c005d00 tx=0x7f421c00b410 comp rx=0 tx=0).stop 2024-09-10T20:55:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 msgr2=0x7f42201056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f42201056d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4218006010 tx=0x7f4218003ce0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 shutdown_connections 2024-09-10T20:55:42.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f420803f510 0x7f42080419d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4220103430 0x7f4220104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4220102a70 0x7f4220103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 --2- 172.21.15.33:0/2839629280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4220101910 0x7f42201056d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:42.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 >> 172.21.15.33:0/2839629280 conn(0x7f42200fd1e0 msgr2=0x7f422006aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:42.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 shutdown_connections 2024-09-10T20:55:42.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:42.549+0000 7f4226e3a640 1 -- 172.21.15.33:0/2839629280 wait complete. 2024-09-10T20:55:42.805 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:42 smithi174 ceph-mon[35635]: Deploying daemon mgr.z on smithi174 2024-09-10T20:55:42.805 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:42 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2131875209' entity='client.admin' 2024-09-10T20:55:42.805 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:42 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2839629280' entity='client.admin' 2024-09-10T20:55:43.018 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:42 smithi033 ceph-mon[30860]: Deploying daemon mgr.z on smithi174 2024-09-10T20:55:43.018 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:42 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2131875209' entity='client.admin' 2024-09-10T20:55:43.018 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:42 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2839629280' entity='client.admin' 2024-09-10T20:55:43.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:42 smithi104 ceph-mon[35325]: Deploying daemon mgr.z on smithi174 2024-09-10T20:55:43.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:42 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2131875209' entity='client.admin' 2024-09-10T20:55:43.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:42 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2839629280' entity='client.admin' 2024-09-10T20:55:43.144 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:42 smithi174 systemd[1]: Starting Ceph mgr.z for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T20:55:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.427+0000 7fbed803b640 1 -- 172.21.15.33:0/1464803691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0102a40 msgr2=0x7fbed0102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:43.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.427+0000 7fbed803b640 1 --2- 172.21.15.33:0/1464803691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0102a40 0x7fbed0102e40 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbecc009930 tx=0x7fbecc02a420 comp rx=0 tx=0).stop 2024-09-10T20:55:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.429+0000 7fbed803b640 1 -- 172.21.15.33:0/1464803691 shutdown_connections 2024-09-10T20:55:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.429+0000 7fbed803b640 1 --2- 172.21.15.33:0/1464803691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbed0104560 0x7fbed010aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.429+0000 7fbed803b640 1 --2- 172.21.15.33:0/1464803691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed0103ba0 0x7fbed0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.429+0000 7fbed803b640 1 --2- 172.21.15.33:0/1464803691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0102a40 0x7fbed0102e40 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.429+0000 7fbed803b640 1 -- 172.21.15.33:0/1464803691 >> 172.21.15.33:0/1464803691 conn(0x7fbed00fe2f0 msgr2=0x7fbed0100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:43.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.430+0000 7fbed803b640 1 -- 172.21.15.33:0/1464803691 shutdown_connections 2024-09-10T20:55:43.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.430+0000 7fbed803b640 1 -- 172.21.15.33:0/1464803691 wait complete. 2024-09-10T20:55:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 Processor -- start 2024-09-10T20:55:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 -- start start 2024-09-10T20:55:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed0102a40 0x7fbed0076c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0103ba0 0x7fbed0075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbed0104560 0x7fbed0075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010e580 con 0x7fbed0103ba0 2024-09-10T20:55:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010e400 con 0x7fbed0104560 2024-09-10T20:55:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed803b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbed010e700 con 0x7fbed0102a40 2024-09-10T20:55:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed55d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0103ba0 0x7fbed0075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed55d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0103ba0 0x7fbed0075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48774/0 (socket says 172.21.15.33:48774) 2024-09-10T20:55:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.431+0000 7fbed55d6640 1 -- 172.21.15.33:0/1025523070 learned_addr learned my addr 172.21.15.33:0/1025523070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:43.437 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:43 smithi174 podman[37204]: 2024-09-10 20:55:43.169921061 +0000 UTC m=+0.099107734 container create 2d5a591f66032b20dda64bf5dba9384128f84f74abab8d43395f818469a0b1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T20:55:43.437 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:43 smithi174 podman[37204]: 2024-09-10 20:55:43.086602546 +0000 UTC m=+0.015789185 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T20:55:43.437 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:43 smithi174 podman[37204]: 2024-09-10 20:55:43.405449478 +0000 UTC m=+0.334636108 container init 2d5a591f66032b20dda64bf5dba9384128f84f74abab8d43395f818469a0b1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T20:55:43.437 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:43 smithi174 podman[37204]: 2024-09-10 20:55:43.408133625 +0000 UTC m=+0.337320256 container start 2d5a591f66032b20dda64bf5dba9384128f84f74abab8d43395f818469a0b1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T20:55:43.437 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mgr[37223]: -- 172.21.15.174:0/37994581 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634edd1b0e0 con 0x5634edcf9400 2024-09-10T20:55:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.432+0000 7fbed55d6640 1 -- 172.21.15.33:0/1025523070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed0102a40 msgr2=0x7fbed0076c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.432+0000 7fbed55d6640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed0102a40 0x7fbed0076c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.432+0000 7fbed55d6640 1 -- 172.21.15.33:0/1025523070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbed0104560 msgr2=0x7fbed0075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.432+0000 7fbed55d6640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbed0104560 0x7fbed0075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.432+0000 7fbed55d6640 1 -- 172.21.15.33:0/1025523070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed0075fc0 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.432+0000 7fbed55d6640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0103ba0 0x7fbed0075540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbec400e9f0 tx=0x7fbec400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.432+0000 7fbec2ffd640 1 -- 172.21.15.33:0/1025523070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec400cd80 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.433+0000 7fbec2ffd640 1 -- 172.21.15.33:0/1025523070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fbec40054b0 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.433+0000 7fbec2ffd640 1 -- 172.21.15.33:0/1025523070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec4010430 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.433+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed00762b0 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.434+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a77e0 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.434+0000 7fbec2ffd640 1 -- 172.21.15.33:0/1025523070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7fbec4010630 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.434+0000 7fbec2ffd640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbebc03f510 0x7fbebc0419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.434+0000 7fbec2ffd640 1 -- 172.21.15.33:0/1025523070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbec4059660 con 0x7fbed0103ba0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.435+0000 7fbed5dd7640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbebc03f510 0x7fbebc0419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.435+0000 7fbed5dd7640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbebc03f510 0x7fbebc0419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02a940 tx=0x7fbecc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.436+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbed0109e40 con 0x7fbed0103ba0 2024-09-10T20:55:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:43.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.447+0000 7fbec2ffd640 1 -- 172.21.15.33:0/1025523070 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbec4017e50 con 0x7fbed0103ba0 2024-09-10T20:55:43.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.667+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7fbed002e780 con 0x7fbed0103ba0 2024-09-10T20:55:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.672+0000 7fbec2ffd640 1 -- 172.21.15.33:0/1025523070 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v15)=0 v15) ==== 116+0+0 (secure 0 0 0) 0x7fbec40047a0 con 0x7fbed0103ba0 2024-09-10T20:55:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbebc03f510 msgr2=0x7fbebc0419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbebc03f510 0x7fbebc0419d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbecc02a940 tx=0x7fbecc000bc0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0103ba0 msgr2=0x7fbed0075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0103ba0 0x7fbed0075540 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbec400e9f0 tx=0x7fbec400edf0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 shutdown_connections 2024-09-10T20:55:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbebc03f510 0x7fbebc0419d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbed0104560 0x7fbed0075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbed0103ba0 0x7fbed0075540 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 --2- 172.21.15.33:0/1025523070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbed0102a40 0x7fbed0076c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 >> 172.21.15.33:0/1025523070 conn(0x7fbed00fe2f0 msgr2=0x7fbed0107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 shutdown_connections 2024-09-10T20:55:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:43.674+0000 7fbed803b640 1 -- 172.21.15.33:0/1025523070 wait complete. 2024-09-10T20:55:43.726 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:43 smithi174 bash[37204]: 2d5a591f66032b20dda64bf5dba9384128f84f74abab8d43395f818469a0b1eb 2024-09-10T20:55:43.726 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 20:55:43 smithi174 systemd[1]: Started Ceph mgr.z for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:43.868 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:43 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1025523070' entity='client.admin' 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:44.023 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:43 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1025523070' entity='client.admin' 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:43 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1025523070' entity='client.admin' 2024-09-10T20:55:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 -- 172.21.15.33:0/892359868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4103b90 msgr2=0x7f07d4104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:44.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 --2- 172.21.15.33:0/892359868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4103b90 0x7f07d4104010 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f07bc009930 tx=0x7f07bc02a420 comp rx=0 tx=0).stop 2024-09-10T20:55:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 -- 172.21.15.33:0/892359868 shutdown_connections 2024-09-10T20:55:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 --2- 172.21.15.33:0/892359868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07d4104550 0x7f07d410aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 --2- 172.21.15.33:0/892359868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4103b90 0x7f07d4104010 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 --2- 172.21.15.33:0/892359868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07d4102a30 0x7f07d4102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 -- 172.21.15.33:0/892359868 >> 172.21.15.33:0/892359868 conn(0x7f07d40fe320 msgr2=0x7f07d4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:44.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 -- 172.21.15.33:0/892359868 shutdown_connections 2024-09-10T20:55:44.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 -- 172.21.15.33:0/892359868 wait complete. 2024-09-10T20:55:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 Processor -- start 2024-09-10T20:55:44.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.411+0000 7f07d9e33640 1 -- start start 2024-09-10T20:55:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d9e33640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07d4102a30 0x7f07d4198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d9e33640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07d4103b90 0x7f07d41993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d9e33640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4104550 0x7f07d419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d9e33640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07d410e620 con 0x7f07d4104550 2024-09-10T20:55:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d9e33640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f07d410e4a0 con 0x7f07d4102a30 2024-09-10T20:55:44.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d9e33640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07d410e7a0 con 0x7f07d4103b90 2024-09-10T20:55:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4104550 0x7f07d419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4104550 0x7f07d419a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48788/0 (socket says 172.21.15.33:48788) 2024-09-10T20:55:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 -- 172.21.15.33:0/86812861 learned_addr learned my addr 172.21.15.33:0/86812861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:44.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07cbfff640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07d4103b90 0x7f07d41993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d37fe640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07d4102a30 0x7f07d4198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 -- 172.21.15.33:0/86812861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07d4103b90 msgr2=0x7f07d41993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07d4103b90 0x7f07d41993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 -- 172.21.15.33:0/86812861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07d4102a30 msgr2=0x7f07d4198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07d4102a30 0x7f07d4198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 -- 172.21.15.33:0/86812861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d419a8a0 con 0x7f07d4104550 2024-09-10T20:55:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.412+0000 7f07d3fff640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4104550 0x7f07d419a000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00b970 tx=0x7f07cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.413+0000 7f07d17fa640 1 -- 172.21.15.33:0/86812861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc019070 con 0x7f07d4104550 2024-09-10T20:55:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.413+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d419ab90 con 0x7f07d4104550 2024-09-10T20:55:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.413+0000 7f07d17fa640 1 -- 172.21.15.33:0/86812861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f07cc014440 con 0x7f07d4104550 2024-09-10T20:55:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.413+0000 7f07d17fa640 1 -- 172.21.15.33:0/86812861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc0075d0 con 0x7f07d4104550 2024-09-10T20:55:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.413+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d41a1270 con 0x7f07d4104550 2024-09-10T20:55:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.414+0000 7f07d17fa640 1 -- 172.21.15.33:0/86812861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f07cc014930 con 0x7f07d4104550 2024-09-10T20:55:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.414+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07d4109db0 con 0x7f07d4104550 2024-09-10T20:55:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.414+0000 7f07d17fa640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07c003f510 0x7f07c00419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:44.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.414+0000 7f07d17fa640 1 -- 172.21.15.33:0/86812861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f07cc053620 con 0x7f07d4104550 2024-09-10T20:55:44.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.414+0000 7f07d37fe640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07c003f510 0x7f07c00419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:44.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.414+0000 7f07d37fe640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07c003f510 0x7f07c00419d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f07c4005d00 tx=0x7f07c400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.419+0000 7f07d17fa640 1 -- 172.21.15.33:0/86812861 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f07cc01c360 con 0x7f07d4104550 2024-09-10T20:55:44.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.607+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f07d4063d50 con 0x7f07d4104550 2024-09-10T20:55:44.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.610+0000 7f07d17fa640 1 -- 172.21.15.33:0/86812861 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7f07cc010ca0 con 0x7f07d4104550 2024-09-10T20:55:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07c003f510 msgr2=0x7f07c00419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07c003f510 0x7f07c00419d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f07c4005d00 tx=0x7f07c400b410 comp rx=0 tx=0).stop 2024-09-10T20:55:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4104550 msgr2=0x7f07d419a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4104550 0x7f07d419a000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f07cc00b970 tx=0x7f07cc00be30 comp rx=0 tx=0).stop 2024-09-10T20:55:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 shutdown_connections 2024-09-10T20:55:44.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07c003f510 0x7f07c00419d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07d4104550 0x7f07d419a000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07d4103b90 0x7f07d41993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 --2- 172.21.15.33:0/86812861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07d4102a30 0x7f07d4198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 >> 172.21.15.33:0/86812861 conn(0x7f07d40fe320 msgr2=0x7f07d4106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 shutdown_connections 2024-09-10T20:55:44.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:44.612+0000 7f07d9e33640 1 -- 172.21.15.33:0/86812861 wait complete. 2024-09-10T20:55:44.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:44 smithi033 ceph-mon[30860]: Deploying daemon mgr.y on smithi104 2024-09-10T20:55:44.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:44 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/86812861' entity='client.admin' 2024-09-10T20:55:44.960 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi033') 2024-09-10T20:55:44.961 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi033') 2024-09-10T20:55:44.961 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi033') 2024-09-10T20:55:44.987 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:44 smithi104 ceph-mon[35325]: Deploying daemon mgr.y on smithi104 2024-09-10T20:55:44.987 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:44 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/86812861' entity='client.admin' 2024-09-10T20:55:45.134 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:45.135 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:45.136 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:44 smithi174 ceph-mon[35635]: Deploying daemon mgr.y on smithi104 2024-09-10T20:55:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:44 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/86812861' entity='client.admin' 2024-09-10T20:55:45.562 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:45 smithi104 systemd[1]: Starting Ceph mgr.y for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T20:55:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 -- 172.21.15.33:0/2969627280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140102a30 msgr2=0x7fb140102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 --2- 172.21.15.33:0/2969627280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140102a30 0x7fb140102e30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb130009cd0 tx=0x7fb13002a8e0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 -- 172.21.15.33:0/2969627280 shutdown_connections 2024-09-10T20:55:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 --2- 172.21.15.33:0/2969627280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140104550 0x7fb14010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 --2- 172.21.15.33:0/2969627280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb140103b90 0x7fb140104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 --2- 172.21.15.33:0/2969627280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140102a30 0x7fb140102e30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 -- 172.21.15.33:0/2969627280 >> 172.21.15.33:0/2969627280 conn(0x7fb1400fe320 msgr2=0x7fb140100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 -- 172.21.15.33:0/2969627280 shutdown_connections 2024-09-10T20:55:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.740+0000 7fb14497b640 1 -- 172.21.15.33:0/2969627280 wait complete. 2024-09-10T20:55:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 Processor -- start 2024-09-10T20:55:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 -- start start 2024-09-10T20:55:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140102a30 0x7fb140198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb140103b90 0x7fb140199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140104550 0x7fb140199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb14010e550 con 0x7fb140104550 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb14010e3d0 con 0x7fb140102a30 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb14497b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb14010e6d0 con 0x7fb140103b90 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb13ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140104550 0x7fb140199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb13ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140104550 0x7fb140199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48798/0 (socket says 172.21.15.33:48798) 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb13ed9d640 1 -- 172.21.15.33:0/268619382 learned_addr learned my addr 172.21.15.33:0/268619382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb13e59c640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140102a30 0x7fb140198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.741+0000 7fb13dd9b640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb140103b90 0x7fb140199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb13ed9d640 1 -- 172.21.15.33:0/268619382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb140103b90 msgr2=0x7fb140199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb13ed9d640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb140103b90 0x7fb140199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb13ed9d640 1 -- 172.21.15.33:0/268619382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140102a30 msgr2=0x7fb140198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb13ed9d640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140102a30 0x7fb140198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb13ed9d640 1 -- 172.21.15.33:0/268619382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14019a870 con 0x7fb140104550 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb13ed9d640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140104550 0x7fb140199fd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb13800d750 tx=0x7fb13800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb12f7fe640 1 -- 172.21.15.33:0/268619382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138017070 con 0x7fb140104550 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14019ab60 con 0x7fb140104550 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb12f7fe640 1 -- 172.21.15.33:0/268619382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb1380054b0 con 0x7fb140104550 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb12f7fe640 1 -- 172.21.15.33:0/268619382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb138004ba0 con 0x7fb140104550 2024-09-10T20:55:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.742+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb140075550 con 0x7fb140104550 2024-09-10T20:55:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.743+0000 7fb12f7fe640 1 -- 172.21.15.33:0/268619382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7fb138010460 con 0x7fb140104550 2024-09-10T20:55:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.743+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb140109e30 con 0x7fb140104550 2024-09-10T20:55:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.744+0000 7fb12f7fe640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb12803f510 0x7fb1280419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.744+0000 7fb12f7fe640 1 -- 172.21.15.33:0/268619382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb138053760 con 0x7fb140104550 2024-09-10T20:55:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.744+0000 7fb13e59c640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb12803f510 0x7fb1280419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.744+0000 7fb13e59c640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb12803f510 0x7fb1280419d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb1300316c0 tx=0x7fb130031000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:45.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.748+0000 7fb12f7fe640 1 -- 172.21.15.33:0/268619382 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb138009bd0 con 0x7fb140104550 2024-09-10T20:55:45.879 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:45.881 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:45 smithi104 ceph-mon[35325]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:45.881 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:45 smithi104 podman[36447]: 2024-09-10 20:55:45.527881884 +0000 UTC m=+0.015751835 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T20:55:45.881 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:45 smithi104 podman[36447]: 2024-09-10 20:55:45.657443108 +0000 UTC m=+0.145313092 container create 8307e8746dc883a4069f4965e081475bd670124128b25b54c58bdec20851d554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-y, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T20:55:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:45.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:45 smithi033 ceph-mon[30860]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:45.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.945+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7fb14019ad90 con 0x7fb140104550 2024-09-10T20:55:45.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.950+0000 7fb12f7fe640 1 -- 172.21.15.33:0/268619382 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v17)=0 v17) ==== 116+0+0 (secure 0 0 0) 0x7fb138010780 con 0x7fb140104550 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb12803f510 msgr2=0x7fb1280419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb12803f510 0x7fb1280419d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb1300316c0 tx=0x7fb130031000 comp rx=0 tx=0).stop 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140104550 msgr2=0x7fb140199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140104550 0x7fb140199fd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb13800d750 tx=0x7fb13800dc10 comp rx=0 tx=0).stop 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 shutdown_connections 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb12803f510 0x7fb1280419d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb140104550 0x7fb140199fd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb140103b90 0x7fb140199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 --2- 172.21.15.33:0/268619382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb140102a30 0x7fb140198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 >> 172.21.15.33:0/268619382 conn(0x7fb1400fe320 msgr2=0x7fb140107000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 shutdown_connections 2024-09-10T20:55:45.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:45.953+0000 7fb14497b640 1 -- 172.21.15.33:0/268619382 wait complete. 2024-09-10T20:55:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:45 smithi174 ceph-mon[35635]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:46.312 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:45 smithi104 podman[36447]: 2024-09-10 20:55:45.980453121 +0000 UTC m=+0.468323063 container init 8307e8746dc883a4069f4965e081475bd670124128b25b54c58bdec20851d554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T20:55:46.312 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:45 smithi104 podman[36447]: 2024-09-10 20:55:45.983474492 +0000 UTC m=+0.471344442 container start 8307e8746dc883a4069f4965e081475bd670124128b25b54c58bdec20851d554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T20:55:46.312 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mgr[36467]: -- 172.21.15.104:0/2190703397 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564719e4d0e0 con 0x564719e2b400 2024-09-10T20:55:46.312 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:46 smithi104 bash[36447]: 8307e8746dc883a4069f4965e081475bd670124128b25b54c58bdec20851d554 2024-09-10T20:55:46.312 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 20:55:46 smithi104 systemd[1]: Started Ceph mgr.y for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T20:55:46.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 -- 172.21.15.33:0/2446474920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 msgr2=0x7f35d4075580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:46.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 --2- 172.21.15.33:0/2446474920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d4075580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009ca0 tx=0x7f35c402a8c0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 -- 172.21.15.33:0/2446474920 shutdown_connections 2024-09-10T20:55:46.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 --2- 172.21.15.33:0/2446474920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d410a9d0 0x7f35d410ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 --2- 172.21.15.33:0/2446474920 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35d4075ac0 0x7f35d4075f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 --2- 172.21.15.33:0/2446474920 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d4075580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 -- 172.21.15.33:0/2446474920 >> 172.21.15.33:0/2446474920 conn(0x7f35d40fe2f0 msgr2=0x7f35d4100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 -- 172.21.15.33:0/2446474920 shutdown_connections 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.679+0000 7f35d963e640 1 -- 172.21.15.33:0/2446474920 wait complete. 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 Processor -- start 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 -- start start 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d4075ac0 0x7f35d419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d419b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35d410a9d0 0x7f35d419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4110780 con 0x7f35d4076f20 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4110600 con 0x7f35d410a9d0 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d963e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35d4110900 con 0x7f35d4075ac0 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d419b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d4075ac0 0x7f35d419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:46.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d419b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48818/0 (socket says 172.21.15.33:48818) 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d27fc640 1 -- 172.21.15.33:0/3939748390 learned_addr learned my addr 172.21.15.33:0/3939748390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d4075ac0 0x7f35d419af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:44042/0 (socket says 172.21.15.33:44042) 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.680+0000 7f35d37fe640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35d410a9d0 0x7f35d419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d27fc640 1 -- 172.21.15.33:0/3939748390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d4075ac0 msgr2=0x7f35d419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d27fc640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d4075ac0 0x7f35d419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d27fc640 1 -- 172.21.15.33:0/3939748390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35d410a9d0 msgr2=0x7f35d419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d27fc640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35d410a9d0 0x7f35d419c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d27fc640 1 -- 172.21.15.33:0/3939748390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d419ca60 con 0x7f35d4076f20 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d2ffd640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d4075ac0 0x7f35d419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d27fc640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d419b580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e9f0 tx=0x7f35c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35bbfff640 1 -- 172.21.15.33:0/3939748390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c800cd80 con 0x7f35d4076f20 2024-09-10T20:55:46.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d419cd50 con 0x7f35d4076f20 2024-09-10T20:55:46.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35bbfff640 1 -- 172.21.15.33:0/3939748390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f35c80108c0 con 0x7f35d4076f20 2024-09-10T20:55:46.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35bbfff640 1 -- 172.21.15.33:0/3939748390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c80187d0 con 0x7f35d4076f20 2024-09-10T20:55:46.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.681+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d41a3430 con 0x7f35d4076f20 2024-09-10T20:55:46.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.682+0000 7f35bbfff640 1 -- 172.21.15.33:0/3939748390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f35c8018990 con 0x7f35d4076f20 2024-09-10T20:55:46.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.682+0000 7f35bbfff640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35bc03f510 0x7f35bc0419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:46.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.682+0000 7f35bbfff640 1 -- 172.21.15.33:0/3939748390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f35c8054330 con 0x7f35d4076f20 2024-09-10T20:55:46.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.682+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d41a3600 con 0x7f35d4076f20 2024-09-10T20:55:46.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.683+0000 7f35d2ffd640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35bc03f510 0x7f35bc0419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:46.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.683+0000 7f35d2ffd640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35bc03f510 0x7f35bc0419d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009430 tx=0x7f35c4031000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:46.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.687+0000 7f35bbfff640 1 -- 172.21.15.33:0/3939748390 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f35c8010d50 con 0x7f35d4076f20 2024-09-10T20:55:46.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.878+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f35d41a37d0 con 0x7f35d4076f20 2024-09-10T20:55:46.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.881+0000 7f35bbfff640 1 -- 172.21.15.33:0/3939748390 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v18)=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f35c8018d50 con 0x7f35d4076f20 2024-09-10T20:55:46.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35bc03f510 msgr2=0x7f35bc0419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:46.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35bc03f510 0x7f35bc0419d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35c4009430 tx=0x7f35c4031000 comp rx=0 tx=0).stop 2024-09-10T20:55:46.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 msgr2=0x7f35d419b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:46.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d419b580 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f35c800e9f0 tx=0x7f35c800eeb0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 shutdown_connections 2024-09-10T20:55:46.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35bc03f510 0x7f35bc0419d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35d410a9d0 0x7f35d419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35d4076f20 0x7f35d419b580 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 --2- 172.21.15.33:0/3939748390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35d4075ac0 0x7f35d419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:46.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 >> 172.21.15.33:0/3939748390 conn(0x7f35d40fe2f0 msgr2=0x7f35d40fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:46.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 shutdown_connections 2024-09-10T20:55:46.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.884+0000 7f35d963e640 1 -- 172.21.15.33:0/3939748390 wait complete. 2024-09-10T20:55:47.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/268619382' entity='client.admin' 2024-09-10T20:55:47.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:47.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:47.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:46 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3939748390' entity='client.admin' 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/268619382' entity='client.admin' 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:46 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3939748390' entity='client.admin' 2024-09-10T20:55:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/268619382' entity='client.admin' 2024-09-10T20:55:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:47.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:46 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3939748390' entity='client.admin' 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 -- 172.21.15.33:0/3322664155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04103ba0 msgr2=0x7f2e04104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 --2- 172.21.15.33:0/3322664155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04103ba0 0x7f2e04104020 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009930 tx=0x7f2df802a420 comp rx=0 tx=0).stop 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 -- 172.21.15.33:0/3322664155 shutdown_connections 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 --2- 172.21.15.33:0/3322664155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e04104560 0x7f2e0410aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 --2- 172.21.15.33:0/3322664155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04103ba0 0x7f2e04104020 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 --2- 172.21.15.33:0/3322664155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e04102a40 0x7f2e04102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 -- 172.21.15.33:0/3322664155 >> 172.21.15.33:0/3322664155 conn(0x7f2e040fe2f0 msgr2=0x7f2e04100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 -- 172.21.15.33:0/3322664155 shutdown_connections 2024-09-10T20:55:47.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 -- 172.21.15.33:0/3322664155 wait complete. 2024-09-10T20:55:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 Processor -- start 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.889+0000 7f2e0946f640 1 -- start start 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e0946f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04102a40 0x7f2e0419afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e0946f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e04103ba0 0x7f2e0419b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e0946f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e04104560 0x7f2e0419c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e0946f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0410e580 con 0x7f2e04102a40 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e0946f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0410e400 con 0x7f2e04104560 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e0946f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0410e700 con 0x7f2e04103ba0 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04102a40 0x7f2e0419afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04102a40 0x7f2e0419afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48848/0 (socket says 172.21.15.33:48848) 2024-09-10T20:55:47.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 -- 172.21.15.33:0/2400557847 learned_addr learned my addr 172.21.15.33:0/2400557847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e037fe640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e04104560 0x7f2e0419c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e027fc640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e04103ba0 0x7f2e0419b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 -- 172.21.15.33:0/2400557847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e04103ba0 msgr2=0x7f2e0419b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e04103ba0 0x7f2e0419b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 -- 172.21.15.33:0/2400557847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e04104560 msgr2=0x7f2e0419c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e04104560 0x7f2e0419c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 -- 172.21.15.33:0/2400557847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e0419cad0 con 0x7f2e04102a40 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2e02ffd640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04102a40 0x7f2e0419afa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d770 tx=0x7f2df400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2debfff640 1 -- 172.21.15.33:0/2400557847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4017070 con 0x7f2e04102a40 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.890+0000 7f2debfff640 1 -- 172.21.15.33:0/2400557847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2df40054b0 con 0x7f2e04102a40 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.891+0000 7f2debfff640 1 -- 172.21.15.33:0/2400557847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df4004c70 con 0x7f2e04102a40 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.891+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e0419cdc0 con 0x7f2e04102a40 2024-09-10T20:55:47.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.891+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e041a34a0 con 0x7f2e04102a40 2024-09-10T20:55:47.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.892+0000 7f2debfff640 1 -- 172.21.15.33:0/2400557847 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f2df4004e10 con 0x7f2e04102a40 2024-09-10T20:55:47.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.892+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e041a3670 con 0x7f2e04102a40 2024-09-10T20:55:47.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.892+0000 7f2debfff640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2dec03f510 0x7f2dec0419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:47.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.892+0000 7f2debfff640 1 -- 172.21.15.33:0/2400557847 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2df4053730 con 0x7f2e04102a40 2024-09-10T20:55:47.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.892+0000 7f2e027fc640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2dec03f510 0x7f2dec0419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:47.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.892+0000 7f2e027fc640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2dec03f510 0x7f2dec0419d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009900 tx=0x7f2df8005720 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.897+0000 7f2debfff640 1 -- 172.21.15.33:0/2400557847 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2df400f3c0 con 0x7f2e04102a40 2024-09-10T20:55:48.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.083+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f2e041a3840 con 0x7f2e04102a40 2024-09-10T20:55:48.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.088+0000 7f2debfff640 1 -- 172.21.15.33:0/2400557847 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v20)=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7f2df40071c0 con 0x7f2e04102a40 2024-09-10T20:55:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2dec03f510 msgr2=0x7f2dec0419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2dec03f510 0x7f2dec0419d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009900 tx=0x7f2df8005720 comp rx=0 tx=0).stop 2024-09-10T20:55:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04102a40 msgr2=0x7f2e0419afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04102a40 0x7f2e0419afa0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2df400d770 tx=0x7f2df400dc30 comp rx=0 tx=0).stop 2024-09-10T20:55:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 shutdown_connections 2024-09-10T20:55:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2dec03f510 0x7f2dec0419d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2e04104560 0x7f2e0419c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e04103ba0 0x7f2e0419b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 --2- 172.21.15.33:0/2400557847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e04102a40 0x7f2e0419afa0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 >> 172.21.15.33:0/2400557847 conn(0x7f2e040fe2f0 msgr2=0x7f2e04109390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 shutdown_connections 2024-09-10T20:55:48.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:48.090+0000 7f2e0946f640 1 -- 172.21.15.33:0/2400557847 wait complete. 2024-09-10T20:55:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:48 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3756691874' entity='client.admin' 2024-09-10T20:55:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:48 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:48 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:48 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:48 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:48 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2400557847' entity='client.admin' 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 -- 172.21.15.33:0/3044209185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7794105b90 msgr2=0x7f7794107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3044209185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7794105b90 0x7f7794107fe0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7788009900 tx=0x7f778802a410 comp rx=0 tx=0).stop 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 -- 172.21.15.33:0/3044209185 shutdown_connections 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3044209185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794108720 0x7f779410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3044209185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7794105b90 0x7f7794107fe0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3044209185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f779406b680 0x7f7794105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 -- 172.21.15.33:0/3044209185 >> 172.21.15.33:0/3044209185 conn(0x7f77940fd1e0 msgr2=0x7f77940ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:48.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 -- 172.21.15.33:0/3044209185 shutdown_connections 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.891+0000 7f7798ad9640 1 -- 172.21.15.33:0/3044209185 wait complete. 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.892+0000 7f7798ad9640 1 Processor -- start 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.892+0000 7f7798ad9640 1 -- start start 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7798ad9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f779406b680 0x7f7794198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7798ad9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7794105b90 0x7f7794199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7798ad9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794108720 0x7f7794199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7798ad9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f779410e560 con 0x7f779406b680 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7798ad9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f779410e3e0 con 0x7f7794105b90 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7798ad9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f779410e6e0 con 0x7f7794108720 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f779406b680 0x7f7794198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f779406b680 0x7f7794198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48832/0 (socket says 172.21.15.33:48832) 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 -- 172.21.15.33:0/3756691874 learned_addr learned my addr 172.21.15.33:0/3756691874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:48.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7791d9b640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7794105b90 0x7f7794199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f7792d9d640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794108720 0x7f7794199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 -- 172.21.15.33:0/3756691874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794108720 msgr2=0x7f7794199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794108720 0x7f7794199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 -- 172.21.15.33:0/3756691874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7794105b90 msgr2=0x7f7794199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7794105b90 0x7f7794199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 -- 172.21.15.33:0/3756691874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779419a850 con 0x7f779406b680 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.893+0000 7f779259c640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f779406b680 0x7f7794198d20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.894+0000 7f77837fe640 1 -- 172.21.15.33:0/3756691874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778400cd80 con 0x7f779406b680 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.894+0000 7f77837fe640 1 -- 172.21.15.33:0/3756691874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f77840108c0 con 0x7f779406b680 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.894+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779419ab40 con 0x7f779406b680 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.894+0000 7f77837fe640 1 -- 172.21.15.33:0/3756691874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840187d0 con 0x7f779406b680 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.894+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77941a1220 con 0x7f779406b680 2024-09-10T20:55:48.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.895+0000 7f77837fe640 1 -- 172.21.15.33:0/3756691874 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f77840189b0 con 0x7f779406b680 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.895+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77940fecd0 con 0x7f779406b680 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.895+0000 7f77837fe640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f777c03f510 0x7f777c0419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.895+0000 7f77837fe640 1 -- 172.21.15.33:0/3756691874 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f77840541e0 con 0x7f779406b680 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.896+0000 7f7791d9b640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f777c03f510 0x7f777c0419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.896+0000 7f7791d9b640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f777c03f510 0x7f777c0419d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7788031440 tx=0x7f7788005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:46.900+0000 7f77837fe640 1 -- 172.21.15.33:0/3756691874 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f778401fe30 con 0x7f779406b680 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.095+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f7794063dc0 con 0x7f779406b680 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.098+0000 7f77837fe640 1 -- 172.21.15.33:0/3756691874 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v19)=0 v19) ==== 122+0+0 (secure 0 0 0) 0x7f77840071c0 con 0x7f779406b680 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f777c03f510 msgr2=0x7f777c0419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f777c03f510 0x7f777c0419d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7788031440 tx=0x7f7788005a20 comp rx=0 tx=0).stop 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f779406b680 msgr2=0x7f7794198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f779406b680 0x7f7794198d20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f778400e9f0 tx=0x7f778400eeb0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 shutdown_connections 2024-09-10T20:55:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f777c03f510 0x7f777c0419d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7794108720 0x7f7794199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7794105b90 0x7f7794199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 --2- 172.21.15.33:0/3756691874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f779406b680 0x7f7794198d20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 >> 172.21.15.33:0/3756691874 conn(0x7f77940fd1e0 msgr2=0x7f7794103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 shutdown_connections 2024-09-10T20:55:48.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:47.100+0000 7f7798ad9640 1 -- 172.21.15.33:0/3756691874 wait complete. 2024-09-10T20:55:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:48 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3756691874' entity='client.admin' 2024-09-10T20:55:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:48 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:48 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:48 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:48 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:48 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2400557847' entity='client.admin' 2024-09-10T20:55:48.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:48 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3756691874' entity='client.admin' 2024-09-10T20:55:48.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:48 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:48 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:48 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:48 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:48.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:48 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2400557847' entity='client.admin' 2024-09-10T20:55:48.624 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:48.625 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/372299978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 msgr2=0x7f3c08101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/372299978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08101d10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8009930 tx=0x7f3bf802a420 comp rx=0 tx=0).stop 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/372299978 shutdown_connections 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/372299978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c08103430 0x7f3c08109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/372299978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c08102a70 0x7f3c08102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/372299978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08101d10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/372299978 >> 172.21.15.33:0/372299978 conn(0x7f3c080fd1e0 msgr2=0x7f3c080ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:49.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/372299978 shutdown_connections 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.360+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/372299978 wait complete. 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 Processor -- start 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 -- start start 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08105660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c08102a70 0x7f3c08103f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c08103430 0x7f3c08104480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810d4a0 con 0x7f3c08101910 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810d320 con 0x7f3c08102a70 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0ea2d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0810d620 con 0x7f3c08103430 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08105660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08105660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48874/0 (socket says 172.21.15.33:48874) 2024-09-10T20:55:49.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c07fff640 1 -- 172.21.15.33:0/2162568857 learned_addr learned my addr 172.21.15.33:0/2162568857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c077fe640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c08102a70 0x7f3c08103f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.361+0000 7f3c0cfca640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c08103430 0x7f3c08104480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c07fff640 1 -- 172.21.15.33:0/2162568857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c08103430 msgr2=0x7f3c08104480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c07fff640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c08103430 0x7f3c08104480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c07fff640 1 -- 172.21.15.33:0/2162568857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c08102a70 msgr2=0x7f3c08103f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c07fff640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c08102a70 0x7f3c08103f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c07fff640 1 -- 172.21.15.33:0/2162568857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c08104c80 con 0x7f3c08101910 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c0cfca640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c08103430 0x7f3c08104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c07fff640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08105660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005ab0 tx=0x7f3bf8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf803f070 con 0x7f3c08101910 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3bf8031d10 con 0x7f3c08101910 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c081a2dc0 con 0x7f3c08101910 2024-09-10T20:55:49.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8035030 con 0x7f3c08101910 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.362+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c081a3200 con 0x7f3c08101910 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.363+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c081a3430 con 0x7f3c08101910 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.363+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) ==== 51480+0+0 (secure 0 0 0) 0x7f3bf803bc70 con 0x7f3c08101910 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.364+0000 7f3c057fa640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3bf003f510 0x7f3bf00419d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.364+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3bf80749e0 con 0x7f3c08101910 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.364+0000 7f3c077fe640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3bf003f510 0x7f3bf00419d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.364+0000 7f3c077fe640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3bf003f510 0x7f3bf00419d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00bbe0 tx=0x7f3bfc00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.368+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3bf8045d60 con 0x7f3c08101910 2024-09-10T20:55:49.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.382+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f3bf804c350 con 0x7f3c08101910 2024-09-10T20:55:49.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:49.527 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: Standby manager daemon z started 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:49.528 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:49.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.559+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f3c081a3600 con 0x7f3c08101910 2024-09-10T20:55:49.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.564+0000 7f3c057fa640 1 -- 172.21.15.33:0/2162568857 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f3bf805a840 con 0x7f3c08101910 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3bf003f510 msgr2=0x7f3bf00419d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3bf003f510 0x7f3bf00419d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00bbe0 tx=0x7f3bfc00b390 comp rx=0 tx=0).stop 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 msgr2=0x7f3c08105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08105660 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3bf8005ab0 tx=0x7f3bf8003c80 comp rx=0 tx=0).stop 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 shutdown_connections 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3bf003f510 0x7f3bf00419d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c08103430 0x7f3c08104480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3c08102a70 0x7f3c08103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 --2- 172.21.15.33:0/2162568857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c08101910 0x7f3c08105660 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 >> 172.21.15.33:0/2162568857 conn(0x7f3c080fd1e0 msgr2=0x7f3c080fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 shutdown_connections 2024-09-10T20:55:49.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.566+0000 7f3c0ea2d640 1 -- 172.21.15.33:0/2162568857 wait complete. 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: Standby manager daemon z started 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T20:55:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:49.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:49.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:49.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: Standby manager daemon z started 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.174:0/3951678465' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:55:49.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.927+0000 7f8f5118d640 1 -- 172.21.15.33:0/1147021429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c107f30 msgr2=0x7f8f4c10a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.927+0000 7f8f5118d640 1 --2- 172.21.15.33:0/1147021429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c107f30 0x7f8f4c10a320 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009900 tx=0x7f8f4002a410 comp rx=0 tx=0).stop 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.928+0000 7f8f5118d640 1 -- 172.21.15.33:0/1147021429 shutdown_connections 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.928+0000 7f8f5118d640 1 --2- 172.21.15.33:0/1147021429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f4c10a940 0x7f8f4c10ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.928+0000 7f8f5118d640 1 --2- 172.21.15.33:0/1147021429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c107f30 0x7f8f4c10a320 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.928+0000 7f8f5118d640 1 --2- 172.21.15.33:0/1147021429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c06b5d0 0x7f8f4c1079f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.928+0000 7f8f5118d640 1 -- 172.21.15.33:0/1147021429 >> 172.21.15.33:0/1147021429 conn(0x7f8f4c0fd1e0 msgr2=0x7f8f4c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.928+0000 7f8f5118d640 1 -- 172.21.15.33:0/1147021429 shutdown_connections 2024-09-10T20:55:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.928+0000 7f8f5118d640 1 -- 172.21.15.33:0/1147021429 wait complete. 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 Processor -- start 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 -- start start 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f4c06b5d0 0x7f8f4c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c107f30 0x7f8f4c19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c10a940 0x7f8f4c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c110730 con 0x7f8f4c10a940 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1105b0 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f5118d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f4c1108b0 con 0x7f8f4c107f30 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f4b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c10a940 0x7f8f4c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f4b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c10a940 0x7f8f4c19c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51450/0 (socket says 172.21.15.33:51450) 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f4b59e640 1 -- 172.21.15.33:0/2485361137 learned_addr learned my addr 172.21.15.33:0/2485361137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f4ad9d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f4c06b5d0 0x7f8f4c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.929+0000 7f8f4a59c640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c107f30 0x7f8f4c19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f4ad9d640 1 -- 172.21.15.33:0/2485361137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c107f30 msgr2=0x7f8f4c19b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f4ad9d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c107f30 0x7f8f4c19b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f4ad9d640 1 -- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c10a940 msgr2=0x7f8f4c19c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f4ad9d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c10a940 0x7f8f4c19c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f4ad9d640 1 -- 172.21.15.33:0/2485361137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f4c19ca30 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f4b59e640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c10a940 0x7f8f4c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f4ad9d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f4c06b5d0 0x7f8f4c19af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d6d0 tx=0x7f8f3c00db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f33fff640 1 -- 172.21.15.33:0/2485361137 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c017070 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f4c19cd20 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f33fff640 1 -- 172.21.15.33:0/2485361137 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8f3c009820 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f4c1a3400 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.930+0000 7f8f33fff640 1 -- 172.21.15.33:0/2485361137 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c004bd0 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.931+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f4c1079f0 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.932+0000 7f8f33fff640 1 -- 172.21.15.33:0/2485361137 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f8f3c005220 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.932+0000 7f8f33fff640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f3407a3b0 0x7f8f3407c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.933+0000 7f8f33fff640 1 -- 172.21.15.33:0/2485361137 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8f3c09b6b0 con 0x7f8f4c06b5d0 2024-09-10T20:55:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.933+0000 7f8f4a59c640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f3407a3b0 0x7f8f3407c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.933+0000 7f8f4a59c640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f3407a3b0 0x7f8f3407c870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f40004770 tx=0x7f8f40000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.937+0000 7f8f33fff640 1 -- 172.21.15.33:0/2485361137 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8f3c005540 con 0x7f8f4c06b5d0 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.820+0000 7f680ba25640 1 -- 172.21.15.33:0/175158837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 msgr2=0x7f6804105650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.820+0000 7f680ba25640 1 --2- 172.21.15.33:0/175158837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804105650 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6800009900 tx=0x7f680002a410 comp rx=0 tx=0).stop 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.820+0000 7f680ba25640 1 -- 172.21.15.33:0/175158837 shutdown_connections 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.820+0000 7f680ba25640 1 --2- 172.21.15.33:0/175158837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6804108720 0x7f680410abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.820+0000 7f680ba25640 1 --2- 172.21.15.33:0/175158837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6804105b90 0x7f6804107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.820+0000 7f680ba25640 1 --2- 172.21.15.33:0/175158837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804105650 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.820+0000 7f680ba25640 1 -- 172.21.15.33:0/175158837 >> 172.21.15.33:0/175158837 conn(0x7f68040fd1e0 msgr2=0x7f68040ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.821+0000 7f680ba25640 1 -- 172.21.15.33:0/175158837 shutdown_connections 2024-09-10T20:55:50.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.821+0000 7f680ba25640 1 -- 172.21.15.33:0/175158837 wait complete. 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.821+0000 7f680ba25640 1 Processor -- start 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.821+0000 7f680ba25640 1 -- start start 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f680ba25640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f680ba25640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6804105b90 0x7f6804199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f680ba25640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6804108720 0x7f6804199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f680ba25640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f680410e590 con 0x7f680406b680 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f680ba25640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f680410e410 con 0x7f6804105b90 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f680ba25640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f680410e710 con 0x7f6804108720 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804198cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51436/0 (socket says 172.21.15.33:51436) 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 -- 172.21.15.33:0/1527047678 learned_addr learned my addr 172.21.15.33:0/1527047678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f6808fc0640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6804105b90 0x7f6804199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f6809fc2640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6804108720 0x7f6804199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 -- 172.21.15.33:0/1527047678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6804108720 msgr2=0x7f6804199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6804108720 0x7f6804199f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 -- 172.21.15.33:0/1527047678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6804105b90 msgr2=0x7f6804199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6804105b90 0x7f6804199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 -- 172.21.15.33:0/1527047678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680419a7f0 con 0x7f680406b680 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f6809fc2640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6804108720 0x7f6804199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.822+0000 7f68097c1640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804198cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6800005ab0 tx=0x7f6800003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.823+0000 7f67fa7fc640 1 -- 172.21.15.33:0/1527047678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680003f070 con 0x7f680406b680 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.823+0000 7f67fa7fc640 1 -- 172.21.15.33:0/1527047678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6800031d80 con 0x7f680406b680 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.823+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680419aa80 con 0x7f680406b680 2024-09-10T20:55:50.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.823+0000 7f67fa7fc640 1 -- 172.21.15.33:0/1527047678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6800035030 con 0x7f680406b680 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.823+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68041a11c0 con 0x7f680406b680 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.824+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6804105fd0 con 0x7f680406b680 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.824+0000 7f67fa7fc640 1 -- 172.21.15.33:0/1527047678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f680003b560 con 0x7f680406b680 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.825+0000 7f67fa7fc640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f67f007a3b0 0x7f67f007c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.825+0000 7f67fa7fc640 1 -- 172.21.15.33:0/1527047678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f68000bc970 con 0x7f680406b680 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.825+0000 7f6808fc0640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f67f007a3b0 0x7f67f007c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.825+0000 7f6808fc0640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f67f007a3b0 0x7f67f007c870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67f400bbe0 tx=0x7f67f400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:50.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.829+0000 7f67fa7fc640 1 -- 172.21.15.33:0/1527047678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6800085050 con 0x7f680406b680 2024-09-10T20:55:50.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.027+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f6804063d50 con 0x7f680406b680 2024-09-10T20:55:50.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.031+0000 7f67fa7fc640 1 -- 172.21.15.33:0/1527047678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v23)=0 v23) ==== 127+0+0 (secure 0 0 0) 0x7f6800089a80 con 0x7f680406b680 2024-09-10T20:55:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f67f007a3b0 msgr2=0x7f67f007c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f67f007a3b0 0x7f67f007c870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67f400bbe0 tx=0x7f67f400b390 comp rx=0 tx=0).stop 2024-09-10T20:55:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 msgr2=0x7f6804198cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804198cc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6800005ab0 tx=0x7f6800003cd0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 shutdown_connections 2024-09-10T20:55:50.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f67f007a3b0 0x7f67f007c870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6804108720 0x7f6804199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6804105b90 0x7f6804199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 --2- 172.21.15.33:0/1527047678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f680406b680 0x7f6804198cc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 >> 172.21.15.33:0/1527047678 conn(0x7f68040fd1e0 msgr2=0x7f6804103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:50.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 shutdown_connections 2024-09-10T20:55:50.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.033+0000 7f680ba25640 1 -- 172.21.15.33:0/1527047678 wait complete. 2024-09-10T20:55:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.126+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f8f4c063d50 con 0x7f8f4c06b5d0 2024-09-10T20:55:50.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.131+0000 7f8f33fff640 1 -- 172.21.15.33:0/2485361137 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v24)=0 v24) ==== 120+0+0 (secure 0 0 0) 0x7f8f3c063cf0 con 0x7f8f4c06b5d0 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f3407a3b0 msgr2=0x7f8f3407c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f3407a3b0 0x7f8f3407c870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f40004770 tx=0x7f8f40000bc0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f4c06b5d0 msgr2=0x7f8f4c19af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f4c06b5d0 0x7f8f4c19af00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d6d0 tx=0x7f8f3c00db90 comp rx=0 tx=0).stop 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 shutdown_connections 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f3407a3b0 0x7f8f3407c870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f4c10a940 0x7f8f4c19c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f4c107f30 0x7f8f4c19b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 --2- 172.21.15.33:0/2485361137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f4c06b5d0 0x7f8f4c19af00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 >> 172.21.15.33:0/2485361137 conn(0x7f8f4c0fd1e0 msgr2=0x7f8f4c108b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 shutdown_connections 2024-09-10T20:55:50.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.133+0000 7f8f5118d640 1 -- 172.21.15.33:0/2485361137 wait complete. 2024-09-10T20:55:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:50 smithi174 ceph-mon[35635]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-10T20:55:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:50 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T20:55:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:50 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2162568857' entity='client.admin' 2024-09-10T20:55:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:50 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2597835592' entity='client.admin' 2024-09-10T20:55:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:50 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1527047678' entity='client.admin' 2024-09-10T20:55:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:50 smithi174 ceph-mon[35635]: from='client.? ' entity='client.admin' 2024-09-10T20:55:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:50 smithi033 ceph-mon[30860]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-10T20:55:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:50 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T20:55:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2162568857' entity='client.admin' 2024-09-10T20:55:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2597835592' entity='client.admin' 2024-09-10T20:55:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1527047678' entity='client.admin' 2024-09-10T20:55:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:50 smithi033 ceph-mon[30860]: from='client.? ' entity='client.admin' 2024-09-10T20:55:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:50 smithi104 ceph-mon[35325]: mgrmap e14: x(active, since 69s), standbys: z 2024-09-10T20:55:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:50 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-10T20:55:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:50 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2162568857' entity='client.admin' 2024-09-10T20:55:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:50 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2597835592' entity='client.admin' 2024-09-10T20:55:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:50 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1527047678' entity='client.admin' 2024-09-10T20:55:50.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:50 smithi104 ceph-mon[35325]: from='client.? ' entity='client.admin' 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 -- 172.21.15.33:0/3520388370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 msgr2=0x7fc788101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 --2- 172.21.15.33:0/3520388370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc788101d10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc778009cd0 tx=0x7fc77802a8e0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 -- 172.21.15.33:0/3520388370 shutdown_connections 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 --2- 172.21.15.33:0/3520388370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc788103430 0x7fc788109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 --2- 172.21.15.33:0/3520388370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc788102a70 0x7fc788102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 --2- 172.21.15.33:0/3520388370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc788101d10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 -- 172.21.15.33:0/3520388370 >> 172.21.15.33:0/3520388370 conn(0x7fc7880fd1e0 msgr2=0x7fc7880ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 -- 172.21.15.33:0/3520388370 shutdown_connections 2024-09-10T20:55:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.937+0000 7fc78ca58640 1 -- 172.21.15.33:0/3520388370 wait complete. 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 Processor -- start 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 -- start start 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc78806b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc788102a70 0x7fc78819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc788103430 0x7fc7881a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d480 con 0x7fc788101910 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d300 con 0x7fc788102a70 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78ca58640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc78810d600 con 0x7fc788103430 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc78806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc78806b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51460/0 (socket says 172.21.15.33:51460) 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc78659c640 1 -- 172.21.15.33:0/2740748748 learned_addr learned my addr 172.21.15.33:0/2740748748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc785d9b640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc788102a70 0x7fc78819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.938+0000 7fc786d9d640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc788103430 0x7fc7881a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78659c640 1 -- 172.21.15.33:0/2740748748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc788103430 msgr2=0x7fc7881a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78659c640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc788103430 0x7fc7881a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78659c640 1 -- 172.21.15.33:0/2740748748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc788102a70 msgr2=0x7fc78819da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78659c640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc788102a70 0x7fc78819da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78659c640 1 -- 172.21.15.33:0/2740748748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7881a0920 con 0x7fc788101910 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78659c640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc78806b450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc778000f80 tx=0x7fc778003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc7777fe640 1 -- 172.21.15.33:0/2740748748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc778004240 con 0x7fc788101910 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7881977f0 con 0x7fc788101910 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc7777fe640 1 -- 172.21.15.33:0/2740748748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc7780305a0 con 0x7fc788101910 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc7777fe640 1 -- 172.21.15.33:0/2740748748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7780466e0 con 0x7fc788101910 2024-09-10T20:55:50.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.939+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc788197d30 con 0x7fc788101910 2024-09-10T20:55:50.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.940+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc788197f90 con 0x7fc788101910 2024-09-10T20:55:50.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.941+0000 7fc7777fe640 1 -- 172.21.15.33:0/2740748748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7fc77804e070 con 0x7fc788101910 2024-09-10T20:55:50.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.941+0000 7fc7777fe640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc77007a3b0 0x7fc77007c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:50.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.941+0000 7fc785d9b640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc77007a3b0 0x7fc77007c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:50.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.941+0000 7fc7777fe640 1 -- 172.21.15.33:0/2740748748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc77802e030 con 0x7fc788101910 2024-09-10T20:55:50.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.942+0000 7fc785d9b640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc77007a3b0 0x7fc77007c870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00bbe0 tx=0x7fc77c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:50.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:50.946+0000 7fc7777fe640 1 -- 172.21.15.33:0/2740748748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc77804b060 con 0x7fc788101910 2024-09-10T20:55:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.418+0000 7f6abd293640 1 -- 172.21.15.33:0/3562153196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8105b90 msgr2=0x7f6ab8107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.418+0000 7f6abd293640 1 --2- 172.21.15.33:0/3562153196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8105b90 0x7f6ab8107fe0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009900 tx=0x7f6aa002a410 comp rx=0 tx=0).stop 2024-09-10T20:55:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.418+0000 7f6abd293640 1 -- 172.21.15.33:0/3562153196 shutdown_connections 2024-09-10T20:55:51.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.418+0000 7f6abd293640 1 --2- 172.21.15.33:0/3562153196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8108720 0x7f6ab810abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.418+0000 7f6abd293640 1 --2- 172.21.15.33:0/3562153196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8105b90 0x7f6ab8107fe0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.418+0000 7f6abd293640 1 --2- 172.21.15.33:0/3562153196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ab806b680 0x7f6ab8105650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.418+0000 7f6abd293640 1 -- 172.21.15.33:0/3562153196 >> 172.21.15.33:0/3562153196 conn(0x7f6ab80fd1e0 msgr2=0x7f6ab80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 -- 172.21.15.33:0/3562153196 shutdown_connections 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 -- 172.21.15.33:0/3562153196 wait complete. 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 Processor -- start 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 -- start start 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ab806b680 0x7f6ab8198cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8105b90 0x7f6ab8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8108720 0x7f6ab8199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab810e530 con 0x7f6ab8108720 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab810e3b0 con 0x7f6ab806b680 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.419+0000 7f6abd293640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ab810e6b0 con 0x7f6ab8105b90 2024-09-10T20:55:51.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8108720 0x7f6ab8199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8108720 0x7f6ab8199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48906/0 (socket says 172.21.15.33:48906) 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 -- 172.21.15.33:0/2597835592 learned_addr learned my addr 172.21.15.33:0/2597835592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab6ffd640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ab806b680 0x7f6ab8198cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab67fc640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8105b90 0x7f6ab8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 -- 172.21.15.33:0/2597835592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8105b90 msgr2=0x7f6ab8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8105b90 0x7f6ab8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 -- 172.21.15.33:0/2597835592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ab806b680 msgr2=0x7f6ab8198cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ab806b680 0x7f6ab8198cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 -- 172.21.15.33:0/2597835592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ab819a7f0 con 0x7f6ab8108720 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6ab77fe640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8108720 0x7f6ab8199f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800eb10 tx=0x7f6aa800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6aaffff640 1 -- 172.21.15.33:0/2597835592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8004b20 con 0x7f6ab8108720 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6aaffff640 1 -- 172.21.15.33:0/2597835592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6aa8004cc0 con 0x7f6ab8108720 2024-09-10T20:55:51.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6aaffff640 1 -- 172.21.15.33:0/2597835592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa8010690 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ab819aae0 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.420+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ab81a11c0 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.421+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ab8105fd0 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.422+0000 7f6aaffff640 1 -- 172.21.15.33:0/2597835592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) ==== 102337+0+0 (secure 0 0 0) 0x7f6aa8005020 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.423+0000 7f6aaffff640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6aa407a3b0 0x7f6aa407c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.423+0000 7f6aaffff640 1 -- 172.21.15.33:0/2597835592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6aa809c320 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.423+0000 7f6ab6ffd640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6aa407a3b0 0x7f6aa407c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.423+0000 7f6ab6ffd640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6aa407a3b0 0x7f6aa407c870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0005d00 tx=0x7f6ab000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.427+0000 7f6aaffff640 1 -- 172.21.15.33:0/2597835592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6aa80648e0 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.622+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f6ab8063d50 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.627+0000 7f6aaffff640 1 -- 172.21.15.33:0/2597835592 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v22)=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f6aa8069310 con 0x7f6ab8108720 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6aa407a3b0 msgr2=0x7f6aa407c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:51.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6aa407a3b0 0x7f6aa407c870 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0005d00 tx=0x7f6ab000b410 comp rx=0 tx=0).stop 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8108720 msgr2=0x7f6ab8199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8108720 0x7f6ab8199f50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6aa800eb10 tx=0x7f6aa800c2d0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 shutdown_connections 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6aa407a3b0 0x7f6aa407c870 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ab8108720 0x7f6ab8199f50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ab8105b90 0x7f6ab8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 --2- 172.21.15.33:0/2597835592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ab806b680 0x7f6ab8198cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 >> 172.21.15.33:0/2597835592 conn(0x7f6ab80fd1e0 msgr2=0x7f6ab8103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 shutdown_connections 2024-09-10T20:55:51.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:49.630+0000 7f6abd293640 1 -- 172.21.15.33:0/2597835592 wait complete. 2024-09-10T20:55:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.135+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fc788198840 con 0x7fc788101910 2024-09-10T20:55:51.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.138+0000 7fc7777fe640 1 -- 172.21.15.33:0/2740748748 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v25)=0 v25) ==== 125+0+0 (secure 0 0 0) 0x7fc778085920 con 0x7fc788101910 2024-09-10T20:55:51.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc77007a3b0 msgr2=0x7fc77007c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc77007a3b0 0x7fc77007c870 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc77c00bbe0 tx=0x7fc77c00b390 comp rx=0 tx=0).stop 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 msgr2=0x7fc78806b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc78806b450 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc778000f80 tx=0x7fc778003ba0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 shutdown_connections 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc77007a3b0 0x7fc77007c870 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc788103430 0x7fc7881a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc788102a70 0x7fc78819da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 --2- 172.21.15.33:0/2740748748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc788101910 0x7fc78806b450 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 >> 172.21.15.33:0/2740748748 conn(0x7fc7880fd1e0 msgr2=0x7fc788103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 shutdown_connections 2024-09-10T20:55:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:51.141+0000 7fc78ca58640 1 -- 172.21.15.33:0/2740748748 wait complete. 2024-09-10T20:55:51.743 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:51.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:51.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: Standby manager daemon y started 2024-09-10T20:55:51.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T20:55:51.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T20:55:51.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T20:55:51.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T20:55:51.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:51 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2740748748' entity='client.admin' 2024-09-10T20:55:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: Standby manager daemon y started 2024-09-10T20:55:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T20:55:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T20:55:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T20:55:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T20:55:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:51 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2740748748' entity='client.admin' 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: Standby manager daemon y started 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: from='mgr.? 172.21.15.104:0/1792507905' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T20:55:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:51 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2740748748' entity='client.admin' 2024-09-10T20:55:52.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 -- 172.21.15.33:0/3524402072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824101910 msgr2=0x7fd824101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:52.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 --2- 172.21.15.33:0/3524402072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824101910 0x7fd824101d10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd80c009930 tx=0x7fd80c02a420 comp rx=0 tx=0).stop 2024-09-10T20:55:52.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 -- 172.21.15.33:0/3524402072 shutdown_connections 2024-09-10T20:55:52.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 --2- 172.21.15.33:0/3524402072 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd824103430 0x7fd824109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 --2- 172.21.15.33:0/3524402072 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd824102a70 0x7fd824102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 --2- 172.21.15.33:0/3524402072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824101910 0x7fd824101d10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 -- 172.21.15.33:0/3524402072 >> 172.21.15.33:0/3524402072 conn(0x7fd8240fd1e0 msgr2=0x7fd8240ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:52.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 -- 172.21.15.33:0/3524402072 shutdown_connections 2024-09-10T20:55:52.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.540+0000 7fd82b185640 1 -- 172.21.15.33:0/3524402072 wait complete. 2024-09-10T20:55:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 Processor -- start 2024-09-10T20:55:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 -- start start 2024-09-10T20:55:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd824101910 0x7fd82419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824102a70 0x7fd82419b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd824103430 0x7fd82419c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd82410d480 con 0x7fd824102a70 2024-09-10T20:55:52.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd82410d300 con 0x7fd824101910 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd82b185640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd82410d600 con 0x7fd824103430 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd823fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824102a70 0x7fd82419b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd823fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824102a70 0x7fd82419b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51476/0 (socket says 172.21.15.33:51476) 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd823fff640 1 -- 172.21.15.33:0/2295053448 learned_addr learned my addr 172.21.15.33:0/2295053448 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.541+0000 7fd828f21640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd824101910 0x7fd82419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd829722640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd824103430 0x7fd82419c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd823fff640 1 -- 172.21.15.33:0/2295053448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd824103430 msgr2=0x7fd82419c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd823fff640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd824103430 0x7fd82419c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd823fff640 1 -- 172.21.15.33:0/2295053448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd824101910 msgr2=0x7fd82419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd823fff640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd824101910 0x7fd82419aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd823fff640 1 -- 172.21.15.33:0/2295053448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82419cab0 con 0x7fd824102a70 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd823fff640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824102a70 0x7fd82419b640 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd81400eab0 tx=0x7fd81400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:52.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd821ffb640 1 -- 172.21.15.33:0/2295053448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81400ce40 con 0x7fd824102a70 2024-09-10T20:55:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd821ffb640 1 -- 172.21.15.33:0/2295053448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd814011900 con 0x7fd824102a70 2024-09-10T20:55:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82419cda0 con 0x7fd824102a70 2024-09-10T20:55:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd821ffb640 1 -- 172.21.15.33:0/2295053448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8140109a0 con 0x7fd824102a70 2024-09-10T20:55:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.542+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8241a3480 con 0x7fd824102a70 2024-09-10T20:55:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.543+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8241a3650 con 0x7fd824102a70 2024-09-10T20:55:52.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.545+0000 7fd821ffb640 1 -- 172.21.15.33:0/2295053448 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fd81400f040 con 0x7fd824102a70 2024-09-10T20:55:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.545+0000 7fd821ffb640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8100b5250 0x7fd8100b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.546+0000 7fd828f21640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8100b5250 0x7fd8100b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.546+0000 7fd821ffb640 1 -- 172.21.15.33:0/2295053448 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd8140e4150 con 0x7fd824102a70 2024-09-10T20:55:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.546+0000 7fd828f21640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8100b5250 0x7fd8100b7710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005ab0 tx=0x7fd80c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:52.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.550+0000 7fd821ffb640 1 -- 172.21.15.33:0/2295053448 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd814016030 con 0x7fd824102a70 2024-09-10T20:55:52.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.735+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fd8241a39e0 con 0x7fd824102a70 2024-09-10T20:55:52.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.739+0000 7fd821ffb640 1 -- 172.21.15.33:0/2295053448 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v26)=0 v26) ==== 119+0+0 (secure 0 0 0) 0x7fd8140acce0 con 0x7fd824102a70 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.741+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8100b5250 msgr2=0x7fd8100b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8100b5250 0x7fd8100b7710 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd80c005ab0 tx=0x7fd80c005a20 comp rx=0 tx=0).stop 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824102a70 msgr2=0x7fd82419b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824102a70 0x7fd82419b640 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd81400eab0 tx=0x7fd81400ef70 comp rx=0 tx=0).stop 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 shutdown_connections 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8100b5250 0x7fd8100b7710 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd824103430 0x7fd82419c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd824102a70 0x7fd82419b640 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 --2- 172.21.15.33:0/2295053448 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd824101910 0x7fd82419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 >> 172.21.15.33:0/2295053448 conn(0x7fd8240fd1e0 msgr2=0x7fd82406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 shutdown_connections 2024-09-10T20:55:52.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:52.742+0000 7fd82b185640 1 -- 172.21.15.33:0/2295053448 wait complete. 2024-09-10T20:55:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:52 smithi174 ceph-mon[35635]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T20:55:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T20:55:52.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:52 smithi033 ceph-mon[30860]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T20:55:52.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T20:55:53.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:52 smithi104 ceph-mon[35325]: mgrmap e15: x(active, since 71s), standbys: z, y 2024-09-10T20:55:53.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T20:55:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.332+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/882679585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd4406b6e0 msgr2=0x7fdd441055f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:53.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.332+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/882679585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd4406b6e0 0x7fdd441055f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009ca0 tx=0x7fdd3402a8c0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.332+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/882679585 shutdown_connections 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.332+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/882679585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd44108730 0x7fdd4410abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/882679585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd44105b80 0x7fdd44107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/882679585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd4406b6e0 0x7fdd441055f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/882679585 >> 172.21.15.33:0/882679585 conn(0x7fdd440fd1e0 msgr2=0x7fdd440ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/882679585 shutdown_connections 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/882679585 wait complete. 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 Processor -- start 2024-09-10T20:55:53.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 -- start start 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd4406b6e0 0x7fdd4419af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd44105b80 0x7fdd4419b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd44108730 0x7fdd4419c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4410e540 con 0x7fdd44108730 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4410e3c0 con 0x7fdd44105b80 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.333+0000 7fdd4b4b9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4410e6c0 con 0x7fdd4406b6e0 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd44108730 0x7fdd4419c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd44108730 0x7fdd4419c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51496/0 (socket says 172.21.15.33:51496) 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 -- 172.21.15.33:0/3054297904 learned_addr learned my addr 172.21.15.33:0/3054297904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:53.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49255640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd4406b6e0 0x7fdd4419af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd48a54640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd44105b80 0x7fdd4419b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 -- 172.21.15.33:0/3054297904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd4406b6e0 msgr2=0x7fdd4419af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd4406b6e0 0x7fdd4419af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 -- 172.21.15.33:0/3054297904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd44105b80 msgr2=0x7fdd4419b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd44105b80 0x7fdd4419b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 -- 172.21.15.33:0/3054297904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4419c9d0 con 0x7fdd44108730 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd48a54640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd44105b80 0x7fdd4419b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:55:53.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd49a56640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd44108730 0x7fdd4419c130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c970 tx=0x7fdd3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd427fc640 1 -- 172.21.15.33:0/3054297904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c007bb0 con 0x7fdd44108730 2024-09-10T20:55:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd427fc640 1 -- 172.21.15.33:0/3054297904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdd3c007d50 con 0x7fdd44108730 2024-09-10T20:55:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4419ccc0 con 0x7fdd44108730 2024-09-10T20:55:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd427fc640 1 -- 172.21.15.33:0/3054297904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c004d60 con 0x7fdd44108730 2024-09-10T20:55:53.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.334+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd441a33a0 con 0x7fdd44108730 2024-09-10T20:55:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.335+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd441a35d0 con 0x7fdd44108730 2024-09-10T20:55:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.337+0000 7fdd427fc640 1 -- 172.21.15.33:0/3054297904 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fdd3c005020 con 0x7fdd44108730 2024-09-10T20:55:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.337+0000 7fdd427fc640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd2c0b52d0 0x7fdd2c0b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.338+0000 7fdd49255640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd2c0b52d0 0x7fdd2c0b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.338+0000 7fdd427fc640 1 -- 172.21.15.33:0/3054297904 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdd3c0e3ba0 con 0x7fdd44108730 2024-09-10T20:55:53.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.338+0000 7fdd49255640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd2c0b52d0 0x7fdd2c0b7790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009a60 tx=0x7fdd34005f80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:53.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.342+0000 7fdd427fc640 1 -- 172.21.15.33:0/3054297904 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdd3c014030 con 0x7fdd44108730 2024-09-10T20:55:53.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.528+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fdd441a37a0 con 0x7fdd44108730 2024-09-10T20:55:53.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.531+0000 7fdd427fc640 1 -- 172.21.15.33:0/3054297904 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v27)=0 v27) ==== 120+0+0 (secure 0 0 0) 0x7fdd3c0ac730 con 0x7fdd44108730 2024-09-10T20:55:53.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.533+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd2c0b52d0 msgr2=0x7fdd2c0b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.533+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd2c0b52d0 0x7fdd2c0b7790 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009a60 tx=0x7fdd34005f80 comp rx=0 tx=0).stop 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.533+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd44108730 msgr2=0x7fdd4419c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd44108730 0x7fdd4419c130 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00c970 tx=0x7fdd3c00ce30 comp rx=0 tx=0).stop 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 shutdown_connections 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd2c0b52d0 0x7fdd2c0b7790 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd44108730 0x7fdd4419c130 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd44105b80 0x7fdd4419b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 --2- 172.21.15.33:0/3054297904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd4406b6e0 0x7fdd4419af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 >> 172.21.15.33:0/3054297904 conn(0x7fdd440fd1e0 msgr2=0x7fdd440ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 shutdown_connections 2024-09-10T20:55:53.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:53.534+0000 7fdd4b4b9640 1 -- 172.21.15.33:0/3054297904 wait complete. 2024-09-10T20:55:53.838 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:53 smithi033 ceph-mon[30860]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:53.838 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:53 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2295053448' entity='client.admin' 2024-09-10T20:55:53.838 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:53 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3054297904' entity='client.admin' 2024-09-10T20:55:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:53 smithi104 ceph-mon[35325]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:53 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2295053448' entity='client.admin' 2024-09-10T20:55:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:53 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3054297904' entity='client.admin' 2024-09-10T20:55:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:53 smithi174 ceph-mon[35635]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:53 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2295053448' entity='client.admin' 2024-09-10T20:55:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:53 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3054297904' entity='client.admin' 2024-09-10T20:55:54.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 -- 172.21.15.33:0/4083827144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 msgr2=0x7f08b0104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 --2- 172.21.15.33:0/4083827144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0104010 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009930 tx=0x7f08a002a420 comp rx=0 tx=0).stop 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 -- 172.21.15.33:0/4083827144 shutdown_connections 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 --2- 172.21.15.33:0/4083827144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b0104550 0x7f08b010aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 --2- 172.21.15.33:0/4083827144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0104010 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 --2- 172.21.15.33:0/4083827144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102a30 0x7f08b0102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 -- 172.21.15.33:0/4083827144 >> 172.21.15.33:0/4083827144 conn(0x7f08b00fe320 msgr2=0x7f08b0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 -- 172.21.15.33:0/4083827144 shutdown_connections 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.536+0000 7f08b7b76640 1 -- 172.21.15.33:0/4083827144 wait complete. 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 Processor -- start 2024-09-10T20:55:54.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 -- start start 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102a30 0x7f08b0198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b0104550 0x7f08b0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010e5a0 con 0x7f08b0103b90 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010e420 con 0x7f08b0104550 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b7b76640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08b010e720 con 0x7f08b0102a30 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b5111640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:54.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b5111640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0199370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51518/0 (socket says 172.21.15.33:51518) 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b5111640 1 -- 172.21.15.33:0/959152161 learned_addr learned my addr 172.21.15.33:0/959152161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b5912640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102a30 0x7f08b0198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.537+0000 7f08b6113640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b0104550 0x7f08b0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b5111640 1 -- 172.21.15.33:0/959152161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102a30 msgr2=0x7f08b0198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b5111640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102a30 0x7f08b0198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b5111640 1 -- 172.21.15.33:0/959152161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b0104550 msgr2=0x7f08b0199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b5111640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b0104550 0x7f08b0199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b5111640 1 -- 172.21.15.33:0/959152161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b019a850 con 0x7f08b0103b90 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b6113640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b0104550 0x7f08b0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b5111640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0199370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009900 tx=0x7f08a0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08a6ffd640 1 -- 172.21.15.33:0/959152161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a003f070 con 0x7f08b0103b90 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b019aae0 con 0x7f08b0103b90 2024-09-10T20:55:54.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08a6ffd640 1 -- 172.21.15.33:0/959152161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f08a0031d10 con 0x7f08b0103b90 2024-09-10T20:55:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08a6ffd640 1 -- 172.21.15.33:0/959152161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a0035030 con 0x7f08b0103b90 2024-09-10T20:55:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.538+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b01a1050 con 0x7f08b0103b90 2024-09-10T20:55:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.539+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b0109d50 con 0x7f08b0103b90 2024-09-10T20:55:54.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.541+0000 7f08a6ffd640 1 -- 172.21.15.33:0/959152161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f08a003b4f0 con 0x7f08b0103b90 2024-09-10T20:55:54.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.542+0000 7f08a6ffd640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f089c0b5250 0x7f089c0b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:54.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.542+0000 7f08a6ffd640 1 -- 172.21.15.33:0/959152161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f08a0105310 con 0x7f08b0103b90 2024-09-10T20:55:54.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.542+0000 7f08b5912640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f089c0b5250 0x7f089c0b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:54.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.543+0000 7f08b5912640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f089c0b5250 0x7f089c0b7710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00abe0 tx=0x7f08ac00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:54.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.545+0000 7f08a6ffd640 1 -- 172.21.15.33:0/959152161 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f08a00cde00 con 0x7f08b0103b90 2024-09-10T20:55:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.735+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f08b002e5c0 con 0x7f08b0103b90 2024-09-10T20:55:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.739+0000 7f08a6ffd640 1 -- 172.21.15.33:0/959152161 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v28)=0 v28) ==== 121+0+0 (secure 0 0 0) 0x7f08a00cd700 con 0x7f08b0103b90 2024-09-10T20:55:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f089c0b5250 msgr2=0x7f089c0b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f089c0b5250 0x7f089c0b7710 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00abe0 tx=0x7f08ac00a2d0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 msgr2=0x7f08b0199370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0199370 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009900 tx=0x7f08a0003d50 comp rx=0 tx=0).stop 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 shutdown_connections 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f089c0b5250 0x7f089c0b7710 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08b0104550 0x7f08b0199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08b0103b90 0x7f08b0199370 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 --2- 172.21.15.33:0/959152161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08b0102a30 0x7f08b0198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.741+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 >> 172.21.15.33:0/959152161 conn(0x7f08b00fe320 msgr2=0x7f08b0106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.742+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 shutdown_connections 2024-09-10T20:55:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:54.742+0000 7f08b7b76640 1 -- 172.21.15.33:0/959152161 wait complete. 2024-09-10T20:55:55.933 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:55 smithi033 ceph-mon[30860]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:55.933 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:55 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/959152161' entity='client.admin' 2024-09-10T20:55:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:55 smithi104 ceph-mon[35325]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:55 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/959152161' entity='client.admin' 2024-09-10T20:55:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:55 smithi174 ceph-mon[35635]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:55 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/959152161' entity='client.admin' 2024-09-10T20:55:57.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:57 smithi033 ceph-mon[30860]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 -- 172.21.15.33:0/337465014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc104560 msgr2=0x7f71cc10aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 --2- 172.21.15.33:0/337465014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc104560 0x7f71cc10aee0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009cd0 tx=0x7f71c402a8e0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 -- 172.21.15.33:0/337465014 shutdown_connections 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 --2- 172.21.15.33:0/337465014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc104560 0x7f71cc10aee0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 --2- 172.21.15.33:0/337465014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71cc103ba0 0x7f71cc104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 --2- 172.21.15.33:0/337465014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71cc102a40 0x7f71cc102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 -- 172.21.15.33:0/337465014 >> 172.21.15.33:0/337465014 conn(0x7f71cc0fe2f0 msgr2=0x7f71cc100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 -- 172.21.15.33:0/337465014 shutdown_connections 2024-09-10T20:55:58.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 -- 172.21.15.33:0/337465014 wait complete. 2024-09-10T20:55:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.036+0000 7f71d4588640 1 Processor -- start 2024-09-10T20:55:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d4588640 1 -- start start 2024-09-10T20:55:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d4588640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc102a40 0x7f71cc076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d4588640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71cc103ba0 0x7f71cc075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d4588640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71cc104560 0x7f71cc075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d4588640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc10e560 con 0x7f71cc102a40 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d4588640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc10e3e0 con 0x7f71cc104560 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d4588640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f71cc10e6e0 con 0x7f71cc103ba0 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc102a40 0x7f71cc076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc102a40 0x7f71cc076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51528/0 (socket says 172.21.15.33:51528) 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 -- 172.21.15.33:0/1884236678 learned_addr learned my addr 172.21.15.33:0/1884236678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d1b23640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71cc103ba0 0x7f71cc075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2b25640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71cc104560 0x7f71cc075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 -- 172.21.15.33:0/1884236678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71cc103ba0 msgr2=0x7f71cc075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71cc103ba0 0x7f71cc075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 -- 172.21.15.33:0/1884236678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71cc104560 msgr2=0x7f71cc075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71cc104560 0x7f71cc075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 -- 172.21.15.33:0/1884236678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71cc076340 con 0x7f71cc102a40 2024-09-10T20:55:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.037+0000 7f71d2324640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc102a40 0x7f71cc076cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e9f0 tx=0x7f71c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.038+0000 7f71c37fe640 1 -- 172.21.15.33:0/1884236678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c800cd80 con 0x7f71cc102a40 2024-09-10T20:55:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.038+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71cc1a2d80 con 0x7f71cc102a40 2024-09-10T20:55:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.038+0000 7f71c37fe640 1 -- 172.21.15.33:0/1884236678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f71c80108c0 con 0x7f71cc102a40 2024-09-10T20:55:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.038+0000 7f71c37fe640 1 -- 172.21.15.33:0/1884236678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c80187d0 con 0x7f71cc102a40 2024-09-10T20:55:58.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.038+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71cc1a31e0 con 0x7f71cc102a40 2024-09-10T20:55:58.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.039+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71cc109d60 con 0x7f71cc102a40 2024-09-10T20:55:58.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.040+0000 7f71c37fe640 1 -- 172.21.15.33:0/1884236678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f71c8010430 con 0x7f71cc102a40 2024-09-10T20:55:58.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.041+0000 7f71c37fe640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71b80b5250 0x7f71b80b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.041+0000 7f71d1b23640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71b80b5250 0x7f71b80b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.041+0000 7f71c37fe640 1 -- 172.21.15.33:0/1884236678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f71c80e42d0 con 0x7f71cc102a40 2024-09-10T20:55:58.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.042+0000 7f71d1b23640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71b80b5250 0x7f71b80b7710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009bd0 tx=0x7f71bc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:58.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.045+0000 7f71c37fe640 1 -- 172.21.15.33:0/1884236678 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f71c80ac770 con 0x7f71cc102a40 2024-09-10T20:55:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:57 smithi104 ceph-mon[35325]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:58.118 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:55:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:57 smithi174 ceph-mon[35635]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:58.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.236+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f71cc063d50 con 0x7f71cc102a40 2024-09-10T20:55:58.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.242+0000 7f71c37fe640 1 -- 172.21.15.33:0/1884236678 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v29)=0 v29) ==== 121+0+0 (secure 0 0 0) 0x7f71c80b11a0 con 0x7f71cc102a40 2024-09-10T20:55:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71b80b5250 msgr2=0x7f71b80b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71b80b5250 0x7f71b80b7710 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009bd0 tx=0x7f71bc005f90 comp rx=0 tx=0).stop 2024-09-10T20:55:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc102a40 msgr2=0x7f71cc076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc102a40 0x7f71cc076cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f71c800e9f0 tx=0x7f71c800eeb0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 shutdown_connections 2024-09-10T20:55:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71b80b5250 0x7f71b80b7710 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71cc104560 0x7f71cc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71cc103ba0 0x7f71cc075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 --2- 172.21.15.33:0/1884236678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71cc102a40 0x7f71cc076cd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 >> 172.21.15.33:0/1884236678 conn(0x7f71cc0fe2f0 msgr2=0x7f71cc106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 shutdown_connections 2024-09-10T20:55:58.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.244+0000 7f71d4588640 1 -- 172.21.15.33:0/1884236678 wait complete. 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 -- 172.21.15.33:0/1541764273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 msgr2=0x7fb384101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 --2- 172.21.15.33:0/1541764273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb384101d50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb380009930 tx=0x7fb38002a420 comp rx=0 tx=0).stop 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 -- 172.21.15.33:0/1541764273 shutdown_connections 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 --2- 172.21.15.33:0/1541764273 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb384103470 0x7fb384109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 --2- 172.21.15.33:0/1541764273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb384102ab0 0x7fb384102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 --2- 172.21.15.33:0/1541764273 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb384101d50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 -- 172.21.15.33:0/1541764273 >> 172.21.15.33:0/1541764273 conn(0x7fb3840fd200 msgr2=0x7fb3840ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 -- 172.21.15.33:0/1541764273 shutdown_connections 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.723+0000 7fb38c06d640 1 -- 172.21.15.33:0/1541764273 wait complete. 2024-09-10T20:55:58.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 Processor -- start 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 -- start start 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb3841056a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb384102ab0 0x7fb384103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb384103470 0x7fb3841044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d4f0 con 0x7fb384101950 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d370 con 0x7fb384102ab0 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38c06d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb38410d670 con 0x7fb384103470 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb3841056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb3841056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51546/0 (socket says 172.21.15.33:51546) 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 -- 172.21.15.33:0/3635481407 learned_addr learned my addr 172.21.15.33:0/3635481407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389608640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb384102ab0 0x7fb384103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb38a60a640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb384103470 0x7fb3841044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 -- 172.21.15.33:0/3635481407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb384103470 msgr2=0x7fb3841044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb384103470 0x7fb3841044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 -- 172.21.15.33:0/3635481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb384102ab0 msgr2=0x7fb384103f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb384102ab0 0x7fb384103f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 -- 172.21.15.33:0/3635481407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb384104cc0 con 0x7fb384101950 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389608640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb384102ab0 0x7fb384103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.724+0000 7fb389e09640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb3841056a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb380009a50 tx=0x7fb380032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.725+0000 7fb37affd640 1 -- 172.21.15.33:0/3635481407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38003f030 con 0x7fb384101950 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.725+0000 7fb37affd640 1 -- 172.21.15.33:0/3635481407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb3800044b0 con 0x7fb384101950 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.725+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3841a2e10 con 0x7fb384101950 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.725+0000 7fb37affd640 1 -- 172.21.15.33:0/3635481407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb380035070 con 0x7fb384101950 2024-09-10T20:55:58.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.725+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3841a3290 con 0x7fb384101950 2024-09-10T20:55:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.726+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3841a34c0 con 0x7fb384101950 2024-09-10T20:55:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.727+0000 7fb37affd640 1 -- 172.21.15.33:0/3635481407 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fb38003baf0 con 0x7fb384101950 2024-09-10T20:55:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.728+0000 7fb37affd640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3700b52d0 0x7fb3700b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:55:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.728+0000 7fb389608640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3700b52d0 0x7fb3700b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:55:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.728+0000 7fb37affd640 1 -- 172.21.15.33:0/3635481407 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb380104f30 con 0x7fb384101950 2024-09-10T20:55:58.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.728+0000 7fb389608640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3700b52d0 0x7fb3700b7790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb374009930 tx=0x7fb374008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:55:58.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.731+0000 7fb37affd640 1 -- 172.21.15.33:0/3635481407 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb3800cdac0 con 0x7fb384101950 2024-09-10T20:55:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.921+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fb3841a3690 con 0x7fb384101950 2024-09-10T20:55:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.927+0000 7fb37affd640 1 -- 172.21.15.33:0/3635481407 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v30)=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7fb3800cd3c0 con 0x7fb384101950 2024-09-10T20:55:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3700b52d0 msgr2=0x7fb3700b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3700b52d0 0x7fb3700b7790 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb374009930 tx=0x7fb374008040 comp rx=0 tx=0).stop 2024-09-10T20:55:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 msgr2=0x7fb3841056a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:55:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb3841056a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb380009a50 tx=0x7fb380032f90 comp rx=0 tx=0).stop 2024-09-10T20:55:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 shutdown_connections 2024-09-10T20:55:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3700b52d0 0x7fb3700b7790 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb384103470 0x7fb3841044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb384102ab0 0x7fb384103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 --2- 172.21.15.33:0/3635481407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb384101950 0x7fb3841056a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:55:58.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 >> 172.21.15.33:0/3635481407 conn(0x7fb3840fd200 msgr2=0x7fb3840fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:55:58.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 shutdown_connections 2024-09-10T20:55:58.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:55:58.929+0000 7fb38c06d640 1 -- 172.21.15.33:0/3635481407 wait complete. 2024-09-10T20:55:59.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:59 smithi104 ceph-mon[35325]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:59.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1884236678' entity='client.admin' 2024-09-10T20:55:59.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:55:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3635481407' entity='client.admin' 2024-09-10T20:55:59.581 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:59 smithi033 ceph-mon[30860]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:59.581 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1884236678' entity='client.admin' 2024-09-10T20:55:59.581 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:55:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3635481407' entity='client.admin' 2024-09-10T20:55:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:59 smithi174 ceph-mon[35635]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:55:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1884236678' entity='client.admin' 2024-09-10T20:55:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:55:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3635481407' entity='client.admin' 2024-09-10T20:56:00.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 -- 172.21.15.33:0/2507483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c104560 msgr2=0x7f464c10aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:00.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 --2- 172.21.15.33:0/2507483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c104560 0x7f464c10aee0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4644009cd0 tx=0x7f464402a8e0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 -- 172.21.15.33:0/2507483005 shutdown_connections 2024-09-10T20:56:00.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 --2- 172.21.15.33:0/2507483005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c104560 0x7f464c10aee0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 --2- 172.21.15.33:0/2507483005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464c103ba0 0x7f464c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 --2- 172.21.15.33:0/2507483005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c102a40 0x7f464c102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 -- 172.21.15.33:0/2507483005 >> 172.21.15.33:0/2507483005 conn(0x7f464c0fe2f0 msgr2=0x7f464c100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:56:00.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 -- 172.21.15.33:0/2507483005 shutdown_connections 2024-09-10T20:56:00.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.326+0000 7f46516d8640 1 -- 172.21.15.33:0/2507483005 wait complete. 2024-09-10T20:56:00.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 Processor -- start 2024-09-10T20:56:00.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 -- start start 2024-09-10T20:56:00.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c102a40 0x7f464c19af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:00.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c103ba0 0x7f464c19b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:00.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464c104560 0x7f464c19c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f464c10e5e0 con 0x7f464c103ba0 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f464c10e460 con 0x7f464c104560 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f46516d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f464c10e760 con 0x7f464c102a40 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f464a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c103ba0 0x7f464c19b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f464a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c103ba0 0x7f464c19b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48630/0 (socket says 172.21.15.33:48630) 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f464a7fc640 1 -- 172.21.15.33:0/3965870884 learned_addr learned my addr 172.21.15.33:0/3965870884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f464affd640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c102a40 0x7f464c19af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.327+0000 7f464b7fe640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464c104560 0x7f464c19c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f464a7fc640 1 -- 172.21.15.33:0/3965870884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c102a40 msgr2=0x7f464c19af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f464a7fc640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c102a40 0x7f464c19af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f464a7fc640 1 -- 172.21.15.33:0/3965870884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464c104560 msgr2=0x7f464c19c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f464a7fc640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464c104560 0x7f464c19c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f464a7fc640 1 -- 172.21.15.33:0/3965870884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c19cab0 con 0x7f464c103ba0 2024-09-10T20:56:00.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f464a7fc640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c103ba0 0x7f464c19b5d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f464000b4a0 tx=0x7f464000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:00.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640014070 con 0x7f464c103ba0 2024-09-10T20:56:00.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c19cda0 con 0x7f464c103ba0 2024-09-10T20:56:00.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4640010aa0 con 0x7f464c103ba0 2024-09-10T20:56:00.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640017b80 con 0x7f464c103ba0 2024-09-10T20:56:00.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.328+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c1a3480 con 0x7f464c103ba0 2024-09-10T20:56:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.329+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464c1a36b0 con 0x7f464c103ba0 2024-09-10T20:56:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.330+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f4640010610 con 0x7f464c103ba0 2024-09-10T20:56:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.331+0000 7f4633fff640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46340b5250 0x7f46340b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.331+0000 7f464affd640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46340b5250 0x7f46340b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.331+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f46400e3840 con 0x7f464c103ba0 2024-09-10T20:56:00.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.332+0000 7f464affd640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46340b5250 0x7f46340b7710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f463c00bbe0 tx=0x7f463c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:00.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.335+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f46400ac330 con 0x7f464c103ba0 2024-09-10T20:56:00.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.521+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f464c1a3da0 con 0x7f464c103ba0 2024-09-10T20:56:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.525+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v31)=0 v31) ==== 123+0+0 (secure 0 0 0) 0x7f46400abc30 con 0x7f464c103ba0 2024-09-10T20:56:00.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.526+0000 7f4633fff640 1 -- 172.21.15.33:0/3965870884 <== mon.0 v2:172.21.15.33:3300/0 8 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4640017360 con 0x7f464c103ba0 2024-09-10T20:56:00.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46340b5250 msgr2=0x7f46340b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:00.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46340b5250 0x7f46340b7710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f463c00bbe0 tx=0x7f463c00b390 comp rx=0 tx=0).stop 2024-09-10T20:56:00.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c103ba0 msgr2=0x7f464c19b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c103ba0 0x7f464c19b5d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f464000b4a0 tx=0x7f464000b960 comp rx=0 tx=0).stop 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 shutdown_connections 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46340b5250 0x7f46340b7710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f464c104560 0x7f464c19c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f464c103ba0 0x7f464c19b5d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.527+0000 7f46516d8640 1 --2- 172.21.15.33:0/3965870884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f464c102a40 0x7f464c19af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.528+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 >> 172.21.15.33:0/3965870884 conn(0x7f464c0fe2f0 msgr2=0x7f464c1092c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.528+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 shutdown_connections 2024-09-10T20:56:00.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:00.528+0000 7f46516d8640 1 -- 172.21.15.33:0/3965870884 wait complete. 2024-09-10T20:56:01.142 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.142 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.143 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.143 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.143 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.143 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.143 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.143 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.143 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.144 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'a07b33bc-6fb6-11ef-bcea-c7b262605968', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi033') 2024-09-10T20:56:01.145 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T20:56:01.145 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T20:56:01.145 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T20:56:01.162 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T20:56:01.163 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-10 20:54:55.240899366 +0000 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-10 20:50:29.060654123 +0000 2024-09-10T20:56:01.219 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-10 20:50:29.060654123 +0000 2024-09-10T20:56:01.220 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-10 20:50:29.060654123 +0000 2024-09-10T20:56:01.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T20:56:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-10T20:56:01.287 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-10T20:56:01.287 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000219919 s, 2.3 MB/s 2024-09-10T20:56:01.288 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T20:56:01.345 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-09-10T20:56:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T20:56:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:01.401 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T20:56:01.401 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:01.402 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:01.402 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-10 20:54:55.270898892 +0000 2024-09-10T20:56:01.402 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-10 20:50:29.397648785 +0000 2024-09-10T20:56:01.402 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-10 20:50:29.397648785 +0000 2024-09-10T20:56:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-10 20:50:29.397648785 +0000 2024-09-10T20:56:01.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T20:56:01.466 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-10T20:56:01.466 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-10T20:56:01.466 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000241951 s, 2.1 MB/s 2024-09-10T20:56:01.467 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T20:56:01.524 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-10 20:54:55.301898402 +0000 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-10 20:50:29.724643606 +0000 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-10 20:50:29.724643606 +0000 2024-09-10T20:56:01.581 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-10 20:50:29.724643606 +0000 2024-09-10T20:56:01.582 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T20:56:01.644 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-10T20:56:01.644 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-10T20:56:01.644 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000218182 s, 2.3 MB/s 2024-09-10T20:56:01.645 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T20:56:01.702 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-10 20:54:55.331897928 +0000 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-10 20:50:30.061638267 +0000 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-10 20:50:30.061638267 +0000 2024-09-10T20:56:01.757 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-10 20:50:30.061638267 +0000 2024-09-10T20:56:01.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T20:56:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:01 smithi104 ceph-mon[35325]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:01 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3965870884' entity='client.admin' 2024-09-10T20:56:01.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:01 smithi033 ceph-mon[30860]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:01.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:01 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3965870884' entity='client.admin' 2024-09-10T20:56:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-09-10T20:56:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-09-10T20:56:01.822 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000209219 s, 2.4 MB/s 2024-09-10T20:56:01.823 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T20:56:01.880 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-09-10T20:56:01.880 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T20:56:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:01 smithi174 ceph-mon[35635]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:01 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3965870884' entity='client.admin' 2024-09-10T20:56:01.901 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T20:56:01.901 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-09-10T20:56:01.960 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-10 20:55:45.353877915 +0000 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-10 20:50:39.088195797 +0000 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-10 20:50:39.088195797 +0000 2024-09-10T20:56:01.961 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-10 20:50:39.088195797 +0000 2024-09-10T20:56:01.961 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T20:56:02.030 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-10T20:56:02.031 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-10T20:56:02.031 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000287288 s, 1.8 MB/s 2024-09-10T20:56:02.032 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T20:56:02.092 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-09-10T20:56:02.148 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T20:56:02.148 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:02.148 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T20:56:02.149 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:02.149 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:02.149 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-10 20:55:45.353877915 +0000 2024-09-10T20:56:02.149 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-10 20:50:39.399192202 +0000 2024-09-10T20:56:02.149 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-10 20:50:39.399192202 +0000 2024-09-10T20:56:02.149 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-10 20:50:39.399192202 +0000 2024-09-10T20:56:02.149 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T20:56:02.214 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-10T20:56:02.214 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-10T20:56:02.214 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000207834 s, 2.5 MB/s 2024-09-10T20:56:02.215 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T20:56:02.273 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-10 20:55:45.353877915 +0000 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-10 20:50:39.717188526 +0000 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-10 20:50:39.717188526 +0000 2024-09-10T20:56:02.329 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-10 20:50:39.717188526 +0000 2024-09-10T20:56:02.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T20:56:02.393 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-10T20:56:02.393 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-10T20:56:02.393 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000303891 s, 1.7 MB/s 2024-09-10T20:56:02.394 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T20:56:02.451 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-09-10 20:55:45.354877903 +0000 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-09-10 20:50:40.037184827 +0000 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-09-10 20:50:40.037184827 +0000 2024-09-10T20:56:02.507 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-09-10 20:50:40.037184827 +0000 2024-09-10T20:56:02.507 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T20:56:02.571 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-09-10T20:56:02.571 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-09-10T20:56:02.571 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000202669 s, 2.5 MB/s 2024-09-10T20:56:02.572 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T20:56:02.629 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-09-10T20:56:02.681 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T20:56:02.702 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T20:56:02.703 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-09-10T20:56:02.761 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T20:56:02.761 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:02.761 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-09-10T20:56:02.761 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:02.762 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:02.762 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 20:55:42.908624024 +0000 2024-09-10T20:56:02.762 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 20:50:37.746445032 +0000 2024-09-10T20:56:02.762 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 20:50:37.746445032 +0000 2024-09-10T20:56:02.762 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 20:50:37.746445032 +0000 2024-09-10T20:56:02.762 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T20:56:02.833 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T20:56:02.833 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T20:56:02.833 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000287346 s, 1.8 MB/s 2024-09-10T20:56:02.834 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T20:56:02.893 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-09-10T20:56:02.950 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 20:55:42.909624012 +0000 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 20:50:38.079441170 +0000 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 20:50:38.079441170 +0000 2024-09-10T20:56:02.951 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 20:50:38.079441170 +0000 2024-09-10T20:56:02.951 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T20:56:03.017 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T20:56:03.017 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T20:56:03.017 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000225347 s, 2.3 MB/s 2024-09-10T20:56:03.018 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T20:56:03.076 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 20:55:42.909624012 +0000 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 20:50:38.408437354 +0000 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 20:50:38.408437354 +0000 2024-09-10T20:56:03.134 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 20:50:38.408437354 +0000 2024-09-10T20:56:03.134 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T20:56:03.200 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T20:56:03.200 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T20:56:03.200 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000208833 s, 2.5 MB/s 2024-09-10T20:56:03.201 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T20:56:03.259 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-09-10T20:56:03.316 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T20:56:03.316 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T20:56:03.316 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-09-10T20:56:03.316 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T20:56:03.317 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T20:56:03.317 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-09-10 20:55:42.909624012 +0000 2024-09-10T20:56:03.317 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-09-10 20:50:38.751433376 +0000 2024-09-10T20:56:03.317 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-09-10 20:50:38.751433376 +0000 2024-09-10T20:56:03.317 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-09-10 20:50:38.751433376 +0000 2024-09-10T20:56:03.317 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T20:56:03.381 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-09-10T20:56:03.381 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-09-10T20:56:03.382 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000201853 s, 2.5 MB/s 2024-09-10T20:56:03.383 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T20:56:03.440 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-09-10T20:56:03.440 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T20:56:03.704 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:56:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:03 smithi104 ceph-mon[35325]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:03.825 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:03 smithi033 ceph-mon[30860]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:03.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:03 smithi174 ceph-mon[35635]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:05 smithi104 ceph-mon[35325]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:05.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:05 smithi033 ceph-mon[30860]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:05 smithi174 ceph-mon[35635]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:06.088 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:56:06.117 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-09-10T20:56:06.378 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:56:06.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 -- 172.21.15.33:0/2004957748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138102a70 msgr2=0x7ff138102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/2004957748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138102a70 0x7ff138102ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009930 tx=0x7ff12c02a420 comp rx=0 tx=0).stop 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 -- 172.21.15.33:0/2004957748 shutdown_connections 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/2004957748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff138103430 0x7ff138109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/2004957748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138102a70 0x7ff138102ef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/2004957748 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff138101910 0x7ff138101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 -- 172.21.15.33:0/2004957748 >> 172.21.15.33:0/2004957748 conn(0x7ff1380fd1e0 msgr2=0x7ff1380ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 -- 172.21.15.33:0/2004957748 shutdown_connections 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 -- 172.21.15.33:0/2004957748 wait complete. 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.855+0000 7ff13e0bf640 1 Processor -- start 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff13e0bf640 1 -- start start 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff13e0bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff138101910 0x7ff138198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:06.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff13e0bf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff138102a70 0x7ff1381994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff13e0bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138103430 0x7ff13819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff13e0bf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810e6c0 con 0x7ff138103430 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff13e0bf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810e540 con 0x7ff138102a70 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff13e0bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff13810e840 con 0x7ff138101910 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138103430 0x7ff13819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138103430 0x7ff13819a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48636/0 (socket says 172.21.15.33:48636) 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 -- 172.21.15.33:0/4028101622 learned_addr learned my addr 172.21.15.33:0/4028101622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff1377fe640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff138101910 0x7ff138198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff136ffd640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff138102a70 0x7ff1381994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 -- 172.21.15.33:0/4028101622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff138101910 msgr2=0x7ff138198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff138101910 0x7ff138198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 -- 172.21.15.33:0/4028101622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff138102a70 msgr2=0x7ff1381994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff138102a70 0x7ff1381994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 -- 172.21.15.33:0/4028101622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13819a9b0 con 0x7ff138103430 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.856+0000 7ff137fff640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138103430 0x7ff13819a110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff13000eaa0 tx=0x7ff13000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.857+0000 7ff134ff9640 1 -- 172.21.15.33:0/4028101622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13000cdb0 con 0x7ff138103430 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.857+0000 7ff134ff9640 1 -- 172.21.15.33:0/4028101622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff130005020 con 0x7ff138103430 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.857+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13819aca0 con 0x7ff138103430 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.857+0000 7ff134ff9640 1 -- 172.21.15.33:0/4028101622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1300116d0 con 0x7ff138103430 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.857+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1381a1380 con 0x7ff138103430 2024-09-10T20:56:06.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.858+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff138108c90 con 0x7ff138103430 2024-09-10T20:56:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.859+0000 7ff134ff9640 1 -- 172.21.15.33:0/4028101622 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7ff13000f040 con 0x7ff138103430 2024-09-10T20:56:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.860+0000 7ff134ff9640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1200b52d0 0x7ff1200b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.860+0000 7ff1377fe640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1200b52d0 0x7ff1200b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.861+0000 7ff134ff9640 1 -- 172.21.15.33:0/4028101622 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff1300e3dd0 con 0x7ff138103430 2024-09-10T20:56:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.861+0000 7ff1377fe640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1200b52d0 0x7ff1200b7790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff128005fd0 tx=0x7ff128005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:06.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:06.865+0000 7ff134ff9640 1 -- 172.21.15.33:0/4028101622 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff130016030 con 0x7ff138103430 2024-09-10T20:56:07.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:07.079+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff138063590 con 0x7ff1200b52d0 2024-09-10T20:56:07.724 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:07 smithi033 ceph-mon[30860]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:07.725 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:07.725 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:07.725 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:07 smithi104 ceph-mon[35325]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:07 smithi174 ceph-mon[35635]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:08.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:08 smithi033 ceph-mon[30860]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:08.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:08 smithi104 ceph-mon[35325]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:08.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:08 smithi174 ceph-mon[35635]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:09 smithi104 ceph-mon[35325]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:09 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2077492806' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "343a51e3-d26a-4f11-9714-d6464a31c878"} : dispatch 2024-09-10T20:56:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:09 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2077492806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "343a51e3-d26a-4f11-9714-d6464a31c878"}]': finished 2024-09-10T20:56:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:09 smithi104 ceph-mon[35325]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T20:56:09.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:09 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:09 smithi174 ceph-mon[35635]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:09 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2077492806' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "343a51e3-d26a-4f11-9714-d6464a31c878"} : dispatch 2024-09-10T20:56:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:09 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2077492806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "343a51e3-d26a-4f11-9714-d6464a31c878"}]': finished 2024-09-10T20:56:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:09 smithi174 ceph-mon[35635]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T20:56:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:09 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:09.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:09 smithi033 ceph-mon[30860]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:09.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:09 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2077492806' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "343a51e3-d26a-4f11-9714-d6464a31c878"} : dispatch 2024-09-10T20:56:09.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:09 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2077492806' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "343a51e3-d26a-4f11-9714-d6464a31c878"}]': finished 2024-09-10T20:56:09.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:09 smithi033 ceph-mon[30860]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T20:56:09.907 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:10.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:10 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3369209028' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:56:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:10 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3369209028' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:56:10.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:10 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3369209028' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:56:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:11 smithi104 ceph-mon[35325]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:11 smithi174 ceph-mon[35635]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:11 smithi033 ceph-mon[30860]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:13 smithi104 ceph-mon[35325]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:13 smithi174 ceph-mon[35635]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:13 smithi033 ceph-mon[30860]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:15 smithi104 ceph-mon[35325]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:15.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:15 smithi033 ceph-mon[30860]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:15 smithi174 ceph-mon[35635]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:17.673 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:17 smithi033 ceph-mon[30860]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:17 smithi104 ceph-mon[35325]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:17 smithi174 ceph-mon[35635]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:18.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:18 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T20:56:18.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:18 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:18.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:18 smithi033 ceph-mon[30860]: Deploying daemon osd.0 on smithi033 2024-09-10T20:56:18.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:18 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T20:56:18.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:18 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:18.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:18 smithi174 ceph-mon[35635]: Deploying daemon osd.0 on smithi033 2024-09-10T20:56:19.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:18 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T20:56:19.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:18 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:19.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:18 smithi104 ceph-mon[35325]: Deploying daemon osd.0 on smithi033 2024-09-10T20:56:19.582 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:19 smithi033 ceph-mon[30860]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:19 smithi174 ceph-mon[35635]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:20.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:19 smithi104 ceph-mon[35325]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:21 smithi174 ceph-mon[35635]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:21 smithi033 ceph-mon[30860]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:21 smithi104 ceph-mon[35325]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:23.829 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:23 smithi033 ceph-mon[30860]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:23 smithi174 ceph-mon[35635]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:23 smithi104 ceph-mon[35325]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:24.748 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:24.749 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:24.749 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:24.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:24.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:24.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:25.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:24 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:25.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:24 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:25.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:24 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:25 smithi033 ceph-mon[30860]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:25 smithi174 ceph-mon[35635]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:25.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:25 smithi104 ceph-mon[35325]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.661+0000 7ff134ff9640 1 -- 172.21.15.33:0/4028101622 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff138063590 con 0x7ff1200b52d0 2024-09-10T20:56:27.666 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-09-10T20:56:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1200b52d0 msgr2=0x7ff1200b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1200b52d0 0x7ff1200b7790 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff128005fd0 tx=0x7ff128005850 comp rx=0 tx=0).stop 2024-09-10T20:56:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138103430 msgr2=0x7ff13819a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138103430 0x7ff13819a110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff13000eaa0 tx=0x7ff13000ef60 comp rx=0 tx=0).stop 2024-09-10T20:56:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 shutdown_connections 2024-09-10T20:56:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1200b52d0 0x7ff1200b7790 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff138103430 0x7ff13819a110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff138102a70 0x7ff1381994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 --2- 172.21.15.33:0/4028101622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff138101910 0x7ff138198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.663+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 >> 172.21.15.33:0/4028101622 conn(0x7ff1380fd1e0 msgr2=0x7ff138105e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:56:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.664+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 shutdown_connections 2024-09-10T20:56:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:56:27.664+0000 7ff13e0bf640 1 -- 172.21.15.33:0/4028101622 wait complete. 2024-09-10T20:56:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:27.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:27.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:27.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:27.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:27.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:27.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:27.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:28.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:28.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:28.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:28.311 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T20:56:28.314 INFO:tasks.cephadm:Deploying osd.1 on smithi104 with /dev/vg_nvme/lv_4... 2024-09-10T20:56:28.314 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T20:56:28.571 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T20:56:29.732 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:29 smithi033 ceph-mon[30860]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:29.861 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:29 smithi104 ceph-mon[35325]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:29 smithi174 ceph-mon[35635]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:30.427 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:56:30 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0[47340]: 2024-09-10T20:56:30.063+0000 7f87e572e640 -1 osd.0 0 log_to_monitors true 2024-09-10T20:56:30.751 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:30 smithi033 ceph-mon[30860]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T20:56:30.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:30 smithi104 ceph-mon[35325]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T20:56:30.996 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:56:31.023 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-09-10T20:56:31.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:30 smithi174 ceph-mon[35635]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T20:56:31.273 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T20:56:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 -- 172.21.15.104:0/1772714637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988103470 msgr2=0x7f6988109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 --2- 172.21.15.104:0/1772714637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988103470 0x7f6988109df0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6980009cd0 tx=0x7f698002a8e0 comp rx=0 tx=0).stop 2024-09-10T20:56:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 -- 172.21.15.104:0/1772714637 shutdown_connections 2024-09-10T20:56:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 --2- 172.21.15.104:0/1772714637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988103470 0x7f6988109df0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 --2- 172.21.15.104:0/1772714637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6988102ab0 0x7f6988102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 --2- 172.21.15.104:0/1772714637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6988101950 0x7f6988101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:31.780 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 -- 172.21.15.104:0/1772714637 >> 172.21.15.104:0/1772714637 conn(0x7f69880fd200 msgr2=0x7f69880ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 -- 172.21.15.104:0/1772714637 shutdown_connections 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 -- 172.21.15.104:0/1772714637 wait complete. 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 Processor -- start 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 -- start start 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6988101950 0x7f6988198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.751+0000 7f698e105640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988102ab0 0x7f69881994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f698e105640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6988103470 0x7f698819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f698e105640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f698810d630 con 0x7f6988103470 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f698e105640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f698810d4b0 con 0x7f6988102ab0 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f698e105640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f698810d7b0 con 0x7f6988101950 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988102ab0 0x7f69881994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:31.781 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988102ab0 0x7f69881994a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:38776/0 (socket says 172.21.15.104:38776) 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 -- 172.21.15.104:0/3062644061 learned_addr learned my addr 172.21.15.104:0/3062644061 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f69877fe640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6988101950 0x7f6988198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6987fff640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6988103470 0x7f698819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 -- 172.21.15.104:0/3062644061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6988101950 msgr2=0x7f6988198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6988101950 0x7f6988198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 -- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6988103470 msgr2=0x7f698819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6988103470 0x7f698819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 -- 172.21.15.104:0/3062644061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f698819a980 con 0x7f6988102ab0 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f69877fe640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6988101950 0x7f6988198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6986ffd640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988102ab0 0x7f69881994a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b4a0 tx=0x7f697c00b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6984ff9640 1 -- 172.21.15.104:0/3062644061 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c014070 con 0x7f6988102ab0 2024-09-10T20:56:31.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f698819ac70 con 0x7f6988102ab0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6984ff9640 1 -- 172.21.15.104:0/3062644061 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f697c010ac0 con 0x7f6988102ab0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f6984ff9640 1 -- 172.21.15.104:0/3062644061 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f697c017c30 con 0x7f6988102ab0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.752+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a1350 con 0x7f6988102ab0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.753+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69881a15e0 con 0x7f6988102ab0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.754+0000 7f6984ff9640 1 -- 172.21.15.104:0/3062644061 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f697c010610 con 0x7f6988102ab0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.755+0000 7f6984ff9640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69700b5250 0x7f69700b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.755+0000 7f6984ff9640 1 -- 172.21.15.104:0/3062644061 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f697c0e3970 con 0x7f6988102ab0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.755+0000 7f69877fe640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69700b5250 0x7f69700b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.756+0000 7f69877fe640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69700b5250 0x7f69700b7710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f697800abe0 tx=0x7f697800a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:31.783 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.759+0000 7f6984ff9640 1 -- 172.21.15.104:0/3062644061 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f697c0ac3c0 con 0x7f6988102ab0 2024-09-10T20:56:31.949 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:31.945+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f698806bba0 con 0x7f69700b5250 2024-09-10T20:56:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T20:56:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T20:56:31.949 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:31.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T20:56:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T20:56:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:56:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:32.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:56:32.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:32.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:32.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.177 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0[47340]: 2024-09-10T20:56:31.682+0000 7f87e16b1640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T20:56:32.177 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0[47340]: 2024-09-10T20:56:31.685+0000 7f87dd4d8640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:56:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:32.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi033 2024-09-10T20:56:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi033 to 17092M 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi033 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi033 to 17092M 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi033 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi033 to 17092M 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: from='osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:33 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:56:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:33 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:56:33.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:33 smithi104 ceph-mon[35325]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:33.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:33 smithi104 ceph-mon[35325]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:33.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:33 smithi104 ceph-mon[35325]: osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] boot 2024-09-10T20:56:33.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:33 smithi104 ceph-mon[35325]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T20:56:33.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:33 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:56:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:33 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:56:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:33 smithi174 ceph-mon[35635]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:33 smithi174 ceph-mon[35635]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:33 smithi174 ceph-mon[35635]: osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] boot 2024-09-10T20:56:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:33 smithi174 ceph-mon[35635]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T20:56:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:33 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:56:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:33 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:56:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:33 smithi033 ceph-mon[30860]: from='client.24248 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:33 smithi033 ceph-mon[30860]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T20:56:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:33 smithi033 ceph-mon[30860]: osd.0 [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] boot 2024-09-10T20:56:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:33 smithi033 ceph-mon[30860]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T20:56:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T20:56:34.807 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:34 smithi104 ceph-mon[35325]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T20:56:34.807 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:34 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/2294789424' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"} : dispatch 2024-09-10T20:56:34.807 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:34 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"} : dispatch 2024-09-10T20:56:34.807 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:34 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"}]': finished 2024-09-10T20:56:34.807 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:34 smithi104 ceph-mon[35325]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T20:56:34.807 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:34 smithi174 ceph-mon[35635]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T20:56:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:34 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/2294789424' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"} : dispatch 2024-09-10T20:56:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:34 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"} : dispatch 2024-09-10T20:56:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:34 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"}]': finished 2024-09-10T20:56:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:34 smithi174 ceph-mon[35635]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T20:56:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:35.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:34 smithi033 ceph-mon[30860]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T20:56:35.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:34 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/2294789424' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"} : dispatch 2024-09-10T20:56:35.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:34 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"} : dispatch 2024-09-10T20:56:35.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:34 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a51a4510-302a-4343-8cce-ee4b0c562880"}]': finished 2024-09-10T20:56:35.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:34 smithi033 ceph-mon[30860]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T20:56:35.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:35 smithi104 ceph-mon[35325]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:35 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/2687392462' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:56:36.078 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:35 smithi033 ceph-mon[30860]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:36.078 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:35 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/2687392462' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:56:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:35 smithi174 ceph-mon[35635]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:35 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/2687392462' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:56:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:37 smithi104 ceph-mon[35325]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:37 smithi174 ceph-mon[35635]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:38.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:37 smithi033 ceph-mon[30860]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:39.957 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:39 smithi104 ceph-mon[35325]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:39 smithi174 ceph-mon[35635]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:40.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:39 smithi033 ceph-mon[30860]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:41.950 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:41 smithi104 ceph-mon[35325]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:41 smithi174 ceph-mon[35635]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:41 smithi033 ceph-mon[30860]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:42 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T20:56:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:42 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:42 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T20:56:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:42 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:43.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:42 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T20:56:43.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:42 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:44.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:43 smithi104 ceph-mon[35325]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:44.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:43 smithi104 ceph-mon[35325]: Deploying daemon osd.1 on smithi104 2024-09-10T20:56:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:43 smithi174 ceph-mon[35635]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:43 smithi174 ceph-mon[35635]: Deploying daemon osd.1 on smithi104 2024-09-10T20:56:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:43 smithi033 ceph-mon[30860]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:43 smithi033 ceph-mon[30860]: Deploying daemon osd.1 on smithi104 2024-09-10T20:56:45.954 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:45 smithi104 ceph-mon[35325]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:45 smithi174 ceph-mon[35635]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:45 smithi033 ceph-mon[30860]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:48.056 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:47 smithi104 ceph-mon[35325]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:47 smithi174 ceph-mon[35635]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:47 smithi033 ceph-mon[30860]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:49 smithi104 ceph-mon[35325]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:49 smithi174 ceph-mon[35635]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:49 smithi033 ceph-mon[30860]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.025 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:51 smithi104 ceph-mon[35325]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:52.025 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.025 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.025 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:52.025 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:52.025 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:51 smithi174 ceph-mon[35635]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:51 smithi033 ceph-mon[30860]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.320 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.313+0000 7f6984ff9640 1 -- 172.21.15.104:0/3062644061 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f698806bba0 con 0x7f69700b5250 2024-09-10T20:56:52.320 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 1 on host 'smithi104' 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.315+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69700b5250 msgr2=0x7f69700b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.315+0000 7f698e105640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69700b5250 0x7f69700b7710 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f697800abe0 tx=0x7f697800a5c0 comp rx=0 tx=0).stop 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.315+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988102ab0 msgr2=0x7f69881994a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.315+0000 7f698e105640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988102ab0 0x7f69881994a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f697c00b4a0 tx=0x7f697c00b960 comp rx=0 tx=0).stop 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.315+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 shutdown_connections 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.315+0000 7f698e105640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69700b5250 0x7f69700b7710 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.315+0000 7f698e105640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6988103470 0x7f698819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.316+0000 7f698e105640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6988102ab0 0x7f69881994a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:52.321 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.316+0000 7f698e105640 1 --2- 172.21.15.104:0/3062644061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6988101950 0x7f6988198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:52.322 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.316+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 >> 172.21.15.104:0/3062644061 conn(0x7f69880fd200 msgr2=0x7f698806b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:56:52.322 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.316+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 shutdown_connections 2024-09-10T20:56:52.322 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:56:52.316+0000 7f698e105640 1 -- 172.21.15.104:0/3062644061 wait complete. 2024-09-10T20:56:52.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:52.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.871 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:52.904 DEBUG:teuthology.orchestra.run.smithi104:osd.1> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T20:56:52.906 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-09-10T20:56:52.907 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T20:56:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:53.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:53.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:53.158 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:56:53.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:56:53.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:53.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:53 smithi174 ceph-mon[35635]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:54.049 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:53 smithi104 ceph-mon[35325]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:53 smithi033 ceph-mon[30860]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:55.267 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:56:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1[39777]: 2024-09-10T20:56:54.926+0000 7f506e0ba640 -1 osd.1 0 log_to_monitors true 2024-09-10T20:56:55.733 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:56:55.760 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-09-10T20:56:56.012 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:56:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T20:56:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T20:56:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:56:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:56.134 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:56.134 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T20:56:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T20:56:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:56:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:56.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:56:56.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3185373410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4103450 msgr2=0x7fa2d4109dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3185373410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4103450 0x7fa2d4109dd0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009d00 tx=0x7fa2cc02a910 comp rx=0 tx=0).stop 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3185373410 shutdown_connections 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3185373410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4103450 0x7fa2d4109dd0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3185373410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2d4102a90 0x7fa2d4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3185373410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d4101930 0x7fa2d4101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3185373410 >> 172.21.15.174:0/3185373410 conn(0x7fa2d40fd200 msgr2=0x7fa2d40ff640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3185373410 shutdown_connections 2024-09-10T20:56:56.542 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.520+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3185373410 wait complete. 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 Processor -- start 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 -- start start 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4101930 0x7fa2d4198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2d4102a90 0x7fa2d41994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d4103450 0x7fa2d419a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d410e760 con 0x7fa2d4103450 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d410e5e0 con 0x7fa2d4102a90 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dc63d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d410e8e0 con 0x7fa2d4101930 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2da3d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4101930 0x7fa2d4198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2da3d9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4101930 0x7fa2d4198e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:37514/0 (socket says 172.21.15.174:37514) 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2da3d9640 1 -- 172.21.15.174:0/3141517611 learned_addr learned my addr 172.21.15.174:0/3141517611 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2dabda640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d4103450 0x7fa2d419a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:56.543 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2da3d9640 1 -- 172.21.15.174:0/3141517611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2d4102a90 msgr2=0x7fa2d41994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.521+0000 7fa2d9bd8640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2d4102a90 0x7fa2d41994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2da3d9640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2d4102a90 0x7fa2d41994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2da3d9640 1 -- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d4103450 msgr2=0x7fa2d419a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2da3d9640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d4103450 0x7fa2d419a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2da3d9640 1 -- 172.21.15.174:0/3141517611 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d419a940 con 0x7fa2d4101930 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2dabda640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d4103450 0x7fa2d419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2d9bd8640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2d4102a90 0x7fa2d41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2da3d9640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4101930 0x7fa2d4198e80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000b4a0 tx=0x7fa2d000b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2cb7fe640 1 -- 172.21.15.174:0/3141517611 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0014070 con 0x7fa2d4101930 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2cb7fe640 1 -- 172.21.15.174:0/3141517611 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2d0010ac0 con 0x7fa2d4101930 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2cb7fe640 1 -- 172.21.15.174:0/3141517611 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0017c30 con 0x7fa2d4101930 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d419ac30 con 0x7fa2d4101930 2024-09-10T20:56:56.544 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.522+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d41a1310 con 0x7fa2d4101930 2024-09-10T20:56:56.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.523+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2d4108be0 con 0x7fa2d4101930 2024-09-10T20:56:56.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.524+0000 7fa2cb7fe640 1 -- 172.21.15.174:0/3141517611 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7fa2d0010610 con 0x7fa2d4101930 2024-09-10T20:56:56.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.525+0000 7fa2cb7fe640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2c00b52d0 0x7fa2c00b7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:56:56.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.525+0000 7fa2cb7fe640 1 -- 172.21.15.174:0/3141517611 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1903+0+0 (secure 0 0 0) 0x7fa2d00e3a80 con 0x7fa2d4101930 2024-09-10T20:56:56.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.525+0000 7fa2d9bd8640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2c00b52d0 0x7fa2c00b7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:56:56.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.526+0000 7fa2d9bd8640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2c00b52d0 0x7fa2c00b7790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400baa0 tx=0x7fa2c4005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:56:56.545 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.529+0000 7fa2cb7fe640 1 -- 172.21.15.174:0/3141517611 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa2d00ac330 con 0x7fa2d4101930 2024-09-10T20:56:56.721 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:56:56.716+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa2d4063590 con 0x7fa2c00b52d0 2024-09-10T20:56:56.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi104 2024-09-10T20:56:56.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-10T20:56:56.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T20:56:56.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: from='osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:56:56.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T20:56:56.993 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:56.994 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:56:56.994 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:56.994 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:56.994 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi104 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: from='osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:57.130 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi104 2024-09-10T20:56:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi104 to 17088M 2024-09-10T20:56:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T20:56:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: from='osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:56:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T20:56:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:56:57.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:56:57.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:56:57.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:56:57.312 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1[39777]: 2024-09-10T20:56:57.192+0000 7f506a03d640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T20:56:57.312 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1[39777]: 2024-09-10T20:56:57.195+0000 7f5061e5c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:56:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-mon[35325]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-mon[35325]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-mon[35325]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:56:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-mon[35325]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T20:56:58.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:58.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:58.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:57 smithi104 ceph-mon[35325]: from='osd.1 ' entity='osd.1' 2024-09-10T20:56:58.072 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:57 smithi174 ceph-mon[35635]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:58.072 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:57 smithi174 ceph-mon[35635]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:58.072 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:57 smithi174 ceph-mon[35635]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:56:58.072 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:57 smithi174 ceph-mon[35635]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T20:56:58.072 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:58.072 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:58.072 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:57 smithi174 ceph-mon[35635]: from='osd.1 ' entity='osd.1' 2024-09-10T20:56:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:57 smithi033 ceph-mon[30860]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:57 smithi033 ceph-mon[30860]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:56:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:57 smithi033 ceph-mon[30860]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:56:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:57 smithi033 ceph-mon[30860]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T20:56:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:57 smithi033 ceph-mon[30860]: from='osd.1 ' entity='osd.1' 2024-09-10T20:56:59.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:58 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:56:59.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:58 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:56:59.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:59.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:58 smithi104 ceph-mon[35325]: osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] boot 2024-09-10T20:56:59.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:58 smithi104 ceph-mon[35325]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T20:56:59.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:58 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:56:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:58 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:56:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:58 smithi174 ceph-mon[35635]: osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] boot 2024-09-10T20:56:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:58 smithi174 ceph-mon[35635]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T20:56:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:59.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:58 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:56:59.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:58 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:56:59.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:59.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:58 smithi033 ceph-mon[30860]: osd.1 [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] boot 2024-09-10T20:56:59.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:58 smithi033 ceph-mon[30860]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T20:56:59.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T20:56:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:59 smithi174 ceph-mon[35635]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:56:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/2118364032' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"} : dispatch 2024-09-10T20:56:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:59 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"} : dispatch 2024-09-10T20:56:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:59 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"}]': finished 2024-09-10T20:56:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:59 smithi174 ceph-mon[35635]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T20:56:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:59 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:56:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:56:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/4161399973' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:59 smithi104 ceph-mon[35325]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:57:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/2118364032' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"} : dispatch 2024-09-10T20:57:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:59 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"} : dispatch 2024-09-10T20:57:00.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:59 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"}]': finished 2024-09-10T20:57:00.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:59 smithi104 ceph-mon[35325]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T20:57:00.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:59 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:00.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:56:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/4161399973' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:59 smithi033 ceph-mon[30860]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T20:57:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/2118364032' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"} : dispatch 2024-09-10T20:57:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:59 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"} : dispatch 2024-09-10T20:57:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:59 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ae175f96-8529-4db0-83bd-7ee10b49dddf"}]': finished 2024-09-10T20:57:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:59 smithi033 ceph-mon[30860]: osdmap e14: 3 total, 2 up, 3 in 2024-09-10T20:57:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:59 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:56:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/4161399973' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:01 smithi104 ceph-mon[35325]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:01 smithi174 ceph-mon[35635]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:01 smithi033 ceph-mon[30860]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:04.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:03 smithi104 ceph-mon[35325]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:03 smithi174 ceph-mon[35635]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:03 smithi033 ceph-mon[30860]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:06.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:05 smithi104 ceph-mon[35325]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:06.103 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:05 smithi174 ceph-mon[35635]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:05 smithi033 ceph-mon[30860]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:07.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:07 smithi174 ceph-mon[35635]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:07.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T20:57:07.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:07 smithi104 ceph-mon[35325]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T20:57:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:07 smithi033 ceph-mon[30860]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T20:57:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:08.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:08 smithi174 ceph-mon[35635]: Deploying daemon osd.2 on smithi174 2024-09-10T20:57:09.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:08 smithi104 ceph-mon[35325]: Deploying daemon osd.2 on smithi174 2024-09-10T20:57:09.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:08 smithi033 ceph-mon[30860]: Deploying daemon osd.2 on smithi174 2024-09-10T20:57:10.061 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:09 smithi174 ceph-mon[35635]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:10.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:09 smithi104 ceph-mon[35325]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:09 smithi033 ceph-mon[30860]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:12.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:11 smithi104 ceph-mon[35325]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:11 smithi174 ceph-mon[35635]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:11 smithi033 ceph-mon[30860]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:13 smithi104 ceph-mon[35325]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:14.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:14.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:13 smithi174 ceph-mon[35635]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:13 smithi033 ceph-mon[30860]: pgmap v80: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:15.953 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:15 smithi174 ceph-mon[35635]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:16.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:15 smithi104 ceph-mon[35325]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:15 smithi033 ceph-mon[30860]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:17.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:17.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:17.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:16 smithi104 ceph-mon[35325]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:17.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:17.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:17.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:16 smithi174 ceph-mon[35635]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:16 smithi033 ceph-mon[30860]: pgmap v82: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:17.788 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.784+0000 7fa2cb7fe640 1 -- 172.21.15.174:0/3141517611 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa2d4063590 con 0x7fa2c00b52d0 2024-09-10T20:57:17.788 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-09-10T20:57:17.791 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2c00b52d0 msgr2=0x7fa2c00b7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:17.791 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2c00b52d0 0x7fa2c00b7790 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400baa0 tx=0x7fa2c4005d90 comp rx=0 tx=0).stop 2024-09-10T20:57:17.791 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4101930 msgr2=0x7fa2d4198e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:17.791 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4101930 0x7fa2d4198e80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000b4a0 tx=0x7fa2d000b960 comp rx=0 tx=0).stop 2024-09-10T20:57:17.791 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 shutdown_connections 2024-09-10T20:57:17.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2c00b52d0 0x7fa2c00b7790 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:17.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2d4103450 0x7fa2d419a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:17.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2d4102a90 0x7fa2d41994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:17.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 --2- 172.21.15.174:0/3141517611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2d4101930 0x7fa2d4198e80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:17.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 >> 172.21.15.174:0/3141517611 conn(0x7fa2d40fd200 msgr2=0x7fa2d4105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:57:17.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 shutdown_connections 2024-09-10T20:57:17.792 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:57:17.787+0000 7fa2dc63d640 1 -- 172.21.15.174:0/3141517611 wait complete. 2024-09-10T20:57:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:18.375 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:18.375 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:18.375 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:18.402 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T20:57:18.405 INFO:tasks.cephadm:Deploying osd.3 on smithi033 with /dev/vg_nvme/lv_3... 2024-09-10T20:57:18.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T20:57:18.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:18.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:18.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:57:19.110 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:19 smithi033 ceph-mon[30860]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:19.289 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:19 smithi174 ceph-mon[35635]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:19.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:19 smithi104 ceph-mon[35325]: pgmap v83: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:20.549 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:20 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2[40634]: 2024-09-10T20:57:20.139+0000 7f4899166640 -1 osd.2 0 log_to_monitors true 2024-09-10T20:57:21.180 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:21 smithi033 ceph-mon[30860]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:21.180 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:21 smithi033 ceph-mon[30860]: from='osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T20:57:21.180 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:21 smithi033 ceph-mon[30860]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T20:57:21.210 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:57:21.229 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:21 smithi174 ceph-mon[35635]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:21.229 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:21 smithi174 ceph-mon[35635]: from='osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T20:57:21.230 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:21 smithi174 ceph-mon[35635]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T20:57:21.240 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-09-10T20:57:21.548 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:57:21.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:21 smithi104 ceph-mon[35325]: pgmap v84: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:21.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:21 smithi104 ceph-mon[35325]: from='osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T20:57:21.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:21 smithi104 ceph-mon[35325]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T20:57:22.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 -- 172.21.15.33:0/1029186856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc102a70 msgr2=0x7f7bbc102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:22.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 --2- 172.21.15.33:0/1029186856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc102a70 0x7f7bbc102ef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0009930 tx=0x7f7bb002a420 comp rx=0 tx=0).stop 2024-09-10T20:57:22.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 -- 172.21.15.33:0/1029186856 shutdown_connections 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 --2- 172.21.15.33:0/1029186856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bbc103430 0x7f7bbc109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 --2- 172.21.15.33:0/1029186856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc102a70 0x7f7bbc102ef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 --2- 172.21.15.33:0/1029186856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bbc101910 0x7f7bbc101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 -- 172.21.15.33:0/1029186856 >> 172.21.15.33:0/1029186856 conn(0x7f7bbc0fd1e0 msgr2=0x7f7bbc0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 -- 172.21.15.33:0/1029186856 shutdown_connections 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.028+0000 7f7bc302c640 1 -- 172.21.15.33:0/1029186856 wait complete. 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 Processor -- start 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 -- start start 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc101910 0x7f7bbc19b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bbc102a70 0x7f7bbc19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:22.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bbc103430 0x7f7bbc19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc10d5f0 con 0x7f7bbc101910 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc10d470 con 0x7f7bbc103430 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc302c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7bbc10d770 con 0x7f7bbc102a70 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc0dc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc101910 0x7f7bbc19b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc0dc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc101910 0x7f7bbc19b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60052/0 (socket says 172.21.15.33:60052) 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc0dc8640 1 -- 172.21.15.33:0/2943598885 learned_addr learned my addr 172.21.15.33:0/2943598885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bbbfff640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bbc102a70 0x7f7bbc19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.029+0000 7f7bc15c9640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bbc103430 0x7f7bbc19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc0dc8640 1 -- 172.21.15.33:0/2943598885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bbc102a70 msgr2=0x7f7bbc19b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc0dc8640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bbc102a70 0x7f7bbc19b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc0dc8640 1 -- 172.21.15.33:0/2943598885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bbc103430 msgr2=0x7f7bbc19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:22.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc0dc8640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bbc103430 0x7f7bbc19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:22.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc0dc8640 1 -- 172.21.15.33:0/2943598885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bbc19cbd0 con 0x7f7bbc101910 2024-09-10T20:57:22.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc0dc8640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc101910 0x7f7bbc19b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00d950 tx=0x7f7bac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:57:22.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac017070 con 0x7f7bbc101910 2024-09-10T20:57:22.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bac009c30 con 0x7f7bbc101910 2024-09-10T20:57:22.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bbc19cec0 con 0x7f7bbc101910 2024-09-10T20:57:22.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bac00f620 con 0x7f7bbc101910 2024-09-10T20:57:22.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.030+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bbc1a35a0 con 0x7f7bbc101910 2024-09-10T20:57:22.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.031+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bbc1a3770 con 0x7f7bbc101910 2024-09-10T20:57:22.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.033+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) ==== 153194+0+0 (secure 0 0 0) 0x7f7bac010460 con 0x7f7bbc101910 2024-09-10T20:57:22.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.033+0000 7f7bb9ffb640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ba40b5250 0x7f7ba40b7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:22.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.034+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 2445+0+0 (secure 0 0 0) 0x7f7bac0e4000 con 0x7f7bbc101910 2024-09-10T20:57:22.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.034+0000 7f7bbbfff640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ba40b5250 0x7f7ba40b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:57:22.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.034+0000 7f7bbbfff640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ba40b5250 0x7f7ba40b7710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0006010 tx=0x7f7bb0031430 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:57:22.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.036+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7bac0ac690 con 0x7f7bbc101910 2024-09-10T20:57:22.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:22.235+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f7bbc0ff350 con 0x7f7ba40b5250 2024-09-10T20:57:22.394 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2[40634]: 2024-09-10T20:57:22.093+0000 7f48950e9640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T20:57:22.394 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2[40634]: 2024-09-10T20:57:22.097+0000 7f4890f10640 -1 osd.2 16 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:57:22.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T20:57:22.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T20:57:22.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi174 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:22.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi174 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:22.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-10T20:57:22.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:22.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:22.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T20:57:22.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T20:57:22.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:22.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi174 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi174 to 17088M 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:22.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:23.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:23.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:57:23.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T20:57:23.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:23.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:23.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:57:23.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:57:23.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:57:23.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:57:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:23.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: pgmap v86: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:23.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:57:23.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T20:57:23.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:23.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:23.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:57:23.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:57:23.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:57:23.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:24 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:57:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:24 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:57:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:24 smithi174 ceph-mon[35635]: osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] boot 2024-09-10T20:57:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:24 smithi174 ceph-mon[35635]: osdmap e17: 3 total, 3 up, 3 in 2024-09-10T20:57:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:24.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:24 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:57:24.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:24 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:57:24.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:24 smithi033 ceph-mon[30860]: osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] boot 2024-09-10T20:57:24.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:24 smithi033 ceph-mon[30860]: osdmap e17: 3 total, 3 up, 3 in 2024-09-10T20:57:24.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:24.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:24 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:57:24.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:24 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:57:24.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:24 smithi104 ceph-mon[35325]: osd.2 [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] boot 2024-09-10T20:57:24.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:24 smithi104 ceph-mon[35325]: osdmap e17: 3 total, 3 up, 3 in 2024-09-10T20:57:24.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:24 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4051934741' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87f711da-d77c-452d-9a29-3418de2ba617"} : dispatch 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4051934741' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87f711da-d77c-452d-9a29-3418de2ba617"}]': finished 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: osdmap e19: 4 total, 3 up, 4 in 2024-09-10T20:57:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:25.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T20:57:25.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/337227349' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:25.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:25.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T20:57:25.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T20:57:25.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4051934741' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87f711da-d77c-452d-9a29-3418de2ba617"} : dispatch 2024-09-10T20:57:25.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T20:57:25.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4051934741' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87f711da-d77c-452d-9a29-3418de2ba617"}]': finished 2024-09-10T20:57:25.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: osdmap e19: 4 total, 3 up, 4 in 2024-09-10T20:57:25.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:25.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T20:57:25.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/337227349' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: pgmap v89: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4051934741' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "87f711da-d77c-452d-9a29-3418de2ba617"} : dispatch 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4051934741' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "87f711da-d77c-452d-9a29-3418de2ba617"}]': finished 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: osdmap e19: 4 total, 3 up, 4 in 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:25.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T20:57:25.465 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/337227349' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:25.812 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43920]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T20:57:25.813 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43920]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.813 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43920]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.813 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43920]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:25.813 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43923]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T20:57:25.813 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43923]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.813 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43923]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.813 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43923]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44560]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44560]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44560]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44560]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44563]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44563]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44563]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.829 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44563]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51815]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51815]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51815]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51815]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51818]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51818]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51818]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51818]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51808]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51808]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51808]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51808]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51811]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51811]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51811]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:25.928 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 20:57:25 smithi033 sudo[51811]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44571]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44571]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44571]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44571]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44574]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44574]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44574]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:25 smithi174 sudo[44574]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:26.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43926]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T20:57:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43926]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43926]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43929]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T20:57:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43929]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T20:57:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43929]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T20:57:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:25 smithi104 sudo[43929]: pam_unix(sudo:session): session closed for user root 2024-09-10T20:57:26.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:26.781+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f7bac0a8200 con 0x7f7bbc101910 2024-09-10T20:57:26.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T20:57:26.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T20:57:26.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:26.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.788 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:26 smithi033 ceph-mon[30860]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T20:57:26.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T20:57:26.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:26.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.815 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.815 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.815 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.815 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.815 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.816 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.817 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:26 smithi104 ceph-mon[35325]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: osdmap e20: 4 total, 3 up, 4 in 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T20:57:26.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:26 smithi174 ceph-mon[35635]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T20:57:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:27 smithi104 ceph-mon[35325]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:27 smithi104 ceph-mon[35325]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T20:57:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:27 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:27 smithi104 ceph-mon[35325]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T20:57:27.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:27 smithi174 ceph-mon[35635]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:27.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:27 smithi174 ceph-mon[35635]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T20:57:27.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:27 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:27.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:27 smithi174 ceph-mon[35635]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T20:57:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:27 smithi033 ceph-mon[30860]: pgmap v93: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:27 smithi033 ceph-mon[30860]: osdmap e21: 4 total, 3 up, 4 in 2024-09-10T20:57:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:27 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:27 smithi033 ceph-mon[30860]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-10T20:57:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:29 smithi104 ceph-mon[35325]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:29.858 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:29 smithi033 ceph-mon[30860]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:29 smithi174 ceph-mon[35635]: pgmap v95: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:31.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:31 smithi033 ceph-mon[30860]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:31 smithi104 ceph-mon[35325]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:31 smithi174 ceph-mon[35635]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:33 smithi104 ceph-mon[35325]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T20:57:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:33.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:33 smithi033 ceph-mon[30860]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:33.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T20:57:33.878 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:33 smithi174 ceph-mon[35635]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:33.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T20:57:33.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:34.582 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:34 smithi033 ceph-mon[30860]: Deploying daemon osd.3 on smithi033 2024-09-10T20:57:34.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:34 smithi104 ceph-mon[35325]: Deploying daemon osd.3 on smithi033 2024-09-10T20:57:34.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:34 smithi174 ceph-mon[35635]: Deploying daemon osd.3 on smithi033 2024-09-10T20:57:35.748 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:35 smithi033 ceph-mon[30860]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:35 smithi104 ceph-mon[35325]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:35 smithi174 ceph-mon[35635]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:37 smithi104 ceph-mon[35325]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:37 smithi174 ceph-mon[35635]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:37.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:37 smithi033 ceph-mon[30860]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:39 smithi104 ceph-mon[35325]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:39.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:39.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:39 smithi174 ceph-mon[35635]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:39 smithi033 ceph-mon[30860]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:41 smithi104 ceph-mon[35325]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:41.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:41 smithi033 ceph-mon[30860]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:41 smithi174 ceph-mon[35635]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:42.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.946+0000 7f7bb9ffb640 1 -- 172.21.15.33:0/2943598885 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7bbc0ff350 con 0x7f7ba40b5250 2024-09-10T20:57:42.950 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 3 on host 'smithi033' 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.948+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ba40b5250 msgr2=0x7f7ba40b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ba40b5250 0x7f7ba40b7710 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7bb0006010 tx=0x7f7bb0031430 comp rx=0 tx=0).stop 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc101910 msgr2=0x7f7bbc19b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc101910 0x7f7bbc19b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00d950 tx=0x7f7bac00de10 comp rx=0 tx=0).stop 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 shutdown_connections 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ba40b5250 0x7f7ba40b7710 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bbc103430 0x7f7bbc19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bbc102a70 0x7f7bbc19b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 --2- 172.21.15.33:0/2943598885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bbc101910 0x7f7bbc19b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 >> 172.21.15.33:0/2943598885 conn(0x7f7bbc0fd1e0 msgr2=0x7f7bbc0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:57:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 shutdown_connections 2024-09-10T20:57:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:57:42.949+0000 7f7bc302c640 1 -- 172.21.15.33:0/2943598885 wait complete. 2024-09-10T20:57:43.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:43.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.567 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:43.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:43.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:43.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.598 DEBUG:teuthology.orchestra.run.smithi033:osd.3> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T20:57:43.601 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2024-09-10T20:57:43.601 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T20:57:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:43.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.854 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:57:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:43.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:45.521 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 20:57:45 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T20:57:45.345+0000 7f85113e1640 -1 osd.3 0 log_to_monitors true 2024-09-10T20:57:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:45 smithi104 ceph-mon[35325]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:45 smithi104 ceph-mon[35325]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T20:57:45.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:45 smithi174 ceph-mon[35635]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:45.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:45 smithi174 ceph-mon[35635]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T20:57:45.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:45 smithi033 ceph-mon[30860]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:45.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:45 smithi033 ceph-mon[30860]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T20:57:46.304 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:57:46.330 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-09-10T20:57:46.573 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T20:57:46.584 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 20:57:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T20:57:46.521+0000 7f850d364640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T20:57:46.584 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 20:57:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T20:57:46.530+0000 7f850918b640 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:57:46.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:46 smithi033 ceph-mon[30860]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T20:57:46.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:46 smithi033 ceph-mon[30860]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T20:57:46.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:46 smithi033 ceph-mon[30860]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:57:46.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:46.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:46 smithi033 ceph-mon[30860]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:57:46.694 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:46 smithi104 ceph-mon[35325]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T20:57:46.694 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:46 smithi104 ceph-mon[35325]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T20:57:46.694 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:46 smithi104 ceph-mon[35325]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:57:46.695 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:46.695 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:46 smithi104 ceph-mon[35325]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:57:46.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:46 smithi174 ceph-mon[35635]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T20:57:46.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:46 smithi174 ceph-mon[35635]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T20:57:46.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:46 smithi174 ceph-mon[35635]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:57:46.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:46.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:46 smithi174 ceph-mon[35635]: from='osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.006+0000 7f09054e1640 1 -- 172.21.15.104:0/3084522485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900104550 msgr2=0x7f090010aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.006+0000 7f09054e1640 1 --2- 172.21.15.104:0/3084522485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900104550 0x7f090010aed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009cd0 tx=0x7f08f802a8e0 comp rx=0 tx=0).stop 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 -- 172.21.15.104:0/3084522485 shutdown_connections 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 --2- 172.21.15.104:0/3084522485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900104550 0x7f090010aed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 --2- 172.21.15.104:0/3084522485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0900103b90 0x7f0900104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 --2- 172.21.15.104:0/3084522485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0900102a30 0x7f0900102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 -- 172.21.15.104:0/3084522485 >> 172.21.15.104:0/3084522485 conn(0x7f09000fe320 msgr2=0x7f0900100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:57:47.020 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 -- 172.21.15.104:0/3084522485 shutdown_connections 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 -- 172.21.15.104:0/3084522485 wait complete. 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 Processor -- start 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.007+0000 7f09054e1640 1 -- start start 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900102a30 0x7f0900198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0900103b90 0x7f0900199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0900104550 0x7f090019a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f090010e710 con 0x7f0900103b90 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f090010e590 con 0x7f0900102a30 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f090010e890 con 0x7f0900104550 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900102a30 0x7f0900198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900102a30 0x7f0900198ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:57732/0 (socket says 172.21.15.104:57732) 2024-09-10T20:57:47.021 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 -- 172.21.15.104:0/1208212130 learned_addr learned my addr 172.21.15.104:0/1208212130 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08fe7fc640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0900103b90 0x7f0900199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 -- 172.21.15.104:0/1208212130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0900104550 msgr2=0x7f090019a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0900104550 0x7f090019a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 -- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0900103b90 msgr2=0x7f0900199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0900103b90 0x7f0900199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 -- 172.21.15.104:0/1208212130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090019a9f0 con 0x7f0900102a30 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08feffd640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900102a30 0x7f0900198ec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b4a0 tx=0x7f08f000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08e7fff640 1 -- 172.21.15.104:0/1208212130 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0014070 con 0x7f0900102a30 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090019ace0 con 0x7f0900102a30 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08e7fff640 1 -- 172.21.15.104:0/1208212130 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08f0010ac0 con 0x7f0900102a30 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f08e7fff640 1 -- 172.21.15.104:0/1208212130 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0017c30 con 0x7f0900102a30 2024-09-10T20:57:47.022 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.008+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09001a13c0 con 0x7f0900102a30 2024-09-10T20:57:47.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.009+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0900109d50 con 0x7f0900102a30 2024-09-10T20:57:47.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.011+0000 7f08e7fff640 1 -- 172.21.15.104:0/1208212130 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f08f0010610 con 0x7f0900102a30 2024-09-10T20:57:47.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.011+0000 7f08e7fff640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08e80b52e0 0x7f08e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:57:47.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.012+0000 7f08e7fff640 1 -- 172.21.15.104:0/1208212130 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3389+0+0 (secure 0 0 0) 0x7f08f00e4160 con 0x7f0900102a30 2024-09-10T20:57:47.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.012+0000 7f08fe7fc640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08e80b52e0 0x7f08e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:57:47.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.012+0000 7f08fe7fc640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08e80b52e0 0x7f08e80b77a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005d00 tx=0x7f08f400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:57:47.023 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.015+0000 7f08e7fff640 1 -- 172.21.15.104:0/1208212130 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f08f00ac360 con 0x7f0900102a30 2024-09-10T20:57:47.204 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:57:47.200+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0900063590 con 0x7f08e80b52e0 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi033 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:47.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi033 to 8546M 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] boot 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T20:57:47.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi033 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi033 to 8546M 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:57:47.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:57:47.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:47.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] boot 2024-09-10T20:57:47.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T20:57:47.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-10T20:57:47.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T20:57:47.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi033 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi033 to 8546M 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:47.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:57:47.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:57:47.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:57:47.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:57:47.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:47.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: osd.3 [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] boot 2024-09-10T20:57:47.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: osdmap e24: 4 total, 4 up, 4 in 2024-09-10T20:57:47.930 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T20:57:48.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:48 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:57:48.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:48 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:57:48.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:48 smithi104 ceph-mon[35325]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:57:48.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:48 smithi104 ceph-mon[35325]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T20:57:48.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:48 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:57:48.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:48 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:57:48.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:48 smithi174 ceph-mon[35635]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:57:48.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:48 smithi174 ceph-mon[35635]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T20:57:48.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:48 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:57:48.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:48 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:57:48.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:48 smithi033 ceph-mon[30860]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:57:48.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:48 smithi033 ceph-mon[30860]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T20:57:49.801 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:49 smithi104 ceph-mon[35325]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:49.801 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:49 smithi104 ceph-mon[35325]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T20:57:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:49 smithi174 ceph-mon[35635]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:49 smithi174 ceph-mon[35635]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T20:57:49.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:49 smithi033 ceph-mon[30860]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:49.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:49 smithi033 ceph-mon[30860]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T20:57:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:50 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/2227628195' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"} : dispatch 2024-09-10T20:57:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:50 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"} : dispatch 2024-09-10T20:57:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:50 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"}]': finished 2024-09-10T20:57:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:50 smithi104 ceph-mon[35325]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T20:57:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:50 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:57:50.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:50 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/2195451059' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:50.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:50 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/2227628195' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"} : dispatch 2024-09-10T20:57:50.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:50 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"} : dispatch 2024-09-10T20:57:50.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:50 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"}]': finished 2024-09-10T20:57:50.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:50 smithi174 ceph-mon[35635]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T20:57:50.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:50 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:57:50.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:50 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/2195451059' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:50.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/2227628195' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"} : dispatch 2024-09-10T20:57:50.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:50 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"} : dispatch 2024-09-10T20:57:50.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:50 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "02b1eb40-cf26-4cee-9c4b-c5a353a8537a"}]': finished 2024-09-10T20:57:50.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:50 smithi033 ceph-mon[30860]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T20:57:50.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:50 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:57:50.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/2195451059' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:57:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:51 smithi174 ceph-mon[35635]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:51.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:51 smithi033 ceph-mon[30860]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:52.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:51 smithi104 ceph-mon[35325]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:53 smithi174 ceph-mon[35635]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:53.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:53 smithi033 ceph-mon[30860]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:53 smithi104 ceph-mon[35325]: pgmap v113: 1 pgs: 1 peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:57:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:55 smithi174 ceph-mon[35635]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T20:57:55.894 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:55 smithi104 ceph-mon[35325]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T20:57:55.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:55 smithi033 ceph-mon[30860]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T20:57:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:57 smithi104 ceph-mon[35325]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T20:57:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:57 smithi174 ceph-mon[35635]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T20:57:57.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:57 smithi033 ceph-mon[30860]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T20:57:58.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T20:57:58.603 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T20:57:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:58.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T20:57:58.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:57:59.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:59 smithi104 ceph-mon[35325]: Deploying daemon osd.4 on smithi104 2024-09-10T20:57:59.872 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:57:59 smithi104 ceph-mon[35325]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T20:57:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:59 smithi174 ceph-mon[35635]: Deploying daemon osd.4 on smithi104 2024-09-10T20:57:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:57:59 smithi174 ceph-mon[35635]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T20:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:59 smithi033 ceph-mon[30860]: Deploying daemon osd.4 on smithi104 2024-09-10T20:57:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:57:59 smithi033 ceph-mon[30860]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T20:58:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:01 smithi174 ceph-mon[35635]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T20:58:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:01 smithi033 ceph-mon[30860]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T20:58:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:01 smithi104 ceph-mon[35325]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T20:58:03.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:03 smithi104 ceph-mon[35325]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:03.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:03 smithi174 ceph-mon[35635]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:03.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:03 smithi033 ceph-mon[30860]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:04.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:04 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:04.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:04 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:04.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:04 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:04.914 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:04 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:04.914 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:04 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:04.914 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:04 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:04.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:04 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:04.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:04 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:04.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:04 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:05.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:05 smithi174 ceph-mon[35635]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:05.917 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:05 smithi104 ceph-mon[35325]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:05.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:05 smithi033 ceph-mon[30860]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:07 smithi104 ceph-mon[35325]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:07 smithi174 ceph-mon[35635]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:07 smithi033 ceph-mon[30860]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:07.954 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.949+0000 7f08e7fff640 1 -- 172.21.15.104:0/1208212130 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0900063590 con 0x7f08e80b52e0 2024-09-10T20:58:07.954 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-09-10T20:58:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08e80b52e0 msgr2=0x7f08e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08e80b52e0 0x7f08e80b77a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08f4005d00 tx=0x7f08f400b410 comp rx=0 tx=0).stop 2024-09-10T20:58:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900102a30 msgr2=0x7f0900198ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900102a30 0x7f0900198ec0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f08f000b4a0 tx=0x7f08f000b960 comp rx=0 tx=0).stop 2024-09-10T20:58:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 shutdown_connections 2024-09-10T20:58:07.956 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08e80b52e0 0x7f08e80b77a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0900104550 0x7f090019a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0900103b90 0x7f0900199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 --2- 172.21.15.104:0/1208212130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0900102a30 0x7f0900198ec0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 >> 172.21.15.104:0/1208212130 conn(0x7f09000fe320 msgr2=0x7f0900106f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 shutdown_connections 2024-09-10T20:58:07.957 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:58:07.952+0000 7f09054e1640 1 -- 172.21.15.104:0/1208212130 wait complete. 2024-09-10T20:58:08.566 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T20:58:08.568 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-09-10T20:58:08.569 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T20:58:08.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:08 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:08.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:08 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:08.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:08 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:08.828 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:58:08.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:08 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:08.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:08 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:08.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:08 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:08.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:08 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:08.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:08 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:08.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:08 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:09.688 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:09 smithi104 ceph-mon[35325]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:09 smithi174 ceph-mon[35635]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:09.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:09 smithi033 ceph-mon[30860]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:09.996 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 20:58:09 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4[46670]: 2024-09-10T20:58:09.801+0000 7f8a4548c640 -1 osd.4 0 log_to_monitors true 2024-09-10T20:58:10.784 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:10 smithi174 ceph-mon[35635]: from='osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T20:58:10.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:10 smithi174 ceph-mon[35635]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T20:58:10.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:10 smithi033 ceph-mon[30860]: from='osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T20:58:10.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:10 smithi033 ceph-mon[30860]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T20:58:11.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:10 smithi104 ceph-mon[35325]: from='osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T20:58:11.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:10 smithi104 ceph-mon[35325]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T20:58:11.377 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:58:11.402 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-09-10T20:58:11.662 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:58:11.812 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4[46670]: 2024-09-10T20:58:11.617+0000 7f8a4140f640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T20:58:11.812 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4[46670]: 2024-09-10T20:58:11.622+0000 7f8a3d236640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:11.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:11.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.884 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:11.884 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T20:58:11.884 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T20:58:11.884 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:11.884 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:11.885 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:11.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:11.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:11.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:11.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:11.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:12.144 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 -- 172.21.15.174:0/2906363035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 msgr2=0x7fa6b8107ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:12.144 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 --2- 172.21.15.174:0/2906363035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8107ff0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009900 tx=0x7fa6a402a410 comp rx=0 tx=0).stop 2024-09-10T20:58:12.144 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 -- 172.21.15.174:0/2906363035 shutdown_connections 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 --2- 172.21.15.174:0/2906363035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b8108730 0x7fa6b810abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 --2- 172.21.15.174:0/2906363035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8107ff0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 --2- 172.21.15.174:0/2906363035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b806bfa0 0x7fa6b81056c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 -- 172.21.15.174:0/2906363035 >> 172.21.15.174:0/2906363035 conn(0x7fa6b80fd210 msgr2=0x7fa6b80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 -- 172.21.15.174:0/2906363035 shutdown_connections 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.109+0000 7fa6bed54640 1 -- 172.21.15.174:0/2906363035 wait complete. 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 Processor -- start 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 -- start start 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b806bfa0 0x7fa6b8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:12.145 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b8108730 0x7fa6b819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b810e680 con 0x7fa6b806bfa0 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b810e500 con 0x7fa6b8108730 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bed54640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b810e800 con 0x7fa6b8105c00 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6b7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6b7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43802/0 (socket says 172.21.15.174:43802) 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6b7fff640 1 -- 172.21.15.174:0/306724672 learned_addr learned my addr 172.21.15.174:0/306724672 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.110+0000 7fa6bd2f1640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b8108730 0x7fa6b819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:12.146 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6bcaf0640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b806bfa0 0x7fa6b8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b7fff640 1 -- 172.21.15.174:0/306724672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b8108730 msgr2=0x7fa6b819a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b7fff640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b8108730 0x7fa6b819a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b7fff640 1 -- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b806bfa0 msgr2=0x7fa6b8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b7fff640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b806bfa0 0x7fa6b8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b7fff640 1 -- 172.21.15.174:0/306724672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b819a8e0 con 0x7fa6b8105c00 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6bd2f1640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b8108730 0x7fa6b819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b7fff640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8199400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4005940 tx=0x7fa6a40042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:12.147 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b5ffb640 1 -- 172.21.15.174:0/306724672 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a403f030 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b5ffb640 1 -- 172.21.15.174:0/306724672 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a4003cb0 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6b5ffb640 1 -- 172.21.15.174:0/306724672 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4035070 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b819ab70 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.111+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b81a12b0 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.112+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6b80fed00 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.114+0000 7fa6b5ffb640 1 -- 172.21.15.174:0/306724672 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fa6a403bbe0 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.114+0000 7fa6b5ffb640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6a80b5360 0x7fa6a80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.115+0000 7fa6b5ffb640 1 -- 172.21.15.174:0/306724672 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3821+0+0 (secure 0 0 0) 0x7fa6a41057f0 con 0x7fa6b8105c00 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.115+0000 7fa6bcaf0640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6a80b5360 0x7fa6a80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.115+0000 7fa6bcaf0640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6a80b5360 0x7fa6a80b7820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009b60 tx=0x7fa6ac005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:12.148 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.118+0000 7fa6b5ffb640 1 -- 172.21.15.174:0/306724672 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa6a40cd840 con 0x7fa6b8105c00 2024-09-10T20:58:12.316 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:12.311+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa6b8063590 con 0x7fa6a80b5360 2024-09-10T20:58:12.888 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi104 2024-09-10T20:58:12.888 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-10T20:58:12.888 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:58:12.888 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T20:58:12.888 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:12.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:12.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:58:12.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:58:12.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:12.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi104 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:58:12.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:58:12.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:12.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:13.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi104 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi104 to 8544M 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:13.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:13.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:13 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:58:13.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:13 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:58:13.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:13 smithi174 ceph-mon[35635]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:13.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:13 smithi174 ceph-mon[35635]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:58:13.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:13 smithi174 ceph-mon[35635]: osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] boot 2024-09-10T20:58:13.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:13 smithi174 ceph-mon[35635]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T20:58:13.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:13 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:58:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:13 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:58:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:13 smithi033 ceph-mon[30860]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:13 smithi033 ceph-mon[30860]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:58:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:13 smithi033 ceph-mon[30860]: osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] boot 2024-09-10T20:58:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:13 smithi033 ceph-mon[30860]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T20:58:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:13 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:58:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:13 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:58:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:13 smithi104 ceph-mon[35325]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-09-10T20:58:14.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:13 smithi104 ceph-mon[35325]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:58:14.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:13 smithi104 ceph-mon[35325]: osd.4 [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] boot 2024-09-10T20:58:14.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:13 smithi104 ceph-mon[35325]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T20:58:14.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T20:58:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:14 smithi174 ceph-mon[35635]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T20:58:14.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:14 smithi033 ceph-mon[30860]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T20:58:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:14 smithi104 ceph-mon[35325]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T20:58:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:15 smithi033 ceph-mon[30860]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:15 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/25454032' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"} : dispatch 2024-09-10T20:58:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:15 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"} : dispatch 2024-09-10T20:58:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:15 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"}]': finished 2024-09-10T20:58:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:15 smithi033 ceph-mon[30860]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T20:58:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:15 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:15 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/422442693' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:58:16.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:15 smithi104 ceph-mon[35325]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:16.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:15 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/25454032' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"} : dispatch 2024-09-10T20:58:16.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:15 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"} : dispatch 2024-09-10T20:58:16.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:15 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"}]': finished 2024-09-10T20:58:16.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:15 smithi104 ceph-mon[35325]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T20:58:16.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:15 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:16.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:15 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/422442693' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:58:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:15 smithi174 ceph-mon[35635]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:15 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/25454032' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"} : dispatch 2024-09-10T20:58:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:15 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"} : dispatch 2024-09-10T20:58:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:15 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e7d7361f-8c71-472e-84a7-bedf288dc9bd"}]': finished 2024-09-10T20:58:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:15 smithi174 ceph-mon[35635]: osdmap e32: 6 total, 5 up, 6 in 2024-09-10T20:58:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:15 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:15 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/422442693' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:58:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:17 smithi033 ceph-mon[30860]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:18.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:17 smithi104 ceph-mon[35325]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:17 smithi174 ceph-mon[35635]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:19 smithi174 ceph-mon[35635]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:19.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:19 smithi033 ceph-mon[30860]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:20.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:19 smithi104 ceph-mon[35325]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 137 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:21.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:21 smithi174 ceph-mon[35635]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:21 smithi033 ceph-mon[30860]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:21 smithi104 ceph-mon[35325]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:23.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:23 smithi174 ceph-mon[35635]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:23.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T20:58:23.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:23.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:23 smithi174 ceph-mon[35635]: Deploying daemon osd.5 on smithi174 2024-09-10T20:58:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:23 smithi033 ceph-mon[30860]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T20:58:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:23 smithi033 ceph-mon[30860]: Deploying daemon osd.5 on smithi174 2024-09-10T20:58:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:23 smithi104 ceph-mon[35325]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T20:58:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:23 smithi104 ceph-mon[35325]: Deploying daemon osd.5 on smithi174 2024-09-10T20:58:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:25 smithi033 ceph-mon[30860]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:25 smithi104 ceph-mon[35325]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:26.078 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:25 smithi174 ceph-mon[35635]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:27.798 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:27 smithi174 ceph-mon[35635]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:27 smithi033 ceph-mon[30860]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:27 smithi104 ceph-mon[35325]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:29.816 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:29 smithi174 ceph-mon[35635]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:29.816 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:29.817 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:29.817 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:29 smithi033 ceph-mon[30860]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:30.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:29 smithi104 ceph-mon[35325]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:30.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:30.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:30.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:31 smithi174 ceph-mon[35635]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:31.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:31.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:31 smithi033 ceph-mon[30860]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:31 smithi104 ceph-mon[35325]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:32.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:32.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:32.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:33.174 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-09-10T20:58:33.175 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.170+0000 7fa6b5ffb640 1 -- 172.21.15.174:0/306724672 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa6b8063590 con 0x7fa6a80b5360 2024-09-10T20:58:33.176 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6a80b5360 msgr2=0x7fa6a80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6a80b5360 0x7fa6a80b7820 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009b60 tx=0x7fa6ac005f90 comp rx=0 tx=0).stop 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 msgr2=0x7fa6b8199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8199400 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4005940 tx=0x7fa6a40042f0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 shutdown_connections 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6a80b5360 0x7fa6a80b7820 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b8108730 0x7fa6b819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b8105c00 0x7fa6b8199400 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 --2- 172.21.15.174:0/306724672 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b806bfa0 0x7fa6b8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 >> 172.21.15.174:0/306724672 conn(0x7fa6b80fd210 msgr2=0x7fa6b8103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 shutdown_connections 2024-09-10T20:58:33.177 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:58:33.173+0000 7fa6bed54640 1 -- 172.21.15.174:0/306724672 wait complete. 2024-09-10T20:58:33.784 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:33 smithi174 ceph-mon[35635]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:33.784 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:33.784 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:33.784 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:33.811 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T20:58:33.813 INFO:tasks.cephadm:Deploying osd.6 on smithi033 with /dev/vg_nvme/lv_2... 2024-09-10T20:58:33.813 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T20:58:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:33 smithi033 ceph-mon[30860]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:33 smithi104 ceph-mon[35325]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:34.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:34.083 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:58:35.527 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 20:58:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T20:58:35.359+0000 7f2b76af7640 -1 osd.5 0 log_to_monitors true 2024-09-10T20:58:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:35 smithi174 ceph-mon[35635]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:35 smithi174 ceph-mon[35635]: from='osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T20:58:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:35 smithi174 ceph-mon[35635]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T20:58:35.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:35 smithi033 ceph-mon[30860]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:35.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:35 smithi033 ceph-mon[30860]: from='osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T20:58:35.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:35 smithi033 ceph-mon[30860]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T20:58:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:35 smithi104 ceph-mon[35325]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:35 smithi104 ceph-mon[35325]: from='osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T20:58:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:35 smithi104 ceph-mon[35325]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T20:58:36.726 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:58:36.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-09-10T20:58:36.814 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 20:58:36 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T20:58:36.689+0000 7f2b72a7a640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T20:58:36.815 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 20:58:36 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T20:58:36.695+0000 7f2b6e8a1640 -1 osd.5 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:58:36.815 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:36 smithi174 ceph-mon[35635]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T20:58:36.815 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:36 smithi174 ceph-mon[35635]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T20:58:36.815 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:36.815 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:36 smithi174 ceph-mon[35635]: from='osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:58:36.815 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:36 smithi174 ceph-mon[35635]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:58:37.022 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:58:37.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:36 smithi104 ceph-mon[35325]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T20:58:37.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:36 smithi104 ceph-mon[35325]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T20:58:37.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:37.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:36 smithi104 ceph-mon[35325]: from='osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:58:37.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:36 smithi104 ceph-mon[35325]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:58:37.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:36 smithi033 ceph-mon[30860]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T20:58:37.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:36 smithi033 ceph-mon[30860]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T20:58:37.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:37.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:36 smithi033 ceph-mon[30860]: from='osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:58:37.144 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:36 smithi033 ceph-mon[30860]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 -- 172.21.15.33:0/3398130392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 msgr2=0x7f7450108320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 --2- 172.21.15.33:0/3398130392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450108320 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7444009900 tx=0x7f744402a410 comp rx=0 tx=0).stop 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 -- 172.21.15.33:0/3398130392 shutdown_connections 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 --2- 172.21.15.33:0/3398130392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450108860 0x7f745010ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 --2- 172.21.15.33:0/3398130392 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450108320 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 --2- 172.21.15.33:0/3398130392 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450100e60 0x7f7450101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 -- 172.21.15.33:0/3398130392 >> 172.21.15.33:0/3398130392 conn(0x7f74500fc970 msgr2=0x7f74500fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:37.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 -- 172.21.15.33:0/3398130392 shutdown_connections 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.483+0000 7f7455ebb640 1 -- 172.21.15.33:0/3398130392 wait complete. 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 Processor -- start 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 -- start start 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450100e60 0x7f7450198e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450108860 0x7f745019a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f745010e700 con 0x7f7450105f30 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f745010e580 con 0x7f7450108860 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f7455ebb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f745010e880 con 0x7f7450100e60 2024-09-10T20:58:37.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f744effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f744effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450199480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54404/0 (socket says 172.21.15.33:54404) 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f744effd640 1 -- 172.21.15.33:0/947209070 learned_addr learned my addr 172.21.15.33:0/947209070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.484+0000 7f744f7fe640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450100e60 0x7f7450198e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744effd640 1 -- 172.21.15.33:0/947209070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450100e60 msgr2=0x7f7450198e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744ffff640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450108860 0x7f745019a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744effd640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450100e60 0x7f7450198e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744effd640 1 -- 172.21.15.33:0/947209070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450108860 msgr2=0x7f745019a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744effd640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450108860 0x7f745019a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744effd640 1 -- 172.21.15.33:0/947209070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745019a960 con 0x7f7450105f30 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744ffff640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450108860 0x7f745019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744effd640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450199480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7444005940 tx=0x7f7444004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:37.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744cff9640 1 -- 172.21.15.33:0/947209070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744403f030 con 0x7f7450105f30 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745019abf0 con 0x7f7450105f30 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744cff9640 1 -- 172.21.15.33:0/947209070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7444003cb0 con 0x7f7450105f30 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f744cff9640 1 -- 172.21.15.33:0/947209070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444035070 con 0x7f7450105f30 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.485+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a1330 con 0x7f7450105f30 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.486+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74500fe460 con 0x7f7450105f30 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.487+0000 7f744cff9640 1 -- 172.21.15.33:0/947209070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f744403bbf0 con 0x7f7450105f30 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.488+0000 7f744cff9640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b52e0 0x7f74380b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.488+0000 7f744f7fe640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b52e0 0x7f74380b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:58:37.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.488+0000 7f744cff9640 1 -- 172.21.15.33:0/947209070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4253+0+0 (secure 0 0 0) 0x7f7444105590 con 0x7f7450105f30 2024-09-10T20:58:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.489+0000 7f744f7fe640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b52e0 0x7f74380b77a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7440009990 tx=0x7f7440009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:58:37.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.493+0000 7f744cff9640 1 -- 172.21.15.33:0/947209070 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f74440cd390 con 0x7f7450105f30 2024-09-10T20:58:37.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:37.688+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7450063590 con 0x7f74380b52e0 2024-09-10T20:58:37.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:37.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:58:37.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T20:58:37.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:37.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:38.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.114 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 138 MiB used, 447 GiB / 447 GiB avail 2024-09-10T20:58:38.114 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:38.115 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:39.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:58:39.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:58:39.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi174 2024-09-10T20:58:39.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-10T20:58:39.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] boot 2024-09-10T20:58:39.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T20:58:39.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:39.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:58:39.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:58:39.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:58:39.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:39.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:38 smithi104 ceph-mon[35325]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi174 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] boot 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:58:39.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:58:39.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:58:39.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:39.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:38 smithi174 ceph-mon[35635]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T20:58:39.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:58:39.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:58:39.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi174 2024-09-10T20:58:39.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi174 to 8544M 2024-09-10T20:58:39.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: osd.5 [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] boot 2024-09-10T20:58:39.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: osdmap e35: 6 total, 6 up, 6 in 2024-09-10T20:58:39.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T20:58:39.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: from='client.14484 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:58:39.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:58:39.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:58:39.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:39.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:38 smithi033 ceph-mon[30860]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T20:58:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:39 smithi033 ceph-mon[30860]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:39 smithi033 ceph-mon[30860]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T20:58:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:39 smithi104 ceph-mon[35325]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:39 smithi104 ceph-mon[35325]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T20:58:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:39 smithi174 ceph-mon[35635]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:39 smithi174 ceph-mon[35635]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T20:58:40.864 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:40 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1937705486' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03e0706e-6a19-42c7-97c4-54dbe8877a69"} : dispatch 2024-09-10T20:58:40.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:40 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1937705486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03e0706e-6a19-42c7-97c4-54dbe8877a69"}]': finished 2024-09-10T20:58:40.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:40 smithi033 ceph-mon[30860]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T20:58:40.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:58:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:40 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1937705486' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03e0706e-6a19-42c7-97c4-54dbe8877a69"} : dispatch 2024-09-10T20:58:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:40 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1937705486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03e0706e-6a19-42c7-97c4-54dbe8877a69"}]': finished 2024-09-10T20:58:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:40 smithi104 ceph-mon[35325]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T20:58:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:58:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:40 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1937705486' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "03e0706e-6a19-42c7-97c4-54dbe8877a69"} : dispatch 2024-09-10T20:58:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:40 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1937705486' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "03e0706e-6a19-42c7-97c4-54dbe8877a69"}]': finished 2024-09-10T20:58:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:40 smithi174 ceph-mon[35635]: osdmap e38: 7 total, 6 up, 7 in 2024-09-10T20:58:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:58:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:41 smithi104 ceph-mon[35325]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:41 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2332118216' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:58:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:41 smithi174 ceph-mon[35635]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:41 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2332118216' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:58:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:41 smithi033 ceph-mon[30860]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:41 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2332118216' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:58:44.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:43 smithi104 ceph-mon[35325]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:43 smithi174 ceph-mon[35635]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:43 smithi033 ceph-mon[30860]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 164 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:45 smithi033 ceph-mon[30860]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T20:58:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:45 smithi104 ceph-mon[35325]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T20:58:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:45 smithi174 ceph-mon[35635]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-10T20:58:48.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:47 smithi104 ceph-mon[35325]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T20:58:48.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:47 smithi033 ceph-mon[30860]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T20:58:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:47 smithi174 ceph-mon[35635]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T20:58:48.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:48 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T20:58:48.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:48 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:49.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:48 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T20:58:49.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:48 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:49.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:48 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T20:58:49.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:48 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:50.024 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:49 smithi033 ceph-mon[30860]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T20:58:50.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:49 smithi033 ceph-mon[30860]: Deploying daemon osd.6 on smithi033 2024-09-10T20:58:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:49 smithi104 ceph-mon[35325]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T20:58:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:49 smithi104 ceph-mon[35325]: Deploying daemon osd.6 on smithi033 2024-09-10T20:58:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:49 smithi174 ceph-mon[35635]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T20:58:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:49 smithi174 ceph-mon[35635]: Deploying daemon osd.6 on smithi033 2024-09-10T20:58:51.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:51 smithi033 ceph-mon[30860]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T20:58:52.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:51 smithi104 ceph-mon[35325]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T20:58:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:51 smithi174 ceph-mon[35635]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T20:58:53.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:53 smithi033 ceph-mon[30860]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T20:58:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:53 smithi104 ceph-mon[35325]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T20:58:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:53 smithi174 ceph-mon[35635]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T20:58:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:55 smithi104 ceph-mon[35325]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:55 smithi174 ceph-mon[35635]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:55 smithi033 ceph-mon[30860]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:58:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:57 smithi033 ceph-mon[30860]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:58.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.031 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.032 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:58.032 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:58.032 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:57 smithi104 ceph-mon[35325]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:58.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:58.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:57 smithi174 ceph-mon[35635]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:58:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:58:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:58.816 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 6 on host 'smithi033' 2024-09-10T20:58:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.812+0000 7f744cff9640 1 -- 172.21.15.33:0/947209070 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7450063590 con 0x7f74380b52e0 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.814+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b52e0 msgr2=0x7f74380b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.814+0000 7f7455ebb640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b52e0 0x7f74380b77a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7440009990 tx=0x7f7440009110 comp rx=0 tx=0).stop 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 msgr2=0x7f7450199480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450199480 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7444005940 tx=0x7f7444004430 comp rx=0 tx=0).stop 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 shutdown_connections 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b52e0 0x7f74380b77a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450108860 0x7f745019a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450105f30 0x7f7450199480 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 --2- 172.21.15.33:0/947209070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450100e60 0x7f7450198e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:58:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 >> 172.21.15.33:0/947209070 conn(0x7f74500fc970 msgr2=0x7f7450106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:58:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 shutdown_connections 2024-09-10T20:58:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:58:58.815+0000 7f7455ebb640 1 -- 172.21.15.33:0/947209070 wait complete. 2024-09-10T20:58:59.506 DEBUG:teuthology.orchestra.run.smithi033:osd.6> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T20:58:59.508 INFO:tasks.cephadm:Deploying osd.7 on smithi104 with /dev/vg_nvme/lv_2... 2024-09-10T20:58:59.508 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T20:58:59.764 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T20:58:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:59 smithi104 ceph-mon[35325]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:58:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:59 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:58:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:59 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:58:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:58:59 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:00.039 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:59 smithi033 ceph-mon[30860]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:00.039 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:59 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:00.039 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:59 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:00.039 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:58:59 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:59 smithi174 ceph-mon[35635]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:59 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:59 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:58:59 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:01.067 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 20:59:00 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T20:59:00.795+0000 7fa01ab8d640 -1 osd.6 0 log_to_monitors true 2024-09-10T20:59:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:01 smithi104 ceph-mon[35325]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:01.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:01 smithi104 ceph-mon[35325]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T20:59:02.126 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:01 smithi033 ceph-mon[30860]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:02.126 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:01 smithi033 ceph-mon[30860]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T20:59:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:01 smithi174 ceph-mon[35635]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:01 smithi174 ceph-mon[35635]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T20:59:02.547 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T20:59:02.574 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-09-10T20:59:02.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:02.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:02 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:02.826 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T20:59:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T20:59:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T20:59:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:59:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:03.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:02 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.164 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:03.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:03.165 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:02 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:03.296 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 -- 172.21.15.104:0/1067574068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 msgr2=0x7f080c1033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 --2- 172.21.15.104:0/1067574068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c1033e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009900 tx=0x7f07fc02a410 comp rx=0 tx=0).stop 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 -- 172.21.15.104:0/1067574068 shutdown_connections 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 --2- 172.21.15.104:0/1067574068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080c106520 0x7f080c1069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 --2- 172.21.15.104:0/1067574068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080c103970 0x7f080c105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 --2- 172.21.15.104:0/1067574068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c1033e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 -- 172.21.15.104:0/1067574068 >> 172.21.15.104:0/1067574068 conn(0x7f080c0fd1e0 msgr2=0x7f080c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 -- 172.21.15.104:0/1067574068 shutdown_connections 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.271+0000 7f0813537640 1 -- 172.21.15.104:0/1067574068 wait complete. 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 Processor -- start 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 -- start start 2024-09-10T20:59:03.297 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080c103970 0x7f080c19fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080c106520 0x7f080c1a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f080c114ce0 con 0x7f080c106520 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f080c114b60 con 0x7f080c06b6e0 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0813537640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f080c114e60 con 0x7f080c103970 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f08112d3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f08112d3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:52504/0 (socket says 172.21.15.104:52504) 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f08112d3640 1 -- 172.21.15.104:0/85314816 learned_addr learned my addr 172.21.15.104:0/85314816 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0810ad2640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080c103970 0x7f080c19fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f0811ad4640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080c106520 0x7f080c1a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.298 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f08112d3640 1 -- 172.21.15.104:0/85314816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080c103970 msgr2=0x7f080c19fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f08112d3640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080c103970 0x7f080c19fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f08112d3640 1 -- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080c106520 msgr2=0x7f080c1a06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.272+0000 7f08112d3640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080c106520 0x7f080c1a06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f08112d3640 1 -- 172.21.15.104:0/85314816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c1a0f70 con 0x7f080c06b6e0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f0811ad4640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080c106520 0x7f080c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f08112d3640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c19f440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07fc005940 tx=0x7f07fc004430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f08027fc640 1 -- 172.21.15.104:0/85314816 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc03f030 con 0x7f080c06b6e0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f08027fc640 1 -- 172.21.15.104:0/85314816 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07fc003cb0 con 0x7f080c06b6e0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f08027fc640 1 -- 172.21.15.104:0/85314816 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc035070 con 0x7f080c06b6e0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c1a1200 con 0x7f080c06b6e0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.273+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c1a77f0 con 0x7f080c06b6e0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.274+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c0fecd0 con 0x7f080c06b6e0 2024-09-10T20:59:03.299 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.276+0000 7f08027fc640 1 -- 172.21.15.104:0/85314816 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f07fc03b460 con 0x7f080c06b6e0 2024-09-10T20:59:03.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.277+0000 7f08027fc640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07f80b52e0 0x7f07f80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:03.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.277+0000 7f08027fc640 1 -- 172.21.15.104:0/85314816 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4685+0+0 (secure 0 0 0) 0x7f07fc105d00 con 0x7f080c06b6e0 2024-09-10T20:59:03.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.277+0000 7f0810ad2640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07f80b52e0 0x7f07f80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:03.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.278+0000 7f0810ad2640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07f80b52e0 0x7f07f80b77a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0808005fd0 tx=0x7f0808005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:03.300 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.280+0000 7f08027fc640 1 -- 172.21.15.104:0/85314816 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f07fc0cd9f0 con 0x7f080c06b6e0 2024-09-10T20:59:03.427 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T20:59:03.158+0000 7fa016b10640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T20:59:03.427 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T20:59:03.165+0000 7fa00e92f640 -1 osd.6 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:59:03.476 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:03.472+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f080c063590 con 0x7f07f80b52e0 2024-09-10T20:59:04.040 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:04.040 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi033 2024-09-10T20:59:04.040 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi033 to 5697M 2024-09-10T20:59:04.040 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:59:04.040 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T20:59:04.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:04.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:04.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' 2024-09-10T20:59:04.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:04.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:04.041 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi033 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi033 to 5697M 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:04.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi033 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi033 to 5697M 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: from='osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940]' entity='osd.6' 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:04.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:04.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:04 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:59:04.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:04 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:59:04.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:04 smithi104 ceph-mon[35325]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:04.911 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:04 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:04.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:04 smithi104 ceph-mon[35325]: osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] boot 2024-09-10T20:59:04.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:04 smithi104 ceph-mon[35325]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T20:59:04.912 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:04 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:04 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:59:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:04 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:59:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:04 smithi174 ceph-mon[35635]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:04 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:04 smithi174 ceph-mon[35635]: osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] boot 2024-09-10T20:59:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:04 smithi174 ceph-mon[35635]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T20:59:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:04 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:04 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:59:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:04 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:59:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:04 smithi033 ceph-mon[30860]: from='client.24353 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:04 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:04 smithi033 ceph-mon[30860]: osd.6 [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] boot 2024-09-10T20:59:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:04 smithi033 ceph-mon[30860]: osdmap e41: 7 total, 7 up, 7 in 2024-09-10T20:59:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:04 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T20:59:06.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:05 smithi104 ceph-mon[35325]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:05 smithi174 ceph-mon[35635]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:05 smithi033 ceph-mon[30860]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 165 MiB used, 536 GiB / 536 GiB avail 2024-09-10T20:59:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:06 smithi104 ceph-mon[35325]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T20:59:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:06 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/1317290991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"} : dispatch 2024-09-10T20:59:07.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:06 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"} : dispatch 2024-09-10T20:59:07.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:06 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"}]': finished 2024-09-10T20:59:07.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:06 smithi104 ceph-mon[35325]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T20:59:07.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:06 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:07.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:06 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/1253395159' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:07.078 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:06 smithi033 ceph-mon[30860]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T20:59:07.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:06 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/1317290991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"} : dispatch 2024-09-10T20:59:07.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:06 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"} : dispatch 2024-09-10T20:59:07.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:06 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"}]': finished 2024-09-10T20:59:07.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:06 smithi033 ceph-mon[30860]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T20:59:07.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:06 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:07.079 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:06 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/1253395159' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:06 smithi174 ceph-mon[35635]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T20:59:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:06 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/1317290991' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"} : dispatch 2024-09-10T20:59:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:06 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"} : dispatch 2024-09-10T20:59:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:06 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5f9abb9d-783e-4ded-93ea-4604eb6df315"}]': finished 2024-09-10T20:59:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:06 smithi174 ceph-mon[35635]: osdmap e43: 8 total, 7 up, 8 in 2024-09-10T20:59:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:06 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:06 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/1253395159' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:07 smithi104 ceph-mon[35325]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:07 smithi174 ceph-mon[35635]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:07 smithi033 ceph-mon[30860]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 195 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:10.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:09 smithi104 ceph-mon[35325]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:09 smithi174 ceph-mon[35635]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:09 smithi033 ceph-mon[30860]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:12.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:11 smithi104 ceph-mon[35325]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:11 smithi174 ceph-mon[35635]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:11 smithi033 ceph-mon[30860]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:13 smithi104 ceph-mon[35325]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:13 smithi174 ceph-mon[35635]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:13 smithi033 ceph-mon[30860]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:15.056 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T20:59:15.056 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:15.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T20:59:15.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T20:59:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:15.880 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:15 smithi104 ceph-mon[35325]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:15.880 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:15 smithi104 ceph-mon[35325]: Deploying daemon osd.7 on smithi104 2024-09-10T20:59:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:15 smithi174 ceph-mon[35635]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:15 smithi174 ceph-mon[35635]: Deploying daemon osd.7 on smithi104 2024-09-10T20:59:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:15 smithi033 ceph-mon[30860]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:15 smithi033 ceph-mon[30860]: Deploying daemon osd.7 on smithi104 2024-09-10T20:59:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:17 smithi104 ceph-mon[35325]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:17 smithi174 ceph-mon[35635]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:18.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:17 smithi033 ceph-mon[30860]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:20.053 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:19 smithi104 ceph-mon[35325]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:19 smithi174 ceph-mon[35635]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:19 smithi033 ceph-mon[30860]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:21 smithi104 ceph-mon[35325]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:21 smithi174 ceph-mon[35635]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:21 smithi033 ceph-mon[30860]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:23 smithi174 ceph-mon[35635]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:23 smithi033 ceph-mon[30860]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:24.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:23 smithi104 ceph-mon[35325]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:24.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:24.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:24.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:24.750 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.746+0000 7f08027fc640 1 -- 172.21.15.104:0/85314816 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f080c063590 con 0x7f07f80b52e0 2024-09-10T20:59:24.750 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 7 on host 'smithi104' 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07f80b52e0 msgr2=0x7f07f80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07f80b52e0 0x7f07f80b77a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0808005fd0 tx=0x7f0808005850 comp rx=0 tx=0).stop 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 msgr2=0x7f080c19f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c19f440 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f07fc005940 tx=0x7f07fc004430 comp rx=0 tx=0).stop 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 shutdown_connections 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07f80b52e0 0x7f07f80b77a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f080c106520 0x7f080c1a06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f080c103970 0x7f080c19fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 --2- 172.21.15.104:0/85314816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f080c06b6e0 0x7f080c19f440 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:24.753 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 >> 172.21.15.104:0/85314816 conn(0x7f080c0fd1e0 msgr2=0x7f080c1047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:24.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 shutdown_connections 2024-09-10T20:59:24.754 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T20:59:24.749+0000 7f0813537640 1 -- 172.21.15.104:0/85314816 wait complete. 2024-09-10T20:59:25.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:24 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:25.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:24 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:25.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:24 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:25.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:25.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:25.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:24 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:25.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:25.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:25.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:24 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:25.507 DEBUG:teuthology.orchestra.run.smithi104:osd.7> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T20:59:25.510 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-09-10T20:59:25.510 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T20:59:25.766 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:59:26.039 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:25 smithi104 ceph-mon[35325]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:26.133 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:25 smithi174 ceph-mon[35635]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:25 smithi033 ceph-mon[30860]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:27.231 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 20:59:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T20:59:27.052+0000 7f57d0713640 -1 osd.7 0 log_to_monitors true 2024-09-10T20:59:27.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:27 smithi104 ceph-mon[35325]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:27.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:27 smithi104 ceph-mon[35325]: from='osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T20:59:27.983 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:27 smithi104 ceph-mon[35325]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T20:59:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:27 smithi174 ceph-mon[35635]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:27 smithi174 ceph-mon[35635]: from='osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T20:59:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:27 smithi174 ceph-mon[35635]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T20:59:28.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:27 smithi033 ceph-mon[30860]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:28.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:27 smithi033 ceph-mon[30860]: from='osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T20:59:28.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:27 smithi033 ceph-mon[30860]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T20:59:28.504 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T20:59:28.533 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-09-10T20:59:28.786 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T20:59:28.969 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T20:59:28.969 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:28.970 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:28.971 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:28 smithi174 ceph-mon[35635]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:59:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T20:59:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T20:59:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:59:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:59:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:28 smithi033 ceph-mon[30860]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: osdmap e44: 8 total, 7 up, 8 in 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:29.214 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:29.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:29.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:29.215 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:28 smithi104 ceph-mon[35325]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/515752395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c108750 msgr2=0x7f1f9c10ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.355 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/515752395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c108750 0x7f1f9c10ac10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009ca0 tx=0x7f1f9402a8c0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/515752395 shutdown_connections 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/515752395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c108750 0x7f1f9c10ac10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/515752395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f9c105bc0 0x7f1f9c108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/515752395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c06c0a0 0x7f1f9c105680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/515752395 >> 172.21.15.174:0/515752395 conn(0x7f1f9c0fd1f0 msgr2=0x7f1f9c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/515752395 shutdown_connections 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.327+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/515752395 wait complete. 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 Processor -- start 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 -- start start 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f9c06c0a0 0x7f1f9c198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.356 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c105bc0 0x7f1f9c1993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c108750 0x7f1f9c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10e710 con 0x7f1f9c108750 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10e590 con 0x7f1f9c06c0a0 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1fa1a3e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9c10e890 con 0x7f1f9c105bc0 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c105bc0 0x7f1f9c1993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1f9a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c105bc0 0x7f1f9c1993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43432/0 (socket says 172.21.15.174:43432) 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1f9a7fc640 1 -- 172.21.15.174:0/2331360647 learned_addr learned my addr 172.21.15.174:0/2331360647 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1f9affd640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f9c06c0a0 0x7f1f9c198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.328+0000 7f1f9b7fe640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c108750 0x7f1f9c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1f9a7fc640 1 -- 172.21.15.174:0/2331360647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f9c06c0a0 msgr2=0x7f1f9c198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.357 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1f9a7fc640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f9c06c0a0 0x7f1f9c198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1f9a7fc640 1 -- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c108750 msgr2=0x7f1f9c19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1f9a7fc640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c108750 0x7f1f9c19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1f9a7fc640 1 -- 172.21.15.174:0/2331360647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9c19a8a0 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1f9b7fe640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c108750 0x7f1f9c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1f9a7fc640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c105bc0 0x7f1f9c1993c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000c920 tx=0x7f1f9000cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1fa0a3c640 1 -- 172.21.15.174:0/2331360647 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90007bb0 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1fa0a3c640 1 -- 172.21.15.174:0/2331360647 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f90007d50 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9c19ab90 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1fa0a3c640 1 -- 172.21.15.174:0/2331360647 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90018730 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.329+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9c1a1270 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.330+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f9c0fece0 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.332+0000 7f1fa0a3c640 1 -- 172.21.15.174:0/2331360647 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f1f9000f450 con 0x7f1f9c105bc0 2024-09-10T20:59:29.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.333+0000 7f1fa0a3c640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f840b52e0 0x7f1f840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.333+0000 7f1fa0a3c640 1 -- 172.21.15.174:0/2331360647 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5117+0+0 (secure 0 0 0) 0x7f1f900e5350 con 0x7f1f9c105bc0 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.333+0000 7f1f9affd640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f840b52e0 0x7f1f840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.333+0000 7f1f9affd640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f840b52e0 0x7f1f840b77a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009990 tx=0x7f1f8c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:29.359 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.335+0000 7f1fa0a3c640 1 -- 172.21.15.174:0/2331360647 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1f90014030 con 0x7f1f9c105bc0 2024-09-10T20:59:29.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:29.527+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1f9c063590 con 0x7f1f840b52e0 2024-09-10T20:59:29.562 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T20:59:29.208+0000 7f57cceaa640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T20:59:29.562 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T20:59:29.218+0000 7f57c44b5640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:59:30.125 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:30.125 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi104 2024-09-10T20:59:30.125 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-10T20:59:30.125 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T20:59:30.125 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.125 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.126 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='osd.7 ' entity='osd.7' 2024-09-10T20:59:30.126 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:30.126 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:30.126 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:30.126 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:30.126 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi104 2024-09-10T20:59:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-10T20:59:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T20:59:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='osd.7 ' entity='osd.7' 2024-09-10T20:59:30.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:30.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:30.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:30.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:30.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:30.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi104 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi104 to 5696M 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='osd.7 ' entity='osd.7' 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:30.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:31.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:30 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:59:31.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:30 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:59:31.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:30 smithi174 ceph-mon[35635]: osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] boot 2024-09-10T20:59:31.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:30 smithi174 ceph-mon[35635]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T20:59:31.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:30 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:31.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:30 smithi174 ceph-mon[35635]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:31.020 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:30 smithi174 ceph-mon[35635]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T20:59:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:30 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:59:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:30 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:59:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:30 smithi033 ceph-mon[30860]: osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] boot 2024-09-10T20:59:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:30 smithi033 ceph-mon[30860]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T20:59:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:30 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:30 smithi033 ceph-mon[30860]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:31.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:30 smithi033 ceph-mon[30860]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T20:59:31.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:30 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:59:31.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:30 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:59:31.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:30 smithi104 ceph-mon[35325]: osd.7 [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] boot 2024-09-10T20:59:31.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:30 smithi104 ceph-mon[35325]: osdmap e46: 8 total, 8 up, 8 in 2024-09-10T20:59:31.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:30 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T20:59:31.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:30 smithi104 ceph-mon[35325]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 196 MiB used, 626 GiB / 626 GiB avail 2024-09-10T20:59:31.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:30 smithi104 ceph-mon[35325]: osdmap e47: 8 total, 8 up, 8 in 2024-09-10T20:59:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T20:59:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/4101383925' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"} : dispatch 2024-09-10T20:59:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"} : dispatch 2024-09-10T20:59:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"}]': finished 2024-09-10T20:59:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T20:59:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:33.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:32 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/388454636' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T20:59:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/4101383925' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"} : dispatch 2024-09-10T20:59:33.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"} : dispatch 2024-09-10T20:59:33.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"}]': finished 2024-09-10T20:59:33.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T20:59:33.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:33.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:32 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/388454636' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:33.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T20:59:33.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:33.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/4101383925' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"} : dispatch 2024-09-10T20:59:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"} : dispatch 2024-09-10T20:59:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69"}]': finished 2024-09-10T20:59:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: osdmap e49: 9 total, 8 up, 9 in 2024-09-10T20:59:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:33.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:32 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/388454636' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:35 smithi174 ceph-mon[35635]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:35 smithi033 ceph-mon[30860]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:35.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:35 smithi104 ceph-mon[35325]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:37.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:37 smithi174 ceph-mon[35635]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T20:59:37.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:37 smithi033 ceph-mon[30860]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T20:59:37.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:37 smithi104 ceph-mon[35325]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T20:59:39.357 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:39 smithi174 ceph-mon[35635]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T20:59:39.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:39 smithi033 ceph-mon[30860]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T20:59:39.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:39 smithi104 ceph-mon[35325]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-10T20:59:41.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:41 smithi174 ceph-mon[35635]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T20:59:41.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T20:59:41.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:41.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:41 smithi174 ceph-mon[35635]: Deploying daemon osd.8 on smithi174 2024-09-10T20:59:41.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:41 smithi033 ceph-mon[30860]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T20:59:41.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T20:59:41.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:41.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:41 smithi033 ceph-mon[30860]: Deploying daemon osd.8 on smithi174 2024-09-10T20:59:41.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:41 smithi104 ceph-mon[35325]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-10T20:59:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-10T20:59:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:41 smithi104 ceph-mon[35325]: Deploying daemon osd.8 on smithi174 2024-09-10T20:59:43.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:43 smithi174 ceph-mon[35635]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T20:59:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:43 smithi033 ceph-mon[30860]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T20:59:43.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:43 smithi104 ceph-mon[35325]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T20:59:45.288 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:45 smithi174 ceph-mon[35635]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:59:45.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:45 smithi033 ceph-mon[30860]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:59:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:45 smithi104 ceph-mon[35325]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:59:47.358 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:47 smithi174 ceph-mon[35635]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:59:47.358 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:47.358 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:47.358 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:47.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:47 smithi033 ceph-mon[30860]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:59:47.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:47.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:47.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:47 smithi104 ceph-mon[35325]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T20:59:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:49.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:49 smithi174 ceph-mon[35635]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:49 smithi033 ceph-mon[30860]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:49.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:49 smithi104 ceph-mon[35325]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:50.400 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-09-10T20:59:50.401 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.395+0000 7f1fa0a3c640 1 -- 172.21.15.174:0/2331360647 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1f9c063590 con 0x7f1f840b52e0 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f840b52e0 msgr2=0x7f1f840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f840b52e0 0x7f1f840b77a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009990 tx=0x7f1f8c009110 comp rx=0 tx=0).stop 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c105bc0 msgr2=0x7f1f9c1993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c105bc0 0x7f1f9c1993c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f9000c920 tx=0x7f1f9000cd20 comp rx=0 tx=0).stop 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 shutdown_connections 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f840b52e0 0x7f1f840b77a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f9c108750 0x7f1f9c19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f9c105bc0 0x7f1f9c1993c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 --2- 172.21.15.174:0/2331360647 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f9c06c0a0 0x7f1f9c198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:50.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 >> 172.21.15.174:0/2331360647 conn(0x7f1f9c0fd1f0 msgr2=0x7f1f9c103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:50.404 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 shutdown_connections 2024-09-10T20:59:50.404 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T20:59:50.398+0000 7f1fa1a3e640 1 -- 172.21.15.174:0/2331360647 wait complete. 2024-09-10T20:59:50.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:50 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:50 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:50 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:50 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:50.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:50 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:50 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:50 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:50 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:50 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:50 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:50 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:50 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:50.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:50 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:50.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:50 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:50.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:50 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:51.097 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.8.service 2024-09-10T20:59:51.100 INFO:tasks.cephadm:Deploying osd.9 on smithi033 with /dev/vg_nvme/lv_1... 2024-09-10T20:59:51.100 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T20:59:51.369 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:59:51.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:51 smithi033 ceph-mon[30860]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:51.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:51.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:51.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:51 smithi174 ceph-mon[35635]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:51 smithi104 ceph-mon[35325]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T20:59:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:52.568 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 20:59:52 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T20:59:52.361+0000 7f5b10c16640 -1 osd.8 0 log_to_monitors true 2024-09-10T20:59:53.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:53 smithi033 ceph-mon[30860]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:53.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:53 smithi033 ceph-mon[30860]: from='osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T20:59:53.472 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:53 smithi033 ceph-mon[30860]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T20:59:53.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:53 smithi174 ceph-mon[35635]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:53.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:53 smithi174 ceph-mon[35635]: from='osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T20:59:53.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:53 smithi174 ceph-mon[35635]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T20:59:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:53 smithi104 ceph-mon[35325]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:53.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:53 smithi104 ceph-mon[35325]: from='osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T20:59:53.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:53 smithi104 ceph-mon[35325]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-10T20:59:54.272 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T20:59:54.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_1 2024-09-10T20:59:54.563 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T20:59:54.644 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T20:59:54.383+0000 7f5b0cb99640 -1 osd.8 0 waiting for initial osdmap 2024-09-10T20:59:54.644 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T20:59:54.392+0000 7f5b089c0640 -1 osd.8 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T20:59:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:54 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T20:59:54.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T20:59:54.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:54.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:59:54.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:54.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:54 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-10T20:59:54.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: osdmap e50: 9 total, 8 up, 9 in 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T20:59:54.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:54 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.199+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1978121013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8104560 msgr2=0x7fd5c810aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.199+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1978121013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8104560 0x7fd5c810aee0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009cd0 tx=0x7fd5c002a8e0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1978121013 shutdown_connections 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1978121013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8104560 0x7fd5c810aee0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1978121013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5c8103ba0 0x7fd5c8104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1978121013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c8102a40 0x7fd5c8102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1978121013 >> 172.21.15.33:0/1978121013 conn(0x7fd5c80fe2f0 msgr2=0x7fd5c8100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1978121013 shutdown_connections 2024-09-10T20:59:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1978121013 wait complete. 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 Processor -- start 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.200+0000 7fd5cf4d8640 1 -- start start 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cf4d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8102a40 0x7fd5c8198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cf4d8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5c8103ba0 0x7fd5c8199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cf4d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c8104560 0x7fd5c819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cf4d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c810e6a0 con 0x7fd5c8102a40 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cf4d8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c810e520 con 0x7fd5c8103ba0 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cf4d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c810e820 con 0x7fd5c8104560 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8102a40 0x7fd5c8198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8102a40 0x7fd5c8198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50504/0 (socket says 172.21.15.33:50504) 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 -- 172.21.15.33:0/1841295145 learned_addr learned my addr 172.21.15.33:0/1841295145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T20:59:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cca73640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5c8103ba0 0x7fd5c8199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cda75640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c8104560 0x7fd5c819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 -- 172.21.15.33:0/1841295145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c8104560 msgr2=0x7fd5c819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c8104560 0x7fd5c819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 -- 172.21.15.33:0/1841295145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5c8103ba0 msgr2=0x7fd5c8199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5c8103ba0 0x7fd5c8199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 -- 172.21.15.33:0/1841295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c819a930 con 0x7fd5c8102a40 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cca73640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5c8103ba0 0x7fd5c8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cda75640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c8104560 0x7fd5c819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5cd274640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8102a40 0x7fd5c8198e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d950 tx=0x7fd5c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.201+0000 7fd5be7fc640 1 -- 172.21.15.33:0/1841295145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4017070 con 0x7fd5c8102a40 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.202+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c819ac20 con 0x7fd5c8102a40 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.202+0000 7fd5be7fc640 1 -- 172.21.15.33:0/1841295145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5c4009c30 con 0x7fd5c8102a40 2024-09-10T20:59:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.202+0000 7fd5be7fc640 1 -- 172.21.15.33:0/1841295145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400f620 con 0x7fd5c8102a40 2024-09-10T20:59:55.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.202+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c81a1300 con 0x7fd5c8102a40 2024-09-10T20:59:55.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.203+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c8109cf0 con 0x7fd5c8102a40 2024-09-10T20:59:55.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.204+0000 7fd5be7fc640 1 -- 172.21.15.33:0/1841295145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fd5c4010460 con 0x7fd5c8102a40 2024-09-10T20:59:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.205+0000 7fd5be7fc640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5b40b52e0 0x7fd5b40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T20:59:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.205+0000 7fd5cca73640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5b40b52e0 0x7fd5b40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T20:59:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.205+0000 7fd5be7fc640 1 -- 172.21.15.33:0/1841295145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5549+0+0 (secure 0 0 0) 0x7fd5c40e4ef0 con 0x7fd5c8102a40 2024-09-10T20:59:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.205+0000 7fd5cca73640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5b40b52e0 0x7fd5b40b77a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009c20 tx=0x7fd5b8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T20:59:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.209+0000 7fd5be7fc640 1 -- 172.21.15.33:0/1841295145 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd5c40ac7e0 con 0x7fd5c8102a40 2024-09-10T20:59:55.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T20:59:55.406+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd5c8063590 con 0x7fd5b40b52e0 2024-09-10T20:59:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T20:59:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T20:59:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi174 2024-09-10T20:59:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-10T20:59:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:59:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T20:59:55.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:55.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T20:59:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T20:59:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi174 2024-09-10T20:59:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-10T20:59:55.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:59:55.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T20:59:55.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:55.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi174 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi174 to 5696M 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: osdmap e51: 9 total, 8 up, 9 in 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:55.714 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] boot 2024-09-10T20:59:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: osdmap e52: 9 total, 9 up, 9 in 2024-09-10T20:59:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:56.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:56.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:56.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] boot 2024-09-10T20:59:56.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: osdmap e52: 9 total, 9 up, 9 in 2024-09-10T20:59:56.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:56.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:56.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:56.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:56.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:56.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:56.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: osd.8 [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] boot 2024-09-10T20:59:56.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: osdmap e52: 9 total, 9 up, 9 in 2024-09-10T20:59:56.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-10T20:59:56.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T20:59:56.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T20:59:56.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T20:59:56.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T20:59:57.454 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:57 smithi033 ceph-mon[30860]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:57.455 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:57 smithi033 ceph-mon[30860]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T20:59:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:57 smithi104 ceph-mon[35325]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:57 smithi104 ceph-mon[35325]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T20:59:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:57 smithi174 ceph-mon[35635]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-10T20:59:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:57 smithi174 ceph-mon[35635]: osdmap e53: 9 total, 9 up, 9 in 2024-09-10T20:59:58.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:58 smithi033 ceph-mon[30860]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T20:59:58.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/444914114' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "dbae656c-994b-4a32-a22d-de19664248e9"} : dispatch 2024-09-10T20:59:58.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/444914114' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbae656c-994b-4a32-a22d-de19664248e9"}]': finished 2024-09-10T20:59:58.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:58 smithi033 ceph-mon[30860]: osdmap e55: 10 total, 9 up, 10 in 2024-09-10T20:59:58.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T20:59:58.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:58 smithi104 ceph-mon[35325]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T20:59:58.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/444914114' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "dbae656c-994b-4a32-a22d-de19664248e9"} : dispatch 2024-09-10T20:59:58.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/444914114' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbae656c-994b-4a32-a22d-de19664248e9"}]': finished 2024-09-10T20:59:58.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:58 smithi104 ceph-mon[35325]: osdmap e55: 10 total, 9 up, 10 in 2024-09-10T20:59:58.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T20:59:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:58 smithi174 ceph-mon[35635]: osdmap e54: 9 total, 9 up, 9 in 2024-09-10T20:59:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/444914114' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "dbae656c-994b-4a32-a22d-de19664248e9"} : dispatch 2024-09-10T20:59:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/444914114' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dbae656c-994b-4a32-a22d-de19664248e9"}]': finished 2024-09-10T20:59:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:58 smithi174 ceph-mon[35635]: osdmap e55: 10 total, 9 up, 10 in 2024-09-10T20:59:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T20:59:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:59 smithi033 ceph-mon[30860]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T20:59:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 20:59:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/67615549' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:59 smithi104 ceph-mon[35325]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T20:59:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 20:59:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/67615549' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T20:59:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:59 smithi174 ceph-mon[35635]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T20:59:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 20:59:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/67615549' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:00:00.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:00 smithi033 ceph-mon[30860]: overall HEALTH_OK 2024-09-10T21:00:00.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:00 smithi104 ceph-mon[35325]: overall HEALTH_OK 2024-09-10T21:00:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:00 smithi174 ceph-mon[35635]: overall HEALTH_OK 2024-09-10T21:00:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:01 smithi033 ceph-mon[30860]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:01 smithi104 ceph-mon[35325]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:01 smithi174 ceph-mon[35635]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:03 smithi033 ceph-mon[30860]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:00:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:03 smithi104 ceph-mon[35325]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:00:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:03 smithi174 ceph-mon[35635]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T21:00:05.619 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:05 smithi033 ceph-mon[30860]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T21:00:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:05 smithi104 ceph-mon[35325]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T21:00:05.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:05 smithi174 ceph-mon[35635]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T21:00:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:07 smithi033 ceph-mon[30860]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-10T21:00:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T21:00:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:07 smithi033 ceph-mon[30860]: Deploying daemon osd.9 on smithi033 2024-09-10T21:00:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:07 smithi104 ceph-mon[35325]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-10T21:00:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T21:00:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:07 smithi104 ceph-mon[35325]: Deploying daemon osd.9 on smithi033 2024-09-10T21:00:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:07 smithi174 ceph-mon[35635]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-10T21:00:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-10T21:00:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:07 smithi174 ceph-mon[35635]: Deploying daemon osd.9 on smithi033 2024-09-10T21:00:09.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:09 smithi033 ceph-mon[30860]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:00:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:09 smithi104 ceph-mon[35325]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:00:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:09 smithi174 ceph-mon[35635]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T21:00:11.786 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:11 smithi033 ceph-mon[30860]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:00:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:11 smithi104 ceph-mon[35325]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:00:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:11 smithi174 ceph-mon[35635]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:00:13.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:13 smithi033 ceph-mon[30860]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:00:13.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:13.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:13.611 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:13 smithi104 ceph-mon[35325]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:00:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:13.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:13.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:13 smithi174 ceph-mon[35635]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:00:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:15.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:15 smithi033 ceph-mon[30860]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:15 smithi104 ceph-mon[35325]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:15 smithi174 ceph-mon[35635]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:16.376 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 9 on host 'smithi033' 2024-09-10T21:00:16.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.372+0000 7fd5be7fc640 1 -- 172.21.15.33:0/1841295145 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd5c8063590 con 0x7fd5b40b52e0 2024-09-10T21:00:16.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5b40b52e0 msgr2=0x7fd5b40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5b40b52e0 0x7fd5b40b77a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009c20 tx=0x7fd5b8005f90 comp rx=0 tx=0).stop 2024-09-10T21:00:16.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8102a40 msgr2=0x7fd5c8198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:16.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8102a40 0x7fd5c8198e00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d950 tx=0x7fd5c400de10 comp rx=0 tx=0).stop 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 shutdown_connections 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5b40b52e0 0x7fd5b40b77a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5c8104560 0x7fd5c819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5c8103ba0 0x7fd5c8199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 --2- 172.21.15.33:0/1841295145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5c8102a40 0x7fd5c8198e00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 >> 172.21.15.33:0/1841295145 conn(0x7fd5c80fe2f0 msgr2=0x7fd5c8106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 shutdown_connections 2024-09-10T21:00:16.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:00:16.373+0000 7fd5cf4d8640 1 -- 172.21.15.33:0/1841295145 wait complete. 2024-09-10T21:00:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:16.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:17.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:17.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:17.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:17.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:17.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:17.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:17.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:17.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:17.065 DEBUG:teuthology.orchestra.run.smithi033:osd.9> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.9.service 2024-09-10T21:00:17.067 INFO:tasks.cephadm:Deploying osd.10 on smithi104 with /dev/vg_nvme/lv_1... 2024-09-10T21:00:17.068 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T21:00:17.324 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:00:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:17 smithi033 ceph-mon[30860]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:17.705 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:17 smithi104 ceph-mon[35325]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:17 smithi174 ceph-mon[35635]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:18.928 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:00:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:00:18.626+0000 7f78e1705640 -1 osd.9 0 log_to_monitors true 2024-09-10T21:00:19.860 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:19 smithi104 ceph-mon[35325]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:19.861 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:19 smithi104 ceph-mon[35325]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:00:19.874 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:19 smithi033 ceph-mon[30860]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:19.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:19 smithi033 ceph-mon[30860]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:00:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:19 smithi174 ceph-mon[35635]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:19 smithi174 ceph-mon[35635]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-10T21:00:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-09-10T21:00:20.073 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_1 2024-09-10T21:00:20.329 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:00:20.729 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:20.730 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 -- 172.21.15.104:0/741643812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb90103460 msgr2=0x7ffb90109fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/741643812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb90103460 0x7ffb90109fb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009ca0 tx=0x7ffb8802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 -- 172.21.15.104:0/741643812 shutdown_connections 2024-09-10T21:00:20.829 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/741643812 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb90103460 0x7ffb90109fb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/741643812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb901028f0 0x7ffb90102d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/741643812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb90101fb0 0x7ffb901023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 -- 172.21.15.104:0/741643812 >> 172.21.15.104:0/741643812 conn(0x7ffb900fc970 msgr2=0x7ffb900fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 -- 172.21.15.104:0/741643812 shutdown_connections 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.808+0000 7ffb96e6f640 1 -- 172.21.15.104:0/741643812 wait complete. 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 Processor -- start 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 -- start start 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb90101fb0 0x7ffb90197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb901028f0 0x7ffb901981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb90103460 0x7ffb90198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010d4e0 con 0x7ffb90103460 2024-09-10T21:00:20.830 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010d360 con 0x7ffb901028f0 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb96e6f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb9010d660 con 0x7ffb90101fb0 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb8ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb901028f0 0x7ffb901981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb8ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb901028f0 0x7ffb901981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:35282/0 (socket says 172.21.15.104:35282) 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.809+0000 7ffb8ffff640 1 -- 172.21.15.104:0/2022367265 learned_addr learned my addr 172.21.15.104:0/2022367265 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb94c0b640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb90101fb0 0x7ffb90197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb9540c640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb90103460 0x7ffb90198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8ffff640 1 -- 172.21.15.104:0/2022367265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb90101fb0 msgr2=0x7ffb90197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8ffff640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb90101fb0 0x7ffb90197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8ffff640 1 -- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb90103460 msgr2=0x7ffb90198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8ffff640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb90103460 0x7ffb90198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:20.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8ffff640 1 -- 172.21.15.104:0/2022367265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb90199650 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8ffff640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb901028f0 0x7ffb901981e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d950 tx=0x7ffb7c00de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8dffb640 1 -- 172.21.15.104:0/2022367265 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c017030 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8dffb640 1 -- 172.21.15.104:0/2022367265 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb7c004020 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb90199940 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb8dffb640 1 -- 172.21.15.104:0/2022367265 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c0047a0 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.810+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901a0020 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.811+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb90108e30 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.812+0000 7ffb8dffb640 1 -- 172.21.15.104:0/2022367265 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ffb7c009a30 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.813+0000 7ffb8dffb640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffb800b52e0 0x7ffb800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.813+0000 7ffb8dffb640 1 -- 172.21.15.104:0/2022367265 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5981+0+0 (secure 0 0 0) 0x7ffb7c0e4dc0 con 0x7ffb901028f0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.813+0000 7ffb94c0b640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffb800b52e0 0x7ffb800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:20.832 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.814+0000 7ffb94c0b640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffb800b52e0 0x7ffb800b77a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005e10 tx=0x7ffb84005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:20.833 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:20.817+0000 7ffb8dffb640 1 -- 172.21.15.104:0/2022367265 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ffb7c0ac5a0 con 0x7ffb901028f0 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.928 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:00:20.626+0000 7f78dd688640 -1 osd.9 0 waiting for initial osdmap 2024-09-10T21:00:20.928 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:00:20.637+0000 7f78d94af640 -1 osd.9 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:00:20.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-10T21:00:20.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: osdmap e56: 10 total, 9 up, 10 in 2024-09-10T21:00:20.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]} : dispatch 2024-09-10T21:00:20.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:20.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:20.929 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:21.007 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:21.003+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffb90063590 con 0x7ffb800b52e0 2024-09-10T21:00:21.749 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi033 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi033 to 4273M 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:00:21.750 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:00:21.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:21.751 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T21:00:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T21:00:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi033 2024-09-10T21:00:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi033 to 4273M 2024-09-10T21:00:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T21:00:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T21:00:21.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:21.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:21.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:00:21.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:00:21.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:21.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi033 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi033 to 4273M 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: from='osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: osdmap e57: 10 total, 9 up, 10 in 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:22.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:00:22.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:00:22.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:22.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:22 smithi033 ceph-mon[30860]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:00:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:22 smithi033 ceph-mon[30860]: osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] boot 2024-09-10T21:00:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:22 smithi033 ceph-mon[30860]: osdmap e58: 10 total, 10 up, 10 in 2024-09-10T21:00:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:22 smithi104 ceph-mon[35325]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:00:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:22 smithi104 ceph-mon[35325]: osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] boot 2024-09-10T21:00:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:22 smithi104 ceph-mon[35325]: osdmap e58: 10 total, 10 up, 10 in 2024-09-10T21:00:22.948 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:22 smithi174 ceph-mon[35635]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:00:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:22 smithi174 ceph-mon[35635]: osd.9 [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] boot 2024-09-10T21:00:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:22 smithi174 ceph-mon[35635]: osdmap e58: 10 total, 10 up, 10 in 2024-09-10T21:00:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-10T21:00:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:23 smithi033 ceph-mon[30860]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:23 smithi033 ceph-mon[30860]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T21:00:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:23 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/4093573877' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"} : dispatch 2024-09-10T21:00:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:23 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"} : dispatch 2024-09-10T21:00:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:23 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"}]': finished 2024-09-10T21:00:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:23 smithi033 ceph-mon[30860]: osdmap e60: 11 total, 10 up, 11 in 2024-09-10T21:00:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:24.013 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:23 smithi104 ceph-mon[35325]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:24.013 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:23 smithi104 ceph-mon[35325]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T21:00:24.013 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:23 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/4093573877' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"} : dispatch 2024-09-10T21:00:24.013 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:23 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"} : dispatch 2024-09-10T21:00:24.013 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:23 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"}]': finished 2024-09-10T21:00:24.013 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:23 smithi104 ceph-mon[35325]: osdmap e60: 11 total, 10 up, 11 in 2024-09-10T21:00:24.013 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:23 smithi174 ceph-mon[35635]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 255 MiB used, 804 GiB / 805 GiB avail 2024-09-10T21:00:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:23 smithi174 ceph-mon[35635]: osdmap e59: 10 total, 10 up, 10 in 2024-09-10T21:00:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:23 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/4093573877' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"} : dispatch 2024-09-10T21:00:24.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:23 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"} : dispatch 2024-09-10T21:00:24.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:23 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f020be25-441a-4f54-8077-3d0ea98e1181"}]': finished 2024-09-10T21:00:24.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:23 smithi174 ceph-mon[35635]: osdmap e60: 11 total, 10 up, 11 in 2024-09-10T21:00:24.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:24.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:24 smithi033 ceph-mon[30860]: from='client.? 172.21.15.104:0/1702442127' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:00:25.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:24 smithi174 ceph-mon[35635]: from='client.? 172.21.15.104:0/1702442127' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:00:25.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:24 smithi104 ceph-mon[35325]: from='client.? 172.21.15.104:0/1702442127' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:00:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:25 smithi033 ceph-mon[30860]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:25 smithi104 ceph-mon[35325]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:25 smithi174 ceph-mon[35635]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:27 smithi104 ceph-mon[35325]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:27 smithi033 ceph-mon[30860]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:27 smithi174 ceph-mon[35635]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:29 smithi033 ceph-mon[30860]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:30.020 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:29 smithi104 ceph-mon[35325]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:30.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:29 smithi174 ceph-mon[35635]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:31 smithi033 ceph-mon[30860]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T21:00:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:31.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:31 smithi104 ceph-mon[35325]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:31.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T21:00:31.978 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:31 smithi174 ceph-mon[35635]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-10T21:00:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:32 smithi104 ceph-mon[35325]: Deploying daemon osd.10 on smithi104 2024-09-10T21:00:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:32 smithi174 ceph-mon[35635]: Deploying daemon osd.10 on smithi104 2024-09-10T21:00:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:32 smithi033 ceph-mon[30860]: Deploying daemon osd.10 on smithi104 2024-09-10T21:00:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:33 smithi104 ceph-mon[35325]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:33 smithi174 ceph-mon[35635]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:33 smithi033 ceph-mon[30860]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:36.057 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:35 smithi104 ceph-mon[35325]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:35 smithi174 ceph-mon[35635]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:35 smithi033 ceph-mon[30860]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:37.943 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:37 smithi104 ceph-mon[35325]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:37.943 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:37.943 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:37.943 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:37.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:37 smithi033 ceph-mon[30860]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:37.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:37.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:37.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:37 smithi174 ceph-mon[35635]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:39.952 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:39 smithi104 ceph-mon[35325]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:39 smithi174 ceph-mon[35635]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:40.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:39 smithi033 ceph-mon[30860]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:41.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.083+0000 7ffb8dffb640 1 -- 172.21.15.104:0/2022367265 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ffb90063590 con 0x7ffb800b52e0 2024-09-10T21:00:41.088 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 10 on host 'smithi104' 2024-09-10T21:00:41.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffb800b52e0 msgr2=0x7ffb800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:41.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffb800b52e0 0x7ffb800b77a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005e10 tx=0x7ffb84005da0 comp rx=0 tx=0).stop 2024-09-10T21:00:41.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb901028f0 msgr2=0x7ffb901981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:41.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb901028f0 0x7ffb901981e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00d950 tx=0x7ffb7c00de10 comp rx=0 tx=0).stop 2024-09-10T21:00:41.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 shutdown_connections 2024-09-10T21:00:41.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffb800b52e0 0x7ffb800b77a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:41.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb90103460 0x7ffb90198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:41.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb901028f0 0x7ffb901981e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:41.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 --2- 172.21.15.104:0/2022367265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb90101fb0 0x7ffb90197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:41.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 >> 172.21.15.104:0/2022367265 conn(0x7ffb900fc970 msgr2=0x7ffb90106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:41.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 shutdown_connections 2024-09-10T21:00:41.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-09-10T21:00:41.086+0000 7ffb96e6f640 1 -- 172.21.15.104:0/2022367265 wait complete. 2024-09-10T21:00:41.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:41.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:00:41.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:41.764 DEBUG:teuthology.orchestra.run.smithi104:osd.10> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.10.service 2024-09-10T21:00:41.767 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-09-10T21:00:41.767 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-10T21:00:42.023 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:00:43.373 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:43 smithi104 ceph-mon[35325]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:43.564 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:43 smithi174 ceph-mon[35635]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:43.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:43 smithi033 ceph-mon[30860]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:43.812 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:00:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:00:43.380+0000 7fdea17e0640 -1 osd.10 0 log_to_monitors true 2024-09-10T21:00:44.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:44 smithi174 ceph-mon[35635]: from='osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:00:44.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:44 smithi174 ceph-mon[35635]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:00:44.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:44 smithi104 ceph-mon[35325]: from='osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:00:44.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:44 smithi104 ceph-mon[35325]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:00:44.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:44 smithi033 ceph-mon[30860]: from='osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:00:44.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:44 smithi033 ceph-mon[30860]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-10T21:00:44.917 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-09-10T21:00:44.943 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-09-10T21:00:45.193 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:00:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T21:00:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T21:00:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:45.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T21:00:45.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T21:00:45.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:45.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:45.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:45.638 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-10T21:00:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: osdmap e61: 11 total, 10 up, 11 in 2024-09-10T21:00:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T21:00:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]} : dispatch 2024-09-10T21:00:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:00:45.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:00:45.742 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 -- 172.21.15.174:0/612255907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814102ab0 msgr2=0x7f0814102f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:45.742 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 --2- 172.21.15.174:0/612255907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814102ab0 0x7f0814102f30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009960 tx=0x7f07fc02a460 comp rx=0 tx=0).stop 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 -- 172.21.15.174:0/612255907 shutdown_connections 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 --2- 172.21.15.174:0/612255907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814103470 0x7f0814109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 --2- 172.21.15.174:0/612255907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814102ab0 0x7f0814102f30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 --2- 172.21.15.174:0/612255907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0814101950 0x7f0814101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 -- 172.21.15.174:0/612255907 >> 172.21.15.174:0/612255907 conn(0x7f08140fd200 msgr2=0x7f08140ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 -- 172.21.15.174:0/612255907 shutdown_connections 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.713+0000 7f0818fee640 1 -- 172.21.15.174:0/612255907 wait complete. 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 Processor -- start 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 -- start start 2024-09-10T21:00:45.743 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101950 0x7f08141057e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:45.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0814102ab0 0x7f0814104050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:45.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814103470 0x7f0814104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:45.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f081410d650 con 0x7f0814102ab0 2024-09-10T21:00:45.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f081410d4d0 con 0x7f0814103470 2024-09-10T21:00:45.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0818fee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f081410d7d0 con 0x7f0814101950 2024-09-10T21:00:45.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101950 0x7f08141057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:45.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101950 0x7f08141057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:43796/0 (socket says 172.21.15.174:43796) 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 -- 172.21.15.174:0/780259530 learned_addr learned my addr 172.21.15.174:0/780259530 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f0812d9d640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814103470 0x7f0814104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f080bfff640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0814102ab0 0x7f0814104050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 -- 172.21.15.174:0/780259530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814103470 msgr2=0x7f0814104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814103470 0x7f0814104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 -- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0814102ab0 msgr2=0x7f0814104050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0814102ab0 0x7f0814104050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.714+0000 7f081259c640 1 -- 172.21.15.174:0/780259530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0814104e50 con 0x7f0814101950 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.715+0000 7f081259c640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101950 0x7f08141057e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f080400b4a0 tx=0x7f080400b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:45.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.715+0000 7f080b7fe640 1 -- 172.21.15.174:0/780259530 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804014070 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.715+0000 7f080b7fe640 1 -- 172.21.15.174:0/780259530 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0804010ac0 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.715+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08141a2fe0 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.715+0000 7f080b7fe640 1 -- 172.21.15.174:0/780259530 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804017c30 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.715+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08141a34c0 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.716+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08141a3750 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.717+0000 7f080b7fe640 1 -- 172.21.15.174:0/780259530 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f0804010610 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.718+0000 7f080b7fe640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08000b5360 0x7f08000b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.718+0000 7f080b7fe640 1 -- 172.21.15.174:0/780259530 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6414+0+0 (secure 0 0 0) 0x7f08040e4c60 con 0x7f0814101950 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.718+0000 7f080bfff640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08000b5360 0x7f08000b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:00:45.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.719+0000 7f080bfff640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08000b5360 0x7f08000b7820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07fc031460 tx=0x7f07fc005d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:00:45.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.721+0000 7f080b7fe640 1 -- 172.21.15.174:0/780259530 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f08040ac290 con 0x7f0814101950 2024-09-10T21:00:45.912 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:00:45.908+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f081406bba0 con 0x7f08000b5360 2024-09-10T21:00:46.062 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:00:45.704+0000 7fde9df77640 -1 osd.10 0 waiting for initial osdmap 2024-09-10T21:00:46.062 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:00:45 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:00:45.714+0000 7fde95582640 -1 osd.10 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:00:46.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi104 2024-09-10T21:00:46.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-10T21:00:46.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T21:00:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T21:00:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: from='osd.10 ' entity='osd.10' 2024-09-10T21:00:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:00:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:00:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi104 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: from='osd.10 ' entity='osd.10' 2024-09-10T21:00:46.596 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:00:46.597 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:00:46.597 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:46.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi104 2024-09-10T21:00:46.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi104 to 4272M 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: osdmap e62: 11 total, 10 up, 11 in 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: from='osd.10 ' entity='osd.10' 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T21:00:46.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:47.375 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T21:00:47.375 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T21:00:47.375 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:00:47.376 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:47.376 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:47.376 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] boot 2024-09-10T21:00:47.376 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: osdmap e63: 11 total, 11 up, 11 in 2024-09-10T21:00:47.376 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:47 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T21:00:47.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T21:00:47.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:00:47.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:47.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:47.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] boot 2024-09-10T21:00:47.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: osdmap e63: 11 total, 11 up, 11 in 2024-09-10T21:00:47.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:47 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: from='client.24448 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 282 MiB used, 894 GiB / 894 GiB avail 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: osd.10 [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] boot 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: osdmap e63: 11 total, 11 up, 11 in 2024-09-10T21:00:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:47 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-10T21:00:48.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:48 smithi174 ceph-mon[35635]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T21:00:48.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:48 smithi033 ceph-mon[30860]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T21:00:48.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:48 smithi104 ceph-mon[35325]: osdmap e64: 11 total, 11 up, 11 in 2024-09-10T21:00:49.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:00:49.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T21:00:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/1958989202' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"} : dispatch 2024-09-10T21:00:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"} : dispatch 2024-09-10T21:00:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"}]': finished 2024-09-10T21:00:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T21:00:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:00:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:49 smithi104 ceph-mon[35325]: from='client.? 172.21.15.174:0/129424402' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:00:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:00:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T21:00:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/1958989202' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"} : dispatch 2024-09-10T21:00:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"} : dispatch 2024-09-10T21:00:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"}]': finished 2024-09-10T21:00:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T21:00:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:00:49.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:49 smithi174 ceph-mon[35635]: from='client.? 172.21.15.174:0/129424402' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 313 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: osdmap e65: 11 total, 11 up, 11 in 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/1958989202' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"} : dispatch 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"} : dispatch 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9b6009d5-69cd-4c9c-9387-bc1fdceb041d"}]': finished 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: osdmap e66: 12 total, 11 up, 12 in 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:00:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:49 smithi033 ceph-mon[30860]: from='client.? 172.21.15.174:0/129424402' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T21:00:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:51 smithi174 ceph-mon[35635]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:00:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:51 smithi033 ceph-mon[30860]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:00:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:51 smithi104 ceph-mon[35325]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:00:53.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:53 smithi174 ceph-mon[35635]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T21:00:53.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:53 smithi033 ceph-mon[30860]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T21:00:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:53 smithi104 ceph-mon[35325]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 100 KiB/s, 0 objects/s recovering 2024-09-10T21:00:55.556 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:55 smithi174 ceph-mon[35635]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T21:00:55.614 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:55 smithi104 ceph-mon[35325]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T21:00:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:55 smithi033 ceph-mon[30860]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-09-10T21:00:57.579 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:57 smithi174 ceph-mon[35635]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:00:57.580 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T21:00:57.580 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:57 smithi033 ceph-mon[30860]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:00:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T21:00:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:57 smithi104 ceph-mon[35325]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T21:00:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-10T21:00:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:00:58.499 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:58 smithi174 ceph-mon[35635]: Deploying daemon osd.11 on smithi174 2024-09-10T21:00:58.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:58 smithi033 ceph-mon[30860]: Deploying daemon osd.11 on smithi174 2024-09-10T21:00:58.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:58 smithi104 ceph-mon[35325]: Deploying daemon osd.11 on smithi174 2024-09-10T21:00:59.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:00:59 smithi174 ceph-mon[35635]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T21:00:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:00:59 smithi033 ceph-mon[30860]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T21:00:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:00:59 smithi104 ceph-mon[35325]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-10T21:01:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:01 smithi174 ceph-mon[35635]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T21:01:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:01 smithi033 ceph-mon[30860]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T21:01:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:01 smithi104 ceph-mon[35325]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-10T21:01:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:03 smithi174 ceph-mon[35635]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:01:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:03 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:03 smithi033 ceph-mon[30860]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:01:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:03 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:03 smithi104 ceph-mon[35325]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T21:01:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:03.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:03 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:05.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:05 smithi174 ceph-mon[35635]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:05.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:05 smithi033 ceph-mon[30860]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:05 smithi104 ceph-mon[35325]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:06.857 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-09-10T21:01:06.857 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.852+0000 7f080b7fe640 1 -- 172.21.15.174:0/780259530 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f081406bba0 con 0x7f08000b5360 2024-09-10T21:01:06.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.854+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08000b5360 msgr2=0x7f08000b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:06.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.854+0000 7f0818fee640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08000b5360 0x7f08000b7820 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f07fc031460 tx=0x7f07fc005d50 comp rx=0 tx=0).stop 2024-09-10T21:01:06.859 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101950 msgr2=0x7f08141057e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101950 0x7f08141057e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f080400b4a0 tx=0x7f080400b960 comp rx=0 tx=0).stop 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 shutdown_connections 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08000b5360 0x7f08000b7820 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0814103470 0x7f0814104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0814102ab0 0x7f0814104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 --2- 172.21.15.174:0/780259530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0814101950 0x7f08141057e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 >> 172.21.15.174:0/780259530 conn(0x7f08140fd200 msgr2=0x7f081406b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 shutdown_connections 2024-09-10T21:01:06.860 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:06.855+0000 7f0818fee640 1 -- 172.21.15.174:0/780259530 wait complete. 2024-09-10T21:01:07.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:07.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:07.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:07.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:07.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:07 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.545 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:07 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:07.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:07.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:07.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:07.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:07 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:07.576 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.11.service 2024-09-10T21:01:07.578 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-10T21:01:07.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T21:01:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:08.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 -- 172.21.15.33:0/450844197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f8106520 msgr2=0x7f64f81069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:08.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 --2- 172.21.15.33:0/450844197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f8106520 0x7f64f81069d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f64f0009ca0 tx=0x7f64f002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 -- 172.21.15.33:0/450844197 shutdown_connections 2024-09-10T21:01:08.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 --2- 172.21.15.33:0/450844197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f8106520 0x7f64f81069d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 --2- 172.21.15.33:0/450844197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f8103970 0x7f64f8105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 --2- 172.21.15.33:0/450844197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f806b6e0 0x7f64f81033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 -- 172.21.15.33:0/450844197 >> 172.21.15.33:0/450844197 conn(0x7f64f80fd1e0 msgr2=0x7f64f80ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 -- 172.21.15.33:0/450844197 shutdown_connections 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 -- 172.21.15.33:0/450844197 wait complete. 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.347+0000 7f64ffac4640 1 Processor -- start 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 -- start start 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f806b6e0 0x7f64f819f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f8103970 0x7f64f819f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f8106520 0x7f64f81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8114c70 con 0x7f64f806b6e0 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8114af0 con 0x7f64f8103970 2024-09-10T21:01:08.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64f8114df0 con 0x7f64f8106520 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f806b6e0 0x7f64f819f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f806b6e0 0x7f64f819f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58570/0 (socket says 172.21.15.33:58570) 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 -- 172.21.15.33:0/478320618 learned_addr learned my addr 172.21.15.33:0/478320618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fe061640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f8106520 0x7f64f81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd05f640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f8103970 0x7f64f819f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 -- 172.21.15.33:0/478320618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f8106520 msgr2=0x7f64f81a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f8106520 0x7f64f81a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 -- 172.21.15.33:0/478320618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f8103970 msgr2=0x7f64f819f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f8103970 0x7f64f819f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 -- 172.21.15.33:0/478320618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64f81a0ec0 con 0x7f64f806b6e0 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64fd860640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f806b6e0 0x7f64f819f390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d950 tx=0x7f64f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:08.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64eeffd640 1 -- 172.21.15.33:0/478320618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f4017070 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64f81a11b0 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64eeffd640 1 -- 172.21.15.33:0/478320618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f4009c30 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.348+0000 7f64eeffd640 1 -- 172.21.15.33:0/478320618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f400f620 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.349+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64f81a7890 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.349+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64f80fecd0 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.351+0000 7f64eeffd640 1 -- 172.21.15.33:0/478320618 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f64f4010460 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.352+0000 7f64eeffd640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64e40b52e0 0x7f64e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.352+0000 7f64fd05f640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64e40b52e0 0x7f64e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.352+0000 7f64eeffd640 1 -- 172.21.15.33:0/478320618 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6809+0+0 (secure 0 0 0) 0x7f64f40e52d0 con 0x7f64f806b6e0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.352+0000 7f64fd05f640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64e40b52e0 0x7f64e40b77a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64f806bb90 tx=0x7f64e8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:08.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.355+0000 7f64eeffd640 1 -- 172.21.15.33:0/478320618 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f64f40ac6d0 con 0x7f64f806b6e0 2024-09-10T21:01:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.539+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f64f802e820 con 0x7f64f806b6e0 2024-09-10T21:01:08.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.540+0000 7f64eeffd640 1 -- 172.21.15.33:0/478320618 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7f64f4009de0 con 0x7f64f806b6e0 2024-09-10T21:01:08.545 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64e40b52e0 msgr2=0x7f64e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64e40b52e0 0x7f64e40b77a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f64f806bb90 tx=0x7f64e8005f90 comp rx=0 tx=0).stop 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f806b6e0 msgr2=0x7f64f819f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f806b6e0 0x7f64f819f390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f64f400d950 tx=0x7f64f400de10 comp rx=0 tx=0).stop 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 shutdown_connections 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64e40b52e0 0x7f64e40b77a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64f8106520 0x7f64f81a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64f8103970 0x7f64f819f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 --2- 172.21.15.33:0/478320618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f64f806b6e0 0x7f64f819f390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 >> 172.21.15.33:0/478320618 conn(0x7f64f80fd1e0 msgr2=0x7f64f8101c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 shutdown_connections 2024-09-10T21:01:08.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:08.542+0000 7f64ffac4640 1 -- 172.21.15.33:0/478320618 wait complete. 2024-09-10T21:01:09.136 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:01:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:01:08.880+0000 7f9d18634640 -1 osd.11 0 log_to_monitors true 2024-09-10T21:01:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:09 smithi174 ceph-mon[35635]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:09 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/478320618' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:01:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:09 smithi174 ceph-mon[35635]: from='osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:01:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:09 smithi174 ceph-mon[35635]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:01:09.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:09 smithi033 ceph-mon[30860]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:09.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:09 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/478320618' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:01:09.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:09 smithi033 ceph-mon[30860]: from='osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:01:09.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:09 smithi033 ceph-mon[30860]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:01:09.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:09 smithi104 ceph-mon[35325]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:09.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:09 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/478320618' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:01:09.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:09 smithi104 ceph-mon[35325]: from='osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:01:09.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:09 smithi104 ceph-mon[35325]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-10T21:01:09.577 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726002046,"num_in_osds":12,"osd_in_since":1726002048,"num_remapped_pgs":0} 2024-09-10T21:01:10.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T21:01:10.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: from='osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T21:01:10.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T21:01:10.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:10.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T21:01:10.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T21:01:10.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T21:01:10.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:10 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:10.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T21:01:10.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: from='osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T21:01:10.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T21:01:10.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:10.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T21:01:10.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T21:01:10.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T21:01:10.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:10 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:10.578 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: from='osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: osdmap e67: 12 total, 11 up, 12 in 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]} : dispatch 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: osdmap e68: 12 total, 11 up, 12 in 2024-09-10T21:01:10.594 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:10.594 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:01:10.160+0000 7f9d145b7640 -1 osd.11 0 waiting for initial osdmap 2024-09-10T21:01:10.595 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:01:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:01:10.173+0000 7f9d103de640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:01:10.852 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:11.244 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: Detected new or changed devices on smithi174 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.245 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:11.246 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:11.246 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.246 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 -- 172.21.15.33:0/2088457292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c101940 msgr2=0x7fb86c101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 --2- 172.21.15.33:0/2088457292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c101940 0x7fb86c101d40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009930 tx=0x7fb85c02a420 comp rx=0 tx=0).stop 2024-09-10T21:01:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 -- 172.21.15.33:0/2088457292 shutdown_connections 2024-09-10T21:01:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 --2- 172.21.15.33:0/2088457292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb86c103460 0x7fb86c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 --2- 172.21.15.33:0/2088457292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102aa0 0x7fb86c102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 --2- 172.21.15.33:0/2088457292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c101940 0x7fb86c101d40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 -- 172.21.15.33:0/2088457292 >> 172.21.15.33:0/2088457292 conn(0x7fb86c0fd210 msgr2=0x7fb86c0ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 -- 172.21.15.33:0/2088457292 shutdown_connections 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.322+0000 7fb8728a4640 1 -- 172.21.15.33:0/2088457292 wait complete. 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 Processor -- start 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 -- start start 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb86c101940 0x7fb86c198ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102aa0 0x7fb86c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c103460 0x7fb86c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10e6f0 con 0x7fb86c103460 2024-09-10T21:01:11.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10e570 con 0x7fb86c101940 2024-09-10T21:01:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb8728a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10e870 con 0x7fb86c102aa0 2024-09-10T21:01:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb870e41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c103460 0x7fb86c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb870e41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c103460 0x7fb86c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55588/0 (socket says 172.21.15.33:55588) 2024-09-10T21:01:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.323+0000 7fb870e41640 1 -- 172.21.15.33:0/1504458436 learned_addr learned my addr 172.21.15.33:0/1504458436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb86b7fe640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102aa0 0x7fb86c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb86bfff640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb86c101940 0x7fb86c198ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:11.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb870e41640 1 -- 172.21.15.33:0/1504458436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102aa0 msgr2=0x7fb86c199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb870e41640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102aa0 0x7fb86c199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb870e41640 1 -- 172.21.15.33:0/1504458436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb86c101940 msgr2=0x7fb86c198ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb870e41640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb86c101940 0x7fb86c198ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb870e41640 1 -- 172.21.15.33:0/1504458436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c19aa20 con 0x7fb86c103460 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb870e41640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c103460 0x7fb86c19a180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb86400cad0 tx=0x7fb86400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb8697fa640 1 -- 172.21.15.33:0/1504458436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86400ae20 con 0x7fb86c103460 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb8697fa640 1 -- 172.21.15.33:0/1504458436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb864005020 con 0x7fb86c103460 2024-09-10T21:01:11.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c19ad10 con 0x7fb86c103460 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb8697fa640 1 -- 172.21.15.33:0/1504458436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86400f6b0 con 0x7fb86c103460 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.324+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c1a13f0 con 0x7fb86c103460 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.325+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb86c108d40 con 0x7fb86c103460 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.327+0000 7fb8697fa640 1 -- 172.21.15.33:0/1504458436 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb8640054f0 con 0x7fb86c103460 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.327+0000 7fb8697fa640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8540b5360 0x7fb8540b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.327+0000 7fb86bfff640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8540b5360 0x7fb8540b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.327+0000 7fb8697fa640 1 -- 172.21.15.33:0/1504458436 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 7127+0+0 (secure 0 0 0) 0x7fb8640e5a30 con 0x7fb86c103460 2024-09-10T21:01:11.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.328+0000 7fb86bfff640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8540b5360 0x7fb8540b7820 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009900 tx=0x7fb85c031430 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:11.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.330+0000 7fb8697fa640 1 -- 172.21.15.33:0/1504458436 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb864014030 con 0x7fb86c103460 2024-09-10T21:01:11.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.515+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb86c02e810 con 0x7fb86c103460 2024-09-10T21:01:11.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.515+0000 7fb8697fa640 1 -- 172.21.15.33:0/1504458436 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v69) ==== 74+0+133 (secure 0 0 0) 0x7fb8640ac7c0 con 0x7fb86c103460 2024-09-10T21:01:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:11.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.517+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8540b5360 msgr2=0x7fb8540b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:11.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.517+0000 7fb8728a4640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8540b5360 0x7fb8540b7820 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009900 tx=0x7fb85c031430 comp rx=0 tx=0).stop 2024-09-10T21:01:11.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.517+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c103460 msgr2=0x7fb86c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:11.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.517+0000 7fb8728a4640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c103460 0x7fb86c19a180 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb86400cad0 tx=0x7fb86400cf90 comp rx=0 tx=0).stop 2024-09-10T21:01:11.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 shutdown_connections 2024-09-10T21:01:11.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8540b5360 0x7fb8540b7820 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb86c103460 0x7fb86c19a180 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c102aa0 0x7fb86c199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 --2- 172.21.15.33:0/1504458436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb86c101940 0x7fb86c198ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:11.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 >> 172.21.15.33:0/1504458436 conn(0x7fb86c0fd210 msgr2=0x7fb86c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:11.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 shutdown_connections 2024-09-10T21:01:11.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:11.518+0000 7fb8728a4640 1 -- 172.21.15.33:0/1504458436 wait complete. 2024-09-10T21:01:11.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:11.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:11.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: Detected new or changed devices on smithi174 2024-09-10T21:01:11.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-10T21:01:11.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:11.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:11.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: Detected new or changed devices on smithi174 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:01:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi174 to 4272M 2024-09-10T21:01:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:12.348 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":69,"num_osds":12,"num_up_osds":12,"osd_up_since":1726002071,"num_in_osds":12,"osd_in_since":1726002048,"num_remapped_pgs":0} 2024-09-10T21:01:12.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:01:12.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:12 smithi104 ceph-mon[35325]: purged_snaps scrub starts 2024-09-10T21:01:12.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:12 smithi104 ceph-mon[35325]: purged_snaps scrub ok 2024-09-10T21:01:12.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:12 smithi104 ceph-mon[35325]: osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] boot 2024-09-10T21:01:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:12 smithi104 ceph-mon[35325]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T21:01:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:12 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1504458436' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:01:12.607 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:12 smithi174 ceph-mon[35635]: purged_snaps scrub starts 2024-09-10T21:01:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:12 smithi174 ceph-mon[35635]: purged_snaps scrub ok 2024-09-10T21:01:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:12 smithi174 ceph-mon[35635]: osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] boot 2024-09-10T21:01:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:12 smithi174 ceph-mon[35635]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T21:01:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:12 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1504458436' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:01:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:12 smithi033 ceph-mon[30860]: purged_snaps scrub starts 2024-09-10T21:01:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:12 smithi033 ceph-mon[30860]: purged_snaps scrub ok 2024-09-10T21:01:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:12 smithi033 ceph-mon[30860]: osd.11 [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] boot 2024-09-10T21:01:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:12 smithi033 ceph-mon[30860]: osdmap e69: 12 total, 12 up, 12 in 2024-09-10T21:01:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-10T21:01:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:12 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1504458436' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T21:01:13.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 -- 172.21.15.33:0/975524785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6081087d0 msgr2=0x7fe608108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 --2- 172.21.15.33:0/975524785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6081087d0 0x7fe608108c10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009900 tx=0x7fe5f802a410 comp rx=0 tx=0).stop 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 -- 172.21.15.33:0/975524785 shutdown_connections 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 --2- 172.21.15.33:0/975524785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe608109150 0x7fe60810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 --2- 172.21.15.33:0/975524785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6081087d0 0x7fe608108c10 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 --2- 172.21.15.33:0/975524785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe608102dc0 0x7fe6081031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 -- 172.21.15.33:0/975524785 >> 172.21.15.33:0/975524785 conn(0x7fe6080fe830 msgr2=0x7fe608100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 -- 172.21.15.33:0/975524785 shutdown_connections 2024-09-10T21:01:13.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.063+0000 7fe60f15c640 1 -- 172.21.15.33:0/975524785 wait complete. 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 Processor -- start 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 -- start start 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe608102dc0 0x7fe6080754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6081087d0 0x7fe6080759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608109150 0x7fe608075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe608112e10 con 0x7fe608109150 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe608112c90 con 0x7fe608102dc0 2024-09-10T21:01:13.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60f15c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe608112f90 con 0x7fe6081087d0 2024-09-10T21:01:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608109150 0x7fe608075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608109150 0x7fe608075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55602/0 (socket says 172.21.15.33:55602) 2024-09-10T21:01:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 -- 172.21.15.33:0/3877445484 learned_addr learned my addr 172.21.15.33:0/3877445484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60cef8640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe608102dc0 0x7fe6080754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe607fff640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6081087d0 0x7fe6080759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 -- 172.21.15.33:0/3877445484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6081087d0 msgr2=0x7fe6080759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:13.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6081087d0 0x7fe6080759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 -- 172.21.15.33:0/3877445484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe608102dc0 msgr2=0x7fe6080754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe608102dc0 0x7fe6080754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60d6f9640 1 -- 172.21.15.33:0/3877445484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6081a76a0 con 0x7fe608109150 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.064+0000 7fe60cef8640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe608102dc0 0x7fe6080754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.065+0000 7fe60d6f9640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608109150 0x7fe608075f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe60000b970 tx=0x7fe60000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.065+0000 7fe605ffb640 1 -- 172.21.15.33:0/3877445484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600019070 con 0x7fe608109150 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.065+0000 7fe605ffb640 1 -- 172.21.15.33:0/3877445484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe600014440 con 0x7fe608109150 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.065+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6081a7990 con 0x7fe608109150 2024-09-10T21:01:13.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.065+0000 7fe605ffb640 1 -- 172.21.15.33:0/3877445484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6000075d0 con 0x7fe608109150 2024-09-10T21:01:13.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.065+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6081a7ed0 con 0x7fe608109150 2024-09-10T21:01:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.066+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6081031a0 con 0x7fe608109150 2024-09-10T21:01:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.067+0000 7fe605ffb640 1 -- 172.21.15.33:0/3877445484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fe60000cb40 con 0x7fe608109150 2024-09-10T21:01:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.068+0000 7fe605ffb640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe5f40b52e0 0x7fe5f40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.069+0000 7fe60cef8640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe5f40b52e0 0x7fe5f40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.069+0000 7fe605ffb640 1 -- 172.21.15.33:0/3877445484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fe6000e4f80 con 0x7fe608109150 2024-09-10T21:01:13.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.069+0000 7fe60cef8640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe5f40b52e0 0x7fe5f40b77a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000baa0 tx=0x7fe5f0005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:13.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.073+0000 7fe605ffb640 1 -- 172.21.15.33:0/3877445484 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe6000ac2e0 con 0x7fe608109150 2024-09-10T21:01:13.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.259+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe60802e5c0 con 0x7fe608109150 2024-09-10T21:01:13.265 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:13 smithi033 ceph-mon[30860]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:13.265 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:13 smithi033 ceph-mon[30860]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T21:01:13.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.260+0000 7fe605ffb640 1 -- 172.21.15.33:0/3877445484 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18470 (secure 0 0 0) 0x7fe6000abbe0 con 0x7fe608109150 2024-09-10T21:01:13.266 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:13.266 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":70,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:01:12.177164+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:01:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.262+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe5f40b52e0 msgr2=0x7fe5f40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.262+0000 7fe60f15c640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe5f40b52e0 0x7fe5f40b77a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000baa0 tx=0x7fe5f0005d90 comp rx=0 tx=0).stop 2024-09-10T21:01:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.262+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608109150 msgr2=0x7fe608075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.262+0000 7fe60f15c640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608109150 0x7fe608075f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe60000b970 tx=0x7fe60000be30 comp rx=0 tx=0).stop 2024-09-10T21:01:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 shutdown_connections 2024-09-10T21:01:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe5f40b52e0 0x7fe5f40b77a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe608109150 0x7fe608075f20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6081087d0 0x7fe6080759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 --2- 172.21.15.33:0/3877445484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe608102dc0 0x7fe6080754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:13.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 >> 172.21.15.33:0/3877445484 conn(0x7fe6080fe830 msgr2=0x7fe608100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:13.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 shutdown_connections 2024-09-10T21:01:13.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:13.263+0000 7fe60f15c640 1 -- 172.21.15.33:0/3877445484 wait complete. 2024-09-10T21:01:13.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:13 smithi104 ceph-mon[35325]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:13.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:13 smithi104 ceph-mon[35325]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T21:01:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:13 smithi174 ceph-mon[35635]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 314 MiB used, 983 GiB / 983 GiB avail 2024-09-10T21:01:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:13 smithi174 ceph-mon[35635]: osdmap e70: 12 total, 12 up, 12 in 2024-09-10T21:01:13.886 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T20:57:24.470902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:01:13.887 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T21:01:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:14.443 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:14 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3877445484' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:01:14.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:14 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3877445484' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:01:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 -- 172.21.15.33:0/764866671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac101c90 msgr2=0x7fdbac102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 --2- 172.21.15.33:0/764866671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac101c90 0x7fdbac102070 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009900 tx=0x7fdba802a410 comp rx=0 tx=0).stop 2024-09-10T21:01:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 -- 172.21.15.33:0/764866671 shutdown_connections 2024-09-10T21:01:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 --2- 172.21.15.33:0/764866671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbac108020 0x7fdbac10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 --2- 172.21.15.33:0/764866671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac1076a0 0x7fdbac107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 --2- 172.21.15.33:0/764866671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac101c90 0x7fdbac102070 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 -- 172.21.15.33:0/764866671 >> 172.21.15.33:0/764866671 conn(0x7fdbac0fd700 msgr2=0x7fdbac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:14.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 -- 172.21.15.33:0/764866671 shutdown_connections 2024-09-10T21:01:14.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 -- 172.21.15.33:0/764866671 wait complete. 2024-09-10T21:01:14.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 Processor -- start 2024-09-10T21:01:14.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.618+0000 7fdbb443b640 1 -- start start 2024-09-10T21:01:14.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb443b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbac101c90 0x7fdbac10f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:14.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb443b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac1076a0 0x7fdbac10f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb443b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac108020 0x7fdbac10fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb443b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac112e20 con 0x7fdbac108020 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb443b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac112ca0 con 0x7fdbac1076a0 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb443b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdbac112fa0 con 0x7fdbac101c90 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac108020 0x7fdbac10fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac108020 0x7fdbac10fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55628/0 (socket says 172.21.15.33:55628) 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 -- 172.21.15.33:0/3862728172 learned_addr learned my addr 172.21.15.33:0/3862728172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb21d7640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbac101c90 0x7fdbac10f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb19d6640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac1076a0 0x7fdbac10f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 -- 172.21.15.33:0/3862728172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbac101c90 msgr2=0x7fdbac10f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbac101c90 0x7fdbac10f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 -- 172.21.15.33:0/3862728172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac1076a0 msgr2=0x7fdbac10f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:14.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac1076a0 0x7fdbac10f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 -- 172.21.15.33:0/3862728172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbac1a77f0 con 0x7fdbac108020 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb19d6640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac1076a0 0x7fdbac10f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb21d7640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbac101c90 0x7fdbac10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdbb29d8640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac108020 0x7fdbac10fdb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b970 tx=0x7fdba400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.619+0000 7fdba37fe640 1 -- 172.21.15.33:0/3862728172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba4019070 con 0x7fdbac108020 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.620+0000 7fdba37fe640 1 -- 172.21.15.33:0/3862728172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdba4014440 con 0x7fdbac108020 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.620+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbac1a7ae0 con 0x7fdbac108020 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.620+0000 7fdba37fe640 1 -- 172.21.15.33:0/3862728172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba40075d0 con 0x7fdbac108020 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.620+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbac1a8050 con 0x7fdbac108020 2024-09-10T21:01:14.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.621+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbac1020f0 con 0x7fdbac108020 2024-09-10T21:01:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.623+0000 7fdba37fe640 1 -- 172.21.15.33:0/3862728172 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fdba400cb40 con 0x7fdbac108020 2024-09-10T21:01:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.624+0000 7fdba37fe640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb980b52e0 0x7fdb980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:14.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.624+0000 7fdbb21d7640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb980b52e0 0x7fdb980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.624+0000 7fdba37fe640 1 -- 172.21.15.33:0/3862728172 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fdba40e53e0 con 0x7fdbac108020 2024-09-10T21:01:14.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.624+0000 7fdbb21d7640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb980b52e0 0x7fdb980b77a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdba8031440 tx=0x7fdba80095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:14.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.627+0000 7fdba37fe640 1 -- 172.21.15.33:0/3862728172 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdba40ac740 con 0x7fdbac108020 2024-09-10T21:01:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:14 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3877445484' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:01:14.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.813+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdbac02e5c0 con 0x7fdbac108020 2024-09-10T21:01:14.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.813+0000 7fdba37fe640 1 -- 172.21.15.33:0/3862728172 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fdba40ac040 con 0x7fdbac108020 2024-09-10T21:01:14.818 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-09-10T21:01:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb980b52e0 msgr2=0x7fdb980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb980b52e0 0x7fdb980b77a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdba8031440 tx=0x7fdba80095f0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac108020 msgr2=0x7fdbac10fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac108020 0x7fdbac10fdb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdba400b970 tx=0x7fdba400be30 comp rx=0 tx=0).stop 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 shutdown_connections 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb980b52e0 0x7fdb980b77a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbac108020 0x7fdbac10fdb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdbac1076a0 0x7fdbac10f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 --2- 172.21.15.33:0/3862728172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdbac101c90 0x7fdbac10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 >> 172.21.15.33:0/3862728172 conn(0x7fdbac0fd700 msgr2=0x7fdbac0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.815+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 shutdown_connections 2024-09-10T21:01:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:14.816+0000 7fdbb443b640 1 -- 172.21.15.33:0/3862728172 wait complete. 2024-09-10T21:01:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:15 smithi033 ceph-mon[30860]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:15 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3862728172' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:01:15.528 INFO:tasks.cephadm:Adding mds.a on smithi033 2024-09-10T21:01:15.528 INFO:tasks.cephadm:Adding mds.d on smithi033 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.g on smithi033 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.j on smithi033 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.b on smithi104 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.e on smithi104 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.h on smithi104 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.k on smithi104 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-09-10T21:01:15.529 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-09-10T21:01:15.529 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-09-10T21:01:15.558 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:15 smithi174 ceph-mon[35635]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:15.558 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:15 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3862728172' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:01:15.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:15 smithi104 ceph-mon[35325]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 341 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:15.565 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:15 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3862728172' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:01:15.792 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:01:16.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 -- 172.21.15.174:0/3007912631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec102a30 msgr2=0x7f20ec102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:16.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 --2- 172.21.15.174:0/3007912631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec102a30 0x7f20ec102e30 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009960 tx=0x7f20dc02a460 comp rx=0 tx=0).stop 2024-09-10T21:01:16.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 -- 172.21.15.174:0/3007912631 shutdown_connections 2024-09-10T21:01:16.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 --2- 172.21.15.174:0/3007912631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec104550 0x7f20ec10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 --2- 172.21.15.174:0/3007912631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ec103b90 0x7f20ec104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.365 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 --2- 172.21.15.174:0/3007912631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec102a30 0x7f20ec102e30 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 -- 172.21.15.174:0/3007912631 >> 172.21.15.174:0/3007912631 conn(0x7f20ec0fe320 msgr2=0x7f20ec100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 -- 172.21.15.174:0/3007912631 shutdown_connections 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 -- 172.21.15.174:0/3007912631 wait complete. 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.310+0000 7f20f1b37640 1 Processor -- start 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20f1b37640 1 -- start start 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20f1b37640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec102a30 0x7f20ec198ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20f1b37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ec103b90 0x7f20ec199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20f1b37640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec104550 0x7f20ec19a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20f1b37640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec10e710 con 0x7f20ec103b90 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20f1b37640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec10e590 con 0x7f20ec102a30 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20f1b37640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20ec10e890 con 0x7f20ec104550 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec104550 0x7f20ec19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:16.366 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec104550 0x7f20ec19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:59010/0 (socket says 172.21.15.174:59010) 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 -- 172.21.15.174:0/2941207666 learned_addr learned my addr 172.21.15.174:0/2941207666 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20eb7fe640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec102a30 0x7f20ec198ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20eaffd640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ec103b90 0x7f20ec199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 -- 172.21.15.174:0/2941207666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec102a30 msgr2=0x7f20ec198ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec102a30 0x7f20ec198ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 -- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ec103b90 msgr2=0x7f20ec199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ec103b90 0x7f20ec199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 -- 172.21.15.174:0/2941207666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ec19a9f0 con 0x7f20ec104550 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20eb7fe640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec102a30 0x7f20ec198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.311+0000 7f20ebfff640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec104550 0x7f20ec19a150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b970 tx=0x7f20e400be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.312+0000 7f20e8ff9640 1 -- 172.21.15.174:0/2941207666 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4019070 con 0x7f20ec104550 2024-09-10T21:01:16.367 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.312+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ec19ace0 con 0x7f20ec104550 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.312+0000 7f20e8ff9640 1 -- 172.21.15.174:0/2941207666 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e4014440 con 0x7f20ec104550 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.312+0000 7f20e8ff9640 1 -- 172.21.15.174:0/2941207666 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e40075d0 con 0x7f20ec104550 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.312+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ec1a13c0 con 0x7f20ec104550 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.313+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20ec109db0 con 0x7f20ec104550 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.314+0000 7f20e8ff9640 1 -- 172.21.15.174:0/2941207666 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f20e400cb40 con 0x7f20ec104550 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.315+0000 7f20e8ff9640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f20d40b52e0 0x7f20d40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.315+0000 7f20e8ff9640 1 -- 172.21.15.174:0/2941207666 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f20e40e53e0 con 0x7f20ec104550 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.315+0000 7f20eb7fe640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f20d40b52e0 0x7f20d40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.315+0000 7f20eb7fe640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f20d40b52e0 0x7f20d40b77a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0096c0 tx=0x7f20dc005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:16.368 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.318+0000 7f20e8ff9640 1 -- 172.21.15.174:0/2941207666 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f20e40ac740 con 0x7f20ec104550 2024-09-10T21:01:16.507 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.504+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) -- 0x7f20ec063690 con 0x7f20d40b52e0 2024-09-10T21:01:16.515 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.512+0000 7f20e8ff9640 1 -- 172.21.15.174:0/2941207666 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f20ec063690 con 0x7f20d40b52e0 2024-09-10T21:01:16.515 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f20d40b52e0 msgr2=0x7f20d40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f20d40b52e0 0x7f20d40b77a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f20dc0096c0 tx=0x7f20dc005fd0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec104550 msgr2=0x7f20ec19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec104550 0x7f20ec19a150 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b970 tx=0x7f20e400be30 comp rx=0 tx=0).stop 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 shutdown_connections 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f20d40b52e0 0x7f20d40b77a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20ec104550 0x7f20ec19a150 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20ec103b90 0x7f20ec199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 --2- 172.21.15.174:0/2941207666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20ec102a30 0x7f20ec198ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 >> 172.21.15.174:0/2941207666 conn(0x7f20ec0fe320 msgr2=0x7f20ec106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:16.518 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 shutdown_connections 2024-09-10T21:01:16.519 INFO:teuthology.orchestra.run.smithi174.stderr:2024-09-10T21:01:16.514+0000 7f20f1b37640 1 -- 172.21.15.174:0/2941207666 wait complete. 2024-09-10T21:01:17.232 DEBUG:teuthology.orchestra.run.smithi033:mds.a> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.a.service 2024-09-10T21:01:17.235 DEBUG:teuthology.orchestra.run.smithi033:mds.d> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.d.service 2024-09-10T21:01:17.279 DEBUG:teuthology.orchestra.run.smithi033:mds.g> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.g.service 2024-09-10T21:01:17.323 DEBUG:teuthology.orchestra.run.smithi033:mds.j> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.j.service 2024-09-10T21:01:17.366 DEBUG:teuthology.orchestra.run.smithi104:mds.b> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.b.service 2024-09-10T21:01:17.368 DEBUG:teuthology.orchestra.run.smithi104:mds.e> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.e.service 2024-09-10T21:01:17.412 DEBUG:teuthology.orchestra.run.smithi104:mds.h> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.h.service 2024-09-10T21:01:17.455 DEBUG:teuthology.orchestra.run.smithi104:mds.k> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.k.service 2024-09-10T21:01:17.498 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.c.service 2024-09-10T21:01:17.501 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.f.service 2024-09-10T21:01:17.545 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.i.service 2024-09-10T21:01:17.587 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.l.service 2024-09-10T21:01:17.632 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-10T21:01:17.633 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:17.788 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:17.789 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:01:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-10T21:01:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='client.24493 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:01:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: Saving service mds.all spec with placement smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-10T21:01:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:17.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:17.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:17.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:17.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:18.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 -- 172.21.15.33:0/201743078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac1087d0 msgr2=0x7fe5ac108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/201743078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac1087d0 0x7fe5ac108c10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe598009900 tx=0x7fe59802a410 comp rx=0 tx=0).stop 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 -- 172.21.15.33:0/201743078 shutdown_connections 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/201743078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5ac109150 0x7fe5ac10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/201743078 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac1087d0 0x7fe5ac108c10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/201743078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5ac102dc0 0x7fe5ac1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 -- 172.21.15.33:0/201743078 >> 172.21.15.33:0/201743078 conn(0x7fe5ac0fe830 msgr2=0x7fe5ac100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 -- 172.21.15.33:0/201743078 shutdown_connections 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.322+0000 7fe5b2a97640 1 -- 172.21.15.33:0/201743078 wait complete. 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 Processor -- start 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 -- start start 2024-09-10T21:01:18.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac102dc0 0x7fe5ac19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5ac1087d0 0x7fe5ac19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5ac109150 0x7fe5ac1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac112e60 con 0x7fe5ac102dc0 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac112ce0 con 0x7fe5ac109150 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.323+0000 7fe5b2a97640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ac112fe0 con 0x7fe5ac1087d0 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac102dc0 0x7fe5ac19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac102dc0 0x7fe5ac19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55646/0 (socket says 172.21.15.33:55646) 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 -- 172.21.15.33:0/1718675277 learned_addr learned my addr 172.21.15.33:0/1718675277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5abfff640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5ac1087d0 0x7fe5ac19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b1034640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5ac109150 0x7fe5ac1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 -- 172.21.15.33:0/1718675277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5ac1087d0 msgr2=0x7fe5ac19ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5ac1087d0 0x7fe5ac19ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 -- 172.21.15.33:0/1718675277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5ac109150 msgr2=0x7fe5ac1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5ac109150 0x7fe5ac1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 -- 172.21.15.33:0/1718675277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ac1a1570 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b1034640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5ac109150 0x7fe5ac1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b0833640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac102dc0 0x7fe5ac19b4c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000d950 tx=0x7fe5a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5a9ffb640 1 -- 172.21.15.33:0/1718675277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0017070 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5a9ffb640 1 -- 172.21.15.33:0/1718675277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5a0009c30 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ac1a1860 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.324+0000 7fe5a9ffb640 1 -- 172.21.15.33:0/1718675277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a000f620 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.325+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ac1a80f0 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.325+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ac104070 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.327+0000 7fe5a9ffb640 1 -- 172.21.15.33:0/1718675277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fe5a0010460 con 0x7fe5ac102dc0 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.327+0000 7fe5a9ffb640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe59c0b53e0 0x7fe59c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:18.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.328+0000 7fe5abfff640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe59c0b53e0 0x7fe59c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:18.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.328+0000 7fe5a9ffb640 1 -- 172.21.15.33:0/1718675277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fe5a00e53d0 con 0x7fe5ac102dc0 2024-09-10T21:01:18.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.328+0000 7fe5abfff640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe59c0b53e0 0x7fe59c0b78a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe598005ab0 tx=0x7fe598005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:18.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.332+0000 7fe5a9ffb640 1 -- 172.21.15.33:0/1718675277 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe5a00ac690 con 0x7fe5ac102dc0 2024-09-10T21:01:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.555+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5ac1031a0 con 0x7fe5ac102dc0 2024-09-10T21:01:18.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.555+0000 7fe5a9ffb640 1 -- 172.21.15.33:0/1718675277 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fe5a0009de0 con 0x7fe5ac102dc0 2024-09-10T21:01:18.561 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:18.561 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":1,"btime":"2024-09-10T20:53:48:210161+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-10T21:01:18.561 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 1 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe59c0b53e0 msgr2=0x7fe59c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe59c0b53e0 0x7fe59c0b78a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe598005ab0 tx=0x7fe598005a20 comp rx=0 tx=0).stop 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac102dc0 msgr2=0x7fe5ac19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac102dc0 0x7fe5ac19b4c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe5a000d950 tx=0x7fe5a000de10 comp rx=0 tx=0).stop 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 shutdown_connections 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe59c0b53e0 0x7fe59c0b78a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5ac109150 0x7fe5ac1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5ac1087d0 0x7fe5ac19ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 --2- 172.21.15.33:0/1718675277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5ac102dc0 0x7fe5ac19b4c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:18.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 >> 172.21.15.33:0/1718675277 conn(0x7fe5ac0fe830 msgr2=0x7fe5ac104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 shutdown_connections 2024-09-10T21:01:18.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:18.558+0000 7fe5b2a97640 1 -- 172.21.15.33:0/1718675277 wait complete. 2024-09-10T21:01:18.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:18 smithi033 ceph-mon[30860]: Deploying daemon mds.l on smithi174 2024-09-10T21:01:18.758 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:18 smithi174 ceph-mon[35635]: Deploying daemon mds.l on smithi174 2024-09-10T21:01:18.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:18 smithi104 ceph-mon[35325]: Deploying daemon mds.l on smithi174 2024-09-10T21:01:19.144 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:18 smithi174 systemd[1]: Starting Ceph mds.l for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:19.431 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 podman[65777]: 2024-09-10 21:01:19.04832739 +0000 UTC m=+0.016045814 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:19.432 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 podman[65777]: 2024-09-10 21:01:19.150609387 +0000 UTC m=+0.118327790 container create 6a4a62d2e96a4ae8db5bd85f55330ec43a10096cc2baf9b7230abb8f152f2f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T21:01:19.432 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 podman[65777]: 2024-09-10 21:01:19.428252285 +0000 UTC m=+0.395970690 container init 6a4a62d2e96a4ae8db5bd85f55330ec43a10096cc2baf9b7230abb8f152f2f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T21:01:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:19 smithi033 ceph-mon[30860]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:19 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1718675277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:19 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6832/34665907,v1:172.21.15.174:6833/34665907] up:boot 2024-09-10T21:01:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:19 smithi033 ceph-mon[30860]: fsmap 1 up:standby 2024-09-10T21:01:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:19 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:01:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:19 smithi104 ceph-mon[35325]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:19.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:19 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1718675277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:19.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:19 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6832/34665907,v1:172.21.15.174:6833/34665907] up:boot 2024-09-10T21:01:19.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:19 smithi104 ceph-mon[35325]: fsmap 1 up:standby 2024-09-10T21:01:19.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:19 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:01:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:19 smithi174 ceph-mon[35635]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:19 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1718675277' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:19 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6832/34665907,v1:172.21.15.174:6833/34665907] up:boot 2024-09-10T21:01:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:19 smithi174 ceph-mon[35635]: fsmap 1 up:standby 2024-09-10T21:01:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:01:19.894 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 podman[65777]: 2024-09-10 21:01:19.431391691 +0000 UTC m=+0.399110095 container start 6a4a62d2e96a4ae8db5bd85f55330ec43a10096cc2baf9b7230abb8f152f2f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T21:01:19.894 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 ceph-mds[65797]: -- 172.21.15.174:0/2788173763 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564780119860 con 0x5647800f7000 2024-09-10T21:01:19.895 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l[65793]: starting mds.l at 2024-09-10T21:01:19.895 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 bash[65777]: 6a4a62d2e96a4ae8db5bd85f55330ec43a10096cc2baf9b7230abb8f152f2f43 2024-09-10T21:01:19.895 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:01:19 smithi174 systemd[1]: Started Ceph mds.l for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:20 smithi174 ceph-mon[35635]: Deploying daemon mds.k on smithi104 2024-09-10T21:01:20.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:20.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:20.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:20.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:20.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:20.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:20.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:20 smithi033 ceph-mon[30860]: Deploying daemon mds.k on smithi104 2024-09-10T21:01:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:20 smithi104 ceph-mon[35325]: Deploying daemon mds.k on smithi104 2024-09-10T21:01:21.359 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:21.629 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:21 smithi033 ceph-mon[30860]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:21.805 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:21 smithi104 ceph-mon[35325]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:21 smithi174 ceph-mon[35635]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 -- 172.21.15.33:0/3994332828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57101087d0 msgr2=0x7f5710108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:22.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 --2- 172.21.15.33:0/3994332828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57101087d0 0x7f5710108c10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5700009900 tx=0x7f570002a410 comp rx=0 tx=0).stop 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 -- 172.21.15.33:0/3994332828 shutdown_connections 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 --2- 172.21.15.33:0/3994332828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5710109150 0x7f571010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 --2- 172.21.15.33:0/3994332828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57101087d0 0x7f5710108c10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 --2- 172.21.15.33:0/3994332828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5710102dc0 0x7f57101031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 -- 172.21.15.33:0/3994332828 >> 172.21.15.33:0/3994332828 conn(0x7f57100fe830 msgr2=0x7f5710100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 -- 172.21.15.33:0/3994332828 shutdown_connections 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 -- 172.21.15.33:0/3994332828 wait complete. 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 Processor -- start 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.083+0000 7f571770c640 1 -- start start 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f571770c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5710102dc0 0x7f57100754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f571770c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57101087d0 0x7f57100759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:22.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f571770c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5710109150 0x7f5710075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f571770c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5710112e60 con 0x7f5710109150 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f571770c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5710112ce0 con 0x7f57101087d0 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f571770c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5710112fe0 con 0x7f5710102dc0 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5710109150 0x7f5710075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5710109150 0x7f5710075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42178/0 (socket says 172.21.15.33:42178) 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 -- 172.21.15.33:0/784804520 learned_addr learned my addr 172.21.15.33:0/784804520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5714ca7640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57101087d0 0x7f57100759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f57154a8640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5710102dc0 0x7f57100754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 -- 172.21.15.33:0/784804520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5710102dc0 msgr2=0x7f57100754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5710102dc0 0x7f57100754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 -- 172.21.15.33:0/784804520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57101087d0 msgr2=0x7f57100759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57101087d0 0x7f57100759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 -- 172.21.15.33:0/784804520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57101a76d0 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f57154a8640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5710102dc0 0x7f57100754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.084+0000 7f5715ca9640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5710109150 0x7f5710075f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f570800cce0 tx=0x7f570800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.085+0000 7f57067fc640 1 -- 172.21.15.33:0/784804520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570800f450 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.085+0000 7f57067fc640 1 -- 172.21.15.33:0/784804520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f570800faa0 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.085+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57101a79c0 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.085+0000 7f57067fc640 1 -- 172.21.15.33:0/784804520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708018b70 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.085+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57101a7f00 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.086+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57101031a0 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.087+0000 7f57067fc640 1 -- 172.21.15.33:0/784804520 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f570800f5f0 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.088+0000 7f57067fc640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56fc0b52e0 0x7f56fc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.088+0000 7f57154a8640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56fc0b52e0 0x7f56fc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.088+0000 7f57067fc640 1 -- 172.21.15.33:0/784804520 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f57080e5f00 con 0x7f5710109150 2024-09-10T21:01:22.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.088+0000 7f57154a8640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56fc0b52e0 0x7f56fc0b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f570c006fd0 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:22.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.091+0000 7f57067fc640 1 -- 172.21.15.33:0/784804520 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5708014030 con 0x7f5710109150 2024-09-10T21:01:22.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.314+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f571002e5c0 con 0x7f5710109150 2024-09-10T21:01:22.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.314+0000 7f57067fc640 1 -- 172.21.15.33:0/784804520 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1432 (secure 0 0 0) 0x7f57080ad2f0 con 0x7f5710109150 2024-09-10T21:01:22.318 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:22.318 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":2,"btime":"2024-09-10T21:01:19:459530+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:01:22.319 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56fc0b52e0 msgr2=0x7f56fc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56fc0b52e0 0x7f56fc0b77a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f570c006fd0 tx=0x7f570c008040 comp rx=0 tx=0).stop 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5710109150 msgr2=0x7f5710075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5710109150 0x7f5710075f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f570800cce0 tx=0x7f570800a620 comp rx=0 tx=0).stop 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 shutdown_connections 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56fc0b52e0 0x7f56fc0b77a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5710109150 0x7f5710075f20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57101087d0 0x7f57100759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 --2- 172.21.15.33:0/784804520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5710102dc0 0x7f57100754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:22.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 >> 172.21.15.33:0/784804520 conn(0x7f57100fe830 msgr2=0x7f5710100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:22.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 shutdown_connections 2024-09-10T21:01:22.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:22.317+0000 7f571770c640 1 -- 172.21.15.33:0/784804520 wait complete. 2024-09-10T21:01:22.562 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:01:22 smithi104 systemd[1]: Started Ceph mds.k for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:23.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:23.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.422 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: Deploying daemon mds.j on smithi033 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/784804520' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6832/2049405106,v1:172.21.15.104:6833/2049405106] up:boot 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: fsmap 2 up:standby 2024-09-10T21:01:23.423 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:23 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:23.536 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: Deploying daemon mds.j on smithi033 2024-09-10T21:01:23.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/784804520' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:23.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6832/2049405106,v1:172.21.15.104:6833/2049405106] up:boot 2024-09-10T21:01:23.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: fsmap 2 up:standby 2024-09-10T21:01:23.537 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:23 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: Deploying daemon mds.j on smithi033 2024-09-10T21:01:23.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/784804520' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:23.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6832/2049405106,v1:172.21.15.104:6833/2049405106] up:boot 2024-09-10T21:01:23.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: fsmap 2 up:standby 2024-09-10T21:01:23.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:23 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:01:24.880 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:24 smithi033 systemd[1]: Starting Ceph mds.j for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:25.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:25.132 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:24 smithi033 podman[72779]: 2024-09-10 21:01:24.876488936 +0000 UTC m=+0.113365050 container create 99e5230000fcd4c2ff0c69b5a7b639e52cedfbc32f4ff7f08cf02f6640dee5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T21:01:25.133 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:24 smithi033 podman[72779]: 2024-09-10 21:01:24.779987309 +0000 UTC m=+0.016863372 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:25.391 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:25.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:25 smithi033 ceph-mon[30860]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:25.427 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:25 smithi033 podman[72779]: 2024-09-10 21:01:25.129530072 +0000 UTC m=+0.366406121 container init 99e5230000fcd4c2ff0c69b5a7b639e52cedfbc32f4ff7f08cf02f6640dee5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:01:25.427 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:25 smithi033 podman[72779]: 2024-09-10 21:01:25.13327899 +0000 UTC m=+0.370155039 container start 99e5230000fcd4c2ff0c69b5a7b639e52cedfbc32f4ff7f08cf02f6640dee5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T21:01:25.427 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:25 smithi033 ceph-mds[72815]: -- 172.21.15.33:0/1545394967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a1dc99860 con 0x555a1dc77000 2024-09-10T21:01:25.427 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:25 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j[72794]: starting mds.j at 2024-09-10T21:01:25.427 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:25 smithi033 bash[72779]: 99e5230000fcd4c2ff0c69b5a7b639e52cedfbc32f4ff7f08cf02f6640dee5d6 2024-09-10T21:01:25.427 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:01:25 smithi033 systemd[1]: Started Ceph mds.j for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:25.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:25 smithi104 ceph-mon[35325]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:25 smithi174 ceph-mon[35635]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 -- 172.21.15.33:0/2799606597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da4108060 msgr2=0x7f7da410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 --2- 172.21.15.33:0/2799606597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da4108060 0x7f7da410e860 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009ca0 tx=0x7f7d9c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:01:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 -- 172.21.15.33:0/2799606597 shutdown_connections 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 --2- 172.21.15.33:0/2799606597 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da4108060 0x7f7da410e860 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 --2- 172.21.15.33:0/2799606597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da41076e0 0x7f7da4107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 --2- 172.21.15.33:0/2799606597 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7da4101cd0 0x7f7da41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 -- 172.21.15.33:0/2799606597 >> 172.21.15.33:0/2799606597 conn(0x7f7da40fd700 msgr2=0x7f7da40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 -- 172.21.15.33:0/2799606597 shutdown_connections 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.857+0000 7f7da9719640 1 -- 172.21.15.33:0/2799606597 wait complete. 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 Processor -- start 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 -- start start 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7da4101cd0 0x7f7da41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da41076e0 0x7f7da4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4108060 0x7f7da419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4111d60 con 0x7f7da41076e0 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4111be0 con 0x7f7da4101cd0 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da9719640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7da4111ee0 con 0x7f7da4108060 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da41076e0 0x7f7da4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da41076e0 0x7f7da4199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42208/0 (socket says 172.21.15.33:42208) 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 -- 172.21.15.33:0/979871652 learned_addr learned my addr 172.21.15.33:0/979871652 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da2ffd640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7da4101cd0 0x7f7da41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da37fe640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4108060 0x7f7da419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 -- 172.21.15.33:0/979871652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4108060 msgr2=0x7f7da419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4108060 0x7f7da419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 -- 172.21.15.33:0/979871652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7da4101cd0 msgr2=0x7f7da41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7da4101cd0 0x7f7da41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.858+0000 7f7da27fc640 1 -- 172.21.15.33:0/979871652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da419f4a0 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.859+0000 7f7da27fc640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da41076e0 0x7f7da4199930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800b4a0 tx=0x7f7d9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.859+0000 7f7d8bfff640 1 -- 172.21.15.33:0/979871652 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98014070 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.859+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da419f790 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.859+0000 7f7d8bfff640 1 -- 172.21.15.33:0/979871652 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d98010ac0 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.859+0000 7f7d8bfff640 1 -- 172.21.15.33:0/979871652 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98017c30 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.859+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da41a5e50 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.860+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da4102130 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.861+0000 7f7d8bfff640 1 -- 172.21.15.33:0/979871652 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f7d98010610 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.862+0000 7f7d8bfff640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d8c0b52e0 0x7f7d8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.862+0000 7f7da2ffd640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d8c0b52e0 0x7f7d8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.862+0000 7f7d8bfff640 1 -- 172.21.15.33:0/979871652 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f7d980e5200 con 0x7f7da41076e0 2024-09-10T21:01:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.863+0000 7f7da2ffd640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d8c0b52e0 0x7f7d8c0b77a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009890 tx=0x7f7d94008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:25.865+0000 7f7d8bfff640 1 -- 172.21.15.33:0/979871652 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7d980ac4c0 con 0x7f7da41076e0 2024-09-10T21:01:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.096+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7da41a6140 con 0x7f7da41076e0 2024-09-10T21:01:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.096+0000 7f7d8bfff640 1 -- 172.21.15.33:0/979871652 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7f7d980abdc0 con 0x7f7da41076e0 2024-09-10T21:01:26.100 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:26.100 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":4,"btime":"2024-09-10T21:01:25:266869+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:01:26.101 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 4 2024-09-10T21:01:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.098+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d8c0b52e0 msgr2=0x7f7d8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:26.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.098+0000 7f7da9719640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d8c0b52e0 0x7f7d8c0b77a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7d94009890 tx=0x7f7d94008040 comp rx=0 tx=0).stop 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.098+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da41076e0 msgr2=0x7f7da4199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.098+0000 7f7da9719640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da41076e0 0x7f7da4199930 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800b4a0 tx=0x7f7d9800b960 comp rx=0 tx=0).stop 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 shutdown_connections 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d8c0b52e0 0x7f7d8c0b77a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7da4108060 0x7f7da419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7da41076e0 0x7f7da4199930 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 --2- 172.21.15.33:0/979871652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7da4101cd0 0x7f7da41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 >> 172.21.15.33:0/979871652 conn(0x7f7da40fd700 msgr2=0x7f7da4104cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 shutdown_connections 2024-09-10T21:01:26.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:26.099+0000 7f7da9719640 1 -- 172.21.15.33:0/979871652 wait complete. 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.33:6834/758676630,v1:172.21.15.33:6835/758676630] up:boot 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: fsmap 3 up:standby 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:26.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: Deploying daemon mds.d on smithi033 2024-09-10T21:01:26.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:26 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/979871652' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:26.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.33:6834/758676630,v1:172.21.15.33:6835/758676630] up:boot 2024-09-10T21:01:26.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: fsmap 3 up:standby 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: Deploying daemon mds.d on smithi033 2024-09-10T21:01:26.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:26 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/979871652' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:26.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.33:6834/758676630,v1:172.21.15.33:6835/758676630] up:boot 2024-09-10T21:01:26.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: fsmap 3 up:standby 2024-09-10T21:01:26.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-10T21:01:26.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:26.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:26.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:26.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:26.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: Deploying daemon mds.d on smithi033 2024-09-10T21:01:26.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:26 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/979871652' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:27.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:27 smithi104 ceph-mon[35325]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:27 smithi174 ceph-mon[35635]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:27.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:27 smithi033 ceph-mon[30860]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:28.427 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 systemd[1]: Starting Ceph mds.d for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:28.783 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 podman[73188]: 2024-09-10 21:01:28.41250133 +0000 UTC m=+0.016579933 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:28.783 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 podman[73188]: 2024-09-10 21:01:28.516268404 +0000 UTC m=+0.120346997 container create b73e89ef5e364920ffa8f6266d3afe84c823279dc5086e48e31b4deb005c7e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T21:01:28.783 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 podman[73188]: 2024-09-10 21:01:28.752292073 +0000 UTC m=+0.356370674 container init b73e89ef5e364920ffa8f6266d3afe84c823279dc5086e48e31b4deb005c7e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T21:01:28.783 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 podman[73188]: 2024-09-10 21:01:28.755057462 +0000 UTC m=+0.359136062 container start b73e89ef5e364920ffa8f6266d3afe84c823279dc5086e48e31b4deb005c7e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T21:01:29.177 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 ceph-mds[73207]: -- 172.21.15.33:0/1518878488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650694b7860 con 0x565069495000 2024-09-10T21:01:29.177 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d[73203]: starting mds.d at 2024-09-10T21:01:29.177 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 bash[73188]: b73e89ef5e364920ffa8f6266d3afe84c823279dc5086e48e31b4deb005c7e5b 2024-09-10T21:01:29.177 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:01:28 smithi033 systemd[1]: Started Ceph mds.d for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:29.382 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:29.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:29 smithi104 ceph-mon[35325]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:29.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:29.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:29.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:29 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:29 smithi174 ceph-mon[35635]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:29 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:29.645 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:29 smithi033 ceph-mon[30860]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:29 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 -- 172.21.15.33:0/2096871910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c1087d0 msgr2=0x7f225c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 --2- 172.21.15.33:0/2096871910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c1087d0 0x7f225c108c10 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f224c009900 tx=0x7f224c02a410 comp rx=0 tx=0).stop 2024-09-10T21:01:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 -- 172.21.15.33:0/2096871910 shutdown_connections 2024-09-10T21:01:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 --2- 172.21.15.33:0/2096871910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f225c109150 0x7f225c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 --2- 172.21.15.33:0/2096871910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c1087d0 0x7f225c108c10 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 --2- 172.21.15.33:0/2096871910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c102dc0 0x7f225c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 -- 172.21.15.33:0/2096871910 >> 172.21.15.33:0/2096871910 conn(0x7f225c0fe830 msgr2=0x7f225c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 -- 172.21.15.33:0/2096871910 shutdown_connections 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.156+0000 7f2264537640 1 -- 172.21.15.33:0/2096871910 wait complete. 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 Processor -- start 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 -- start start 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c102dc0 0x7f225c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f225c1087d0 0x7f225c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c109150 0x7f225c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f225c112e10 con 0x7f225c109150 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f225c112c90 con 0x7f225c102dc0 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2264537640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f225c112f90 con 0x7f225c1087d0 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.157+0000 7f2262ad4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c109150 0x7f225c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c109150 0x7f225c075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55840/0 (socket says 172.21.15.33:55840) 2024-09-10T21:01:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 -- 172.21.15.33:0/3223964186 learned_addr learned my addr 172.21.15.33:0/3223964186 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f22622d3640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c102dc0 0x7f225c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2261ad2640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f225c1087d0 0x7f225c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 -- 172.21.15.33:0/3223964186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f225c1087d0 msgr2=0x7f225c0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f225c1087d0 0x7f225c0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 -- 172.21.15.33:0/3223964186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c102dc0 msgr2=0x7f225c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c102dc0 0x7f225c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 -- 172.21.15.33:0/3223964186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c1a7700 con 0x7f225c109150 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2261ad2640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f225c1087d0 0x7f225c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f22622d3640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c102dc0 0x7f225c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2262ad4640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c109150 0x7f225c075f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f225400b970 tx=0x7f225400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f22537fe640 1 -- 172.21.15.33:0/3223964186 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254019070 con 0x7f225c109150 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f22537fe640 1 -- 172.21.15.33:0/3223964186 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2254014440 con 0x7f225c109150 2024-09-10T21:01:30.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2264537640 1 -- 172.21.15.33:0/3223964186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c1a79f0 con 0x7f225c109150 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f22537fe640 1 -- 172.21.15.33:0/3223964186 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22540075d0 con 0x7f225c109150 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.158+0000 7f2264537640 1 -- 172.21.15.33:0/3223964186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c1a7fd0 con 0x7f225c109150 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.159+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2244005290 con 0x7f225c109150 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.162+0000 7f22537fe640 1 -- 172.21.15.33:0/3223964186 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f225400cb40 con 0x7f225c109150 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.162+0000 7f22537fe640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22480b5360 0x7f22480b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.163+0000 7f22622d3640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22480b5360 0x7f22480b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.163+0000 7f22537fe640 1 -- 172.21.15.33:0/3223964186 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f22540e52e0 con 0x7f225c109150 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.163+0000 7f22622d3640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22480b5360 0x7f22480b7820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2258009fc0 tx=0x7f2258005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:30.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.166+0000 7f22537fe640 1 -- 172.21.15.33:0/3223964186 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f22540abff0 con 0x7f225c109150 2024-09-10T21:01:30.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.390+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2244005580 con 0x7f225c109150 2024-09-10T21:01:30.394 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:30 smithi033 ceph-mon[30860]: Deploying daemon mds.e on smithi104 2024-09-10T21:01:30.394 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:30 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.33:6836/1550204890,v1:172.21.15.33:6837/1550204890] up:boot 2024-09-10T21:01:30.394 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:30 smithi033 ceph-mon[30860]: fsmap 4 up:standby 2024-09-10T21:01:30.394 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:30 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T21:01:30.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.390+0000 7f22537fe640 1 -- 172.21.15.33:0/3223964186 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7f22540b0a20 con 0x7f225c109150 2024-09-10T21:01:30.395 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:30.395 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":5,"btime":"2024-09-10T21:01:29:304675+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:01:30.395 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 5 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.392+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22480b5360 msgr2=0x7f22480b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.392+0000 7f22517fa640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22480b5360 0x7f22480b7820 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2258009fc0 tx=0x7f2258005f90 comp rx=0 tx=0).stop 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.392+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c109150 msgr2=0x7f225c075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.392+0000 7f22517fa640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c109150 0x7f225c075f20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f225400b970 tx=0x7f225400be30 comp rx=0 tx=0).stop 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 shutdown_connections 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22480b5360 0x7f22480b7820 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f225c109150 0x7f225c075f20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f225c1087d0 0x7f225c0759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 --2- 172.21.15.33:0/3223964186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f225c102dc0 0x7f225c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 >> 172.21.15.33:0/3223964186 conn(0x7f225c0fe830 msgr2=0x7f225c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:30.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 shutdown_connections 2024-09-10T21:01:30.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:30.393+0000 7f22517fa640 1 -- 172.21.15.33:0/3223964186 wait complete. 2024-09-10T21:01:30.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:30 smithi174 ceph-mon[35635]: Deploying daemon mds.e on smithi104 2024-09-10T21:01:30.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:30 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.33:6836/1550204890,v1:172.21.15.33:6837/1550204890] up:boot 2024-09-10T21:01:30.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:30 smithi174 ceph-mon[35635]: fsmap 4 up:standby 2024-09-10T21:01:30.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:30 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T21:01:30.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:30 smithi104 ceph-mon[35325]: Deploying daemon mds.e on smithi104 2024-09-10T21:01:30.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:30 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.33:6836/1550204890,v1:172.21.15.33:6837/1550204890] up:boot 2024-09-10T21:01:30.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:30 smithi104 ceph-mon[35325]: fsmap 4 up:standby 2024-09-10T21:01:30.763 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:30 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-10T21:01:31.063 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:30 smithi104 systemd[1]: Starting Ceph mds.e for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:31.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:31 smithi033 ceph-mon[30860]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:31.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:31 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3223964186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:31.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:31 smithi104 ceph-mon[35325]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:31.464 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:31 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3223964186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:31.464 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 podman[65043]: 2024-09-10 21:01:31.056229474 +0000 UTC m=+0.015690603 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:31.464 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 podman[65043]: 2024-09-10 21:01:31.166642714 +0000 UTC m=+0.126103839 container create 97bde8c1f2eec9c03ca2a15b0bbd8b073ab78ed610e1acbcdcb7c60584741a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T21:01:31.465 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 podman[65043]: 2024-09-10 21:01:31.460992876 +0000 UTC m=+0.420454002 container init 97bde8c1f2eec9c03ca2a15b0bbd8b073ab78ed610e1acbcdcb7c60584741a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:01:31.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:31 smithi174 ceph-mon[35635]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:31.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:31 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3223964186' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:31.812 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 podman[65043]: 2024-09-10 21:01:31.464050937 +0000 UTC m=+0.423512051 container start 97bde8c1f2eec9c03ca2a15b0bbd8b073ab78ed610e1acbcdcb7c60584741a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:01:31.813 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 ceph-mds[65062]: -- 172.21.15.104:0/3880402686 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579cb431c20 con 0x5579cb40f000 2024-09-10T21:01:31.813 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e[65058]: starting mds.e at 2024-09-10T21:01:31.813 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 bash[65043]: 97bde8c1f2eec9c03ca2a15b0bbd8b073ab78ed610e1acbcdcb7c60584741a44 2024-09-10T21:01:31.813 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:01:31 smithi104 systemd[1]: Started Ceph mds.e for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:32.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:32.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:32.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:32.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:32.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:32.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:32.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: Deploying daemon mds.f on smithi174 2024-09-10T21:01:32.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6834/1051931570,v1:172.21.15.104:6835/1051931570] up:boot 2024-09-10T21:01:32.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: fsmap 5 up:standby 2024-09-10T21:01:32.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:32 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:01:32.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:32.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:32.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:32.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:32.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:32.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:32.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: Deploying daemon mds.f on smithi174 2024-09-10T21:01:32.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6834/1051931570,v1:172.21.15.104:6835/1051931570] up:boot 2024-09-10T21:01:32.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: fsmap 5 up:standby 2024-09-10T21:01:32.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:32 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:01:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: Deploying daemon mds.f on smithi174 2024-09-10T21:01:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6834/1051931570,v1:172.21.15.104:6835/1051931570] up:boot 2024-09-10T21:01:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: fsmap 5 up:standby 2024-09-10T21:01:33.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:32 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:01:33.112 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:33.644 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:33 smithi174 systemd[1]: Starting Ceph mds.f for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:33.819 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:33 smithi033 ceph-mon[30860]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:33.906 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:33 smithi174 ceph-mon[35635]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:33.907 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:33 smithi174 podman[66080]: 2024-09-10 21:01:33.678263148 +0000 UTC m=+0.015555758 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:33.907 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:33 smithi174 podman[66080]: 2024-09-10 21:01:33.798370673 +0000 UTC m=+0.135663284 container create 121a532764b9dcbc0cecd931b57ac216a95df69195bbfa15031edd6ba606dbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 -- 172.21.15.33:0/972247498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c109150 msgr2=0x7f521c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 --2- 172.21.15.33:0/972247498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c109150 0x7f521c10f950 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f520c009ca0 tx=0x7f520c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 -- 172.21.15.33:0/972247498 shutdown_connections 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 --2- 172.21.15.33:0/972247498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c109150 0x7f521c10f950 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 --2- 172.21.15.33:0/972247498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f521c1087d0 0x7f521c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 --2- 172.21.15.33:0/972247498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f521c102dc0 0x7f521c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 -- 172.21.15.33:0/972247498 >> 172.21.15.33:0/972247498 conn(0x7f521c0fe830 msgr2=0x7f521c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 -- 172.21.15.33:0/972247498 shutdown_connections 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 -- 172.21.15.33:0/972247498 wait complete. 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.898+0000 7f5223a81640 1 Processor -- start 2024-09-10T21:01:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f5223a81640 1 -- start start 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f5223a81640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c102dc0 0x7f521c199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f5223a81640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f521c1087d0 0x7f521c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f5223a81640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f521c109150 0x7f521c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f5223a81640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f521c112e10 con 0x7f521c102dc0 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f5223a81640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f521c112c90 con 0x7f521c109150 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f5223a81640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f521c112f90 con 0x7f521c1087d0 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c102dc0 0x7f521c199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c102dc0 0x7f521c199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55848/0 (socket says 172.21.15.33:55848) 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 -- 172.21.15.33:0/1386254590 learned_addr learned my addr 172.21.15.33:0/1386254590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522101c640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f521c1087d0 0x7f521c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522201e640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f521c109150 0x7f521c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 -- 172.21.15.33:0/1386254590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f521c1087d0 msgr2=0x7f521c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f521c1087d0 0x7f521c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 -- 172.21.15.33:0/1386254590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f521c109150 msgr2=0x7f521c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:33.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f521c109150 0x7f521c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 -- 172.21.15.33:0/1386254590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f521c19f3b0 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.899+0000 7f522181d640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c102dc0 0x7f521c199300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f521400e9f0 tx=0x7f521400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.900+0000 7f521affd640 1 -- 172.21.15.33:0/1386254590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521400cd80 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.900+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f521c19f6a0 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.900+0000 7f521affd640 1 -- 172.21.15.33:0/1386254590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52140054d0 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.900+0000 7f521affd640 1 -- 172.21.15.33:0/1386254590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5214010690 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.900+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521c1a5d60 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.901+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f521c1031a0 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.902+0000 7f521affd640 1 -- 172.21.15.33:0/1386254590 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f5214004e30 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.903+0000 7f521affd640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52040b52e0 0x7f52040b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.903+0000 7f522101c640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52040b52e0 0x7f52040b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.903+0000 7f521affd640 1 -- 172.21.15.33:0/1386254590 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f52140e5d10 con 0x7f521c102dc0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.903+0000 7f522101c640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52040b52e0 0x7f52040b77a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f521c102070 tx=0x7f521000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:33.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:33.906+0000 7f521affd640 1 -- 172.21.15.33:0/1386254590 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52140ad070 con 0x7f521c102dc0 2024-09-10T21:01:33.997 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:33 smithi104 ceph-mon[35325]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:34.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.127+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f521c19a430 con 0x7f521c102dc0 2024-09-10T21:01:34.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.128+0000 7f521affd640 1 -- 172.21.15.33:0/1386254590 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7f52140ac970 con 0x7f521c102dc0 2024-09-10T21:01:34.132 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:34.132 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":6,"btime":"2024-09-10T21:01:32:317554+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-10T21:01:34.132 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 6 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52040b52e0 msgr2=0x7f52040b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52040b52e0 0x7f52040b77a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f521c102070 tx=0x7f521000b410 comp rx=0 tx=0).stop 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c102dc0 msgr2=0x7f521c199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c102dc0 0x7f521c199300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f521400e9f0 tx=0x7f521400edf0 comp rx=0 tx=0).stop 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 shutdown_connections 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52040b52e0 0x7f52040b77a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f521c109150 0x7f521c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f521c1087d0 0x7f521c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 --2- 172.21.15.33:0/1386254590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f521c102dc0 0x7f521c199300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 >> 172.21.15.33:0/1386254590 conn(0x7f521c0fe830 msgr2=0x7f521c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 shutdown_connections 2024-09-10T21:01:34.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:34.131+0000 7f5223a81640 1 -- 172.21.15.33:0/1386254590 wait complete. 2024-09-10T21:01:34.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:34 smithi174 podman[66080]: 2024-09-10 21:01:34.026362712 +0000 UTC m=+0.363655335 container init 121a532764b9dcbc0cecd931b57ac216a95df69195bbfa15031edd6ba606dbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-09-10T21:01:34.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:34 smithi174 podman[66080]: 2024-09-10 21:01:34.029353272 +0000 UTC m=+0.366645887 container start 121a532764b9dcbc0cecd931b57ac216a95df69195bbfa15031edd6ba606dbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:01:34.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mds[66099]: -- 172.21.15.174:0/1672444588 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e0af91860 con 0x559e0af6f400 2024-09-10T21:01:34.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f[66095]: starting mds.f at 2024-09-10T21:01:34.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:34 smithi174 bash[66080]: 121a532764b9dcbc0cecd931b57ac216a95df69195bbfa15031edd6ba606dbbb 2024-09-10T21:01:34.395 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:01:34 smithi174 systemd[1]: Started Ceph mds.f for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:34.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:34 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1386254590' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:34.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:34.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:34.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:34.809 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:34.810 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:34.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1386254590' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:34.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:34.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:34.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:34.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:34.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:34.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:35.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:34 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1386254590' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:35.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:35.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:35 smithi174 ceph-mon[35635]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:35.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:35 smithi174 ceph-mon[35635]: Deploying daemon mds.c on smithi174 2024-09-10T21:01:35.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:35 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] up:boot 2024-09-10T21:01:35.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:35 smithi174 ceph-mon[35635]: fsmap 6 up:standby 2024-09-10T21:01:35.747 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T21:01:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:35 smithi033 ceph-mon[30860]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:35 smithi033 ceph-mon[30860]: Deploying daemon mds.c on smithi174 2024-09-10T21:01:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:35 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] up:boot 2024-09-10T21:01:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:35 smithi033 ceph-mon[30860]: fsmap 6 up:standby 2024-09-10T21:01:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T21:01:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:35 smithi104 ceph-mon[35325]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:36.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:35 smithi104 ceph-mon[35325]: Deploying daemon mds.c on smithi174 2024-09-10T21:01:36.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:35 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] up:boot 2024-09-10T21:01:36.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:35 smithi104 ceph-mon[35325]: fsmap 6 up:standby 2024-09-10T21:01:36.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T21:01:36.394 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 systemd[1]: Starting Ceph mds.c for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:36.718 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 podman[66382]: 2024-09-10 21:01:36.325011222 +0000 UTC m=+0.015774019 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:36.718 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 podman[66382]: 2024-09-10 21:01:36.445818105 +0000 UTC m=+0.136580888 container create b261a3a6c7b86220561791759b778a3b396a66dbc090e1fac4f8962cd0c54884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T21:01:36.719 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 podman[66382]: 2024-09-10 21:01:36.715207782 +0000 UTC m=+0.405970566 container init b261a3a6c7b86220561791759b778a3b396a66dbc090e1fac4f8962cd0c54884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T21:01:36.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:37.144 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 podman[66382]: 2024-09-10 21:01:36.718161578 +0000 UTC m=+0.408924365 container start b261a3a6c7b86220561791759b778a3b396a66dbc090e1fac4f8962cd0c54884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T21:01:37.144 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 ceph-mds[66402]: -- 172.21.15.174:0/208250794 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa2734f860 con 0x55aa2732d400 2024-09-10T21:01:37.144 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c[66398]: starting mds.c at 2024-09-10T21:01:37.144 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 bash[66382]: b261a3a6c7b86220561791759b778a3b396a66dbc090e1fac4f8962cd0c54884 2024-09-10T21:01:37.144 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:01:36 smithi174 systemd[1]: Started Ceph mds.c for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:37.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 -- 172.21.15.33:0/2854434515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8101830 msgr2=0x7f95c8101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:37.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/2854434515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8101830 0x7f95c8101c10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009900 tx=0x7f95b802a480 comp rx=0 tx=0).stop 2024-09-10T21:01:37.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 -- 172.21.15.33:0/2854434515 shutdown_connections 2024-09-10T21:01:37.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/2854434515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c810f3e0 0x7f95c81117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/2854434515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8102150 0x7f95c810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/2854434515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8101830 0x7f95c8101c10 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 -- 172.21.15.33:0/2854434515 >> 172.21.15.33:0/2854434515 conn(0x7f95c80fd700 msgr2=0x7f95c80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 -- 172.21.15.33:0/2854434515 shutdown_connections 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.598+0000 7f95ccd4f640 1 -- 172.21.15.33:0/2854434515 wait complete. 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 Processor -- start 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 -- start start 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8101830 0x7f95c819b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8102150 0x7f95c819b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c810f3e0 0x7f95c81a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8114f30 con 0x7f95c8102150 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f95c8114db0 con 0x7f95c8101830 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95ccd4f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95c81150b0 con 0x7f95c810f3e0 2024-09-10T21:01:37.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95c659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8101830 0x7f95c819b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95c659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8101830 0x7f95c819b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:32950/0 (socket says 172.21.15.33:32950) 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95c659c640 1 -- 172.21.15.33:0/122661205 learned_addr learned my addr 172.21.15.33:0/122661205 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.599+0000 7f95c5d9b640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8102150 0x7f95c819b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c6d9d640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c810f3e0 0x7f95c81a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c5d9b640 1 -- 172.21.15.33:0/122661205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c810f3e0 msgr2=0x7f95c81a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c5d9b640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c810f3e0 0x7f95c81a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c5d9b640 1 -- 172.21.15.33:0/122661205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8101830 msgr2=0x7f95c819b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c5d9b640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8101830 0x7f95c819b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c5d9b640 1 -- 172.21.15.33:0/122661205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c81a1530 con 0x7f95c8102150 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c659c640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8101830 0x7f95c819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95c5d9b640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8102150 0x7f95c819b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95b77fe640 1 -- 172.21.15.33:0/122661205 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc00ccc0 con 0x7f95c8102150 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95b77fe640 1 -- 172.21.15.33:0/122661205 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95bc0108e0 con 0x7f95c8102150 2024-09-10T21:01:37.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c81a1820 con 0x7f95c8102150 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95b77fe640 1 -- 172.21.15.33:0/122661205 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc018880 con 0x7f95c8102150 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.600+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c81a7ee0 con 0x7f95c8102150 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.601+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c8101c10 con 0x7f95c8102150 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.604+0000 7f95b77fe640 1 -- 172.21.15.33:0/122661205 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f95bc010430 con 0x7f95c8102150 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.605+0000 7f95b77fe640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95b00b52e0 0x7f95b00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.605+0000 7f95c659c640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95b00b52e0 0x7f95b00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.605+0000 7f95b77fe640 1 -- 172.21.15.33:0/122661205 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f95bc0e5ed0 con 0x7f95c8102150 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.606+0000 7f95c659c640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95b00b52e0 0x7f95b00b77a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f95b8031440 tx=0x7f95b80058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:37.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.607+0000 7f95b77fe640 1 -- 172.21.15.33:0/122661205 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f95bc0acbc0 con 0x7f95c8102150 2024-09-10T21:01:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.837+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f95c819c5b0 con 0x7f95c8102150 2024-09-10T21:01:37.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:37 smithi033 ceph-mon[30860]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:37.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:37.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:37.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:37.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:37.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:37.842 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.838+0000 7f95b77fe640 1 -- 172.21.15.33:0/122661205 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6388 (secure 0 0 0) 0x7f95bc0b15f0 con 0x7f95c8102150 2024-09-10T21:01:37.843 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:37.843 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":8,"btime":"2024-09-10T21:01:37:654759+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-10T21:01:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 8 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.840+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95b00b52e0 msgr2=0x7f95b00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.840+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95b00b52e0 0x7f95b00b77a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f95b8031440 tx=0x7f95b80058d0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.840+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8102150 msgr2=0x7f95c819b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.840+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8102150 0x7f95c819b9c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f95bc00e9f0 tx=0x7f95bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.840+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 shutdown_connections 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.841+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95b00b52e0 0x7f95b00b77a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.841+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c810f3e0 0x7f95c81a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.841+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95c8102150 0x7f95c819b9c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.841+0000 7f95ccd4f640 1 --2- 172.21.15.33:0/122661205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95c8101830 0x7f95c819b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.841+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 >> 172.21.15.33:0/122661205 conn(0x7f95c80fd700 msgr2=0x7f95c80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.841+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 shutdown_connections 2024-09-10T21:01:37.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:37.841+0000 7f95ccd4f640 1 -- 172.21.15.33:0/122661205 wait complete. 2024-09-10T21:01:38.046 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:37 smithi104 ceph-mon[35325]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:38.047 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:38.047 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:38.047 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:38.047 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:38.047 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:38.047 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:37 smithi174 ceph-mon[35635]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:38.754 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:38 smithi104 ceph-mon[35325]: Deploying daemon mds.h on smithi104 2024-09-10T21:01:38.754 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:38 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] up:boot 2024-09-10T21:01:38.754 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:38 smithi104 ceph-mon[35325]: fsmap 7 up:standby 2024-09-10T21:01:38.754 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:38 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T21:01:38.754 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:38 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/122661205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:38.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:38 smithi033 ceph-mon[30860]: Deploying daemon mds.h on smithi104 2024-09-10T21:01:38.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:38 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] up:boot 2024-09-10T21:01:38.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:38 smithi033 ceph-mon[30860]: fsmap 7 up:standby 2024-09-10T21:01:38.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:38 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T21:01:38.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:38 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/122661205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:39.062 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:38 smithi104 systemd[1]: Starting Ceph mds.h for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:38 smithi174 ceph-mon[35635]: Deploying daemon mds.h on smithi104 2024-09-10T21:01:39.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:38 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] up:boot 2024-09-10T21:01:39.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:38 smithi174 ceph-mon[35635]: fsmap 7 up:standby 2024-09-10T21:01:39.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:38 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T21:01:39.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:38 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/122661205' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:39.419 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 podman[65344]: 2024-09-10 21:01:39.049450938 +0000 UTC m=+0.016198529 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:39.419 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 podman[65344]: 2024-09-10 21:01:39.171267105 +0000 UTC m=+0.138014692 container create 4f038929f650d89ab48b569171516a53f8739e98d70d61e144676d0d78ee6566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T21:01:39.419 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 podman[65344]: 2024-09-10 21:01:39.415682945 +0000 UTC m=+0.382430523 container init 4f038929f650d89ab48b569171516a53f8739e98d70d61e144676d0d78ee6566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T21:01:39.671 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 podman[65344]: 2024-09-10 21:01:39.418530351 +0000 UTC m=+0.385277940 container start 4f038929f650d89ab48b569171516a53f8739e98d70d61e144676d0d78ee6566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T21:01:39.671 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mds[65364]: -- 172.21.15.104:0/34570451 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d2adc63860 con 0x55d2adc40800 2024-09-10T21:01:39.671 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h[65360]: starting mds.h at 2024-09-10T21:01:39.671 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 bash[65344]: 4f038929f650d89ab48b569171516a53f8739e98d70d61e144676d0d78ee6566 2024-09-10T21:01:39.671 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:01:39 smithi104 systemd[1]: Started Ceph mds.h for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:39 smithi033 ceph-mon[30860]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:39.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:39 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:40.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:39 smithi174 ceph-mon[35635]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:40.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:40.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:40.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:40.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:40.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:40.056 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:39 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mon[35325]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:39 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:40.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:40.797 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:40.918 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:40 smithi033 ceph-mon[30860]: Deploying daemon mds.i on smithi174 2024-09-10T21:01:40.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:40 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] up:boot 2024-09-10T21:01:40.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:40 smithi033 ceph-mon[30860]: fsmap 8 up:standby 2024-09-10T21:01:40.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T21:01:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:40 smithi104 ceph-mon[35325]: Deploying daemon mds.i on smithi174 2024-09-10T21:01:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:40 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] up:boot 2024-09-10T21:01:41.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:40 smithi104 ceph-mon[35325]: fsmap 8 up:standby 2024-09-10T21:01:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T21:01:41.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:40 smithi174 ceph-mon[35635]: Deploying daemon mds.i on smithi174 2024-09-10T21:01:41.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:40 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] up:boot 2024-09-10T21:01:41.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:40 smithi174 ceph-mon[35635]: fsmap 8 up:standby 2024-09-10T21:01:41.109 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T21:01:41.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 -- 172.21.15.33:0/3856997276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 msgr2=0x7f4d5410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:41.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/3856997276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d5410cab0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d48009900 tx=0x7f4d4802a410 comp rx=0 tx=0).stop 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 -- 172.21.15.33:0/3856997276 shutdown_connections 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/3856997276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d5410d100 0x7f4d5410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/3856997276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d5410cab0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/3856997276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d54103ca0 0x7f4d54104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 -- 172.21.15.33:0/3856997276 >> 172.21.15.33:0/3856997276 conn(0x7f4d540fd700 msgr2=0x7f4d540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 -- 172.21.15.33:0/3856997276 shutdown_connections 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.259+0000 7f4d5ab04640 1 -- 172.21.15.33:0/3856997276 wait complete. 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 Processor -- start 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 -- start start 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54103ca0 0x7f4d54199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d54199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d5410d100 0x7f4d5419ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54112e30 con 0x7f4d541045c0 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54112cb0 con 0x7f4d5410d100 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d5ab04640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54112fb0 con 0x7f4d54103ca0 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d54199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d54199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60984/0 (socket says 172.21.15.33:60984) 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d53fff640 1 -- 172.21.15.33:0/2465543687 learned_addr learned my addr 172.21.15.33:0/2465543687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.260+0000 7f4d588a0640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54103ca0 0x7f4d54199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d590a1640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d5410d100 0x7f4d5419ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d53fff640 1 -- 172.21.15.33:0/2465543687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54103ca0 msgr2=0x7f4d54199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d53fff640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54103ca0 0x7f4d54199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d53fff640 1 -- 172.21.15.33:0/2465543687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d5410d100 msgr2=0x7f4d5419ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d53fff640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d5410d100 0x7f4d5419ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d53fff640 1 -- 172.21.15.33:0/2465543687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d5419f4d0 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d590a1640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d5410d100 0x7f4d5419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d53fff640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d54199960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4d48004270 tx=0x7f4d480042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d51ffb640 1 -- 172.21.15.33:0/2465543687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4803f030 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d51ffb640 1 -- 172.21.15.33:0/2465543687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d480044c0 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d51ffb640 1 -- 172.21.15.33:0/2465543687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d48035070 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d5419f760 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.261+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d541a5e80 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.262+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d54104100 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.264+0000 7f4d51ffb640 1 -- 172.21.15.33:0/2465543687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f4d4803bad0 con 0x7f4d541045c0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.264+0000 7f4d51ffb640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d3c0b5360 0x7f4d3c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.265+0000 7f4d588a0640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d3c0b5360 0x7f4d3c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.265+0000 7f4d51ffb640 1 -- 172.21.15.33:0/2465543687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f4d481062c0 con 0x7f4d541045c0 2024-09-10T21:01:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.265+0000 7f4d588a0640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d3c0b5360 0x7f4d3c0b7820 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009990 tx=0x7f4d44009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.267+0000 7f4d51ffb640 1 -- 172.21.15.33:0/2465543687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4d480cd620 con 0x7f4d541045c0 2024-09-10T21:01:41.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.488+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d5419a550 con 0x7f4d541045c0 2024-09-10T21:01:41.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.489+0000 7f4d51ffb640 1 -- 172.21.15.33:0/2465543687 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7216 (secure 0 0 0) 0x7f4d480ccf20 con 0x7f4d541045c0 2024-09-10T21:01:41.493 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:41.493 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":9,"btime":"2024-09-10T21:01:39:666542+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-10T21:01:41.494 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 9 2024-09-10T21:01:41.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d3c0b5360 msgr2=0x7f4d3c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:41.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d3c0b5360 0x7f4d3c0b7820 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4d44009990 tx=0x7f4d44009110 comp rx=0 tx=0).stop 2024-09-10T21:01:41.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 msgr2=0x7f4d54199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d54199960 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4d48004270 tx=0x7f4d480042a0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 shutdown_connections 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d3c0b5360 0x7f4d3c0b7820 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d5410d100 0x7f4d5419ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d541045c0 0x7f4d54199960 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 --2- 172.21.15.33:0/2465543687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d54103ca0 0x7f4d54199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.491+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 >> 172.21.15.33:0/2465543687 conn(0x7f4d540fd700 msgr2=0x7f4d540fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.492+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 shutdown_connections 2024-09-10T21:01:41.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:41.492+0000 7f4d5ab04640 1 -- 172.21.15.33:0/2465543687 wait complete. 2024-09-10T21:01:41.644 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:41 smithi174 systemd[1]: Starting Ceph mds.i for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:41.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:41 smithi033 ceph-mon[30860]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:41.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:41 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2465543687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:41.937 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:41 smithi174 ceph-mon[35635]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:41.937 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:41 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2465543687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:41.937 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:41 smithi174 podman[66683]: 2024-09-10 21:01:41.69250713 +0000 UTC m=+0.016906710 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:41.937 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:41 smithi174 podman[66683]: 2024-09-10 21:01:41.808908199 +0000 UTC m=+0.133307821 container create d48521b7c06ec6a08e9dcc8ae56327ac1b8421a9060862537289141e0d440ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:01:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:41 smithi104 ceph-mon[35325]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:41 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2465543687' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:42.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:42 smithi174 podman[66683]: 2024-09-10 21:01:42.044879322 +0000 UTC m=+0.369278895 container init d48521b7c06ec6a08e9dcc8ae56327ac1b8421a9060862537289141e0d440ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T21:01:42.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:42 smithi174 podman[66683]: 2024-09-10 21:01:42.047861195 +0000 UTC m=+0.372260771 container start d48521b7c06ec6a08e9dcc8ae56327ac1b8421a9060862537289141e0d440ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:01:42.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:42 smithi174 ceph-mds[66702]: -- 172.21.15.174:0/2752637758 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bbc4df7860 con 0x55bbc4dd5000 2024-09-10T21:01:42.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:42 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i[66698]: starting mds.i at 2024-09-10T21:01:42.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:42 smithi174 bash[66683]: d48521b7c06ec6a08e9dcc8ae56327ac1b8421a9060862537289141e0d440ede 2024-09-10T21:01:42.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:01:42 smithi174 systemd[1]: Started Ceph mds.i for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: Deploying daemon mds.g on smithi033 2024-09-10T21:01:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] up:boot 2024-09-10T21:01:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: fsmap 9 up:standby 2024-09-10T21:01:43.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:43 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T21:01:43.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:43.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: Deploying daemon mds.g on smithi033 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] up:boot 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: fsmap 9 up:standby 2024-09-10T21:01:43.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:43 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T21:01:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:43.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:43.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: Deploying daemon mds.g on smithi033 2024-09-10T21:01:43.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] up:boot 2024-09-10T21:01:43.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: fsmap 9 up:standby 2024-09-10T21:01:43.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:43 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T21:01:44.191 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:44.427 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 systemd[1]: Starting Ceph mds.g for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:44.749 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 podman[73976]: 2024-09-10 21:01:44.374090929 +0000 UTC m=+0.016640305 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:44.749 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 podman[73976]: 2024-09-10 21:01:44.517442052 +0000 UTC m=+0.159991410 container create dd8736cf14872c8885025478e38b596476becad1824d2bea9f1e68a9ff13179b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T21:01:44.749 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 podman[73976]: 2024-09-10 21:01:44.745337083 +0000 UTC m=+0.387886451 container init dd8736cf14872c8885025478e38b596476becad1824d2bea9f1e68a9ff13179b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T21:01:44.942 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:45.064 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 podman[73976]: 2024-09-10 21:01:44.748624914 +0000 UTC m=+0.391174273 container start dd8736cf14872c8885025478e38b596476becad1824d2bea9f1e68a9ff13179b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T21:01:45.064 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 ceph-mds[74020]: -- 172.21.15.33:0/1322860831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fc76c37c20 con 0x55fc76c15400 2024-09-10T21:01:45.064 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g[74004]: starting mds.g at 2024-09-10T21:01:45.064 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 bash[73976]: dd8736cf14872c8885025478e38b596476becad1824d2bea9f1e68a9ff13179b 2024-09-10T21:01:45.064 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:01:44 smithi033 systemd[1]: Started Ceph mds.g for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:45.353 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:45 smithi033 ceph-mon[30860]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:45.353 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.353 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.353 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.353 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:45.353 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:45.354 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:45.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 -- 172.21.15.33:0/4091461174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c41076a0 msgr2=0x7f80c4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 --2- 172.21.15.33:0/4091461174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c41076a0 0x7f80c4107ae0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f80ac009900 tx=0x7f80ac02a410 comp rx=0 tx=0).stop 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 -- 172.21.15.33:0/4091461174 shutdown_connections 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 --2- 172.21.15.33:0/4091461174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80c4108020 0x7f80c410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 --2- 172.21.15.33:0/4091461174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c41076a0 0x7f80c4107ae0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 --2- 172.21.15.33:0/4091461174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80c4101c90 0x7f80c4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 -- 172.21.15.33:0/4091461174 >> 172.21.15.33:0/4091461174 conn(0x7f80c40fd700 msgr2=0x7f80c40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 -- 172.21.15.33:0/4091461174 shutdown_connections 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.432+0000 7f80ca386640 1 -- 172.21.15.33:0/4091461174 wait complete. 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 Processor -- start 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 -- start start 2024-09-10T21:01:45.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80c4101c90 0x7f80c40fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80c41076a0 0x7f80c40fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c4108020 0x7f80c40fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4111ce0 con 0x7f80c4108020 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4111b60 con 0x7f80c4101c90 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80ca386640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80c4111e60 con 0x7f80c41076a0 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80c8923640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c4108020 0x7f80c40fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80c8923640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c4108020 0x7f80c40fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32788/0 (socket says 172.21.15.33:32788) 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80c8923640 1 -- 172.21.15.33:0/113449185 learned_addr learned my addr 172.21.15.33:0/113449185 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.433+0000 7f80c3fff640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80c4101c90 0x7f80c40fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80bbfff640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80c41076a0 0x7f80c40fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:45.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c8923640 1 -- 172.21.15.33:0/113449185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80c41076a0 msgr2=0x7f80c40fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c8923640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80c41076a0 0x7f80c40fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c8923640 1 -- 172.21.15.33:0/113449185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80c4101c90 msgr2=0x7f80c40fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c8923640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80c4101c90 0x7f80c40fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c8923640 1 -- 172.21.15.33:0/113449185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c41a7690 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c8923640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c4108020 0x7f80c40fe680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00eaa0 tx=0x7f80bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c1ffb640 1 -- 172.21.15.33:0/113449185 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc00cdb0 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c41a7980 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c1ffb640 1 -- 172.21.15.33:0/113449185 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80bc005020 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80c1ffb640 1 -- 172.21.15.33:0/113449185 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0116d0 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.434+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c41a7e40 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.435+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c4102070 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.437+0000 7f80c1ffb640 1 -- 172.21.15.33:0/113449185 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f80bc00f040 con 0x7f80c4108020 2024-09-10T21:01:45.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.437+0000 7f80c1ffb640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80b00b52e0 0x7f80b00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.438+0000 7f80c3fff640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80b00b52e0 0x7f80b00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.438+0000 7f80c1ffb640 1 -- 172.21.15.33:0/113449185 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f80bc0e5980 con 0x7f80c4108020 2024-09-10T21:01:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.438+0000 7f80c3fff640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80b00b52e0 0x7f80b00b77a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80b400bbe0 tx=0x7f80b400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:45.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.440+0000 7f80c1ffb640 1 -- 172.21.15.33:0/113449185 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f80bc016030 con 0x7f80c4108020 2024-09-10T21:01:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:45 smithi104 ceph-mon[35325]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:45.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:45 smithi174 ceph-mon[35635]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:45.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.667+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80c40fef00 con 0x7f80c4108020 2024-09-10T21:01:45.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.667+0000 7f80c1ffb640 1 -- 172.21.15.33:0/113449185 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8869 (secure 0 0 0) 0x7f80bc0ac5f0 con 0x7f80c4108020 2024-09-10T21:01:45.671 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:45.671 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":11,"btime":"2024-09-10T21:01:45:197007+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T21:01:45.672 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 11 2024-09-10T21:01:45.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80b00b52e0 msgr2=0x7f80b00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:45.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80b00b52e0 0x7f80b00b77a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f80b400bbe0 tx=0x7f80b400b390 comp rx=0 tx=0).stop 2024-09-10T21:01:45.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c4108020 msgr2=0x7f80c40fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:45.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c4108020 0x7f80c40fe680 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f80bc00eaa0 tx=0x7f80bc00ef60 comp rx=0 tx=0).stop 2024-09-10T21:01:45.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 shutdown_connections 2024-09-10T21:01:45.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80b00b52e0 0x7f80b00b77a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80c4108020 0x7f80c40fe680 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80c41076a0 0x7f80c40fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 --2- 172.21.15.33:0/113449185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80c4101c90 0x7f80c40fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:45.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.670+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 >> 172.21.15.33:0/113449185 conn(0x7f80c40fd700 msgr2=0x7f80c406acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:45.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.671+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 shutdown_connections 2024-09-10T21:01:45.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:45.671+0000 7f80ca386640 1 -- 172.21.15.33:0/113449185 wait complete. 2024-09-10T21:01:46.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:46 smithi033 ceph-mon[30860]: Deploying daemon mds.a on smithi033 2024-09-10T21:01:46.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:46 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.33:6838/295723844,v1:172.21.15.33:6839/295723844] up:boot 2024-09-10T21:01:46.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:46 smithi033 ceph-mon[30860]: fsmap 10 up:standby 2024-09-10T21:01:46.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:46 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T21:01:46.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:46 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/113449185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:46.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:46 smithi104 ceph-mon[35325]: Deploying daemon mds.a on smithi033 2024-09-10T21:01:46.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:46 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.33:6838/295723844,v1:172.21.15.33:6839/295723844] up:boot 2024-09-10T21:01:46.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:46 smithi104 ceph-mon[35325]: fsmap 10 up:standby 2024-09-10T21:01:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:46 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T21:01:46.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:46 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/113449185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:46.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:46 smithi174 ceph-mon[35635]: Deploying daemon mds.a on smithi033 2024-09-10T21:01:46.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:46 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.33:6838/295723844,v1:172.21.15.33:6839/295723844] up:boot 2024-09-10T21:01:46.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:46 smithi174 ceph-mon[35635]: fsmap 10 up:standby 2024-09-10T21:01:46.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:46 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T21:01:46.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:46 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/113449185' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:47 smithi104 ceph-mon[35325]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:47.612 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:47 smithi033 ceph-mon[30860]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:47.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:47 smithi174 ceph-mon[35635]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:47.927 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:47 smithi033 systemd[1]: Starting Ceph mds.a for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:48.273 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 podman[74376]: 2024-09-10 21:01:47.909769367 +0000 UTC m=+0.016745624 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:48.274 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 podman[74376]: 2024-09-10 21:01:48.015944306 +0000 UTC m=+0.122920563 container create be3eb55e3891b24a515e796b87b173a076dc1cdb4494108c48f6aa50079c5f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T21:01:48.274 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 podman[74376]: 2024-09-10 21:01:48.243772868 +0000 UTC m=+0.350749124 container init be3eb55e3891b24a515e796b87b173a076dc1cdb4494108c48f6aa50079c5f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-10T21:01:48.274 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 podman[74376]: 2024-09-10 21:01:48.246642184 +0000 UTC m=+0.353618440 container start be3eb55e3891b24a515e796b87b173a076dc1cdb4494108c48f6aa50079c5f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T21:01:48.677 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 ceph-mds[74395]: -- 172.21.15.33:0/3867812312 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af08f5b860 con 0x55af08f38800 2024-09-10T21:01:48.677 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a[74391]: starting mds.a at 2024-09-10T21:01:48.677 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 bash[74376]: be3eb55e3891b24a515e796b87b173a076dc1cdb4494108c48f6aa50079c5f71 2024-09-10T21:01:48.677 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:01:48 smithi033 systemd[1]: Started Ceph mds.a for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:48.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:49.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: Deploying daemon mds.b on smithi104 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.33:6840/1476321443,v1:172.21.15.33:6841/1476321443] up:boot 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: fsmap 11 up:standby 2024-09-10T21:01:49.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: Deploying daemon mds.b on smithi104 2024-09-10T21:01:49.568 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.33:6840/1476321443,v1:172.21.15.33:6841/1476321443] up:boot 2024-09-10T21:01:49.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: fsmap 11 up:standby 2024-09-10T21:01:49.569 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T21:01:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 -- 172.21.15.33:0/1459574464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 msgr2=0x7fd5a0108cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/1459574464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a0108cc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd594009900 tx=0x7fd59402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 -- 172.21.15.33:0/1459574464 shutdown_connections 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/1459574464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5a0109200 0x7fd5a010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/1459574464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a0108cc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/1459574464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a0100df0 0x7fd5a01011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 -- 172.21.15.33:0/1459574464 >> 172.21.15.33:0/1459574464 conn(0x7fd5a00fca20 msgr2=0x7fd5a00fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 -- 172.21.15.33:0/1459574464 shutdown_connections 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 -- 172.21.15.33:0/1459574464 wait complete. 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 Processor -- start 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.564+0000 7fd5a69a9640 1 -- start start 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd5a69a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a0100df0 0x7fd5a010ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd5a69a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a010f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd5a69a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5a0109200 0x7fd5a01a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd5a69a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0112d80 con 0x7fd5a0108860 2024-09-10T21:01:49.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd5a69a9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0112c00 con 0x7fd5a0100df0 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd5a69a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a0112f00 con 0x7fd5a0109200 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a010f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a010f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35224/0 (socket says 172.21.15.33:35224) 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 -- 172.21.15.33:0/3922817796 learned_addr learned my addr 172.21.15.33:0/3922817796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd5a4f46640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5a0109200 0x7fd5a01a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59ffff640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a0100df0 0x7fd5a010ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 -- 172.21.15.33:0/3922817796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5a0109200 msgr2=0x7fd5a01a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5a0109200 0x7fd5a01a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 -- 172.21.15.33:0/3922817796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a0100df0 msgr2=0x7fd5a010ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a0100df0 0x7fd5a010ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.565+0000 7fd59f7fe640 1 -- 172.21.15.33:0/3922817796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a01a5980 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.566+0000 7fd59f7fe640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a010f2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd594005940 tx=0x7fd594004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.566+0000 7fd59d7fa640 1 -- 172.21.15.33:0/3922817796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59403f030 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.566+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a01a5c10 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.566+0000 7fd59d7fa640 1 -- 172.21.15.33:0/3922817796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd594003aa0 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.566+0000 7fd59d7fa640 1 -- 172.21.15.33:0/3922817796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594035070 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.566+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a01ac330 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.567+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a01011d0 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.568+0000 7fd59d7fa640 1 -- 172.21.15.33:0/3922817796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fd59403bcc0 con 0x7fd5a0108860 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.569+0000 7fd59d7fa640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5880b52e0 0x7fd5880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.569+0000 7fd59ffff640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5880b52e0 0x7fd5880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.569+0000 7fd59d7fa640 1 -- 172.21.15.33:0/3922817796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fd594106390 con 0x7fd5a0108860 2024-09-10T21:01:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.570+0000 7fd59ffff640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5880b52e0 0x7fd5880b77a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd590009bd0 tx=0x7fd590005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.572+0000 7fd59d7fa640 1 -- 172.21.15.33:0/3922817796 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd5940cd000 con 0x7fd5a0108860 2024-09-10T21:01:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.796+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5a01ac590 con 0x7fd5a0108860 2024-09-10T21:01:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.796+0000 7fd59d7fa640 1 -- 172.21.15.33:0/3922817796 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9695 (secure 0 0 0) 0x7fd5940b2e20 con 0x7fd5a0108860 2024-09-10T21:01:49.800 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:49.800 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":12,"btime":"2024-09-10T21:01:49:218778+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-10T21:01:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 12 2024-09-10T21:01:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5880b52e0 msgr2=0x7fd5880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5880b52e0 0x7fd5880b77a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd590009bd0 tx=0x7fd590005f90 comp rx=0 tx=0).stop 2024-09-10T21:01:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 msgr2=0x7fd5a010f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a010f2a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd594005940 tx=0x7fd594004270 comp rx=0 tx=0).stop 2024-09-10T21:01:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 shutdown_connections 2024-09-10T21:01:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5880b52e0 0x7fd5880b77a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5a0109200 0x7fd5a01a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0108860 0x7fd5a010f2a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 --2- 172.21.15.33:0/3922817796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5a0100df0 0x7fd5a010ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 >> 172.21.15.33:0/3922817796 conn(0x7fd5a00fca20 msgr2=0x7fd5a00fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.798+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 shutdown_connections 2024-09-10T21:01:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:49.799+0000 7fd5a69a9640 1 -- 172.21.15.33:0/3922817796 wait complete. 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: Deploying daemon mds.b on smithi104 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.33:6840/1476321443,v1:172.21.15.33:6841/1476321443] up:boot 2024-09-10T21:01:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: fsmap 11 up:standby 2024-09-10T21:01:49.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T21:01:50.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:50 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3922817796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:50.563 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:50 smithi104 systemd[1]: Starting Ceph mds.b for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:01:50.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3922817796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:50.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:50 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3922817796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:50.948 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:50 smithi104 podman[65642]: 2024-09-10 21:01:50.592962357 +0000 UTC m=+0.016023327 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:01:50.948 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:50 smithi104 podman[65642]: 2024-09-10 21:01:50.692225577 +0000 UTC m=+0.115286536 container create da5a9bb67f8f19856f42fcb39ba90695a2d436a59fb773ce68f070e03d1283c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:01:50.948 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:50 smithi104 podman[65642]: 2024-09-10 21:01:50.94472036 +0000 UTC m=+0.367781319 container init da5a9bb67f8f19856f42fcb39ba90695a2d436a59fb773ce68f070e03d1283c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:01:51.312 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:50 smithi104 podman[65642]: 2024-09-10 21:01:50.947739963 +0000 UTC m=+0.370800920 container start da5a9bb67f8f19856f42fcb39ba90695a2d436a59fb773ce68f070e03d1283c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T21:01:51.312 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:50 smithi104 ceph-mds[65661]: -- 172.21.15.104:0/4223177555 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0fad85860 con 0x55f0fad63400 2024-09-10T21:01:51.312 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:50 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: starting mds.b at 2024-09-10T21:01:51.313 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:51 smithi104 bash[65642]: da5a9bb67f8f19856f42fcb39ba90695a2d436a59fb773ce68f070e03d1283c8 2024-09-10T21:01:51.313 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:01:51 smithi104 systemd[1]: Started Ceph mds.b for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:01:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:51 smithi033 ceph-mon[30860]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:51 smithi104 ceph-mon[35325]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:51.845 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:51 smithi174 ceph-mon[35635]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:51.845 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.845 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.845 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.845 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.845 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:51.845 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:52.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:01:52.742 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:52.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:52 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] up:boot 2024-09-10T21:01:52.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:52 smithi104 ceph-mon[35325]: fsmap 12 up:standby 2024-09-10T21:01:52.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T21:01:52.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:52 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] up:boot 2024-09-10T21:01:52.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:52 smithi033 ceph-mon[30860]: fsmap 12 up:standby 2024-09-10T21:01:52.865 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T21:01:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:52 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] up:boot 2024-09-10T21:01:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:52 smithi174 ceph-mon[35635]: fsmap 12 up:standby 2024-09-10T21:01:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 -- 172.21.15.33:0/207092452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc10d120 msgr2=0x7fccfc10f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 --2- 172.21.15.33:0/207092452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc10d120 0x7fccfc10f580 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009ca0 tx=0x7fccf402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 -- 172.21.15.33:0/207092452 shutdown_connections 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 --2- 172.21.15.33:0/207092452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc10d120 0x7fccfc10f580 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 --2- 172.21.15.33:0/207092452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccfc1045e0 0x7fccfc10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 --2- 172.21.15.33:0/207092452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccfc103cc0 0x7fccfc1040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 -- 172.21.15.33:0/207092452 >> 172.21.15.33:0/207092452 conn(0x7fccfc0fd740 msgr2=0x7fccfc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.252+0000 7fcd017d4640 1 -- 172.21.15.33:0/207092452 shutdown_connections 2024-09-10T21:01:53.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 -- 172.21.15.33:0/207092452 wait complete. 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 Processor -- start 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 -- start start 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc103cc0 0x7fccfc101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccfc1045e0 0x7fccfc101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccfc10d120 0x7fccfc1023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc112d90 con 0x7fccfc103cc0 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc112c10 con 0x7fccfc1045e0 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.253+0000 7fcd017d4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fccfc112f10 con 0x7fccfc10d120 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc103cc0 0x7fccfc101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc103cc0 0x7fccfc101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35252/0 (socket says 172.21.15.33:35252) 2024-09-10T21:01:53.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 -- 172.21.15.33:0/299773360 learned_addr learned my addr 172.21.15.33:0/299773360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfa7fc640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccfc1045e0 0x7fccfc101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfb7fe640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccfc10d120 0x7fccfc1023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 -- 172.21.15.33:0/299773360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccfc10d120 msgr2=0x7fccfc1023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccfc10d120 0x7fccfc1023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 -- 172.21.15.33:0/299773360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccfc1045e0 msgr2=0x7fccfc101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccfc1045e0 0x7fccfc101eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 -- 172.21.15.33:0/299773360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccfc1a75c0 con 0x7fccfc103cc0 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fccfaffd640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc103cc0 0x7fccfc101970 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c920 tx=0x7fccec00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fcce3fff640 1 -- 172.21.15.33:0/299773360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec007c70 con 0x7fccfc103cc0 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fcce3fff640 1 -- 172.21.15.33:0/299773360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccec007e10 con 0x7fccfc103cc0 2024-09-10T21:01:53.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccfc1a78b0 con 0x7fccfc103cc0 2024-09-10T21:01:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fcce3fff640 1 -- 172.21.15.33:0/299773360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccec018820 con 0x7fccfc103cc0 2024-09-10T21:01:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.254+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccfc1a7df0 con 0x7fccfc103cc0 2024-09-10T21:01:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.255+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccfc1040a0 con 0x7fccfc103cc0 2024-09-10T21:01:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.257+0000 7fcce3fff640 1 -- 172.21.15.33:0/299773360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fccec00f450 con 0x7fccfc103cc0 2024-09-10T21:01:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.258+0000 7fcce3fff640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcce40b52e0 0x7fcce40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.258+0000 7fccfa7fc640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcce40b52e0 0x7fcce40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:53.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.258+0000 7fcce3fff640 1 -- 172.21.15.33:0/299773360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fccec0e5b00 con 0x7fccfc103cc0 2024-09-10T21:01:53.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.258+0000 7fccfa7fc640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcce40b52e0 0x7fcce40b77a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fccfc103460 tx=0x7fccf0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:53.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.262+0000 7fcce3fff640 1 -- 172.21.15.33:0/299773360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fccec014030 con 0x7fccfc103cc0 2024-09-10T21:01:53.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.500+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccfc1a81f0 con 0x7fccfc103cc0 2024-09-10T21:01:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.501+0000 7fcce3fff640 1 -- 172.21.15.33:0/299773360 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7fccec0ace50 con 0x7fccfc103cc0 2024-09-10T21:01:53.505 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:53.505 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":13,"btime":"2024-09-10T21:01:51:415107+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14697,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14709,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24494,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24518,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24526,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24532,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24539,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24547,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24566,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-10T21:01:53.506 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 13 2024-09-10T21:01:53.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcce40b52e0 msgr2=0x7fcce40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:53.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcce40b52e0 0x7fcce40b77a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fccfc103460 tx=0x7fccf0005f90 comp rx=0 tx=0).stop 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc103cc0 msgr2=0x7fccfc101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc103cc0 0x7fccfc101970 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fccec00c920 tx=0x7fccec00cde0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 shutdown_connections 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcce40b52e0 0x7fcce40b77a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fccfc10d120 0x7fccfc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccfc1045e0 0x7fccfc101eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 --2- 172.21.15.33:0/299773360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccfc103cc0 0x7fccfc101970 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.503+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 >> 172.21.15.33:0/299773360 conn(0x7fccfc0fd740 msgr2=0x7fccfc0fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.504+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 shutdown_connections 2024-09-10T21:01:53.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:53.504+0000 7fcd017d4640 1 -- 172.21.15.33:0/299773360 wait complete. 2024-09-10T21:01:53.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:53.860 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:53.861 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:53 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/299773360' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:54.049 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:54.050 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:53 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/299773360' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:54.051 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:54.052 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:01:54.052 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:53 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/299773360' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:01:54.210 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-10T21:01:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:55.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 -- 172.21.15.33:0/138530053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278104550 msgr2=0x7fb27810cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:55.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 --2- 172.21.15.33:0/138530053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278104550 0x7fb27810cb30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb26c009ca0 tx=0x7fb26c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 -- 172.21.15.33:0/138530053 shutdown_connections 2024-09-10T21:01:55.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 --2- 172.21.15.33:0/138530053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb27810d100 0x7fb27810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 --2- 172.21.15.33:0/138530053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278104550 0x7fb27810cb30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 --2- 172.21.15.33:0/138530053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb278103b50 0x7fb278103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 -- 172.21.15.33:0/138530053 >> 172.21.15.33:0/138530053 conn(0x7fb2780fd700 msgr2=0x7fb2780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 -- 172.21.15.33:0/138530053 shutdown_connections 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 -- 172.21.15.33:0/138530053 wait complete. 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.598+0000 7fb27f96e640 1 Processor -- start 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27f96e640 1 -- start start 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27f96e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278103b50 0x7fb2781019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27f96e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb278104550 0x7fb278101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27f96e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb27810d100 0x7fb278102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27f96e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb278112d60 con 0x7fb278103b50 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27f96e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb278112be0 con 0x7fb27810d100 2024-09-10T21:01:55.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27f96e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb278112ee0 con 0x7fb278104550 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278103b50 0x7fb2781019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278103b50 0x7fb2781019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35264/0 (socket says 172.21.15.33:35264) 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 -- 172.21.15.33:0/3261883350 learned_addr learned my addr 172.21.15.33:0/3261883350 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27cf09640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb278104550 0x7fb278101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27df0b640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb27810d100 0x7fb278102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 -- 172.21.15.33:0/3261883350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb278104550 msgr2=0x7fb278101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb278104550 0x7fb278101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 -- 172.21.15.33:0/3261883350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb27810d100 msgr2=0x7fb278102450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb27810d100 0x7fb278102450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 -- 172.21.15.33:0/3261883350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2781a76b0 con 0x7fb278103b50 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27cf09640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb278104550 0x7fb278101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27df0b640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb27810d100 0x7fb278102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:01:55.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.599+0000 7fb27d70a640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278103b50 0x7fb2781019d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d770 tx=0x7fb26800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.600+0000 7fb2767fc640 1 -- 172.21.15.33:0/3261883350 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268017070 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.600+0000 7fb2767fc640 1 -- 172.21.15.33:0/3261883350 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2680054d0 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.600+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2781a79a0 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.600+0000 7fb2767fc640 1 -- 172.21.15.33:0/3261883350 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268004c30 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.600+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2781a7e90 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.601+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb278103f30 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.602+0000 7fb2767fc640 1 -- 172.21.15.33:0/3261883350 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fb268004dd0 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.603+0000 7fb2767fc640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb2600b52e0 0x7fb2600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.603+0000 7fb27cf09640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb2600b52e0 0x7fb2600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.603+0000 7fb2767fc640 1 -- 172.21.15.33:0/3261883350 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7fb2680e5710 con 0x7fb278103b50 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.603+0000 7fb27cf09640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb2600b52e0 0x7fb2600b77a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb2781034c0 tx=0x7fb26c009360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:55.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.606+0000 7fb2767fc640 1 -- 172.21.15.33:0/3261883350 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb2680ac9d0 con 0x7fb278103b50 2024-09-10T21:01:55.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.808+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb27802e5c0 con 0x7fb278103b50 2024-09-10T21:01:55.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:55 smithi033 ceph-mon[30860]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:55.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:55.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:55.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:55.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:55.813 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:55 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:55.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.808+0000 7fb2767fc640 1 -- 172.21.15.33:0/3261883350 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fb2680ac2d0 con 0x7fb278103b50 2024-09-10T21:01:55.813 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:01:55.813 INFO:teuthology.orchestra.run.smithi033.stdout:[ 2024-09-10T21:01:55.813 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:01:55.814 INFO:teuthology.orchestra.run.smithi033.stdout: "poolnum": 1, 2024-09-10T21:01:55.814 INFO:teuthology.orchestra.run.smithi033.stdout: "poolname": ".mgr" 2024-09-10T21:01:55.814 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:01:55.814 INFO:teuthology.orchestra.run.smithi033.stdout:] 2024-09-10T21:01:55.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb2600b52e0 msgr2=0x7fb2600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:55.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb2600b52e0 0x7fb2600b77a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb2781034c0 tx=0x7fb26c009360 comp rx=0 tx=0).stop 2024-09-10T21:01:55.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278103b50 msgr2=0x7fb2781019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:55.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278103b50 0x7fb2781019d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb26800d770 tx=0x7fb26800dc30 comp rx=0 tx=0).stop 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 shutdown_connections 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb2600b52e0 0x7fb2600b77a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb27810d100 0x7fb278102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb278104550 0x7fb278101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 --2- 172.21.15.33:0/3261883350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb278103b50 0x7fb2781019d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 >> 172.21.15.33:0/3261883350 conn(0x7fb2780fd700 msgr2=0x7fb2780fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.811+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 shutdown_connections 2024-09-10T21:01:55.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:55.812+0000 7fb27f96e640 1 -- 172.21.15.33:0/3261883350 wait complete. 2024-09-10T21:01:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:55 smithi104 ceph-mon[35325]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:55 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:55 smithi174 ceph-mon[35635]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:55 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:56.538 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:01:56.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:01:56.795 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:57.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:56 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3261883350' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T21:01:57.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:57.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:57.025 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:56 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:56 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3261883350' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T21:01:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:57.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:56 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:56 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3261883350' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-10T21:01:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:01:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:01:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:56 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:01:57.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 -- 172.21.15.33:0/1440601044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c109150 msgr2=0x7f925c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 --2- 172.21.15.33:0/1440601044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c109150 0x7f925c10f950 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9254009ca0 tx=0x7f925402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 -- 172.21.15.33:0/1440601044 shutdown_connections 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 --2- 172.21.15.33:0/1440601044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c109150 0x7f925c10f950 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 --2- 172.21.15.33:0/1440601044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f925c1087d0 0x7f925c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 --2- 172.21.15.33:0/1440601044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f925c102dc0 0x7f925c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 -- 172.21.15.33:0/1440601044 >> 172.21.15.33:0/1440601044 conn(0x7f925c0fe830 msgr2=0x7f925c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 -- 172.21.15.33:0/1440601044 shutdown_connections 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.242+0000 7f9263bf3640 1 -- 172.21.15.33:0/1440601044 wait complete. 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 Processor -- start 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 -- start start 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f925c102dc0 0x7f925c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:57.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c1087d0 0x7f925c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f925c109150 0x7f925c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f925c112e10 con 0x7f925c1087d0 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f925c112c90 con 0x7f925c109150 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9263bf3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f925c112f90 con 0x7f925c102dc0 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f926118e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c1087d0 0x7f925c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f926118e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c1087d0 0x7f925c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35276/0 (socket says 172.21.15.33:35276) 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f926118e640 1 -- 172.21.15.33:0/393530783 learned_addr learned my addr 172.21.15.33:0/393530783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f926198f640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f925c102dc0 0x7f925c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.243+0000 7f9262190640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f925c109150 0x7f925c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f926198f640 1 -- 172.21.15.33:0/393530783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f925c109150 msgr2=0x7f925c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f926198f640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f925c109150 0x7f925c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f926198f640 1 -- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c1087d0 msgr2=0x7f925c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f926198f640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c1087d0 0x7f925c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f926198f640 1 -- 172.21.15.33:0/393530783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925c19f350 con 0x7f925c102dc0 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f926118e640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c1087d0 0x7f925c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f926198f640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f925c102dc0 0x7f925c1992a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f925800e9f0 tx=0x7f925800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f9252ffd640 1 -- 172.21.15.33:0/393530783 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925800ccc0 con 0x7f925c102dc0 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f925c19f640 con 0x7f925c102dc0 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f9252ffd640 1 -- 172.21.15.33:0/393530783 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9258005020 con 0x7f925c102dc0 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f9252ffd640 1 -- 172.21.15.33:0/393530783 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258004a00 con 0x7f925c102dc0 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.244+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f925c1a5d00 con 0x7f925c102dc0 2024-09-10T21:01:57.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.245+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925c1031a0 con 0x7f925c102dc0 2024-09-10T21:01:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.247+0000 7f9252ffd640 1 -- 172.21.15.33:0/393530783 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f9258004ba0 con 0x7f925c102dc0 2024-09-10T21:01:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.247+0000 7f9252ffd640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92480b52e0 0x7f92480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.247+0000 7f926118e640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92480b52e0 0x7f92480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.247+0000 7f9252ffd640 1 -- 172.21.15.33:0/393530783 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7127+0+0 (secure 0 0 0) 0x7f92580e5800 con 0x7f925c102dc0 2024-09-10T21:01:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.248+0000 7f926118e640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92480b52e0 0x7f92480b77a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f925c102030 tx=0x7f924c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:57.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.250+0000 7f9252ffd640 1 -- 172.21.15.33:0/393530783 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f92580acb50 con 0x7f925c102dc0 2024-09-10T21:01:57.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.435+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f925c19a3d0 con 0x7f925c102dc0 2024-09-10T21:01:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.768+0000 7f9252ffd640 1 -- 172.21.15.33:0/393530783 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f92580ac450 con 0x7f925c102dc0 2024-09-10T21:01:57.772 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' created 2024-09-10T21:01:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92480b52e0 msgr2=0x7f92480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:57.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92480b52e0 0x7f92480b77a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f925c102030 tx=0x7f924c005f90 comp rx=0 tx=0).stop 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f925c102dc0 msgr2=0x7f925c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f925c102dc0 0x7f925c1992a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f925800e9f0 tx=0x7f925800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 shutdown_connections 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92480b52e0 0x7f92480b77a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f925c109150 0x7f925c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f925c1087d0 0x7f925c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 --2- 172.21.15.33:0/393530783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f925c102dc0 0x7f925c1992a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.772+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 >> 172.21.15.33:0/393530783 conn(0x7f925c0fe830 msgr2=0x7f925c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.773+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 shutdown_connections 2024-09-10T21:01:57.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:57.773+0000 7f9263bf3640 1 -- 172.21.15.33:0/393530783 wait complete. 2024-09-10T21:01:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:57 smithi104 ceph-mon[35325]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:57 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/393530783' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T21:01:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:57 smithi104 ceph-mon[35325]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T21:01:58.139 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:57 smithi033 ceph-mon[30860]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:58.139 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:57 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/393530783' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T21:01:58.139 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:57 smithi033 ceph-mon[30860]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T21:01:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:57 smithi174 ceph-mon[35635]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:01:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:57 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/393530783' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T21:01:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:57 smithi174 ceph-mon[35635]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-10T21:01:58.430 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:01:58.691 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:01:59.053 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:58 smithi033 ceph-mon[30860]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T21:01:59.053 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:58 smithi033 ceph-mon[30860]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T21:01:59.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:58 smithi104 ceph-mon[35325]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T21:01:59.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:58 smithi104 ceph-mon[35325]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T21:01:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:58 smithi174 ceph-mon[35635]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-10T21:01:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:58 smithi174 ceph-mon[35635]: osdmap e71: 12 total, 12 up, 12 in 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.139+0000 7f01f407f640 1 -- 172.21.15.33:0/3462184614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec10d120 msgr2=0x7f01ec10f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.139+0000 7f01f407f640 1 --2- 172.21.15.33:0/3462184614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec10d120 0x7f01ec10f590 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f01e4009900 tx=0x7f01e402a410 comp rx=0 tx=0).stop 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 -- 172.21.15.33:0/3462184614 shutdown_connections 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 --2- 172.21.15.33:0/3462184614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec10d120 0x7f01ec10f590 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 --2- 172.21.15.33:0/3462184614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01ec104570 0x7f01ec10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 --2- 172.21.15.33:0/3462184614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec103b70 0x7f01ec103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 -- 172.21.15.33:0/3462184614 >> 172.21.15.33:0/3462184614 conn(0x7f01ec0fd700 msgr2=0x7f01ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:59.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 -- 172.21.15.33:0/3462184614 shutdown_connections 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 -- 172.21.15.33:0/3462184614 wait complete. 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 Processor -- start 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.140+0000 7f01f407f640 1 -- start start 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f407f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01ec103b70 0x7f01ec199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f407f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec104570 0x7f01ec199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f407f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec10d120 0x7f01ec19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f407f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec112da0 con 0x7f01ec104570 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f407f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec112c20 con 0x7f01ec103b70 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f407f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01ec112f20 con 0x7f01ec10d120 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec104570 0x7f01ec199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec104570 0x7f01ec199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35296/0 (socket says 172.21.15.33:35296) 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 -- 172.21.15.33:0/4045913057 learned_addr learned my addr 172.21.15.33:0/4045913057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:01:59.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f1e1b640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01ec103b70 0x7f01ec199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 -- 172.21.15.33:0/4045913057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec10d120 msgr2=0x7f01ec19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f261c640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec10d120 0x7f01ec19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec10d120 0x7f01ec19eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 -- 172.21.15.33:0/4045913057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01ec103b70 msgr2=0x7f01ec199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01ec103b70 0x7f01ec199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 -- 172.21.15.33:0/4045913057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ec19f2c0 con 0x7f01ec104570 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f1e1b640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01ec103b70 0x7f01ec199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.141+0000 7f01f161a640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec104570 0x7f01ec199750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00d950 tx=0x7f01dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.142+0000 7f01e2ffd640 1 -- 172.21.15.33:0/4045913057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc017070 con 0x7f01ec104570 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.142+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ec19f5b0 con 0x7f01ec104570 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.142+0000 7f01e2ffd640 1 -- 172.21.15.33:0/4045913057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01dc009c30 con 0x7f01ec104570 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.142+0000 7f01e2ffd640 1 -- 172.21.15.33:0/4045913057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00f620 con 0x7f01ec104570 2024-09-10T21:01:59.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.142+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ec1a5c70 con 0x7f01ec104570 2024-09-10T21:01:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.143+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01ec103f50 con 0x7f01ec104570 2024-09-10T21:01:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.144+0000 7f01e2ffd640 1 -- 172.21.15.33:0/4045913057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f01dc010460 con 0x7f01ec104570 2024-09-10T21:01:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.145+0000 7f01e2ffd640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01d80b52e0 0x7f01d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:01:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.145+0000 7f01f1e1b640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01d80b52e0 0x7f01d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:01:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.145+0000 7f01e2ffd640 1 -- 172.21.15.33:0/4045913057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7497+0+0 (secure 0 0 0) 0x7f01dc0e5590 con 0x7f01ec104570 2024-09-10T21:01:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.145+0000 7f01f1e1b640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01d80b52e0 0x7f01d80b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005fd0 tx=0x7f01e8009520 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:01:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.148+0000 7f01e2ffd640 1 -- 172.21.15.33:0/4045913057 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f01dc0ac780 con 0x7f01ec104570 2024-09-10T21:01:59.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.335+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f01ec1a5ff0 con 0x7f01ec104570 2024-09-10T21:01:59.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.777+0000 7f01e2ffd640 1 -- 172.21.15.33:0/4045913057 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7f01dc009de0 con 0x7f01ec104570 2024-09-10T21:01:59.781 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' created 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01d80b52e0 msgr2=0x7f01d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01d80b52e0 0x7f01d80b77a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f01e8005fd0 tx=0x7f01e8009520 comp rx=0 tx=0).stop 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec104570 msgr2=0x7f01ec199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec104570 0x7f01ec199750 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00d950 tx=0x7f01dc00de10 comp rx=0 tx=0).stop 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 shutdown_connections 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01d80b52e0 0x7f01d80b77a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ec10d120 0x7f01ec19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01ec104570 0x7f01ec199750 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 --2- 172.21.15.33:0/4045913057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01ec103b70 0x7f01ec199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 >> 172.21.15.33:0/4045913057 conn(0x7f01ec0fd700 msgr2=0x7f01ec0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 shutdown_connections 2024-09-10T21:01:59.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:01:59.780+0000 7f01f407f640 1 -- 172.21.15.33:0/4045913057 wait complete. 2024-09-10T21:02:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:59 smithi104 ceph-mon[35325]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:59 smithi104 ceph-mon[35325]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T21:02:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:59 smithi104 ceph-mon[35325]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T21:02:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:01:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4045913057' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T21:02:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:59 smithi174 ceph-mon[35635]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:59 smithi174 ceph-mon[35635]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T21:02:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:59 smithi174 ceph-mon[35635]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T21:02:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:01:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4045913057' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T21:02:00.166 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:59 smithi033 ceph-mon[30860]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:00.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:59 smithi033 ceph-mon[30860]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T21:02:00.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:59 smithi033 ceph-mon[30860]: osdmap e72: 12 total, 12 up, 12 in 2024-09-10T21:02:00.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:01:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4045913057' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-10T21:02:00.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:02:00.761 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:01.048 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:00 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4045913057' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T21:02:01.048 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:00 smithi033 ceph-mon[30860]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T21:02:01.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:00 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4045913057' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T21:02:01.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:00 smithi104 ceph-mon[35325]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T21:02:01.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:00 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4045913057' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-10T21:02:01.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:00 smithi174 ceph-mon[35635]: osdmap e73: 12 total, 12 up, 12 in 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 -- 172.21.15.33:0/4119623788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 msgr2=0x7f3ac410cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/4119623788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac410cd10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3aac009900 tx=0x7f3aac02a410 comp rx=0 tx=0).stop 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 -- 172.21.15.33:0/4119623788 shutdown_connections 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/4119623788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ac410d250 0x7f3ac410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/4119623788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac410cd10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/4119623788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ac4101810 0x7f3ac4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 -- 172.21.15.33:0/4119623788 >> 172.21.15.33:0/4119623788 conn(0x7f3ac4076f50 msgr2=0x7f3ac4077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:01.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 -- 172.21.15.33:0/4119623788 shutdown_connections 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 -- 172.21.15.33:0/4119623788 wait complete. 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 Processor -- start 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.212+0000 7f3aca9a1640 1 -- start start 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3aca9a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ac4101810 0x7f3ac4199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3aca9a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3aca9a1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ac410d250 0x7f3ac419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3aca9a1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4112e30 con 0x7f3ac410a920 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3aca9a1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4112cb0 con 0x7f3ac410d250 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3aca9a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ac4112fb0 con 0x7f3ac4101810 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac4199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35766/0 (socket says 172.21.15.33:35766) 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 -- 172.21.15.33:0/1211992758 learned_addr learned my addr 172.21.15.33:0/1211992758 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:01.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3ac8f3e640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ac410d250 0x7f3ac419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3ac3fff640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ac4101810 0x7f3ac4199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 -- 172.21.15.33:0/1211992758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ac4101810 msgr2=0x7f3ac4199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ac4101810 0x7f3ac4199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 -- 172.21.15.33:0/1211992758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ac410d250 msgr2=0x7f3ac419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ac410d250 0x7f3ac419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 -- 172.21.15.33:0/1211992758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ac419f4b0 con 0x7f3ac410a920 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.213+0000 7f3abbfff640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac4199940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.214+0000 7f3ac1ffb640 1 -- 172.21.15.33:0/1211992758 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac03f030 con 0x7f3ac410a920 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.214+0000 7f3ac1ffb640 1 -- 172.21.15.33:0/1211992758 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aac003800 con 0x7f3ac410a920 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.214+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ac419f740 con 0x7f3ac410a920 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.214+0000 7f3ac1ffb640 1 -- 172.21.15.33:0/1211992758 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aac035070 con 0x7f3ac410a920 2024-09-10T21:02:01.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.214+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ac41a5e60 con 0x7f3ac410a920 2024-09-10T21:02:01.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.215+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ac4101c70 con 0x7f3ac410a920 2024-09-10T21:02:01.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.216+0000 7f3ac1ffb640 1 -- 172.21.15.33:0/1211992758 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f3aac031b10 con 0x7f3ac410a920 2024-09-10T21:02:01.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.217+0000 7f3ac1ffb640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3ab00b52e0 0x7f3ab00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:01.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.217+0000 7f3ac3fff640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3ab00b52e0 0x7f3ab00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:01.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.217+0000 7f3ac1ffb640 1 -- 172.21.15.33:0/1211992758 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 7879+0+0 (secure 0 0 0) 0x7f3aac1065f0 con 0x7f3ac410a920 2024-09-10T21:02:01.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.218+0000 7f3ac3fff640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3ab00b52e0 0x7f3ab00b77a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009890 tx=0x7f3ab4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:01.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.220+0000 7f3ac1ffb640 1 -- 172.21.15.33:0/1211992758 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3aac0cd660 con 0x7f3ac410a920 2024-09-10T21:02:01.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.444+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3ac41a60c0 con 0x7f3ac410a920 2024-09-10T21:02:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.784+0000 7f3ac1ffb640 1 -- 172.21.15.33:0/1211992758 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f3aac0ccf60 con 0x7f3ac410a920 2024-09-10T21:02:01.788 INFO:teuthology.orchestra.run.smithi033.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3ab00b52e0 msgr2=0x7f3ab00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3ab00b52e0 0x7f3ab00b77a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4009890 tx=0x7f3ab4008040 comp rx=0 tx=0).stop 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 msgr2=0x7f3ac4199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac4199940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006010 tx=0x7f3aac0042f0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 shutdown_connections 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3ab00b52e0 0x7f3ab00b77a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3ac410d250 0x7f3ac419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ac410a920 0x7f3ac4199940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 --2- 172.21.15.33:0/1211992758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ac4101810 0x7f3ac4199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 >> 172.21.15.33:0/1211992758 conn(0x7f3ac4076f50 msgr2=0x7f3ac41005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 shutdown_connections 2024-09-10T21:02:01.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:01.787+0000 7f3aca9a1640 1 -- 172.21.15.33:0/1211992758 wait complete. 2024-09-10T21:02:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:01 smithi104 ceph-mon[35325]: pgmap v283: 97 pgs: 1 creating+activating, 3 creating+peering, 86 unknown, 7 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:02.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:01 smithi104 ceph-mon[35325]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T21:02:02.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:01 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1211992758' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T21:02:02.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:01 smithi104 ceph-mon[35325]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T21:02:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:01 smithi174 ceph-mon[35635]: pgmap v283: 97 pgs: 1 creating+activating, 3 creating+peering, 86 unknown, 7 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:01 smithi174 ceph-mon[35635]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T21:02:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:01 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1211992758' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T21:02:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:01 smithi174 ceph-mon[35635]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T21:02:02.167 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:01 smithi033 ceph-mon[30860]: pgmap v283: 97 pgs: 1 creating+activating, 3 creating+peering, 86 unknown, 7 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:02.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:01 smithi033 ceph-mon[30860]: osdmap e74: 12 total, 12 up, 12 in 2024-09-10T21:02:02.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:01 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1211992758' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-10T21:02:02.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:01 smithi033 ceph-mon[30860]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-10T21:02:02.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:02:02.734 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:03.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:02 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1211992758' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T21:02:03.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:02 smithi104 ceph-mon[35325]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T21:02:03.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:02 smithi104 ceph-mon[35325]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T21:02:03.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:02 smithi104 ceph-mon[35325]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T21:02:03.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:02 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1211992758' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T21:02:03.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:02 smithi033 ceph-mon[30860]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T21:02:03.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:02 smithi033 ceph-mon[30860]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T21:02:03.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:02 smithi033 ceph-mon[30860]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T21:02:03.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:02 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1211992758' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-10T21:02:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:02 smithi174 ceph-mon[35635]: osdmap e75: 12 total, 12 up, 12 in 2024-09-10T21:02:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:02 smithi174 ceph-mon[35635]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-10T21:02:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:02 smithi174 ceph-mon[35635]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 -- 172.21.15.33:0/3542376683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 msgr2=0x7f455410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 --2- 172.21.15.33:0/3542376683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455410f5a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f454c009ca0 tx=0x7f454c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 -- 172.21.15.33:0/3542376683 shutdown_connections 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 --2- 172.21.15.33:0/3542376683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455410f5a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 --2- 172.21.15.33:0/3542376683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4554104600 0x7f455410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 --2- 172.21.15.33:0/3542376683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4554103ce0 0x7f45541040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 -- 172.21.15.33:0/3542376683 >> 172.21.15.33:0/3542376683 conn(0x7f45540fd700 msgr2=0x7f45540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.217+0000 7f455b2c6640 1 -- 172.21.15.33:0/3542376683 shutdown_connections 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 -- 172.21.15.33:0/3542376683 wait complete. 2024-09-10T21:02:03.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 Processor -- start 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 -- start start 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4554103ce0 0x7f45541991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4554104600 0x7f4554199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455419eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112db0 con 0x7f455410d140 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112c30 con 0x7f4554103ce0 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f455b2c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112f30 con 0x7f4554104600 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f4559863640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455419eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f4559863640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455419eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35794/0 (socket says 172.21.15.33:35794) 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.218+0000 7f4559863640 1 -- 172.21.15.33:0/1251402156 learned_addr learned my addr 172.21.15.33:0/1251402156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4558861640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4554104600 0x7f4554199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:03.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559062640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4554103ce0 0x7f45541991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559863640 1 -- 172.21.15.33:0/1251402156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4554104600 msgr2=0x7f4554199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559863640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4554104600 0x7f4554199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559863640 1 -- 172.21.15.33:0/1251402156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4554103ce0 msgr2=0x7f45541991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559863640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4554103ce0 0x7f45541991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559863640 1 -- 172.21.15.33:0/1251402156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419f270 con 0x7f455410d140 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559062640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4554103ce0 0x7f45541991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f4559863640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455419eaa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f454c005cc0 tx=0x7f454c02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f454a7fc640 1 -- 172.21.15.33:0/1251402156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c003bf0 con 0x7f455410d140 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f454a7fc640 1 -- 172.21.15.33:0/1251402156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c003d90 con 0x7f455410d140 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419f500 con 0x7f455410d140 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f454a7fc640 1 -- 172.21.15.33:0/1251402156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c02fbc0 con 0x7f455410d140 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.219+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a5c20 con 0x7f455410d140 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.220+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45541040c0 con 0x7f455410d140 2024-09-10T21:02:03.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.222+0000 7f454a7fc640 1 -- 172.21.15.33:0/1251402156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f454c004610 con 0x7f455410d140 2024-09-10T21:02:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.222+0000 7f454a7fc640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45400b5360 0x7f45400b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.222+0000 7f4559062640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45400b5360 0x7f45400b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.222+0000 7f454a7fc640 1 -- 172.21.15.33:0/1251402156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f454c107070 con 0x7f455410d140 2024-09-10T21:02:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.223+0000 7f4559062640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45400b5360 0x7f45400b7820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f455000baa0 tx=0x7f4550005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:03.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.226+0000 7f454a7fc640 1 -- 172.21.15.33:0/1251402156 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f454c0ce080 con 0x7f455410d140 2024-09-10T21:02:03.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.413+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f455419a2f0 con 0x7f455410d140 2024-09-10T21:02:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.414+0000 7f454a7fc640 1 -- 172.21.15.33:0/1251402156 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22281 (secure 0 0 0) 0x7f455419a2f0 con 0x7f455410d140 2024-09-10T21:02:03.418 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:03.418 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:01.445442+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.416+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45400b5360 msgr2=0x7f45400b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.416+0000 7f455b2c6640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45400b5360 0x7f45400b7820 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f455000baa0 tx=0x7f4550005d90 comp rx=0 tx=0).stop 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.416+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 msgr2=0x7f455419eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.416+0000 7f455b2c6640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455419eaa0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f454c005cc0 tx=0x7f454c02f380 comp rx=0 tx=0).stop 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 shutdown_connections 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45400b5360 0x7f45400b7820 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f455410d140 0x7f455419eaa0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4554104600 0x7f4554199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 --2- 172.21.15.33:0/1251402156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4554103ce0 0x7f45541991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 >> 172.21.15.33:0/1251402156 conn(0x7f45540fd700 msgr2=0x7f455410f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 shutdown_connections 2024-09-10T21:02:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:03.417+0000 7f455b2c6640 1 -- 172.21.15.33:0/1251402156 wait complete. 2024-09-10T21:02:04.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:03 smithi033 ceph-mon[30860]: pgmap v286: 97 pgs: 1 creating+activating, 3 creating+peering, 65 unknown, 28 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:04.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:03 smithi033 ceph-mon[30860]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T21:02:04.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:03 smithi033 ceph-mon[30860]: Cluster is now healthy 2024-09-10T21:02:04.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:03 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] up:active 2024-09-10T21:02:04.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:03 smithi033 ceph-mon[30860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:04.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:03 smithi033 ceph-mon[30860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:04.029 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:03 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1251402156' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:04.060 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:02:04.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:03 smithi104 ceph-mon[35325]: pgmap v286: 97 pgs: 1 creating+activating, 3 creating+peering, 65 unknown, 28 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:04.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:03 smithi104 ceph-mon[35325]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T21:02:04.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:03 smithi104 ceph-mon[35325]: Cluster is now healthy 2024-09-10T21:02:04.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:03 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] up:active 2024-09-10T21:02:04.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:03 smithi104 ceph-mon[35325]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:04.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:03 smithi104 ceph-mon[35325]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:04.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:03 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1251402156' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:03 smithi174 ceph-mon[35635]: pgmap v286: 97 pgs: 1 creating+activating, 3 creating+peering, 65 unknown, 28 active+clean; 577 KiB data, 342 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:02:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:03 smithi174 ceph-mon[35635]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-10T21:02:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:03 smithi174 ceph-mon[35635]: Cluster is now healthy 2024-09-10T21:02:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:03 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] up:active 2024-09-10T21:02:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:03 smithi174 ceph-mon[35635]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:03 smithi174 ceph-mon[35635]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:03 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1251402156' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:04.320 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:04.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 -- 172.21.15.33:0/3775917873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 msgr2=0x7fcb50107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 --2- 172.21.15.33:0/3775917873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb50107b20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009900 tx=0x7fcb3802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 -- 172.21.15.33:0/3775917873 shutdown_connections 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 --2- 172.21.15.33:0/3775917873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb50108060 0x7fcb5010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 --2- 172.21.15.33:0/3775917873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb50107b20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 --2- 172.21.15.33:0/3775917873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb50101cd0 0x7fcb501020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 -- 172.21.15.33:0/3775917873 >> 172.21.15.33:0/3775917873 conn(0x7fcb500fd700 msgr2=0x7fcb500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 -- 172.21.15.33:0/3775917873 shutdown_connections 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.928+0000 7fcb559aa640 1 -- 172.21.15.33:0/3775917873 wait complete. 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 Processor -- start 2024-09-10T21:02:04.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 -- start start 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb50101cd0 0x7fcb500fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb500fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb50108060 0x7fcb500fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111d00 con 0x7fcb501076e0 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111b80 con 0x7fcb50101cd0 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb559aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb50111e80 con 0x7fcb50108060 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb467fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb500fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb467fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb500fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35820/0 (socket says 172.21.15.33:35820) 2024-09-10T21:02:04.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.929+0000 7fcb467fc640 1 -- 172.21.15.33:0/1548147382 learned_addr learned my addr 172.21.15.33:0/1548147382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb4effd640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb50101cd0 0x7fcb500fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb4f7fe640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb50108060 0x7fcb500fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb467fc640 1 -- 172.21.15.33:0/1548147382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb50108060 msgr2=0x7fcb500fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb467fc640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb50108060 0x7fcb500fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb467fc640 1 -- 172.21.15.33:0/1548147382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb50101cd0 msgr2=0x7fcb500fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb467fc640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb50101cd0 0x7fcb500fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb467fc640 1 -- 172.21.15.33:0/1548147382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb501a76c0 con 0x7fcb501076e0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb467fc640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb500fe140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb38005ab0 tx=0x7fcb38003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb549a8640 1 -- 172.21.15.33:0/1548147382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3803f070 con 0x7fcb501076e0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb549a8640 1 -- 172.21.15.33:0/1548147382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb38031d60 con 0x7fcb501076e0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb549a8640 1 -- 172.21.15.33:0/1548147382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38035030 con 0x7fcb501076e0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb501a7950 con 0x7fcb501076e0 2024-09-10T21:02:04.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.930+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb501a7de0 con 0x7fcb501076e0 2024-09-10T21:02:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.931+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb501020b0 con 0x7fcb501076e0 2024-09-10T21:02:04.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.933+0000 7fcb549a8640 1 -- 172.21.15.33:0/1548147382 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fcb3803b580 con 0x7fcb501076e0 2024-09-10T21:02:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.934+0000 7fcb549a8640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb3c0b5360 0x7fcb3c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.934+0000 7fcb4effd640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb3c0b5360 0x7fcb3c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.934+0000 7fcb549a8640 1 -- 172.21.15.33:0/1548147382 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fcb38106c30 con 0x7fcb501076e0 2024-09-10T21:02:04.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.935+0000 7fcb4effd640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb3c0b5360 0x7fcb3c0b7820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb400058c0 tx=0x7fcb4000a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:04.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:04.937+0000 7fcb549a8640 1 -- 172.21.15.33:0/1548147382 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcb380cdc40 con 0x7fcb501076e0 2024-09-10T21:02:05.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.124+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb501a8240 con 0x7fcb501076e0 2024-09-10T21:02:05.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.125+0000 7fcb549a8640 1 -- 172.21.15.33:0/1548147382 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22281 (secure 0 0 0) 0x7fcb380cd540 con 0x7fcb501076e0 2024-09-10T21:02:05.129 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:05.129 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:01.445442+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:02:05.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb3c0b5360 msgr2=0x7fcb3c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:05.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb3c0b5360 0x7fcb3c0b7820 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb400058c0 tx=0x7fcb4000a4c0 comp rx=0 tx=0).stop 2024-09-10T21:02:05.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 msgr2=0x7fcb500fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:05.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb500fe140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fcb38005ab0 tx=0x7fcb38003cd0 comp rx=0 tx=0).stop 2024-09-10T21:02:05.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 shutdown_connections 2024-09-10T21:02:05.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb3c0b5360 0x7fcb3c0b7820 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:05.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb50108060 0x7fcb500fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:05.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb501076e0 0x7fcb500fe140 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:05.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 --2- 172.21.15.33:0/1548147382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb50101cd0 0x7fcb500fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:05.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 >> 172.21.15.33:0/1548147382 conn(0x7fcb500fd700 msgr2=0x7fcb5006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:05.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.127+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 shutdown_connections 2024-09-10T21:02:05.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:05.128+0000 7fcb559aa640 1 -- 172.21.15.33:0/1548147382 wait complete. 2024-09-10T21:02:05.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:02:05.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:05 smithi033 ceph-mon[30860]: pgmap v287: 97 pgs: 10 unknown, 87 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-09-10T21:02:05.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:05 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1548147382' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:06.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:05 smithi104 ceph-mon[35325]: pgmap v287: 97 pgs: 10 unknown, 87 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-09-10T21:02:06.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:05 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1548147382' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:06.070 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:05 smithi174 ceph-mon[35635]: pgmap v287: 97 pgs: 10 unknown, 87 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 3 op/s 2024-09-10T21:02:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:05 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1548147382' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 -- 172.21.15.33:0/2719535815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8103ce0 msgr2=0x7f81e81040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2719535815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8103ce0 0x7f81e81040c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009900 tx=0x7f81d802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 -- 172.21.15.33:0/2719535815 shutdown_connections 2024-09-10T21:02:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2719535815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e810d140 0x7f81e810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2719535815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81e8104600 0x7f81e810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2719535815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8103ce0 0x7f81e81040c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 -- 172.21.15.33:0/2719535815 >> 172.21.15.33:0/2719535815 conn(0x7f81e80fd700 msgr2=0x7f81e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 -- 172.21.15.33:0/2719535815 shutdown_connections 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.540+0000 7f81ecf40640 1 -- 172.21.15.33:0/2719535815 wait complete. 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 Processor -- start 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 -- start start 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81e8103ce0 0x7f81e81992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8104600 0x7f81e8199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e810d140 0x7f81e819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8112db0 con 0x7f81e8104600 2024-09-10T21:02:06.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8112c30 con 0x7f81e810d140 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81ecf40640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81e8112f30 con 0x7f81e8103ce0 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81e5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8104600 0x7f81e8199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81e5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8104600 0x7f81e8199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35842/0 (socket says 172.21.15.33:35842) 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81e5d9b640 1 -- 172.21.15.33:0/2156929760 learned_addr learned my addr 172.21.15.33:0/2156929760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81e6d9d640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e810d140 0x7f81e819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.541+0000 7f81e659c640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81e8103ce0 0x7f81e81992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81e5d9b640 1 -- 172.21.15.33:0/2156929760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81e8103ce0 msgr2=0x7f81e81992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81e5d9b640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81e8103ce0 0x7f81e81992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81e5d9b640 1 -- 172.21.15.33:0/2156929760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e810d140 msgr2=0x7f81e819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81e5d9b640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e810d140 0x7f81e819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81e5d9b640 1 -- 172.21.15.33:0/2156929760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e819f370 con 0x7f81e8104600 2024-09-10T21:02:06.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81e659c640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81e8103ce0 0x7f81e81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81e5d9b640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8104600 0x7f81e8199800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9b0 tx=0x7f81dc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81d77fe640 1 -- 172.21.15.33:0/2156929760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc016070 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81d77fe640 1 -- 172.21.15.33:0/2156929760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81dc00cdc0 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81d77fe640 1 -- 172.21.15.33:0/2156929760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc011470 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e819f660 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.542+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e81a5d20 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.543+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e81040c0 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.544+0000 7f81d77fe640 1 -- 172.21.15.33:0/2156929760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f81dc00f040 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.545+0000 7f81d77fe640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d00b52e0 0x7f81d00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.545+0000 7f81e659c640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d00b52e0 0x7f81d00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.545+0000 7f81d77fe640 1 -- 172.21.15.33:0/2156929760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f81dc0e5eb0 con 0x7f81e8104600 2024-09-10T21:02:06.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.546+0000 7f81e659c640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d00b52e0 0x7f81d00b77a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f81d8031440 tx=0x7f81d8005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:06.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.548+0000 7f81d77fe640 1 -- 172.21.15.33:0/2156929760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f81dc0acec0 con 0x7f81e8104600 2024-09-10T21:02:06.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.770+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f81e802e5c0 con 0x7f81e8104600 2024-09-10T21:02:06.810 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:02:06 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: 2024-09-10T21:02:06.425+0000 7f8073f1c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:02:06.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.814+0000 7f81d77fe640 1 -- 172.21.15.33:0/2156929760 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f81dc0ac7c0 con 0x7f81e8104600 2024-09-10T21:02:06.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d00b52e0 msgr2=0x7f81d00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:06.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d00b52e0 0x7f81d00b77a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f81d8031440 tx=0x7f81d8005a20 comp rx=0 tx=0).stop 2024-09-10T21:02:06.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8104600 msgr2=0x7f81e8199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:06.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8104600 0x7f81e8199800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00e9b0 tx=0x7f81dc00ee70 comp rx=0 tx=0).stop 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 shutdown_connections 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d00b52e0 0x7f81d00b77a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81e810d140 0x7f81e819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81e8104600 0x7f81e8199800 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 --2- 172.21.15.33:0/2156929760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81e8103ce0 0x7f81e81992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.816+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 >> 172.21.15.33:0/2156929760 conn(0x7f81e80fd700 msgr2=0x7f81e80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.817+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 shutdown_connections 2024-09-10T21:02:06.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:06.817+0000 7f81ecf40640 1 -- 172.21.15.33:0/2156929760 wait complete. 2024-09-10T21:02:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:06 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2156929760' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T21:02:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:06 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2156929760' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T21:02:07.163 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:06 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2156929760' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-10T21:02:07.503 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-09-10T21:02:07.503 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-10T21:02:07.806 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:08.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:07 smithi033 ceph-mon[30860]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-09-10T21:02:08.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:07 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2156929760' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T21:02:08.127 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:07 smithi033 ceph-mon[30860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:07 smithi174 ceph-mon[35635]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-09-10T21:02:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:07 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2156929760' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T21:02:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:07 smithi174 ceph-mon[35635]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.209+0000 7ff91f1af640 1 -- 172.21.15.33:0/3000053884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff918101c90 msgr2=0x7ff918102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.209+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3000053884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff918101c90 0x7ff918102070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009900 tx=0x7ff90c02a410 comp rx=0 tx=0).stop 2024-09-10T21:02:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- 172.21.15.33:0/3000053884 shutdown_connections 2024-09-10T21:02:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3000053884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff918108020 0x7ff91810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3000053884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9181076a0 0x7ff918107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3000053884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff918101c90 0x7ff918102070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- 172.21.15.33:0/3000053884 >> 172.21.15.33:0/3000053884 conn(0x7ff9180fd700 msgr2=0x7ff9180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- 172.21.15.33:0/3000053884 shutdown_connections 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- 172.21.15.33:0/3000053884 wait complete. 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 Processor -- start 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- start start 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918101c90 0x7ff918103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9181076a0 0x7ff918103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff918108020 0x7ff918103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111ce0 con 0x7ff9181076a0 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111b60 con 0x7ff918101c90 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.210+0000 7ff91f1af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff918111e60 con 0x7ff918108020 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9181076a0 0x7ff918103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9181076a0 0x7ff918103a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35858/0 (socket says 172.21.15.33:35858) 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 -- 172.21.15.33:0/3387522873 learned_addr learned my addr 172.21.15.33:0/3387522873 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff91d74c640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff918108020 0x7ff918103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff91cf4b640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918101c90 0x7ff918103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 -- 172.21.15.33:0/3387522873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff918108020 msgr2=0x7ff918103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff918108020 0x7ff918103fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 -- 172.21.15.33:0/3387522873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918101c90 msgr2=0x7ff918103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918101c90 0x7ff918103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 -- 172.21.15.33:0/3387522873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9181a7720 con 0x7ff9181076a0 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff91d74c640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff918108020 0x7ff918103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff91cf4b640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918101c90 0x7ff918103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff917fff640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9181076a0 0x7ff918103a90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff90400d950 tx=0x7ff90400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff915ffb640 1 -- 172.21.15.33:0/3387522873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904017030 con 0x7ff9181076a0 2024-09-10T21:02:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff915ffb640 1 -- 172.21.15.33:0/3387522873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff904004020 con 0x7ff9181076a0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff915ffb640 1 -- 172.21.15.33:0/3387522873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9040047a0 con 0x7ff9181076a0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.211+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9181a7a10 con 0x7ff9181076a0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.212+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9181a7f00 con 0x7ff9181076a0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.213+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff918102070 con 0x7ff9181076a0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.215+0000 7ff915ffb640 1 -- 172.21.15.33:0/3387522873 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7ff904009a30 con 0x7ff9181076a0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.216+0000 7ff915ffb640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9080b52e0 0x7ff9080b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.216+0000 7ff91cf4b640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9080b52e0 0x7ff9080b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.216+0000 7ff915ffb640 1 -- 172.21.15.33:0/3387522873 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7ff9040e5600 con 0x7ff9181076a0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.216+0000 7ff91cf4b640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9080b52e0 0x7ff9080b77a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c00 tx=0x7ff90c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.220+0000 7ff915ffb640 1 -- 172.21.15.33:0/3387522873 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff9040ac610 con 0x7ff9181076a0 2024-09-10T21:02:08.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:07 smithi104 ceph-mon[35325]: pgmap v288: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-09-10T21:02:08.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:07 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2156929760' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-10T21:02:08.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:07 smithi104 ceph-mon[35325]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:08.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.447+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7ff9181a82f0 con 0x7ff9181076a0 2024-09-10T21:02:08.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.825+0000 7ff915ffb640 1 -- 172.21.15.33:0/3387522873 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff9040abf10 con 0x7ff9181076a0 2024-09-10T21:02:08.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9080b52e0 msgr2=0x7ff9080b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:08.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9080b52e0 0x7ff9080b77a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000c00 tx=0x7ff90c031400 comp rx=0 tx=0).stop 2024-09-10T21:02:08.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9181076a0 msgr2=0x7ff918103a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:08.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9181076a0 0x7ff918103a90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff90400d950 tx=0x7ff90400de10 comp rx=0 tx=0).stop 2024-09-10T21:02:08.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 shutdown_connections 2024-09-10T21:02:08.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9080b52e0 0x7ff9080b77a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff918108020 0x7ff918103fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9181076a0 0x7ff918103a90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 --2- 172.21.15.33:0/3387522873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff918101c90 0x7ff918103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 >> 172.21.15.33:0/3387522873 conn(0x7ff9180fd700 msgr2=0x7ff91806ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 shutdown_connections 2024-09-10T21:02:08.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:08.828+0000 7ff91f1af640 1 -- 172.21.15.33:0/3387522873 wait complete. 2024-09-10T21:02:09.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:08 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3387522873' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T21:02:09.156 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:08 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3387522873' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T21:02:09.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:08 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3387522873' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} : dispatch 2024-09-10T21:02:09.519 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-10T21:02:09.780 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:09 smithi174 ceph-mon[35635]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T21:02:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:09 smithi174 ceph-mon[35635]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T21:02:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:09 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3387522873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T21:02:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:09 smithi174 ceph-mon[35635]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T21:02:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:09 smithi174 ceph-mon[35635]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:09 smithi174 ceph-mon[35635]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T21:02:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:09 smithi174 ceph-mon[35635]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T21:02:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:09 smithi033 ceph-mon[30860]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T21:02:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:09 smithi033 ceph-mon[30860]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T21:02:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:09 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3387522873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T21:02:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:09 smithi033 ceph-mon[30860]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T21:02:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:09 smithi033 ceph-mon[30860]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:09 smithi033 ceph-mon[30860]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T21:02:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:09 smithi033 ceph-mon[30860]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T21:02:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:09 smithi104 ceph-mon[35325]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-10T21:02:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:09 smithi104 ceph-mon[35325]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T21:02:10.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:09 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3387522873' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-10T21:02:10.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:09 smithi104 ceph-mon[35325]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T21:02:10.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:09 smithi104 ceph-mon[35325]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-10T21:02:10.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:09 smithi104 ceph-mon[35325]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-09-10T21:02:10.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:09 smithi104 ceph-mon[35325]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 -- 172.21.15.33:0/3764185242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 msgr2=0x7f5c641040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/3764185242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c641040c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009900 tx=0x7f5c6002a410 comp rx=0 tx=0).stop 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 -- 172.21.15.33:0/3764185242 shutdown_connections 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/3764185242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c6410d140 0x7f5c6410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/3764185242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c64104600 0x7f5c6410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/3764185242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c641040c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 -- 172.21.15.33:0/3764185242 >> 172.21.15.33:0/3764185242 conn(0x7f5c640fd700 msgr2=0x7f5c640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:10.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 -- 172.21.15.33:0/3764185242 shutdown_connections 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 -- 172.21.15.33:0/3764185242 wait complete. 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.302+0000 7f5c6b74c640 1 Processor -- start 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c6b74c640 1 -- start start 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c6b74c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c64199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c6b74c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c64104600 0x7f5c64199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c6b74c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c6410d140 0x7f5c6419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c6b74c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64112db0 con 0x7f5c64103ce0 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c6b74c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64112c30 con 0x7f5c6410d140 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c6b74c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c64112f30 con 0x7f5c64104600 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c64199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c64199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56318/0 (socket says 172.21.15.33:56318) 2024-09-10T21:02:10.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 -- 172.21.15.33:0/2205779028 learned_addr learned my addr 172.21.15.33:0/2205779028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c69ce9640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c6410d140 0x7f5c6419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c68ce7640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c64104600 0x7f5c64199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 -- 172.21.15.33:0/2205779028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c64104600 msgr2=0x7f5c64199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c64104600 0x7f5c64199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 -- 172.21.15.33:0/2205779028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c6410d140 msgr2=0x7f5c6419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c6410d140 0x7f5c6419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c694e8640 1 -- 172.21.15.33:0/2205779028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6419f2f0 con 0x7f5c64103ce0 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.303+0000 7f5c69ce9640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c6410d140 0x7f5c6419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.304+0000 7f5c694e8640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c64199240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005940 tx=0x7f5c600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.304+0000 7f5c5a7fc640 1 -- 172.21.15.33:0/2205779028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6003f030 con 0x7f5c64103ce0 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.304+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6419f580 con 0x7f5c64103ce0 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.304+0000 7f5c5a7fc640 1 -- 172.21.15.33:0/2205779028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c60003cb0 con 0x7f5c64103ce0 2024-09-10T21:02:10.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.304+0000 7f5c5a7fc640 1 -- 172.21.15.33:0/2205779028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60035070 con 0x7f5c64103ce0 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.304+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c641a5ca0 con 0x7f5c64103ce0 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.305+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c641040c0 con 0x7f5c64103ce0 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.306+0000 7f5c5a7fc640 1 -- 172.21.15.33:0/2205779028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f5c6003bbf0 con 0x7f5c64103ce0 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.307+0000 7f5c5a7fc640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c500b52e0 0x7f5c500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.307+0000 7f5c68ce7640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c500b52e0 0x7f5c500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.307+0000 7f5c5a7fc640 1 -- 172.21.15.33:0/2205779028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5c60106920 con 0x7f5c64103ce0 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.308+0000 7f5c68ce7640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c500b52e0 0x7f5c500b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c54005fd0 tx=0x7f5c54005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:10.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.310+0000 7f5c5a7fc640 1 -- 172.21.15.33:0/2205779028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5c600cd930 con 0x7f5c64103ce0 2024-09-10T21:02:10.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.533+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f5c6419a370 con 0x7f5c64103ce0 2024-09-10T21:02:10.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.843+0000 7f5c5a7fc640 1 -- 172.21.15.33:0/2205779028 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f5c600cd230 con 0x7f5c64103ce0 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c500b52e0 msgr2=0x7f5c500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c500b52e0 0x7f5c500b77a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5c54005fd0 tx=0x7f5c54005d80 comp rx=0 tx=0).stop 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 msgr2=0x7f5c64199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c64199240 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5c60005940 tx=0x7f5c600042f0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 shutdown_connections 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c500b52e0 0x7f5c500b77a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c6410d140 0x7f5c6419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c64104600 0x7f5c64199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 --2- 172.21.15.33:0/2205779028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c64103ce0 0x7f5c64199240 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:10.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 >> 172.21.15.33:0/2205779028 conn(0x7f5c640fd700 msgr2=0x7f5c640fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:10.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 shutdown_connections 2024-09-10T21:02:10.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:10.846+0000 7f5c6b74c640 1 -- 172.21.15.33:0/2205779028 wait complete. 2024-09-10T21:02:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:10 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] up:active 2024-09-10T21:02:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:10 smithi174 ceph-mon[35635]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T21:02:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:10 smithi174 ceph-mon[35635]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T21:02:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:10 smithi174 ceph-mon[35635]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T21:02:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:10 smithi174 ceph-mon[35635]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T21:02:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:10 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2205779028' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T21:02:11.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:10 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] up:active 2024-09-10T21:02:11.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:10 smithi033 ceph-mon[30860]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T21:02:11.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:10 smithi033 ceph-mon[30860]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T21:02:11.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:10 smithi033 ceph-mon[30860]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T21:02:11.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:10 smithi033 ceph-mon[30860]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T21:02:11.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:10 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2205779028' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T21:02:11.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:10 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] up:active 2024-09-10T21:02:11.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:10 smithi104 ceph-mon[35325]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-10T21:02:11.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:10 smithi104 ceph-mon[35325]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-09-10T21:02:11.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:10 smithi104 ceph-mon[35325]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-09-10T21:02:11.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:10 smithi104 ceph-mon[35325]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-09-10T21:02:11.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:10 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2205779028' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-10T21:02:11.524 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-10T21:02:11.785 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:11.812 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: 2024-09-10T21:02:11.425+0000 7f8073f1c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-10T21:02:11.813 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: 2024-09-10T21:02:11.425+0000 7f8073f1c640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-10T21:02:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:11 smithi174 ceph-mon[35635]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-10T21:02:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:11 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2205779028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T21:02:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:11 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] up:active 2024-09-10T21:02:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:11 smithi174 ceph-mon[35635]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T21:02:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:11 smithi174 ceph-mon[35635]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T21:02:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:11 smithi174 ceph-mon[35635]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-10T21:02:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:11 smithi174 ceph-mon[35635]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T21:02:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:11 smithi033 ceph-mon[30860]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-10T21:02:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:11 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2205779028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T21:02:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:11 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] up:active 2024-09-10T21:02:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:11 smithi033 ceph-mon[30860]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T21:02:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:11 smithi033 ceph-mon[30860]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T21:02:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:11 smithi033 ceph-mon[30860]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-10T21:02:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:11 smithi033 ceph-mon[30860]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T21:02:12.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-mon[35325]: pgmap v290: 97 pgs: 97 active+clean; 579 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-10T21:02:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2205779028' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T21:02:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] up:active 2024-09-10T21:02:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-mon[35325]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-10T21:02:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-mon[35325]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-09-10T21:02:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-mon[35325]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-09-10T21:02:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:11 smithi104 ceph-mon[35325]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 -- 172.21.15.33:0/3174913576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0102d90 msgr2=0x7fc2a0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 --2- 172.21.15.33:0/3174913576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0102d90 0x7fc2a0103170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc298009900 tx=0x7fc29802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 -- 172.21.15.33:0/3174913576 shutdown_connections 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 --2- 172.21.15.33:0/3174913576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2a0109120 0x7fc2a010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 --2- 172.21.15.33:0/3174913576 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2a01087a0 0x7fc2a0108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 --2- 172.21.15.33:0/3174913576 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0102d90 0x7fc2a0103170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 -- 172.21.15.33:0/3174913576 >> 172.21.15.33:0/3174913576 conn(0x7fc2a00fe820 msgr2=0x7fc2a0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 -- 172.21.15.33:0/3174913576 shutdown_connections 2024-09-10T21:02:12.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.294+0000 7fc2a7513640 1 -- 172.21.15.33:0/3174913576 wait complete. 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 Processor -- start 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 -- start start 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2a0102d90 0x7fc2a01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2a01087a0 0x7fc2a0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0109120 0x7fc2a019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0112de0 con 0x7fc2a0109120 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0112c60 con 0x7fc2a01087a0 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a7513640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2a0112f60 con 0x7fc2a0102d90 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0109120 0x7fc2a019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0109120 0x7fc2a019ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56350/0 (socket says 172.21.15.33:56350) 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 -- 172.21.15.33:0/1386787544 learned_addr learned my addr 172.21.15.33:0/1386787544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:12.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a4aae640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2a01087a0 0x7fc2a0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a52af640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2a0102d90 0x7fc2a01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 -- 172.21.15.33:0/1386787544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2a0102d90 msgr2=0x7fc2a01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2a0102d90 0x7fc2a01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 -- 172.21.15.33:0/1386787544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2a01087a0 msgr2=0x7fc2a0199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2a01087a0 0x7fc2a0199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 -- 172.21.15.33:0/1386787544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2a019f3a0 con 0x7fc2a0109120 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.295+0000 7fc2a5ab0640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0109120 0x7fc2a019ebd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b970 tx=0x7fc29000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.296+0000 7fc29e7fc640 1 -- 172.21.15.33:0/1386787544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290019070 con 0x7fc2a0109120 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.296+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2a019f690 con 0x7fc2a0109120 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.296+0000 7fc29e7fc640 1 -- 172.21.15.33:0/1386787544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc290014440 con 0x7fc2a0109120 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.296+0000 7fc29e7fc640 1 -- 172.21.15.33:0/1386787544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2900075d0 con 0x7fc2a0109120 2024-09-10T21:02:12.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.296+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2a01a5d50 con 0x7fc2a0109120 2024-09-10T21:02:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.297+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2a0103170 con 0x7fc2a0109120 2024-09-10T21:02:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.298+0000 7fc29e7fc640 1 -- 172.21.15.33:0/1386787544 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fc29000cb40 con 0x7fc2a0109120 2024-09-10T21:02:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.299+0000 7fc29e7fc640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2880b52e0 0x7fc2880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.299+0000 7fc2a52af640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2880b52e0 0x7fc2880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.299+0000 7fc29e7fc640 1 -- 172.21.15.33:0/1386787544 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fc2900e5700 con 0x7fc2a0109120 2024-09-10T21:02:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.299+0000 7fc2a52af640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2880b52e0 0x7fc2880b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc298031440 tx=0x7fc2980095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:12.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.302+0000 7fc29e7fc640 1 -- 172.21.15.33:0/1386787544 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc2900ac0c0 con 0x7fc2a0109120 2024-09-10T21:02:12.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.528+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fc2a019a420 con 0x7fc2a0109120 2024-09-10T21:02:12.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.860+0000 7fc29e7fc640 1 -- 172.21.15.33:0/1386787544 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v26) ==== 115+0+0 (secure 0 0 0) 0x7fc2a019a420 con 0x7fc2a0109120 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2880b52e0 msgr2=0x7fc2880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2880b52e0 0x7fc2880b77a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc298031440 tx=0x7fc2980095f0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0109120 msgr2=0x7fc2a019ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0109120 0x7fc2a019ebd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc29000b970 tx=0x7fc29000be30 comp rx=0 tx=0).stop 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 shutdown_connections 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2880b52e0 0x7fc2880b77a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2a0109120 0x7fc2a019ebd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc2a01087a0 0x7fc2a0199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 --2- 172.21.15.33:0/1386787544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2a0102d90 0x7fc2a01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 >> 172.21.15.33:0/1386787544 conn(0x7fc2a00fe820 msgr2=0x7fc2a0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 shutdown_connections 2024-09-10T21:02:12.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:12.863+0000 7fc2a7513640 1 -- 172.21.15.33:0/1386787544 wait complete. 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] up:active 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: Cluster is now healthy 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-10T21:02:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:12 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1386787544' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] up:active 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: Cluster is now healthy 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-10T21:02:13.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:12 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1386787544' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T21:02:13.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] up:active 2024-09-10T21:02:13.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-10T21:02:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T21:02:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: Cluster is now healthy 2024-09-10T21:02:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-09-10T21:02:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-10T21:02:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-09-10T21:02:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:12 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1386787544' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-10T21:02:13.548 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-09-10T21:02:13.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-10T21:02:13.815 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:14.117 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:13 smithi033 ceph-mon[30860]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-10T21:02:14.118 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:13 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1386787544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T21:02:14.118 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:13 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] up:active 2024-09-10T21:02:14.118 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:13 smithi033 ceph-mon[30860]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T21:02:14.118 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:13 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:02:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:13 smithi174 ceph-mon[35635]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-10T21:02:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:13 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1386787544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T21:02:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:13 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] up:active 2024-09-10T21:02:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:13 smithi174 ceph-mon[35635]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T21:02:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:13 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:02:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:13 smithi104 ceph-mon[35325]: pgmap v291: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 7 op/s 2024-09-10T21:02:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:13 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1386787544' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-10T21:02:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:13 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] up:active 2024-09-10T21:02:14.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:13 smithi104 ceph-mon[35325]: fsmap cephfs:5 7 up:standby 5 up:active 2024-09-10T21:02:14.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:13 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 -- 172.21.15.33:0/722740042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90801087d0 msgr2=0x7f9080108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 --2- 172.21.15.33:0/722740042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90801087d0 0x7f9080108c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9068009900 tx=0x7f906802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 -- 172.21.15.33:0/722740042 shutdown_connections 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 --2- 172.21.15.33:0/722740042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9080109150 0x7f908010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 --2- 172.21.15.33:0/722740042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90801087d0 0x7f9080108c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 --2- 172.21.15.33:0/722740042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9080102dc0 0x7f90801031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 -- 172.21.15.33:0/722740042 >> 172.21.15.33:0/722740042 conn(0x7f90800fe830 msgr2=0x7f9080100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 -- 172.21.15.33:0/722740042 shutdown_connections 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 -- 172.21.15.33:0/722740042 wait complete. 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.306+0000 7f9085c1d640 1 Processor -- start 2024-09-10T21:02:14.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9085c1d640 1 -- start start 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9085c1d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9080102dc0 0x7f9080199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9085c1d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90801087d0 0x7f90801997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9085c1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9080109150 0x7f908019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9085c1d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9080112e10 con 0x7f9080109150 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9085c1d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9080112c90 con 0x7f90801087d0 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9085c1d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9080112f90 con 0x7f9080102dc0 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9080109150 0x7f908019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9080109150 0x7f908019eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56362/0 (socket says 172.21.15.33:56362) 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 -- 172.21.15.33:0/463475904 learned_addr learned my addr 172.21.15.33:0/463475904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f9077fff640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90801087d0 0x7f90801997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907f7fe640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9080102dc0 0x7f9080199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:14.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 -- 172.21.15.33:0/463475904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9080102dc0 msgr2=0x7f9080199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9080102dc0 0x7f9080199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 -- 172.21.15.33:0/463475904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90801087d0 msgr2=0x7f90801997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90801087d0 0x7f90801997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 -- 172.21.15.33:0/463475904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f908019f330 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.307+0000 7f907ffff640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9080109150 0x7f908019eb60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f907800b970 tx=0x7f907800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.308+0000 7f907d7fa640 1 -- 172.21.15.33:0/463475904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9078019070 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.308+0000 7f907d7fa640 1 -- 172.21.15.33:0/463475904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9078014440 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.308+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f908019f620 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.308+0000 7f907d7fa640 1 -- 172.21.15.33:0/463475904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90780075d0 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.308+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90801a5d60 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.309+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90801031a0 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.310+0000 7f907d7fa640 1 -- 172.21.15.33:0/463475904 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7f907800cb40 con 0x7f9080109150 2024-09-10T21:02:14.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.311+0000 7f907d7fa640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f906c0b5360 0x7f906c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.311+0000 7f907f7fe640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f906c0b5360 0x7f906c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.311+0000 7f907d7fa640 1 -- 172.21.15.33:0/463475904 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f90780e5670 con 0x7f9080109150 2024-09-10T21:02:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.311+0000 7f907f7fe640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f906c0b5360 0x7f906c0b7820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9070005e10 tx=0x7f9070005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:14.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.314+0000 7f907d7fa640 1 -- 172.21.15.33:0/463475904 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f90780abf90 con 0x7f9080109150 2024-09-10T21:02:14.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.528+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9080063590 con 0x7f906c0b5360 2024-09-10T21:02:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.562+0000 7f907d7fa640 1 -- 172.21.15.33:0/463475904 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9080063590 con 0x7f906c0b5360 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f906c0b5360 msgr2=0x7f906c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f906c0b5360 0x7f906c0b7820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9070005e10 tx=0x7f9070005da0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9080109150 msgr2=0x7f908019eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9080109150 0x7f908019eb60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f907800b970 tx=0x7f907800be30 comp rx=0 tx=0).stop 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 shutdown_connections 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f906c0b5360 0x7f906c0b7820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9080109150 0x7f908019eb60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90801087d0 0x7f90801997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 --2- 172.21.15.33:0/463475904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9080102dc0 0x7f9080199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 >> 172.21.15.33:0/463475904 conn(0x7f90800fe830 msgr2=0x7f9080100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 shutdown_connections 2024-09-10T21:02:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:14.564+0000 7f9085c1d640 1 -- 172.21.15.33:0/463475904 wait complete. 2024-09-10T21:02:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:02:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T21:02:14.534+0000 7f0ddcbe8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T21:02:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:02:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T21:02:14.534+0000 7f0ddcbe8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T21:02:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:02:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T21:02:14.534+0000 7f0ddcbe8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T21:02:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:02:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T21:02:14.534+0000 7f0ddcbe8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T21:02:14.927 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:02:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T21:02:14.534+0000 7f0ddcbe8640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-10T21:02:15.391 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-10T21:02:15.655 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:16.027 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:15 smithi033 ceph-mon[30860]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-10T21:02:16.027 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:15 smithi033 ceph-mon[30860]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.110+0000 7fa83dd73640 1 -- 172.21.15.33:0/1898974954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 msgr2=0x7fa8381031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.110+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1898974954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa8381031a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa828009900 tx=0x7fa82802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 -- 172.21.15.33:0/1898974954 shutdown_connections 2024-09-10T21:02:16.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1898974954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838109150 0x7fa83810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1898974954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8381087d0 0x7fa838108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1898974954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa8381031a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 -- 172.21.15.33:0/1898974954 >> 172.21.15.33:0/1898974954 conn(0x7fa8380fe830 msgr2=0x7fa838100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 -- 172.21.15.33:0/1898974954 shutdown_connections 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 -- 172.21.15.33:0/1898974954 wait complete. 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 Processor -- start 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.111+0000 7fa83dd73640 1 -- start start 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa83dd73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa838199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa83dd73640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8381087d0 0x7fa838199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa83dd73640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838109150 0x7fa83819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa83dd73640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa838112e50 con 0x7fa838102dc0 2024-09-10T21:02:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa83dd73640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa838112cd0 con 0x7fa838109150 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa83dd73640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa838112fd0 con 0x7fa8381087d0 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa838199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa838199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56406/0 (socket says 172.21.15.33:56406) 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 -- 172.21.15.33:0/1435015469 learned_addr learned my addr 172.21.15.33:0/1435015469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa836ffd640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8381087d0 0x7fa838199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa837fff640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838109150 0x7fa83819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 -- 172.21.15.33:0/1435015469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8381087d0 msgr2=0x7fa838199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8381087d0 0x7fa838199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 -- 172.21.15.33:0/1435015469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838109150 msgr2=0x7fa83819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838109150 0x7fa83819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 -- 172.21.15.33:0/1435015469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83819f4f0 con 0x7fa838102dc0 2024-09-10T21:02:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa837fff640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838109150 0x7fa83819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa8377fe640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa838199440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa828006010 tx=0x7fa8280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa834ff9640 1 -- 172.21.15.33:0/1435015469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82803f030 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83819f780 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa834ff9640 1 -- 172.21.15.33:0/1435015469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa828003800 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.112+0000 7fa834ff9640 1 -- 172.21.15.33:0/1435015469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828035070 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.113+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8381a5ea0 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.114+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa838103220 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.115+0000 7fa834ff9640 1 -- 172.21.15.33:0/1435015469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 16) ==== 153250+0+0 (secure 0 0 0) 0x7fa828003cb0 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.116+0000 7fa834ff9640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa8240b52e0 0x7fa8240b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.116+0000 7fa836ffd640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa8240b52e0 0x7fa8240b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.116+0000 7fa834ff9640 1 -- 172.21.15.33:0/1435015469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fa8281065b0 con 0x7fa838102dc0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.116+0000 7fa836ffd640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa8240b52e0 0x7fa8240b77a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa838108330 tx=0x7fa820009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:16.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.119+0000 7fa834ff9640 1 -- 172.21.15.33:0/1435015469 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa8280cd5c0 con 0x7fa838102dc0 2024-09-10T21:02:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:15 smithi174 ceph-mon[35635]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-10T21:02:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:15 smithi174 ceph-mon[35635]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:15 smithi104 ceph-mon[35325]: pgmap v292: 97 pgs: 97 active+clean; 582 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 4.0 KiB/s wr, 13 op/s 2024-09-10T21:02:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:15 smithi104 ceph-mon[35325]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:16.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.333+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa838063590 con 0x7fa8240b52e0 2024-09-10T21:02:16.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.356+0000 7fa834ff9640 1 -- 172.21.15.33:0/1435015469 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa838063590 con 0x7fa8240b52e0 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa8240b52e0 msgr2=0x7fa8240b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa8240b52e0 0x7fa8240b77a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa838108330 tx=0x7fa820009110 comp rx=0 tx=0).stop 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 msgr2=0x7fa838199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa838199440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa828006010 tx=0x7fa8280042f0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 shutdown_connections 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa8240b52e0 0x7fa8240b77a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa838109150 0x7fa83819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8381087d0 0x7fa838199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 --2- 172.21.15.33:0/1435015469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa838102dc0 0x7fa838199440 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:16.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 >> 172.21.15.33:0/1435015469 conn(0x7fa8380fe830 msgr2=0x7fa838100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 shutdown_connections 2024-09-10T21:02:16.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:16.359+0000 7fa83dd73640 1 -- 172.21.15.33:0/1435015469 wait complete. 2024-09-10T21:02:16.812 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:02:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: 2024-09-10T21:02:16.426+0000 7f8073f1c640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-10T21:02:17.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-10T21:02:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:16 smithi174 ceph-mon[35635]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2024-09-10T21:02:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:16 smithi174 ceph-mon[35635]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:17.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:16 smithi033 ceph-mon[30860]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2024-09-10T21:02:17.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:16 smithi033 ceph-mon[30860]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:17.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:16 smithi104 ceph-mon[35325]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 4.6 KiB/s wr, 16 op/s 2024-09-10T21:02:17.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:16 smithi104 ceph-mon[35325]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:17.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1180418011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5410a920 msgr2=0x7f5e5410cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1180418011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5410a920 0x7f5e5410cd10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009900 tx=0x7f5e4402a410 comp rx=0 tx=0).stop 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1180418011 shutdown_connections 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1180418011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d250 0x7f5e5410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1180418011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e5410a920 0x7f5e5410cd10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1180418011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e54101810 0x7f5e54101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1180418011 >> 172.21.15.33:0/1180418011 conn(0x7f5e54076f50 msgr2=0x7f5e54077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1180418011 shutdown_connections 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1180418011 wait complete. 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 Processor -- start 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.721+0000 7f5e5b77b640 1 -- start start 2024-09-10T21:02:17.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e5b77b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e54101810 0x7f5e54199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e5b77b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e5410a920 0x7f5e54199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e5b77b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d250 0x7f5e5419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e5b77b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54112d90 con 0x7f5e54101810 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e5b77b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54112c10 con 0x7f5e5410a920 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e5b77b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e54112f10 con 0x7f5e5410d250 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e54101810 0x7f5e54199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e54101810 0x7f5e54199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56422/0 (socket says 172.21.15.33:56422) 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 -- 172.21.15.33:0/1388354853 learned_addr learned my addr 172.21.15.33:0/1388354853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e58d16640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e5410a920 0x7f5e54199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:17.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59d18640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d250 0x7f5e5419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 -- 172.21.15.33:0/1388354853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d250 msgr2=0x7f5e5419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d250 0x7f5e5419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 -- 172.21.15.33:0/1388354853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e5410a920 msgr2=0x7f5e54199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e5410a920 0x7f5e54199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 -- 172.21.15.33:0/1388354853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e5419f2f0 con 0x7f5e54101810 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e58d16640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e5410a920 0x7f5e54199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59d18640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d250 0x7f5e5419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.722+0000 7f5e59517640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e54101810 0x7f5e54199240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c890 tx=0x7f5e5000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.723+0000 7f5e4a7fc640 1 -- 172.21.15.33:0/1388354853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50004220 con 0x7f5e54101810 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.723+0000 7f5e4a7fc640 1 -- 172.21.15.33:0/1388354853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e500043c0 con 0x7f5e54101810 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.723+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e5419f5e0 con 0x7f5e54101810 2024-09-10T21:02:17.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.723+0000 7f5e4a7fc640 1 -- 172.21.15.33:0/1388354853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50004a00 con 0x7f5e54101810 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.723+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e54102890 con 0x7f5e54101810 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.724+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e5406ae10 con 0x7f5e54101810 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.725+0000 7f5e4a7fc640 1 -- 172.21.15.33:0/1388354853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f5e500079b0 con 0x7f5e54101810 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.726+0000 7f5e4a7fc640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5e400b53f0 0x7f5e400b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.726+0000 7f5e58d16640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5e400b53f0 0x7f5e400b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.726+0000 7f5e4a7fc640 1 -- 172.21.15.33:0/1388354853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5e500e5a80 con 0x7f5e54101810 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.726+0000 7f5e58d16640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5e400b53f0 0x7f5e400b78b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5e44031440 tx=0x7f5e440095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:17.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.730+0000 7f5e4a7fc640 1 -- 172.21.15.33:0/1388354853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5e500e7e20 con 0x7f5e54101810 2024-09-10T21:02:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:17 smithi033 ceph-mon[30860]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T21:02:17.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.946+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f5e54100cb0 con 0x7f5e400b53f0 2024-09-10T21:02:17.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.951+0000 7f5e4a7fc640 1 -- 172.21.15.33:0/1388354853 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f5e54100cb0 con 0x7f5e400b53f0 2024-09-10T21:02:17.956 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_0/a5ee1cc4-a75a-4b7f-a8d2-27fb8f118cf2 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5e400b53f0 msgr2=0x7f5e400b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5e400b53f0 0x7f5e400b78b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5e44031440 tx=0x7f5e440095f0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e54101810 msgr2=0x7f5e54199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e54101810 0x7f5e54199240 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5e5000c890 tx=0x7f5e5000cd50 comp rx=0 tx=0).stop 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 shutdown_connections 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5e400b53f0 0x7f5e400b78b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d250 0x7f5e5419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e5410a920 0x7f5e54199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 --2- 172.21.15.33:0/1388354853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e54101810 0x7f5e54199240 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 >> 172.21.15.33:0/1388354853 conn(0x7f5e54076f50 msgr2=0x7f5e541005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 shutdown_connections 2024-09-10T21:02:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:17.954+0000 7f5e5b77b640 1 -- 172.21.15.33:0/1388354853 wait complete. 2024-09-10T21:02:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:17 smithi174 ceph-mon[35635]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T21:02:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:17 smithi104 ceph-mon[35325]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-10T21:02:18.619 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-10T21:02:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:19.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:18 smithi033 ceph-mon[30860]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:19.140 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:18 smithi033 ceph-mon[30860]: pgmap v294: 97 pgs: 97 active+clean; 605 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-10T21:02:19.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:18 smithi174 ceph-mon[35635]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:19.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:18 smithi174 ceph-mon[35635]: pgmap v294: 97 pgs: 97 active+clean; 605 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-10T21:02:19.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:18 smithi104 ceph-mon[35325]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:19.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:18 smithi104 ceph-mon[35325]: pgmap v294: 97 pgs: 97 active+clean; 605 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 6.3 KiB/s wr, 22 op/s 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 -- 172.21.15.33:0/3511021616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968104600 msgr2=0x7f196810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 --2- 172.21.15.33:0/3511021616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968104600 0x7f196810caf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f195c009900 tx=0x7f195c02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 -- 172.21.15.33:0/3511021616 shutdown_connections 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 --2- 172.21.15.33:0/3511021616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f196810d140 0x7f196810f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 --2- 172.21.15.33:0/3511021616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968104600 0x7f196810caf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 --2- 172.21.15.33:0/3511021616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1968103ce0 0x7f19681040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 -- 172.21.15.33:0/3511021616 >> 172.21.15.33:0/3511021616 conn(0x7f19680fd700 msgr2=0x7f19680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:19.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.347+0000 7f196f005640 1 -- 172.21.15.33:0/3511021616 shutdown_connections 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 -- 172.21.15.33:0/3511021616 wait complete. 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 Processor -- start 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 -- start start 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968103ce0 0x7f1968199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1968104600 0x7f19681997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f196810d140 0x7f196819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112db0 con 0x7f1968103ce0 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112c30 con 0x7f196810d140 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196f005640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1968112f30 con 0x7f1968104600 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196cda1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968103ce0 0x7f1968199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196cda1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968103ce0 0x7f1968199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56446/0 (socket says 172.21.15.33:56446) 2024-09-10T21:02:19.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.348+0000 7f196cda1640 1 -- 172.21.15.33:0/696090428 learned_addr learned my addr 172.21.15.33:0/696090428 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f1967fff640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1968104600 0x7f19681997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196d5a2640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f196810d140 0x7f196819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196cda1640 1 -- 172.21.15.33:0/696090428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1968104600 msgr2=0x7f19681997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196cda1640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1968104600 0x7f19681997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196cda1640 1 -- 172.21.15.33:0/696090428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f196810d140 msgr2=0x7f196819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196cda1640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f196810d140 0x7f196819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196cda1640 1 -- 172.21.15.33:0/696090428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f196819f310 con 0x7f1968103ce0 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196d5a2640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f196810d140 0x7f196819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196cda1640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968103ce0 0x7f1968199260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f195800c890 tx=0x7f195800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f1965ffb640 1 -- 172.21.15.33:0/696090428 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958004220 con 0x7f1968103ce0 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f196819f600 con 0x7f1968103ce0 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f1965ffb640 1 -- 172.21.15.33:0/696090428 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19580043c0 con 0x7f1968103ce0 2024-09-10T21:02:19.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f1965ffb640 1 -- 172.21.15.33:0/696090428 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958004a00 con 0x7f1968103ce0 2024-09-10T21:02:19.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.349+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19681a5e90 con 0x7f1968103ce0 2024-09-10T21:02:19.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.350+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f196806bb90 con 0x7f1968103ce0 2024-09-10T21:02:19.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.352+0000 7f1965ffb640 1 -- 172.21.15.33:0/696090428 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f19580079b0 con 0x7f1968103ce0 2024-09-10T21:02:19.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.352+0000 7f1965ffb640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f19500b53f0 0x7f19500b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:19.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.353+0000 7f1967fff640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f19500b53f0 0x7f19500b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:19.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.353+0000 7f1965ffb640 1 -- 172.21.15.33:0/696090428 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f19580e5a30 con 0x7f1968103ce0 2024-09-10T21:02:19.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.353+0000 7f1967fff640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f19500b53f0 0x7f19500b78b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f195c000c00 tx=0x7f195c005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:19.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.357+0000 7f1965ffb640 1 -- 172.21.15.33:0/696090428 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1958014030 con 0x7f1968103ce0 2024-09-10T21:02:19.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.568+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f19680ffa20 con 0x7f19500b53f0 2024-09-10T21:02:19.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.593+0000 7f1965ffb640 1 -- 172.21.15.33:0/696090428 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f19680ffa20 con 0x7f19500b53f0 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f19500b53f0 msgr2=0x7f19500b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f19500b53f0 0x7f19500b78b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f195c000c00 tx=0x7f195c005e80 comp rx=0 tx=0).stop 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968103ce0 msgr2=0x7f1968199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968103ce0 0x7f1968199260 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f195800c890 tx=0x7f195800cd50 comp rx=0 tx=0).stop 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 shutdown_connections 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f19500b53f0 0x7f19500b78b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f196810d140 0x7f196819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1968104600 0x7f19681997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 --2- 172.21.15.33:0/696090428 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1968103ce0 0x7f1968199260 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 >> 172.21.15.33:0/696090428 conn(0x7f19680fd700 msgr2=0x7f19680fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 shutdown_connections 2024-09-10T21:02:19.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:19.595+0000 7f196f005640 1 -- 172.21.15.33:0/696090428 wait complete. 2024-09-10T21:02:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:19 smithi174 ceph-mon[35635]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:20.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:19 smithi033 ceph-mon[30860]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:20.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:19 smithi104 ceph-mon[35325]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:20.406 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-10T21:02:20.667 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:21.049 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:20 smithi033 ceph-mon[30860]: pgmap v295: 97 pgs: 97 active+clean; 606 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-10T21:02:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:20 smithi174 ceph-mon[35635]: pgmap v295: 97 pgs: 97 active+clean; 606 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-10T21:02:21.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.131+0000 7fd6d65df640 1 -- 172.21.15.33:0/3403517169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 msgr2=0x7fd6d0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 --2- 172.21.15.33:0/3403517169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0108c10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8009900 tx=0x7fd6b802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 -- 172.21.15.33:0/3403517169 shutdown_connections 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 --2- 172.21.15.33:0/3403517169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d0109150 0x7fd6d010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 --2- 172.21.15.33:0/3403517169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0108c10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 --2- 172.21.15.33:0/3403517169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6d0102dc0 0x7fd6d01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 -- 172.21.15.33:0/3403517169 >> 172.21.15.33:0/3403517169 conn(0x7fd6d00fe870 msgr2=0x7fd6d0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 -- 172.21.15.33:0/3403517169 shutdown_connections 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 -- 172.21.15.33:0/3403517169 wait complete. 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 Processor -- start 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 -- start start 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d0102dc0 0x7fd6d01993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:21.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.132+0000 7fd6d65df640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d65df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6d0109150 0x7fd6d019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d65df640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0112e50 con 0x7fd6d01087d0 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d65df640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0112cd0 con 0x7fd6d0102dc0 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d65df640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6d0112fd0 con 0x7fd6d0109150 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55972/0 (socket says 172.21.15.33:55972) 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 -- 172.21.15.33:0/1218400007 learned_addr learned my addr 172.21.15.33:0/1218400007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6cffff640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d0102dc0 0x7fd6d01993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:21.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d4b7c640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6d0109150 0x7fd6d019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 -- 172.21.15.33:0/1218400007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6d0109150 msgr2=0x7fd6d019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6d0109150 0x7fd6d019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 -- 172.21.15.33:0/1218400007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d0102dc0 msgr2=0x7fd6d01993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d0102dc0 0x7fd6d01993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 -- 172.21.15.33:0/1218400007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6d019f470 con 0x7fd6d01087d0 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d4b7c640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6d0109150 0x7fd6d019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6c7fff640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0199900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8006010 tx=0x7fd6b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6cdffb640 1 -- 172.21.15.33:0/1218400007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b803f030 con 0x7fd6d01087d0 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6cdffb640 1 -- 172.21.15.33:0/1218400007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b8003800 con 0x7fd6d01087d0 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6cdffb640 1 -- 172.21.15.33:0/1218400007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8035070 con 0x7fd6d01087d0 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6d019f700 con 0x7fd6d01087d0 2024-09-10T21:02:21.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.133+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6d01a5e20 con 0x7fd6d01087d0 2024-09-10T21:02:21.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.134+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6d0103220 con 0x7fd6d01087d0 2024-09-10T21:02:21.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.136+0000 7fd6cdffb640 1 -- 172.21.15.33:0/1218400007 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fd6b8031b10 con 0x7fd6d01087d0 2024-09-10T21:02:21.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.136+0000 7fd6cdffb640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6bc0b5370 0x7fd6bc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:21.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.137+0000 7fd6cffff640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6bc0b5370 0x7fd6bc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:21.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.137+0000 7fd6cdffb640 1 -- 172.21.15.33:0/1218400007 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fd6b8106830 con 0x7fd6d01087d0 2024-09-10T21:02:21.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.137+0000 7fd6cffff640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6bc0b5370 0x7fd6bc0b7830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009890 tx=0x7fd6c0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:21.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.139+0000 7fd6cdffb640 1 -- 172.21.15.33:0/1218400007 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd6b80cd740 con 0x7fd6d01087d0 2024-09-10T21:02:21.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:20 smithi104 ceph-mon[35325]: pgmap v295: 97 pgs: 97 active+clean; 606 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 22 op/s 2024-09-10T21:02:21.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.353+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fd6d0063590 con 0x7fd6bc0b5370 2024-09-10T21:02:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.357+0000 7fd6cdffb640 1 -- 172.21.15.33:0/1218400007 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fd6d0063590 con 0x7fd6bc0b5370 2024-09-10T21:02:21.362 INFO:teuthology.orchestra.run.smithi033.stdout:/volumes/qa/sv_1/44306e77-b631-4a07-b5a9-d08fc1590f9a 2024-09-10T21:02:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.359+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6bc0b5370 msgr2=0x7fd6bc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.359+0000 7fd6d65df640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6bc0b5370 0x7fd6bc0b7830 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009890 tx=0x7fd6c0008040 comp rx=0 tx=0).stop 2024-09-10T21:02:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.359+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 msgr2=0x7fd6d0199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.359+0000 7fd6d65df640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0199900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8006010 tx=0x7fd6b80042f0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.359+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 shutdown_connections 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.360+0000 7fd6d65df640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6bc0b5370 0x7fd6bc0b7830 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.360+0000 7fd6d65df640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6d0109150 0x7fd6d019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.360+0000 7fd6d65df640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d01087d0 0x7fd6d0199900 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.360+0000 7fd6d65df640 1 --2- 172.21.15.33:0/1218400007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d0102dc0 0x7fd6d01993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.360+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 >> 172.21.15.33:0/1218400007 conn(0x7fd6d00fe870 msgr2=0x7fd6d0100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.360+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 shutdown_connections 2024-09-10T21:02:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:21.360+0000 7fd6d65df640 1 -- 172.21.15.33:0/1218400007 wait complete. 2024-09-10T21:02:21.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:21 smithi033 ceph-mon[30860]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:22.019 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:02:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:21 smithi174 ceph-mon[35635]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:22.290 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:22.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:21 smithi104 ceph-mon[35325]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 -- 172.21.15.33:0/2860546703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 msgr2=0x7f5c2c10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 --2- 172.21.15.33:0/2860546703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c10f5a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5c24009ca0 tx=0x7f5c2402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 -- 172.21.15.33:0/2860546703 shutdown_connections 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 --2- 172.21.15.33:0/2860546703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c10f5a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 --2- 172.21.15.33:0/2860546703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c2c104600 0x7f5c2c10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 --2- 172.21.15.33:0/2860546703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c2c103ce0 0x7f5c2c1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 -- 172.21.15.33:0/2860546703 >> 172.21.15.33:0/2860546703 conn(0x7f5c2c0fd700 msgr2=0x7f5c2c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 -- 172.21.15.33:0/2860546703 shutdown_connections 2024-09-10T21:02:22.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.770+0000 7f5c30859640 1 -- 172.21.15.33:0/2860546703 wait complete. 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 Processor -- start 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 -- start start 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c2c103ce0 0x7f5c2c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c2c104600 0x7f5c2c101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c112d70 con 0x7f5c2c10d140 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c112bf0 con 0x7f5c2c103ce0 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c30859640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c2c112ef0 con 0x7f5c2c104600 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c102410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55998/0 (socket says 172.21.15.33:55998) 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 -- 172.21.15.33:0/1423876957 learned_addr learned my addr 172.21.15.33:0/1423876957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2a59c640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c2c103ce0 0x7f5c2c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:22.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c29d9b640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c2c104600 0x7f5c2c101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 -- 172.21.15.33:0/1423876957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c2c104600 msgr2=0x7f5c2c101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c2c104600 0x7f5c2c101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 -- 172.21.15.33:0/1423876957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c2c103ce0 msgr2=0x7f5c2c101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c2c103ce0 0x7f5c2c101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.771+0000 7f5c2ad9d640 1 -- 172.21.15.33:0/1423876957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c2c1a7680 con 0x7f5c2c10d140 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.772+0000 7f5c2ad9d640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c102410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5c24005cc0 tx=0x7f5c2402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.772+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/1423876957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c24003bf0 con 0x7f5c2c10d140 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.772+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/1423876957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c24003d90 con 0x7f5c2c10d140 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.772+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2c1a7910 con 0x7f5c2c10d140 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.772+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/1423876957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c2402fbc0 con 0x7f5c2c10d140 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.772+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2c1a7ea0 con 0x7f5c2c10d140 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.773+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2c1040c0 con 0x7f5c2c10d140 2024-09-10T21:02:22.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.774+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/1423876957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f5c24004610 con 0x7f5c2c10d140 2024-09-10T21:02:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.775+0000 7f5c1b7fe640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c140b53f0 0x7f5c140b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.775+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/1423876957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f5c24107190 con 0x7f5c2c10d140 2024-09-10T21:02:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.775+0000 7f5c2a59c640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c140b53f0 0x7f5c140b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.775+0000 7f5c2a59c640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c140b53f0 0x7f5c140b78b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005d00 tx=0x7f5c1c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:22.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:22.778+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/1423876957 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5c240ce0a0 con 0x7f5c2c10d140 2024-09-10T21:02:23.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.006+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c2c02e5c0 con 0x7f5c2c10d140 2024-09-10T21:02:23.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.007+0000 7f5c1b7fe640 1 -- 172.21.15.33:0/1423876957 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11973 (secure 0 0 0) 0x7f5c240cd9a0 con 0x7f5c2c10d140 2024-09-10T21:02:23.011 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:23.011 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"btime":"2024-09-10T21:02:12:865816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:02:12.865787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:02:23.011 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c140b53f0 msgr2=0x7f5c140b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c140b53f0 0x7f5c140b78b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c005d00 tx=0x7f5c1c00b410 comp rx=0 tx=0).stop 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 msgr2=0x7f5c2c102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c102410 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5c24005cc0 tx=0x7f5c2402f380 comp rx=0 tx=0).stop 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 shutdown_connections 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5c140b53f0 0x7f5c140b78b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c2c10d140 0x7f5c2c102410 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c2c104600 0x7f5c2c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 --2- 172.21.15.33:0/1423876957 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c2c103ce0 0x7f5c2c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 >> 172.21.15.33:0/1423876957 conn(0x7f5c2c0fd700 msgr2=0x7f5c2c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 shutdown_connections 2024-09-10T21:02:23.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:23.009+0000 7f5c30859640 1 -- 172.21.15.33:0/1423876957 wait complete. 2024-09-10T21:02:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:22 smithi174 ceph-mon[35635]: pgmap v296: 97 pgs: 97 active+clean; 606 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2024-09-10T21:02:23.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:22 smithi033 ceph-mon[30860]: pgmap v296: 97 pgs: 97 active+clean; 606 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2024-09-10T21:02:23.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:22 smithi104 ceph-mon[35325]: pgmap v296: 97 pgs: 97 active+clean; 606 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.5 KiB/s wr, 22 op/s 2024-09-10T21:02:23.809 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:02:24.075 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:23 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1423876957' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:02:24.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:23 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1423876957' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:02:24.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:23 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1423876957' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 -- 172.21.15.33:0/1412752704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 msgr2=0x7fef8410f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 --2- 172.21.15.33:0/1412752704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8410f590 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009ca0 tx=0x7fef7c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 -- 172.21.15.33:0/1412752704 shutdown_connections 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 --2- 172.21.15.33:0/1412752704 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8410f590 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 --2- 172.21.15.33:0/1412752704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef84104570 0x7fef8410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 --2- 172.21.15.33:0/1412752704 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef84103b70 0x7fef84103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 -- 172.21.15.33:0/1412752704 >> 172.21.15.33:0/1412752704 conn(0x7fef840fd700 msgr2=0x7fef840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 -- 172.21.15.33:0/1412752704 shutdown_connections 2024-09-10T21:02:24.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.547+0000 7fef8aa91640 1 -- 172.21.15.33:0/1412752704 wait complete. 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 Processor -- start 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 -- start start 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef84103b70 0x7fef84199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef84104570 0x7fef84199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fef84112df0 con 0x7fef8410d120 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fef84112c70 con 0x7fef84103b70 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8aa91640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef84112f70 con 0x7fef84104570 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8902e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8902e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56014/0 (socket says 172.21.15.33:56014) 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8902e640 1 -- 172.21.15.33:0/3814287148 learned_addr learned my addr 172.21.15.33:0/3814287148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef8882d640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef84103b70 0x7fef84199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:24.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.548+0000 7fef83fff640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef84104570 0x7fef84199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8902e640 1 -- 172.21.15.33:0/3814287148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef84104570 msgr2=0x7fef84199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8902e640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef84104570 0x7fef84199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8902e640 1 -- 172.21.15.33:0/3814287148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef84103b70 msgr2=0x7fef84199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8902e640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef84103b70 0x7fef84199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8902e640 1 -- 172.21.15.33:0/3814287148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8419f2f0 con 0x7fef8410d120 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8902e640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8419eb20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009430 tx=0x7fef7c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef81ffb640 1 -- 172.21.15.33:0/3814287148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c003c30 con 0x7fef8410d120 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef81ffb640 1 -- 172.21.15.33:0/3814287148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef7c003dd0 con 0x7fef8410d120 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8419f580 con 0x7fef8410d120 2024-09-10T21:02:24.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef81ffb640 1 -- 172.21.15.33:0/3814287148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c02faa0 con 0x7fef8410d120 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.549+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef841a5ca0 con 0x7fef8410d120 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.550+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef84103f50 con 0x7fef8410d120 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.552+0000 7fef81ffb640 1 -- 172.21.15.33:0/3814287148 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fef7c0045f0 con 0x7fef8410d120 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.553+0000 7fef81ffb640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fef740b5370 0x7fef740b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.553+0000 7fef8882d640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fef740b5370 0x7fef740b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.553+0000 7fef81ffb640 1 -- 172.21.15.33:0/3814287148 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fef7c107030 con 0x7fef8410d120 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.553+0000 7fef8882d640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fef740b5370 0x7fef740b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef7800bbe0 tx=0x7fef7800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.556+0000 7fef81ffb640 1 -- 172.21.15.33:0/3814287148 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fef7c0cdf40 con 0x7fef8410d120 2024-09-10T21:02:24.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.740+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef841a5f60 con 0x7fef8410d120 2024-09-10T21:02:24.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.742+0000 7fef81ffb640 1 -- 172.21.15.33:0/3814287148 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22281 (secure 0 0 0) 0x7fef7c0cd840 con 0x7fef8410d120 2024-09-10T21:02:24.747 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:24.747 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:01.445442+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:02:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fef740b5370 msgr2=0x7fef740b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fef740b5370 0x7fef740b7830 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef7800bbe0 tx=0x7fef7800b390 comp rx=0 tx=0).stop 2024-09-10T21:02:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 msgr2=0x7fef8419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8419eb20 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fef7c009430 tx=0x7fef7c0319f0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 shutdown_connections 2024-09-10T21:02:24.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fef740b5370 0x7fef740b7830 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef8410d120 0x7fef8419eb20 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef84104570 0x7fef84199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 --2- 172.21.15.33:0/3814287148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fef84103b70 0x7fef84199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 >> 172.21.15.33:0/3814287148 conn(0x7fef840fd700 msgr2=0x7fef840fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 shutdown_connections 2024-09-10T21:02:24.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:24.745+0000 7fef8aa91640 1 -- 172.21.15.33:0/3814287148 wait complete. 2024-09-10T21:02:25.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:24 smithi033 ceph-mon[30860]: pgmap v297: 97 pgs: 97 active+clean; 630 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-09-10T21:02:25.107 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:24 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3814287148' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:25.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:24 smithi104 ceph-mon[35325]: pgmap v297: 97 pgs: 97 active+clean; 630 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-09-10T21:02:25.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:24 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3814287148' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:24 smithi174 ceph-mon[35635]: pgmap v297: 97 pgs: 97 active+clean; 630 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.2 KiB/s wr, 24 op/s 2024-09-10T21:02:25.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:24 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3814287148' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:25.421 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T21:02:25.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T21:02:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.158+0000 7f70aa9b3640 1 -- 172.21.15.33:0/2966121636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 msgr2=0x7f70a41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/2966121636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41031a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7094009900 tx=0x7f709402a410 comp rx=0 tx=0).stop 2024-09-10T21:02:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 -- 172.21.15.33:0/2966121636 shutdown_connections 2024-09-10T21:02:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/2966121636 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a4109150 0x7f70a410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/2966121636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70a41087d0 0x7f70a4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/2966121636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41031a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 -- 172.21.15.33:0/2966121636 >> 172.21.15.33:0/2966121636 conn(0x7f70a40fe830 msgr2=0x7f70a4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 -- 172.21.15.33:0/2966121636 shutdown_connections 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 -- 172.21.15.33:0/2966121636 wait complete. 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 Processor -- start 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.159+0000 7f70aa9b3640 1 -- start start 2024-09-10T21:02:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70aa9b3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70aa9b3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70a41087d0 0x7f70a4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70aa9b3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a4109150 0x7f70a419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70aa9b3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4112ea0 con 0x7f70a4102dc0 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70aa9b3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4112d20 con 0x7f70a4109150 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70aa9b3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70a4113020 con 0x7f70a41087d0 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56036/0 (socket says 172.21.15.33:56036) 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 -- 172.21.15.33:0/819141433 learned_addr learned my addr 172.21.15.33:0/819141433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a37fe640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70a41087d0 0x7f70a4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a8f50640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a4109150 0x7f70a419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 -- 172.21.15.33:0/819141433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70a41087d0 msgr2=0x7f70a4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70a41087d0 0x7f70a4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 -- 172.21.15.33:0/819141433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a4109150 msgr2=0x7f70a419ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a4109150 0x7f70a419ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 -- 172.21.15.33:0/819141433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70a419f4a0 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a37fe640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70a41087d0 0x7f70a4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a8f50640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a4109150 0x7f70a419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a3fff640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41993f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7094031440 tx=0x7f7094003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a17fa640 1 -- 172.21.15.33:0/819141433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f709403f070 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70a419f730 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a17fa640 1 -- 172.21.15.33:0/819141433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7094031d60 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.160+0000 7f70a17fa640 1 -- 172.21.15.33:0/819141433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7094035030 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.161+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70a41a5e50 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.162+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70a4103220 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.163+0000 7f70a17fa640 1 -- 172.21.15.33:0/819141433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f709403b4f0 con 0x7f70a4102dc0 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.164+0000 7f70a17fa640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f708c0b53b0 0x7f708c0b7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.164+0000 7f70a37fe640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f708c0b53b0 0x7f708c0b7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.164+0000 7f70a17fa640 1 -- 172.21.15.33:0/819141433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f709403be00 con 0x7f70a4102dc0 2024-09-10T21:02:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.164+0000 7f70a37fe640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f708c0b53b0 0x7f708c0b7870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7098006fd0 tx=0x7f7098008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.167+0000 7f70a17fa640 1 -- 172.21.15.33:0/819141433 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f70940cde20 con 0x7f70a4102dc0 2024-09-10T21:02:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.351+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f70a4063590 con 0x7f708c0b53b0 2024-09-10T21:02:26.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.355+0000 7f70a17fa640 1 -- 172.21.15.33:0/819141433 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258905 (secure 0 0 0) 0x7f70a4063590 con 0x7f708c0b53b0 2024-09-10T21:02:26.362 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-10T21:02:26.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f708c0b53b0 msgr2=0x7f708c0b7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f708c0b53b0 0x7f708c0b7870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7098006fd0 tx=0x7f7098008040 comp rx=0 tx=0).stop 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 msgr2=0x7f70a41993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41993f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7094031440 tx=0x7f7094003d80 comp rx=0 tx=0).stop 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 shutdown_connections 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f708c0b53b0 0x7f708c0b7870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70a4109150 0x7f70a419ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70a41087d0 0x7f70a4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 --2- 172.21.15.33:0/819141433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a4102dc0 0x7f70a41993f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 >> 172.21.15.33:0/819141433 conn(0x7f70a40fe830 msgr2=0x7f70a4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 shutdown_connections 2024-09-10T21:02:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:26.360+0000 7f70aa9b3640 1 -- 172.21.15.33:0/819141433 wait complete. 2024-09-10T21:02:27.164 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-10T21:02:26.088861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645388,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":224,"num_read_kb":198,"num_write":357,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355180,"kb_used_data":10092,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586644,"statfs":{"total":1151940427776,"available":1151576723456,"internally_reserved":0,"allocated":10334208,"data_stored":6599916,"data_compressed":107421,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49562,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":139,"num_read_kb":126,"num_write":76,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003362"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788385+0000","last_change":"2024-09-10T21:02:00.805421+0000","last_active":"2024-09-10T21:02:01.788385+0000","last_peered":"2024-09-10T21:02:01.788385+0000","last_clean":"2024-09-10T21:02:01.788385+0000","last_became_active":"2024-09-10T21:02:00.804274+0000","last_became_peered":"2024-09-10T21:02:00.804274+0000","last_unstale":"2024-09-10T21:02:01.788385+0000","last_undegraded":"2024-09-10T21:02:01.788385+0000","last_fullsized":"2024-09-10T21:02:01.788385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:12.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803249+0000","last_change":"2024-09-10T21:02:00.806104+0000","last_active":"2024-09-10T21:02:01.803249+0000","last_peered":"2024-09-10T21:02:01.803249+0000","last_clean":"2024-09-10T21:02:01.803249+0000","last_became_active":"2024-09-10T21:02:00.805510+0000","last_became_peered":"2024-09-10T21:02:00.805510+0000","last_unstale":"2024-09-10T21:02:01.803249+0000","last_undegraded":"2024-09-10T21:02:01.803249+0000","last_fullsized":"2024-09-10T21:02:01.803249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:37.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802601+0000","last_change":"2024-09-10T21:02:00.807691+0000","last_active":"2024-09-10T21:02:01.802601+0000","last_peered":"2024-09-10T21:02:01.802601+0000","last_clean":"2024-09-10T21:02:01.802601+0000","last_became_active":"2024-09-10T21:02:00.807364+0000","last_became_peered":"2024-09-10T21:02:00.807364+0000","last_unstale":"2024-09-10T21:02:01.802601+0000","last_undegraded":"2024-09-10T21:02:01.802601+0000","last_fullsized":"2024-09-10T21:02:01.802601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:27.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796533+0000","last_change":"2024-09-10T21:02:00.809542+0000","last_active":"2024-09-10T21:02:01.796533+0000","last_peered":"2024-09-10T21:02:01.796533+0000","last_clean":"2024-09-10T21:02:01.796533+0000","last_became_active":"2024-09-10T21:02:00.809034+0000","last_became_peered":"2024-09-10T21:02:00.809034+0000","last_unstale":"2024-09-10T21:02:01.796533+0000","last_undegraded":"2024-09-10T21:02:01.796533+0000","last_fullsized":"2024-09-10T21:02:01.796533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:29.923532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796807+0000","last_change":"2024-09-10T21:02:00.809355+0000","last_active":"2024-09-10T21:02:01.796807+0000","last_peered":"2024-09-10T21:02:01.796807+0000","last_clean":"2024-09-10T21:02:01.796807+0000","last_became_active":"2024-09-10T21:02:00.808863+0000","last_became_peered":"2024-09-10T21:02:00.808863+0000","last_unstale":"2024-09-10T21:02:01.796807+0000","last_undegraded":"2024-09-10T21:02:01.796807+0000","last_fullsized":"2024-09-10T21:02:01.796807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:29.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.790553+0000","last_change":"2024-09-10T21:02:00.803756+0000","last_active":"2024-09-10T21:02:01.790553+0000","last_peered":"2024-09-10T21:02:01.790553+0000","last_clean":"2024-09-10T21:02:01.790553+0000","last_became_active":"2024-09-10T21:02:00.803406+0000","last_became_peered":"2024-09-10T21:02:00.803406+0000","last_unstale":"2024-09-10T21:02:01.790553+0000","last_undegraded":"2024-09-10T21:02:01.790553+0000","last_fullsized":"2024-09-10T21:02:01.790553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:53.437629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798836+0000","last_change":"2024-09-10T21:02:00.808845+0000","last_active":"2024-09-10T21:02:01.798836+0000","last_peered":"2024-09-10T21:02:01.798836+0000","last_clean":"2024-09-10T21:02:01.798836+0000","last_became_active":"2024-09-10T21:02:00.807759+0000","last_became_peered":"2024-09-10T21:02:00.807759+0000","last_unstale":"2024-09-10T21:02:01.798836+0000","last_undegraded":"2024-09-10T21:02:01.798836+0000","last_fullsized":"2024-09-10T21:02:01.798836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.809389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800503+0000","last_change":"2024-09-10T21:02:00.811199+0000","last_active":"2024-09-10T21:02:01.800503+0000","last_peered":"2024-09-10T21:02:01.800503+0000","last_clean":"2024-09-10T21:02:01.800503+0000","last_became_active":"2024-09-10T21:02:00.810692+0000","last_became_peered":"2024-09-10T21:02:00.810692+0000","last_unstale":"2024-09-10T21:02:01.800503+0000","last_undegraded":"2024-09-10T21:02:01.800503+0000","last_fullsized":"2024-09-10T21:02:01.800503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:59.557973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799691+0000","last_change":"2024-09-10T21:02:00.812686+0000","last_active":"2024-09-10T21:02:01.799691+0000","last_peered":"2024-09-10T21:02:01.799691+0000","last_clean":"2024-09-10T21:02:01.799691+0000","last_became_active":"2024-09-10T21:02:00.812363+0000","last_became_peered":"2024-09-10T21:02:00.812363+0000","last_unstale":"2024-09-10T21:02:01.799691+0000","last_undegraded":"2024-09-10T21:02:01.799691+0000","last_fullsized":"2024-09-10T21:02:01.799691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:37.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.791003+0000","last_change":"2024-09-10T21:02:00.812058+0000","last_active":"2024-09-10T21:02:01.791003+0000","last_peered":"2024-09-10T21:02:01.791003+0000","last_clean":"2024-09-10T21:02:01.791003+0000","last_became_active":"2024-09-10T21:02:00.811490+0000","last_became_peered":"2024-09-10T21:02:00.811490+0000","last_unstale":"2024-09-10T21:02:01.791003+0000","last_undegraded":"2024-09-10T21:02:01.791003+0000","last_fullsized":"2024-09-10T21:02:01.791003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:31.557485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800320+0000","last_change":"2024-09-10T21:02:00.809991+0000","last_active":"2024-09-10T21:02:01.800320+0000","last_peered":"2024-09-10T21:02:01.800320+0000","last_clean":"2024-09-10T21:02:01.800320+0000","last_became_active":"2024-09-10T21:02:00.809197+0000","last_became_peered":"2024-09-10T21:02:00.809197+0000","last_unstale":"2024-09-10T21:02:01.800320+0000","last_undegraded":"2024-09-10T21:02:01.800320+0000","last_fullsized":"2024-09-10T21:02:01.800320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796740+0000","last_change":"2024-09-10T21:02:00.805122+0000","last_active":"2024-09-10T21:02:01.796740+0000","last_peered":"2024-09-10T21:02:01.796740+0000","last_clean":"2024-09-10T21:02:01.796740+0000","last_became_active":"2024-09-10T21:02:00.804784+0000","last_became_peered":"2024-09-10T21:02:00.804784+0000","last_unstale":"2024-09-10T21:02:01.796740+0000","last_undegraded":"2024-09-10T21:02:01.796740+0000","last_fullsized":"2024-09-10T21:02:01.796740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:04.823870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798520+0000","last_change":"2024-09-10T21:02:00.807280+0000","last_active":"2024-09-10T21:02:01.798520+0000","last_peered":"2024-09-10T21:02:01.798520+0000","last_clean":"2024-09-10T21:02:01.798520+0000","last_became_active":"2024-09-10T21:02:00.804247+0000","last_became_peered":"2024-09-10T21:02:00.804247+0000","last_unstale":"2024-09-10T21:02:01.798520+0000","last_undegraded":"2024-09-10T21:02:01.798520+0000","last_fullsized":"2024-09-10T21:02:01.798520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:28.163960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802774+0000","last_change":"2024-09-10T21:02:00.805723+0000","last_active":"2024-09-10T21:02:01.802774+0000","last_peered":"2024-09-10T21:02:01.802774+0000","last_clean":"2024-09-10T21:02:01.802774+0000","last_became_active":"2024-09-10T21:02:00.805104+0000","last_became_peered":"2024-09-10T21:02:00.805104+0000","last_unstale":"2024-09-10T21:02:01.802774+0000","last_undegraded":"2024-09-10T21:02:01.802774+0000","last_fullsized":"2024-09-10T21:02:01.802774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.056884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802581+0000","last_change":"2024-09-10T21:02:00.810497+0000","last_active":"2024-09-10T21:02:01.802581+0000","last_peered":"2024-09-10T21:02:01.802581+0000","last_clean":"2024-09-10T21:02:01.802581+0000","last_became_active":"2024-09-10T21:02:00.809392+0000","last_became_peered":"2024-09-10T21:02:00.809392+0000","last_unstale":"2024-09-10T21:02:01.802581+0000","last_undegraded":"2024-09-10T21:02:01.802581+0000","last_fullsized":"2024-09-10T21:02:01.802581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:51.214771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798089+0000","last_change":"2024-09-10T21:02:00.813973+0000","last_active":"2024-09-10T21:02:01.798089+0000","last_peered":"2024-09-10T21:02:01.798089+0000","last_clean":"2024-09-10T21:02:01.798089+0000","last_became_active":"2024-09-10T21:02:00.813580+0000","last_became_peered":"2024-09-10T21:02:00.813580+0000","last_unstale":"2024-09-10T21:02:01.798089+0000","last_undegraded":"2024-09-10T21:02:01.798089+0000","last_fullsized":"2024-09-10T21:02:01.798089+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:48.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788416+0000","last_change":"2024-09-10T21:02:00.804844+0000","last_active":"2024-09-10T21:02:01.788416+0000","last_peered":"2024-09-10T21:02:01.788416+0000","last_clean":"2024-09-10T21:02:01.788416+0000","last_became_active":"2024-09-10T21:02:00.803866+0000","last_became_peered":"2024-09-10T21:02:00.803866+0000","last_unstale":"2024-09-10T21:02:01.788416+0000","last_undegraded":"2024-09-10T21:02:01.788416+0000","last_fullsized":"2024-09-10T21:02:01.788416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:05.231877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.789932+0000","last_change":"2024-09-10T21:02:00.801343+0000","last_active":"2024-09-10T21:02:01.789932+0000","last_peered":"2024-09-10T21:02:01.789932+0000","last_clean":"2024-09-10T21:02:01.789932+0000","last_became_active":"2024-09-10T21:02:00.801008+0000","last_became_peered":"2024-09-10T21:02:00.801008+0000","last_unstale":"2024-09-10T21:02:01.789932+0000","last_undegraded":"2024-09-10T21:02:01.789932+0000","last_fullsized":"2024-09-10T21:02:01.789932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:31.167925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.797891+0000","last_change":"2024-09-10T21:02:00.813026+0000","last_active":"2024-09-10T21:02:01.797891+0000","last_peered":"2024-09-10T21:02:01.797891+0000","last_clean":"2024-09-10T21:02:01.797891+0000","last_became_active":"2024-09-10T21:02:00.812074+0000","last_became_peered":"2024-09-10T21:02:00.812074+0000","last_unstale":"2024-09-10T21:02:01.797891+0000","last_undegraded":"2024-09-10T21:02:01.797891+0000","last_fullsized":"2024-09-10T21:02:01.797891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:04.866402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788036+0000","last_change":"2024-09-10T21:02:00.810502+0000","last_active":"2024-09-10T21:02:01.788036+0000","last_peered":"2024-09-10T21:02:01.788036+0000","last_clean":"2024-09-10T21:02:01.788036+0000","last_became_active":"2024-09-10T21:02:00.810209+0000","last_became_peered":"2024-09-10T21:02:00.810209+0000","last_unstale":"2024-09-10T21:02:01.788036+0000","last_undegraded":"2024-09-10T21:02:01.788036+0000","last_fullsized":"2024-09-10T21:02:01.788036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:51.311272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798556+0000","last_change":"2024-09-10T21:02:00.797119+0000","last_active":"2024-09-10T21:02:01.798556+0000","last_peered":"2024-09-10T21:02:01.798556+0000","last_clean":"2024-09-10T21:02:01.798556+0000","last_became_active":"2024-09-10T21:02:00.796788+0000","last_became_peered":"2024-09-10T21:02:00.796788+0000","last_unstale":"2024-09-10T21:02:01.798556+0000","last_undegraded":"2024-09-10T21:02:01.798556+0000","last_fullsized":"2024-09-10T21:02:01.798556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:07.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798899+0000","last_change":"2024-09-10T21:02:00.802284+0000","last_active":"2024-09-10T21:02:01.798899+0000","last_peered":"2024-09-10T21:02:01.798899+0000","last_clean":"2024-09-10T21:02:01.798899+0000","last_became_active":"2024-09-10T21:02:00.801964+0000","last_became_peered":"2024-09-10T21:02:00.801964+0000","last_unstale":"2024-09-10T21:02:01.798899+0000","last_undegraded":"2024-09-10T21:02:01.798899+0000","last_fullsized":"2024-09-10T21:02:01.798899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:45.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800645+0000","last_change":"2024-09-10T21:02:00.809196+0000","last_active":"2024-09-10T21:02:01.800645+0000","last_peered":"2024-09-10T21:02:01.800645+0000","last_clean":"2024-09-10T21:02:01.800645+0000","last_became_active":"2024-09-10T21:02:00.808903+0000","last_became_peered":"2024-09-10T21:02:00.808903+0000","last_unstale":"2024-09-10T21:02:01.800645+0000","last_undegraded":"2024-09-10T21:02:01.800645+0000","last_fullsized":"2024-09-10T21:02:01.800645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:01.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803568+0000","last_change":"2024-09-10T21:02:00.808195+0000","last_active":"2024-09-10T21:02:01.803568+0000","last_peered":"2024-09-10T21:02:01.803568+0000","last_clean":"2024-09-10T21:02:01.803568+0000","last_became_active":"2024-09-10T21:02:00.807670+0000","last_became_peered":"2024-09-10T21:02:00.807670+0000","last_unstale":"2024-09-10T21:02:01.803568+0000","last_undegraded":"2024-09-10T21:02:01.803568+0000","last_fullsized":"2024-09-10T21:02:01.803568+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:12.549344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802944+0000","last_change":"2024-09-10T21:02:00.811401+0000","last_active":"2024-09-10T21:02:01.802944+0000","last_peered":"2024-09-10T21:02:01.802944+0000","last_clean":"2024-09-10T21:02:01.802944+0000","last_became_active":"2024-09-10T21:02:00.811055+0000","last_became_peered":"2024-09-10T21:02:00.811055+0000","last_unstale":"2024-09-10T21:02:01.802944+0000","last_undegraded":"2024-09-10T21:02:01.802944+0000","last_fullsized":"2024-09-10T21:02:01.802944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:58.641873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798711+0000","last_change":"2024-09-10T21:02:00.813796+0000","last_active":"2024-09-10T21:02:01.798711+0000","last_peered":"2024-09-10T21:02:01.798711+0000","last_clean":"2024-09-10T21:02:01.798711+0000","last_became_active":"2024-09-10T21:02:00.813468+0000","last_became_peered":"2024-09-10T21:02:00.813468+0000","last_unstale":"2024-09-10T21:02:01.798711+0000","last_undegraded":"2024-09-10T21:02:01.798711+0000","last_fullsized":"2024-09-10T21:02:01.798711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:25.964890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788213+0000","last_change":"2024-09-10T21:02:00.811436+0000","last_active":"2024-09-10T21:02:01.788213+0000","last_peered":"2024-09-10T21:02:01.788213+0000","last_clean":"2024-09-10T21:02:01.788213+0000","last_became_active":"2024-09-10T21:02:00.810956+0000","last_became_peered":"2024-09-10T21:02:00.810956+0000","last_unstale":"2024-09-10T21:02:01.788213+0000","last_undegraded":"2024-09-10T21:02:01.788213+0000","last_fullsized":"2024-09-10T21:02:01.788213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.306832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798715+0000","last_change":"2024-09-10T21:02:00.799370+0000","last_active":"2024-09-10T21:02:01.798715+0000","last_peered":"2024-09-10T21:02:01.798715+0000","last_clean":"2024-09-10T21:02:01.798715+0000","last_became_active":"2024-09-10T21:02:00.799044+0000","last_became_peered":"2024-09-10T21:02:00.799044+0000","last_unstale":"2024-09-10T21:02:01.798715+0000","last_undegraded":"2024-09-10T21:02:01.798715+0000","last_fullsized":"2024-09-10T21:02:01.798715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:40.332088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803457+0000","last_change":"2024-09-10T21:02:00.799830+0000","last_active":"2024-09-10T21:02:01.803457+0000","last_peered":"2024-09-10T21:02:01.803457+0000","last_clean":"2024-09-10T21:02:01.803457+0000","last_became_active":"2024-09-10T21:02:00.799533+0000","last_became_peered":"2024-09-10T21:02:00.799533+0000","last_unstale":"2024-09-10T21:02:01.803457+0000","last_undegraded":"2024-09-10T21:02:01.803457+0000","last_fullsized":"2024-09-10T21:02:01.803457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:23.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799264+0000","last_change":"2024-09-10T21:02:00.812739+0000","last_active":"2024-09-10T21:02:01.799264+0000","last_peered":"2024-09-10T21:02:01.799264+0000","last_clean":"2024-09-10T21:02:01.799264+0000","last_became_active":"2024-09-10T21:02:00.812395+0000","last_became_peered":"2024-09-10T21:02:00.812395+0000","last_unstale":"2024-09-10T21:02:01.799264+0000","last_undegraded":"2024-09-10T21:02:01.799264+0000","last_fullsized":"2024-09-10T21:02:01.799264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:55.727155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.797481+0000","last_change":"2024-09-10T21:02:00.806291+0000","last_active":"2024-09-10T21:02:01.797481+0000","last_peered":"2024-09-10T21:02:01.797481+0000","last_clean":"2024-09-10T21:02:01.797481+0000","last_became_active":"2024-09-10T21:02:00.805943+0000","last_became_peered":"2024-09-10T21:02:00.805943+0000","last_unstale":"2024-09-10T21:02:01.797481+0000","last_undegraded":"2024-09-10T21:02:01.797481+0000","last_fullsized":"2024-09-10T21:02:01.797481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:21.354105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788016+0000","last_change":"2024-09-10T21:02:00.808774+0000","last_active":"2024-09-10T21:02:01.788016+0000","last_peered":"2024-09-10T21:02:01.788016+0000","last_clean":"2024-09-10T21:02:01.788016+0000","last_became_active":"2024-09-10T21:02:00.808027+0000","last_became_peered":"2024-09-10T21:02:00.808027+0000","last_unstale":"2024-09-10T21:02:01.788016+0000","last_undegraded":"2024-09-10T21:02:01.788016+0000","last_fullsized":"2024-09-10T21:02:01.788016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796627+0000","last_change":"2024-09-10T21:02:00.808349+0000","last_active":"2024-09-10T21:02:01.796627+0000","last_peered":"2024-09-10T21:02:01.796627+0000","last_clean":"2024-09-10T21:02:01.796627+0000","last_became_active":"2024-09-10T21:02:00.807839+0000","last_became_peered":"2024-09-10T21:02:00.807839+0000","last_unstale":"2024-09-10T21:02:01.796627+0000","last_undegraded":"2024-09-10T21:02:01.796627+0000","last_fullsized":"2024-09-10T21:02:01.796627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:53.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:20.887759+0000","last_change":"2024-09-10T21:01:58.789380+0000","last_active":"2024-09-10T21:02:20.887759+0000","last_peered":"2024-09-10T21:02:20.887759+0000","last_clean":"2024-09-10T21:02:20.887759+0000","last_became_active":"2024-09-10T21:01:58.789008+0000","last_became_peered":"2024-09-10T21:01:58.789008+0000","last_unstale":"2024-09-10T21:02:20.887759+0000","last_undegraded":"2024-09-10T21:02:20.887759+0000","last_fullsized":"2024-09-10T21:02:20.887759+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:01.488478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803608+0000","last_change":"2024-09-10T21:02:00.807068+0000","last_active":"2024-09-10T21:02:01.803608+0000","last_peered":"2024-09-10T21:02:01.803608+0000","last_clean":"2024-09-10T21:02:01.803608+0000","last_became_active":"2024-09-10T21:02:00.806764+0000","last_became_peered":"2024-09-10T21:02:00.806764+0000","last_unstale":"2024-09-10T21:02:01.803608+0000","last_undegraded":"2024-09-10T21:02:01.803608+0000","last_fullsized":"2024-09-10T21:02:01.803608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:03.307495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:10.860997+0000","last_change":"2024-09-10T21:01:58.787619+0000","last_active":"2024-09-10T21:02:10.860997+0000","last_peered":"2024-09-10T21:02:10.860997+0000","last_clean":"2024-09-10T21:02:10.860997+0000","last_became_active":"2024-09-10T21:01:58.787138+0000","last_became_peered":"2024-09-10T21:01:58.787138+0000","last_unstale":"2024-09-10T21:02:10.860997+0000","last_undegraded":"2024-09-10T21:02:10.860997+0000","last_fullsized":"2024-09-10T21:02:10.860997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:33.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:09.849049+0000","last_change":"2024-09-10T21:01:58.780115+0000","last_active":"2024-09-10T21:02:09.849049+0000","last_peered":"2024-09-10T21:02:09.849049+0000","last_clean":"2024-09-10T21:02:09.849049+0000","last_became_active":"2024-09-10T21:01:58.779757+0000","last_became_peered":"2024-09-10T21:01:58.779757+0000","last_unstale":"2024-09-10T21:02:09.849049+0000","last_undegraded":"2024-09-10T21:02:09.849049+0000","last_fullsized":"2024-09-10T21:02:09.849049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:29.337978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800485+0000","last_change":"2024-09-10T21:02:00.800925+0000","last_active":"2024-09-10T21:02:01.800485+0000","last_peered":"2024-09-10T21:02:01.800485+0000","last_clean":"2024-09-10T21:02:01.800485+0000","last_became_active":"2024-09-10T21:02:00.798567+0000","last_became_peered":"2024-09-10T21:02:00.798567+0000","last_unstale":"2024-09-10T21:02:01.800485+0000","last_undegraded":"2024-09-10T21:02:01.800485+0000","last_fullsized":"2024-09-10T21:02:01.800485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:01.873716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.868222+0000","last_change":"2024-09-10T21:01:58.786040+0000","last_active":"2024-09-10T21:02:11.868222+0000","last_peered":"2024-09-10T21:02:11.868222+0000","last_clean":"2024-09-10T21:02:11.868222+0000","last_became_active":"2024-09-10T21:01:58.785703+0000","last_became_peered":"2024-09-10T21:01:58.785703+0000","last_unstale":"2024-09-10T21:02:11.868222+0000","last_undegraded":"2024-09-10T21:02:11.868222+0000","last_fullsized":"2024-09-10T21:02:11.868222+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:39.741614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.789677+0000","last_change":"2024-09-10T21:02:00.798368+0000","last_active":"2024-09-10T21:02:01.789677+0000","last_peered":"2024-09-10T21:02:01.789677+0000","last_clean":"2024-09-10T21:02:01.789677+0000","last_became_active":"2024-09-10T21:02:00.798080+0000","last_became_peered":"2024-09-10T21:02:00.798080+0000","last_unstale":"2024-09-10T21:02:01.789677+0000","last_undegraded":"2024-09-10T21:02:01.789677+0000","last_fullsized":"2024-09-10T21:02:01.789677+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:31.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:21.888298+0000","last_change":"2024-09-10T21:01:58.788119+0000","last_active":"2024-09-10T21:02:21.888298+0000","last_peered":"2024-09-10T21:02:21.888298+0000","last_clean":"2024-09-10T21:02:21.888298+0000","last_became_active":"2024-09-10T21:01:58.787724+0000","last_became_peered":"2024-09-10T21:01:58.787724+0000","last_unstale":"2024-09-10T21:02:21.888298+0000","last_undegraded":"2024-09-10T21:02:21.888298+0000","last_fullsized":"2024-09-10T21:02:21.888298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:42.872858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800024+0000","last_change":"2024-09-10T21:02:00.804425+0000","last_active":"2024-09-10T21:02:01.800024+0000","last_peered":"2024-09-10T21:02:01.800024+0000","last_clean":"2024-09-10T21:02:01.800024+0000","last_became_active":"2024-09-10T21:02:00.804093+0000","last_became_peered":"2024-09-10T21:02:00.804093+0000","last_unstale":"2024-09-10T21:02:01.800024+0000","last_undegraded":"2024-09-10T21:02:01.800024+0000","last_fullsized":"2024-09-10T21:02:01.800024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:46.183326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867156+0000","last_change":"2024-09-10T21:01:58.791323+0000","last_active":"2024-09-10T21:02:11.867156+0000","last_peered":"2024-09-10T21:02:11.867156+0000","last_clean":"2024-09-10T21:02:11.867156+0000","last_became_active":"2024-09-10T21:01:58.790551+0000","last_became_peered":"2024-09-10T21:01:58.790551+0000","last_unstale":"2024-09-10T21:02:11.867156+0000","last_undegraded":"2024-09-10T21:02:11.867156+0000","last_fullsized":"2024-09-10T21:02:11.867156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:18.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803889+0000","last_change":"2024-09-10T21:02:00.804025+0000","last_active":"2024-09-10T21:02:01.803889+0000","last_peered":"2024-09-10T21:02:01.803889+0000","last_clean":"2024-09-10T21:02:01.803889+0000","last_became_active":"2024-09-10T21:02:00.803663+0000","last_became_peered":"2024-09-10T21:02:00.803663+0000","last_unstale":"2024-09-10T21:02:01.803889+0000","last_undegraded":"2024-09-10T21:02:01.803889+0000","last_fullsized":"2024-09-10T21:02:01.803889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:30.609020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:10.860671+0000","last_change":"2024-09-10T21:01:58.887067+0000","last_active":"2024-09-10T21:02:10.860671+0000","last_peered":"2024-09-10T21:02:10.860671+0000","last_clean":"2024-09-10T21:02:10.860671+0000","last_became_active":"2024-09-10T21:01:58.886611+0000","last_became_peered":"2024-09-10T21:01:58.886611+0000","last_unstale":"2024-09-10T21:02:10.860671+0000","last_undegraded":"2024-09-10T21:02:10.860671+0000","last_fullsized":"2024-09-10T21:02:10.860671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:06.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798417+0000","last_change":"2024-09-10T21:02:00.814120+0000","last_active":"2024-09-10T21:02:01.798417+0000","last_peered":"2024-09-10T21:02:01.798417+0000","last_clean":"2024-09-10T21:02:01.798417+0000","last_became_active":"2024-09-10T21:02:00.813769+0000","last_became_peered":"2024-09-10T21:02:00.813769+0000","last_unstale":"2024-09-10T21:02:01.798417+0000","last_undegraded":"2024-09-10T21:02:01.798417+0000","last_fullsized":"2024-09-10T21:02:01.798417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:10.601319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'24","reported_seq":69,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:21.889517+0000","last_change":"2024-09-10T21:01:58.790966+0000","last_active":"2024-09-10T21:02:21.889517+0000","last_peered":"2024-09-10T21:02:21.889517+0000","last_clean":"2024-09-10T21:02:21.889517+0000","last_became_active":"2024-09-10T21:01:58.790636+0000","last_became_peered":"2024-09-10T21:01:58.790636+0000","last_unstale":"2024-09-10T21:02:21.889517+0000","last_undegraded":"2024-09-10T21:02:21.889517+0000","last_fullsized":"2024-09-10T21:02:21.889517+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:06.209074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46800,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799643+0000","last_change":"2024-09-10T21:02:00.810610+0000","last_active":"2024-09-10T21:02:01.799643+0000","last_peered":"2024-09-10T21:02:01.799643+0000","last_clean":"2024-09-10T21:02:01.799643+0000","last_became_active":"2024-09-10T21:02:00.810224+0000","last_became_peered":"2024-09-10T21:02:00.810224+0000","last_unstale":"2024-09-10T21:02:01.799643+0000","last_undegraded":"2024-09-10T21:02:01.799643+0000","last_fullsized":"2024-09-10T21:02:01.799643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:32.785312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:12.872396+0000","last_change":"2024-09-10T21:01:58.786476+0000","last_active":"2024-09-10T21:02:12.872396+0000","last_peered":"2024-09-10T21:02:12.872396+0000","last_clean":"2024-09-10T21:02:12.872396+0000","last_became_active":"2024-09-10T21:01:58.786153+0000","last_became_peered":"2024-09-10T21:01:58.786153+0000","last_unstale":"2024-09-10T21:02:12.872396+0000","last_undegraded":"2024-09-10T21:02:12.872396+0000","last_fullsized":"2024-09-10T21:02:12.872396+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:00.118591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798264+0000","last_change":"2024-09-10T21:02:00.796138+0000","last_active":"2024-09-10T21:02:01.798264+0000","last_peered":"2024-09-10T21:02:01.798264+0000","last_clean":"2024-09-10T21:02:01.798264+0000","last_became_active":"2024-09-10T21:02:00.795847+0000","last_became_peered":"2024-09-10T21:02:00.795847+0000","last_unstale":"2024-09-10T21:02:01.798264+0000","last_undegraded":"2024-09-10T21:02:01.798264+0000","last_fullsized":"2024-09-10T21:02:01.798264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:29.462134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867188+0000","last_change":"2024-09-10T21:01:58.789314+0000","last_active":"2024-09-10T21:02:11.867188+0000","last_peered":"2024-09-10T21:02:11.867188+0000","last_clean":"2024-09-10T21:02:11.867188+0000","last_became_active":"2024-09-10T21:01:58.788594+0000","last_became_peered":"2024-09-10T21:01:58.788594+0000","last_unstale":"2024-09-10T21:02:11.867188+0000","last_undegraded":"2024-09-10T21:02:11.867188+0000","last_fullsized":"2024-09-10T21:02:11.867188+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:44.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798980+0000","last_change":"2024-09-10T21:02:00.799828+0000","last_active":"2024-09-10T21:02:01.798980+0000","last_peered":"2024-09-10T21:02:01.798980+0000","last_clean":"2024-09-10T21:02:01.798980+0000","last_became_active":"2024-09-10T21:02:00.799465+0000","last_became_peered":"2024-09-10T21:02:00.799465+0000","last_unstale":"2024-09-10T21:02:01.798980+0000","last_undegraded":"2024-09-10T21:02:01.798980+0000","last_fullsized":"2024-09-10T21:02:01.798980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:56.794326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:21.888452+0000","last_change":"2024-09-10T21:01:58.788730+0000","last_active":"2024-09-10T21:02:21.888452+0000","last_peered":"2024-09-10T21:02:21.888452+0000","last_clean":"2024-09-10T21:02:21.888452+0000","last_became_active":"2024-09-10T21:01:58.788317+0000","last_became_peered":"2024-09-10T21:01:58.788317+0000","last_unstale":"2024-09-10T21:02:21.888452+0000","last_undegraded":"2024-09-10T21:02:21.888452+0000","last_fullsized":"2024-09-10T21:02:21.888452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:52.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799475+0000","last_change":"2024-09-10T21:02:00.807104+0000","last_active":"2024-09-10T21:02:01.799475+0000","last_peered":"2024-09-10T21:02:01.799475+0000","last_clean":"2024-09-10T21:02:01.799475+0000","last_became_active":"2024-09-10T21:02:00.806815+0000","last_became_peered":"2024-09-10T21:02:00.806815+0000","last_unstale":"2024-09-10T21:02:01.799475+0000","last_undegraded":"2024-09-10T21:02:01.799475+0000","last_fullsized":"2024-09-10T21:02:01.799475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:24.747662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867889+0000","last_change":"2024-09-10T21:01:58.886877+0000","last_active":"2024-09-10T21:02:11.867889+0000","last_peered":"2024-09-10T21:02:11.867889+0000","last_clean":"2024-09-10T21:02:11.867889+0000","last_became_active":"2024-09-10T21:01:58.886437+0000","last_became_peered":"2024-09-10T21:01:58.886437+0000","last_unstale":"2024-09-10T21:02:11.867889+0000","last_undegraded":"2024-09-10T21:02:11.867889+0000","last_fullsized":"2024-09-10T21:02:11.867889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:34.786880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803016+0000","last_change":"2024-09-10T21:02:00.806796+0000","last_active":"2024-09-10T21:02:01.803016+0000","last_peered":"2024-09-10T21:02:01.803016+0000","last_clean":"2024-09-10T21:02:01.803016+0000","last_became_active":"2024-09-10T21:02:00.805802+0000","last_became_peered":"2024-09-10T21:02:00.805802+0000","last_unstale":"2024-09-10T21:02:01.803016+0000","last_undegraded":"2024-09-10T21:02:01.803016+0000","last_fullsized":"2024-09-10T21:02:01.803016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.962870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.845176+0000","last_change":"2024-09-10T21:01:58.781088+0000","last_active":"2024-09-10T21:02:11.845176+0000","last_peered":"2024-09-10T21:02:11.845176+0000","last_clean":"2024-09-10T21:02:11.845176+0000","last_became_active":"2024-09-10T21:01:58.780731+0000","last_became_peered":"2024-09-10T21:01:58.780731+0000","last_unstale":"2024-09-10T21:02:11.845176+0000","last_undegraded":"2024-09-10T21:02:11.845176+0000","last_fullsized":"2024-09-10T21:02:11.845176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:50.636077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788274+0000","last_change":"2024-09-10T21:02:00.806551+0000","last_active":"2024-09-10T21:02:01.788274+0000","last_peered":"2024-09-10T21:02:01.788274+0000","last_clean":"2024-09-10T21:02:01.788274+0000","last_became_active":"2024-09-10T21:02:00.806134+0000","last_became_peered":"2024-09-10T21:02:00.806134+0000","last_unstale":"2024-09-10T21:02:01.788274+0000","last_undegraded":"2024-09-10T21:02:01.788274+0000","last_fullsized":"2024-09-10T21:02:01.788274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.574372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.866161+0000","last_change":"2024-09-10T21:01:58.780294+0000","last_active":"2024-09-10T21:02:11.866161+0000","last_peered":"2024-09-10T21:02:11.866161+0000","last_clean":"2024-09-10T21:02:11.866161+0000","last_became_active":"2024-09-10T21:01:58.779767+0000","last_became_peered":"2024-09-10T21:01:58.779767+0000","last_unstale":"2024-09-10T21:02:11.866161+0000","last_undegraded":"2024-09-10T21:02:11.866161+0000","last_fullsized":"2024-09-10T21:02:11.866161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:21.734159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799903+0000","last_change":"2024-09-10T21:02:00.809393+0000","last_active":"2024-09-10T21:02:01.799903+0000","last_peered":"2024-09-10T21:02:01.799903+0000","last_clean":"2024-09-10T21:02:01.799903+0000","last_became_active":"2024-09-10T21:02:00.809012+0000","last_became_peered":"2024-09-10T21:02:00.809012+0000","last_unstale":"2024-09-10T21:02:01.799903+0000","last_undegraded":"2024-09-10T21:02:01.799903+0000","last_fullsized":"2024-09-10T21:02:01.799903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:03.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867304+0000","last_change":"2024-09-10T21:01:58.786507+0000","last_active":"2024-09-10T21:02:11.867304+0000","last_peered":"2024-09-10T21:02:11.867304+0000","last_clean":"2024-09-10T21:02:11.867304+0000","last_became_active":"2024-09-10T21:01:58.786115+0000","last_became_peered":"2024-09-10T21:01:58.786115+0000","last_unstale":"2024-09-10T21:02:11.867304+0000","last_undegraded":"2024-09-10T21:02:11.867304+0000","last_fullsized":"2024-09-10T21:02:11.867304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.985353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799601+0000","last_change":"2024-09-10T21:02:00.805050+0000","last_active":"2024-09-10T21:02:01.799601+0000","last_peered":"2024-09-10T21:02:01.799601+0000","last_clean":"2024-09-10T21:02:01.799601+0000","last_became_active":"2024-09-10T21:02:00.804706+0000","last_became_peered":"2024-09-10T21:02:00.804706+0000","last_unstale":"2024-09-10T21:02:01.799601+0000","last_undegraded":"2024-09-10T21:02:01.799601+0000","last_fullsized":"2024-09-10T21:02:01.799601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:15.463986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:22.890050+0000","last_change":"2024-09-10T21:01:59.114525+0000","last_active":"2024-09-10T21:02:22.890050+0000","last_peered":"2024-09-10T21:02:22.890050+0000","last_clean":"2024-09-10T21:02:22.890050+0000","last_became_active":"2024-09-10T21:01:59.114076+0000","last_became_peered":"2024-09-10T21:01:59.114076+0000","last_unstale":"2024-09-10T21:02:22.890050+0000","last_undegraded":"2024-09-10T21:02:22.890050+0000","last_fullsized":"2024-09-10T21:02:22.890050+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:53.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800841+0000","last_change":"2024-09-10T21:02:00.799803+0000","last_active":"2024-09-10T21:02:01.800841+0000","last_peered":"2024-09-10T21:02:01.800841+0000","last_clean":"2024-09-10T21:02:01.800841+0000","last_became_active":"2024-09-10T21:02:00.799459+0000","last_became_peered":"2024-09-10T21:02:00.799459+0000","last_unstale":"2024-09-10T21:02:01.800841+0000","last_undegraded":"2024-09-10T21:02:01.800841+0000","last_fullsized":"2024-09-10T21:02:01.800841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:57.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799946+0000","last_change":"2024-09-10T21:00:48.381903+0000","last_active":"2024-09-10T21:02:01.799946+0000","last_peered":"2024-09-10T21:02:01.799946+0000","last_clean":"2024-09-10T21:02:01.799946+0000","last_became_active":"2024-09-10T21:00:48.376874+0000","last_became_peered":"2024-09-10T21:00:48.376874+0000","last_unstale":"2024-09-10T21:02:01.799946+0000","last_undegraded":"2024-09-10T21:02:01.799946+0000","last_fullsized":"2024-09-10T21:02:01.799946+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_clean_scrub_stamp":"2024-09-10T20:57:24.516525+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:35.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:12.872339+0000","last_change":"2024-09-10T21:01:58.788005+0000","last_active":"2024-09-10T21:02:12.872339+0000","last_peered":"2024-09-10T21:02:12.872339+0000","last_clean":"2024-09-10T21:02:12.872339+0000","last_became_active":"2024-09-10T21:01:58.786471+0000","last_became_peered":"2024-09-10T21:01:58.786471+0000","last_unstale":"2024-09-10T21:02:12.872339+0000","last_undegraded":"2024-09-10T21:02:12.872339+0000","last_fullsized":"2024-09-10T21:02:12.872339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:15.622814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.801307+0000","last_change":"2024-09-10T21:02:00.802573+0000","last_active":"2024-09-10T21:02:01.801307+0000","last_peered":"2024-09-10T21:02:01.801307+0000","last_clean":"2024-09-10T21:02:01.801307+0000","last_became_active":"2024-09-10T21:02:00.802263+0000","last_became_peered":"2024-09-10T21:02:00.802263+0000","last_unstale":"2024-09-10T21:02:01.801307+0000","last_undegraded":"2024-09-10T21:02:01.801307+0000","last_fullsized":"2024-09-10T21:02:01.801307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:54.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:09.850177+0000","last_change":"2024-09-10T21:01:58.785700+0000","last_active":"2024-09-10T21:02:09.850177+0000","last_peered":"2024-09-10T21:02:09.850177+0000","last_clean":"2024-09-10T21:02:09.850177+0000","last_became_active":"2024-09-10T21:01:58.785346+0000","last_became_peered":"2024-09-10T21:01:58.785346+0000","last_unstale":"2024-09-10T21:02:09.850177+0000","last_undegraded":"2024-09-10T21:02:09.850177+0000","last_fullsized":"2024-09-10T21:02:09.850177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:39.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798022+0000","last_change":"2024-09-10T21:02:00.808045+0000","last_active":"2024-09-10T21:02:01.798022+0000","last_peered":"2024-09-10T21:02:01.798022+0000","last_clean":"2024-09-10T21:02:01.798022+0000","last_became_active":"2024-09-10T21:02:00.807757+0000","last_became_peered":"2024-09-10T21:02:00.807757+0000","last_unstale":"2024-09-10T21:02:01.798022+0000","last_undegraded":"2024-09-10T21:02:01.798022+0000","last_fullsized":"2024-09-10T21:02:01.798022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:16.198290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:22.889163+0000","last_change":"2024-09-10T21:01:58.884939+0000","last_active":"2024-09-10T21:02:22.889163+0000","last_peered":"2024-09-10T21:02:22.889163+0000","last_clean":"2024-09-10T21:02:22.889163+0000","last_became_active":"2024-09-10T21:01:58.884517+0000","last_became_peered":"2024-09-10T21:01:58.884517+0000","last_unstale":"2024-09-10T21:02:22.889163+0000","last_undegraded":"2024-09-10T21:02:22.889163+0000","last_fullsized":"2024-09-10T21:02:22.889163+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.579662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803370+0000","last_change":"2024-09-10T21:02:00.813021+0000","last_active":"2024-09-10T21:02:01.803370+0000","last_peered":"2024-09-10T21:02:01.803370+0000","last_clean":"2024-09-10T21:02:01.803370+0000","last_became_active":"2024-09-10T21:02:00.812622+0000","last_became_peered":"2024-09-10T21:02:00.812622+0000","last_unstale":"2024-09-10T21:02:01.803370+0000","last_undegraded":"2024-09-10T21:02:01.803370+0000","last_fullsized":"2024-09-10T21:02:01.803370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:48.348100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:23.890807+0000","last_change":"2024-09-10T21:01:58.792187+0000","last_active":"2024-09-10T21:02:23.890807+0000","last_peered":"2024-09-10T21:02:23.890807+0000","last_clean":"2024-09-10T21:02:23.890807+0000","last_became_active":"2024-09-10T21:01:58.791806+0000","last_became_peered":"2024-09-10T21:01:58.791806+0000","last_unstale":"2024-09-10T21:02:23.890807+0000","last_undegraded":"2024-09-10T21:02:23.890807+0000","last_fullsized":"2024-09-10T21:02:23.890807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:48.493396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.790867+0000","last_change":"2024-09-10T21:02:00.804073+0000","last_active":"2024-09-10T21:02:01.790867+0000","last_peered":"2024-09-10T21:02:01.790867+0000","last_clean":"2024-09-10T21:02:01.790867+0000","last_became_active":"2024-09-10T21:02:00.803760+0000","last_became_peered":"2024-09-10T21:02:00.803760+0000","last_unstale":"2024-09-10T21:02:01.790867+0000","last_undegraded":"2024-09-10T21:02:01.790867+0000","last_fullsized":"2024-09-10T21:02:01.790867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:01.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:02.635141+0000","last_change":"2024-09-10T21:01:58.885088+0000","last_active":"2024-09-10T21:02:02.635141+0000","last_peered":"2024-09-10T21:02:02.635141+0000","last_clean":"2024-09-10T21:02:02.635141+0000","last_became_active":"2024-09-10T21:01:58.884729+0000","last_became_peered":"2024-09-10T21:01:58.884729+0000","last_unstale":"2024-09-10T21:02:02.635141+0000","last_undegraded":"2024-09-10T21:02:02.635141+0000","last_fullsized":"2024-09-10T21:02:02.635141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:37.451629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:02.303367+0000","last_change":"2024-09-10T21:02:00.810329+0000","last_active":"2024-09-10T21:02:02.303367+0000","last_peered":"2024-09-10T21:02:02.303367+0000","last_clean":"2024-09-10T21:02:02.303367+0000","last_became_active":"2024-09-10T21:02:00.809985+0000","last_became_peered":"2024-09-10T21:02:00.809985+0000","last_unstale":"2024-09-10T21:02:02.303367+0000","last_undegraded":"2024-09-10T21:02:02.303367+0000","last_fullsized":"2024-09-10T21:02:02.303367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:39.631969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867915+0000","last_change":"2024-09-10T21:01:58.884987+0000","last_active":"2024-09-10T21:02:11.867915+0000","last_peered":"2024-09-10T21:02:11.867915+0000","last_clean":"2024-09-10T21:02:11.867915+0000","last_became_active":"2024-09-10T21:01:58.884648+0000","last_became_peered":"2024-09-10T21:01:58.884648+0000","last_unstale":"2024-09-10T21:02:11.867915+0000","last_undegraded":"2024-09-10T21:02:11.867915+0000","last_fullsized":"2024-09-10T21:02:11.867915+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:35.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798746+0000","last_change":"2024-09-10T21:02:00.804961+0000","last_active":"2024-09-10T21:02:01.798746+0000","last_peered":"2024-09-10T21:02:01.798746+0000","last_clean":"2024-09-10T21:02:01.798746+0000","last_became_active":"2024-09-10T21:02:00.804147+0000","last_became_peered":"2024-09-10T21:02:00.804147+0000","last_unstale":"2024-09-10T21:02:01.798746+0000","last_undegraded":"2024-09-10T21:02:01.798746+0000","last_fullsized":"2024-09-10T21:02:01.798746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:15.801815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'11","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:19.794040+0000","last_change":"2024-09-10T21:01:58.781292+0000","last_active":"2024-09-10T21:02:19.794040+0000","last_peered":"2024-09-10T21:02:19.794040+0000","last_clean":"2024-09-10T21:02:19.794040+0000","last_became_active":"2024-09-10T21:01:58.780995+0000","last_became_peered":"2024-09-10T21:01:58.780995+0000","last_unstale":"2024-09-10T21:02:19.794040+0000","last_undegraded":"2024-09-10T21:02:19.794040+0000","last_fullsized":"2024-09-10T21:02:19.794040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:56.914723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803222+0000","last_change":"2024-09-10T21:02:00.805878+0000","last_active":"2024-09-10T21:02:01.803222+0000","last_peered":"2024-09-10T21:02:01.803222+0000","last_clean":"2024-09-10T21:02:01.803222+0000","last_became_active":"2024-09-10T21:02:00.804565+0000","last_became_peered":"2024-09-10T21:02:00.804565+0000","last_unstale":"2024-09-10T21:02:01.803222+0000","last_undegraded":"2024-09-10T21:02:01.803222+0000","last_fullsized":"2024-09-10T21:02:01.803222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:24.359643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.868277+0000","last_change":"2024-09-10T21:01:59.114862+0000","last_active":"2024-09-10T21:02:11.868277+0000","last_peered":"2024-09-10T21:02:11.868277+0000","last_clean":"2024-09-10T21:02:11.868277+0000","last_became_active":"2024-09-10T21:01:59.114411+0000","last_became_peered":"2024-09-10T21:01:59.114411+0000","last_unstale":"2024-09-10T21:02:11.868277+0000","last_undegraded":"2024-09-10T21:02:11.868277+0000","last_fullsized":"2024-09-10T21:02:11.868277+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.645829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803074+0000","last_change":"2024-09-10T21:02:00.799230+0000","last_active":"2024-09-10T21:02:01.803074+0000","last_peered":"2024-09-10T21:02:01.803074+0000","last_clean":"2024-09-10T21:02:01.803074+0000","last_became_active":"2024-09-10T21:02:00.798992+0000","last_became_peered":"2024-09-10T21:02:00.798992+0000","last_unstale":"2024-09-10T21:02:01.803074+0000","last_undegraded":"2024-09-10T21:02:01.803074+0000","last_fullsized":"2024-09-10T21:02:01.803074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:40.322066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:08.842955+0000","last_change":"2024-09-10T21:01:58.887042+0000","last_active":"2024-09-10T21:02:08.842955+0000","last_peered":"2024-09-10T21:02:08.842955+0000","last_clean":"2024-09-10T21:02:08.842955+0000","last_became_active":"2024-09-10T21:01:58.886710+0000","last_became_peered":"2024-09-10T21:01:58.886710+0000","last_unstale":"2024-09-10T21:02:08.842955+0000","last_undegraded":"2024-09-10T21:02:08.842955+0000","last_fullsized":"2024-09-10T21:02:08.842955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:42.672899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796506+0000","last_change":"2024-09-10T21:02:00.809763+0000","last_active":"2024-09-10T21:02:01.796506+0000","last_peered":"2024-09-10T21:02:01.796506+0000","last_clean":"2024-09-10T21:02:01.796506+0000","last_became_active":"2024-09-10T21:02:00.808955+0000","last_became_peered":"2024-09-10T21:02:00.808955+0000","last_unstale":"2024-09-10T21:02:01.796506+0000","last_undegraded":"2024-09-10T21:02:01.796506+0000","last_fullsized":"2024-09-10T21:02:01.796506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:22.622852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.865716+0000","last_change":"2024-09-10T21:01:58.786312+0000","last_active":"2024-09-10T21:02:11.865716+0000","last_peered":"2024-09-10T21:02:11.865716+0000","last_clean":"2024-09-10T21:02:11.865716+0000","last_became_active":"2024-09-10T21:01:58.785894+0000","last_became_peered":"2024-09-10T21:01:58.785894+0000","last_unstale":"2024-09-10T21:02:11.865716+0000","last_undegraded":"2024-09-10T21:02:11.865716+0000","last_fullsized":"2024-09-10T21:02:11.865716+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:11.682904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788285+0000","last_change":"2024-09-10T21:02:00.808374+0000","last_active":"2024-09-10T21:02:01.788285+0000","last_peered":"2024-09-10T21:02:01.788285+0000","last_clean":"2024-09-10T21:02:01.788285+0000","last_became_active":"2024-09-10T21:02:00.807565+0000","last_became_peered":"2024-09-10T21:02:00.807565+0000","last_unstale":"2024-09-10T21:02:01.788285+0000","last_undegraded":"2024-09-10T21:02:01.788285+0000","last_fullsized":"2024-09-10T21:02:01.788285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:24.227480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:10.861693+0000","last_change":"2024-09-10T21:01:58.782375+0000","last_active":"2024-09-10T21:02:10.861693+0000","last_peered":"2024-09-10T21:02:10.861693+0000","last_clean":"2024-09-10T21:02:10.861693+0000","last_became_active":"2024-09-10T21:01:58.781964+0000","last_became_peered":"2024-09-10T21:01:58.781964+0000","last_unstale":"2024-09-10T21:02:10.861693+0000","last_undegraded":"2024-09-10T21:02:10.861693+0000","last_fullsized":"2024-09-10T21:02:10.861693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.460126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802972+0000","last_change":"2024-09-10T21:02:00.806612+0000","last_active":"2024-09-10T21:02:01.802972+0000","last_peered":"2024-09-10T21:02:01.802972+0000","last_clean":"2024-09-10T21:02:01.802972+0000","last_became_active":"2024-09-10T21:02:00.803470+0000","last_became_peered":"2024-09-10T21:02:00.803470+0000","last_unstale":"2024-09-10T21:02:01.802972+0000","last_undegraded":"2024-09-10T21:02:01.802972+0000","last_fullsized":"2024-09-10T21:02:01.802972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:08.350931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:08.843968+0000","last_change":"2024-09-10T21:01:58.788987+0000","last_active":"2024-09-10T21:02:08.843968+0000","last_peered":"2024-09-10T21:02:08.843968+0000","last_clean":"2024-09-10T21:02:08.843968+0000","last_became_active":"2024-09-10T21:01:58.788605+0000","last_became_peered":"2024-09-10T21:01:58.788605+0000","last_unstale":"2024-09-10T21:02:08.843968+0000","last_undegraded":"2024-09-10T21:02:08.843968+0000","last_fullsized":"2024-09-10T21:02:08.843968+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:45.798350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798367+0000","last_change":"2024-09-10T21:02:00.804200+0000","last_active":"2024-09-10T21:02:01.798367+0000","last_peered":"2024-09-10T21:02:01.798367+0000","last_clean":"2024-09-10T21:02:01.798367+0000","last_became_active":"2024-09-10T21:02:00.803875+0000","last_became_peered":"2024-09-10T21:02:00.803875+0000","last_unstale":"2024-09-10T21:02:01.798367+0000","last_undegraded":"2024-09-10T21:02:01.798367+0000","last_fullsized":"2024-09-10T21:02:01.798367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:15.930163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803354+0000","last_change":"2024-09-10T21:02:00.814655+0000","last_active":"2024-09-10T21:02:01.803354+0000","last_peered":"2024-09-10T21:02:01.803354+0000","last_clean":"2024-09-10T21:02:01.803354+0000","last_became_active":"2024-09-10T21:02:00.813996+0000","last_became_peered":"2024-09-10T21:02:00.813996+0000","last_unstale":"2024-09-10T21:02:01.803354+0000","last_undegraded":"2024-09-10T21:02:01.803354+0000","last_fullsized":"2024-09-10T21:02:01.803354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:59.860942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.868895+0000","last_change":"2024-09-10T21:01:58.784834+0000","last_active":"2024-09-10T21:02:11.868895+0000","last_peered":"2024-09-10T21:02:11.868895+0000","last_clean":"2024-09-10T21:02:11.868895+0000","last_became_active":"2024-09-10T21:01:58.784464+0000","last_became_peered":"2024-09-10T21:01:58.784464+0000","last_unstale":"2024-09-10T21:02:11.868895+0000","last_undegraded":"2024-09-10T21:02:11.868895+0000","last_fullsized":"2024-09-10T21:02:11.868895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:09.864824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798208+0000","last_change":"2024-09-10T21:02:00.805677+0000","last_active":"2024-09-10T21:02:01.798208+0000","last_peered":"2024-09-10T21:02:01.798208+0000","last_clean":"2024-09-10T21:02:01.798208+0000","last_became_active":"2024-09-10T21:02:00.805341+0000","last_became_peered":"2024-09-10T21:02:00.805341+0000","last_unstale":"2024-09-10T21:02:01.798208+0000","last_undegraded":"2024-09-10T21:02:01.798208+0000","last_fullsized":"2024-09-10T21:02:01.798208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.236954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:12.872819+0000","last_change":"2024-09-10T21:01:58.786087+0000","last_active":"2024-09-10T21:02:12.872819+0000","last_peered":"2024-09-10T21:02:12.872819+0000","last_clean":"2024-09-10T21:02:12.872819+0000","last_became_active":"2024-09-10T21:01:58.785661+0000","last_became_peered":"2024-09-10T21:01:58.785661+0000","last_unstale":"2024-09-10T21:02:12.872819+0000","last_undegraded":"2024-09-10T21:02:12.872819+0000","last_fullsized":"2024-09-10T21:02:12.872819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:41.045302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803772+0000","last_change":"2024-09-10T21:02:00.801514+0000","last_active":"2024-09-10T21:02:01.803772+0000","last_peered":"2024-09-10T21:02:01.803772+0000","last_clean":"2024-09-10T21:02:01.803772+0000","last_became_active":"2024-09-10T21:02:00.801221+0000","last_became_peered":"2024-09-10T21:02:00.801221+0000","last_unstale":"2024-09-10T21:02:01.803772+0000","last_undegraded":"2024-09-10T21:02:01.803772+0000","last_fullsized":"2024-09-10T21:02:01.803772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:35.564890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:02.552335+0000","last_change":"2024-09-10T21:01:58.884876+0000","last_active":"2024-09-10T21:02:02.552335+0000","last_peered":"2024-09-10T21:02:02.552335+0000","last_clean":"2024-09-10T21:02:02.552335+0000","last_became_active":"2024-09-10T21:01:58.884530+0000","last_became_peered":"2024-09-10T21:01:58.884530+0000","last_unstale":"2024-09-10T21:02:02.552335+0000","last_undegraded":"2024-09-10T21:02:02.552335+0000","last_fullsized":"2024-09-10T21:02:02.552335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.851017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.801716+0000","last_change":"2024-09-10T21:02:00.811148+0000","last_active":"2024-09-10T21:02:01.801716+0000","last_peered":"2024-09-10T21:02:01.801716+0000","last_clean":"2024-09-10T21:02:01.801716+0000","last_became_active":"2024-09-10T21:02:00.809265+0000","last_became_peered":"2024-09-10T21:02:00.809265+0000","last_unstale":"2024-09-10T21:02:01.801716+0000","last_undegraded":"2024-09-10T21:02:01.801716+0000","last_fullsized":"2024-09-10T21:02:01.801716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:25.269876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:20.886576+0000","last_change":"2024-09-10T21:01:58.786917+0000","last_active":"2024-09-10T21:02:20.886576+0000","last_peered":"2024-09-10T21:02:20.886576+0000","last_clean":"2024-09-10T21:02:20.886576+0000","last_became_active":"2024-09-10T21:01:58.786595+0000","last_became_peered":"2024-09-10T21:01:58.786595+0000","last_unstale":"2024-09-10T21:02:20.886576+0000","last_undegraded":"2024-09-10T21:02:20.886576+0000","last_fullsized":"2024-09-10T21:02:20.886576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:30.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54786,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":148,"num_read_kb":134,"num_write":232,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":164358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":390229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1028058,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103195,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":389768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299423,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":390146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495652,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":983446,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":389876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855407,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":982342,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":389737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215161,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":389310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444095,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":438200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":437589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":391215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":49554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:02:27.169 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T21:02:27.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:27 smithi033 ceph-mon[30860]: pgmap v298: 97 pgs: 97 active+clean; 630 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 17 op/s 2024-09-10T21:02:27.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:27 smithi033 ceph-mon[30860]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:02:27.433 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:27.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:27 smithi104 ceph-mon[35325]: pgmap v298: 97 pgs: 97 active+clean; 630 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 17 op/s 2024-09-10T21:02:27.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:27 smithi104 ceph-mon[35325]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:02:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:27 smithi174 ceph-mon[35635]: pgmap v298: 97 pgs: 97 active+clean; 630 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.1 KiB/s wr, 17 op/s 2024-09-10T21:02:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:27 smithi174 ceph-mon[35635]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:02:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 -- 172.21.15.33:0/2029790058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf01087d0 msgr2=0x7f0cf0108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 --2- 172.21.15.33:0/2029790058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf01087d0 0x7f0cf0108c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8009900 tx=0x7f0cd802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 -- 172.21.15.33:0/2029790058 shutdown_connections 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 --2- 172.21.15.33:0/2029790058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf0109150 0x7f0cf010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 --2- 172.21.15.33:0/2029790058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf01087d0 0x7f0cf0108c10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 --2- 172.21.15.33:0/2029790058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf0102dc0 0x7f0cf01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 -- 172.21.15.33:0/2029790058 >> 172.21.15.33:0/2029790058 conn(0x7f0cf00fe830 msgr2=0x7f0cf0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 -- 172.21.15.33:0/2029790058 shutdown_connections 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.891+0000 7f0cf5334640 1 -- 172.21.15.33:0/2029790058 wait complete. 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.892+0000 7f0cf5334640 1 Processor -- start 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.892+0000 7f0cf5334640 1 -- start start 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.892+0000 7f0cf5334640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf0102dc0 0x7f0cf0199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cf5334640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf01087d0 0x7f0cf01996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:27.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cf5334640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf0109150 0x7f0cf019ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cf5334640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0112e10 con 0x7f0cf0109150 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cf5334640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0112c90 con 0x7f0cf0102dc0 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cf5334640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0cf0112f90 con 0x7f0cf01087d0 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf0109150 0x7f0cf019ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf0109150 0x7f0cf019ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56046/0 (socket says 172.21.15.33:56046) 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 -- 172.21.15.33:0/1018181982 learned_addr learned my addr 172.21.15.33:0/1018181982 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0ce67fc640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf01087d0 0x7f0cf01996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0ceeffd640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf0102dc0 0x7f0cf0199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 -- 172.21.15.33:0/1018181982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf01087d0 msgr2=0x7f0cf01996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf01087d0 0x7f0cf01996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 -- 172.21.15.33:0/1018181982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf0102dc0 msgr2=0x7f0cf0199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf0102dc0 0x7f0cf0199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:27.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 -- 172.21.15.33:0/1018181982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cf019f230 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0ceeffd640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf0102dc0 0x7f0cf0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cef7fe640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf0109150 0x7f0cf019ea60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800cce0 tx=0x7f0ce800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cecff9640 1 -- 172.21.15.33:0/1018181982 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800f450 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cecff9640 1 -- 172.21.15.33:0/1018181982 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ce800faa0 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cf019f520 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cecff9640 1 -- 172.21.15.33:0/1018181982 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8018b70 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.893+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cf01a5be0 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.895+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cf0104070 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.896+0000 7f0cecff9640 1 -- 172.21.15.33:0/1018181982 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f0ce800f5f0 con 0x7f0cf0109150 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.897+0000 7f0cecff9640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0cdc0b5370 0x7f0cdc0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.897+0000 7f0ceeffd640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0cdc0b5370 0x7f0cdc0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:27.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.897+0000 7f0cecff9640 1 -- 172.21.15.33:0/1018181982 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f0ce80e63d0 con 0x7f0cf0109150 2024-09-10T21:02:27.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.897+0000 7f0ceeffd640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0cdc0b5370 0x7f0cdc0b7830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000baa0 tx=0x7f0ce0005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:27.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:27.901+0000 7f0cecff9640 1 -- 172.21.15.33:0/1018181982 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0ce8014030 con 0x7f0cf0109150 2024-09-10T21:02:28.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.082+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0cf0105d90 con 0x7f0cdc0b5370 2024-09-10T21:02:28.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.088+0000 7f0cecff9640 1 -- 172.21.15.33:0/1018181982 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258905 (secure 0 0 0) 0x7f0cf0105d90 con 0x7f0cdc0b5370 2024-09-10T21:02:28.094 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:28.095 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-10T21:02:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0cdc0b5370 msgr2=0x7f0cdc0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0cdc0b5370 0x7f0cdc0b7830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000baa0 tx=0x7f0ce0005d90 comp rx=0 tx=0).stop 2024-09-10T21:02:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf0109150 msgr2=0x7f0cf019ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:28.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf0109150 0x7f0cf019ea60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800cce0 tx=0x7f0ce800a620 comp rx=0 tx=0).stop 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 shutdown_connections 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0cdc0b5370 0x7f0cdc0b7830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cf0109150 0x7f0cf019ea60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0cf01087d0 0x7f0cf01996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 --2- 172.21.15.33:0/1018181982 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0cf0102dc0 0x7f0cf0199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 >> 172.21.15.33:0/1018181982 conn(0x7f0cf00fe830 msgr2=0x7f0cf0101d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 shutdown_connections 2024-09-10T21:02:28.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:28.092+0000 7f0cf5334640 1 -- 172.21.15.33:0/1018181982 wait complete. 2024-09-10T21:02:28.828 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-10T21:02:26.088861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645388,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":224,"num_read_kb":198,"num_write":357,"num_write_kb":2271,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":295,"ondisk_log_size":295,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355180,"kb_used_data":10092,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586644,"statfs":{"total":1151940427776,"available":1151576723456,"internally_reserved":0,"allocated":10334208,"data_stored":6599916,"data_compressed":107421,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":49562,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":139,"num_read_kb":126,"num_write":76,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003362"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788385+0000","last_change":"2024-09-10T21:02:00.805421+0000","last_active":"2024-09-10T21:02:01.788385+0000","last_peered":"2024-09-10T21:02:01.788385+0000","last_clean":"2024-09-10T21:02:01.788385+0000","last_became_active":"2024-09-10T21:02:00.804274+0000","last_became_peered":"2024-09-10T21:02:00.804274+0000","last_unstale":"2024-09-10T21:02:01.788385+0000","last_undegraded":"2024-09-10T21:02:01.788385+0000","last_fullsized":"2024-09-10T21:02:01.788385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:12.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803249+0000","last_change":"2024-09-10T21:02:00.806104+0000","last_active":"2024-09-10T21:02:01.803249+0000","last_peered":"2024-09-10T21:02:01.803249+0000","last_clean":"2024-09-10T21:02:01.803249+0000","last_became_active":"2024-09-10T21:02:00.805510+0000","last_became_peered":"2024-09-10T21:02:00.805510+0000","last_unstale":"2024-09-10T21:02:01.803249+0000","last_undegraded":"2024-09-10T21:02:01.803249+0000","last_fullsized":"2024-09-10T21:02:01.803249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:37.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802601+0000","last_change":"2024-09-10T21:02:00.807691+0000","last_active":"2024-09-10T21:02:01.802601+0000","last_peered":"2024-09-10T21:02:01.802601+0000","last_clean":"2024-09-10T21:02:01.802601+0000","last_became_active":"2024-09-10T21:02:00.807364+0000","last_became_peered":"2024-09-10T21:02:00.807364+0000","last_unstale":"2024-09-10T21:02:01.802601+0000","last_undegraded":"2024-09-10T21:02:01.802601+0000","last_fullsized":"2024-09-10T21:02:01.802601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:27.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796533+0000","last_change":"2024-09-10T21:02:00.809542+0000","last_active":"2024-09-10T21:02:01.796533+0000","last_peered":"2024-09-10T21:02:01.796533+0000","last_clean":"2024-09-10T21:02:01.796533+0000","last_became_active":"2024-09-10T21:02:00.809034+0000","last_became_peered":"2024-09-10T21:02:00.809034+0000","last_unstale":"2024-09-10T21:02:01.796533+0000","last_undegraded":"2024-09-10T21:02:01.796533+0000","last_fullsized":"2024-09-10T21:02:01.796533+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:29.923532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796807+0000","last_change":"2024-09-10T21:02:00.809355+0000","last_active":"2024-09-10T21:02:01.796807+0000","last_peered":"2024-09-10T21:02:01.796807+0000","last_clean":"2024-09-10T21:02:01.796807+0000","last_became_active":"2024-09-10T21:02:00.808863+0000","last_became_peered":"2024-09-10T21:02:00.808863+0000","last_unstale":"2024-09-10T21:02:01.796807+0000","last_undegraded":"2024-09-10T21:02:01.796807+0000","last_fullsized":"2024-09-10T21:02:01.796807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:29.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.790553+0000","last_change":"2024-09-10T21:02:00.803756+0000","last_active":"2024-09-10T21:02:01.790553+0000","last_peered":"2024-09-10T21:02:01.790553+0000","last_clean":"2024-09-10T21:02:01.790553+0000","last_became_active":"2024-09-10T21:02:00.803406+0000","last_became_peered":"2024-09-10T21:02:00.803406+0000","last_unstale":"2024-09-10T21:02:01.790553+0000","last_undegraded":"2024-09-10T21:02:01.790553+0000","last_fullsized":"2024-09-10T21:02:01.790553+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:53.437629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798836+0000","last_change":"2024-09-10T21:02:00.808845+0000","last_active":"2024-09-10T21:02:01.798836+0000","last_peered":"2024-09-10T21:02:01.798836+0000","last_clean":"2024-09-10T21:02:01.798836+0000","last_became_active":"2024-09-10T21:02:00.807759+0000","last_became_peered":"2024-09-10T21:02:00.807759+0000","last_unstale":"2024-09-10T21:02:01.798836+0000","last_undegraded":"2024-09-10T21:02:01.798836+0000","last_fullsized":"2024-09-10T21:02:01.798836+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.809389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800503+0000","last_change":"2024-09-10T21:02:00.811199+0000","last_active":"2024-09-10T21:02:01.800503+0000","last_peered":"2024-09-10T21:02:01.800503+0000","last_clean":"2024-09-10T21:02:01.800503+0000","last_became_active":"2024-09-10T21:02:00.810692+0000","last_became_peered":"2024-09-10T21:02:00.810692+0000","last_unstale":"2024-09-10T21:02:01.800503+0000","last_undegraded":"2024-09-10T21:02:01.800503+0000","last_fullsized":"2024-09-10T21:02:01.800503+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:59.557973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799691+0000","last_change":"2024-09-10T21:02:00.812686+0000","last_active":"2024-09-10T21:02:01.799691+0000","last_peered":"2024-09-10T21:02:01.799691+0000","last_clean":"2024-09-10T21:02:01.799691+0000","last_became_active":"2024-09-10T21:02:00.812363+0000","last_became_peered":"2024-09-10T21:02:00.812363+0000","last_unstale":"2024-09-10T21:02:01.799691+0000","last_undegraded":"2024-09-10T21:02:01.799691+0000","last_fullsized":"2024-09-10T21:02:01.799691+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:37.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.791003+0000","last_change":"2024-09-10T21:02:00.812058+0000","last_active":"2024-09-10T21:02:01.791003+0000","last_peered":"2024-09-10T21:02:01.791003+0000","last_clean":"2024-09-10T21:02:01.791003+0000","last_became_active":"2024-09-10T21:02:00.811490+0000","last_became_peered":"2024-09-10T21:02:00.811490+0000","last_unstale":"2024-09-10T21:02:01.791003+0000","last_undegraded":"2024-09-10T21:02:01.791003+0000","last_fullsized":"2024-09-10T21:02:01.791003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:31.557485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800320+0000","last_change":"2024-09-10T21:02:00.809991+0000","last_active":"2024-09-10T21:02:01.800320+0000","last_peered":"2024-09-10T21:02:01.800320+0000","last_clean":"2024-09-10T21:02:01.800320+0000","last_became_active":"2024-09-10T21:02:00.809197+0000","last_became_peered":"2024-09-10T21:02:00.809197+0000","last_unstale":"2024-09-10T21:02:01.800320+0000","last_undegraded":"2024-09-10T21:02:01.800320+0000","last_fullsized":"2024-09-10T21:02:01.800320+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796740+0000","last_change":"2024-09-10T21:02:00.805122+0000","last_active":"2024-09-10T21:02:01.796740+0000","last_peered":"2024-09-10T21:02:01.796740+0000","last_clean":"2024-09-10T21:02:01.796740+0000","last_became_active":"2024-09-10T21:02:00.804784+0000","last_became_peered":"2024-09-10T21:02:00.804784+0000","last_unstale":"2024-09-10T21:02:01.796740+0000","last_undegraded":"2024-09-10T21:02:01.796740+0000","last_fullsized":"2024-09-10T21:02:01.796740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:04.823870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798520+0000","last_change":"2024-09-10T21:02:00.807280+0000","last_active":"2024-09-10T21:02:01.798520+0000","last_peered":"2024-09-10T21:02:01.798520+0000","last_clean":"2024-09-10T21:02:01.798520+0000","last_became_active":"2024-09-10T21:02:00.804247+0000","last_became_peered":"2024-09-10T21:02:00.804247+0000","last_unstale":"2024-09-10T21:02:01.798520+0000","last_undegraded":"2024-09-10T21:02:01.798520+0000","last_fullsized":"2024-09-10T21:02:01.798520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:28.163960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802774+0000","last_change":"2024-09-10T21:02:00.805723+0000","last_active":"2024-09-10T21:02:01.802774+0000","last_peered":"2024-09-10T21:02:01.802774+0000","last_clean":"2024-09-10T21:02:01.802774+0000","last_became_active":"2024-09-10T21:02:00.805104+0000","last_became_peered":"2024-09-10T21:02:00.805104+0000","last_unstale":"2024-09-10T21:02:01.802774+0000","last_undegraded":"2024-09-10T21:02:01.802774+0000","last_fullsized":"2024-09-10T21:02:01.802774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.056884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802581+0000","last_change":"2024-09-10T21:02:00.810497+0000","last_active":"2024-09-10T21:02:01.802581+0000","last_peered":"2024-09-10T21:02:01.802581+0000","last_clean":"2024-09-10T21:02:01.802581+0000","last_became_active":"2024-09-10T21:02:00.809392+0000","last_became_peered":"2024-09-10T21:02:00.809392+0000","last_unstale":"2024-09-10T21:02:01.802581+0000","last_undegraded":"2024-09-10T21:02:01.802581+0000","last_fullsized":"2024-09-10T21:02:01.802581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:51.214771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798089+0000","last_change":"2024-09-10T21:02:00.813973+0000","last_active":"2024-09-10T21:02:01.798089+0000","last_peered":"2024-09-10T21:02:01.798089+0000","last_clean":"2024-09-10T21:02:01.798089+0000","last_became_active":"2024-09-10T21:02:00.813580+0000","last_became_peered":"2024-09-10T21:02:00.813580+0000","last_unstale":"2024-09-10T21:02:01.798089+0000","last_undegraded":"2024-09-10T21:02:01.798089+0000","last_fullsized":"2024-09-10T21:02:01.798089+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:48.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788416+0000","last_change":"2024-09-10T21:02:00.804844+0000","last_active":"2024-09-10T21:02:01.788416+0000","last_peered":"2024-09-10T21:02:01.788416+0000","last_clean":"2024-09-10T21:02:01.788416+0000","last_became_active":"2024-09-10T21:02:00.803866+0000","last_became_peered":"2024-09-10T21:02:00.803866+0000","last_unstale":"2024-09-10T21:02:01.788416+0000","last_undegraded":"2024-09-10T21:02:01.788416+0000","last_fullsized":"2024-09-10T21:02:01.788416+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:05.231877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.789932+0000","last_change":"2024-09-10T21:02:00.801343+0000","last_active":"2024-09-10T21:02:01.789932+0000","last_peered":"2024-09-10T21:02:01.789932+0000","last_clean":"2024-09-10T21:02:01.789932+0000","last_became_active":"2024-09-10T21:02:00.801008+0000","last_became_peered":"2024-09-10T21:02:00.801008+0000","last_unstale":"2024-09-10T21:02:01.789932+0000","last_undegraded":"2024-09-10T21:02:01.789932+0000","last_fullsized":"2024-09-10T21:02:01.789932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:31.167925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.797891+0000","last_change":"2024-09-10T21:02:00.813026+0000","last_active":"2024-09-10T21:02:01.797891+0000","last_peered":"2024-09-10T21:02:01.797891+0000","last_clean":"2024-09-10T21:02:01.797891+0000","last_became_active":"2024-09-10T21:02:00.812074+0000","last_became_peered":"2024-09-10T21:02:00.812074+0000","last_unstale":"2024-09-10T21:02:01.797891+0000","last_undegraded":"2024-09-10T21:02:01.797891+0000","last_fullsized":"2024-09-10T21:02:01.797891+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:50:04.866402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788036+0000","last_change":"2024-09-10T21:02:00.810502+0000","last_active":"2024-09-10T21:02:01.788036+0000","last_peered":"2024-09-10T21:02:01.788036+0000","last_clean":"2024-09-10T21:02:01.788036+0000","last_became_active":"2024-09-10T21:02:00.810209+0000","last_became_peered":"2024-09-10T21:02:00.810209+0000","last_unstale":"2024-09-10T21:02:01.788036+0000","last_undegraded":"2024-09-10T21:02:01.788036+0000","last_fullsized":"2024-09-10T21:02:01.788036+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:51.311272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798556+0000","last_change":"2024-09-10T21:02:00.797119+0000","last_active":"2024-09-10T21:02:01.798556+0000","last_peered":"2024-09-10T21:02:01.798556+0000","last_clean":"2024-09-10T21:02:01.798556+0000","last_became_active":"2024-09-10T21:02:00.796788+0000","last_became_peered":"2024-09-10T21:02:00.796788+0000","last_unstale":"2024-09-10T21:02:01.798556+0000","last_undegraded":"2024-09-10T21:02:01.798556+0000","last_fullsized":"2024-09-10T21:02:01.798556+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:07.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798899+0000","last_change":"2024-09-10T21:02:00.802284+0000","last_active":"2024-09-10T21:02:01.798899+0000","last_peered":"2024-09-10T21:02:01.798899+0000","last_clean":"2024-09-10T21:02:01.798899+0000","last_became_active":"2024-09-10T21:02:00.801964+0000","last_became_peered":"2024-09-10T21:02:00.801964+0000","last_unstale":"2024-09-10T21:02:01.798899+0000","last_undegraded":"2024-09-10T21:02:01.798899+0000","last_fullsized":"2024-09-10T21:02:01.798899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:45.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800645+0000","last_change":"2024-09-10T21:02:00.809196+0000","last_active":"2024-09-10T21:02:01.800645+0000","last_peered":"2024-09-10T21:02:01.800645+0000","last_clean":"2024-09-10T21:02:01.800645+0000","last_became_active":"2024-09-10T21:02:00.808903+0000","last_became_peered":"2024-09-10T21:02:00.808903+0000","last_unstale":"2024-09-10T21:02:01.800645+0000","last_undegraded":"2024-09-10T21:02:01.800645+0000","last_fullsized":"2024-09-10T21:02:01.800645+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:01.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803568+0000","last_change":"2024-09-10T21:02:00.808195+0000","last_active":"2024-09-10T21:02:01.803568+0000","last_peered":"2024-09-10T21:02:01.803568+0000","last_clean":"2024-09-10T21:02:01.803568+0000","last_became_active":"2024-09-10T21:02:00.807670+0000","last_became_peered":"2024-09-10T21:02:00.807670+0000","last_unstale":"2024-09-10T21:02:01.803568+0000","last_undegraded":"2024-09-10T21:02:01.803568+0000","last_fullsized":"2024-09-10T21:02:01.803568+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:12.549344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802944+0000","last_change":"2024-09-10T21:02:00.811401+0000","last_active":"2024-09-10T21:02:01.802944+0000","last_peered":"2024-09-10T21:02:01.802944+0000","last_clean":"2024-09-10T21:02:01.802944+0000","last_became_active":"2024-09-10T21:02:00.811055+0000","last_became_peered":"2024-09-10T21:02:00.811055+0000","last_unstale":"2024-09-10T21:02:01.802944+0000","last_undegraded":"2024-09-10T21:02:01.802944+0000","last_fullsized":"2024-09-10T21:02:01.802944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:58.641873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798711+0000","last_change":"2024-09-10T21:02:00.813796+0000","last_active":"2024-09-10T21:02:01.798711+0000","last_peered":"2024-09-10T21:02:01.798711+0000","last_clean":"2024-09-10T21:02:01.798711+0000","last_became_active":"2024-09-10T21:02:00.813468+0000","last_became_peered":"2024-09-10T21:02:00.813468+0000","last_unstale":"2024-09-10T21:02:01.798711+0000","last_undegraded":"2024-09-10T21:02:01.798711+0000","last_fullsized":"2024-09-10T21:02:01.798711+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:25.964890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788213+0000","last_change":"2024-09-10T21:02:00.811436+0000","last_active":"2024-09-10T21:02:01.788213+0000","last_peered":"2024-09-10T21:02:01.788213+0000","last_clean":"2024-09-10T21:02:01.788213+0000","last_became_active":"2024-09-10T21:02:00.810956+0000","last_became_peered":"2024-09-10T21:02:00.810956+0000","last_unstale":"2024-09-10T21:02:01.788213+0000","last_undegraded":"2024-09-10T21:02:01.788213+0000","last_fullsized":"2024-09-10T21:02:01.788213+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.306832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798715+0000","last_change":"2024-09-10T21:02:00.799370+0000","last_active":"2024-09-10T21:02:01.798715+0000","last_peered":"2024-09-10T21:02:01.798715+0000","last_clean":"2024-09-10T21:02:01.798715+0000","last_became_active":"2024-09-10T21:02:00.799044+0000","last_became_peered":"2024-09-10T21:02:00.799044+0000","last_unstale":"2024-09-10T21:02:01.798715+0000","last_undegraded":"2024-09-10T21:02:01.798715+0000","last_fullsized":"2024-09-10T21:02:01.798715+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:40.332088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803457+0000","last_change":"2024-09-10T21:02:00.799830+0000","last_active":"2024-09-10T21:02:01.803457+0000","last_peered":"2024-09-10T21:02:01.803457+0000","last_clean":"2024-09-10T21:02:01.803457+0000","last_became_active":"2024-09-10T21:02:00.799533+0000","last_became_peered":"2024-09-10T21:02:00.799533+0000","last_unstale":"2024-09-10T21:02:01.803457+0000","last_undegraded":"2024-09-10T21:02:01.803457+0000","last_fullsized":"2024-09-10T21:02:01.803457+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:23.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799264+0000","last_change":"2024-09-10T21:02:00.812739+0000","last_active":"2024-09-10T21:02:01.799264+0000","last_peered":"2024-09-10T21:02:01.799264+0000","last_clean":"2024-09-10T21:02:01.799264+0000","last_became_active":"2024-09-10T21:02:00.812395+0000","last_became_peered":"2024-09-10T21:02:00.812395+0000","last_unstale":"2024-09-10T21:02:01.799264+0000","last_undegraded":"2024-09-10T21:02:01.799264+0000","last_fullsized":"2024-09-10T21:02:01.799264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:55.727155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.797481+0000","last_change":"2024-09-10T21:02:00.806291+0000","last_active":"2024-09-10T21:02:01.797481+0000","last_peered":"2024-09-10T21:02:01.797481+0000","last_clean":"2024-09-10T21:02:01.797481+0000","last_became_active":"2024-09-10T21:02:00.805943+0000","last_became_peered":"2024-09-10T21:02:00.805943+0000","last_unstale":"2024-09-10T21:02:01.797481+0000","last_undegraded":"2024-09-10T21:02:01.797481+0000","last_fullsized":"2024-09-10T21:02:01.797481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:21.354105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788016+0000","last_change":"2024-09-10T21:02:00.808774+0000","last_active":"2024-09-10T21:02:01.788016+0000","last_peered":"2024-09-10T21:02:01.788016+0000","last_clean":"2024-09-10T21:02:01.788016+0000","last_became_active":"2024-09-10T21:02:00.808027+0000","last_became_peered":"2024-09-10T21:02:00.808027+0000","last_unstale":"2024-09-10T21:02:01.788016+0000","last_undegraded":"2024-09-10T21:02:01.788016+0000","last_fullsized":"2024-09-10T21:02:01.788016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796627+0000","last_change":"2024-09-10T21:02:00.808349+0000","last_active":"2024-09-10T21:02:01.796627+0000","last_peered":"2024-09-10T21:02:01.796627+0000","last_clean":"2024-09-10T21:02:01.796627+0000","last_became_active":"2024-09-10T21:02:00.807839+0000","last_became_peered":"2024-09-10T21:02:00.807839+0000","last_unstale":"2024-09-10T21:02:01.796627+0000","last_undegraded":"2024-09-10T21:02:01.796627+0000","last_fullsized":"2024-09-10T21:02:01.796627+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:53.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:20.887759+0000","last_change":"2024-09-10T21:01:58.789380+0000","last_active":"2024-09-10T21:02:20.887759+0000","last_peered":"2024-09-10T21:02:20.887759+0000","last_clean":"2024-09-10T21:02:20.887759+0000","last_became_active":"2024-09-10T21:01:58.789008+0000","last_became_peered":"2024-09-10T21:01:58.789008+0000","last_unstale":"2024-09-10T21:02:20.887759+0000","last_undegraded":"2024-09-10T21:02:20.887759+0000","last_fullsized":"2024-09-10T21:02:20.887759+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:01.488478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803608+0000","last_change":"2024-09-10T21:02:00.807068+0000","last_active":"2024-09-10T21:02:01.803608+0000","last_peered":"2024-09-10T21:02:01.803608+0000","last_clean":"2024-09-10T21:02:01.803608+0000","last_became_active":"2024-09-10T21:02:00.806764+0000","last_became_peered":"2024-09-10T21:02:00.806764+0000","last_unstale":"2024-09-10T21:02:01.803608+0000","last_undegraded":"2024-09-10T21:02:01.803608+0000","last_fullsized":"2024-09-10T21:02:01.803608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:03.307495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:10.860997+0000","last_change":"2024-09-10T21:01:58.787619+0000","last_active":"2024-09-10T21:02:10.860997+0000","last_peered":"2024-09-10T21:02:10.860997+0000","last_clean":"2024-09-10T21:02:10.860997+0000","last_became_active":"2024-09-10T21:01:58.787138+0000","last_became_peered":"2024-09-10T21:01:58.787138+0000","last_unstale":"2024-09-10T21:02:10.860997+0000","last_undegraded":"2024-09-10T21:02:10.860997+0000","last_fullsized":"2024-09-10T21:02:10.860997+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:33.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:09.849049+0000","last_change":"2024-09-10T21:01:58.780115+0000","last_active":"2024-09-10T21:02:09.849049+0000","last_peered":"2024-09-10T21:02:09.849049+0000","last_clean":"2024-09-10T21:02:09.849049+0000","last_became_active":"2024-09-10T21:01:58.779757+0000","last_became_peered":"2024-09-10T21:01:58.779757+0000","last_unstale":"2024-09-10T21:02:09.849049+0000","last_undegraded":"2024-09-10T21:02:09.849049+0000","last_fullsized":"2024-09-10T21:02:09.849049+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:29.337978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800485+0000","last_change":"2024-09-10T21:02:00.800925+0000","last_active":"2024-09-10T21:02:01.800485+0000","last_peered":"2024-09-10T21:02:01.800485+0000","last_clean":"2024-09-10T21:02:01.800485+0000","last_became_active":"2024-09-10T21:02:00.798567+0000","last_became_peered":"2024-09-10T21:02:00.798567+0000","last_unstale":"2024-09-10T21:02:01.800485+0000","last_undegraded":"2024-09-10T21:02:01.800485+0000","last_fullsized":"2024-09-10T21:02:01.800485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:01.873716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.868222+0000","last_change":"2024-09-10T21:01:58.786040+0000","last_active":"2024-09-10T21:02:11.868222+0000","last_peered":"2024-09-10T21:02:11.868222+0000","last_clean":"2024-09-10T21:02:11.868222+0000","last_became_active":"2024-09-10T21:01:58.785703+0000","last_became_peered":"2024-09-10T21:01:58.785703+0000","last_unstale":"2024-09-10T21:02:11.868222+0000","last_undegraded":"2024-09-10T21:02:11.868222+0000","last_fullsized":"2024-09-10T21:02:11.868222+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:39.741614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.789677+0000","last_change":"2024-09-10T21:02:00.798368+0000","last_active":"2024-09-10T21:02:01.789677+0000","last_peered":"2024-09-10T21:02:01.789677+0000","last_clean":"2024-09-10T21:02:01.789677+0000","last_became_active":"2024-09-10T21:02:00.798080+0000","last_became_peered":"2024-09-10T21:02:00.798080+0000","last_unstale":"2024-09-10T21:02:01.789677+0000","last_undegraded":"2024-09-10T21:02:01.789677+0000","last_fullsized":"2024-09-10T21:02:01.789677+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:31.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:21.888298+0000","last_change":"2024-09-10T21:01:58.788119+0000","last_active":"2024-09-10T21:02:21.888298+0000","last_peered":"2024-09-10T21:02:21.888298+0000","last_clean":"2024-09-10T21:02:21.888298+0000","last_became_active":"2024-09-10T21:01:58.787724+0000","last_became_peered":"2024-09-10T21:01:58.787724+0000","last_unstale":"2024-09-10T21:02:21.888298+0000","last_undegraded":"2024-09-10T21:02:21.888298+0000","last_fullsized":"2024-09-10T21:02:21.888298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:42.872858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800024+0000","last_change":"2024-09-10T21:02:00.804425+0000","last_active":"2024-09-10T21:02:01.800024+0000","last_peered":"2024-09-10T21:02:01.800024+0000","last_clean":"2024-09-10T21:02:01.800024+0000","last_became_active":"2024-09-10T21:02:00.804093+0000","last_became_peered":"2024-09-10T21:02:00.804093+0000","last_unstale":"2024-09-10T21:02:01.800024+0000","last_undegraded":"2024-09-10T21:02:01.800024+0000","last_fullsized":"2024-09-10T21:02:01.800024+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:46.183326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867156+0000","last_change":"2024-09-10T21:01:58.791323+0000","last_active":"2024-09-10T21:02:11.867156+0000","last_peered":"2024-09-10T21:02:11.867156+0000","last_clean":"2024-09-10T21:02:11.867156+0000","last_became_active":"2024-09-10T21:01:58.790551+0000","last_became_peered":"2024-09-10T21:01:58.790551+0000","last_unstale":"2024-09-10T21:02:11.867156+0000","last_undegraded":"2024-09-10T21:02:11.867156+0000","last_fullsized":"2024-09-10T21:02:11.867156+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:18.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803889+0000","last_change":"2024-09-10T21:02:00.804025+0000","last_active":"2024-09-10T21:02:01.803889+0000","last_peered":"2024-09-10T21:02:01.803889+0000","last_clean":"2024-09-10T21:02:01.803889+0000","last_became_active":"2024-09-10T21:02:00.803663+0000","last_became_peered":"2024-09-10T21:02:00.803663+0000","last_unstale":"2024-09-10T21:02:01.803889+0000","last_undegraded":"2024-09-10T21:02:01.803889+0000","last_fullsized":"2024-09-10T21:02:01.803889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:30.609020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:10.860671+0000","last_change":"2024-09-10T21:01:58.887067+0000","last_active":"2024-09-10T21:02:10.860671+0000","last_peered":"2024-09-10T21:02:10.860671+0000","last_clean":"2024-09-10T21:02:10.860671+0000","last_became_active":"2024-09-10T21:01:58.886611+0000","last_became_peered":"2024-09-10T21:01:58.886611+0000","last_unstale":"2024-09-10T21:02:10.860671+0000","last_undegraded":"2024-09-10T21:02:10.860671+0000","last_fullsized":"2024-09-10T21:02:10.860671+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:06.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798417+0000","last_change":"2024-09-10T21:02:00.814120+0000","last_active":"2024-09-10T21:02:01.798417+0000","last_peered":"2024-09-10T21:02:01.798417+0000","last_clean":"2024-09-10T21:02:01.798417+0000","last_became_active":"2024-09-10T21:02:00.813769+0000","last_became_peered":"2024-09-10T21:02:00.813769+0000","last_unstale":"2024-09-10T21:02:01.798417+0000","last_undegraded":"2024-09-10T21:02:01.798417+0000","last_fullsized":"2024-09-10T21:02:01.798417+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:10.601319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'24","reported_seq":69,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:21.889517+0000","last_change":"2024-09-10T21:01:58.790966+0000","last_active":"2024-09-10T21:02:21.889517+0000","last_peered":"2024-09-10T21:02:21.889517+0000","last_clean":"2024-09-10T21:02:21.889517+0000","last_became_active":"2024-09-10T21:01:58.790636+0000","last_became_peered":"2024-09-10T21:01:58.790636+0000","last_unstale":"2024-09-10T21:02:21.889517+0000","last_undegraded":"2024-09-10T21:02:21.889517+0000","last_fullsized":"2024-09-10T21:02:21.889517+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:06.209074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46800,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":58,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799643+0000","last_change":"2024-09-10T21:02:00.810610+0000","last_active":"2024-09-10T21:02:01.799643+0000","last_peered":"2024-09-10T21:02:01.799643+0000","last_clean":"2024-09-10T21:02:01.799643+0000","last_became_active":"2024-09-10T21:02:00.810224+0000","last_became_peered":"2024-09-10T21:02:00.810224+0000","last_unstale":"2024-09-10T21:02:01.799643+0000","last_undegraded":"2024-09-10T21:02:01.799643+0000","last_fullsized":"2024-09-10T21:02:01.799643+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:32.785312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:12.872396+0000","last_change":"2024-09-10T21:01:58.786476+0000","last_active":"2024-09-10T21:02:12.872396+0000","last_peered":"2024-09-10T21:02:12.872396+0000","last_clean":"2024-09-10T21:02:12.872396+0000","last_became_active":"2024-09-10T21:01:58.786153+0000","last_became_peered":"2024-09-10T21:01:58.786153+0000","last_unstale":"2024-09-10T21:02:12.872396+0000","last_undegraded":"2024-09-10T21:02:12.872396+0000","last_fullsized":"2024-09-10T21:02:12.872396+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:00.118591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798264+0000","last_change":"2024-09-10T21:02:00.796138+0000","last_active":"2024-09-10T21:02:01.798264+0000","last_peered":"2024-09-10T21:02:01.798264+0000","last_clean":"2024-09-10T21:02:01.798264+0000","last_became_active":"2024-09-10T21:02:00.795847+0000","last_became_peered":"2024-09-10T21:02:00.795847+0000","last_unstale":"2024-09-10T21:02:01.798264+0000","last_undegraded":"2024-09-10T21:02:01.798264+0000","last_fullsized":"2024-09-10T21:02:01.798264+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:29.462134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867188+0000","last_change":"2024-09-10T21:01:58.789314+0000","last_active":"2024-09-10T21:02:11.867188+0000","last_peered":"2024-09-10T21:02:11.867188+0000","last_clean":"2024-09-10T21:02:11.867188+0000","last_became_active":"2024-09-10T21:01:58.788594+0000","last_became_peered":"2024-09-10T21:01:58.788594+0000","last_unstale":"2024-09-10T21:02:11.867188+0000","last_undegraded":"2024-09-10T21:02:11.867188+0000","last_fullsized":"2024-09-10T21:02:11.867188+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:44.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798980+0000","last_change":"2024-09-10T21:02:00.799828+0000","last_active":"2024-09-10T21:02:01.798980+0000","last_peered":"2024-09-10T21:02:01.798980+0000","last_clean":"2024-09-10T21:02:01.798980+0000","last_became_active":"2024-09-10T21:02:00.799465+0000","last_became_peered":"2024-09-10T21:02:00.799465+0000","last_unstale":"2024-09-10T21:02:01.798980+0000","last_undegraded":"2024-09-10T21:02:01.798980+0000","last_fullsized":"2024-09-10T21:02:01.798980+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:56.794326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:21.888452+0000","last_change":"2024-09-10T21:01:58.788730+0000","last_active":"2024-09-10T21:02:21.888452+0000","last_peered":"2024-09-10T21:02:21.888452+0000","last_clean":"2024-09-10T21:02:21.888452+0000","last_became_active":"2024-09-10T21:01:58.788317+0000","last_became_peered":"2024-09-10T21:01:58.788317+0000","last_unstale":"2024-09-10T21:02:21.888452+0000","last_undegraded":"2024-09-10T21:02:21.888452+0000","last_fullsized":"2024-09-10T21:02:21.888452+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:52.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799475+0000","last_change":"2024-09-10T21:02:00.807104+0000","last_active":"2024-09-10T21:02:01.799475+0000","last_peered":"2024-09-10T21:02:01.799475+0000","last_clean":"2024-09-10T21:02:01.799475+0000","last_became_active":"2024-09-10T21:02:00.806815+0000","last_became_peered":"2024-09-10T21:02:00.806815+0000","last_unstale":"2024-09-10T21:02:01.799475+0000","last_undegraded":"2024-09-10T21:02:01.799475+0000","last_fullsized":"2024-09-10T21:02:01.799475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:24.747662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867889+0000","last_change":"2024-09-10T21:01:58.886877+0000","last_active":"2024-09-10T21:02:11.867889+0000","last_peered":"2024-09-10T21:02:11.867889+0000","last_clean":"2024-09-10T21:02:11.867889+0000","last_became_active":"2024-09-10T21:01:58.886437+0000","last_became_peered":"2024-09-10T21:01:58.886437+0000","last_unstale":"2024-09-10T21:02:11.867889+0000","last_undegraded":"2024-09-10T21:02:11.867889+0000","last_fullsized":"2024-09-10T21:02:11.867889+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:31:34.786880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803016+0000","last_change":"2024-09-10T21:02:00.806796+0000","last_active":"2024-09-10T21:02:01.803016+0000","last_peered":"2024-09-10T21:02:01.803016+0000","last_clean":"2024-09-10T21:02:01.803016+0000","last_became_active":"2024-09-10T21:02:00.805802+0000","last_became_peered":"2024-09-10T21:02:00.805802+0000","last_unstale":"2024-09-10T21:02:01.803016+0000","last_undegraded":"2024-09-10T21:02:01.803016+0000","last_fullsized":"2024-09-10T21:02:01.803016+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.962870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.845176+0000","last_change":"2024-09-10T21:01:58.781088+0000","last_active":"2024-09-10T21:02:11.845176+0000","last_peered":"2024-09-10T21:02:11.845176+0000","last_clean":"2024-09-10T21:02:11.845176+0000","last_became_active":"2024-09-10T21:01:58.780731+0000","last_became_peered":"2024-09-10T21:01:58.780731+0000","last_unstale":"2024-09-10T21:02:11.845176+0000","last_undegraded":"2024-09-10T21:02:11.845176+0000","last_fullsized":"2024-09-10T21:02:11.845176+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:50.636077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788274+0000","last_change":"2024-09-10T21:02:00.806551+0000","last_active":"2024-09-10T21:02:01.788274+0000","last_peered":"2024-09-10T21:02:01.788274+0000","last_clean":"2024-09-10T21:02:01.788274+0000","last_became_active":"2024-09-10T21:02:00.806134+0000","last_became_peered":"2024-09-10T21:02:00.806134+0000","last_unstale":"2024-09-10T21:02:01.788274+0000","last_undegraded":"2024-09-10T21:02:01.788274+0000","last_fullsized":"2024-09-10T21:02:01.788274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.574372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.866161+0000","last_change":"2024-09-10T21:01:58.780294+0000","last_active":"2024-09-10T21:02:11.866161+0000","last_peered":"2024-09-10T21:02:11.866161+0000","last_clean":"2024-09-10T21:02:11.866161+0000","last_became_active":"2024-09-10T21:01:58.779767+0000","last_became_peered":"2024-09-10T21:01:58.779767+0000","last_unstale":"2024-09-10T21:02:11.866161+0000","last_undegraded":"2024-09-10T21:02:11.866161+0000","last_fullsized":"2024-09-10T21:02:11.866161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:21.734159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799903+0000","last_change":"2024-09-10T21:02:00.809393+0000","last_active":"2024-09-10T21:02:01.799903+0000","last_peered":"2024-09-10T21:02:01.799903+0000","last_clean":"2024-09-10T21:02:01.799903+0000","last_became_active":"2024-09-10T21:02:00.809012+0000","last_became_peered":"2024-09-10T21:02:00.809012+0000","last_unstale":"2024-09-10T21:02:01.799903+0000","last_undegraded":"2024-09-10T21:02:01.799903+0000","last_fullsized":"2024-09-10T21:02:01.799903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:03.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867304+0000","last_change":"2024-09-10T21:01:58.786507+0000","last_active":"2024-09-10T21:02:11.867304+0000","last_peered":"2024-09-10T21:02:11.867304+0000","last_clean":"2024-09-10T21:02:11.867304+0000","last_became_active":"2024-09-10T21:01:58.786115+0000","last_became_peered":"2024-09-10T21:01:58.786115+0000","last_unstale":"2024-09-10T21:02:11.867304+0000","last_undegraded":"2024-09-10T21:02:11.867304+0000","last_fullsized":"2024-09-10T21:02:11.867304+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.985353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799601+0000","last_change":"2024-09-10T21:02:00.805050+0000","last_active":"2024-09-10T21:02:01.799601+0000","last_peered":"2024-09-10T21:02:01.799601+0000","last_clean":"2024-09-10T21:02:01.799601+0000","last_became_active":"2024-09-10T21:02:00.804706+0000","last_became_peered":"2024-09-10T21:02:00.804706+0000","last_unstale":"2024-09-10T21:02:01.799601+0000","last_undegraded":"2024-09-10T21:02:01.799601+0000","last_fullsized":"2024-09-10T21:02:01.799601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:15.463986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:22.890050+0000","last_change":"2024-09-10T21:01:59.114525+0000","last_active":"2024-09-10T21:02:22.890050+0000","last_peered":"2024-09-10T21:02:22.890050+0000","last_clean":"2024-09-10T21:02:22.890050+0000","last_became_active":"2024-09-10T21:01:59.114076+0000","last_became_peered":"2024-09-10T21:01:59.114076+0000","last_unstale":"2024-09-10T21:02:22.890050+0000","last_undegraded":"2024-09-10T21:02:22.890050+0000","last_fullsized":"2024-09-10T21:02:22.890050+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:53.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.800841+0000","last_change":"2024-09-10T21:02:00.799803+0000","last_active":"2024-09-10T21:02:01.800841+0000","last_peered":"2024-09-10T21:02:01.800841+0000","last_clean":"2024-09-10T21:02:01.800841+0000","last_became_active":"2024-09-10T21:02:00.799459+0000","last_became_peered":"2024-09-10T21:02:00.799459+0000","last_unstale":"2024-09-10T21:02:01.800841+0000","last_undegraded":"2024-09-10T21:02:01.800841+0000","last_fullsized":"2024-09-10T21:02:01.800841+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:57.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":94,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.799946+0000","last_change":"2024-09-10T21:00:48.381903+0000","last_active":"2024-09-10T21:02:01.799946+0000","last_peered":"2024-09-10T21:02:01.799946+0000","last_clean":"2024-09-10T21:02:01.799946+0000","last_became_active":"2024-09-10T21:00:48.376874+0000","last_became_peered":"2024-09-10T21:00:48.376874+0000","last_unstale":"2024-09-10T21:02:01.799946+0000","last_undegraded":"2024-09-10T21:02:01.799946+0000","last_fullsized":"2024-09-10T21:02:01.799946+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_clean_scrub_stamp":"2024-09-10T20:57:24.516525+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:35.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:12.872339+0000","last_change":"2024-09-10T21:01:58.788005+0000","last_active":"2024-09-10T21:02:12.872339+0000","last_peered":"2024-09-10T21:02:12.872339+0000","last_clean":"2024-09-10T21:02:12.872339+0000","last_became_active":"2024-09-10T21:01:58.786471+0000","last_became_peered":"2024-09-10T21:01:58.786471+0000","last_unstale":"2024-09-10T21:02:12.872339+0000","last_undegraded":"2024-09-10T21:02:12.872339+0000","last_fullsized":"2024-09-10T21:02:12.872339+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:15.622814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.801307+0000","last_change":"2024-09-10T21:02:00.802573+0000","last_active":"2024-09-10T21:02:01.801307+0000","last_peered":"2024-09-10T21:02:01.801307+0000","last_clean":"2024-09-10T21:02:01.801307+0000","last_became_active":"2024-09-10T21:02:00.802263+0000","last_became_peered":"2024-09-10T21:02:00.802263+0000","last_unstale":"2024-09-10T21:02:01.801307+0000","last_undegraded":"2024-09-10T21:02:01.801307+0000","last_fullsized":"2024-09-10T21:02:01.801307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:54.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":23,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:09.850177+0000","last_change":"2024-09-10T21:01:58.785700+0000","last_active":"2024-09-10T21:02:09.850177+0000","last_peered":"2024-09-10T21:02:09.850177+0000","last_clean":"2024-09-10T21:02:09.850177+0000","last_became_active":"2024-09-10T21:01:58.785346+0000","last_became_peered":"2024-09-10T21:01:58.785346+0000","last_unstale":"2024-09-10T21:02:09.850177+0000","last_undegraded":"2024-09-10T21:02:09.850177+0000","last_fullsized":"2024-09-10T21:02:09.850177+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:39.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798022+0000","last_change":"2024-09-10T21:02:00.808045+0000","last_active":"2024-09-10T21:02:01.798022+0000","last_peered":"2024-09-10T21:02:01.798022+0000","last_clean":"2024-09-10T21:02:01.798022+0000","last_became_active":"2024-09-10T21:02:00.807757+0000","last_became_peered":"2024-09-10T21:02:00.807757+0000","last_unstale":"2024-09-10T21:02:01.798022+0000","last_undegraded":"2024-09-10T21:02:01.798022+0000","last_fullsized":"2024-09-10T21:02:01.798022+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:16.198290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:22.889163+0000","last_change":"2024-09-10T21:01:58.884939+0000","last_active":"2024-09-10T21:02:22.889163+0000","last_peered":"2024-09-10T21:02:22.889163+0000","last_clean":"2024-09-10T21:02:22.889163+0000","last_became_active":"2024-09-10T21:01:58.884517+0000","last_became_peered":"2024-09-10T21:01:58.884517+0000","last_unstale":"2024-09-10T21:02:22.889163+0000","last_undegraded":"2024-09-10T21:02:22.889163+0000","last_fullsized":"2024-09-10T21:02:22.889163+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.579662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803370+0000","last_change":"2024-09-10T21:02:00.813021+0000","last_active":"2024-09-10T21:02:01.803370+0000","last_peered":"2024-09-10T21:02:01.803370+0000","last_clean":"2024-09-10T21:02:01.803370+0000","last_became_active":"2024-09-10T21:02:00.812622+0000","last_became_peered":"2024-09-10T21:02:00.812622+0000","last_unstale":"2024-09-10T21:02:01.803370+0000","last_undegraded":"2024-09-10T21:02:01.803370+0000","last_fullsized":"2024-09-10T21:02:01.803370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:48.348100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:23.890807+0000","last_change":"2024-09-10T21:01:58.792187+0000","last_active":"2024-09-10T21:02:23.890807+0000","last_peered":"2024-09-10T21:02:23.890807+0000","last_clean":"2024-09-10T21:02:23.890807+0000","last_became_active":"2024-09-10T21:01:58.791806+0000","last_became_peered":"2024-09-10T21:01:58.791806+0000","last_unstale":"2024-09-10T21:02:23.890807+0000","last_undegraded":"2024-09-10T21:02:23.890807+0000","last_fullsized":"2024-09-10T21:02:23.890807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:48.493396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.790867+0000","last_change":"2024-09-10T21:02:00.804073+0000","last_active":"2024-09-10T21:02:01.790867+0000","last_peered":"2024-09-10T21:02:01.790867+0000","last_clean":"2024-09-10T21:02:01.790867+0000","last_became_active":"2024-09-10T21:02:00.803760+0000","last_became_peered":"2024-09-10T21:02:00.803760+0000","last_unstale":"2024-09-10T21:02:01.790867+0000","last_undegraded":"2024-09-10T21:02:01.790867+0000","last_fullsized":"2024-09-10T21:02:01.790867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:01.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:02.635141+0000","last_change":"2024-09-10T21:01:58.885088+0000","last_active":"2024-09-10T21:02:02.635141+0000","last_peered":"2024-09-10T21:02:02.635141+0000","last_clean":"2024-09-10T21:02:02.635141+0000","last_became_active":"2024-09-10T21:01:58.884729+0000","last_became_peered":"2024-09-10T21:01:58.884729+0000","last_unstale":"2024-09-10T21:02:02.635141+0000","last_undegraded":"2024-09-10T21:02:02.635141+0000","last_fullsized":"2024-09-10T21:02:02.635141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:37.451629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:02.303367+0000","last_change":"2024-09-10T21:02:00.810329+0000","last_active":"2024-09-10T21:02:02.303367+0000","last_peered":"2024-09-10T21:02:02.303367+0000","last_clean":"2024-09-10T21:02:02.303367+0000","last_became_active":"2024-09-10T21:02:00.809985+0000","last_became_peered":"2024-09-10T21:02:00.809985+0000","last_unstale":"2024-09-10T21:02:02.303367+0000","last_undegraded":"2024-09-10T21:02:02.303367+0000","last_fullsized":"2024-09-10T21:02:02.303367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:39.631969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.867915+0000","last_change":"2024-09-10T21:01:58.884987+0000","last_active":"2024-09-10T21:02:11.867915+0000","last_peered":"2024-09-10T21:02:11.867915+0000","last_clean":"2024-09-10T21:02:11.867915+0000","last_became_active":"2024-09-10T21:01:58.884648+0000","last_became_peered":"2024-09-10T21:01:58.884648+0000","last_unstale":"2024-09-10T21:02:11.867915+0000","last_undegraded":"2024-09-10T21:02:11.867915+0000","last_fullsized":"2024-09-10T21:02:11.867915+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:35.195855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798746+0000","last_change":"2024-09-10T21:02:00.804961+0000","last_active":"2024-09-10T21:02:01.798746+0000","last_peered":"2024-09-10T21:02:01.798746+0000","last_clean":"2024-09-10T21:02:01.798746+0000","last_became_active":"2024-09-10T21:02:00.804147+0000","last_became_peered":"2024-09-10T21:02:00.804147+0000","last_unstale":"2024-09-10T21:02:01.798746+0000","last_undegraded":"2024-09-10T21:02:01.798746+0000","last_fullsized":"2024-09-10T21:02:01.798746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:15.801815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'11","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:19.794040+0000","last_change":"2024-09-10T21:01:58.781292+0000","last_active":"2024-09-10T21:02:19.794040+0000","last_peered":"2024-09-10T21:02:19.794040+0000","last_clean":"2024-09-10T21:02:19.794040+0000","last_became_active":"2024-09-10T21:01:58.780995+0000","last_became_peered":"2024-09-10T21:01:58.780995+0000","last_unstale":"2024-09-10T21:02:19.794040+0000","last_undegraded":"2024-09-10T21:02:19.794040+0000","last_fullsized":"2024-09-10T21:02:19.794040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:56.914723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803222+0000","last_change":"2024-09-10T21:02:00.805878+0000","last_active":"2024-09-10T21:02:01.803222+0000","last_peered":"2024-09-10T21:02:01.803222+0000","last_clean":"2024-09-10T21:02:01.803222+0000","last_became_active":"2024-09-10T21:02:00.804565+0000","last_became_peered":"2024-09-10T21:02:00.804565+0000","last_unstale":"2024-09-10T21:02:01.803222+0000","last_undegraded":"2024-09-10T21:02:01.803222+0000","last_fullsized":"2024-09-10T21:02:01.803222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:24.359643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.868277+0000","last_change":"2024-09-10T21:01:59.114862+0000","last_active":"2024-09-10T21:02:11.868277+0000","last_peered":"2024-09-10T21:02:11.868277+0000","last_clean":"2024-09-10T21:02:11.868277+0000","last_became_active":"2024-09-10T21:01:59.114411+0000","last_became_peered":"2024-09-10T21:01:59.114411+0000","last_unstale":"2024-09-10T21:02:11.868277+0000","last_undegraded":"2024-09-10T21:02:11.868277+0000","last_fullsized":"2024-09-10T21:02:11.868277+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.645829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803074+0000","last_change":"2024-09-10T21:02:00.799230+0000","last_active":"2024-09-10T21:02:01.803074+0000","last_peered":"2024-09-10T21:02:01.803074+0000","last_clean":"2024-09-10T21:02:01.803074+0000","last_became_active":"2024-09-10T21:02:00.798992+0000","last_became_peered":"2024-09-10T21:02:00.798992+0000","last_unstale":"2024-09-10T21:02:01.803074+0000","last_undegraded":"2024-09-10T21:02:01.803074+0000","last_fullsized":"2024-09-10T21:02:01.803074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:40.322066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:08.842955+0000","last_change":"2024-09-10T21:01:58.887042+0000","last_active":"2024-09-10T21:02:08.842955+0000","last_peered":"2024-09-10T21:02:08.842955+0000","last_clean":"2024-09-10T21:02:08.842955+0000","last_became_active":"2024-09-10T21:01:58.886710+0000","last_became_peered":"2024-09-10T21:01:58.886710+0000","last_unstale":"2024-09-10T21:02:08.842955+0000","last_undegraded":"2024-09-10T21:02:08.842955+0000","last_fullsized":"2024-09-10T21:02:08.842955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:42.672899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.796506+0000","last_change":"2024-09-10T21:02:00.809763+0000","last_active":"2024-09-10T21:02:01.796506+0000","last_peered":"2024-09-10T21:02:01.796506+0000","last_clean":"2024-09-10T21:02:01.796506+0000","last_became_active":"2024-09-10T21:02:00.808955+0000","last_became_peered":"2024-09-10T21:02:00.808955+0000","last_unstale":"2024-09-10T21:02:01.796506+0000","last_undegraded":"2024-09-10T21:02:01.796506+0000","last_fullsized":"2024-09-10T21:02:01.796506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:22.622852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":26,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.865716+0000","last_change":"2024-09-10T21:01:58.786312+0000","last_active":"2024-09-10T21:02:11.865716+0000","last_peered":"2024-09-10T21:02:11.865716+0000","last_clean":"2024-09-10T21:02:11.865716+0000","last_became_active":"2024-09-10T21:01:58.785894+0000","last_became_peered":"2024-09-10T21:01:58.785894+0000","last_unstale":"2024-09-10T21:02:11.865716+0000","last_undegraded":"2024-09-10T21:02:11.865716+0000","last_fullsized":"2024-09-10T21:02:11.865716+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:11.682904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.788285+0000","last_change":"2024-09-10T21:02:00.808374+0000","last_active":"2024-09-10T21:02:01.788285+0000","last_peered":"2024-09-10T21:02:01.788285+0000","last_clean":"2024-09-10T21:02:01.788285+0000","last_became_active":"2024-09-10T21:02:00.807565+0000","last_became_peered":"2024-09-10T21:02:00.807565+0000","last_unstale":"2024-09-10T21:02:01.788285+0000","last_undegraded":"2024-09-10T21:02:01.788285+0000","last_fullsized":"2024-09-10T21:02:01.788285+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:24.227480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:10.861693+0000","last_change":"2024-09-10T21:01:58.782375+0000","last_active":"2024-09-10T21:02:10.861693+0000","last_peered":"2024-09-10T21:02:10.861693+0000","last_clean":"2024-09-10T21:02:10.861693+0000","last_became_active":"2024-09-10T21:01:58.781964+0000","last_became_peered":"2024-09-10T21:01:58.781964+0000","last_unstale":"2024-09-10T21:02:10.861693+0000","last_undegraded":"2024-09-10T21:02:10.861693+0000","last_fullsized":"2024-09-10T21:02:10.861693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.460126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.802972+0000","last_change":"2024-09-10T21:02:00.806612+0000","last_active":"2024-09-10T21:02:01.802972+0000","last_peered":"2024-09-10T21:02:01.802972+0000","last_clean":"2024-09-10T21:02:01.802972+0000","last_became_active":"2024-09-10T21:02:00.803470+0000","last_became_peered":"2024-09-10T21:02:00.803470+0000","last_unstale":"2024-09-10T21:02:01.802972+0000","last_undegraded":"2024-09-10T21:02:01.802972+0000","last_fullsized":"2024-09-10T21:02:01.802972+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:08.350931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:08.843968+0000","last_change":"2024-09-10T21:01:58.788987+0000","last_active":"2024-09-10T21:02:08.843968+0000","last_peered":"2024-09-10T21:02:08.843968+0000","last_clean":"2024-09-10T21:02:08.843968+0000","last_became_active":"2024-09-10T21:01:58.788605+0000","last_became_peered":"2024-09-10T21:01:58.788605+0000","last_unstale":"2024-09-10T21:02:08.843968+0000","last_undegraded":"2024-09-10T21:02:08.843968+0000","last_fullsized":"2024-09-10T21:02:08.843968+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:45.798350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798367+0000","last_change":"2024-09-10T21:02:00.804200+0000","last_active":"2024-09-10T21:02:01.798367+0000","last_peered":"2024-09-10T21:02:01.798367+0000","last_clean":"2024-09-10T21:02:01.798367+0000","last_became_active":"2024-09-10T21:02:00.803875+0000","last_became_peered":"2024-09-10T21:02:00.803875+0000","last_unstale":"2024-09-10T21:02:01.798367+0000","last_undegraded":"2024-09-10T21:02:01.798367+0000","last_fullsized":"2024-09-10T21:02:01.798367+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:15.930163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803354+0000","last_change":"2024-09-10T21:02:00.814655+0000","last_active":"2024-09-10T21:02:01.803354+0000","last_peered":"2024-09-10T21:02:01.803354+0000","last_clean":"2024-09-10T21:02:01.803354+0000","last_became_active":"2024-09-10T21:02:00.813996+0000","last_became_peered":"2024-09-10T21:02:00.813996+0000","last_unstale":"2024-09-10T21:02:01.803354+0000","last_undegraded":"2024-09-10T21:02:01.803354+0000","last_fullsized":"2024-09-10T21:02:01.803354+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:59.860942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:11.868895+0000","last_change":"2024-09-10T21:01:58.784834+0000","last_active":"2024-09-10T21:02:11.868895+0000","last_peered":"2024-09-10T21:02:11.868895+0000","last_clean":"2024-09-10T21:02:11.868895+0000","last_became_active":"2024-09-10T21:01:58.784464+0000","last_became_peered":"2024-09-10T21:01:58.784464+0000","last_unstale":"2024-09-10T21:02:11.868895+0000","last_undegraded":"2024-09-10T21:02:11.868895+0000","last_fullsized":"2024-09-10T21:02:11.868895+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:09.864824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.798208+0000","last_change":"2024-09-10T21:02:00.805677+0000","last_active":"2024-09-10T21:02:01.798208+0000","last_peered":"2024-09-10T21:02:01.798208+0000","last_clean":"2024-09-10T21:02:01.798208+0000","last_became_active":"2024-09-10T21:02:00.805341+0000","last_became_peered":"2024-09-10T21:02:00.805341+0000","last_unstale":"2024-09-10T21:02:01.798208+0000","last_undegraded":"2024-09-10T21:02:01.798208+0000","last_fullsized":"2024-09-10T21:02:01.798208+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.236954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:12.872819+0000","last_change":"2024-09-10T21:01:58.786087+0000","last_active":"2024-09-10T21:02:12.872819+0000","last_peered":"2024-09-10T21:02:12.872819+0000","last_clean":"2024-09-10T21:02:12.872819+0000","last_became_active":"2024-09-10T21:01:58.785661+0000","last_became_peered":"2024-09-10T21:01:58.785661+0000","last_unstale":"2024-09-10T21:02:12.872819+0000","last_undegraded":"2024-09-10T21:02:12.872819+0000","last_fullsized":"2024-09-10T21:02:12.872819+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:41.045302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.803772+0000","last_change":"2024-09-10T21:02:00.801514+0000","last_active":"2024-09-10T21:02:01.803772+0000","last_peered":"2024-09-10T21:02:01.803772+0000","last_clean":"2024-09-10T21:02:01.803772+0000","last_became_active":"2024-09-10T21:02:00.801221+0000","last_became_peered":"2024-09-10T21:02:00.801221+0000","last_unstale":"2024-09-10T21:02:01.803772+0000","last_undegraded":"2024-09-10T21:02:01.803772+0000","last_fullsized":"2024-09-10T21:02:01.803772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:35.564890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:02.552335+0000","last_change":"2024-09-10T21:01:58.884876+0000","last_active":"2024-09-10T21:02:02.552335+0000","last_peered":"2024-09-10T21:02:02.552335+0000","last_clean":"2024-09-10T21:02:02.552335+0000","last_became_active":"2024-09-10T21:01:58.884530+0000","last_became_peered":"2024-09-10T21:01:58.884530+0000","last_unstale":"2024-09-10T21:02:02.552335+0000","last_undegraded":"2024-09-10T21:02:02.552335+0000","last_fullsized":"2024-09-10T21:02:02.552335+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.851017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:01.801716+0000","last_change":"2024-09-10T21:02:00.811148+0000","last_active":"2024-09-10T21:02:01.801716+0000","last_peered":"2024-09-10T21:02:01.801716+0000","last_clean":"2024-09-10T21:02:01.801716+0000","last_became_active":"2024-09-10T21:02:00.809265+0000","last_became_peered":"2024-09-10T21:02:00.809265+0000","last_unstale":"2024-09-10T21:02:01.801716+0000","last_undegraded":"2024-09-10T21:02:01.801716+0000","last_fullsized":"2024-09-10T21:02:01.801716+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:25.269876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-10T21:02:20.886576+0000","last_change":"2024-09-10T21:01:58.786917+0000","last_active":"2024-09-10T21:02:20.886576+0000","last_peered":"2024-09-10T21:02:20.886576+0000","last_clean":"2024-09-10T21:02:20.886576+0000","last_became_active":"2024-09-10T21:01:58.786595+0000","last_became_peered":"2024-09-10T21:01:58.786595+0000","last_unstale":"2024-09-10T21:02:20.886576+0000","last_undegraded":"2024-09-10T21:02:20.886576+0000","last_fullsized":"2024-09-10T21:02:20.886576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:30.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54786,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":148,"num_read_kb":134,"num_write":232,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":164358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":201,"ondisk_log_size":201,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":815104,"data_stored":390229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939670,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":1028058,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103195,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":389768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299423,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":835584,"data_stored":390146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495652,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":983446,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659178,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31988,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713164,"statfs":{"total":95995035648,"available":95962279936,"internally_reserved":0,"allocated":823296,"data_stored":389876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855407,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":982342,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018933,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717324,"statfs":{"total":95995035648,"available":95966539776,"internally_reserved":0,"allocated":823296,"data_stored":389737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215161,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":839680,"data_stored":389310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444095,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":892928,"data_stored":438200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":437589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":843776,"data_stored":391215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":48943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":49554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":48927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:02:28.832 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T21:02:28.833 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T21:02:28.833 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T21:02:29.097 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:29.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:29 smithi033 ceph-mon[30860]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:02:29.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:29 smithi033 ceph-mon[30860]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-10T21:02:29.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:29 smithi104 ceph-mon[35325]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:02:29.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:29 smithi104 ceph-mon[35325]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 -- 172.21.15.33:0/2828339143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 msgr2=0x7fca08101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 --2- 172.21.15.33:0/2828339143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08101d10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8009930 tx=0x7fc9f802a420 comp rx=0 tx=0).stop 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 -- 172.21.15.33:0/2828339143 shutdown_connections 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 --2- 172.21.15.33:0/2828339143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca08103430 0x7fca08109db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 --2- 172.21.15.33:0/2828339143 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca08102a70 0x7fca08102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 --2- 172.21.15.33:0/2828339143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08101d10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 -- 172.21.15.33:0/2828339143 >> 172.21.15.33:0/2828339143 conn(0x7fca080fd1e0 msgr2=0x7fca080ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 -- 172.21.15.33:0/2828339143 shutdown_connections 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 -- 172.21.15.33:0/2828339143 wait complete. 2024-09-10T21:02:29.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.570+0000 7fca0e140640 1 Processor -- start 2024-09-10T21:02:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca0e140640 1 -- start start 2024-09-10T21:02:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca0e140640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca0e140640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca08102a70 0x7fca081994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca0e140640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca08103430 0x7fca0819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca0e140640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fca0810d5f0 con 0x7fca08101910 2024-09-10T21:02:29.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca0e140640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fca0810d470 con 0x7fca08103430 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca0e140640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca0810d770 con 0x7fca08102a70 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08198e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46426/0 (socket says 172.21.15.33:46426) 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 -- 172.21.15.33:0/2387315939 learned_addr learned my addr 172.21.15.33:0/2387315939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca06ffd640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca08102a70 0x7fca081994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca07fff640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca08103430 0x7fca0819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 -- 172.21.15.33:0/2387315939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca08102a70 msgr2=0x7fca081994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca08102a70 0x7fca081994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 -- 172.21.15.33:0/2387315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca08103430 msgr2=0x7fca0819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca08103430 0x7fca0819a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.571+0000 7fca077fe640 1 -- 172.21.15.33:0/2387315939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca0819a9c0 con 0x7fca08101910 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.572+0000 7fca077fe640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08198e90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8005ec0 tx=0x7fc9f8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:29.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.572+0000 7fca04ff9640 1 -- 172.21.15.33:0/2387315939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f803f030 con 0x7fca08101910 2024-09-10T21:02:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.572+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca0819ac50 con 0x7fca08101910 2024-09-10T21:02:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.572+0000 7fca04ff9640 1 -- 172.21.15.33:0/2387315939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9f8003cb0 con 0x7fca08101910 2024-09-10T21:02:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.572+0000 7fca04ff9640 1 -- 172.21.15.33:0/2387315939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8035070 con 0x7fca08101910 2024-09-10T21:02:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.572+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca081a11c0 con 0x7fca08101910 2024-09-10T21:02:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.573+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca081a14e0 con 0x7fca08101910 2024-09-10T21:02:29.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.574+0000 7fca04ff9640 1 -- 172.21.15.33:0/2387315939 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fc9f803b460 con 0x7fca08101910 2024-09-10T21:02:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.575+0000 7fca04ff9640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc9f00b5370 0x7fc9f00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.575+0000 7fca04ff9640 1 -- 172.21.15.33:0/2387315939 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fc9f8106a20 con 0x7fca08101910 2024-09-10T21:02:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.575+0000 7fca06ffd640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc9f00b5370 0x7fc9f00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:29.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.576+0000 7fca06ffd640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc9f00b5370 0x7fc9f00b7830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0058c0 tx=0x7fc9fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:29.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.578+0000 7fca04ff9640 1 -- 172.21.15.33:0/2387315939 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc9f80cd930 con 0x7fca08101910 2024-09-10T21:02:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:29 smithi174 ceph-mon[35635]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:02:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:29 smithi174 ceph-mon[35635]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-09-10T21:02:29.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.833+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fca081a16b0 con 0x7fca08101910 2024-09-10T21:02:29.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.836+0000 7fca04ff9640 1 -- 172.21.15.33:0/2387315939 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fc9f80cd230 con 0x7fca08101910 2024-09-10T21:02:29.840 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-09-10T21:02:29.840 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQDls+Bmh3/IMRAAZL5Wxc1xl3fStEZnWEJayw== 2024-09-10T21:02:29.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc9f00b5370 msgr2=0x7fc9f00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc9f00b5370 0x7fc9f00b7830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc0058c0 tx=0x7fc9fc00a2d0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 msgr2=0x7fca08198e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08198e90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8005ec0 tx=0x7fc9f8004360 comp rx=0 tx=0).stop 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 shutdown_connections 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc9f00b5370 0x7fc9f00b7830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fca08103430 0x7fca0819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca08102a70 0x7fca081994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 --2- 172.21.15.33:0/2387315939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca08101910 0x7fca08198e90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 >> 172.21.15.33:0/2387315939 conn(0x7fca080fd1e0 msgr2=0x7fca0806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 shutdown_connections 2024-09-10T21:02:29.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:29.839+0000 7fca0e140640 1 -- 172.21.15.33:0/2387315939 wait complete. 2024-09-10T21:02:30.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:30 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2387315939' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T21:02:30.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:30 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2387315939' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T21:02:30.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:30 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2387315939' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T21:02:30.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:30 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2387315939' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T21:02:30.587 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T21:02:30.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T21:02:30.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T21:02:30.627 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-10T21:02:30.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:30 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2387315939' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T21:02:30.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:30 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2387315939' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T21:02:30.928 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.398+0000 7f61bb685640 1 -- 172.21.15.33:0/2401846469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4109150 msgr2=0x7f61b410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.398+0000 7f61bb685640 1 --2- 172.21.15.33:0/2401846469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4109150 0x7f61b410f950 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009ca0 tx=0x7f61ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.398+0000 7f61bb685640 1 -- 172.21.15.33:0/2401846469 shutdown_connections 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.398+0000 7f61bb685640 1 --2- 172.21.15.33:0/2401846469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4109150 0x7f61b410f950 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.398+0000 7f61bb685640 1 --2- 172.21.15.33:0/2401846469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61b41087d0 0x7f61b4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.398+0000 7f61bb685640 1 --2- 172.21.15.33:0/2401846469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b4102dc0 0x7f61b41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.398+0000 7f61bb685640 1 -- 172.21.15.33:0/2401846469 >> 172.21.15.33:0/2401846469 conn(0x7f61b40fe830 msgr2=0x7f61b4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 -- 172.21.15.33:0/2401846469 shutdown_connections 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 -- 172.21.15.33:0/2401846469 wait complete. 2024-09-10T21:02:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 Processor -- start 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 -- start start 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4102dc0 0x7f61b41993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b41087d0 0x7f61b4199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61b4109150 0x7f61b419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4112e50 con 0x7f61b4102dc0 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4112cd0 con 0x7f61b41087d0 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61bb685640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61b4112fd0 con 0x7f61b4109150 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61b9421640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4102dc0 0x7f61b41993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61b9421640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4102dc0 0x7f61b41993e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46440/0 (socket says 172.21.15.33:46440) 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.399+0000 7f61b9421640 1 -- 172.21.15.33:0/1680428790 learned_addr learned my addr 172.21.15.33:0/1680428790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9c22640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61b4109150 0x7f61b419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b8c20640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b41087d0 0x7f61b4199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9421640 1 -- 172.21.15.33:0/1680428790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61b4109150 msgr2=0x7f61b419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9421640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61b4109150 0x7f61b419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9421640 1 -- 172.21.15.33:0/1680428790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b41087d0 msgr2=0x7f61b4199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9421640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b41087d0 0x7f61b4199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9421640 1 -- 172.21.15.33:0/1680428790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b419f490 con 0x7f61b4102dc0 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9c22640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61b4109150 0x7f61b419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b8c20640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b41087d0 0x7f61b4199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61b9421640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4102dc0 0x7f61b41993e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b4a0 tx=0x7f61b000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61aa7fc640 1 -- 172.21.15.33:0/1680428790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0014070 con 0x7f61b4102dc0 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b419f780 con 0x7f61b4102dc0 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61aa7fc640 1 -- 172.21.15.33:0/1680428790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0010ac0 con 0x7f61b4102dc0 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61aa7fc640 1 -- 172.21.15.33:0/1680428790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0017c30 con 0x7f61b4102dc0 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.400+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b41a5ec0 con 0x7f61b4102dc0 2024-09-10T21:02:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.401+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61b4103220 con 0x7f61b4102dc0 2024-09-10T21:02:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.403+0000 7f61aa7fc640 1 -- 172.21.15.33:0/1680428790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f61b0010610 con 0x7f61b4102dc0 2024-09-10T21:02:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.403+0000 7f61aa7fc640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61a00b53f0 0x7f61a00b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.403+0000 7f61b8c20640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61a00b53f0 0x7f61a00b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.403+0000 7f61aa7fc640 1 -- 172.21.15.33:0/1680428790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f61b00e5650 con 0x7f61b4102dc0 2024-09-10T21:02:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.404+0000 7f61b8c20640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61a00b53f0 0x7f61a00b78b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61a4006fd0 tx=0x7f61a4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.407+0000 7f61aa7fc640 1 -- 172.21.15.33:0/1680428790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f61b00ac460 con 0x7f61b4102dc0 2024-09-10T21:02:31.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.594+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f61b402e5c0 con 0x7f61b4102dc0 2024-09-10T21:02:31.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.595+0000 7f61aa7fc640 1 -- 172.21.15.33:0/1680428790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v48) ==== 86+0+133 (secure 0 0 0) 0x7f61b00abd60 con 0x7f61b4102dc0 2024-09-10T21:02:31.599 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61a00b53f0 msgr2=0x7f61a00b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61a00b53f0 0x7f61a00b78b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61a4006fd0 tx=0x7f61a4008040 comp rx=0 tx=0).stop 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4102dc0 msgr2=0x7f61b41993e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4102dc0 0x7f61b41993e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f61b000b4a0 tx=0x7f61b000b960 comp rx=0 tx=0).stop 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 shutdown_connections 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61a00b53f0 0x7f61a00b78b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61b4109150 0x7f61b419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61b41087d0 0x7f61b4199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 --2- 172.21.15.33:0/1680428790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61b4102dc0 0x7f61b41993e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 >> 172.21.15.33:0/1680428790 conn(0x7f61b40fe830 msgr2=0x7f61b4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 shutdown_connections 2024-09-10T21:02:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:31.597+0000 7f61bb685640 1 -- 172.21.15.33:0/1680428790 wait complete. 2024-09-10T21:02:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:31 smithi033 ceph-mon[30860]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T21:02:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:31 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1680428790' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T21:02:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:31 smithi174 ceph-mon[35635]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T21:02:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:31 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1680428790' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T21:02:32.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:31 smithi104 ceph-mon[35325]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 11 op/s 2024-09-10T21:02:32.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:31 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1680428790' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T21:02:32.359 INFO:teuthology.orchestra.run.smithi033.stdout:[{"version":48,"timestamp":"2024-09-10T21:01:51.107425+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-10T21:02:32.360 INFO:tasks.ceph_manager:config epoch is 48 2024-09-10T21:02:32.360 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T21:02:32.360 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T21:02:32.360 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T21:02:32.625 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 -- 172.21.15.33:0/900411896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 msgr2=0x7fbc001020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 --2- 172.21.15.33:0/900411896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc001020b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009900 tx=0x7fbbf002a410 comp rx=0 tx=0).stop 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 -- 172.21.15.33:0/900411896 shutdown_connections 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 --2- 172.21.15.33:0/900411896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc00108060 0x7fbc0010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 --2- 172.21.15.33:0/900411896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc001076e0 0x7fbc00107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 --2- 172.21.15.33:0/900411896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc001020b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 -- 172.21.15.33:0/900411896 >> 172.21.15.33:0/900411896 conn(0x7fbc000fd700 msgr2=0x7fbc000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 -- 172.21.15.33:0/900411896 shutdown_connections 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 -- 172.21.15.33:0/900411896 wait complete. 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 Processor -- start 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.124+0000 7fbc06a70640 1 -- start start 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc06a70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc00198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc06a70640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc001076e0 0x7fbc001986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc06a70640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc00108060 0x7fbc0019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc06a70640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00111d20 con 0x7fbc00101cd0 2024-09-10T21:02:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc06a70640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00111ba0 con 0x7fbc00108060 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc06a70640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc00111ea0 con 0x7fbc001076e0 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc00198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc00198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46456/0 (socket says 172.21.15.33:46456) 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 -- 172.21.15.33:0/3212814842 learned_addr learned my addr 172.21.15.33:0/3212814842 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbbfffff640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc001076e0 0x7fbc001986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 -- 172.21.15.33:0/3212814842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc001076e0 msgr2=0x7fbc001986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc001076e0 0x7fbc001986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 -- 172.21.15.33:0/3212814842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc00108060 msgr2=0x7fbc0019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0500d640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc00108060 0x7fbc0019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc00108060 0x7fbc0019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 -- 172.21.15.33:0/3212814842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0019e220 con 0x7fbc00101cd0 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc0480c640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc00198170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005940 tx=0x7fbbf00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:33.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbbfdffb640 1 -- 172.21.15.33:0/3212814842 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf003f030 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0019e4b0 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbbfdffb640 1 -- 172.21.15.33:0/3212814842 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf0003cb0 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.125+0000 7fbbfdffb640 1 -- 172.21.15.33:0/3212814842 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0035070 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.126+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0006b490 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.127+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc001020b0 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.128+0000 7fbbfdffb640 1 -- 172.21.15.33:0/3212814842 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fbbf003bbf0 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.129+0000 7fbbfdffb640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbbe80b7580 0x7fbbe80b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.129+0000 7fbbfffff640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbbe80b7580 0x7fbbe80b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.129+0000 7fbbfdffb640 1 -- 172.21.15.33:0/3212814842 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fbbf0106b60 con 0x7fbc00101cd0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.129+0000 7fbbfffff640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbbe80b7580 0x7fbbe80b9a40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4005fd0 tx=0x7fbbf4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.132+0000 7fbbfdffb640 1 -- 172.21.15.33:0/3212814842 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbbf00cd970 con 0x7fbc00101cd0 2024-09-10T21:02:33.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.367+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbc0006bf10 con 0x7fbc00101cd0 2024-09-10T21:02:33.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.372+0000 7fbbfdffb640 1 -- 172.21.15.33:0/3212814842 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+294215 (secure 0 0 0) 0x7fbbf00cd270 con 0x7fbc00101cd0 2024-09-10T21:02:33.380 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:33.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.375+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbbe80b7580 msgr2=0x7fbbe80b9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:33.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.375+0000 7fbc06a70640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbbe80b7580 0x7fbbe80b9a40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4005fd0 tx=0x7fbbf4005850 comp rx=0 tx=0).stop 2024-09-10T21:02:33.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.375+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 msgr2=0x7fbc00198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:33.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.375+0000 7fbc06a70640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc00198170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0005940 tx=0x7fbbf00042f0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 shutdown_connections 2024-09-10T21:02:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbbe80b7580 0x7fbbe80b9a40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc00108060 0x7fbc0019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc001076e0 0x7fbc001986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 --2- 172.21.15.33:0/3212814842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbc00101cd0 0x7fbc00198170 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 >> 172.21.15.33:0/3212814842 conn(0x7fbc000fd700 msgr2=0x7fbc000ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 shutdown_connections 2024-09-10T21:02:33.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:33.376+0000 7fbc06a70640 1 -- 172.21.15.33:0/3212814842 wait complete. 2024-09-10T21:02:34.134 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:33 smithi033 ceph-mon[30860]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T21:02:34.134 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:33 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3212814842' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T21:02:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:33 smithi174 ceph-mon[35635]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T21:02:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:33 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3212814842' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T21:02:34.166 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":3102318098},{"type":"v1","addr":"172.21.15.33:6801","nonce":3102318098}]},"active_addr":"172.21.15.33:6801/3102318098","active_change":"2024-09-10T20:54:39.992688+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24136,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24152,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2582306509}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":4010722974}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":1011618733}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2607078036}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":251255603}]}]} 2024-09-10T21:02:34.172 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T21:02:34.172 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T21:02:34.172 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:02:34.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:33 smithi104 ceph-mon[35325]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.2 KiB/s wr, 10 op/s 2024-09-10T21:02:34.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:33 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3212814842' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T21:02:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 -- 172.21.15.33:0/3719159946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630103ce0 msgr2=0x7fa6301040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 --2- 172.21.15.33:0/3719159946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630103ce0 0x7fa6301040c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa620009900 tx=0x7fa62002a410 comp rx=0 tx=0).stop 2024-09-10T21:02:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 -- 172.21.15.33:0/3719159946 shutdown_connections 2024-09-10T21:02:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 --2- 172.21.15.33:0/3719159946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa63010d140 0x7fa63010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:34.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 --2- 172.21.15.33:0/3719159946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa630104600 0x7fa63010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 --2- 172.21.15.33:0/3719159946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630103ce0 0x7fa6301040c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 -- 172.21.15.33:0/3719159946 >> 172.21.15.33:0/3719159946 conn(0x7fa6300fd700 msgr2=0x7fa6300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 -- 172.21.15.33:0/3719159946 shutdown_connections 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.932+0000 7fa635c4b640 1 -- 172.21.15.33:0/3719159946 wait complete. 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 Processor -- start 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 -- start start 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa630103ce0 0x7fa6301992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630104600 0x7fa630199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d140 0x7fa63019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa630112db0 con 0x7fa63010d140 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa630112c30 con 0x7fa630104600 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa635c4b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa630112f30 con 0x7fa630103ce0 2024-09-10T21:02:34.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d140 0x7fa63019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d140 0x7fa63019eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46470/0 (socket says 172.21.15.33:46470) 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.933+0000 7fa62ffff640 1 -- 172.21.15.33:0/4253164029 learned_addr learned my addr 172.21.15.33:0/4253164029 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa630103ce0 0x7fa6301992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62effd640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630104600 0x7fa630199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62ffff640 1 -- 172.21.15.33:0/4253164029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa630103ce0 msgr2=0x7fa6301992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62ffff640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa630103ce0 0x7fa6301992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62ffff640 1 -- 172.21.15.33:0/4253164029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630104600 msgr2=0x7fa630199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62ffff640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630104600 0x7fa630199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62ffff640 1 -- 172.21.15.33:0/4253164029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63019f370 con 0x7fa63010d140 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa630103ce0 0x7fa6301992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62ffff640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d140 0x7fa63019eba0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa62800eaa0 tx=0x7fa62800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62cff9640 1 -- 172.21.15.33:0/4253164029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62800cdb0 con 0x7fa63010d140 2024-09-10T21:02:34.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62cff9640 1 -- 172.21.15.33:0/4253164029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa628005020 con 0x7fa63010d140 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63019f660 con 0x7fa63010d140 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa62cff9640 1 -- 172.21.15.33:0/4253164029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280116d0 con 0x7fa63010d140 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.934+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6301a5d20 con 0x7fa63010d140 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.935+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6301040c0 con 0x7fa63010d140 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.937+0000 7fa62cff9640 1 -- 172.21.15.33:0/4253164029 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fa62800f040 con 0x7fa63010d140 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.937+0000 7fa62cff9640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b5370 0x7fa6180b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.937+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b5370 0x7fa6180b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.938+0000 7fa62cff9640 1 -- 172.21.15.33:0/4253164029 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fa6280e5ee0 con 0x7fa63010d140 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.938+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b5370 0x7fa6180b7830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa620031440 tx=0x7fa620005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:34.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:34.940+0000 7fa62cff9640 1 -- 172.21.15.33:0/4253164029 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa628016030 con 0x7fa63010d140 2024-09-10T21:02:35.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.128+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa63019a3f0 con 0x7fa63010d140 2024-09-10T21:02:35.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.129+0000 7fa62cff9640 1 -- 172.21.15.33:0/4253164029 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22281 (secure 0 0 0) 0x7fa6280acdf0 con 0x7fa63010d140 2024-09-10T21:02:35.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:35.134 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:01.445442+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:02:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b5370 msgr2=0x7fa6180b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b5370 0x7fa6180b7830 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa620031440 tx=0x7fa620005a20 comp rx=0 tx=0).stop 2024-09-10T21:02:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d140 msgr2=0x7fa63019eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d140 0x7fa63019eba0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa62800eaa0 tx=0x7fa62800ef60 comp rx=0 tx=0).stop 2024-09-10T21:02:35.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 shutdown_connections 2024-09-10T21:02:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b5370 0x7fa6180b7830 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d140 0x7fa63019eba0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630104600 0x7fa630199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 --2- 172.21.15.33:0/4253164029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa630103ce0 0x7fa6301992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 >> 172.21.15.33:0/4253164029 conn(0x7fa6300fd700 msgr2=0x7fa6300fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 shutdown_connections 2024-09-10T21:02:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:35.132+0000 7fa635c4b640 1 -- 172.21.15.33:0/4253164029 wait complete. 2024-09-10T21:02:35.957 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T21:02:35.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:02:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:35 smithi174 ceph-mon[35635]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-10T21:02:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:35 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4253164029' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:35 smithi033 ceph-mon[30860]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-10T21:02:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:35 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4253164029' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:36.223 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:35 smithi104 ceph-mon[35325]: pgmap v302: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-09-10T21:02:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:35 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4253164029' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 -- 172.21.15.33:0/396696328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac141087d0 msgr2=0x7fac14108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 --2- 172.21.15.33:0/396696328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac141087d0 0x7fac14108c10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fabfc009900 tx=0x7fabfc02a410 comp rx=0 tx=0).stop 2024-09-10T21:02:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 -- 172.21.15.33:0/396696328 shutdown_connections 2024-09-10T21:02:36.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 --2- 172.21.15.33:0/396696328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac14109150 0x7fac1410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 --2- 172.21.15.33:0/396696328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac141087d0 0x7fac14108c10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 --2- 172.21.15.33:0/396696328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac14102dc0 0x7fac141031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 -- 172.21.15.33:0/396696328 >> 172.21.15.33:0/396696328 conn(0x7fac140fe830 msgr2=0x7fac14100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 -- 172.21.15.33:0/396696328 shutdown_connections 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.676+0000 7fac195f9640 1 -- 172.21.15.33:0/396696328 wait complete. 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 Processor -- start 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 -- start start 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac14102dc0 0x7fac141992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac141087d0 0x7fac14199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac14109150 0x7fac1419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac14112e10 con 0x7fac14109150 2024-09-10T21:02:36.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac14112c90 con 0x7fac14102dc0 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac195f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac14112f90 con 0x7fac141087d0 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac14109150 0x7fac1419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac137fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac14109150 0x7fac1419ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46486/0 (socket says 172.21.15.33:46486) 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac137fe640 1 -- 172.21.15.33:0/329750570 learned_addr learned my addr 172.21.15.33:0/329750570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac12ffd640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac14102dc0 0x7fac141992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.677+0000 7fac0a7fc640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac141087d0 0x7fac14199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac137fe640 1 -- 172.21.15.33:0/329750570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac141087d0 msgr2=0x7fac14199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac137fe640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac141087d0 0x7fac14199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac137fe640 1 -- 172.21.15.33:0/329750570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac14102dc0 msgr2=0x7fac141992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac137fe640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac14102dc0 0x7fac141992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac137fe640 1 -- 172.21.15.33:0/329750570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1419f390 con 0x7fac14109150 2024-09-10T21:02:36.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac12ffd640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac14102dc0 0x7fac141992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac137fe640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac14109150 0x7fac1419ebc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b970 tx=0x7fac0c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac10ff9640 1 -- 172.21.15.33:0/329750570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c019070 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac10ff9640 1 -- 172.21.15.33:0/329750570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac0c014440 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac10ff9640 1 -- 172.21.15.33:0/329750570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0075d0 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1419f680 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.678+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac141a5d40 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.679+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac141031a0 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.680+0000 7fac10ff9640 1 -- 172.21.15.33:0/329750570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fac0c00cb40 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.681+0000 7fac10ff9640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac000b5370 0x7fac000b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.681+0000 7fac12ffd640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac000b5370 0x7fac000b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.681+0000 7fac10ff9640 1 -- 172.21.15.33:0/329750570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7fac0c0e58e0 con 0x7fac14109150 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.682+0000 7fac12ffd640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac000b5370 0x7fac000b7830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac040058c0 tx=0x7fac04005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:36.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.684+0000 7fac10ff9640 1 -- 172.21.15.33:0/329750570 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac0c0ac7f0 con 0x7fac14109150 2024-09-10T21:02:36.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.872+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac1419a410 con 0x7fac14109150 2024-09-10T21:02:36.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.873+0000 7fac10ff9640 1 -- 172.21.15.33:0/329750570 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22281 (secure 0 0 0) 0x7fac1419a410 con 0x7fac14109150 2024-09-10T21:02:36.877 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:02:36.878 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":75,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:01.445442+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.875+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac000b5370 msgr2=0x7fac000b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.875+0000 7fac195f9640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac000b5370 0x7fac000b7830 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fac040058c0 tx=0x7fac04005850 comp rx=0 tx=0).stop 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.875+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac14109150 msgr2=0x7fac1419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.875+0000 7fac195f9640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac14109150 0x7fac1419ebc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00b970 tx=0x7fac0c00be30 comp rx=0 tx=0).stop 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 shutdown_connections 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac000b5370 0x7fac000b7830 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac14109150 0x7fac1419ebc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac141087d0 0x7fac14199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 --2- 172.21.15.33:0/329750570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac14102dc0 0x7fac141992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 >> 172.21.15.33:0/329750570 conn(0x7fac140fe830 msgr2=0x7fac14100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:36.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 shutdown_connections 2024-09-10T21:02:36.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:36.876+0000 7fac195f9640 1 -- 172.21.15.33:0/329750570 wait complete. 2024-09-10T21:02:37.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T21:02:37.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T21:02:37.511 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T21:02:37.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T21:02:37.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T21:02:37.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T21:02:37.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T21:02:37.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T21:02:37.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-10T21:02:37.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-10T21:02:37.514 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-10T21:02:37.515 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-10T21:02:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:37 smithi174 ceph-mon[35635]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T21:02:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:37 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/329750570' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:38.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:37 smithi033 ceph-mon[30860]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T21:02:38.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:37 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/329750570' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:38.293 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.293 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.293 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.294 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.298 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.299 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:37 smithi104 ceph-mon[35325]: pgmap v303: 97 pgs: 97 active+clean; 641 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.1 KiB/s wr, 10 op/s 2024-09-10T21:02:38.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:37 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/329750570' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:02:38.322 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:38.322 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:40.056 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:39 smithi033 ceph-mon[30860]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T21:02:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:39 smithi174 ceph-mon[35635]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T21:02:40.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:39 smithi104 ceph-mon[35325]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]} : dispatch 2024-09-10T21:02:40.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:40 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]} : dispatch 2024-09-10T21:02:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:40 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/3724967638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4109130 msgr2=0x7ffaf410f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 --2- 172.21.15.33:0/3724967638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4109130 0x7ffaf410f930 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009ca0 tx=0x7ffaec02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/3724967638 shutdown_connections 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 --2- 172.21.15.33:0/3724967638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4109130 0x7ffaf410f930 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 --2- 172.21.15.33:0/3724967638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf41087b0 0x7ffaf4108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 --2- 172.21.15.33:0/3724967638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf4102da0 0x7ffaf4103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/3724967638 >> 172.21.15.33:0/3724967638 conn(0x7ffaf40fe810 msgr2=0x7ffaf4100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/3724967638 shutdown_connections 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.222+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/3724967638 wait complete. 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 Processor -- start 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 -- start start 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4102da0 0x7ffaf4109920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf41087b0 0x7ffaf4109e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf4109130 0x7ffaf410a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4112e30 con 0x7ffaf4102da0 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4112cb0 con 0x7ffaf41087b0 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf9d4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf4112fb0 con 0x7ffaf4109130 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4102da0 0x7ffaf4109920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4102da0 0x7ffaf4109920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44424/0 (socket says 172.21.15.33:44424) 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf37fe640 1 -- 172.21.15.33:0/2025038566 learned_addr learned my addr 172.21.15.33:0/2025038566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf2ffd640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf41087b0 0x7ffaf4109e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf3fff640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf4109130 0x7ffaf410a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf37fe640 1 -- 172.21.15.33:0/2025038566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf4109130 msgr2=0x7ffaf410a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf37fe640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf4109130 0x7ffaf410a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf37fe640 1 -- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf41087b0 msgr2=0x7ffaf4109e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.223+0000 7ffaf37fe640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf41087b0 0x7ffaf4109e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.224+0000 7ffaf37fe640 1 -- 172.21.15.33:0/2025038566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf41abbe0 con 0x7ffaf4102da0 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.224+0000 7ffaf2ffd640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffaf41087b0 0x7ffaf4109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.224+0000 7ffaf3fff640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffaf4109130 0x7ffaf410a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.224+0000 7ffaf37fe640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4102da0 0x7ffaf4109920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffae400b4a0 tx=0x7ffae400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.224+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4014070 con 0x7ffaf4102da0 2024-09-10T21:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.224+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf41abed0 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.224+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4010ac0 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.225+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4017c30 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.225+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf41ac450 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.225+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaf40638e0 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.226+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ffae4010610 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.227+0000 7ffaf0ff9640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffadc0b53f0 0x7ffadc0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.227+0000 7ffaf2ffd640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffadc0b53f0 0x7ffadc0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.227+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8240+0+0 (secure 0 0 0) 0x7ffae40e5620 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.228+0000 7ffaf0ff9640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] conn(0x7ffadc0bef00 0x7ffadc0c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.228+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 --> [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffae4012e80 con 0x7ffadc0bef00 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.228+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ffae40ec050 con 0x7ffaf4102da0 2024-09-10T21:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.228+0000 7ffaf2ffd640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffadc0b53f0 0x7ffadc0b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffae8006fd0 tx=0x7ffae8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.228+0000 7ffaf3fff640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] conn(0x7ffadc0bef00 0x7ffadc0c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.228+0000 7ffaf3fff640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] conn(0x7ffadc0bef00 0x7ffadc0c13a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.229+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== osd.10 v2:172.21.15.104:6824/575213460 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffae4012e80 con 0x7ffadc0bef00 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.243+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 --> [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffaf4063d90 con 0x7ffadc0bef00 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.243+0000 7ffaf0ff9640 1 -- 172.21.15.33:0/2025038566 <== osd.10 v2:172.21.15.104:6824/575213460 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffaf4063d90 con 0x7ffadc0bef00 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] conn(0x7ffadc0bef00 msgr2=0x7ffadc0c13a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] conn(0x7ffadc0bef00 0x7ffadc0c13a0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffadc0b53f0 msgr2=0x7ffadc0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ffadc0b53f0 0x7ffadc0b78b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ffae8006fd0 tx=0x7ffae8008040 comp rx=0 tx=0).stop 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4102da0 msgr2=0x7ffaf4109920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 --2- 172.21.15.33:0/2025038566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffaf4102da0 0x7ffaf4109920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ffae400b4a0 tx=0x7ffae400b960 comp rx=0 tx=0).stop 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf3fff640 1 -- 172.21.15.33:0/2025038566 reap_dead start 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 shutdown_connections 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 >> 172.21.15.33:0/2025038566 conn(0x7ffaf40fe810 msgr2=0x7ffaf4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 shutdown_connections 2024-09-10T21:02:41.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.244+0000 7ffaf9d4c640 1 -- 172.21.15.33:0/2025038566 wait complete. 2024-09-10T21:02:41.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]} : dispatch 2024-09-10T21:02:41.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]} : dispatch 2024-09-10T21:02:41.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]} : dispatch 2024-09-10T21:02:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]} : dispatch 2024-09-10T21:02:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]} : dispatch 2024-09-10T21:02:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]} : dispatch 2024-09-10T21:02:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]} : dispatch 2024-09-10T21:02:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]} : dispatch 2024-09-10T21:02:41.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:40 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]} : dispatch 2024-09-10T21:02:41.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 -- 172.21.15.33:0/136610015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c1087d0 msgr2=0x7f8e4c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 --2- 172.21.15.33:0/136610015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c1087d0 0x7f8e4c108c10 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009900 tx=0x7f8e3c02a410 comp rx=0 tx=0).stop 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 -- 172.21.15.33:0/136610015 shutdown_connections 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 --2- 172.21.15.33:0/136610015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e4c109150 0x7f8e4c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 --2- 172.21.15.33:0/136610015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c1087d0 0x7f8e4c108c10 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 --2- 172.21.15.33:0/136610015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c102dc0 0x7f8e4c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 -- 172.21.15.33:0/136610015 >> 172.21.15.33:0/136610015 conn(0x7f8e4c0fe830 msgr2=0x7f8e4c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 -- 172.21.15.33:0/136610015 shutdown_connections 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.238+0000 7f8e5397d640 1 -- 172.21.15.33:0/136610015 wait complete. 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 Processor -- start 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 -- start start 2024-09-10T21:02:41.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c102dc0 0x7f8e4c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e4c1087d0 0x7f8e4c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c109150 0x7f8e4c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c112e10 con 0x7f8e4c102dc0 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c112c90 con 0x7f8e4c1087d0 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e5397d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c112f90 con 0x7f8e4c109150 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e51719640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c102dc0 0x7f8e4c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e51719640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c102dc0 0x7f8e4c1992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44400/0 (socket says 172.21.15.33:44400) 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.239+0000 7f8e51719640 1 -- 172.21.15.33:0/3958005854 learned_addr learned my addr 172.21.15.33:0/3958005854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e51f1a640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c109150 0x7f8e4c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e50f18640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e4c1087d0 0x7f8e4c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e51719640 1 -- 172.21.15.33:0/3958005854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c109150 msgr2=0x7f8e4c19eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e51719640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c109150 0x7f8e4c19eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e51719640 1 -- 172.21.15.33:0/3958005854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e4c1087d0 msgr2=0x7f8e4c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e51719640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8e4c1087d0 0x7f8e4c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e51719640 1 -- 172.21.15.33:0/3958005854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c19f370 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e51719640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c102dc0 0x7f8e4c1992c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d770 tx=0x7f8e4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48017070 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e480054d0 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c19f660 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e48004c30 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.240+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a5d20 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.241+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e4c0638e0 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.243+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8e48004dd0 con 0x7f8e4c102dc0 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.243+0000 7f8e427fc640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8e380b53f0 0x7f8e380b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.243+0000 7f8e50f18640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8e380b53f0 0x7f8e380b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.244+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f8e480e5960 con 0x7f8e4c102dc0 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.244+0000 7f8e427fc640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] conn(0x7f8e380beb30 0x7f8e380c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.244+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 --> [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e48021e50 con 0x7f8e380beb30 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.244+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f8e480e5e40 con 0x7f8e4c102dc0 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.244+0000 7f8e51f1a640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] conn(0x7f8e380beb30 0x7f8e380c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.244+0000 7f8e50f18640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8e380b53f0 0x7f8e380b78b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c031440 tx=0x7f8e3c005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.244+0000 7f8e51f1a640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] conn(0x7f8e380beb30 0x7f8e380c0f50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.245+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== osd.3 v2:172.21.15.33:6810/2053264902 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e48021e50 con 0x7f8e380beb30 2024-09-10T21:02:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.259+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 --> [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e4c063d60 con 0x7f8e380beb30 2024-09-10T21:02:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.259+0000 7f8e427fc640 1 -- 172.21.15.33:0/3958005854 <== osd.3 v2:172.21.15.33:6810/2053264902 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e4c063d60 con 0x7f8e380beb30 2024-09-10T21:02:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] conn(0x7f8e380beb30 msgr2=0x7f8e380c0f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] conn(0x7f8e380beb30 0x7f8e380c0f50 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8e380b53f0 msgr2=0x7f8e380b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8e380b53f0 0x7f8e380b78b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c031440 tx=0x7f8e3c005b50 comp rx=0 tx=0).stop 2024-09-10T21:02:41.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c102dc0 msgr2=0x7f8e4c1992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 --2- 172.21.15.33:0/3958005854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8e4c102dc0 0x7f8e4c1992c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8e4800d770 tx=0x7f8e4800dc30 comp rx=0 tx=0).stop 2024-09-10T21:02:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e51f1a640 1 -- 172.21.15.33:0/3958005854 reap_dead start 2024-09-10T21:02:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 shutdown_connections 2024-09-10T21:02:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 >> 172.21.15.33:0/3958005854 conn(0x7f8e4c0fe830 msgr2=0x7f8e4c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 shutdown_connections 2024-09-10T21:02:41.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.260+0000 7f8e5397d640 1 -- 172.21.15.33:0/3958005854 wait complete. 2024-09-10T21:02:41.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 -- 172.21.15.33:0/311372202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 msgr2=0x7f420c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 --2- 172.21.15.33:0/311372202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 0x7f420c107b20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4200009ca0 tx=0x7f420002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 -- 172.21.15.33:0/311372202 shutdown_connections 2024-09-10T21:02:41.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 --2- 172.21.15.33:0/311372202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f420c108060 0x7f420c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 --2- 172.21.15.33:0/311372202 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 0x7f420c107b20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 --2- 172.21.15.33:0/311372202 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420c101cd0 0x7f420c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 -- 172.21.15.33:0/311372202 >> 172.21.15.33:0/311372202 conn(0x7f420c0fd720 msgr2=0x7f420c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 -- 172.21.15.33:0/311372202 shutdown_connections 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.136+0000 7f4210c37640 1 -- 172.21.15.33:0/311372202 wait complete. 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 Processor -- start 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 -- start start 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420c101cd0 0x7f420c19a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 0x7f420c19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f420c108060 0x7f420c1a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f420c111d20 con 0x7f420c1076e0 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f420c111ba0 con 0x7f420c101cd0 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.137+0000 7f4210c37640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f420c111ea0 con 0x7f420c108060 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 0x7f420c19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 0x7f420c19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44380/0 (socket says 172.21.15.33:44380) 2024-09-10T21:02:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 -- 172.21.15.33:0/3867509469 learned_addr learned my addr 172.21.15.33:0/3867509469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f420a59c640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420c101cd0 0x7f420c19a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f420ad9d640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f420c108060 0x7f420c1a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 -- 172.21.15.33:0/3867509469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f420c108060 msgr2=0x7f420c1a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f420c108060 0x7f420c1a51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 -- 172.21.15.33:0/3867509469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420c101cd0 msgr2=0x7f420c19a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f420c101cd0 0x7f420c19a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 -- 172.21.15.33:0/3867509469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c1a56e0 con 0x7f420c1076e0 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4209d9b640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 0x7f420c19a660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4200006010 tx=0x7f42000319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4200003bf0 con 0x7f420c1076e0 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c1a58b0 con 0x7f420c1076e0 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4200003d90 con 0x7f420c1076e0 2024-09-10T21:02:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420002fc20 con 0x7f420c1076e0 2024-09-10T21:02:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.138+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c1a5d40 con 0x7f420c1076e0 2024-09-10T21:02:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.139+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f420c0638e0 con 0x7f420c1076e0 2024-09-10T21:02:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.141+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f42000046a0 con 0x7f420c1076e0 2024-09-10T21:02:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f41fb7fe640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41f40b5370 0x7f41f40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f420a59c640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41f40b5370 0x7f41f40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7979+0+0 (secure 0 0 0) 0x7f4200107260 con 0x7f420c1076e0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f41fb7fe640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] conn(0x7f41f40beab0 0x7f41f40c0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 --> [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f420003ee50 con 0x7f41f40beab0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) ==== 24+0+0 (secure 0 0 0) 0x7f4200107610 con 0x7f420c1076e0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f420ad9d640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] conn(0x7f41f40beab0 0x7f41f40c0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f420a59c640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41f40b5370 0x7f41f40b7830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00bbe0 tx=0x7f41fc00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.142+0000 7f420ad9d640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] conn(0x7f41f40beab0 0x7f41f40c0ed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.143+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== osd.6 v2:172.21.15.33:6818/3359699940 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f420003ee50 con 0x7f41f40beab0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.157+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 --> [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f420c063d60 con 0x7f41f40beab0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.157+0000 7f41fb7fe640 1 -- 172.21.15.33:0/3867509469 <== osd.6 v2:172.21.15.33:6818/3359699940 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f420c063d60 con 0x7f41f40beab0 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] conn(0x7f41f40beab0 msgr2=0x7f41f40c0ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] conn(0x7f41f40beab0 0x7f41f40c0ed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41f40b5370 msgr2=0x7f41f40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41f40b5370 0x7f41f40b7830 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00bbe0 tx=0x7f41fc00b390 comp rx=0 tx=0).stop 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 msgr2=0x7f420c19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 --2- 172.21.15.33:0/3867509469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f420c1076e0 0x7f420c19a660 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4200006010 tx=0x7f42000319f0 comp rx=0 tx=0).stop 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f420ad9d640 1 -- 172.21.15.33:0/3867509469 reap_dead start 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 shutdown_connections 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 >> 172.21.15.33:0/3867509469 conn(0x7f420c0fd720 msgr2=0x7f420c0ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 shutdown_connections 2024-09-10T21:02:41.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:40.158+0000 7f4210c37640 1 -- 172.21.15.33:0/3867509469 wait complete. 2024-09-10T21:02:42.085 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T21:02:42.086 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:41 smithi033 ceph-mon[30860]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T21:02:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-10T21:02:42.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-09-10T21:02:42.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T21:02:42.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:41 smithi174 ceph-mon[35635]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T21:02:42.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2d", "id": [3, 9]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2024-09-10T21:02:42.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:41 smithi104 ceph-mon[35325]: osdmap e76: 12 total, 12 up, 12 in 2024-09-10T21:02:42.496 INFO:teuthology.orchestra.run.smithi033.stdout:176093659182 2024-09-10T21:02:42.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 -- 172.21.15.33:0/960851205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 msgr2=0x7f8f1c10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 --2- 172.21.15.33:0/960851205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 0x7f8f1c10e860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009ca0 tx=0x7f8f1402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 -- 172.21.15.33:0/960851205 shutdown_connections 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 --2- 172.21.15.33:0/960851205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 0x7f8f1c10e860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 --2- 172.21.15.33:0/960851205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f1c1076e0 0x7f8f1c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 --2- 172.21.15.33:0/960851205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f1c101cd0 0x7f8f1c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 -- 172.21.15.33:0/960851205 >> 172.21.15.33:0/960851205 conn(0x7f8f1c0fd720 msgr2=0x7f8f1c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 -- 172.21.15.33:0/960851205 shutdown_connections 2024-09-10T21:02:42.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 -- 172.21.15.33:0/960851205 wait complete. 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 Processor -- start 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 -- start start 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f1c101cd0 0x7f8f1c19a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f1c1076e0 0x7f8f1c19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.113+0000 7f8f2250d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 0x7f8f1c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f2250d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c111d70 con 0x7f8f1c108060 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f2250d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c111bf0 con 0x7f8f1c101cd0 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f2250d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f1c111ef0 con 0x7f8f1c1076e0 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 0x7f8f1c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 0x7f8f1c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44410/0 (socket says 172.21.15.33:44410) 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 -- 172.21.15.33:0/1673872669 learned_addr learned my addr 172.21.15.33:0/1673872669 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:42.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f13fff640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f1c1076e0 0x7f8f1c19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:42.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f1bfff640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f1c101cd0 0x7f8f1c19a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:42.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 -- 172.21.15.33:0/1673872669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f1c1076e0 msgr2=0x7f8f1c19a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:42.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f1c1076e0 0x7f8f1c19a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 -- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f1c101cd0 msgr2=0x7f8f1c19a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:42.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f1c101cd0 0x7f8f1c19a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 -- 172.21.15.33:0/1673872669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f1c1a5440 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f1bfff640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8f1c101cd0 0x7f8f1c19a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f20aaa640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 0x7f8f1c19ab00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009430 tx=0x7f8f140319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14003c30 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f14003dd0 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f1c1a56d0 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1402faa0 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.114+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f1c1a5c60 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.115+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8f1c104ff0 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.117+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8f1402fc40 con 0x7f8f1c108060 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.117+0000 7f8f19ffb640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f080b53f0 0x7f8f080b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.118+0000 7f8f1bfff640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f080b53f0 0x7f8f080b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:42.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.118+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 8240+0+0 (secure 0 0 0) 0x7f8f141072d0 con 0x7f8f1c108060 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.118+0000 7f8f19ffb640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] conn(0x7f8f080bef10 0x7f8f080c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.118+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 --> [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8f080c1a80 con 0x7f8f080bef10 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.118+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8f14107700 con 0x7f8f1c108060 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.118+0000 7f8f1bfff640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f080b53f0 0x7f8f080b78b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0058c0 tx=0x7f8f0c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.118+0000 7f8f13fff640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] conn(0x7f8f080bef10 0x7f8f080c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.119+0000 7f8f13fff640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] conn(0x7f8f080bef10 0x7f8f080c13b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.119+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== osd.1 v2:172.21.15.104:6800/2112783762 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8f080c1a80 con 0x7f8f080bef10 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.133+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 --> [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8f1c102f80 con 0x7f8f080bef10 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f19ffb640 1 -- 172.21.15.33:0/1673872669 <== osd.1 v2:172.21.15.104:6800/2112783762 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8f1c102f80 con 0x7f8f080bef10 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] conn(0x7f8f080bef10 msgr2=0x7f8f080c13b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] conn(0x7f8f080bef10 0x7f8f080c13b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f080b53f0 msgr2=0x7f8f080b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8f080b53f0 0x7f8f080b78b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c0058c0 tx=0x7f8f0c005850 comp rx=0 tx=0).stop 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 msgr2=0x7f8f1c19ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 --2- 172.21.15.33:0/1673872669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f1c108060 0x7f8f1c19ab00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8f14009430 tx=0x7f8f140319f0 comp rx=0 tx=0).stop 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f20aaa640 1 -- 172.21.15.33:0/1673872669 reap_dead start 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 shutdown_connections 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 >> 172.21.15.33:0/1673872669 conn(0x7f8f1c0fd720 msgr2=0x7f8f1c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 shutdown_connections 2024-09-10T21:02:42.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:41.134+0000 7f8f2250d640 1 -- 172.21.15.33:0/1673872669 wait complete. 2024-09-10T21:02:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:42 smithi174 ceph-mon[35635]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T21:02:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:42 smithi174 ceph-mon[35635]: pgmap v308: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 639 B/s wr, 10 op/s 2024-09-10T21:02:43.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:42 smithi033 ceph-mon[30860]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T21:02:43.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:42 smithi033 ceph-mon[30860]: pgmap v308: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 639 B/s wr, 10 op/s 2024-09-10T21:02:43.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:42 smithi104 ceph-mon[35325]: osdmap e77: 12 total, 12 up, 12 in 2024-09-10T21:02:43.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:42 smithi104 ceph-mon[35325]: pgmap v308: 97 pgs: 2 peering, 95 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 639 B/s wr, 10 op/s 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 -- 172.21.15.33:0/1538408877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc1087d0 msgr2=0x7fe9fc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 --2- 172.21.15.33:0/1538408877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc1087d0 0x7fe9fc108c10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0009900 tx=0x7fe9f002a410 comp rx=0 tx=0).stop 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 -- 172.21.15.33:0/1538408877 shutdown_connections 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 --2- 172.21.15.33:0/1538408877 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc109150 0x7fe9fc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 --2- 172.21.15.33:0/1538408877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc1087d0 0x7fe9fc108c10 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 --2- 172.21.15.33:0/1538408877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9fc102dc0 0x7fe9fc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 -- 172.21.15.33:0/1538408877 >> 172.21.15.33:0/1538408877 conn(0x7fe9fc0fe830 msgr2=0x7fe9fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 -- 172.21.15.33:0/1538408877 shutdown_connections 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 -- 172.21.15.33:0/1538408877 wait complete. 2024-09-10T21:02:43.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 Processor -- start 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.988+0000 7fea02ec1640 1 -- start start 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea02ec1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9fc102dc0 0x7fe9fc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea02ec1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc1087d0 0x7fe9fc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea02ec1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc109150 0x7fe9fc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea02ec1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc112e10 con 0x7fe9fc109150 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea02ec1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc112c90 con 0x7fe9fc1087d0 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea02ec1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc112f90 con 0x7fe9fc102dc0 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc109150 0x7fe9fc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc109150 0x7fe9fc075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44442/0 (socket says 172.21.15.33:44442) 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 -- 172.21.15.33:0/1946625813 learned_addr learned my addr 172.21.15.33:0/1946625813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fe9fbfff640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc1087d0 0x7fe9fc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea00c5d640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9fc102dc0 0x7fe9fc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 -- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9fc102dc0 msgr2=0x7fe9fc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9fc102dc0 0x7fe9fc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 -- 172.21.15.33:0/1946625813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc1087d0 msgr2=0x7fe9fc0759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe9fc1087d0 0x7fe9fc0759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 -- 172.21.15.33:0/1946625813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc1a76c0 con 0x7fe9fc109150 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea00c5d640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9fc102dc0 0x7fe9fc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea0145e640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc109150 0x7fe9fc075f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400cce0 tx=0x7fe9f400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400f450 con 0x7fe9fc109150 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9f400faa0 con 0x7fe9fc109150 2024-09-10T21:02:43.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.989+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc1a79b0 con 0x7fe9fc109150 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.990+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4018b70 con 0x7fe9fc109150 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.990+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1a7f40 con 0x7fe9fc109150 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.991+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9fc0638e0 con 0x7fe9fc109150 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.992+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fe9f400f5f0 con 0x7fe9fc109150 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.993+0000 7fe9f9ffb640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9e40b53f0 0x7fe9e40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.993+0000 7fea00c5d640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9e40b53f0 0x7fe9e40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.993+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe9f40e6770 con 0x7fe9fc109150 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.993+0000 7fe9f9ffb640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] conn(0x7fe9e40bef10 0x7fe9e40c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.993+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 --> [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9f4012ec0 con 0x7fe9e40bef10 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.993+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fe9f40e6b20 con 0x7fe9fc109150 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.994+0000 7fea00c5d640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9e40b53f0 0x7fe9e40b78b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009930 tx=0x7fe9ec008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.994+0000 7fe9fbfff640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] conn(0x7fe9e40bef10 0x7fe9e40c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.994+0000 7fe9fbfff640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] conn(0x7fe9e40bef10 0x7fe9e40c13b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:42.995+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== osd.8 v2:172.21.15.174:6816/758997036 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9f4012ec0 con 0x7fe9e40bef10 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.008+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 --> [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9fc063d60 con 0x7fe9e40bef10 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fe9f9ffb640 1 -- 172.21.15.33:0/1946625813 <== osd.8 v2:172.21.15.174:6816/758997036 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe9fc063d60 con 0x7fe9e40bef10 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] conn(0x7fe9e40bef10 msgr2=0x7fe9e40c13b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fea02ec1640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] conn(0x7fe9e40bef10 0x7fe9e40c13b0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9e40b53f0 msgr2=0x7fe9e40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fea02ec1640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9e40b53f0 0x7fe9e40b78b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009930 tx=0x7fe9ec008040 comp rx=0 tx=0).stop 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc109150 msgr2=0x7fe9fc075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fea02ec1640 1 --2- 172.21.15.33:0/1946625813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc109150 0x7fe9fc075f20 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400cce0 tx=0x7fe9f400a620 comp rx=0 tx=0).stop 2024-09-10T21:02:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.009+0000 7fea0145e640 1 -- 172.21.15.33:0/1946625813 reap_dead start 2024-09-10T21:02:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.010+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 shutdown_connections 2024-09-10T21:02:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.010+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 >> 172.21.15.33:0/1946625813 conn(0x7fe9fc0fe830 msgr2=0x7fe9fc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.010+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 shutdown_connections 2024-09-10T21:02:43.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.010+0000 7fea02ec1640 1 -- 172.21.15.33:0/1946625813 wait complete. 2024-09-10T21:02:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 -- 172.21.15.33:0/3281493386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb010d110 msgr2=0x7efcb010f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 --2- 172.21.15.33:0/3281493386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb010d110 0x7efcb010f570 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efca0009ca0 tx=0x7efca002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 -- 172.21.15.33:0/3281493386 shutdown_connections 2024-09-10T21:02:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 --2- 172.21.15.33:0/3281493386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb010d110 0x7efcb010f570 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 --2- 172.21.15.33:0/3281493386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcb01045d0 0x7efcb010cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 --2- 172.21.15.33:0/3281493386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb0103cb0 0x7efcb0104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 -- 172.21.15.33:0/3281493386 >> 172.21.15.33:0/3281493386 conn(0x7efcb00fd6d0 msgr2=0x7efcb00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 -- 172.21.15.33:0/3281493386 shutdown_connections 2024-09-10T21:02:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 -- 172.21.15.33:0/3281493386 wait complete. 2024-09-10T21:02:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.684+0000 7efcb52d7640 1 Processor -- start 2024-09-10T21:02:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcb52d7640 1 -- start start 2024-09-10T21:02:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcb52d7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb0103cb0 0x7efcb0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcb52d7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb01045d0 0x7efcb0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcb52d7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcb010d110 0x7efcb019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcb52d7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0112d80 con 0x7efcb01045d0 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcb52d7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0112c00 con 0x7efcb0103cb0 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcb52d7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efcb0112f00 con 0x7efcb010d110 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb01045d0 0x7efcb0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb01045d0 0x7efcb0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44460/0 (socket says 172.21.15.33:44460) 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 -- 172.21.15.33:0/1354903330 learned_addr learned my addr 172.21.15.33:0/1354903330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcaf7fe640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcb010d110 0x7efcb019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcaeffd640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb0103cb0 0x7efcb0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 -- 172.21.15.33:0/1354903330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcb010d110 msgr2=0x7efcb019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efcb010d110 0x7efcb019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 -- 172.21.15.33:0/1354903330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb0103cb0 msgr2=0x7efcb0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efcb0103cb0 0x7efcb0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.685+0000 7efcae7fc640 1 -- 172.21.15.33:0/1354903330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb019f2e0 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.686+0000 7efcae7fc640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb01045d0 0x7efcb0199770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d750 tx=0x7efc9800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.686+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98017070 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.686+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb019f5d0 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.686+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc98004020 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.686+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98003c00 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.686+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb01a5e60 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.687+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efcb00ffd40 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.688+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7efc98009a30 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.689+0000 7efca7fff640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc9c0b5370 0x7efc9c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.689+0000 7efcaeffd640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc9c0b5370 0x7efc9c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.689+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efc980e5960 con 0x7efcb01045d0 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.690+0000 7efca7fff640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] conn(0x7efc9c0bee90 0x7efc9c0c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.690+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 --> [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc9c0c1a00 con 0x7efc9c0bee90 2024-09-10T21:02:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.690+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7efc980e5d90 con 0x7efcb01045d0 2024-09-10T21:02:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.690+0000 7efcaf7fe640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] conn(0x7efc9c0bee90 0x7efc9c0c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.690+0000 7efcaeffd640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc9c0b5370 0x7efc9c0b7830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efca800bbe0 tx=0x7efca800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.690+0000 7efcaf7fe640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] conn(0x7efc9c0bee90 0x7efc9c0c1330 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.691+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== osd.9 v2:172.21.15.33:6826/3501032223 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc9c0c1a00 con 0x7efc9c0bee90 2024-09-10T21:02:43.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.706+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 --> [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efcb006bbd0 con 0x7efc9c0bee90 2024-09-10T21:02:43.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.706+0000 7efca7fff640 1 -- 172.21.15.33:0/1354903330 <== osd.9 v2:172.21.15.33:6826/3501032223 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7efcb006bbd0 con 0x7efc9c0bee90 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] conn(0x7efc9c0bee90 msgr2=0x7efc9c0c1330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] conn(0x7efc9c0bee90 0x7efc9c0c1330 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc9c0b5370 msgr2=0x7efc9c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc9c0b5370 0x7efc9c0b7830 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efca800bbe0 tx=0x7efca800b390 comp rx=0 tx=0).stop 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb01045d0 msgr2=0x7efcb0199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 --2- 172.21.15.33:0/1354903330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efcb01045d0 0x7efcb0199770 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d750 tx=0x7efc9800dc10 comp rx=0 tx=0).stop 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcaf7fe640 1 -- 172.21.15.33:0/1354903330 reap_dead start 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 shutdown_connections 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 >> 172.21.15.33:0/1354903330 conn(0x7efcb00fd6d0 msgr2=0x7efcb00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:43.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 shutdown_connections 2024-09-10T21:02:43.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.707+0000 7efcb52d7640 1 -- 172.21.15.33:0/1354903330 wait complete. 2024-09-10T21:02:44.788 INFO:teuthology.orchestra.run.smithi033.stdout:270582939674 2024-09-10T21:02:44.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-10T21:02:45.231 INFO:teuthology.orchestra.run.smithi033.stdout:103079215165 2024-09-10T21:02:45.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T21:02:45.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:45 smithi033 ceph-mon[30860]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 639 B/s wr, 10 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:45 smithi104 ceph-mon[35325]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 639 B/s wr, 10 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:45 smithi174 ceph-mon[35635]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 639 B/s wr, 10 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 -- 172.21.15.33:0/1456197424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc468102dc0 msgr2=0x7fc4681031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 --2- 172.21.15.33:0/1456197424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc468102dc0 0x7fc4681031a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009900 tx=0x7fc45c02a410 comp rx=0 tx=0).stop 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 -- 172.21.15.33:0/1456197424 shutdown_connections 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 --2- 172.21.15.33:0/1456197424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc468109150 0x7fc46810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 --2- 172.21.15.33:0/1456197424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4681087d0 0x7fc468108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 --2- 172.21.15.33:0/1456197424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc468102dc0 0x7fc4681031a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 -- 172.21.15.33:0/1456197424 >> 172.21.15.33:0/1456197424 conn(0x7fc4680fe830 msgr2=0x7fc468100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 -- 172.21.15.33:0/1456197424 shutdown_connections 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.946+0000 7fc46ecf4640 1 -- 172.21.15.33:0/1456197424 wait complete. 2024-09-10T21:02:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 Processor -- start 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 -- start start 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc468102dc0 0x7fc4681992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4681087d0 0x7fc468199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc468109150 0x7fc46819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc468112e60 con 0x7fc4681087d0 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc468112ce0 con 0x7fc468102dc0 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc46ecf4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc468112fe0 con 0x7fc468109150 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.947+0000 7fc467fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4681087d0 0x7fc468199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4681087d0 0x7fc468199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44490/0 (socket says 172.21.15.33:44490) 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 -- 172.21.15.33:0/349230034 learned_addr learned my addr 172.21.15.33:0/349230034 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc46d291640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc468109150 0x7fc46819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc46ca90640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc468102dc0 0x7fc4681992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 -- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc468109150 msgr2=0x7fc46819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc468109150 0x7fc46819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 -- 172.21.15.33:0/349230034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc468102dc0 msgr2=0x7fc4681992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc468102dc0 0x7fc4681992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 -- 172.21.15.33:0/349230034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46819f380 con 0x7fc4681087d0 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc46ca90640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc468102dc0 0x7fc4681992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc467fff640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4681087d0 0x7fc468199810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc45400b4a0 tx=0x7fc45400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454014070 con 0x7fc4681087d0 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46819f670 con 0x7fc4681087d0 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc454010ac0 con 0x7fc4681087d0 2024-09-10T21:02:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454017c30 con 0x7fc4681087d0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.948+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4681a5db0 con 0x7fc4681087d0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.949+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4680638a0 con 0x7fc4681087d0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.951+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fc454010610 con 0x7fc4681087d0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.951+0000 7fc465ffb640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4580b5370 0x7fc4580b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.951+0000 7fc46ca90640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4580b5370 0x7fc4580b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.951+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc4540e56d0 con 0x7fc4681087d0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.952+0000 7fc465ffb640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] conn(0x7fc4580bee90 0x7fc4580c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.952+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 --> [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc45400ee50 con 0x7fc4580bee90 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.952+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fc4540e5b00 con 0x7fc4681087d0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.952+0000 7fc46ca90640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4580b5370 0x7fc4580b7830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005ab0 tx=0x7fc45c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.952+0000 7fc46d291640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] conn(0x7fc4580bee90 0x7fc4580c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.952+0000 7fc46d291640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] conn(0x7fc4580bee90 0x7fc4580c1330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.953+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== osd.2 v2:172.21.15.174:6800/4030111255 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc45400ee50 con 0x7fc4580bee90 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.967+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 --> [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc468063ca0 con 0x7fc4580bee90 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc465ffb640 1 -- 172.21.15.33:0/349230034 <== osd.2 v2:172.21.15.174:6800/4030111255 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc468063ca0 con 0x7fc4580bee90 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] conn(0x7fc4580bee90 msgr2=0x7fc4580c1330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] conn(0x7fc4580bee90 0x7fc4580c1330 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4580b5370 msgr2=0x7fc4580b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4580b5370 0x7fc4580b7830 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc45c005ab0 tx=0x7fc45c000bc0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4681087d0 msgr2=0x7fc468199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 --2- 172.21.15.33:0/349230034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4681087d0 0x7fc468199810 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc45400b4a0 tx=0x7fc45400b960 comp rx=0 tx=0).stop 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46d291640 1 -- 172.21.15.33:0/349230034 reap_dead start 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 shutdown_connections 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 >> 172.21.15.33:0/349230034 conn(0x7fc4680fe830 msgr2=0x7fc468100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 shutdown_connections 2024-09-10T21:02:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:45.968+0000 7fc46ecf4640 1 -- 172.21.15.33:0/349230034 wait complete. 2024-09-10T21:02:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 -- 172.21.15.33:0/2835937783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8109150 msgr2=0x7f9ec810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 --2- 172.21.15.33:0/2835937783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8109150 0x7f9ec810f950 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009ca0 tx=0x7f9ec002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 -- 172.21.15.33:0/2835937783 shutdown_connections 2024-09-10T21:02:46.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 --2- 172.21.15.33:0/2835937783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8109150 0x7f9ec810f950 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 --2- 172.21.15.33:0/2835937783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ec81087d0 0x7f9ec8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 --2- 172.21.15.33:0/2835937783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec8102dc0 0x7f9ec81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 -- 172.21.15.33:0/2835937783 >> 172.21.15.33:0/2835937783 conn(0x7f9ec80fe830 msgr2=0x7f9ec8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.780+0000 7f9ed0315640 1 -- 172.21.15.33:0/2835937783 shutdown_connections 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.781+0000 7f9ed0315640 1 -- 172.21.15.33:0/2835937783 wait complete. 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.781+0000 7f9ed0315640 1 Processor -- start 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.781+0000 7f9ed0315640 1 -- start start 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8102dc0 0x7f9ec8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ec81087d0 0x7f9ec8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec8109150 0x7f9ec819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec8112e10 con 0x7f9ec8102dc0 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec8112c90 con 0x7f9ec8109150 2024-09-10T21:02:46.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec8112f90 con 0x7f9ec81087d0 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8102dc0 0x7f9ec8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8102dc0 0x7f9ec8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44540/0 (socket says 172.21.15.33:44540) 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 -- 172.21.15.33:0/2108758638 learned_addr learned my addr 172.21.15.33:0/2108758638 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ecd8b0640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ec81087d0 0x7f9ec8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece8b2640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec8109150 0x7f9ec819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 -- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ec81087d0 msgr2=0x7f9ec8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ec81087d0 0x7f9ec8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 -- 172.21.15.33:0/2108758638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec8109150 msgr2=0x7f9ec819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec8109150 0x7f9ec819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 -- 172.21.15.33:0/2108758638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec819f2f0 con 0x7f9ec8102dc0 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ecd8b0640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ec81087d0 0x7f9ec8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece8b2640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9ec8109150 0x7f9ec819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ece0b1640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8102dc0 0x7f9ec8199240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400c890 tx=0x7f9ec400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:46.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4004220 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ec40043c0 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec819f5e0 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4004a00 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.782+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec81a5e70 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.783+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ec81060e0 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.785+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f9ec40079b0 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ebf7fe640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9eb40b53f0 0x7f9eb40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ecd8b0640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9eb40b53f0 0x7f9eb40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9ec40e5f00 con 0x7f9ec8102dc0 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ebf7fe640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] conn(0x7f9eb40bef10 0x7f9eb40c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 --> [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9eb40c1a80 con 0x7f9eb40bef10 2024-09-10T21:02:46.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f9ec40e6330 con 0x7f9ec8102dc0 2024-09-10T21:02:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ecd8b0640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9eb40b53f0 0x7f9eb40b78b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8108300 tx=0x7f9eb800b5f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.786+0000 7f9ece8b2640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] conn(0x7f9eb40bef10 0x7f9eb40c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.787+0000 7f9ece8b2640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] conn(0x7f9eb40bef10 0x7f9eb40c13b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.788+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== osd.5 v2:172.21.15.174:6808/1209422453 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9eb40c1a80 con 0x7f9eb40bef10 2024-09-10T21:02:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.801+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 --> [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ec8104070 con 0x7f9eb40bef10 2024-09-10T21:02:46.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ebf7fe640 1 -- 172.21.15.33:0/2108758638 <== osd.5 v2:172.21.15.174:6808/1209422453 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ec8104070 con 0x7f9eb40bef10 2024-09-10T21:02:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] conn(0x7f9eb40bef10 msgr2=0x7f9eb40c13b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ed0315640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] conn(0x7f9eb40bef10 0x7f9eb40c13b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9eb40b53f0 msgr2=0x7f9eb40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ed0315640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9eb40b53f0 0x7f9eb40b78b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8108300 tx=0x7f9eb800b5f0 comp rx=0 tx=0).stop 2024-09-10T21:02:46.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8102dc0 msgr2=0x7f9ec8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:46.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ed0315640 1 --2- 172.21.15.33:0/2108758638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ec8102dc0 0x7f9ec8199240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400c890 tx=0x7f9ec400cd50 comp rx=0 tx=0).stop 2024-09-10T21:02:46.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.802+0000 7f9ece8b2640 1 -- 172.21.15.33:0/2108758638 reap_dead start 2024-09-10T21:02:46.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.803+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 shutdown_connections 2024-09-10T21:02:46.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.803+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 >> 172.21.15.33:0/2108758638 conn(0x7f9ec80fe830 msgr2=0x7f9ec8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:46.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.803+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 shutdown_connections 2024-09-10T21:02:46.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.803+0000 7f9ed0315640 1 -- 172.21.15.33:0/2108758638 wait complete. 2024-09-10T21:02:47.243 INFO:teuthology.orchestra.run.smithi033.stdout:249108103200 2024-09-10T21:02:47.244 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 -- 172.21.15.33:0/2385416676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c1087a0 msgr2=0x7f0c7c108be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 --2- 172.21.15.33:0/2385416676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c1087a0 0x7f0c7c108be0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009900 tx=0x7f0c7002a410 comp rx=0 tx=0).stop 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 -- 172.21.15.33:0/2385416676 shutdown_connections 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 --2- 172.21.15.33:0/2385416676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c109120 0x7f0c7c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 --2- 172.21.15.33:0/2385416676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c1087a0 0x7f0c7c108be0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 --2- 172.21.15.33:0/2385416676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c7c102d90 0x7f0c7c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 -- 172.21.15.33:0/2385416676 >> 172.21.15.33:0/2385416676 conn(0x7f0c7c0fe820 msgr2=0x7f0c7c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 -- 172.21.15.33:0/2385416676 shutdown_connections 2024-09-10T21:02:47.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 -- 172.21.15.33:0/2385416676 wait complete. 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.677+0000 7f0c8300d640 1 Processor -- start 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c8300d640 1 -- start start 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c8300d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c7c102d90 0x7f0c7c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c8300d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c1087a0 0x7f0c7c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c8300d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c109120 0x7f0c7c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c8300d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c112d70 con 0x7f0c7c109120 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c8300d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c112bf0 con 0x7f0c7c1087a0 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c8300d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c7c112ef0 con 0x7f0c7c102d90 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c109120 0x7f0c7c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c109120 0x7f0c7c19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44520/0 (socket says 172.21.15.33:44520) 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 -- 172.21.15.33:0/1608782934 learned_addr learned my addr 172.21.15.33:0/1608782934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:47.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c7bfff640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c1087a0 0x7f0c7c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c80da9640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c7c102d90 0x7f0c7c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 -- 172.21.15.33:0/1608782934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c7c102d90 msgr2=0x7f0c7c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c7c102d90 0x7f0c7c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 -- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c1087a0 msgr2=0x7f0c7c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c7c1087a0 0x7f0c7c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:47.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 -- 172.21.15.33:0/1608782934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c7c19f330 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.678+0000 7f0c815aa640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c109120 0x7f0c7c19eb60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400eb10 tx=0x7f0c7400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.679+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74004b20 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.679+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c7c19f620 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.679+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c74004cc0 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.679+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c74010670 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.679+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c7c1a5ce0 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.680+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0c7c0638e0 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.681+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f0c74005000 con 0x7f0c7c109120 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c79ffb640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c640b5370 0x7f0c640b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c80da9640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c640b5370 0x7f0c640b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0c740e6780 con 0x7f0c7c109120 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c79ffb640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] conn(0x7f0c640bee90 0x7f0c640c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 --> [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0c74015e80 con 0x7f0c640bee90 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f0c740e6b30 con 0x7f0c7c109120 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c7bfff640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] conn(0x7f0c640bee90 0x7f0c640c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.682+0000 7f0c80da9640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c640b5370 0x7f0c640b7830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005d00 tx=0x7f0c6c005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.683+0000 7f0c7bfff640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] conn(0x7f0c640bee90 0x7f0c640c1330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.683+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== osd.7 v2:172.21.15.104:6816/947117996 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0c74015e80 con 0x7f0c640bee90 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.698+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 --> [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0c7c063d60 con 0x7f0c640bee90 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.698+0000 7f0c79ffb640 1 -- 172.21.15.33:0/1608782934 <== osd.7 v2:172.21.15.104:6816/947117996 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0c7c063d60 con 0x7f0c640bee90 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] conn(0x7f0c640bee90 msgr2=0x7f0c640c1330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] conn(0x7f0c640bee90 0x7f0c640c1330 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c640b5370 msgr2=0x7f0c640b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c640b5370 0x7f0c640b7830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c005d00 tx=0x7f0c6c005c50 comp rx=0 tx=0).stop 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c109120 msgr2=0x7f0c7c19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 --2- 172.21.15.33:0/1608782934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c7c109120 0x7f0c7c19eb60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0c7400eb10 tx=0x7f0c7400c2d0 comp rx=0 tx=0).stop 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c815aa640 1 -- 172.21.15.33:0/1608782934 reap_dead start 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 shutdown_connections 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 >> 172.21.15.33:0/1608782934 conn(0x7f0c7c0fe820 msgr2=0x7f0c7c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 shutdown_connections 2024-09-10T21:02:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:46.699+0000 7f0c8300d640 1 -- 172.21.15.33:0/1608782934 wait complete. 2024-09-10T21:02:47.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:47 smithi033 ceph-mon[30860]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:47 smithi104 ceph-mon[35325]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:47 smithi174 ceph-mon[35635]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 7 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:48.018 INFO:teuthology.orchestra.run.smithi033.stdout:197568495657 2024-09-10T21:02:48.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T21:02:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 -- 172.21.15.33:0/942075041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 msgr2=0x7fd128101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 --2- 172.21.15.33:0/942075041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 0x7fd128101c50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd124009860 tx=0x7fd12402a3d0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 -- 172.21.15.33:0/942075041 shutdown_connections 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 --2- 172.21.15.33:0/942075041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd12810f420 0x7fd128111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 --2- 172.21.15.33:0/942075041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128102190 0x7fd12810eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 --2- 172.21.15.33:0/942075041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 0x7fd128101c50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 -- 172.21.15.33:0/942075041 >> 172.21.15.33:0/942075041 conn(0x7fd1280fd740 msgr2=0x7fd1280ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 -- 172.21.15.33:0/942075041 shutdown_connections 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.211+0000 7fd12fd41640 1 -- 172.21.15.33:0/942075041 wait complete. 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 Processor -- start 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 -- start start 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 0x7fd12819b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128102190 0x7fd12819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd12810f420 0x7fd1281a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd128114fd0 con 0x7fd128101870 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd128114e50 con 0x7fd12810f420 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12fd41640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd128115150 con 0x7fd128102190 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.212+0000 7fd12dadd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 0x7fd12819b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 0x7fd12819b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44564/0 (socket says 172.21.15.33:44564) 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 -- 172.21.15.33:0/710067435 learned_addr learned my addr 172.21.15.33:0/710067435 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12e2de640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd12810f420 0x7fd1281a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12d2dc640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128102190 0x7fd12819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 -- 172.21.15.33:0/710067435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128102190 msgr2=0x7fd12819b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd128102190 0x7fd12819b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 -- 172.21.15.33:0/710067435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd12810f420 msgr2=0x7fd1281a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd12810f420 0x7fd1281a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 -- 172.21.15.33:0/710067435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1281a1550 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12e2de640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd12810f420 0x7fd1281a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12dadd640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 0x7fd12819b4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd124005940 tx=0x7fd124030fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124040070 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd124005570 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1281a17e0 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240046c0 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.213+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1281a7f00 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.214+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd128110ad0 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.216+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fd1240050c0 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd11effd640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1140b5370 0x7fd1140b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd12d2dc640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1140b5370 0x7fd1140b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd124107010 con 0x7fd128101870 2024-09-10T21:02:48.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd11effd640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] conn(0x7fd1140bee90 0x7fd1140c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 --> [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1140c1a00 con 0x7fd1140bee90 2024-09-10T21:02:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd124107440 con 0x7fd128101870 2024-09-10T21:02:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd12e2de640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] conn(0x7fd1140bee90 0x7fd1140c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd12d2dc640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1140b5370 0x7fd1140b7830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd12810fad0 tx=0x7fd118005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.217+0000 7fd12e2de640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] conn(0x7fd1140bee90 0x7fd1140c1330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.218+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== osd.0 v2:172.21.15.33:6802/3008534631 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd1140c1a00 con 0x7fd1140bee90 2024-09-10T21:02:48.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.232+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 --> [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd12806b9d0 con 0x7fd1140bee90 2024-09-10T21:02:48.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.232+0000 7fd11effd640 1 -- 172.21.15.33:0/710067435 <== osd.0 v2:172.21.15.33:6802/3008534631 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd12806b9d0 con 0x7fd1140bee90 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] conn(0x7fd1140bee90 msgr2=0x7fd1140c1330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] conn(0x7fd1140bee90 0x7fd1140c1330 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1140b5370 msgr2=0x7fd1140b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1140b5370 0x7fd1140b7830 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd12810fad0 tx=0x7fd118005f90 comp rx=0 tx=0).stop 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 msgr2=0x7fd12819b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 --2- 172.21.15.33:0/710067435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd128101870 0x7fd12819b4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd124005940 tx=0x7fd124030fa0 comp rx=0 tx=0).stop 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12e2de640 1 -- 172.21.15.33:0/710067435 reap_dead start 2024-09-10T21:02:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 shutdown_connections 2024-09-10T21:02:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 >> 172.21.15.33:0/710067435 conn(0x7fd1280fd740 msgr2=0x7fd1280ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 shutdown_connections 2024-09-10T21:02:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:48.233+0000 7fd12fd41640 1 -- 172.21.15.33:0/710067435 wait complete. 2024-09-10T21:02:48.696 INFO:teuthology.orchestra.run.smithi033.stdout:55834574920 2024-09-10T21:02:48.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T21:02:49.325 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:49 smithi033 ceph-mon[30860]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:49.352 INFO:teuthology.orchestra.run.smithi033.stdout:34359738445 2024-09-10T21:02:49.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T21:02:49.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:49 smithi104 ceph-mon[35325]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:49 smithi174 ceph-mon[35635]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s; 154 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:49.757 INFO:teuthology.orchestra.run.smithi033.stdout:223338299428 2024-09-10T21:02:49.757 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-10T21:02:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 -- 172.21.15.33:0/462198157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc108060 msgr2=0x7f11bc10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 --2- 172.21.15.33:0/462198157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc108060 0x7f11bc10e860 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f11b4009ca0 tx=0x7f11b402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 -- 172.21.15.33:0/462198157 shutdown_connections 2024-09-10T21:02:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 --2- 172.21.15.33:0/462198157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc108060 0x7f11bc10e860 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 --2- 172.21.15.33:0/462198157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11bc1076e0 0x7f11bc107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 --2- 172.21.15.33:0/462198157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11bc101cd0 0x7f11bc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 -- 172.21.15.33:0/462198157 >> 172.21.15.33:0/462198157 conn(0x7f11bc0fd700 msgr2=0x7f11bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 -- 172.21.15.33:0/462198157 shutdown_connections 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.896+0000 7f11c35c6640 1 -- 172.21.15.33:0/462198157 wait complete. 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 Processor -- start 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 -- start start 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11bc101cd0 0x7f11bc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc1076e0 0x7f11bc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11bc108060 0x7f11bc19eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc111d70 con 0x7f11bc1076e0 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc111bf0 con 0x7f11bc101cd0 2024-09-10T21:02:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c35c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc111ef0 con 0x7f11bc108060 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.897+0000 7f11c0b61640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc1076e0 0x7f11bc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc1076e0 0x7f11bc199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44470/0 (socket says 172.21.15.33:44470) 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 -- 172.21.15.33:0/1106096596 learned_addr learned my addr 172.21.15.33:0/1106096596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c1b63640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11bc108060 0x7f11bc19eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c1362640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11bc101cd0 0x7f11bc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 -- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11bc108060 msgr2=0x7f11bc19eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11bc108060 0x7f11bc19eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 -- 172.21.15.33:0/1106096596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11bc101cd0 msgr2=0x7f11bc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11bc101cd0 0x7f11bc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 -- 172.21.15.33:0/1106096596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc19f280 con 0x7f11bc1076e0 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c1362640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11bc101cd0 0x7f11bc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c0b61640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc1076e0 0x7f11bc199710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c920 tx=0x7f11ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac007c70 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11ac007e10 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc19f570 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac018820 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.898+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1a5c30 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.899+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11bc0638e0 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.901+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f11ac00f450 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.902+0000 7f11b27fc640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11a80b5370 0x7f11a80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.902+0000 7f11c1362640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11a80b5370 0x7f11a80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.902+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f11ac0e62d0 con 0x7f11bc1076e0 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.902+0000 7f11b27fc640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] conn(0x7f11a80bee90 0x7f11a80c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.902+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 --> [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f11ac023e50 con 0x7f11a80bee90 2024-09-10T21:02:50.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.902+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f11ac0e6700 con 0x7f11bc1076e0 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.903+0000 7f11c1362640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11a80b5370 0x7f11a80b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009b40 tx=0x7f11b8005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.903+0000 7f11c1b63640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] conn(0x7f11a80bee90 0x7f11a80c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.903+0000 7f11c1b63640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] conn(0x7f11a80bee90 0x7f11a80c1330 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.904+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== osd.11 v2:172.21.15.174:6824/117399142 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f11ac023e50 con 0x7f11a80bee90 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.918+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 --> [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11bc063d60 con 0x7f11a80bee90 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.918+0000 7f11b27fc640 1 -- 172.21.15.33:0/1106096596 <== osd.11 v2:172.21.15.174:6824/117399142 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f11bc063d60 con 0x7f11a80bee90 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] conn(0x7f11a80bee90 msgr2=0x7f11a80c1330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] conn(0x7f11a80bee90 0x7f11a80c1330 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11a80b5370 msgr2=0x7f11a80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11a80b5370 0x7f11a80b7830 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009b40 tx=0x7f11b8005e50 comp rx=0 tx=0).stop 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc1076e0 msgr2=0x7f11bc199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 --2- 172.21.15.33:0/1106096596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11bc1076e0 0x7f11bc199710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c920 tx=0x7f11ac00cde0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c1b63640 1 -- 172.21.15.33:0/1106096596 reap_dead start 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 shutdown_connections 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 >> 172.21.15.33:0/1106096596 conn(0x7f11bc0fd700 msgr2=0x7f11bc06b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 shutdown_connections 2024-09-10T21:02:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:43.919+0000 7f11c35c6640 1 -- 172.21.15.33:0/1106096596 wait complete. 2024-09-10T21:02:50.736 INFO:teuthology.orchestra.run.smithi033.stdout:296352743445 2024-09-10T21:02:50.736 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 -- 172.21.15.33:0/190036681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c810d100 msgr2=0x7fa1c810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 --2- 172.21.15.33:0/190036681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c810d100 0x7fa1c810f550 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009ca0 tx=0x7fa1c002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 -- 172.21.15.33:0/190036681 shutdown_connections 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 --2- 172.21.15.33:0/190036681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c810d100 0x7fa1c810f550 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 --2- 172.21.15.33:0/190036681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1c8104550 0x7fa1c810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 --2- 172.21.15.33:0/190036681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c8103b50 0x7fa1c8103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 -- 172.21.15.33:0/190036681 >> 172.21.15.33:0/190036681 conn(0x7fa1c80fd700 msgr2=0x7fa1c80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 -- 172.21.15.33:0/190036681 shutdown_connections 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.946+0000 7fa1ccb0e640 1 -- 172.21.15.33:0/190036681 wait complete. 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 Processor -- start 2024-09-10T21:02:50.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 -- start start 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c8103b50 0x7fa1c8199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1c8104550 0x7fa1c81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c810d100 0x7fa1c819ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8112d60 con 0x7fa1c8103b50 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8112be0 con 0x7fa1c810d100 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1ccb0e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c8112ee0 con 0x7fa1c8104550 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c8103b50 0x7fa1c8199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1c659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c8103b50 0x7fa1c8199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41000/0 (socket says 172.21.15.33:41000) 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1c659c640 1 -- 172.21.15.33:0/3764561519 learned_addr learned my addr 172.21.15.33:0/3764561519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1c6d9d640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c810d100 0x7fa1c819ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.947+0000 7fa1bffff640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1c8104550 0x7fa1c81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1c659c640 1 -- 172.21.15.33:0/3764561519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1c8104550 msgr2=0x7fa1c81996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1c659c640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1c8104550 0x7fa1c81996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1c659c640 1 -- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c810d100 msgr2=0x7fa1c819ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1c659c640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1c810d100 0x7fa1c819ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1c659c640 1 -- 172.21.15.33:0/3764561519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c819f240 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1bffff640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1c8104550 0x7fa1c81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1c659c640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c8103b50 0x7fa1c8199190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800c920 tx=0x7fa1b800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8007c70 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1b8007e10 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b8018820 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1ccb0e640 1 -- 172.21.15.33:0/3764561519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c819f530 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.948+0000 7fa1ccb0e640 1 -- 172.21.15.33:0/3764561519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c81a5bf0 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.949+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa1c80638e0 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.951+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fa1b800f450 con 0x7fa1c8103b50 2024-09-10T21:02:50.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.952+0000 7fa1bf7fe640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1b40b53f0 0x7fa1b40b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.952+0000 7fa1bffff640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1b40b53f0 0x7fa1b40b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.952+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa1b80e6230 con 0x7fa1c8103b50 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.952+0000 7fa1bf7fe640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] conn(0x7fa1b40bef10 0x7fa1b40c13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.952+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 --> [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1b40c1a80 con 0x7fa1b40bef10 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.952+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fa1b80e6660 con 0x7fa1c8103b50 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.953+0000 7fa1bffff640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1b40b53f0 0x7fa1b40b78b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa1c810f770 tx=0x7fa1b0005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.953+0000 7fa1c6d9d640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] conn(0x7fa1b40bef10 0x7fa1b40c13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:50.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.953+0000 7fa1c6d9d640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] conn(0x7fa1b40bef10 0x7fa1b40c13b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.954+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== osd.4 v2:172.21.15.104:6808/3963281579 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa1b40c1a80 con 0x7fa1b40bef10 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.967+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 --> [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa1c8063d30 con 0x7fa1b40bef10 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bf7fe640 1 -- 172.21.15.33:0/3764561519 <== osd.4 v2:172.21.15.104:6808/3963281579 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa1c8063d30 con 0x7fa1b40bef10 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] conn(0x7fa1b40bef10 msgr2=0x7fa1b40c13b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] conn(0x7fa1b40bef10 0x7fa1b40c13b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1b40b53f0 msgr2=0x7fa1b40b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1b40b53f0 0x7fa1b40b78b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa1c810f770 tx=0x7fa1b0005d30 comp rx=0 tx=0).stop 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c8103b50 msgr2=0x7fa1c8199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 --2- 172.21.15.33:0/3764561519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1c8103b50 0x7fa1c8199190 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa1b800c920 tx=0x7fa1b800cde0 comp rx=0 tx=0).stop 2024-09-10T21:02:50.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1c6d9d640 1 -- 172.21.15.33:0/3764561519 reap_dead start 2024-09-10T21:02:50.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 shutdown_connections 2024-09-10T21:02:50.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.968+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 >> 172.21.15.33:0/3764561519 conn(0x7fa1c80fd700 msgr2=0x7fa1c80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:50.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.969+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 shutdown_connections 2024-09-10T21:02:50.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:50.969+0000 7fa1bd7fa640 1 -- 172.21.15.33:0/3764561519 wait complete. 2024-09-10T21:02:51.369 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:51 smithi033 ceph-mon[30860]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 133 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:51 smithi104 ceph-mon[35325]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 133 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:51 smithi174 ceph-mon[35635]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s; 133 B/s, 1 keys/s, 2 objects/s recovering 2024-09-10T21:02:51.698 INFO:teuthology.orchestra.run.smithi033.stdout:150323855412 2024-09-10T21:02:51.699 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T21:02:52.160 INFO:teuthology.orchestra.run.smithi033.stdout:73014444100 2024-09-10T21:02:52.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T21:02:52.842 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:52.856 INFO:teuthology.orchestra.run.smithi033.stdout:128849018939 2024-09-10T21:02:52.856 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T21:02:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:52.891 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:53.376 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:53 smithi033 ceph-mon[30860]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 120 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T21:02:53.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:53 smithi104 ceph-mon[35325]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 120 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T21:02:53.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:53 smithi174 ceph-mon[35635]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 7 op/s; 120 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T21:02:53.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/2404729499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 msgr2=0x7fc6c0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:53.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/2404729499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c0107ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009900 tx=0x7fc6a802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:53.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/2404729499 shutdown_connections 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/2404729499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0108020 0x7fc6c010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/2404729499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c0107ae0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/2404729499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0101c90 0x7fc6c0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/2404729499 >> 172.21.15.33:0/2404729499 conn(0x7fc6c00fd700 msgr2=0x7fc6c00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/2404729499 shutdown_connections 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.887+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/2404729499 wait complete. 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.888+0000 7fc6c5ee0640 1 Processor -- start 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.888+0000 7fc6c5ee0640 1 -- start start 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.888+0000 7fc6c5ee0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0101c90 0x7fc6c0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.888+0000 7fc6c5ee0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.888+0000 7fc6c5ee0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0108020 0x7fc6c019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:53.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.888+0000 7fc6c5ee0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111c70 con 0x7fc6c01076a0 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6c5ee0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111af0 con 0x7fc6c0101c90 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6c5ee0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c0111df0 con 0x7fc6c0108020 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41028/0 (socket says 172.21.15.33:41028) 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 -- 172.21.15.33:0/1276804824 learned_addr learned my addr 172.21.15.33:0/1276804824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6bffff640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0108020 0x7fc6c019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6bf7fe640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0101c90 0x7fc6c0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 -- 172.21.15.33:0/1276804824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0108020 msgr2=0x7fc6c019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0108020 0x7fc6c019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 -- 172.21.15.33:0/1276804824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0101c90 msgr2=0x7fc6c0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0101c90 0x7fc6c0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:53.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 -- 172.21.15.33:0/1276804824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c019e230 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6b6ffd640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c01986c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8005940 tx=0x7fc6a80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6bd7fa640 1 -- 172.21.15.33:0/1276804824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a803f030 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6bd7fa640 1 -- 172.21.15.33:0/1276804824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6a8003cb0 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c019e4c0 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.889+0000 7fc6bd7fa640 1 -- 172.21.15.33:0/1276804824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8035070 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.890+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c006aab0 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.891+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c0102070 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.892+0000 7fc6bd7fa640 1 -- 172.21.15.33:0/1276804824 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fc6a803bbf0 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.893+0000 7fc6bd7fa640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6ac0b5370 0x7fc6ac0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.893+0000 7fc6bf7fe640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6ac0b5370 0x7fc6ac0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.893+0000 7fc6bd7fa640 1 -- 172.21.15.33:0/1276804824 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc6a8106990 con 0x7fc6c01076a0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.893+0000 7fc6bf7fe640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6ac0b5370 0x7fc6ac0b7830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00058c0 tx=0x7fc6b000a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:53.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.897+0000 7fc6bd7fa640 1 -- 172.21.15.33:0/1276804824 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc6a80cd6f0 con 0x7fc6c01076a0 2024-09-10T21:02:54.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.122+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc6c006b4d0 con 0x7fc6c01076a0 2024-09-10T21:02:54.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.123+0000 7fc6bd7fa640 1 -- 172.21.15.33:0/1276804824 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc6a80ccff0 con 0x7fc6c01076a0 2024-09-10T21:02:54.127 INFO:teuthology.orchestra.run.smithi033.stdout:103079215167 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6ac0b5370 msgr2=0x7fc6ac0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6ac0b5370 0x7fc6ac0b7830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00058c0 tx=0x7fc6b000a4c0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 msgr2=0x7fc6c01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c01986c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8005940 tx=0x7fc6a80042f0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 shutdown_connections 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6ac0b5370 0x7fc6ac0b7830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6c0108020 0x7fc6c019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6c01076a0 0x7fc6c01986c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 --2- 172.21.15.33:0/1276804824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6c0101c90 0x7fc6c0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 >> 172.21.15.33:0/1276804824 conn(0x7fc6c00fd700 msgr2=0x7fc6c00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 shutdown_connections 2024-09-10T21:02:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.125+0000 7fc6c5ee0640 1 -- 172.21.15.33:0/1276804824 wait complete. 2024-09-10T21:02:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 -- 172.21.15.33:0/3989131039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1701076a0 msgr2=0x7fe170107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 --2- 172.21.15.33:0/3989131039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1701076a0 0x7fe170107ae0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe158009900 tx=0x7fe15802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 -- 172.21.15.33:0/3989131039 shutdown_connections 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 --2- 172.21.15.33:0/3989131039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe170108020 0x7fe17010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 --2- 172.21.15.33:0/3989131039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe1701076a0 0x7fe170107ae0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 --2- 172.21.15.33:0/3989131039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe170101c90 0x7fe170102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 -- 172.21.15.33:0/3989131039 >> 172.21.15.33:0/3989131039 conn(0x7fe1700fd700 msgr2=0x7fe1700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 -- 172.21.15.33:0/3989131039 shutdown_connections 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 -- 172.21.15.33:0/3989131039 wait complete. 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.777+0000 7fe1770c8640 1 Processor -- start 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe1770c8640 1 -- start start 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe1770c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe170101c90 0x7fe17019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe1770c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1701076a0 0x7fe17019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe1770c8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe170108020 0x7fe17019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe1770c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe170111ce0 con 0x7fe170101c90 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe1770c8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe170111b60 con 0x7fe170108020 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe1770c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe170111e60 con 0x7fe1701076a0 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe170101c90 0x7fe17019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe170101c90 0x7fe17019a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41012/0 (socket says 172.21.15.33:41012) 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 -- 172.21.15.33:0/1482073201 learned_addr learned my addr 172.21.15.33:0/1482073201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe175665640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe170108020 0x7fe17019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe16ffff640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1701076a0 0x7fe17019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 -- 172.21.15.33:0/1482073201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1701076a0 msgr2=0x7fe17019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1701076a0 0x7fe17019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 -- 172.21.15.33:0/1482073201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe170108020 msgr2=0x7fe17019abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe170108020 0x7fe17019abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 -- 172.21.15.33:0/1482073201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1701a54f0 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe175665640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe170108020 0x7fe17019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe16ffff640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1701076a0 0x7fe17019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.778+0000 7fe174e64640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe170101c90 0x7fe17019a0f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b4a0 tx=0x7fe16000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.779+0000 7fe16dffb640 1 -- 172.21.15.33:0/1482073201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160014070 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.779+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1701a57e0 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.779+0000 7fe16dffb640 1 -- 172.21.15.33:0/1482073201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe160010ac0 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.779+0000 7fe16dffb640 1 -- 172.21.15.33:0/1482073201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160017c30 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.779+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1701a5d70 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.780+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe170102070 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.781+0000 7fe16dffb640 1 -- 172.21.15.33:0/1482073201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fe160010610 con 0x7fe170101c90 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.782+0000 7fe16dffb640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe15c0b5370 0x7fe15c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.782+0000 7fe16ffff640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe15c0b5370 0x7fe15c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.782+0000 7fe16dffb640 1 -- 172.21.15.33:0/1482073201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe1600e56e0 con 0x7fe170101c90 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.782+0000 7fe16ffff640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe15c0b5370 0x7fe15c0b7830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe158031440 tx=0x7fe1580095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stdout:296352743446 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:53.785+0000 7fe16dffb640 1 -- 172.21.15.33:0/1482073201 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe1600ac440 con 0x7fe170101c90 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.008+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe17006aee0 con 0x7fe170101c90 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.008+0000 7fe16dffb640 1 -- 172.21.15.33:0/1482073201 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe17006aee0 con 0x7fe170101c90 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe15c0b5370 msgr2=0x7fe15c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe15c0b5370 0x7fe15c0b7830 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe158031440 tx=0x7fe1580095f0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe170101c90 msgr2=0x7fe17019a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe170101c90 0x7fe17019a0f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b4a0 tx=0x7fe16000b960 comp rx=0 tx=0).stop 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 shutdown_connections 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe15c0b5370 0x7fe15c0b7830 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe170108020 0x7fe17019abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1701076a0 0x7fe17019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 --2- 172.21.15.33:0/1482073201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe170101c90 0x7fe17019a0f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 >> 172.21.15.33:0/1482073201 conn(0x7fe1700fd700 msgr2=0x7fe1700ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 shutdown_connections 2024-09-10T21:02:54.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.012+0000 7fe1770c8640 1 -- 172.21.15.33:0/1482073201 wait complete. 2024-09-10T21:02:54.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:54 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1482073201' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T21:02:54.328 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:54 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1276804824' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T21:02:54.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 -- 172.21.15.33:0/1195695676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 msgr2=0x7fbdbc10cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/1195695676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc10cb10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009900 tx=0x7fbda402a410 comp rx=0 tx=0).stop 2024-09-10T21:02:54.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 -- 172.21.15.33:0/1195695676 shutdown_connections 2024-09-10T21:02:54.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/1195695676 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdbc10d0e0 0x7fbdbc10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/1195695676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc10cb10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/1195695676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc103b30 0x7fbdbc103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 -- 172.21.15.33:0/1195695676 >> 172.21.15.33:0/1195695676 conn(0x7fbdbc0fd700 msgr2=0x7fbdbc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:54.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 -- 172.21.15.33:0/1195695676 shutdown_connections 2024-09-10T21:02:54.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.330+0000 7fbdc0cae640 1 -- 172.21.15.33:0/1195695676 wait complete. 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 Processor -- start 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 -- start start 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc103b30 0x7fbdbc199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdbc10d0e0 0x7fbdbc19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112d60 con 0x7fbdbc104530 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112be0 con 0x7fbdbc10d0e0 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdc0cae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdbc112ee0 con 0x7fbdbc103b30 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.331+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41060/0 (socket says 172.21.15.33:41060) 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 -- 172.21.15.33:0/3340258193 learned_addr learned my addr 172.21.15.33:0/3340258193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:54.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdba59c640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc103b30 0x7fbdbc199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdbad9d640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdbc10d0e0 0x7fbdbc19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 -- 172.21.15.33:0/3340258193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc103b30 msgr2=0x7fbdbc199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc103b30 0x7fbdbc199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 -- 172.21.15.33:0/3340258193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdbc10d0e0 msgr2=0x7fbdbc19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdbc10d0e0 0x7fbdbc19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 -- 172.21.15.33:0/3340258193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdbc19f2c0 con 0x7fbdbc104530 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb3fff640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc199750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbda4005940 tx=0x7fbda40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3340258193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda403f030 con 0x7fbdbc104530 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdbc19f550 con 0x7fbdbc104530 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3340258193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbda4003cb0 con 0x7fbdbc104530 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.332+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3340258193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda4035070 con 0x7fbdbc104530 2024-09-10T21:02:54.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.333+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdbc1a5c70 con 0x7fbdbc104530 2024-09-10T21:02:54.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.333+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdbc103f10 con 0x7fbdbc104530 2024-09-10T21:02:54.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.335+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3340258193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fbda403bbf0 con 0x7fbdbc104530 2024-09-10T21:02:54.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.336+0000 7fbdb37fe640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbda80b5370 0x7fbda80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:54.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.336+0000 7fbdba59c640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbda80b5370 0x7fbda80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:54.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.336+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3340258193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbda4106ba0 con 0x7fbdbc104530 2024-09-10T21:02:54.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.336+0000 7fbdba59c640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbda80b5370 0x7fbda80b7830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbdac005fd0 tx=0x7fbdac00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:54.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.339+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3340258193 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fbda40cd2b0 con 0x7fbdbc104530 2024-09-10T21:02:54.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.525+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbdbc108130 con 0x7fbdbc104530 2024-09-10T21:02:54.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.525+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3340258193 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbda40d1ce0 con 0x7fbdbc104530 2024-09-10T21:02:54.529 INFO:teuthology.orchestra.run.smithi033.stdout:223338299430 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbda80b5370 msgr2=0x7fbda80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbda80b5370 0x7fbda80b7830 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbdac005fd0 tx=0x7fbdac00b390 comp rx=0 tx=0).stop 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 msgr2=0x7fbdbc199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc199750 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbda4005940 tx=0x7fbda40042f0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 shutdown_connections 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbda80b5370 0x7fbda80b7830 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdbc10d0e0 0x7fbdbc19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdbc104530 0x7fbdbc199750 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 --2- 172.21.15.33:0/3340258193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdbc103b30 0x7fbdbc199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:54.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 >> 172.21.15.33:0/3340258193 conn(0x7fbdbc0fd700 msgr2=0x7fbdbc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:54.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 shutdown_connections 2024-09-10T21:02:54.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:54.527+0000 7fbdc0cae640 1 -- 172.21.15.33:0/3340258193 wait complete. 2024-09-10T21:02:54.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:54 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1482073201' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T21:02:54.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:54 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1276804824' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T21:02:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:54 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1482073201' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-10T21:02:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:54 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1276804824' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T21:02:55.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:55 smithi033 ceph-mon[30860]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 103 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T21:02:55.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:55 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3340258193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T21:02:55.459 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215165 got 103079215167 for osd.3 2024-09-10T21:02:55.460 DEBUG:teuthology.parallel:result is None 2024-09-10T21:02:55.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:55 smithi104 ceph-mon[35325]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 103 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T21:02:55.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:55 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3340258193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T21:02:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:55 smithi174 ceph-mon[35635]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s; 103 B/s, 0 keys/s, 1 objects/s recovering 2024-09-10T21:02:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:55 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3340258193' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-10T21:02:55.902 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299428 got 223338299430 for osd.8 2024-09-10T21:02:55.902 DEBUG:teuthology.parallel:result is None 2024-09-10T21:02:56.415 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743445 got 296352743446 for osd.11 2024-09-10T21:02:56.415 DEBUG:teuthology.parallel:result is None 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 -- 172.21.15.33:0/3354175796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 msgr2=0x7fc70c102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 --2- 172.21.15.33:0/3354175796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c102d00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc708009900 tx=0x7fc70802a410 comp rx=0 tx=0).stop 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 -- 172.21.15.33:0/3354175796 shutdown_connections 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 --2- 172.21.15.33:0/3354175796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc70c108120 0x7fc70c10e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 --2- 172.21.15.33:0/3354175796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc70c107780 0x7fc70c107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 --2- 172.21.15.33:0/3354175796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c102d00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 -- 172.21.15.33:0/3354175796 >> 172.21.15.33:0/3354175796 conn(0x7fc70c076f50 msgr2=0x7fc70c077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 -- 172.21.15.33:0/3354175796 shutdown_connections 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.943+0000 7fc713b26640 1 -- 172.21.15.33:0/3354175796 wait complete. 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 Processor -- start 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 -- start start 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc70c107780 0x7fc70c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc70c108120 0x7fc70c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c111ca0 con 0x7fc70c102920 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c111b20 con 0x7fc70c107780 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc713b26640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c111e20 con 0x7fc70c108120 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc7118c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc7118c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41076/0 (socket says 172.21.15.33:41076) 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.944+0000 7fc7118c2640 1 -- 172.21.15.33:0/3273090212 learned_addr learned my addr 172.21.15.33:0/3273090212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7120c3640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc70c108120 0x7fc70c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7110c1640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc70c107780 0x7fc70c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7118c2640 1 -- 172.21.15.33:0/3273090212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc70c108120 msgr2=0x7fc70c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7118c2640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc70c108120 0x7fc70c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7118c2640 1 -- 172.21.15.33:0/3273090212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc70c107780 msgr2=0x7fc70c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7118c2640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc70c107780 0x7fc70c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7118c2640 1 -- 172.21.15.33:0/3273090212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c19e220 con 0x7fc70c102920 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7120c3640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc70c108120 0x7fc70c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc7118c2640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c198170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc708005940 tx=0x7fc708004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc702ffd640 1 -- 172.21.15.33:0/3273090212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70803f030 con 0x7fc70c102920 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c19e4b0 con 0x7fc70c102920 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc702ffd640 1 -- 172.21.15.33:0/3273090212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc708003aa0 con 0x7fc70c102920 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc702ffd640 1 -- 172.21.15.33:0/3273090212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc708035070 con 0x7fc70c102920 2024-09-10T21:02:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.945+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a4bd0 con 0x7fc70c102920 2024-09-10T21:02:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.946+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c102d00 con 0x7fc70c102920 2024-09-10T21:02:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.948+0000 7fc702ffd640 1 -- 172.21.15.33:0/3273090212 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fc70803bcc0 con 0x7fc70c102920 2024-09-10T21:02:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.949+0000 7fc702ffd640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6f80b5370 0x7fc6f80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.949+0000 7fc7110c1640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6f80b5370 0x7fc6f80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.949+0000 7fc702ffd640 1 -- 172.21.15.33:0/3273090212 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc708106ac0 con 0x7fc70c102920 2024-09-10T21:02:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.949+0000 7fc7110c1640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6f80b5370 0x7fc6f80b7830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005fd0 tx=0x7fc6fc005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:56.952+0000 7fc702ffd640 1 -- 172.21.15.33:0/3273090212 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc7080cd7e0 con 0x7fc70c102920 2024-09-10T21:02:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.082 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.087 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.149 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.149 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:02:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.193+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc70c109e70 con 0x7fc70c102920 2024-09-10T21:02:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.193+0000 7fc702ffd640 1 -- 172.21.15.33:0/3273090212 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc7080cd0e0 con 0x7fc70c102920 2024-09-10T21:02:57.198 INFO:teuthology.orchestra.run.smithi033.stdout:34359738447 2024-09-10T21:02:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6f80b5370 msgr2=0x7fc6f80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6f80b5370 0x7fc6f80b7830 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc005fd0 tx=0x7fc6fc005850 comp rx=0 tx=0).stop 2024-09-10T21:02:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 msgr2=0x7fc70c198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c198170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc708005940 tx=0x7fc708004270 comp rx=0 tx=0).stop 2024-09-10T21:02:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 shutdown_connections 2024-09-10T21:02:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6f80b5370 0x7fc6f80b7830 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc70c108120 0x7fc70c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc70c107780 0x7fc70c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 --2- 172.21.15.33:0/3273090212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc70c102920 0x7fc70c198170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 >> 172.21.15.33:0/3273090212 conn(0x7fc70c076f50 msgr2=0x7fc70c100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:57.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 shutdown_connections 2024-09-10T21:02:57.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.196+0000 7fc713b26640 1 -- 172.21.15.33:0/3273090212 wait complete. 2024-09-10T21:02:57.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:57 smithi033 ceph-mon[30860]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-10T21:02:57.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:57 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:02:57.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:57 smithi104 ceph-mon[35325]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-10T21:02:57.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:57 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:02:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:57 smithi174 ceph-mon[35635]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-09-10T21:02:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:57 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:02:57.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 -- 172.21.15.33:0/1550086839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 msgr2=0x7f66101031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 --2- 172.21.15.33:0/1550086839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101031a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6600009900 tx=0x7f660002a410 comp rx=0 tx=0).stop 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 -- 172.21.15.33:0/1550086839 shutdown_connections 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 --2- 172.21.15.33:0/1550086839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6610109150 0x7f661010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 --2- 172.21.15.33:0/1550086839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66101087d0 0x7f6610108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 --2- 172.21.15.33:0/1550086839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101031a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 -- 172.21.15.33:0/1550086839 >> 172.21.15.33:0/1550086839 conn(0x7f66100fe830 msgr2=0x7f6610100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 -- 172.21.15.33:0/1550086839 shutdown_connections 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 -- 172.21.15.33:0/1550086839 wait complete. 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 Processor -- start 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.884+0000 7f6616f44640 1 -- start start 2024-09-10T21:02:57.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6616f44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6616f44640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66101087d0 0x7f6610199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6616f44640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6610109150 0x7f661019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6616f44640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6610112e20 con 0x7f6610102dc0 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6616f44640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6610112ca0 con 0x7f66101087d0 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6616f44640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6610112fa0 con 0x7f6610109150 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41094/0 (socket says 172.21.15.33:41094) 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 -- 172.21.15.33:0/2412994966 learned_addr learned my addr 172.21.15.33:0/2412994966 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f660ffff640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66101087d0 0x7f6610199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f66154e1640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6610109150 0x7f661019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 -- 172.21.15.33:0/2412994966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6610109150 msgr2=0x7f661019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6610109150 0x7f661019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 -- 172.21.15.33:0/2412994966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66101087d0 msgr2=0x7f6610199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:57.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66101087d0 0x7f6610199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 -- 172.21.15.33:0/2412994966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661019f370 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6614ce0640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101992c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6600005ab0 tx=0x7f6600003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f660dffb640 1 -- 172.21.15.33:0/2412994966 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660003f070 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f660dffb640 1 -- 172.21.15.33:0/2412994966 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6600031d60 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661019f600 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.885+0000 7f660dffb640 1 -- 172.21.15.33:0/2412994966 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600035030 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.886+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66101a5d20 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.887+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66101031a0 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.888+0000 7f660dffb640 1 -- 172.21.15.33:0/2412994966 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f660004c060 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.889+0000 7f660dffb640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65fc0b53f0 0x7f65fc0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.889+0000 7f660ffff640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65fc0b53f0 0x7f65fc0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.889+0000 7f660dffb640 1 -- 172.21.15.33:0/2412994966 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f66001066f0 con 0x7f6610102dc0 2024-09-10T21:02:57.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.890+0000 7f660ffff640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65fc0b53f0 0x7f65fc0b78b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6610102180 tx=0x7f65f800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:57.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:57.892+0000 7f660dffb640 1 -- 172.21.15.33:0/2412994966 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f66000ccdc0 con 0x7f6610102dc0 2024-09-10T21:02:58.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.083+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f661010af60 con 0x7f6610102dc0 2024-09-10T21:02:58.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.083+0000 7f660dffb640 1 -- 172.21.15.33:0/2412994966 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f66000d17f0 con 0x7f6610102dc0 2024-09-10T21:02:58.087 INFO:teuthology.orchestra.run.smithi033.stdout:197568495659 2024-09-10T21:02:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65fc0b53f0 msgr2=0x7f65fc0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65fc0b53f0 0x7f65fc0b78b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6610102180 tx=0x7f65f800b410 comp rx=0 tx=0).stop 2024-09-10T21:02:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 msgr2=0x7f66101992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:58.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101992c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6600005ab0 tx=0x7f6600003cd0 comp rx=0 tx=0).stop 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 shutdown_connections 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65fc0b53f0 0x7f65fc0b78b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6610109150 0x7f661019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66101087d0 0x7f6610199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 --2- 172.21.15.33:0/2412994966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6610102dc0 0x7f66101992c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 >> 172.21.15.33:0/2412994966 conn(0x7f66100fe830 msgr2=0x7f6610100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 shutdown_connections 2024-09-10T21:02:58.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:58.085+0000 7f6616f44640 1 -- 172.21.15.33:0/2412994966 wait complete. 2024-09-10T21:02:58.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3273090212' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T21:02:58.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:58 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2412994966' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T21:02:58.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3273090212' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T21:02:58.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:58 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2412994966' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T21:02:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3273090212' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T21:02:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:58 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:02:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2412994966' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T21:02:58.818 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738447 for osd.0 2024-09-10T21:02:58.819 DEBUG:teuthology.parallel:result is None 2024-09-10T21:02:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.526+0000 7f82df29c640 1 -- 172.21.15.33:0/1858984385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d81076a0 msgr2=0x7f82d8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.526+0000 7f82df29c640 1 --2- 172.21.15.33:0/1858984385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d81076a0 0x7f82d8107ae0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009900 tx=0x7f82cc02a410 comp rx=0 tx=0).stop 2024-09-10T21:02:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.527+0000 7f82df29c640 1 -- 172.21.15.33:0/1858984385 shutdown_connections 2024-09-10T21:02:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.527+0000 7f82df29c640 1 --2- 172.21.15.33:0/1858984385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d8108020 0x7f82d810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.527+0000 7f82df29c640 1 --2- 172.21.15.33:0/1858984385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d81076a0 0x7f82d8107ae0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.527+0000 7f82df29c640 1 --2- 172.21.15.33:0/1858984385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82d8101c90 0x7f82d8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.527+0000 7f82df29c640 1 -- 172.21.15.33:0/1858984385 >> 172.21.15.33:0/1858984385 conn(0x7f82d80fd700 msgr2=0x7f82d80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.527+0000 7f82df29c640 1 -- 172.21.15.33:0/1858984385 shutdown_connections 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.527+0000 7f82df29c640 1 -- 172.21.15.33:0/1858984385 wait complete. 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 Processor -- start 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 -- start start 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82d8101c90 0x7f82d8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81076a0 0x7f82d8199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d8108020 0x7f82d819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8111ce0 con 0x7f82d8108020 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8111b60 con 0x7f82d81076a0 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82df29c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f82d8111e60 con 0x7f82d8101c90 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d8108020 0x7f82d819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d8108020 0x7f82d819ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49608/0 (socket says 172.21.15.33:49608) 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 -- 172.21.15.33:0/2711920317 learned_addr learned my addr 172.21.15.33:0/2711920317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dc837640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81076a0 0x7f82d8199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd038640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82d8101c90 0x7f82d8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 -- 172.21.15.33:0/2711920317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82d8101c90 msgr2=0x7f82d8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82d8101c90 0x7f82d8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 -- 172.21.15.33:0/2711920317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81076a0 msgr2=0x7f82d8199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81076a0 0x7f82d8199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.528+0000 7f82dd839640 1 -- 172.21.15.33:0/2711920317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d819f3c0 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.529+0000 7f82dd839640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d8108020 0x7f82d819ebf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c970 tx=0x7f82d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.529+0000 7f82d67fc640 1 -- 172.21.15.33:0/2711920317 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0007bb0 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.529+0000 7f82d67fc640 1 -- 172.21.15.33:0/2711920317 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82d0007d50 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.529+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d819f6b0 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.529+0000 7f82d67fc640 1 -- 172.21.15.33:0/2711920317 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d0004e60 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.529+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d81a5d70 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.530+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d8102f40 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.532+0000 7f82d67fc640 1 -- 172.21.15.33:0/2711920317 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f82d0005000 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.533+0000 7f82d67fc640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f82c00b5370 0x7f82c00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.533+0000 7f82dd038640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f82c00b5370 0x7f82c00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.533+0000 7f82d67fc640 1 -- 172.21.15.33:0/2711920317 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f82d00e5dd0 con 0x7f82d8108020 2024-09-10T21:02:59.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.533+0000 7f82dd038640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f82c00b5370 0x7f82c00b7830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005fd0 tx=0x7f82c8009870 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:59.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.536+0000 7f82d67fc640 1 -- 172.21.15.33:0/2711920317 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f82d0014030 con 0x7f82d8108020 2024-09-10T21:02:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.426+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1484182835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4108020 msgr2=0x7ff0f410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.426+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1484182835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4108020 0x7ff0f410e820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff0ec009900 tx=0x7ff0ec02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.426+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1484182835 shutdown_connections 2024-09-10T21:02:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.426+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1484182835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4108020 0x7ff0f410e820 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.426+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1484182835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f41076a0 0x7ff0f4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.426+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1484182835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f4101c90 0x7ff0f4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.426+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1484182835 >> 172.21.15.33:0/1484182835 conn(0x7ff0f40fd700 msgr2=0x7ff0f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.427+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1484182835 shutdown_connections 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.427+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1484182835 wait complete. 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.427+0000 7ff0f9c79640 1 Processor -- start 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.427+0000 7ff0f9c79640 1 -- start start 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.427+0000 7ff0f9c79640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4101c90 0x7ff0f4199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f9c79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f41076a0 0x7ff0f41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f9c79640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f4108020 0x7ff0f419ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f9c79640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4111ca0 con 0x7ff0f4101c90 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f9c79640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4111b20 con 0x7ff0f4108020 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f9c79640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f4111e20 con 0x7ff0f41076a0 2024-09-10T21:02:59.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4101c90 0x7ff0f4199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4101c90 0x7ff0f4199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49600/0 (socket says 172.21.15.33:49600) 2024-09-10T21:02:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 -- 172.21.15.33:0/1413126549 learned_addr learned my addr 172.21.15.33:0/1413126549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:02:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f3fff640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f4108020 0x7ff0f419ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f2ffd640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f41076a0 0x7ff0f41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.683 INFO:teuthology.orchestra.run.smithi033.stdout:249108103203 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 -- 172.21.15.33:0/1413126549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f41076a0 msgr2=0x7ff0f41996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f41076a0 0x7ff0f41996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 -- 172.21.15.33:0/1413126549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f4108020 msgr2=0x7ff0f419ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f4108020 0x7ff0f419ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 -- 172.21.15.33:0/1413126549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f419f240 con 0x7ff0f4101c90 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f37fe640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4101c90 0x7ff0f4199190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400cce0 tx=0x7ff0e400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f0ff9640 1 -- 172.21.15.33:0/1413126549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e400f650 con 0x7ff0f4101c90 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f419f530 con 0x7ff0f4101c90 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f0ff9640 1 -- 172.21.15.33:0/1413126549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e400fca0 con 0x7ff0f4101c90 2024-09-10T21:02:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f0ff9640 1 -- 172.21.15.33:0/1413126549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4018b40 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.428+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f41a5bf0 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.429+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f4102070 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.431+0000 7ff0f0ff9640 1 -- 172.21.15.33:0/1413126549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff0e400f7f0 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.431+0000 7ff0f0ff9640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0e00b5370 0x7ff0e00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.432+0000 7ff0f2ffd640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0e00b5370 0x7ff0e00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.432+0000 7ff0f0ff9640 1 -- 172.21.15.33:0/1413126549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff0e40e6350 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.432+0000 7ff0f2ffd640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0e00b5370 0x7ff0e00b7830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff0f41000c0 tx=0x7ff0dc005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.436+0000 7ff0f0ff9640 1 -- 172.21.15.33:0/1413126549 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff0e4014030 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.634+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7ff0f41a6010 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.634+0000 7ff0f0ff9640 1 -- 172.21.15.33:0/1413126549 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff0e40ad070 con 0x7ff0f4101c90 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0e00b5370 msgr2=0x7ff0e00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0e00b5370 0x7ff0e00b7830 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff0f41000c0 tx=0x7ff0dc005c50 comp rx=0 tx=0).stop 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4101c90 msgr2=0x7ff0f4199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4101c90 0x7ff0f4199190 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff0e400cce0 tx=0x7ff0e400a620 comp rx=0 tx=0).stop 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 shutdown_connections 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0e00b5370 0x7ff0e00b7830 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0f4108020 0x7ff0f419ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0f41076a0 0x7ff0f41996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 --2- 172.21.15.33:0/1413126549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0f4101c90 0x7ff0f4199190 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 >> 172.21.15.33:0/1413126549 conn(0x7ff0f40fd700 msgr2=0x7ff0f41066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 shutdown_connections 2024-09-10T21:02:59.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.637+0000 7ff0f9c79640 1 -- 172.21.15.33:0/1413126549 wait complete. 2024-09-10T21:02:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.727+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f82d8102070 con 0x7f82d8108020 2024-09-10T21:02:59.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.728+0000 7f82d67fc640 1 -- 172.21.15.33:0/2711920317 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f82d00acaf0 con 0x7f82d8108020 2024-09-10T21:02:59.732 INFO:teuthology.orchestra.run.smithi033.stdout:150323855415 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f82c00b5370 msgr2=0x7f82c00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f82c00b5370 0x7f82c00b7830 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f82c8005fd0 tx=0x7f82c8009870 comp rx=0 tx=0).stop 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d8108020 msgr2=0x7f82d819ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d8108020 0x7f82d819ebf0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82d000c970 tx=0x7f82d000ce30 comp rx=0 tx=0).stop 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 shutdown_connections 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f82c00b5370 0x7f82c00b7830 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82d8108020 0x7f82d819ebf0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f82d81076a0 0x7f82d8199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 --2- 172.21.15.33:0/2711920317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82d8101c90 0x7f82d8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:02:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 >> 172.21.15.33:0/2711920317 conn(0x7f82d80fd700 msgr2=0x7f82d806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:02:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 shutdown_connections 2024-09-10T21:02:59.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:02:59.730+0000 7f82df29c640 1 -- 172.21.15.33:0/2711920317 wait complete. 2024-09-10T21:02:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:59 smithi033 ceph-mon[30860]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T21:02:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1413126549' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T21:02:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:02:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2711920317' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T21:03:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:59 smithi104 ceph-mon[35325]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T21:03:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1413126549' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T21:03:00.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:02:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2711920317' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T21:03:00.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 -- 172.21.15.33:0/2504662321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb101076e0 msgr2=0x7fcb10107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 --2- 172.21.15.33:0/2504662321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb101076e0 0x7fcb10107b20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcb04009900 tx=0x7fcb0402a410 comp rx=0 tx=0).stop 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 -- 172.21.15.33:0/2504662321 shutdown_connections 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 --2- 172.21.15.33:0/2504662321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb10108060 0x7fcb1010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 --2- 172.21.15.33:0/2504662321 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb101076e0 0x7fcb10107b20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 --2- 172.21.15.33:0/2504662321 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb10101cd0 0x7fcb101020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 -- 172.21.15.33:0/2504662321 >> 172.21.15.33:0/2504662321 conn(0x7fcb100fd700 msgr2=0x7fcb100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 -- 172.21.15.33:0/2504662321 shutdown_connections 2024-09-10T21:03:00.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.106+0000 7fcb1485d640 1 -- 172.21.15.33:0/2504662321 wait complete. 2024-09-10T21:03:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.107+0000 7fcb1485d640 1 Processor -- start 2024-09-10T21:03:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.107+0000 7fcb1485d640 1 -- start start 2024-09-10T21:03:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb1485d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb10101cd0 0x7fcb101982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb1485d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb101076e0 0x7fcb10198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb1485d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb10108060 0x7fcb1019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb1485d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10111d60 con 0x7fcb10101cd0 2024-09-10T21:03:00.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb1485d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10111be0 con 0x7fcb10108060 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb1485d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb10111ee0 con 0x7fcb101076e0 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb10101cd0 0x7fcb101982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb10101cd0 0x7fcb101982d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49622/0 (socket says 172.21.15.33:49622) 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 -- 172.21.15.33:0/3827135345 learned_addr learned my addr 172.21.15.33:0/3827135345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0dd9b640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb101076e0 0x7fcb10198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0ed9d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb10108060 0x7fcb1019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 -- 172.21.15.33:0/3827135345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb101076e0 msgr2=0x7fcb10198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb101076e0 0x7fcb10198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 -- 172.21.15.33:0/3827135345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb10108060 msgr2=0x7fcb1019dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb10108060 0x7fcb1019dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 -- 172.21.15.33:0/3827135345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019e380 con 0x7fcb10101cd0 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0ed9d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb10108060 0x7fcb1019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:00.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcb0e59c640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb10101cd0 0x7fcb101982d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000ba50 tx=0x7fcb0000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.108+0000 7fcaff7fe640 1 -- 172.21.15.33:0/3827135345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb00019070 con 0x7fcb10101cd0 2024-09-10T21:03:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.109+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019e670 con 0x7fcb10101cd0 2024-09-10T21:03:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.109+0000 7fcaff7fe640 1 -- 172.21.15.33:0/3827135345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb0000cbe0 con 0x7fcb10101cd0 2024-09-10T21:03:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.109+0000 7fcaff7fe640 1 -- 172.21.15.33:0/3827135345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb000075d0 con 0x7fcb10101cd0 2024-09-10T21:03:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.109+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb1006b510 con 0x7fcb10101cd0 2024-09-10T21:03:00.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.110+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb10102130 con 0x7fcb10101cd0 2024-09-10T21:03:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.111+0000 7fcaff7fe640 1 -- 172.21.15.33:0/3827135345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fcb00007830 con 0x7fcb10101cd0 2024-09-10T21:03:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.112+0000 7fcaff7fe640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcaf80b7580 0x7fcaf80b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.112+0000 7fcb0dd9b640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcaf80b7580 0x7fcaf80b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.112+0000 7fcaff7fe640 1 -- 172.21.15.33:0/3827135345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcb000e5580 con 0x7fcb10101cd0 2024-09-10T21:03:00.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.112+0000 7fcb0dd9b640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcaf80b7580 0x7fcaf80b9a40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb04031440 tx=0x7fcb04005e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.116+0000 7fcaff7fe640 1 -- 172.21.15.33:0/3827135345 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcb000abc50 con 0x7fcb10101cd0 2024-09-10T21:03:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:59 smithi174 ceph-mon[35635]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-10T21:03:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1413126549' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-10T21:03:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:02:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2711920317' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T21:03:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.303+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcb1006c020 con 0x7fcb10101cd0 2024-09-10T21:03:00.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.304+0000 7fcaff7fe640 1 -- 172.21.15.33:0/3827135345 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcb000b0680 con 0x7fcb10101cd0 2024-09-10T21:03:00.308 INFO:teuthology.orchestra.run.smithi033.stdout:55834574923 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcaf80b7580 msgr2=0x7fcaf80b9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcaf80b7580 0x7fcaf80b9a40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcb04031440 tx=0x7fcb04005e80 comp rx=0 tx=0).stop 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb10101cd0 msgr2=0x7fcb101982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb10101cd0 0x7fcb101982d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcb0000ba50 tx=0x7fcb0000bf10 comp rx=0 tx=0).stop 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 shutdown_connections 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcaf80b7580 0x7fcaf80b9a40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb10108060 0x7fcb1019dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb101076e0 0x7fcb10198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 --2- 172.21.15.33:0/3827135345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb10101cd0 0x7fcb101982d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 >> 172.21.15.33:0/3827135345 conn(0x7fcb100fd700 msgr2=0x7fcb100ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:00.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 shutdown_connections 2024-09-10T21:03:00.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:00.306+0000 7fcb1485d640 1 -- 172.21.15.33:0/3827135345 wait complete. 2024-09-10T21:03:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:00 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3827135345' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T21:03:01.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:00 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3827135345' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T21:03:01.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:00 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3827135345' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T21:03:01.742 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495657 got 197568495659 for osd.7 2024-09-10T21:03:01.742 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:01 smithi104 ceph-mon[35325]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:02.106 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:01 smithi033 ceph-mon[30860]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:01 smithi174 ceph-mon[35635]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:02.255 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574923 for osd.1 2024-09-10T21:03:02.255 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 -- 172.21.15.33:0/249230063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 msgr2=0x7ff924108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 --2- 172.21.15.33:0/249230063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff924108c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff918009900 tx=0x7ff91802a410 comp rx=0 tx=0).stop 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 -- 172.21.15.33:0/249230063 shutdown_connections 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 --2- 172.21.15.33:0/249230063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924109150 0x7ff92410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 --2- 172.21.15.33:0/249230063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff924108c10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 --2- 172.21.15.33:0/249230063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924102dc0 0x7ff9241031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 -- 172.21.15.33:0/249230063 >> 172.21.15.33:0/249230063 conn(0x7ff9240fe830 msgr2=0x7ff924100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 -- 172.21.15.33:0/249230063 shutdown_connections 2024-09-10T21:03:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 -- 172.21.15.33:0/249230063 wait complete. 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 Processor -- start 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.905+0000 7ff92adea640 1 -- start start 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff92adea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924102dc0 0x7ff924199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff92adea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff9241997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff92adea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924109150 0x7ff92419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff92adea640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff924112dd0 con 0x7ff9241087d0 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff92adea640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff924112c50 con 0x7ff924102dc0 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff92adea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff924112f50 con 0x7ff924109150 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff9241997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff9241997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49630/0 (socket says 172.21.15.33:49630) 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 -- 172.21.15.33:0/3168244738 learned_addr learned my addr 172.21.15.33:0/3168244738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff928b86640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924102dc0 0x7ff924199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff929387640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924109150 0x7ff92419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 -- 172.21.15.33:0/3168244738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924109150 msgr2=0x7ff92419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924109150 0x7ff92419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 -- 172.21.15.33:0/3168244738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924102dc0 msgr2=0x7ff924199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924102dc0 0x7ff924199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 -- 172.21.15.33:0/3168244738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92419f330 con 0x7ff9241087d0 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff929387640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924109150 0x7ff92419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff923fff640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff9241997c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff918005940 tx=0x7ff918004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.906+0000 7ff921ffb640 1 -- 172.21.15.33:0/3168244738 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91803f030 con 0x7ff9241087d0 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.907+0000 7ff921ffb640 1 -- 172.21.15.33:0/3168244738 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff918003aa0 con 0x7ff9241087d0 2024-09-10T21:03:02.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.907+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92419f5c0 con 0x7ff9241087d0 2024-09-10T21:03:02.365 INFO:teuthology.orchestra.run.smithi033.stdout:270582939678 2024-09-10T21:03:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.907+0000 7ff921ffb640 1 -- 172.21.15.33:0/3168244738 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff918035070 con 0x7ff9241087d0 2024-09-10T21:03:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.907+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241a5ce0 con 0x7ff9241087d0 2024-09-10T21:03:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.908+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9241031a0 con 0x7ff9241087d0 2024-09-10T21:03:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.909+0000 7ff921ffb640 1 -- 172.21.15.33:0/3168244738 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff918003c50 con 0x7ff9241087d0 2024-09-10T21:03:02.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.910+0000 7ff921ffb640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff90c0b5370 0x7ff90c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.910+0000 7ff928b86640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff90c0b5370 0x7ff90c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.910+0000 7ff921ffb640 1 -- 172.21.15.33:0/3168244738 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff918106a40 con 0x7ff9241087d0 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.911+0000 7ff928b86640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff90c0b5370 0x7ff90c0b7830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff914005d00 tx=0x7ff914005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:01.913+0000 7ff921ffb640 1 -- 172.21.15.33:0/3168244738 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff9180cd760 con 0x7ff9241087d0 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.102+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ff92419a3b0 con 0x7ff9241087d0 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.103+0000 7ff921ffb640 1 -- 172.21.15.33:0/3168244738 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff9180cd060 con 0x7ff9241087d0 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff90c0b5370 msgr2=0x7ff90c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff90c0b5370 0x7ff90c0b7830 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff914005d00 tx=0x7ff914005c50 comp rx=0 tx=0).stop 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 msgr2=0x7ff9241997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff9241997c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff918005940 tx=0x7ff918004270 comp rx=0 tx=0).stop 2024-09-10T21:03:02.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 shutdown_connections 2024-09-10T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff90c0b5370 0x7ff90c0b7830 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924109150 0x7ff92419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241087d0 0x7ff9241997c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 --2- 172.21.15.33:0/3168244738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924102dc0 0x7ff924199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 >> 172.21.15.33:0/3168244738 conn(0x7ff9240fe830 msgr2=0x7ff924100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 shutdown_connections 2024-09-10T21:03:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.105+0000 7ff92adea640 1 -- 172.21.15.33:0/3168244738 wait complete. 2024-09-10T21:03:02.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.777+0000 7fb0dc277640 1 -- 172.21.15.33:0/2583469080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 msgr2=0x7fb0d4108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.777+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2583469080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d4108c10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009900 tx=0x7fb0c402a410 comp rx=0 tx=0).stop 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 -- 172.21.15.33:0/2583469080 shutdown_connections 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2583469080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0d4109150 0x7fb0d410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2583469080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d4108c10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2583469080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4102dc0 0x7fb0d41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 -- 172.21.15.33:0/2583469080 >> 172.21.15.33:0/2583469080 conn(0x7fb0d40fe830 msgr2=0x7fb0d4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 -- 172.21.15.33:0/2583469080 shutdown_connections 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 -- 172.21.15.33:0/2583469080 wait complete. 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 Processor -- start 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.778+0000 7fb0dc277640 1 -- start start 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0dc277640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4102dc0 0x7fb0d41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0dc277640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0dc277640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0d4109150 0x7fb0d419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0dc277640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4112e10 con 0x7fb0d41087d0 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0dc277640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4112c90 con 0x7fb0d4109150 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0dc277640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0d4112f90 con 0x7fb0d4102dc0 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d41997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49650/0 (socket says 172.21.15.33:49650) 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 -- 172.21.15.33:0/2647025023 learned_addr learned my addr 172.21.15.33:0/2647025023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0da013640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4102dc0 0x7fb0d41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0da814640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0d4109150 0x7fb0d419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 -- 172.21.15.33:0/2647025023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4102dc0 msgr2=0x7fb0d41992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4102dc0 0x7fb0d41992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 -- 172.21.15.33:0/2647025023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0d4109150 msgr2=0x7fb0d419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0d4109150 0x7fb0d419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 -- 172.21.15.33:0/2647025023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0d419f350 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0da814640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0d4109150 0x7fb0d419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0d9812640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d41997e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4005940 tx=0x7fb0c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0cb7fe640 1 -- 172.21.15.33:0/2647025023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c403f030 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0d419f5e0 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0cb7fe640 1 -- 172.21.15.33:0/2647025023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0c4003cb0 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.779+0000 7fb0cb7fe640 1 -- 172.21.15.33:0/2647025023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4035070 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.780+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0d41a5d00 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.781+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0d41031a0 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.782+0000 7fb0cb7fe640 1 -- 172.21.15.33:0/2647025023 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb0c403bbf0 con 0x7fb0d41087d0 2024-09-10T21:03:02.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.783+0000 7fb0cb7fe640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0c00b5370 0x7fb0c00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:02.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.783+0000 7fb0da013640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0c00b5370 0x7fb0c00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:02.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.783+0000 7fb0cb7fe640 1 -- 172.21.15.33:0/2647025023 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb0c41068e0 con 0x7fb0d41087d0 2024-09-10T21:03:02.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.784+0000 7fb0da013640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0c00b5370 0x7fb0c00b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000bbe0 tx=0x7fb0d000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:02.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.787+0000 7fb0cb7fe640 1 -- 172.21.15.33:0/2647025023 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb0c40cd640 con 0x7fb0d41087d0 2024-09-10T21:03:02.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.977+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fb0d419a3d0 con 0x7fb0d41087d0 2024-09-10T21:03:02.982 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:02 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3168244738' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T21:03:02.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.978+0000 7fb0cb7fe640 1 -- 172.21.15.33:0/2647025023 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb0c40ccf40 con 0x7fb0d41087d0 2024-09-10T21:03:02.982 INFO:teuthology.orchestra.run.smithi033.stdout:128849018941 2024-09-10T21:03:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0c00b5370 msgr2=0x7fb0c00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0c00b5370 0x7fb0c00b7830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000bbe0 tx=0x7fb0d000b390 comp rx=0 tx=0).stop 2024-09-10T21:03:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 msgr2=0x7fb0d41997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:02.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d41997e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4005940 tx=0x7fb0c40042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 shutdown_connections 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0c00b5370 0x7fb0c00b7830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0d4109150 0x7fb0d419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0d41087d0 0x7fb0d41997e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 --2- 172.21.15.33:0/2647025023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0d4102dc0 0x7fb0d41992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 >> 172.21.15.33:0/2647025023 conn(0x7fb0d40fe830 msgr2=0x7fb0d4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 shutdown_connections 2024-09-10T21:03:02.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:02.980+0000 7fb0dc277640 1 -- 172.21.15.33:0/2647025023 wait complete. 2024-09-10T21:03:03.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:02 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3168244738' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T21:03:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:02 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3168244738' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-10T21:03:03.738 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018939 got 128849018941 for osd.4 2024-09-10T21:03:03.738 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:04.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:03 smithi033 ceph-mon[30860]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:04.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:03 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2647025023' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.003+0000 7f65f3708640 1 -- 172.21.15.33:0/1782487396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec10d100 msgr2=0x7f65ec10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.003+0000 7f65f3708640 1 --2- 172.21.15.33:0/1782487396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec10d100 0x7f65ec10f550 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009ca0 tx=0x7f65e402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.003+0000 7f65f3708640 1 -- 172.21.15.33:0/1782487396 shutdown_connections 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.003+0000 7f65f3708640 1 --2- 172.21.15.33:0/1782487396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec10d100 0x7f65ec10f550 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.003+0000 7f65f3708640 1 --2- 172.21.15.33:0/1782487396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec104550 0x7f65ec10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.003+0000 7f65f3708640 1 --2- 172.21.15.33:0/1782487396 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec103b50 0x7f65ec103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.003+0000 7f65f3708640 1 -- 172.21.15.33:0/1782487396 >> 172.21.15.33:0/1782487396 conn(0x7f65ec0fd700 msgr2=0x7f65ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.004+0000 7f65f3708640 1 -- 172.21.15.33:0/1782487396 shutdown_connections 2024-09-10T21:03:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.004+0000 7f65f3708640 1 -- 172.21.15.33:0/1782487396 wait complete. 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 Processor -- start 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 -- start start 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec103b50 0x7f65ec199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec104550 0x7f65ec199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec10d100 0x7f65ec19eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec112d60 con 0x7f65ec104550 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec112be0 con 0x7f65ec10d100 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f3708640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec112ee0 con 0x7f65ec103b50 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f0ca3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec104550 0x7f65ec199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f0ca3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec104550 0x7f65ec199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49692/0 (socket says 172.21.15.33:49692) 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f0ca3640 1 -- 172.21.15.33:0/4072380224 learned_addr learned my addr 172.21.15.33:0/4072380224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f1ca5640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec10d100 0x7f65ec19eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.005+0000 7f65f14a4640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec103b50 0x7f65ec199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f0ca3640 1 -- 172.21.15.33:0/4072380224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec103b50 msgr2=0x7f65ec199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f0ca3640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec103b50 0x7f65ec199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f0ca3640 1 -- 172.21.15.33:0/4072380224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec10d100 msgr2=0x7f65ec19eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f0ca3640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec10d100 0x7f65ec19eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f0ca3640 1 -- 172.21.15.33:0/4072380224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec19f300 con 0x7f65ec104550 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f14a4640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec103b50 0x7f65ec199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f0ca3640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec104550 0x7f65ec199790 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c970 tx=0x7f65dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65e27fc640 1 -- 172.21.15.33:0/4072380224 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc007bb0 con 0x7f65ec104550 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec19f5f0 con 0x7f65ec104550 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65e27fc640 1 -- 172.21.15.33:0/4072380224 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65dc007d50 con 0x7f65ec104550 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65e27fc640 1 -- 172.21.15.33:0/4072380224 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00f450 con 0x7f65ec104550 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.006+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1a5cb0 con 0x7f65ec104550 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.007+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec103f30 con 0x7f65ec104550 2024-09-10T21:03:04.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.010+0000 7f65e27fc640 1 -- 172.21.15.33:0/4072380224 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f65dc004f00 con 0x7f65ec104550 2024-09-10T21:03:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.010+0000 7f65e27fc640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65d80b53f0 0x7f65d80b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.011+0000 7f65f14a4640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65d80b53f0 0x7f65d80b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.011+0000 7f65e27fc640 1 -- 172.21.15.33:0/4072380224 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f65dc0e5e10 con 0x7f65ec104550 2024-09-10T21:03:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.011+0000 7f65f14a4640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65d80b53f0 0x7f65d80b78b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65e80058c0 tx=0x7f65e8005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:04.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.014+0000 7f65e27fc640 1 -- 172.21.15.33:0/4072380224 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f65dc014030 con 0x7f65ec104550 2024-09-10T21:03:04.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:03 smithi104 ceph-mon[35325]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:04.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:03 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2647025023' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T21:03:04.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.898+0000 7ff9c9776640 1 -- 172.21.15.33:0/1085808706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 msgr2=0x7ff9c410cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.898+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1085808706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c410cae0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009900 tx=0x7ff9ac02a410 comp rx=0 tx=0).stop 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.898+0000 7ff9c9776640 1 -- 172.21.15.33:0/1085808706 shutdown_connections 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.898+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1085808706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c410d130 0x7ff9c410f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.898+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1085808706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c410cae0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.898+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1085808706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9c4103cd0 0x7ff9c41040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.898+0000 7ff9c9776640 1 -- 172.21.15.33:0/1085808706 >> 172.21.15.33:0/1085808706 conn(0x7ff9c40fd770 msgr2=0x7ff9c40ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.899+0000 7ff9c9776640 1 -- 172.21.15.33:0/1085808706 shutdown_connections 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.899+0000 7ff9c9776640 1 -- 172.21.15.33:0/1085808706 wait complete. 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.899+0000 7ff9c9776640 1 Processor -- start 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.899+0000 7ff9c9776640 1 -- start start 2024-09-10T21:03:04.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.899+0000 7ff9c9776640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c4103cd0 0x7ff9c4101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c9776640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c4101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c9776640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9c410d130 0x7ff9c4102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c9776640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4112eb0 con 0x7ff9c41045f0 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c9776640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4112d30 con 0x7ff9c410d130 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c9776640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c4113030 con 0x7ff9c4103cd0 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c4101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c4101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49668/0 (socket says 172.21.15.33:49668) 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 -- 172.21.15.33:0/1300666797 learned_addr learned my addr 172.21.15.33:0/1300666797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:04.143 INFO:teuthology.orchestra.run.smithi033.stdout:176093659186 2024-09-10T21:03:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c2ffd640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c4103cd0 0x7ff9c4101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c37fe640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9c410d130 0x7ff9c4102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 -- 172.21.15.33:0/1300666797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c4103cd0 msgr2=0x7ff9c4101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c4103cd0 0x7ff9c4101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 -- 172.21.15.33:0/1300666797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9c410d130 msgr2=0x7ff9c4102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9c410d130 0x7ff9c4102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 -- 172.21.15.33:0/1300666797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c41a7840 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c27fc640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c4101ec0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac031440 tx=0x7ff9ac003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9bbfff640 1 -- 172.21.15.33:0/1300666797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac03f070 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c41a7ad0 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9bbfff640 1 -- 172.21.15.33:0/1300666797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ac031d60 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9bbfff640 1 -- 172.21.15.33:0/1300666797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac035030 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.900+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41a8000 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.902+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9c4104130 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.904+0000 7ff9bbfff640 1 -- 172.21.15.33:0/1300666797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7ff9ac03b580 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.905+0000 7ff9bbfff640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9b00b5370 0x7ff9b00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.905+0000 7ff9c2ffd640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9b00b5370 0x7ff9b00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.905+0000 7ff9bbfff640 1 -- 172.21.15.33:0/1300666797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff9ac106e30 con 0x7ff9c41045f0 2024-09-10T21:03:04.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.905+0000 7ff9c2ffd640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9b00b5370 0x7ff9b00b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400a400 tx=0x7ff9b400d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:03.908+0000 7ff9bbfff640 1 -- 172.21.15.33:0/1300666797 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff9ac0cdb50 con 0x7ff9c41045f0 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.100+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff9c402e5c0 con 0x7ff9c41045f0 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.101+0000 7ff9bbfff640 1 -- 172.21.15.33:0/1300666797 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff9ac0cd450 con 0x7ff9c41045f0 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9b00b5370 msgr2=0x7ff9b00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9b00b5370 0x7ff9b00b7830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400a400 tx=0x7ff9b400d040 comp rx=0 tx=0).stop 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 msgr2=0x7ff9c4101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c4101ec0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac031440 tx=0x7ff9ac003c80 comp rx=0 tx=0).stop 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 shutdown_connections 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9b00b5370 0x7ff9b00b7830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9c410d130 0x7ff9c4102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9c41045f0 0x7ff9c4101ec0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 --2- 172.21.15.33:0/1300666797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9c4103cd0 0x7ff9c4101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 >> 172.21.15.33:0/1300666797 conn(0x7ff9c40fd770 msgr2=0x7ff9c40fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 shutdown_connections 2024-09-10T21:03:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.103+0000 7ff9c9776640 1 -- 172.21.15.33:0/1300666797 wait complete. 2024-09-10T21:03:04.147 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:03 smithi174 ceph-mon[35635]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:04.147 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:03 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2647025023' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T21:03:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.207+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f65ec107be0 con 0x7f65ec104550 2024-09-10T21:03:04.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.207+0000 7f65e27fc640 1 -- 172.21.15.33:0/4072380224 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f65dc0acbb0 con 0x7f65ec104550 2024-09-10T21:03:04.211 INFO:teuthology.orchestra.run.smithi033.stdout:73014444103 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65d80b53f0 msgr2=0x7f65d80b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65d80b53f0 0x7f65d80b78b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f65e80058c0 tx=0x7f65e8005ec0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec104550 msgr2=0x7f65ec199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec104550 0x7f65ec199790 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00c970 tx=0x7f65dc00ce30 comp rx=0 tx=0).stop 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 shutdown_connections 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f65d80b53f0 0x7f65d80b78b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f65ec10d100 0x7f65ec19eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65ec104550 0x7f65ec199790 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 --2- 172.21.15.33:0/4072380224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ec103b50 0x7f65ec199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 >> 172.21.15.33:0/4072380224 conn(0x7f65ec0fd700 msgr2=0x7f65ec0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 shutdown_connections 2024-09-10T21:03:04.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:04.210+0000 7f65f3708640 1 -- 172.21.15.33:0/4072380224 wait complete. 2024-09-10T21:03:04.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939674 got 270582939678 for osd.10 2024-09-10T21:03:04.750 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:05.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:04 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1300666797' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T21:03:05.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:04 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4072380224' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T21:03:05.076 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:04 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1300666797' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T21:03:05.077 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:04 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4072380224' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T21:03:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:04 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1300666797' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T21:03:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:04 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4072380224' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T21:03:05.408 INFO:tasks.cephadm.ceph_manager.ceph:need seq 73014444100 got 73014444103 for osd.2 2024-09-10T21:03:05.408 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:06.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:05 smithi104 ceph-mon[35325]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:05 smithi174 ceph-mon[35635]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:06.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:05 smithi033 ceph-mon[30860]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:06.188 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855412 got 150323855415 for osd.5 2024-09-10T21:03:06.189 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:06.640 INFO:tasks.cephadm.ceph_manager.ceph:need seq 176093659182 got 176093659186 for osd.6 2024-09-10T21:03:06.641 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:07.443 INFO:tasks.cephadm.ceph_manager.ceph:need seq 249108103200 got 249108103203 for osd.9 2024-09-10T21:03:07.443 DEBUG:teuthology.parallel:result is None 2024-09-10T21:03:07.443 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T21:03:07.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T21:03:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:08.011 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:07 smithi033 ceph-mon[30860]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:07 smithi104 ceph-mon[35325]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:07 smithi174 ceph-mon[35635]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.219+0000 7f2a95081640 1 -- 172.21.15.33:0/4286578890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901021d0 msgr2=0x7f2a9010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.219+0000 7f2a95081640 1 --2- 172.21.15.33:0/4286578890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901021d0 0x7f2a9010ef20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009900 tx=0x7f2a8402a480 comp rx=0 tx=0).stop 2024-09-10T21:03:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.219+0000 7f2a95081640 1 -- 172.21.15.33:0/4286578890 shutdown_connections 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.219+0000 7f2a95081640 1 --2- 172.21.15.33:0/4286578890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a9010f460 0x7f2a90111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.219+0000 7f2a95081640 1 --2- 172.21.15.33:0/4286578890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901021d0 0x7f2a9010ef20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.219+0000 7f2a95081640 1 --2- 172.21.15.33:0/4286578890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a901018b0 0x7f2a90101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.219+0000 7f2a95081640 1 -- 172.21.15.33:0/4286578890 >> 172.21.15.33:0/4286578890 conn(0x7f2a900fd700 msgr2=0x7f2a900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.220+0000 7f2a95081640 1 -- 172.21.15.33:0/4286578890 shutdown_connections 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.220+0000 7f2a95081640 1 -- 172.21.15.33:0/4286578890 wait complete. 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.220+0000 7f2a95081640 1 Processor -- start 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a95081640 1 -- start start 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a95081640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901018b0 0x7f2a9019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a95081640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a901021d0 0x7f2a9019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a95081640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a9010f460 0x7f2a901a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a95081640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90114f50 con 0x7f2a901018b0 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a95081640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a90114dd0 con 0x7f2a9010f460 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a95081640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a901150d0 con 0x7f2a901021d0 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901018b0 0x7f2a9019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901018b0 0x7f2a9019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49720/0 (socket says 172.21.15.33:49720) 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 -- 172.21.15.33:0/2155301862 learned_addr learned my addr 172.21.15.33:0/2155301862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8f59e640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a9010f460 0x7f2a901a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8e59c640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a901021d0 0x7f2a9019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 -- 172.21.15.33:0/2155301862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a901021d0 msgr2=0x7f2a9019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a901021d0 0x7f2a9019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 -- 172.21.15.33:0/2155301862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a9010f460 msgr2=0x7f2a901a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a9010f460 0x7f2a901a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.221+0000 7f2a8ed9d640 1 -- 172.21.15.33:0/2155301862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a901a14b0 con 0x7f2a901018b0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.222+0000 7f2a8e59c640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a901021d0 0x7f2a9019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.222+0000 7f2a8ed9d640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901018b0 0x7f2a9019b400 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c920 tx=0x7f2a8000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.222+0000 7f2a77fff640 1 -- 172.21.15.33:0/2155301862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80007c70 con 0x7f2a901018b0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.222+0000 7f2a77fff640 1 -- 172.21.15.33:0/2155301862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a80007e10 con 0x7f2a901018b0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.222+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a901a17a0 con 0x7f2a901018b0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.222+0000 7f2a77fff640 1 -- 172.21.15.33:0/2155301862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a80018820 con 0x7f2a901018b0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.222+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a901a8030 con 0x7f2a901018b0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.223+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9006b130 con 0x7f2a901018b0 2024-09-10T21:03:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.225+0000 7f2a77fff640 1 -- 172.21.15.33:0/2155301862 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f2a8000f450 con 0x7f2a901018b0 2024-09-10T21:03:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.226+0000 7f2a77fff640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a780b53f0 0x7f2a780b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.226+0000 7f2a8e59c640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a780b53f0 0x7f2a780b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.226+0000 7f2a77fff640 1 -- 172.21.15.33:0/2155301862 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2a800e6020 con 0x7f2a901018b0 2024-09-10T21:03:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.227+0000 7f2a8e59c640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a780b53f0 0x7f2a780b78b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a84031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.229+0000 7f2a77fff640 1 -- 172.21.15.33:0/2155301862 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2a80014030 con 0x7f2a901018b0 2024-09-10T21:03:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.432+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2a90111640 con 0x7f2a780b53f0 2024-09-10T21:03:08.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.437+0000 7f2a77fff640 1 -- 172.21.15.33:0/2155301862 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259011 (secure 0 0 0) 0x7f2a90111640 con 0x7f2a780b53f0 2024-09-10T21:03:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:08.444 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a780b53f0 msgr2=0x7f2a780b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a780b53f0 0x7f2a780b78b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2a84000c00 tx=0x7f2a84031400 comp rx=0 tx=0).stop 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901018b0 msgr2=0x7f2a9019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901018b0 0x7f2a9019b400 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2a8000c920 tx=0x7f2a8000cde0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 shutdown_connections 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a780b53f0 0x7f2a780b78b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a9010f460 0x7f2a901a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a901021d0 0x7f2a9019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 --2- 172.21.15.33:0/2155301862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a901018b0 0x7f2a9019b400 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:08.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 >> 172.21.15.33:0/2155301862 conn(0x7f2a900fd700 msgr2=0x7f2a90110840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 shutdown_connections 2024-09-10T21:03:08.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:08.441+0000 7f2a95081640 1 -- 172.21.15.33:0/2155301862 wait complete. 2024-09-10T21:03:09.120 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T21:03:08.099787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659758,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":648,"num_read_kb":428,"num_write":370,"num_write_kb":2289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":2953077,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355580,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586244,"statfs":{"total":1151940427776,"available":1151576313856,"internally_reserved":0,"allocated":10665984,"data_stored":6849294,"data_compressed":176985,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002754"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867485+0000","last_change":"2024-09-10T21:02:00.805421+0000","last_active":"2024-09-10T21:02:41.867485+0000","last_peered":"2024-09-10T21:02:41.867485+0000","last_clean":"2024-09-10T21:02:41.867485+0000","last_became_active":"2024-09-10T21:02:00.804274+0000","last_became_peered":"2024-09-10T21:02:00.804274+0000","last_unstale":"2024-09-10T21:02:41.867485+0000","last_undegraded":"2024-09-10T21:02:41.867485+0000","last_fullsized":"2024-09-10T21:02:41.867485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:12.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870587+0000","last_change":"2024-09-10T21:02:00.806104+0000","last_active":"2024-09-10T21:02:41.870587+0000","last_peered":"2024-09-10T21:02:41.870587+0000","last_clean":"2024-09-10T21:02:41.870587+0000","last_became_active":"2024-09-10T21:02:00.805510+0000","last_became_peered":"2024-09-10T21:02:00.805510+0000","last_unstale":"2024-09-10T21:02:41.870587+0000","last_undegraded":"2024-09-10T21:02:41.870587+0000","last_fullsized":"2024-09-10T21:02:41.870587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:37.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870594+0000","last_change":"2024-09-10T21:02:00.807691+0000","last_active":"2024-09-10T21:02:41.870594+0000","last_peered":"2024-09-10T21:02:41.870594+0000","last_clean":"2024-09-10T21:02:41.870594+0000","last_became_active":"2024-09-10T21:02:00.807364+0000","last_became_peered":"2024-09-10T21:02:00.807364+0000","last_unstale":"2024-09-10T21:02:41.870594+0000","last_undegraded":"2024-09-10T21:02:41.870594+0000","last_fullsized":"2024-09-10T21:02:41.870594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:27.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866175+0000","last_change":"2024-09-10T21:02:00.809542+0000","last_active":"2024-09-10T21:02:41.866175+0000","last_peered":"2024-09-10T21:02:41.866175+0000","last_clean":"2024-09-10T21:02:41.866175+0000","last_became_active":"2024-09-10T21:02:00.809034+0000","last_became_peered":"2024-09-10T21:02:00.809034+0000","last_unstale":"2024-09-10T21:02:41.866175+0000","last_undegraded":"2024-09-10T21:02:41.866175+0000","last_fullsized":"2024-09-10T21:02:41.866175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:29.923532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871352+0000","last_change":"2024-09-10T21:02:00.809355+0000","last_active":"2024-09-10T21:02:41.871352+0000","last_peered":"2024-09-10T21:02:41.871352+0000","last_clean":"2024-09-10T21:02:41.871352+0000","last_became_active":"2024-09-10T21:02:00.808863+0000","last_became_peered":"2024-09-10T21:02:00.808863+0000","last_unstale":"2024-09-10T21:02:41.871352+0000","last_undegraded":"2024-09-10T21:02:41.871352+0000","last_fullsized":"2024-09-10T21:02:41.871352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:29.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916934+0000","last_change":"2024-09-10T21:02:00.803756+0000","last_active":"2024-09-10T21:02:41.916934+0000","last_peered":"2024-09-10T21:02:41.916934+0000","last_clean":"2024-09-10T21:02:41.916934+0000","last_became_active":"2024-09-10T21:02:00.803406+0000","last_became_peered":"2024-09-10T21:02:00.803406+0000","last_unstale":"2024-09-10T21:02:41.916934+0000","last_undegraded":"2024-09-10T21:02:41.916934+0000","last_fullsized":"2024-09-10T21:02:41.916934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:53.437629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.869942+0000","last_change":"2024-09-10T21:02:00.808845+0000","last_active":"2024-09-10T21:02:41.869942+0000","last_peered":"2024-09-10T21:02:41.869942+0000","last_clean":"2024-09-10T21:02:41.869942+0000","last_became_active":"2024-09-10T21:02:00.807759+0000","last_became_peered":"2024-09-10T21:02:00.807759+0000","last_unstale":"2024-09-10T21:02:41.869942+0000","last_undegraded":"2024-09-10T21:02:41.869942+0000","last_fullsized":"2024-09-10T21:02:41.869942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.809389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977534+0000","last_change":"2024-09-10T21:02:00.811199+0000","last_active":"2024-09-10T21:02:41.977534+0000","last_peered":"2024-09-10T21:02:41.977534+0000","last_clean":"2024-09-10T21:02:41.977534+0000","last_became_active":"2024-09-10T21:02:00.810692+0000","last_became_peered":"2024-09-10T21:02:00.810692+0000","last_unstale":"2024-09-10T21:02:41.977534+0000","last_undegraded":"2024-09-10T21:02:41.977534+0000","last_fullsized":"2024-09-10T21:02:41.977534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:59.557973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868394+0000","last_change":"2024-09-10T21:02:00.812686+0000","last_active":"2024-09-10T21:02:41.868394+0000","last_peered":"2024-09-10T21:02:41.868394+0000","last_clean":"2024-09-10T21:02:41.868394+0000","last_became_active":"2024-09-10T21:02:00.812363+0000","last_became_peered":"2024-09-10T21:02:00.812363+0000","last_unstale":"2024-09-10T21:02:41.868394+0000","last_undegraded":"2024-09-10T21:02:41.868394+0000","last_fullsized":"2024-09-10T21:02:41.868394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:37.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917309+0000","last_change":"2024-09-10T21:02:00.812058+0000","last_active":"2024-09-10T21:02:41.917309+0000","last_peered":"2024-09-10T21:02:41.917309+0000","last_clean":"2024-09-10T21:02:41.917309+0000","last_became_active":"2024-09-10T21:02:00.811490+0000","last_became_peered":"2024-09-10T21:02:00.811490+0000","last_unstale":"2024-09-10T21:02:41.917309+0000","last_undegraded":"2024-09-10T21:02:41.917309+0000","last_fullsized":"2024-09-10T21:02:41.917309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:31.557485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870301+0000","last_change":"2024-09-10T21:02:00.809991+0000","last_active":"2024-09-10T21:02:41.870301+0000","last_peered":"2024-09-10T21:02:41.870301+0000","last_clean":"2024-09-10T21:02:41.870301+0000","last_became_active":"2024-09-10T21:02:00.809197+0000","last_became_peered":"2024-09-10T21:02:00.809197+0000","last_unstale":"2024-09-10T21:02:41.870301+0000","last_undegraded":"2024-09-10T21:02:41.870301+0000","last_fullsized":"2024-09-10T21:02:41.870301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.919031+0000","last_change":"2024-09-10T21:02:41.918484+0000","last_active":"2024-09-10T21:02:41.919031+0000","last_peered":"2024-09-10T21:02:41.919031+0000","last_clean":"2024-09-10T21:02:41.919031+0000","last_became_active":"2024-09-10T21:02:41.918068+0000","last_became_peered":"2024-09-10T21:02:41.918068+0000","last_unstale":"2024-09-10T21:02:41.919031+0000","last_undegraded":"2024-09-10T21:02:41.919031+0000","last_fullsized":"2024-09-10T21:02:41.919031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:08.954072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917107+0000","last_change":"2024-09-10T21:02:00.807280+0000","last_active":"2024-09-10T21:02:41.917107+0000","last_peered":"2024-09-10T21:02:41.917107+0000","last_clean":"2024-09-10T21:02:41.917107+0000","last_became_active":"2024-09-10T21:02:00.804247+0000","last_became_peered":"2024-09-10T21:02:00.804247+0000","last_unstale":"2024-09-10T21:02:41.917107+0000","last_undegraded":"2024-09-10T21:02:41.917107+0000","last_fullsized":"2024-09-10T21:02:41.917107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:28.163960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870460+0000","last_change":"2024-09-10T21:02:00.805723+0000","last_active":"2024-09-10T21:02:41.870460+0000","last_peered":"2024-09-10T21:02:41.870460+0000","last_clean":"2024-09-10T21:02:41.870460+0000","last_became_active":"2024-09-10T21:02:00.805104+0000","last_became_peered":"2024-09-10T21:02:00.805104+0000","last_unstale":"2024-09-10T21:02:41.870460+0000","last_undegraded":"2024-09-10T21:02:41.870460+0000","last_fullsized":"2024-09-10T21:02:41.870460+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.056884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870384+0000","last_change":"2024-09-10T21:02:00.810497+0000","last_active":"2024-09-10T21:02:41.870384+0000","last_peered":"2024-09-10T21:02:41.870384+0000","last_clean":"2024-09-10T21:02:41.870384+0000","last_became_active":"2024-09-10T21:02:00.809392+0000","last_became_peered":"2024-09-10T21:02:00.809392+0000","last_unstale":"2024-09-10T21:02:41.870384+0000","last_undegraded":"2024-09-10T21:02:41.870384+0000","last_fullsized":"2024-09-10T21:02:41.870384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:51.214771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867732+0000","last_change":"2024-09-10T21:02:00.813973+0000","last_active":"2024-09-10T21:02:41.867732+0000","last_peered":"2024-09-10T21:02:41.867732+0000","last_clean":"2024-09-10T21:02:41.867732+0000","last_became_active":"2024-09-10T21:02:00.813580+0000","last_became_peered":"2024-09-10T21:02:00.813580+0000","last_unstale":"2024-09-10T21:02:41.867732+0000","last_undegraded":"2024-09-10T21:02:41.867732+0000","last_fullsized":"2024-09-10T21:02:41.867732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:48.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867600+0000","last_change":"2024-09-10T21:02:00.804844+0000","last_active":"2024-09-10T21:02:41.867600+0000","last_peered":"2024-09-10T21:02:41.867600+0000","last_clean":"2024-09-10T21:02:41.867600+0000","last_became_active":"2024-09-10T21:02:00.803866+0000","last_became_peered":"2024-09-10T21:02:00.803866+0000","last_unstale":"2024-09-10T21:02:41.867600+0000","last_undegraded":"2024-09-10T21:02:41.867600+0000","last_fullsized":"2024-09-10T21:02:41.867600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:05.231877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865210+0000","last_change":"2024-09-10T21:02:00.801343+0000","last_active":"2024-09-10T21:02:41.865210+0000","last_peered":"2024-09-10T21:02:41.865210+0000","last_clean":"2024-09-10T21:02:41.865210+0000","last_became_active":"2024-09-10T21:02:00.801008+0000","last_became_peered":"2024-09-10T21:02:00.801008+0000","last_unstale":"2024-09-10T21:02:41.865210+0000","last_undegraded":"2024-09-10T21:02:41.865210+0000","last_fullsized":"2024-09-10T21:02:41.865210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:31.167925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.918116+0000","last_change":"2024-09-10T21:02:41.917997+0000","last_active":"2024-09-10T21:02:41.918116+0000","last_peered":"2024-09-10T21:02:41.918116+0000","last_clean":"2024-09-10T21:02:41.918116+0000","last_became_active":"2024-09-10T21:02:41.917621+0000","last_became_peered":"2024-09-10T21:02:41.917621+0000","last_unstale":"2024-09-10T21:02:41.918116+0000","last_undegraded":"2024-09-10T21:02:41.918116+0000","last_fullsized":"2024-09-10T21:02:41.918116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:12.742475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867954+0000","last_change":"2024-09-10T21:02:00.810502+0000","last_active":"2024-09-10T21:02:41.867954+0000","last_peered":"2024-09-10T21:02:41.867954+0000","last_clean":"2024-09-10T21:02:41.867954+0000","last_became_active":"2024-09-10T21:02:00.810209+0000","last_became_peered":"2024-09-10T21:02:00.810209+0000","last_unstale":"2024-09-10T21:02:41.867954+0000","last_undegraded":"2024-09-10T21:02:41.867954+0000","last_fullsized":"2024-09-10T21:02:41.867954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:51.311272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917146+0000","last_change":"2024-09-10T21:02:00.797119+0000","last_active":"2024-09-10T21:02:41.917146+0000","last_peered":"2024-09-10T21:02:41.917146+0000","last_clean":"2024-09-10T21:02:41.917146+0000","last_became_active":"2024-09-10T21:02:00.796788+0000","last_became_peered":"2024-09-10T21:02:00.796788+0000","last_unstale":"2024-09-10T21:02:41.917146+0000","last_undegraded":"2024-09-10T21:02:41.917146+0000","last_fullsized":"2024-09-10T21:02:41.917146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:07.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867912+0000","last_change":"2024-09-10T21:02:00.802284+0000","last_active":"2024-09-10T21:02:41.867912+0000","last_peered":"2024-09-10T21:02:41.867912+0000","last_clean":"2024-09-10T21:02:41.867912+0000","last_became_active":"2024-09-10T21:02:00.801964+0000","last_became_peered":"2024-09-10T21:02:00.801964+0000","last_unstale":"2024-09-10T21:02:41.867912+0000","last_undegraded":"2024-09-10T21:02:41.867912+0000","last_fullsized":"2024-09-10T21:02:41.867912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:45.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977578+0000","last_change":"2024-09-10T21:02:00.809196+0000","last_active":"2024-09-10T21:02:41.977578+0000","last_peered":"2024-09-10T21:02:41.977578+0000","last_clean":"2024-09-10T21:02:41.977578+0000","last_became_active":"2024-09-10T21:02:00.808903+0000","last_became_peered":"2024-09-10T21:02:00.808903+0000","last_unstale":"2024-09-10T21:02:41.977578+0000","last_undegraded":"2024-09-10T21:02:41.977578+0000","last_fullsized":"2024-09-10T21:02:41.977578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:01.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866856+0000","last_change":"2024-09-10T21:02:00.808195+0000","last_active":"2024-09-10T21:02:41.866856+0000","last_peered":"2024-09-10T21:02:41.866856+0000","last_clean":"2024-09-10T21:02:41.866856+0000","last_became_active":"2024-09-10T21:02:00.807670+0000","last_became_peered":"2024-09-10T21:02:00.807670+0000","last_unstale":"2024-09-10T21:02:41.866856+0000","last_undegraded":"2024-09-10T21:02:41.866856+0000","last_fullsized":"2024-09-10T21:02:41.866856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:12.549344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978030+0000","last_change":"2024-09-10T21:02:00.811401+0000","last_active":"2024-09-10T21:02:41.978030+0000","last_peered":"2024-09-10T21:02:41.978030+0000","last_clean":"2024-09-10T21:02:41.978030+0000","last_became_active":"2024-09-10T21:02:00.811055+0000","last_became_peered":"2024-09-10T21:02:00.811055+0000","last_unstale":"2024-09-10T21:02:41.978030+0000","last_undegraded":"2024-09-10T21:02:41.978030+0000","last_fullsized":"2024-09-10T21:02:41.978030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:58.641873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871070+0000","last_change":"2024-09-10T21:02:00.813796+0000","last_active":"2024-09-10T21:02:41.871070+0000","last_peered":"2024-09-10T21:02:41.871070+0000","last_clean":"2024-09-10T21:02:41.871070+0000","last_became_active":"2024-09-10T21:02:00.813468+0000","last_became_peered":"2024-09-10T21:02:00.813468+0000","last_unstale":"2024-09-10T21:02:41.871070+0000","last_undegraded":"2024-09-10T21:02:41.871070+0000","last_fullsized":"2024-09-10T21:02:41.871070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:25.964890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868732+0000","last_change":"2024-09-10T21:02:00.811436+0000","last_active":"2024-09-10T21:02:41.868732+0000","last_peered":"2024-09-10T21:02:41.868732+0000","last_clean":"2024-09-10T21:02:41.868732+0000","last_became_active":"2024-09-10T21:02:00.810956+0000","last_became_peered":"2024-09-10T21:02:00.810956+0000","last_unstale":"2024-09-10T21:02:41.868732+0000","last_undegraded":"2024-09-10T21:02:41.868732+0000","last_fullsized":"2024-09-10T21:02:41.868732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.306832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917070+0000","last_change":"2024-09-10T21:02:00.799370+0000","last_active":"2024-09-10T21:02:41.917070+0000","last_peered":"2024-09-10T21:02:41.917070+0000","last_clean":"2024-09-10T21:02:41.917070+0000","last_became_active":"2024-09-10T21:02:00.799044+0000","last_became_peered":"2024-09-10T21:02:00.799044+0000","last_unstale":"2024-09-10T21:02:41.917070+0000","last_undegraded":"2024-09-10T21:02:41.917070+0000","last_fullsized":"2024-09-10T21:02:41.917070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:40.332088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870566+0000","last_change":"2024-09-10T21:02:00.799830+0000","last_active":"2024-09-10T21:02:41.870566+0000","last_peered":"2024-09-10T21:02:41.870566+0000","last_clean":"2024-09-10T21:02:41.870566+0000","last_became_active":"2024-09-10T21:02:00.799533+0000","last_became_peered":"2024-09-10T21:02:00.799533+0000","last_unstale":"2024-09-10T21:02:41.870566+0000","last_undegraded":"2024-09-10T21:02:41.870566+0000","last_fullsized":"2024-09-10T21:02:41.870566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:23.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870452+0000","last_change":"2024-09-10T21:02:00.812739+0000","last_active":"2024-09-10T21:02:41.870452+0000","last_peered":"2024-09-10T21:02:41.870452+0000","last_clean":"2024-09-10T21:02:41.870452+0000","last_became_active":"2024-09-10T21:02:00.812395+0000","last_became_peered":"2024-09-10T21:02:00.812395+0000","last_unstale":"2024-09-10T21:02:41.870452+0000","last_undegraded":"2024-09-10T21:02:41.870452+0000","last_fullsized":"2024-09-10T21:02:41.870452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:55.727155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870327+0000","last_change":"2024-09-10T21:02:00.806291+0000","last_active":"2024-09-10T21:02:41.870327+0000","last_peered":"2024-09-10T21:02:41.870327+0000","last_clean":"2024-09-10T21:02:41.870327+0000","last_became_active":"2024-09-10T21:02:00.805943+0000","last_became_peered":"2024-09-10T21:02:00.805943+0000","last_unstale":"2024-09-10T21:02:41.870327+0000","last_undegraded":"2024-09-10T21:02:41.870327+0000","last_fullsized":"2024-09-10T21:02:41.870327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:21.354105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867304+0000","last_change":"2024-09-10T21:02:00.808774+0000","last_active":"2024-09-10T21:02:41.867304+0000","last_peered":"2024-09-10T21:02:41.867304+0000","last_clean":"2024-09-10T21:02:41.867304+0000","last_became_active":"2024-09-10T21:02:00.808027+0000","last_became_peered":"2024-09-10T21:02:00.808027+0000","last_unstale":"2024-09-10T21:02:41.867304+0000","last_undegraded":"2024-09-10T21:02:41.867304+0000","last_fullsized":"2024-09-10T21:02:41.867304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870862+0000","last_change":"2024-09-10T21:02:00.808349+0000","last_active":"2024-09-10T21:02:41.870862+0000","last_peered":"2024-09-10T21:02:41.870862+0000","last_clean":"2024-09-10T21:02:41.870862+0000","last_became_active":"2024-09-10T21:02:00.807839+0000","last_became_peered":"2024-09-10T21:02:00.807839+0000","last_unstale":"2024-09-10T21:02:41.870862+0000","last_undegraded":"2024-09-10T21:02:41.870862+0000","last_fullsized":"2024-09-10T21:02:41.870862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:53.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:05.906805+0000","last_change":"2024-09-10T21:02:41.877376+0000","last_active":"2024-09-10T21:03:05.906805+0000","last_peered":"2024-09-10T21:03:05.906805+0000","last_clean":"2024-09-10T21:03:05.906805+0000","last_became_active":"2024-09-10T21:02:41.871397+0000","last_became_peered":"2024-09-10T21:02:41.871397+0000","last_unstale":"2024-09-10T21:03:05.906805+0000","last_undegraded":"2024-09-10T21:03:05.906805+0000","last_fullsized":"2024-09-10T21:03:05.906805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:59.961597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.979721+0000","last_change":"2024-09-10T21:02:41.979196+0000","last_active":"2024-09-10T21:02:41.979721+0000","last_peered":"2024-09-10T21:02:41.979721+0000","last_clean":"2024-09-10T21:02:41.979721+0000","last_became_active":"2024-09-10T21:02:41.978781+0000","last_became_peered":"2024-09-10T21:02:41.978781+0000","last_unstale":"2024-09-10T21:02:41.979721+0000","last_undegraded":"2024-09-10T21:02:41.979721+0000","last_fullsized":"2024-09-10T21:02:41.979721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:15.100055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917027+0000","last_change":"2024-09-10T21:01:58.787619+0000","last_active":"2024-09-10T21:02:41.917027+0000","last_peered":"2024-09-10T21:02:41.917027+0000","last_clean":"2024-09-10T21:02:41.917027+0000","last_became_active":"2024-09-10T21:01:58.787138+0000","last_became_peered":"2024-09-10T21:01:58.787138+0000","last_unstale":"2024-09-10T21:02:41.917027+0000","last_undegraded":"2024-09-10T21:02:41.917027+0000","last_fullsized":"2024-09-10T21:02:41.917027+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:33.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916753+0000","last_change":"2024-09-10T21:01:58.780115+0000","last_active":"2024-09-10T21:02:41.916753+0000","last_peered":"2024-09-10T21:02:41.916753+0000","last_clean":"2024-09-10T21:02:41.916753+0000","last_became_active":"2024-09-10T21:01:58.779757+0000","last_became_peered":"2024-09-10T21:01:58.779757+0000","last_unstale":"2024-09-10T21:02:41.916753+0000","last_undegraded":"2024-09-10T21:02:41.916753+0000","last_fullsized":"2024-09-10T21:02:41.916753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:29.337978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977623+0000","last_change":"2024-09-10T21:02:00.800925+0000","last_active":"2024-09-10T21:02:41.977623+0000","last_peered":"2024-09-10T21:02:41.977623+0000","last_clean":"2024-09-10T21:02:41.977623+0000","last_became_active":"2024-09-10T21:02:00.798567+0000","last_became_peered":"2024-09-10T21:02:00.798567+0000","last_unstale":"2024-09-10T21:02:41.977623+0000","last_undegraded":"2024-09-10T21:02:41.977623+0000","last_fullsized":"2024-09-10T21:02:41.977623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:01.873716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867127+0000","last_change":"2024-09-10T21:01:58.786040+0000","last_active":"2024-09-10T21:02:41.867127+0000","last_peered":"2024-09-10T21:02:41.867127+0000","last_clean":"2024-09-10T21:02:41.867127+0000","last_became_active":"2024-09-10T21:01:58.785703+0000","last_became_peered":"2024-09-10T21:01:58.785703+0000","last_unstale":"2024-09-10T21:02:41.867127+0000","last_undegraded":"2024-09-10T21:02:41.867127+0000","last_fullsized":"2024-09-10T21:02:41.867127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:39.741614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.864899+0000","last_change":"2024-09-10T21:02:00.798368+0000","last_active":"2024-09-10T21:02:41.864899+0000","last_peered":"2024-09-10T21:02:41.864899+0000","last_clean":"2024-09-10T21:02:41.864899+0000","last_became_active":"2024-09-10T21:02:00.798080+0000","last_became_peered":"2024-09-10T21:02:00.798080+0000","last_unstale":"2024-09-10T21:02:41.864899+0000","last_undegraded":"2024-09-10T21:02:41.864899+0000","last_fullsized":"2024-09-10T21:02:41.864899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:31.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:06.907626+0000","last_change":"2024-09-10T21:01:58.788119+0000","last_active":"2024-09-10T21:03:06.907626+0000","last_peered":"2024-09-10T21:03:06.907626+0000","last_clean":"2024-09-10T21:03:06.907626+0000","last_became_active":"2024-09-10T21:01:58.787724+0000","last_became_peered":"2024-09-10T21:01:58.787724+0000","last_unstale":"2024-09-10T21:03:06.907626+0000","last_undegraded":"2024-09-10T21:03:06.907626+0000","last_fullsized":"2024-09-10T21:03:06.907626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:42.872858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868220+0000","last_change":"2024-09-10T21:02:00.804425+0000","last_active":"2024-09-10T21:02:41.868220+0000","last_peered":"2024-09-10T21:02:41.868220+0000","last_clean":"2024-09-10T21:02:41.868220+0000","last_became_active":"2024-09-10T21:02:00.804093+0000","last_became_peered":"2024-09-10T21:02:00.804093+0000","last_unstale":"2024-09-10T21:02:41.868220+0000","last_undegraded":"2024-09-10T21:02:41.868220+0000","last_fullsized":"2024-09-10T21:02:41.868220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:46.183326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870381+0000","last_change":"2024-09-10T21:01:58.791323+0000","last_active":"2024-09-10T21:02:41.870381+0000","last_peered":"2024-09-10T21:02:41.870381+0000","last_clean":"2024-09-10T21:02:41.870381+0000","last_became_active":"2024-09-10T21:01:58.790551+0000","last_became_peered":"2024-09-10T21:01:58.790551+0000","last_unstale":"2024-09-10T21:02:41.870381+0000","last_undegraded":"2024-09-10T21:02:41.870381+0000","last_fullsized":"2024-09-10T21:02:41.870381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:18.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870908+0000","last_change":"2024-09-10T21:02:00.804025+0000","last_active":"2024-09-10T21:02:41.870908+0000","last_peered":"2024-09-10T21:02:41.870908+0000","last_clean":"2024-09-10T21:02:41.870908+0000","last_became_active":"2024-09-10T21:02:00.803663+0000","last_became_peered":"2024-09-10T21:02:00.803663+0000","last_unstale":"2024-09-10T21:02:41.870908+0000","last_undegraded":"2024-09-10T21:02:41.870908+0000","last_fullsized":"2024-09-10T21:02:41.870908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:30.609020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868035+0000","last_change":"2024-09-10T21:01:58.887067+0000","last_active":"2024-09-10T21:02:41.868035+0000","last_peered":"2024-09-10T21:02:41.868035+0000","last_clean":"2024-09-10T21:02:41.868035+0000","last_became_active":"2024-09-10T21:01:58.886611+0000","last_became_peered":"2024-09-10T21:01:58.886611+0000","last_unstale":"2024-09-10T21:02:41.868035+0000","last_undegraded":"2024-09-10T21:02:41.868035+0000","last_fullsized":"2024-09-10T21:02:41.868035+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:06.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.918116+0000","last_change":"2024-09-10T21:02:41.917998+0000","last_active":"2024-09-10T21:02:41.918116+0000","last_peered":"2024-09-10T21:02:41.918116+0000","last_clean":"2024-09-10T21:02:41.918116+0000","last_became_active":"2024-09-10T21:02:41.917665+0000","last_became_peered":"2024-09-10T21:02:41.917665+0000","last_unstale":"2024-09-10T21:02:41.918116+0000","last_undegraded":"2024-09-10T21:02:41.918116+0000","last_fullsized":"2024-09-10T21:02:41.918116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:15.147379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'31","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:06.907600+0000","last_change":"2024-09-10T21:01:58.790966+0000","last_active":"2024-09-10T21:03:06.907600+0000","last_peered":"2024-09-10T21:03:06.907600+0000","last_clean":"2024-09-10T21:03:06.907600+0000","last_became_active":"2024-09-10T21:01:58.790636+0000","last_became_peered":"2024-09-10T21:01:58.790636+0000","last_unstale":"2024-09-10T21:03:06.907600+0000","last_undegraded":"2024-09-10T21:03:06.907600+0000","last_fullsized":"2024-09-10T21:03:06.907600+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:06.209074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61170,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.872335+0000","last_change":"2024-09-10T21:02:00.810610+0000","last_active":"2024-09-10T21:02:41.872335+0000","last_peered":"2024-09-10T21:02:41.872335+0000","last_clean":"2024-09-10T21:02:41.872335+0000","last_became_active":"2024-09-10T21:02:00.810224+0000","last_became_peered":"2024-09-10T21:02:00.810224+0000","last_unstale":"2024-09-10T21:02:41.872335+0000","last_undegraded":"2024-09-10T21:02:41.872335+0000","last_fullsized":"2024-09-10T21:02:41.872335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:32.785312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870285+0000","last_change":"2024-09-10T21:01:58.786476+0000","last_active":"2024-09-10T21:02:41.870285+0000","last_peered":"2024-09-10T21:02:41.870285+0000","last_clean":"2024-09-10T21:02:41.870285+0000","last_became_active":"2024-09-10T21:01:58.786153+0000","last_became_peered":"2024-09-10T21:01:58.786153+0000","last_unstale":"2024-09-10T21:02:41.870285+0000","last_undegraded":"2024-09-10T21:02:41.870285+0000","last_fullsized":"2024-09-10T21:02:41.870285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:00.118591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916861+0000","last_change":"2024-09-10T21:02:00.796138+0000","last_active":"2024-09-10T21:02:41.916861+0000","last_peered":"2024-09-10T21:02:41.916861+0000","last_clean":"2024-09-10T21:02:41.916861+0000","last_became_active":"2024-09-10T21:02:00.795847+0000","last_became_peered":"2024-09-10T21:02:00.795847+0000","last_unstale":"2024-09-10T21:02:41.916861+0000","last_undegraded":"2024-09-10T21:02:41.916861+0000","last_fullsized":"2024-09-10T21:02:41.916861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:29.462134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867576+0000","last_change":"2024-09-10T21:01:58.789314+0000","last_active":"2024-09-10T21:02:41.867576+0000","last_peered":"2024-09-10T21:02:41.867576+0000","last_clean":"2024-09-10T21:02:41.867576+0000","last_became_active":"2024-09-10T21:01:58.788594+0000","last_became_peered":"2024-09-10T21:01:58.788594+0000","last_unstale":"2024-09-10T21:02:41.867576+0000","last_undegraded":"2024-09-10T21:02:41.867576+0000","last_fullsized":"2024-09-10T21:02:41.867576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:44.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870196+0000","last_change":"2024-09-10T21:02:00.799828+0000","last_active":"2024-09-10T21:02:41.870196+0000","last_peered":"2024-09-10T21:02:41.870196+0000","last_clean":"2024-09-10T21:02:41.870196+0000","last_became_active":"2024-09-10T21:02:00.799465+0000","last_became_peered":"2024-09-10T21:02:00.799465+0000","last_unstale":"2024-09-10T21:02:41.870196+0000","last_undegraded":"2024-09-10T21:02:41.870196+0000","last_fullsized":"2024-09-10T21:02:41.870196+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:56.794326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:01.942060+0000","last_change":"2024-09-10T21:01:58.788730+0000","last_active":"2024-09-10T21:03:01.942060+0000","last_peered":"2024-09-10T21:03:01.942060+0000","last_clean":"2024-09-10T21:03:01.942060+0000","last_became_active":"2024-09-10T21:01:58.788317+0000","last_became_peered":"2024-09-10T21:01:58.788317+0000","last_unstale":"2024-09-10T21:03:01.942060+0000","last_undegraded":"2024-09-10T21:03:01.942060+0000","last_fullsized":"2024-09-10T21:03:01.942060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:52.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.869845+0000","last_change":"2024-09-10T21:02:00.807104+0000","last_active":"2024-09-10T21:02:41.869845+0000","last_peered":"2024-09-10T21:02:41.869845+0000","last_clean":"2024-09-10T21:02:41.869845+0000","last_became_active":"2024-09-10T21:02:00.806815+0000","last_became_peered":"2024-09-10T21:02:00.806815+0000","last_unstale":"2024-09-10T21:02:41.869845+0000","last_undegraded":"2024-09-10T21:02:41.869845+0000","last_fullsized":"2024-09-10T21:02:41.869845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:24.747662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.890722+0000","last_change":"2024-09-10T21:02:41.890657+0000","last_active":"2024-09-10T21:02:41.890722+0000","last_peered":"2024-09-10T21:02:41.890722+0000","last_clean":"2024-09-10T21:02:41.890722+0000","last_became_active":"2024-09-10T21:02:41.873778+0000","last_became_peered":"2024-09-10T21:02:41.873778+0000","last_unstale":"2024-09-10T21:02:41.890722+0000","last_undegraded":"2024-09-10T21:02:41.890722+0000","last_fullsized":"2024-09-10T21:02:41.890722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:58.678679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867451+0000","last_change":"2024-09-10T21:02:00.806796+0000","last_active":"2024-09-10T21:02:41.867451+0000","last_peered":"2024-09-10T21:02:41.867451+0000","last_clean":"2024-09-10T21:02:41.867451+0000","last_became_active":"2024-09-10T21:02:00.805802+0000","last_became_peered":"2024-09-10T21:02:00.805802+0000","last_unstale":"2024-09-10T21:02:41.867451+0000","last_undegraded":"2024-09-10T21:02:41.867451+0000","last_fullsized":"2024-09-10T21:02:41.867451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.962870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871648+0000","last_change":"2024-09-10T21:01:58.781088+0000","last_active":"2024-09-10T21:02:41.871648+0000","last_peered":"2024-09-10T21:02:41.871648+0000","last_clean":"2024-09-10T21:02:41.871648+0000","last_became_active":"2024-09-10T21:01:58.780731+0000","last_became_peered":"2024-09-10T21:01:58.780731+0000","last_unstale":"2024-09-10T21:02:41.871648+0000","last_undegraded":"2024-09-10T21:02:41.871648+0000","last_fullsized":"2024-09-10T21:02:41.871648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:50.636077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867188+0000","last_change":"2024-09-10T21:02:00.806551+0000","last_active":"2024-09-10T21:02:41.867188+0000","last_peered":"2024-09-10T21:02:41.867188+0000","last_clean":"2024-09-10T21:02:41.867188+0000","last_became_active":"2024-09-10T21:02:00.806134+0000","last_became_peered":"2024-09-10T21:02:00.806134+0000","last_unstale":"2024-09-10T21:02:41.867188+0000","last_undegraded":"2024-09-10T21:02:41.867188+0000","last_fullsized":"2024-09-10T21:02:41.867188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.574372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916954+0000","last_change":"2024-09-10T21:01:58.780294+0000","last_active":"2024-09-10T21:02:41.916954+0000","last_peered":"2024-09-10T21:02:41.916954+0000","last_clean":"2024-09-10T21:02:41.916954+0000","last_became_active":"2024-09-10T21:01:58.779767+0000","last_became_peered":"2024-09-10T21:01:58.779767+0000","last_unstale":"2024-09-10T21:02:41.916954+0000","last_undegraded":"2024-09-10T21:02:41.916954+0000","last_fullsized":"2024-09-10T21:02:41.916954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:21.734159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870426+0000","last_change":"2024-09-10T21:02:00.809393+0000","last_active":"2024-09-10T21:02:41.870426+0000","last_peered":"2024-09-10T21:02:41.870426+0000","last_clean":"2024-09-10T21:02:41.870426+0000","last_became_active":"2024-09-10T21:02:00.809012+0000","last_became_peered":"2024-09-10T21:02:00.809012+0000","last_unstale":"2024-09-10T21:02:41.870426+0000","last_undegraded":"2024-09-10T21:02:41.870426+0000","last_fullsized":"2024-09-10T21:02:41.870426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:03.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868189+0000","last_change":"2024-09-10T21:01:58.786507+0000","last_active":"2024-09-10T21:02:41.868189+0000","last_peered":"2024-09-10T21:02:41.868189+0000","last_clean":"2024-09-10T21:02:41.868189+0000","last_became_active":"2024-09-10T21:01:58.786115+0000","last_became_peered":"2024-09-10T21:01:58.786115+0000","last_unstale":"2024-09-10T21:02:41.868189+0000","last_undegraded":"2024-09-10T21:02:41.868189+0000","last_fullsized":"2024-09-10T21:02:41.868189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.985353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870255+0000","last_change":"2024-09-10T21:02:00.805050+0000","last_active":"2024-09-10T21:02:41.870255+0000","last_peered":"2024-09-10T21:02:41.870255+0000","last_clean":"2024-09-10T21:02:41.870255+0000","last_became_active":"2024-09-10T21:02:00.804706+0000","last_became_peered":"2024-09-10T21:02:00.804706+0000","last_unstale":"2024-09-10T21:02:41.870255+0000","last_undegraded":"2024-09-10T21:02:41.870255+0000","last_fullsized":"2024-09-10T21:02:41.870255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:15.463986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:02.945473+0000","last_change":"2024-09-10T21:01:59.114525+0000","last_active":"2024-09-10T21:03:02.945473+0000","last_peered":"2024-09-10T21:03:02.945473+0000","last_clean":"2024-09-10T21:03:02.945473+0000","last_became_active":"2024-09-10T21:01:59.114076+0000","last_became_peered":"2024-09-10T21:01:59.114076+0000","last_unstale":"2024-09-10T21:03:02.945473+0000","last_undegraded":"2024-09-10T21:03:02.945473+0000","last_fullsized":"2024-09-10T21:03:02.945473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:53.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870338+0000","last_change":"2024-09-10T21:02:00.799803+0000","last_active":"2024-09-10T21:02:41.870338+0000","last_peered":"2024-09-10T21:02:41.870338+0000","last_clean":"2024-09-10T21:02:41.870338+0000","last_became_active":"2024-09-10T21:02:00.799459+0000","last_became_peered":"2024-09-10T21:02:00.799459+0000","last_unstale":"2024-09-10T21:02:41.870338+0000","last_undegraded":"2024-09-10T21:02:41.870338+0000","last_fullsized":"2024-09-10T21:02:41.870338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:57.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870056+0000","last_change":"2024-09-10T21:00:48.381903+0000","last_active":"2024-09-10T21:02:41.870056+0000","last_peered":"2024-09-10T21:02:41.870056+0000","last_clean":"2024-09-10T21:02:41.870056+0000","last_became_active":"2024-09-10T21:00:48.376874+0000","last_became_peered":"2024-09-10T21:00:48.376874+0000","last_unstale":"2024-09-10T21:02:41.870056+0000","last_undegraded":"2024-09-10T21:02:41.870056+0000","last_fullsized":"2024-09-10T21:02:41.870056+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_clean_scrub_stamp":"2024-09-10T20:57:24.516525+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:35.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866334+0000","last_change":"2024-09-10T21:01:58.788005+0000","last_active":"2024-09-10T21:02:41.866334+0000","last_peered":"2024-09-10T21:02:41.866334+0000","last_clean":"2024-09-10T21:02:41.866334+0000","last_became_active":"2024-09-10T21:01:58.786471+0000","last_became_peered":"2024-09-10T21:01:58.786471+0000","last_unstale":"2024-09-10T21:02:41.866334+0000","last_undegraded":"2024-09-10T21:02:41.866334+0000","last_fullsized":"2024-09-10T21:02:41.866334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:15.622814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977860+0000","last_change":"2024-09-10T21:02:00.802573+0000","last_active":"2024-09-10T21:02:41.977860+0000","last_peered":"2024-09-10T21:02:41.977860+0000","last_clean":"2024-09-10T21:02:41.977860+0000","last_became_active":"2024-09-10T21:02:00.802263+0000","last_became_peered":"2024-09-10T21:02:00.802263+0000","last_unstale":"2024-09-10T21:02:41.977860+0000","last_undegraded":"2024-09-10T21:02:41.977860+0000","last_fullsized":"2024-09-10T21:02:41.977860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:54.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870004+0000","last_change":"2024-09-10T21:01:58.785700+0000","last_active":"2024-09-10T21:02:41.870004+0000","last_peered":"2024-09-10T21:02:41.870004+0000","last_clean":"2024-09-10T21:02:41.870004+0000","last_became_active":"2024-09-10T21:01:58.785346+0000","last_became_peered":"2024-09-10T21:01:58.785346+0000","last_unstale":"2024-09-10T21:02:41.870004+0000","last_undegraded":"2024-09-10T21:02:41.870004+0000","last_fullsized":"2024-09-10T21:02:41.870004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:39.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916719+0000","last_change":"2024-09-10T21:02:00.808045+0000","last_active":"2024-09-10T21:02:41.916719+0000","last_peered":"2024-09-10T21:02:41.916719+0000","last_clean":"2024-09-10T21:02:41.916719+0000","last_became_active":"2024-09-10T21:02:00.807757+0000","last_became_peered":"2024-09-10T21:02:00.807757+0000","last_unstale":"2024-09-10T21:02:41.916719+0000","last_undegraded":"2024-09-10T21:02:41.916719+0000","last_fullsized":"2024-09-10T21:02:41.916719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:16.198290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":81,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:02.942753+0000","last_change":"2024-09-10T21:01:58.884939+0000","last_active":"2024-09-10T21:03:02.942753+0000","last_peered":"2024-09-10T21:03:02.942753+0000","last_clean":"2024-09-10T21:03:02.942753+0000","last_became_active":"2024-09-10T21:01:58.884517+0000","last_became_peered":"2024-09-10T21:01:58.884517+0000","last_unstale":"2024-09-10T21:03:02.942753+0000","last_undegraded":"2024-09-10T21:03:02.942753+0000","last_fullsized":"2024-09-10T21:03:02.942753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.579662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866940+0000","last_change":"2024-09-10T21:02:00.813021+0000","last_active":"2024-09-10T21:02:41.866940+0000","last_peered":"2024-09-10T21:02:41.866940+0000","last_clean":"2024-09-10T21:02:41.866940+0000","last_became_active":"2024-09-10T21:02:00.812622+0000","last_became_peered":"2024-09-10T21:02:00.812622+0000","last_unstale":"2024-09-10T21:02:41.866940+0000","last_undegraded":"2024-09-10T21:02:41.866940+0000","last_fullsized":"2024-09-10T21:02:41.866940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:48.348100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:03.943896+0000","last_change":"2024-09-10T21:01:58.792187+0000","last_active":"2024-09-10T21:03:03.943896+0000","last_peered":"2024-09-10T21:03:03.943896+0000","last_clean":"2024-09-10T21:03:03.943896+0000","last_became_active":"2024-09-10T21:01:58.791806+0000","last_became_peered":"2024-09-10T21:01:58.791806+0000","last_unstale":"2024-09-10T21:03:03.943896+0000","last_undegraded":"2024-09-10T21:03:03.943896+0000","last_fullsized":"2024-09-10T21:03:03.943896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:48.493396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917092+0000","last_change":"2024-09-10T21:02:00.804073+0000","last_active":"2024-09-10T21:02:41.917092+0000","last_peered":"2024-09-10T21:02:41.917092+0000","last_clean":"2024-09-10T21:02:41.917092+0000","last_became_active":"2024-09-10T21:02:00.803760+0000","last_became_peered":"2024-09-10T21:02:00.803760+0000","last_unstale":"2024-09-10T21:02:41.917092+0000","last_undegraded":"2024-09-10T21:02:41.917092+0000","last_fullsized":"2024-09-10T21:02:41.917092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:01.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865323+0000","last_change":"2024-09-10T21:01:58.885088+0000","last_active":"2024-09-10T21:02:41.865323+0000","last_peered":"2024-09-10T21:02:41.865323+0000","last_clean":"2024-09-10T21:02:41.865323+0000","last_became_active":"2024-09-10T21:01:58.884729+0000","last_became_peered":"2024-09-10T21:01:58.884729+0000","last_unstale":"2024-09-10T21:02:41.865323+0000","last_undegraded":"2024-09-10T21:02:41.865323+0000","last_fullsized":"2024-09-10T21:02:41.865323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:37.451629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917111+0000","last_change":"2024-09-10T21:02:00.810329+0000","last_active":"2024-09-10T21:02:41.917111+0000","last_peered":"2024-09-10T21:02:41.917111+0000","last_clean":"2024-09-10T21:02:41.917111+0000","last_became_active":"2024-09-10T21:02:00.809985+0000","last_became_peered":"2024-09-10T21:02:00.809985+0000","last_unstale":"2024-09-10T21:02:41.917111+0000","last_undegraded":"2024-09-10T21:02:41.917111+0000","last_fullsized":"2024-09-10T21:02:41.917111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:39.631969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.880605+0000","last_change":"2024-09-10T21:02:41.880117+0000","last_active":"2024-09-10T21:02:41.880605+0000","last_peered":"2024-09-10T21:02:41.880605+0000","last_clean":"2024-09-10T21:02:41.880605+0000","last_became_active":"2024-09-10T21:02:41.872494+0000","last_became_peered":"2024-09-10T21:02:41.872494+0000","last_unstale":"2024-09-10T21:02:41.880605+0000","last_undegraded":"2024-09-10T21:02:41.880605+0000","last_fullsized":"2024-09-10T21:02:41.880605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:06.965479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917090+0000","last_change":"2024-09-10T21:02:00.804961+0000","last_active":"2024-09-10T21:02:41.917090+0000","last_peered":"2024-09-10T21:02:41.917090+0000","last_clean":"2024-09-10T21:02:41.917090+0000","last_became_active":"2024-09-10T21:02:00.804147+0000","last_became_peered":"2024-09-10T21:02:00.804147+0000","last_unstale":"2024-09-10T21:02:41.917090+0000","last_undegraded":"2024-09-10T21:02:41.917090+0000","last_fullsized":"2024-09-10T21:02:41.917090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:15.801815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'14","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865328+0000","last_change":"2024-09-10T21:01:58.781292+0000","last_active":"2024-09-10T21:02:41.865328+0000","last_peered":"2024-09-10T21:02:41.865328+0000","last_clean":"2024-09-10T21:02:41.865328+0000","last_became_active":"2024-09-10T21:01:58.780995+0000","last_became_peered":"2024-09-10T21:01:58.780995+0000","last_unstale":"2024-09-10T21:02:41.865328+0000","last_undegraded":"2024-09-10T21:02:41.865328+0000","last_fullsized":"2024-09-10T21:02:41.865328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:56.914723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978081+0000","last_change":"2024-09-10T21:02:00.805878+0000","last_active":"2024-09-10T21:02:41.978081+0000","last_peered":"2024-09-10T21:02:41.978081+0000","last_clean":"2024-09-10T21:02:41.978081+0000","last_became_active":"2024-09-10T21:02:00.804565+0000","last_became_peered":"2024-09-10T21:02:00.804565+0000","last_unstale":"2024-09-10T21:02:41.978081+0000","last_undegraded":"2024-09-10T21:02:41.978081+0000","last_fullsized":"2024-09-10T21:02:41.978081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:24.359643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866263+0000","last_change":"2024-09-10T21:01:59.114862+0000","last_active":"2024-09-10T21:02:41.866263+0000","last_peered":"2024-09-10T21:02:41.866263+0000","last_clean":"2024-09-10T21:02:41.866263+0000","last_became_active":"2024-09-10T21:01:59.114411+0000","last_became_peered":"2024-09-10T21:01:59.114411+0000","last_unstale":"2024-09-10T21:02:41.866263+0000","last_undegraded":"2024-09-10T21:02:41.866263+0000","last_fullsized":"2024-09-10T21:02:41.866263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.645829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870778+0000","last_change":"2024-09-10T21:02:00.799230+0000","last_active":"2024-09-10T21:02:41.870778+0000","last_peered":"2024-09-10T21:02:41.870778+0000","last_clean":"2024-09-10T21:02:41.870778+0000","last_became_active":"2024-09-10T21:02:00.798992+0000","last_became_peered":"2024-09-10T21:02:00.798992+0000","last_unstale":"2024-09-10T21:02:41.870778+0000","last_undegraded":"2024-09-10T21:02:41.870778+0000","last_fullsized":"2024-09-10T21:02:41.870778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:40.322066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867967+0000","last_change":"2024-09-10T21:01:58.887042+0000","last_active":"2024-09-10T21:02:41.867967+0000","last_peered":"2024-09-10T21:02:41.867967+0000","last_clean":"2024-09-10T21:02:41.867967+0000","last_became_active":"2024-09-10T21:01:58.886710+0000","last_became_peered":"2024-09-10T21:01:58.886710+0000","last_unstale":"2024-09-10T21:02:41.867967+0000","last_undegraded":"2024-09-10T21:02:41.867967+0000","last_fullsized":"2024-09-10T21:02:41.867967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:42.672899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870544+0000","last_change":"2024-09-10T21:02:00.809763+0000","last_active":"2024-09-10T21:02:41.870544+0000","last_peered":"2024-09-10T21:02:41.870544+0000","last_clean":"2024-09-10T21:02:41.870544+0000","last_became_active":"2024-09-10T21:02:00.808955+0000","last_became_peered":"2024-09-10T21:02:00.808955+0000","last_unstale":"2024-09-10T21:02:41.870544+0000","last_undegraded":"2024-09-10T21:02:41.870544+0000","last_fullsized":"2024-09-10T21:02:41.870544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:22.622852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977719+0000","last_change":"2024-09-10T21:01:58.786312+0000","last_active":"2024-09-10T21:02:41.977719+0000","last_peered":"2024-09-10T21:02:41.977719+0000","last_clean":"2024-09-10T21:02:41.977719+0000","last_became_active":"2024-09-10T21:01:58.785894+0000","last_became_peered":"2024-09-10T21:01:58.785894+0000","last_unstale":"2024-09-10T21:02:41.977719+0000","last_undegraded":"2024-09-10T21:02:41.977719+0000","last_fullsized":"2024-09-10T21:02:41.977719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:11.682904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867398+0000","last_change":"2024-09-10T21:02:00.808374+0000","last_active":"2024-09-10T21:02:41.867398+0000","last_peered":"2024-09-10T21:02:41.867398+0000","last_clean":"2024-09-10T21:02:41.867398+0000","last_became_active":"2024-09-10T21:02:00.807565+0000","last_became_peered":"2024-09-10T21:02:00.807565+0000","last_unstale":"2024-09-10T21:02:41.867398+0000","last_undegraded":"2024-09-10T21:02:41.867398+0000","last_fullsized":"2024-09-10T21:02:41.867398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:24.227480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870458+0000","last_change":"2024-09-10T21:01:58.782375+0000","last_active":"2024-09-10T21:02:41.870458+0000","last_peered":"2024-09-10T21:02:41.870458+0000","last_clean":"2024-09-10T21:02:41.870458+0000","last_became_active":"2024-09-10T21:01:58.781964+0000","last_became_peered":"2024-09-10T21:01:58.781964+0000","last_unstale":"2024-09-10T21:02:41.870458+0000","last_undegraded":"2024-09-10T21:02:41.870458+0000","last_fullsized":"2024-09-10T21:02:41.870458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.460126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977861+0000","last_change":"2024-09-10T21:02:00.806612+0000","last_active":"2024-09-10T21:02:41.977861+0000","last_peered":"2024-09-10T21:02:41.977861+0000","last_clean":"2024-09-10T21:02:41.977861+0000","last_became_active":"2024-09-10T21:02:00.803470+0000","last_became_peered":"2024-09-10T21:02:00.803470+0000","last_unstale":"2024-09-10T21:02:41.977861+0000","last_undegraded":"2024-09-10T21:02:41.977861+0000","last_fullsized":"2024-09-10T21:02:41.977861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:08.350931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871476+0000","last_change":"2024-09-10T21:01:58.788987+0000","last_active":"2024-09-10T21:02:41.871476+0000","last_peered":"2024-09-10T21:02:41.871476+0000","last_clean":"2024-09-10T21:02:41.871476+0000","last_became_active":"2024-09-10T21:01:58.788605+0000","last_became_peered":"2024-09-10T21:01:58.788605+0000","last_unstale":"2024-09-10T21:02:41.871476+0000","last_undegraded":"2024-09-10T21:02:41.871476+0000","last_fullsized":"2024-09-10T21:02:41.871476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:45.798350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870058+0000","last_change":"2024-09-10T21:02:00.804200+0000","last_active":"2024-09-10T21:02:41.870058+0000","last_peered":"2024-09-10T21:02:41.870058+0000","last_clean":"2024-09-10T21:02:41.870058+0000","last_became_active":"2024-09-10T21:02:00.803875+0000","last_became_peered":"2024-09-10T21:02:00.803875+0000","last_unstale":"2024-09-10T21:02:41.870058+0000","last_undegraded":"2024-09-10T21:02:41.870058+0000","last_fullsized":"2024-09-10T21:02:41.870058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:15.930163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.874681+0000","last_change":"2024-09-10T21:02:41.874580+0000","last_active":"2024-09-10T21:02:41.874681+0000","last_peered":"2024-09-10T21:02:41.874681+0000","last_clean":"2024-09-10T21:02:41.874681+0000","last_became_active":"2024-09-10T21:02:41.873545+0000","last_became_peered":"2024-09-10T21:02:41.873545+0000","last_unstale":"2024-09-10T21:02:41.874681+0000","last_undegraded":"2024-09-10T21:02:41.874681+0000","last_fullsized":"2024-09-10T21:02:41.874681+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:05.278855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870983+0000","last_change":"2024-09-10T21:01:58.784834+0000","last_active":"2024-09-10T21:02:41.870983+0000","last_peered":"2024-09-10T21:02:41.870983+0000","last_clean":"2024-09-10T21:02:41.870983+0000","last_became_active":"2024-09-10T21:01:58.784464+0000","last_became_peered":"2024-09-10T21:01:58.784464+0000","last_unstale":"2024-09-10T21:02:41.870983+0000","last_undegraded":"2024-09-10T21:02:41.870983+0000","last_fullsized":"2024-09-10T21:02:41.870983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:09.864824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867905+0000","last_change":"2024-09-10T21:02:00.805677+0000","last_active":"2024-09-10T21:02:41.867905+0000","last_peered":"2024-09-10T21:02:41.867905+0000","last_clean":"2024-09-10T21:02:41.867905+0000","last_became_active":"2024-09-10T21:02:00.805341+0000","last_became_peered":"2024-09-10T21:02:00.805341+0000","last_unstale":"2024-09-10T21:02:41.867905+0000","last_undegraded":"2024-09-10T21:02:41.867905+0000","last_fullsized":"2024-09-10T21:02:41.867905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.236954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.889813+0000","last_change":"2024-09-10T21:02:41.889336+0000","last_active":"2024-09-10T21:02:41.889813+0000","last_peered":"2024-09-10T21:02:41.889813+0000","last_clean":"2024-09-10T21:02:41.889813+0000","last_became_active":"2024-09-10T21:02:41.874164+0000","last_became_peered":"2024-09-10T21:02:41.874164+0000","last_unstale":"2024-09-10T21:02:41.889813+0000","last_undegraded":"2024-09-10T21:02:41.889813+0000","last_fullsized":"2024-09-10T21:02:41.889813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.799546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870893+0000","last_change":"2024-09-10T21:02:00.801514+0000","last_active":"2024-09-10T21:02:41.870893+0000","last_peered":"2024-09-10T21:02:41.870893+0000","last_clean":"2024-09-10T21:02:41.870893+0000","last_became_active":"2024-09-10T21:02:00.801221+0000","last_became_peered":"2024-09-10T21:02:00.801221+0000","last_unstale":"2024-09-10T21:02:41.870893+0000","last_undegraded":"2024-09-10T21:02:41.870893+0000","last_fullsized":"2024-09-10T21:02:41.870893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:35.564890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978209+0000","last_change":"2024-09-10T21:01:58.884876+0000","last_active":"2024-09-10T21:02:41.978209+0000","last_peered":"2024-09-10T21:02:41.978209+0000","last_clean":"2024-09-10T21:02:41.978209+0000","last_became_active":"2024-09-10T21:01:58.884530+0000","last_became_peered":"2024-09-10T21:01:58.884530+0000","last_unstale":"2024-09-10T21:02:41.978209+0000","last_undegraded":"2024-09-10T21:02:41.978209+0000","last_fullsized":"2024-09-10T21:02:41.978209+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.851017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978109+0000","last_change":"2024-09-10T21:02:00.811148+0000","last_active":"2024-09-10T21:02:41.978109+0000","last_peered":"2024-09-10T21:02:41.978109+0000","last_clean":"2024-09-10T21:02:41.978109+0000","last_became_active":"2024-09-10T21:02:00.809265+0000","last_became_peered":"2024-09-10T21:02:00.809265+0000","last_unstale":"2024-09-10T21:02:41.978109+0000","last_undegraded":"2024-09-10T21:02:41.978109+0000","last_fullsized":"2024-09-10T21:02:41.978109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:25.269876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:05.946051+0000","last_change":"2024-09-10T21:01:58.786917+0000","last_active":"2024-09-10T21:03:05.946051+0000","last_peered":"2024-09-10T21:03:05.946051+0000","last_clean":"2024-09-10T21:03:05.946051+0000","last_became_active":"2024-09-10T21:01:58.786595+0000","last_became_peered":"2024-09-10T21:01:58.786595+0000","last_unstale":"2024-09-10T21:03:05.946051+0000","last_undegraded":"2024-09-10T21:03:05.946051+0000","last_fullsized":"2024-09-10T21:03:05.946051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:30.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69156,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":572,"num_read_kb":364,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1237,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743450,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":407530,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939680,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":995328,"data_stored":1059617,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":864256,"data_stored":406957,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":407223,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495661,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":1000635,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":847872,"data_stored":407065,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":999531,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":407939,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215171,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":864256,"data_stored":406499,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":468634,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":469260,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":408404,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:03:09.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T21:03:09.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:09 smithi033 ceph-mon[30860]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:09.348 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.392 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:09 smithi174 ceph-mon[35635]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:09 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:09 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:09 smithi174 ceph-mon[35635]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:03:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:09 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:03:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:09 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:03:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:09 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:09 smithi033 ceph-mon[30860]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:03:09.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:03:09.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:03:09.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:09 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:09 smithi104 ceph-mon[35325]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:09 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:09 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:09 smithi104 ceph-mon[35325]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:03:09.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:09 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:03:09.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:09 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:03:09.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:09 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:03:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 -- 172.21.15.33:0/3322155931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 msgr2=0x7f3dac107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 --2- 172.21.15.33:0/3322155931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac107ae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3da0009900 tx=0x7f3da002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 -- 172.21.15.33:0/3322155931 shutdown_connections 2024-09-10T21:03:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 --2- 172.21.15.33:0/3322155931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3dac108020 0x7f3dac10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:09.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 --2- 172.21.15.33:0/3322155931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac107ae0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 --2- 172.21.15.33:0/3322155931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dac101c90 0x7f3dac102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 -- 172.21.15.33:0/3322155931 >> 172.21.15.33:0/3322155931 conn(0x7f3dac0fd700 msgr2=0x7f3dac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 -- 172.21.15.33:0/3322155931 shutdown_connections 2024-09-10T21:03:09.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.877+0000 7f3db21b0640 1 -- 172.21.15.33:0/3322155931 wait complete. 2024-09-10T21:03:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 Processor -- start 2024-09-10T21:03:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 -- start start 2024-09-10T21:03:09.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dac101c90 0x7f3dac19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3dac108020 0x7f3dac19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac111ce0 con 0x7f3dac1076a0 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac111b60 con 0x7f3dac101c90 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3db21b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3dac111e60 con 0x7f3dac108020 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3daaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3daaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56658/0 (socket says 172.21.15.33:56658) 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3daaffd640 1 -- 172.21.15.33:0/625595805 learned_addr learned my addr 172.21.15.33:0/625595805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3dab7fe640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dac101c90 0x7f3dac19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.878+0000 7f3dabfff640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3dac108020 0x7f3dac19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3daaffd640 1 -- 172.21.15.33:0/625595805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3dac108020 msgr2=0x7f3dac19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3daaffd640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3dac108020 0x7f3dac19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:09.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3daaffd640 1 -- 172.21.15.33:0/625595805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dac101c90 msgr2=0x7f3dac19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3daaffd640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dac101c90 0x7f3dac19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3daaffd640 1 -- 172.21.15.33:0/625595805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dac1a5450 con 0x7f3dac1076a0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3daaffd640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac19a5b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3da0006010 tx=0x7f3da00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3da8ff9640 1 -- 172.21.15.33:0/625595805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da003f030 con 0x7f3dac1076a0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3da8ff9640 1 -- 172.21.15.33:0/625595805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da0003800 con 0x7f3dac1076a0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3da8ff9640 1 -- 172.21.15.33:0/625595805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0035070 con 0x7f3dac1076a0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dac1a56e0 con 0x7f3dac1076a0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.879+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dac1a5b70 con 0x7f3dac1076a0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.880+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dac102070 con 0x7f3dac1076a0 2024-09-10T21:03:09.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.882+0000 7f3da8ff9640 1 -- 172.21.15.33:0/625595805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f3da0031b10 con 0x7f3dac1076a0 2024-09-10T21:03:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.882+0000 7f3da8ff9640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3d940b53f0 0x7f3d940b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.883+0000 7f3dab7fe640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3d940b53f0 0x7f3d940b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.883+0000 7f3da8ff9640 1 -- 172.21.15.33:0/625595805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3da01068d0 con 0x7f3dac1076a0 2024-09-10T21:03:09.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.883+0000 7f3dab7fe640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3d940b53f0 0x7f3d940b78b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c005c40 tx=0x7f3d9c00a590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:09.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:09.885+0000 7f3da8ff9640 1 -- 172.21.15.33:0/625595805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3da00cd630 con 0x7f3dac1076a0 2024-09-10T21:03:10.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.069+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3dac02e660 con 0x7f3d940b53f0 2024-09-10T21:03:10.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.073+0000 7f3da8ff9640 1 -- 172.21.15.33:0/625595805 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259011 (secure 0 0 0) 0x7f3dac02e660 con 0x7f3d940b53f0 2024-09-10T21:03:10.079 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:10.080 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3d940b53f0 msgr2=0x7f3d940b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3d940b53f0 0x7f3d940b78b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c005c40 tx=0x7f3d9c00a590 comp rx=0 tx=0).stop 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 msgr2=0x7f3dac19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac19a5b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3da0006010 tx=0x7f3da00042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 shutdown_connections 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3d940b53f0 0x7f3d940b78b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3dac108020 0x7f3dac19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3dac1076a0 0x7f3dac19a5b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 --2- 172.21.15.33:0/625595805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3dac101c90 0x7f3dac19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 >> 172.21.15.33:0/625595805 conn(0x7f3dac0fd700 msgr2=0x7f3dac0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 shutdown_connections 2024-09-10T21:03:10.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:10.076+0000 7f3db21b0640 1 -- 172.21.15.33:0/625595805 wait complete. 2024-09-10T21:03:10.780 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-10T21:03:08.099787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659758,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":648,"num_read_kb":428,"num_write":370,"num_write_kb":2289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":2953077,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355580,"kb_used_data":10416,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586244,"statfs":{"total":1151940427776,"available":1151576313856,"internally_reserved":0,"allocated":10665984,"data_stored":6849294,"data_compressed":176985,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":70,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002754"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867485+0000","last_change":"2024-09-10T21:02:00.805421+0000","last_active":"2024-09-10T21:02:41.867485+0000","last_peered":"2024-09-10T21:02:41.867485+0000","last_clean":"2024-09-10T21:02:41.867485+0000","last_became_active":"2024-09-10T21:02:00.804274+0000","last_became_peered":"2024-09-10T21:02:00.804274+0000","last_unstale":"2024-09-10T21:02:41.867485+0000","last_undegraded":"2024-09-10T21:02:41.867485+0000","last_fullsized":"2024-09-10T21:02:41.867485+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:12.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870587+0000","last_change":"2024-09-10T21:02:00.806104+0000","last_active":"2024-09-10T21:02:41.870587+0000","last_peered":"2024-09-10T21:02:41.870587+0000","last_clean":"2024-09-10T21:02:41.870587+0000","last_became_active":"2024-09-10T21:02:00.805510+0000","last_became_peered":"2024-09-10T21:02:00.805510+0000","last_unstale":"2024-09-10T21:02:41.870587+0000","last_undegraded":"2024-09-10T21:02:41.870587+0000","last_fullsized":"2024-09-10T21:02:41.870587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:37.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870594+0000","last_change":"2024-09-10T21:02:00.807691+0000","last_active":"2024-09-10T21:02:41.870594+0000","last_peered":"2024-09-10T21:02:41.870594+0000","last_clean":"2024-09-10T21:02:41.870594+0000","last_became_active":"2024-09-10T21:02:00.807364+0000","last_became_peered":"2024-09-10T21:02:00.807364+0000","last_unstale":"2024-09-10T21:02:41.870594+0000","last_undegraded":"2024-09-10T21:02:41.870594+0000","last_fullsized":"2024-09-10T21:02:41.870594+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:27.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866175+0000","last_change":"2024-09-10T21:02:00.809542+0000","last_active":"2024-09-10T21:02:41.866175+0000","last_peered":"2024-09-10T21:02:41.866175+0000","last_clean":"2024-09-10T21:02:41.866175+0000","last_became_active":"2024-09-10T21:02:00.809034+0000","last_became_peered":"2024-09-10T21:02:00.809034+0000","last_unstale":"2024-09-10T21:02:41.866175+0000","last_undegraded":"2024-09-10T21:02:41.866175+0000","last_fullsized":"2024-09-10T21:02:41.866175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:29.923532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871352+0000","last_change":"2024-09-10T21:02:00.809355+0000","last_active":"2024-09-10T21:02:41.871352+0000","last_peered":"2024-09-10T21:02:41.871352+0000","last_clean":"2024-09-10T21:02:41.871352+0000","last_became_active":"2024-09-10T21:02:00.808863+0000","last_became_peered":"2024-09-10T21:02:00.808863+0000","last_unstale":"2024-09-10T21:02:41.871352+0000","last_undegraded":"2024-09-10T21:02:41.871352+0000","last_fullsized":"2024-09-10T21:02:41.871352+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:29.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916934+0000","last_change":"2024-09-10T21:02:00.803756+0000","last_active":"2024-09-10T21:02:41.916934+0000","last_peered":"2024-09-10T21:02:41.916934+0000","last_clean":"2024-09-10T21:02:41.916934+0000","last_became_active":"2024-09-10T21:02:00.803406+0000","last_became_peered":"2024-09-10T21:02:00.803406+0000","last_unstale":"2024-09-10T21:02:41.916934+0000","last_undegraded":"2024-09-10T21:02:41.916934+0000","last_fullsized":"2024-09-10T21:02:41.916934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:53.437629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.869942+0000","last_change":"2024-09-10T21:02:00.808845+0000","last_active":"2024-09-10T21:02:41.869942+0000","last_peered":"2024-09-10T21:02:41.869942+0000","last_clean":"2024-09-10T21:02:41.869942+0000","last_became_active":"2024-09-10T21:02:00.807759+0000","last_became_peered":"2024-09-10T21:02:00.807759+0000","last_unstale":"2024-09-10T21:02:41.869942+0000","last_undegraded":"2024-09-10T21:02:41.869942+0000","last_fullsized":"2024-09-10T21:02:41.869942+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.809389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977534+0000","last_change":"2024-09-10T21:02:00.811199+0000","last_active":"2024-09-10T21:02:41.977534+0000","last_peered":"2024-09-10T21:02:41.977534+0000","last_clean":"2024-09-10T21:02:41.977534+0000","last_became_active":"2024-09-10T21:02:00.810692+0000","last_became_peered":"2024-09-10T21:02:00.810692+0000","last_unstale":"2024-09-10T21:02:41.977534+0000","last_undegraded":"2024-09-10T21:02:41.977534+0000","last_fullsized":"2024-09-10T21:02:41.977534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:59.557973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868394+0000","last_change":"2024-09-10T21:02:00.812686+0000","last_active":"2024-09-10T21:02:41.868394+0000","last_peered":"2024-09-10T21:02:41.868394+0000","last_clean":"2024-09-10T21:02:41.868394+0000","last_became_active":"2024-09-10T21:02:00.812363+0000","last_became_peered":"2024-09-10T21:02:00.812363+0000","last_unstale":"2024-09-10T21:02:41.868394+0000","last_undegraded":"2024-09-10T21:02:41.868394+0000","last_fullsized":"2024-09-10T21:02:41.868394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:37.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917309+0000","last_change":"2024-09-10T21:02:00.812058+0000","last_active":"2024-09-10T21:02:41.917309+0000","last_peered":"2024-09-10T21:02:41.917309+0000","last_clean":"2024-09-10T21:02:41.917309+0000","last_became_active":"2024-09-10T21:02:00.811490+0000","last_became_peered":"2024-09-10T21:02:00.811490+0000","last_unstale":"2024-09-10T21:02:41.917309+0000","last_undegraded":"2024-09-10T21:02:41.917309+0000","last_fullsized":"2024-09-10T21:02:41.917309+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:31.557485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870301+0000","last_change":"2024-09-10T21:02:00.809991+0000","last_active":"2024-09-10T21:02:41.870301+0000","last_peered":"2024-09-10T21:02:41.870301+0000","last_clean":"2024-09-10T21:02:41.870301+0000","last_became_active":"2024-09-10T21:02:00.809197+0000","last_became_peered":"2024-09-10T21:02:00.809197+0000","last_unstale":"2024-09-10T21:02:41.870301+0000","last_undegraded":"2024-09-10T21:02:41.870301+0000","last_fullsized":"2024-09-10T21:02:41.870301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.919031+0000","last_change":"2024-09-10T21:02:41.918484+0000","last_active":"2024-09-10T21:02:41.919031+0000","last_peered":"2024-09-10T21:02:41.919031+0000","last_clean":"2024-09-10T21:02:41.919031+0000","last_became_active":"2024-09-10T21:02:41.918068+0000","last_became_peered":"2024-09-10T21:02:41.918068+0000","last_unstale":"2024-09-10T21:02:41.919031+0000","last_undegraded":"2024-09-10T21:02:41.919031+0000","last_fullsized":"2024-09-10T21:02:41.919031+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:08.954072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917107+0000","last_change":"2024-09-10T21:02:00.807280+0000","last_active":"2024-09-10T21:02:41.917107+0000","last_peered":"2024-09-10T21:02:41.917107+0000","last_clean":"2024-09-10T21:02:41.917107+0000","last_became_active":"2024-09-10T21:02:00.804247+0000","last_became_peered":"2024-09-10T21:02:00.804247+0000","last_unstale":"2024-09-10T21:02:41.917107+0000","last_undegraded":"2024-09-10T21:02:41.917107+0000","last_fullsized":"2024-09-10T21:02:41.917107+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:28.163960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870460+0000","last_change":"2024-09-10T21:02:00.805723+0000","last_active":"2024-09-10T21:02:41.870460+0000","last_peered":"2024-09-10T21:02:41.870460+0000","last_clean":"2024-09-10T21:02:41.870460+0000","last_became_active":"2024-09-10T21:02:00.805104+0000","last_became_peered":"2024-09-10T21:02:00.805104+0000","last_unstale":"2024-09-10T21:02:41.870460+0000","last_undegraded":"2024-09-10T21:02:41.870460+0000","last_fullsized":"2024-09-10T21:02:41.870460+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.056884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870384+0000","last_change":"2024-09-10T21:02:00.810497+0000","last_active":"2024-09-10T21:02:41.870384+0000","last_peered":"2024-09-10T21:02:41.870384+0000","last_clean":"2024-09-10T21:02:41.870384+0000","last_became_active":"2024-09-10T21:02:00.809392+0000","last_became_peered":"2024-09-10T21:02:00.809392+0000","last_unstale":"2024-09-10T21:02:41.870384+0000","last_undegraded":"2024-09-10T21:02:41.870384+0000","last_fullsized":"2024-09-10T21:02:41.870384+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:51.214771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867732+0000","last_change":"2024-09-10T21:02:00.813973+0000","last_active":"2024-09-10T21:02:41.867732+0000","last_peered":"2024-09-10T21:02:41.867732+0000","last_clean":"2024-09-10T21:02:41.867732+0000","last_became_active":"2024-09-10T21:02:00.813580+0000","last_became_peered":"2024-09-10T21:02:00.813580+0000","last_unstale":"2024-09-10T21:02:41.867732+0000","last_undegraded":"2024-09-10T21:02:41.867732+0000","last_fullsized":"2024-09-10T21:02:41.867732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:48.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867600+0000","last_change":"2024-09-10T21:02:00.804844+0000","last_active":"2024-09-10T21:02:41.867600+0000","last_peered":"2024-09-10T21:02:41.867600+0000","last_clean":"2024-09-10T21:02:41.867600+0000","last_became_active":"2024-09-10T21:02:00.803866+0000","last_became_peered":"2024-09-10T21:02:00.803866+0000","last_unstale":"2024-09-10T21:02:41.867600+0000","last_undegraded":"2024-09-10T21:02:41.867600+0000","last_fullsized":"2024-09-10T21:02:41.867600+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:05.231877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865210+0000","last_change":"2024-09-10T21:02:00.801343+0000","last_active":"2024-09-10T21:02:41.865210+0000","last_peered":"2024-09-10T21:02:41.865210+0000","last_clean":"2024-09-10T21:02:41.865210+0000","last_became_active":"2024-09-10T21:02:00.801008+0000","last_became_peered":"2024-09-10T21:02:00.801008+0000","last_unstale":"2024-09-10T21:02:41.865210+0000","last_undegraded":"2024-09-10T21:02:41.865210+0000","last_fullsized":"2024-09-10T21:02:41.865210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:31.167925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.918116+0000","last_change":"2024-09-10T21:02:41.917997+0000","last_active":"2024-09-10T21:02:41.918116+0000","last_peered":"2024-09-10T21:02:41.918116+0000","last_clean":"2024-09-10T21:02:41.918116+0000","last_became_active":"2024-09-10T21:02:41.917621+0000","last_became_peered":"2024-09-10T21:02:41.917621+0000","last_unstale":"2024-09-10T21:02:41.918116+0000","last_undegraded":"2024-09-10T21:02:41.918116+0000","last_fullsized":"2024-09-10T21:02:41.918116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:12.742475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867954+0000","last_change":"2024-09-10T21:02:00.810502+0000","last_active":"2024-09-10T21:02:41.867954+0000","last_peered":"2024-09-10T21:02:41.867954+0000","last_clean":"2024-09-10T21:02:41.867954+0000","last_became_active":"2024-09-10T21:02:00.810209+0000","last_became_peered":"2024-09-10T21:02:00.810209+0000","last_unstale":"2024-09-10T21:02:41.867954+0000","last_undegraded":"2024-09-10T21:02:41.867954+0000","last_fullsized":"2024-09-10T21:02:41.867954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:51.311272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917146+0000","last_change":"2024-09-10T21:02:00.797119+0000","last_active":"2024-09-10T21:02:41.917146+0000","last_peered":"2024-09-10T21:02:41.917146+0000","last_clean":"2024-09-10T21:02:41.917146+0000","last_became_active":"2024-09-10T21:02:00.796788+0000","last_became_peered":"2024-09-10T21:02:00.796788+0000","last_unstale":"2024-09-10T21:02:41.917146+0000","last_undegraded":"2024-09-10T21:02:41.917146+0000","last_fullsized":"2024-09-10T21:02:41.917146+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:07.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867912+0000","last_change":"2024-09-10T21:02:00.802284+0000","last_active":"2024-09-10T21:02:41.867912+0000","last_peered":"2024-09-10T21:02:41.867912+0000","last_clean":"2024-09-10T21:02:41.867912+0000","last_became_active":"2024-09-10T21:02:00.801964+0000","last_became_peered":"2024-09-10T21:02:00.801964+0000","last_unstale":"2024-09-10T21:02:41.867912+0000","last_undegraded":"2024-09-10T21:02:41.867912+0000","last_fullsized":"2024-09-10T21:02:41.867912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:45.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977578+0000","last_change":"2024-09-10T21:02:00.809196+0000","last_active":"2024-09-10T21:02:41.977578+0000","last_peered":"2024-09-10T21:02:41.977578+0000","last_clean":"2024-09-10T21:02:41.977578+0000","last_became_active":"2024-09-10T21:02:00.808903+0000","last_became_peered":"2024-09-10T21:02:00.808903+0000","last_unstale":"2024-09-10T21:02:41.977578+0000","last_undegraded":"2024-09-10T21:02:41.977578+0000","last_fullsized":"2024-09-10T21:02:41.977578+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:01.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866856+0000","last_change":"2024-09-10T21:02:00.808195+0000","last_active":"2024-09-10T21:02:41.866856+0000","last_peered":"2024-09-10T21:02:41.866856+0000","last_clean":"2024-09-10T21:02:41.866856+0000","last_became_active":"2024-09-10T21:02:00.807670+0000","last_became_peered":"2024-09-10T21:02:00.807670+0000","last_unstale":"2024-09-10T21:02:41.866856+0000","last_undegraded":"2024-09-10T21:02:41.866856+0000","last_fullsized":"2024-09-10T21:02:41.866856+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:12.549344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978030+0000","last_change":"2024-09-10T21:02:00.811401+0000","last_active":"2024-09-10T21:02:41.978030+0000","last_peered":"2024-09-10T21:02:41.978030+0000","last_clean":"2024-09-10T21:02:41.978030+0000","last_became_active":"2024-09-10T21:02:00.811055+0000","last_became_peered":"2024-09-10T21:02:00.811055+0000","last_unstale":"2024-09-10T21:02:41.978030+0000","last_undegraded":"2024-09-10T21:02:41.978030+0000","last_fullsized":"2024-09-10T21:02:41.978030+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:58.641873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871070+0000","last_change":"2024-09-10T21:02:00.813796+0000","last_active":"2024-09-10T21:02:41.871070+0000","last_peered":"2024-09-10T21:02:41.871070+0000","last_clean":"2024-09-10T21:02:41.871070+0000","last_became_active":"2024-09-10T21:02:00.813468+0000","last_became_peered":"2024-09-10T21:02:00.813468+0000","last_unstale":"2024-09-10T21:02:41.871070+0000","last_undegraded":"2024-09-10T21:02:41.871070+0000","last_fullsized":"2024-09-10T21:02:41.871070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:25.964890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868732+0000","last_change":"2024-09-10T21:02:00.811436+0000","last_active":"2024-09-10T21:02:41.868732+0000","last_peered":"2024-09-10T21:02:41.868732+0000","last_clean":"2024-09-10T21:02:41.868732+0000","last_became_active":"2024-09-10T21:02:00.810956+0000","last_became_peered":"2024-09-10T21:02:00.810956+0000","last_unstale":"2024-09-10T21:02:41.868732+0000","last_undegraded":"2024-09-10T21:02:41.868732+0000","last_fullsized":"2024-09-10T21:02:41.868732+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.306832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917070+0000","last_change":"2024-09-10T21:02:00.799370+0000","last_active":"2024-09-10T21:02:41.917070+0000","last_peered":"2024-09-10T21:02:41.917070+0000","last_clean":"2024-09-10T21:02:41.917070+0000","last_became_active":"2024-09-10T21:02:00.799044+0000","last_became_peered":"2024-09-10T21:02:00.799044+0000","last_unstale":"2024-09-10T21:02:41.917070+0000","last_undegraded":"2024-09-10T21:02:41.917070+0000","last_fullsized":"2024-09-10T21:02:41.917070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:40.332088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870566+0000","last_change":"2024-09-10T21:02:00.799830+0000","last_active":"2024-09-10T21:02:41.870566+0000","last_peered":"2024-09-10T21:02:41.870566+0000","last_clean":"2024-09-10T21:02:41.870566+0000","last_became_active":"2024-09-10T21:02:00.799533+0000","last_became_peered":"2024-09-10T21:02:00.799533+0000","last_unstale":"2024-09-10T21:02:41.870566+0000","last_undegraded":"2024-09-10T21:02:41.870566+0000","last_fullsized":"2024-09-10T21:02:41.870566+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:23.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870452+0000","last_change":"2024-09-10T21:02:00.812739+0000","last_active":"2024-09-10T21:02:41.870452+0000","last_peered":"2024-09-10T21:02:41.870452+0000","last_clean":"2024-09-10T21:02:41.870452+0000","last_became_active":"2024-09-10T21:02:00.812395+0000","last_became_peered":"2024-09-10T21:02:00.812395+0000","last_unstale":"2024-09-10T21:02:41.870452+0000","last_undegraded":"2024-09-10T21:02:41.870452+0000","last_fullsized":"2024-09-10T21:02:41.870452+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:55.727155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870327+0000","last_change":"2024-09-10T21:02:00.806291+0000","last_active":"2024-09-10T21:02:41.870327+0000","last_peered":"2024-09-10T21:02:41.870327+0000","last_clean":"2024-09-10T21:02:41.870327+0000","last_became_active":"2024-09-10T21:02:00.805943+0000","last_became_peered":"2024-09-10T21:02:00.805943+0000","last_unstale":"2024-09-10T21:02:41.870327+0000","last_undegraded":"2024-09-10T21:02:41.870327+0000","last_fullsized":"2024-09-10T21:02:41.870327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:21.354105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867304+0000","last_change":"2024-09-10T21:02:00.808774+0000","last_active":"2024-09-10T21:02:41.867304+0000","last_peered":"2024-09-10T21:02:41.867304+0000","last_clean":"2024-09-10T21:02:41.867304+0000","last_became_active":"2024-09-10T21:02:00.808027+0000","last_became_peered":"2024-09-10T21:02:00.808027+0000","last_unstale":"2024-09-10T21:02:41.867304+0000","last_undegraded":"2024-09-10T21:02:41.867304+0000","last_fullsized":"2024-09-10T21:02:41.867304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870862+0000","last_change":"2024-09-10T21:02:00.808349+0000","last_active":"2024-09-10T21:02:41.870862+0000","last_peered":"2024-09-10T21:02:41.870862+0000","last_clean":"2024-09-10T21:02:41.870862+0000","last_became_active":"2024-09-10T21:02:00.807839+0000","last_became_peered":"2024-09-10T21:02:00.807839+0000","last_unstale":"2024-09-10T21:02:41.870862+0000","last_undegraded":"2024-09-10T21:02:41.870862+0000","last_fullsized":"2024-09-10T21:02:41.870862+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:53.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'6","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:05.906805+0000","last_change":"2024-09-10T21:02:41.877376+0000","last_active":"2024-09-10T21:03:05.906805+0000","last_peered":"2024-09-10T21:03:05.906805+0000","last_clean":"2024-09-10T21:03:05.906805+0000","last_became_active":"2024-09-10T21:02:41.871397+0000","last_became_peered":"2024-09-10T21:02:41.871397+0000","last_unstale":"2024-09-10T21:03:05.906805+0000","last_undegraded":"2024-09-10T21:03:05.906805+0000","last_fullsized":"2024-09-10T21:03:05.906805+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:59.961597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.979721+0000","last_change":"2024-09-10T21:02:41.979196+0000","last_active":"2024-09-10T21:02:41.979721+0000","last_peered":"2024-09-10T21:02:41.979721+0000","last_clean":"2024-09-10T21:02:41.979721+0000","last_became_active":"2024-09-10T21:02:41.978781+0000","last_became_peered":"2024-09-10T21:02:41.978781+0000","last_unstale":"2024-09-10T21:02:41.979721+0000","last_undegraded":"2024-09-10T21:02:41.979721+0000","last_fullsized":"2024-09-10T21:02:41.979721+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:15.100055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917027+0000","last_change":"2024-09-10T21:01:58.787619+0000","last_active":"2024-09-10T21:02:41.917027+0000","last_peered":"2024-09-10T21:02:41.917027+0000","last_clean":"2024-09-10T21:02:41.917027+0000","last_became_active":"2024-09-10T21:01:58.787138+0000","last_became_peered":"2024-09-10T21:01:58.787138+0000","last_unstale":"2024-09-10T21:02:41.917027+0000","last_undegraded":"2024-09-10T21:02:41.917027+0000","last_fullsized":"2024-09-10T21:02:41.917027+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:33.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916753+0000","last_change":"2024-09-10T21:01:58.780115+0000","last_active":"2024-09-10T21:02:41.916753+0000","last_peered":"2024-09-10T21:02:41.916753+0000","last_clean":"2024-09-10T21:02:41.916753+0000","last_became_active":"2024-09-10T21:01:58.779757+0000","last_became_peered":"2024-09-10T21:01:58.779757+0000","last_unstale":"2024-09-10T21:02:41.916753+0000","last_undegraded":"2024-09-10T21:02:41.916753+0000","last_fullsized":"2024-09-10T21:02:41.916753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:29.337978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977623+0000","last_change":"2024-09-10T21:02:00.800925+0000","last_active":"2024-09-10T21:02:41.977623+0000","last_peered":"2024-09-10T21:02:41.977623+0000","last_clean":"2024-09-10T21:02:41.977623+0000","last_became_active":"2024-09-10T21:02:00.798567+0000","last_became_peered":"2024-09-10T21:02:00.798567+0000","last_unstale":"2024-09-10T21:02:41.977623+0000","last_undegraded":"2024-09-10T21:02:41.977623+0000","last_fullsized":"2024-09-10T21:02:41.977623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:01.873716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867127+0000","last_change":"2024-09-10T21:01:58.786040+0000","last_active":"2024-09-10T21:02:41.867127+0000","last_peered":"2024-09-10T21:02:41.867127+0000","last_clean":"2024-09-10T21:02:41.867127+0000","last_became_active":"2024-09-10T21:01:58.785703+0000","last_became_peered":"2024-09-10T21:01:58.785703+0000","last_unstale":"2024-09-10T21:02:41.867127+0000","last_undegraded":"2024-09-10T21:02:41.867127+0000","last_fullsized":"2024-09-10T21:02:41.867127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:39.741614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.864899+0000","last_change":"2024-09-10T21:02:00.798368+0000","last_active":"2024-09-10T21:02:41.864899+0000","last_peered":"2024-09-10T21:02:41.864899+0000","last_clean":"2024-09-10T21:02:41.864899+0000","last_became_active":"2024-09-10T21:02:00.798080+0000","last_became_peered":"2024-09-10T21:02:00.798080+0000","last_unstale":"2024-09-10T21:02:41.864899+0000","last_undegraded":"2024-09-10T21:02:41.864899+0000","last_fullsized":"2024-09-10T21:02:41.864899+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:31.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"75'4","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:06.907626+0000","last_change":"2024-09-10T21:01:58.788119+0000","last_active":"2024-09-10T21:03:06.907626+0000","last_peered":"2024-09-10T21:03:06.907626+0000","last_clean":"2024-09-10T21:03:06.907626+0000","last_became_active":"2024-09-10T21:01:58.787724+0000","last_became_peered":"2024-09-10T21:01:58.787724+0000","last_unstale":"2024-09-10T21:03:06.907626+0000","last_undegraded":"2024-09-10T21:03:06.907626+0000","last_fullsized":"2024-09-10T21:03:06.907626+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:42.872858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868220+0000","last_change":"2024-09-10T21:02:00.804425+0000","last_active":"2024-09-10T21:02:41.868220+0000","last_peered":"2024-09-10T21:02:41.868220+0000","last_clean":"2024-09-10T21:02:41.868220+0000","last_became_active":"2024-09-10T21:02:00.804093+0000","last_became_peered":"2024-09-10T21:02:00.804093+0000","last_unstale":"2024-09-10T21:02:41.868220+0000","last_undegraded":"2024-09-10T21:02:41.868220+0000","last_fullsized":"2024-09-10T21:02:41.868220+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:46.183326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870381+0000","last_change":"2024-09-10T21:01:58.791323+0000","last_active":"2024-09-10T21:02:41.870381+0000","last_peered":"2024-09-10T21:02:41.870381+0000","last_clean":"2024-09-10T21:02:41.870381+0000","last_became_active":"2024-09-10T21:01:58.790551+0000","last_became_peered":"2024-09-10T21:01:58.790551+0000","last_unstale":"2024-09-10T21:02:41.870381+0000","last_undegraded":"2024-09-10T21:02:41.870381+0000","last_fullsized":"2024-09-10T21:02:41.870381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:18.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870908+0000","last_change":"2024-09-10T21:02:00.804025+0000","last_active":"2024-09-10T21:02:41.870908+0000","last_peered":"2024-09-10T21:02:41.870908+0000","last_clean":"2024-09-10T21:02:41.870908+0000","last_became_active":"2024-09-10T21:02:00.803663+0000","last_became_peered":"2024-09-10T21:02:00.803663+0000","last_unstale":"2024-09-10T21:02:41.870908+0000","last_undegraded":"2024-09-10T21:02:41.870908+0000","last_fullsized":"2024-09-10T21:02:41.870908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:30.609020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868035+0000","last_change":"2024-09-10T21:01:58.887067+0000","last_active":"2024-09-10T21:02:41.868035+0000","last_peered":"2024-09-10T21:02:41.868035+0000","last_clean":"2024-09-10T21:02:41.868035+0000","last_became_active":"2024-09-10T21:01:58.886611+0000","last_became_peered":"2024-09-10T21:01:58.886611+0000","last_unstale":"2024-09-10T21:02:41.868035+0000","last_undegraded":"2024-09-10T21:02:41.868035+0000","last_fullsized":"2024-09-10T21:02:41.868035+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:06.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.918116+0000","last_change":"2024-09-10T21:02:41.917998+0000","last_active":"2024-09-10T21:02:41.918116+0000","last_peered":"2024-09-10T21:02:41.918116+0000","last_clean":"2024-09-10T21:02:41.918116+0000","last_became_active":"2024-09-10T21:02:41.917665+0000","last_became_peered":"2024-09-10T21:02:41.917665+0000","last_unstale":"2024-09-10T21:02:41.918116+0000","last_undegraded":"2024-09-10T21:02:41.918116+0000","last_fullsized":"2024-09-10T21:02:41.918116+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:15.147379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'31","reported_seq":171,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:06.907600+0000","last_change":"2024-09-10T21:01:58.790966+0000","last_active":"2024-09-10T21:03:06.907600+0000","last_peered":"2024-09-10T21:03:06.907600+0000","last_clean":"2024-09-10T21:03:06.907600+0000","last_became_active":"2024-09-10T21:01:58.790636+0000","last_became_peered":"2024-09-10T21:01:58.790636+0000","last_unstale":"2024-09-10T21:03:06.907600+0000","last_undegraded":"2024-09-10T21:03:06.907600+0000","last_fullsized":"2024-09-10T21:03:06.907600+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:06.209074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61170,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.872335+0000","last_change":"2024-09-10T21:02:00.810610+0000","last_active":"2024-09-10T21:02:41.872335+0000","last_peered":"2024-09-10T21:02:41.872335+0000","last_clean":"2024-09-10T21:02:41.872335+0000","last_became_active":"2024-09-10T21:02:00.810224+0000","last_became_peered":"2024-09-10T21:02:00.810224+0000","last_unstale":"2024-09-10T21:02:41.872335+0000","last_undegraded":"2024-09-10T21:02:41.872335+0000","last_fullsized":"2024-09-10T21:02:41.872335+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:32.785312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870285+0000","last_change":"2024-09-10T21:01:58.786476+0000","last_active":"2024-09-10T21:02:41.870285+0000","last_peered":"2024-09-10T21:02:41.870285+0000","last_clean":"2024-09-10T21:02:41.870285+0000","last_became_active":"2024-09-10T21:01:58.786153+0000","last_became_peered":"2024-09-10T21:01:58.786153+0000","last_unstale":"2024-09-10T21:02:41.870285+0000","last_undegraded":"2024-09-10T21:02:41.870285+0000","last_fullsized":"2024-09-10T21:02:41.870285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:00.118591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916861+0000","last_change":"2024-09-10T21:02:00.796138+0000","last_active":"2024-09-10T21:02:41.916861+0000","last_peered":"2024-09-10T21:02:41.916861+0000","last_clean":"2024-09-10T21:02:41.916861+0000","last_became_active":"2024-09-10T21:02:00.795847+0000","last_became_peered":"2024-09-10T21:02:00.795847+0000","last_unstale":"2024-09-10T21:02:41.916861+0000","last_undegraded":"2024-09-10T21:02:41.916861+0000","last_fullsized":"2024-09-10T21:02:41.916861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:29.462134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867576+0000","last_change":"2024-09-10T21:01:58.789314+0000","last_active":"2024-09-10T21:02:41.867576+0000","last_peered":"2024-09-10T21:02:41.867576+0000","last_clean":"2024-09-10T21:02:41.867576+0000","last_became_active":"2024-09-10T21:01:58.788594+0000","last_became_peered":"2024-09-10T21:01:58.788594+0000","last_unstale":"2024-09-10T21:02:41.867576+0000","last_undegraded":"2024-09-10T21:02:41.867576+0000","last_fullsized":"2024-09-10T21:02:41.867576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:44.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870196+0000","last_change":"2024-09-10T21:02:00.799828+0000","last_active":"2024-09-10T21:02:41.870196+0000","last_peered":"2024-09-10T21:02:41.870196+0000","last_clean":"2024-09-10T21:02:41.870196+0000","last_became_active":"2024-09-10T21:02:00.799465+0000","last_became_peered":"2024-09-10T21:02:00.799465+0000","last_unstale":"2024-09-10T21:02:41.870196+0000","last_undegraded":"2024-09-10T21:02:41.870196+0000","last_fullsized":"2024-09-10T21:02:41.870196+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:56.794326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"75'6","reported_seq":83,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:01.942060+0000","last_change":"2024-09-10T21:01:58.788730+0000","last_active":"2024-09-10T21:03:01.942060+0000","last_peered":"2024-09-10T21:03:01.942060+0000","last_clean":"2024-09-10T21:03:01.942060+0000","last_became_active":"2024-09-10T21:01:58.788317+0000","last_became_peered":"2024-09-10T21:01:58.788317+0000","last_unstale":"2024-09-10T21:03:01.942060+0000","last_undegraded":"2024-09-10T21:03:01.942060+0000","last_fullsized":"2024-09-10T21:03:01.942060+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:52.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.869845+0000","last_change":"2024-09-10T21:02:00.807104+0000","last_active":"2024-09-10T21:02:41.869845+0000","last_peered":"2024-09-10T21:02:41.869845+0000","last_clean":"2024-09-10T21:02:41.869845+0000","last_became_active":"2024-09-10T21:02:00.806815+0000","last_became_peered":"2024-09-10T21:02:00.806815+0000","last_unstale":"2024-09-10T21:02:41.869845+0000","last_undegraded":"2024-09-10T21:02:41.869845+0000","last_fullsized":"2024-09-10T21:02:41.869845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:24.747662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.890722+0000","last_change":"2024-09-10T21:02:41.890657+0000","last_active":"2024-09-10T21:02:41.890722+0000","last_peered":"2024-09-10T21:02:41.890722+0000","last_clean":"2024-09-10T21:02:41.890722+0000","last_became_active":"2024-09-10T21:02:41.873778+0000","last_became_peered":"2024-09-10T21:02:41.873778+0000","last_unstale":"2024-09-10T21:02:41.890722+0000","last_undegraded":"2024-09-10T21:02:41.890722+0000","last_fullsized":"2024-09-10T21:02:41.890722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:58.678679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867451+0000","last_change":"2024-09-10T21:02:00.806796+0000","last_active":"2024-09-10T21:02:41.867451+0000","last_peered":"2024-09-10T21:02:41.867451+0000","last_clean":"2024-09-10T21:02:41.867451+0000","last_became_active":"2024-09-10T21:02:00.805802+0000","last_became_peered":"2024-09-10T21:02:00.805802+0000","last_unstale":"2024-09-10T21:02:41.867451+0000","last_undegraded":"2024-09-10T21:02:41.867451+0000","last_fullsized":"2024-09-10T21:02:41.867451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.962870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871648+0000","last_change":"2024-09-10T21:01:58.781088+0000","last_active":"2024-09-10T21:02:41.871648+0000","last_peered":"2024-09-10T21:02:41.871648+0000","last_clean":"2024-09-10T21:02:41.871648+0000","last_became_active":"2024-09-10T21:01:58.780731+0000","last_became_peered":"2024-09-10T21:01:58.780731+0000","last_unstale":"2024-09-10T21:02:41.871648+0000","last_undegraded":"2024-09-10T21:02:41.871648+0000","last_fullsized":"2024-09-10T21:02:41.871648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:50.636077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867188+0000","last_change":"2024-09-10T21:02:00.806551+0000","last_active":"2024-09-10T21:02:41.867188+0000","last_peered":"2024-09-10T21:02:41.867188+0000","last_clean":"2024-09-10T21:02:41.867188+0000","last_became_active":"2024-09-10T21:02:00.806134+0000","last_became_peered":"2024-09-10T21:02:00.806134+0000","last_unstale":"2024-09-10T21:02:41.867188+0000","last_undegraded":"2024-09-10T21:02:41.867188+0000","last_fullsized":"2024-09-10T21:02:41.867188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.574372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916954+0000","last_change":"2024-09-10T21:01:58.780294+0000","last_active":"2024-09-10T21:02:41.916954+0000","last_peered":"2024-09-10T21:02:41.916954+0000","last_clean":"2024-09-10T21:02:41.916954+0000","last_became_active":"2024-09-10T21:01:58.779767+0000","last_became_peered":"2024-09-10T21:01:58.779767+0000","last_unstale":"2024-09-10T21:02:41.916954+0000","last_undegraded":"2024-09-10T21:02:41.916954+0000","last_fullsized":"2024-09-10T21:02:41.916954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:21.734159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870426+0000","last_change":"2024-09-10T21:02:00.809393+0000","last_active":"2024-09-10T21:02:41.870426+0000","last_peered":"2024-09-10T21:02:41.870426+0000","last_clean":"2024-09-10T21:02:41.870426+0000","last_became_active":"2024-09-10T21:02:00.809012+0000","last_became_peered":"2024-09-10T21:02:00.809012+0000","last_unstale":"2024-09-10T21:02:41.870426+0000","last_undegraded":"2024-09-10T21:02:41.870426+0000","last_fullsized":"2024-09-10T21:02:41.870426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:03.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868189+0000","last_change":"2024-09-10T21:01:58.786507+0000","last_active":"2024-09-10T21:02:41.868189+0000","last_peered":"2024-09-10T21:02:41.868189+0000","last_clean":"2024-09-10T21:02:41.868189+0000","last_became_active":"2024-09-10T21:01:58.786115+0000","last_became_peered":"2024-09-10T21:01:58.786115+0000","last_unstale":"2024-09-10T21:02:41.868189+0000","last_undegraded":"2024-09-10T21:02:41.868189+0000","last_fullsized":"2024-09-10T21:02:41.868189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.985353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870255+0000","last_change":"2024-09-10T21:02:00.805050+0000","last_active":"2024-09-10T21:02:41.870255+0000","last_peered":"2024-09-10T21:02:41.870255+0000","last_clean":"2024-09-10T21:02:41.870255+0000","last_became_active":"2024-09-10T21:02:00.804706+0000","last_became_peered":"2024-09-10T21:02:00.804706+0000","last_unstale":"2024-09-10T21:02:41.870255+0000","last_undegraded":"2024-09-10T21:02:41.870255+0000","last_fullsized":"2024-09-10T21:02:41.870255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:15.463986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:02.945473+0000","last_change":"2024-09-10T21:01:59.114525+0000","last_active":"2024-09-10T21:03:02.945473+0000","last_peered":"2024-09-10T21:03:02.945473+0000","last_clean":"2024-09-10T21:03:02.945473+0000","last_became_active":"2024-09-10T21:01:59.114076+0000","last_became_peered":"2024-09-10T21:01:59.114076+0000","last_unstale":"2024-09-10T21:03:02.945473+0000","last_undegraded":"2024-09-10T21:03:02.945473+0000","last_fullsized":"2024-09-10T21:03:02.945473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:53.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":55,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870338+0000","last_change":"2024-09-10T21:02:00.799803+0000","last_active":"2024-09-10T21:02:41.870338+0000","last_peered":"2024-09-10T21:02:41.870338+0000","last_clean":"2024-09-10T21:02:41.870338+0000","last_became_active":"2024-09-10T21:02:00.799459+0000","last_became_peered":"2024-09-10T21:02:00.799459+0000","last_unstale":"2024-09-10T21:02:41.870338+0000","last_undegraded":"2024-09-10T21:02:41.870338+0000","last_fullsized":"2024-09-10T21:02:41.870338+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:57.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":98,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870056+0000","last_change":"2024-09-10T21:00:48.381903+0000","last_active":"2024-09-10T21:02:41.870056+0000","last_peered":"2024-09-10T21:02:41.870056+0000","last_clean":"2024-09-10T21:02:41.870056+0000","last_became_active":"2024-09-10T21:00:48.376874+0000","last_became_peered":"2024-09-10T21:00:48.376874+0000","last_unstale":"2024-09-10T21:02:41.870056+0000","last_undegraded":"2024-09-10T21:02:41.870056+0000","last_fullsized":"2024-09-10T21:02:41.870056+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_clean_scrub_stamp":"2024-09-10T20:57:24.516525+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:35.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866334+0000","last_change":"2024-09-10T21:01:58.788005+0000","last_active":"2024-09-10T21:02:41.866334+0000","last_peered":"2024-09-10T21:02:41.866334+0000","last_clean":"2024-09-10T21:02:41.866334+0000","last_became_active":"2024-09-10T21:01:58.786471+0000","last_became_peered":"2024-09-10T21:01:58.786471+0000","last_unstale":"2024-09-10T21:02:41.866334+0000","last_undegraded":"2024-09-10T21:02:41.866334+0000","last_fullsized":"2024-09-10T21:02:41.866334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:15.622814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977860+0000","last_change":"2024-09-10T21:02:00.802573+0000","last_active":"2024-09-10T21:02:41.977860+0000","last_peered":"2024-09-10T21:02:41.977860+0000","last_clean":"2024-09-10T21:02:41.977860+0000","last_became_active":"2024-09-10T21:02:00.802263+0000","last_became_peered":"2024-09-10T21:02:00.802263+0000","last_unstale":"2024-09-10T21:02:41.977860+0000","last_undegraded":"2024-09-10T21:02:41.977860+0000","last_fullsized":"2024-09-10T21:02:41.977860+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:54.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870004+0000","last_change":"2024-09-10T21:01:58.785700+0000","last_active":"2024-09-10T21:02:41.870004+0000","last_peered":"2024-09-10T21:02:41.870004+0000","last_clean":"2024-09-10T21:02:41.870004+0000","last_became_active":"2024-09-10T21:01:58.785346+0000","last_became_peered":"2024-09-10T21:01:58.785346+0000","last_unstale":"2024-09-10T21:02:41.870004+0000","last_undegraded":"2024-09-10T21:02:41.870004+0000","last_fullsized":"2024-09-10T21:02:41.870004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:39.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916719+0000","last_change":"2024-09-10T21:02:00.808045+0000","last_active":"2024-09-10T21:02:41.916719+0000","last_peered":"2024-09-10T21:02:41.916719+0000","last_clean":"2024-09-10T21:02:41.916719+0000","last_became_active":"2024-09-10T21:02:00.807757+0000","last_became_peered":"2024-09-10T21:02:00.807757+0000","last_unstale":"2024-09-10T21:02:41.916719+0000","last_undegraded":"2024-09-10T21:02:41.916719+0000","last_fullsized":"2024-09-10T21:02:41.916719+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:16.198290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"75'5","reported_seq":81,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:02.942753+0000","last_change":"2024-09-10T21:01:58.884939+0000","last_active":"2024-09-10T21:03:02.942753+0000","last_peered":"2024-09-10T21:03:02.942753+0000","last_clean":"2024-09-10T21:03:02.942753+0000","last_became_active":"2024-09-10T21:01:58.884517+0000","last_became_peered":"2024-09-10T21:01:58.884517+0000","last_unstale":"2024-09-10T21:03:02.942753+0000","last_undegraded":"2024-09-10T21:03:02.942753+0000","last_fullsized":"2024-09-10T21:03:02.942753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.579662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":53,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866940+0000","last_change":"2024-09-10T21:02:00.813021+0000","last_active":"2024-09-10T21:02:41.866940+0000","last_peered":"2024-09-10T21:02:41.866940+0000","last_clean":"2024-09-10T21:02:41.866940+0000","last_became_active":"2024-09-10T21:02:00.812622+0000","last_became_peered":"2024-09-10T21:02:00.812622+0000","last_unstale":"2024-09-10T21:02:41.866940+0000","last_undegraded":"2024-09-10T21:02:41.866940+0000","last_fullsized":"2024-09-10T21:02:41.866940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:48.348100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"75'6","reported_seq":84,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:03.943896+0000","last_change":"2024-09-10T21:01:58.792187+0000","last_active":"2024-09-10T21:03:03.943896+0000","last_peered":"2024-09-10T21:03:03.943896+0000","last_clean":"2024-09-10T21:03:03.943896+0000","last_became_active":"2024-09-10T21:01:58.791806+0000","last_became_peered":"2024-09-10T21:01:58.791806+0000","last_unstale":"2024-09-10T21:03:03.943896+0000","last_undegraded":"2024-09-10T21:03:03.943896+0000","last_fullsized":"2024-09-10T21:03:03.943896+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:48.493396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":55,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917092+0000","last_change":"2024-09-10T21:02:00.804073+0000","last_active":"2024-09-10T21:02:41.917092+0000","last_peered":"2024-09-10T21:02:41.917092+0000","last_clean":"2024-09-10T21:02:41.917092+0000","last_became_active":"2024-09-10T21:02:00.803760+0000","last_became_peered":"2024-09-10T21:02:00.803760+0000","last_unstale":"2024-09-10T21:02:41.917092+0000","last_undegraded":"2024-09-10T21:02:41.917092+0000","last_fullsized":"2024-09-10T21:02:41.917092+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:01.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865323+0000","last_change":"2024-09-10T21:01:58.885088+0000","last_active":"2024-09-10T21:02:41.865323+0000","last_peered":"2024-09-10T21:02:41.865323+0000","last_clean":"2024-09-10T21:02:41.865323+0000","last_became_active":"2024-09-10T21:01:58.884729+0000","last_became_peered":"2024-09-10T21:01:58.884729+0000","last_unstale":"2024-09-10T21:02:41.865323+0000","last_undegraded":"2024-09-10T21:02:41.865323+0000","last_fullsized":"2024-09-10T21:02:41.865323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:37.451629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917111+0000","last_change":"2024-09-10T21:02:00.810329+0000","last_active":"2024-09-10T21:02:41.917111+0000","last_peered":"2024-09-10T21:02:41.917111+0000","last_clean":"2024-09-10T21:02:41.917111+0000","last_became_active":"2024-09-10T21:02:00.809985+0000","last_became_peered":"2024-09-10T21:02:00.809985+0000","last_unstale":"2024-09-10T21:02:41.917111+0000","last_undegraded":"2024-09-10T21:02:41.917111+0000","last_fullsized":"2024-09-10T21:02:41.917111+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:39.631969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.880605+0000","last_change":"2024-09-10T21:02:41.880117+0000","last_active":"2024-09-10T21:02:41.880605+0000","last_peered":"2024-09-10T21:02:41.880605+0000","last_clean":"2024-09-10T21:02:41.880605+0000","last_became_active":"2024-09-10T21:02:41.872494+0000","last_became_peered":"2024-09-10T21:02:41.872494+0000","last_unstale":"2024-09-10T21:02:41.880605+0000","last_undegraded":"2024-09-10T21:02:41.880605+0000","last_fullsized":"2024-09-10T21:02:41.880605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:06.965479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917090+0000","last_change":"2024-09-10T21:02:00.804961+0000","last_active":"2024-09-10T21:02:41.917090+0000","last_peered":"2024-09-10T21:02:41.917090+0000","last_clean":"2024-09-10T21:02:41.917090+0000","last_became_active":"2024-09-10T21:02:00.804147+0000","last_became_peered":"2024-09-10T21:02:00.804147+0000","last_unstale":"2024-09-10T21:02:41.917090+0000","last_undegraded":"2024-09-10T21:02:41.917090+0000","last_fullsized":"2024-09-10T21:02:41.917090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:15.801815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'14","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865328+0000","last_change":"2024-09-10T21:01:58.781292+0000","last_active":"2024-09-10T21:02:41.865328+0000","last_peered":"2024-09-10T21:02:41.865328+0000","last_clean":"2024-09-10T21:02:41.865328+0000","last_became_active":"2024-09-10T21:01:58.780995+0000","last_became_peered":"2024-09-10T21:01:58.780995+0000","last_unstale":"2024-09-10T21:02:41.865328+0000","last_undegraded":"2024-09-10T21:02:41.865328+0000","last_fullsized":"2024-09-10T21:02:41.865328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:56.914723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978081+0000","last_change":"2024-09-10T21:02:00.805878+0000","last_active":"2024-09-10T21:02:41.978081+0000","last_peered":"2024-09-10T21:02:41.978081+0000","last_clean":"2024-09-10T21:02:41.978081+0000","last_became_active":"2024-09-10T21:02:00.804565+0000","last_became_peered":"2024-09-10T21:02:00.804565+0000","last_unstale":"2024-09-10T21:02:41.978081+0000","last_undegraded":"2024-09-10T21:02:41.978081+0000","last_fullsized":"2024-09-10T21:02:41.978081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:24.359643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866263+0000","last_change":"2024-09-10T21:01:59.114862+0000","last_active":"2024-09-10T21:02:41.866263+0000","last_peered":"2024-09-10T21:02:41.866263+0000","last_clean":"2024-09-10T21:02:41.866263+0000","last_became_active":"2024-09-10T21:01:59.114411+0000","last_became_peered":"2024-09-10T21:01:59.114411+0000","last_unstale":"2024-09-10T21:02:41.866263+0000","last_undegraded":"2024-09-10T21:02:41.866263+0000","last_fullsized":"2024-09-10T21:02:41.866263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.645829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870778+0000","last_change":"2024-09-10T21:02:00.799230+0000","last_active":"2024-09-10T21:02:41.870778+0000","last_peered":"2024-09-10T21:02:41.870778+0000","last_clean":"2024-09-10T21:02:41.870778+0000","last_became_active":"2024-09-10T21:02:00.798992+0000","last_became_peered":"2024-09-10T21:02:00.798992+0000","last_unstale":"2024-09-10T21:02:41.870778+0000","last_undegraded":"2024-09-10T21:02:41.870778+0000","last_fullsized":"2024-09-10T21:02:41.870778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:40.322066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867967+0000","last_change":"2024-09-10T21:01:58.887042+0000","last_active":"2024-09-10T21:02:41.867967+0000","last_peered":"2024-09-10T21:02:41.867967+0000","last_clean":"2024-09-10T21:02:41.867967+0000","last_became_active":"2024-09-10T21:01:58.886710+0000","last_became_peered":"2024-09-10T21:01:58.886710+0000","last_unstale":"2024-09-10T21:02:41.867967+0000","last_undegraded":"2024-09-10T21:02:41.867967+0000","last_fullsized":"2024-09-10T21:02:41.867967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:42.672899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870544+0000","last_change":"2024-09-10T21:02:00.809763+0000","last_active":"2024-09-10T21:02:41.870544+0000","last_peered":"2024-09-10T21:02:41.870544+0000","last_clean":"2024-09-10T21:02:41.870544+0000","last_became_active":"2024-09-10T21:02:00.808955+0000","last_became_peered":"2024-09-10T21:02:00.808955+0000","last_unstale":"2024-09-10T21:02:41.870544+0000","last_undegraded":"2024-09-10T21:02:41.870544+0000","last_fullsized":"2024-09-10T21:02:41.870544+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:22.622852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977719+0000","last_change":"2024-09-10T21:01:58.786312+0000","last_active":"2024-09-10T21:02:41.977719+0000","last_peered":"2024-09-10T21:02:41.977719+0000","last_clean":"2024-09-10T21:02:41.977719+0000","last_became_active":"2024-09-10T21:01:58.785894+0000","last_became_peered":"2024-09-10T21:01:58.785894+0000","last_unstale":"2024-09-10T21:02:41.977719+0000","last_undegraded":"2024-09-10T21:02:41.977719+0000","last_fullsized":"2024-09-10T21:02:41.977719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:11.682904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867398+0000","last_change":"2024-09-10T21:02:00.808374+0000","last_active":"2024-09-10T21:02:41.867398+0000","last_peered":"2024-09-10T21:02:41.867398+0000","last_clean":"2024-09-10T21:02:41.867398+0000","last_became_active":"2024-09-10T21:02:00.807565+0000","last_became_peered":"2024-09-10T21:02:00.807565+0000","last_unstale":"2024-09-10T21:02:41.867398+0000","last_undegraded":"2024-09-10T21:02:41.867398+0000","last_fullsized":"2024-09-10T21:02:41.867398+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:24.227480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870458+0000","last_change":"2024-09-10T21:01:58.782375+0000","last_active":"2024-09-10T21:02:41.870458+0000","last_peered":"2024-09-10T21:02:41.870458+0000","last_clean":"2024-09-10T21:02:41.870458+0000","last_became_active":"2024-09-10T21:01:58.781964+0000","last_became_peered":"2024-09-10T21:01:58.781964+0000","last_unstale":"2024-09-10T21:02:41.870458+0000","last_undegraded":"2024-09-10T21:02:41.870458+0000","last_fullsized":"2024-09-10T21:02:41.870458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.460126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977861+0000","last_change":"2024-09-10T21:02:00.806612+0000","last_active":"2024-09-10T21:02:41.977861+0000","last_peered":"2024-09-10T21:02:41.977861+0000","last_clean":"2024-09-10T21:02:41.977861+0000","last_became_active":"2024-09-10T21:02:00.803470+0000","last_became_peered":"2024-09-10T21:02:00.803470+0000","last_unstale":"2024-09-10T21:02:41.977861+0000","last_undegraded":"2024-09-10T21:02:41.977861+0000","last_fullsized":"2024-09-10T21:02:41.977861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:08.350931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871476+0000","last_change":"2024-09-10T21:01:58.788987+0000","last_active":"2024-09-10T21:02:41.871476+0000","last_peered":"2024-09-10T21:02:41.871476+0000","last_clean":"2024-09-10T21:02:41.871476+0000","last_became_active":"2024-09-10T21:01:58.788605+0000","last_became_peered":"2024-09-10T21:01:58.788605+0000","last_unstale":"2024-09-10T21:02:41.871476+0000","last_undegraded":"2024-09-10T21:02:41.871476+0000","last_fullsized":"2024-09-10T21:02:41.871476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:45.798350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870058+0000","last_change":"2024-09-10T21:02:00.804200+0000","last_active":"2024-09-10T21:02:41.870058+0000","last_peered":"2024-09-10T21:02:41.870058+0000","last_clean":"2024-09-10T21:02:41.870058+0000","last_became_active":"2024-09-10T21:02:00.803875+0000","last_became_peered":"2024-09-10T21:02:00.803875+0000","last_unstale":"2024-09-10T21:02:41.870058+0000","last_undegraded":"2024-09-10T21:02:41.870058+0000","last_fullsized":"2024-09-10T21:02:41.870058+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:15.930163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.874681+0000","last_change":"2024-09-10T21:02:41.874580+0000","last_active":"2024-09-10T21:02:41.874681+0000","last_peered":"2024-09-10T21:02:41.874681+0000","last_clean":"2024-09-10T21:02:41.874681+0000","last_became_active":"2024-09-10T21:02:41.873545+0000","last_became_peered":"2024-09-10T21:02:41.873545+0000","last_unstale":"2024-09-10T21:02:41.874681+0000","last_undegraded":"2024-09-10T21:02:41.874681+0000","last_fullsized":"2024-09-10T21:02:41.874681+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:05.278855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870983+0000","last_change":"2024-09-10T21:01:58.784834+0000","last_active":"2024-09-10T21:02:41.870983+0000","last_peered":"2024-09-10T21:02:41.870983+0000","last_clean":"2024-09-10T21:02:41.870983+0000","last_became_active":"2024-09-10T21:01:58.784464+0000","last_became_peered":"2024-09-10T21:01:58.784464+0000","last_unstale":"2024-09-10T21:02:41.870983+0000","last_undegraded":"2024-09-10T21:02:41.870983+0000","last_fullsized":"2024-09-10T21:02:41.870983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:09.864824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867905+0000","last_change":"2024-09-10T21:02:00.805677+0000","last_active":"2024-09-10T21:02:41.867905+0000","last_peered":"2024-09-10T21:02:41.867905+0000","last_clean":"2024-09-10T21:02:41.867905+0000","last_became_active":"2024-09-10T21:02:00.805341+0000","last_became_peered":"2024-09-10T21:02:00.805341+0000","last_unstale":"2024-09-10T21:02:41.867905+0000","last_undegraded":"2024-09-10T21:02:41.867905+0000","last_fullsized":"2024-09-10T21:02:41.867905+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.236954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.889813+0000","last_change":"2024-09-10T21:02:41.889336+0000","last_active":"2024-09-10T21:02:41.889813+0000","last_peered":"2024-09-10T21:02:41.889813+0000","last_clean":"2024-09-10T21:02:41.889813+0000","last_became_active":"2024-09-10T21:02:41.874164+0000","last_became_peered":"2024-09-10T21:02:41.874164+0000","last_unstale":"2024-09-10T21:02:41.889813+0000","last_undegraded":"2024-09-10T21:02:41.889813+0000","last_fullsized":"2024-09-10T21:02:41.889813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.799546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870893+0000","last_change":"2024-09-10T21:02:00.801514+0000","last_active":"2024-09-10T21:02:41.870893+0000","last_peered":"2024-09-10T21:02:41.870893+0000","last_clean":"2024-09-10T21:02:41.870893+0000","last_became_active":"2024-09-10T21:02:00.801221+0000","last_became_peered":"2024-09-10T21:02:00.801221+0000","last_unstale":"2024-09-10T21:02:41.870893+0000","last_undegraded":"2024-09-10T21:02:41.870893+0000","last_fullsized":"2024-09-10T21:02:41.870893+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:35.564890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978209+0000","last_change":"2024-09-10T21:01:58.884876+0000","last_active":"2024-09-10T21:02:41.978209+0000","last_peered":"2024-09-10T21:02:41.978209+0000","last_clean":"2024-09-10T21:02:41.978209+0000","last_became_active":"2024-09-10T21:01:58.884530+0000","last_became_peered":"2024-09-10T21:01:58.884530+0000","last_unstale":"2024-09-10T21:02:41.978209+0000","last_undegraded":"2024-09-10T21:02:41.978209+0000","last_fullsized":"2024-09-10T21:02:41.978209+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.851017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978109+0000","last_change":"2024-09-10T21:02:00.811148+0000","last_active":"2024-09-10T21:02:41.978109+0000","last_peered":"2024-09-10T21:02:41.978109+0000","last_clean":"2024-09-10T21:02:41.978109+0000","last_became_active":"2024-09-10T21:02:00.809265+0000","last_became_peered":"2024-09-10T21:02:00.809265+0000","last_unstale":"2024-09-10T21:02:41.978109+0000","last_undegraded":"2024-09-10T21:02:41.978109+0000","last_fullsized":"2024-09-10T21:02:41.978109+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:25.269876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'13","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:05.946051+0000","last_change":"2024-09-10T21:01:58.786917+0000","last_active":"2024-09-10T21:03:05.946051+0000","last_peered":"2024-09-10T21:03:05.946051+0000","last_clean":"2024-09-10T21:03:05.946051+0000","last_became_active":"2024-09-10T21:01:58.786595+0000","last_became_peered":"2024-09-10T21:01:58.786595+0000","last_unstale":"2024-09-10T21:03:05.946051+0000","last_undegraded":"2024-09-10T21:03:05.946051+0000","last_fullsized":"2024-09-10T21:03:05.946051+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:30.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":69156,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":572,"num_read_kb":364,"num_write":245,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1237,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":207468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743450,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":847872,"data_stored":407530,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939680,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":995328,"data_stored":1059617,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":864256,"data_stored":406957,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":407223,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495661,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":1000635,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659187,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32020,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713132,"statfs":{"total":95995035648,"available":95962247168,"internally_reserved":0,"allocated":847872,"data_stored":407065,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855417,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":925696,"data_stored":999531,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":856064,"data_stored":407939,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215171,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":864256,"data_stored":406499,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444104,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":468634,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":469260,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":408404,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:03:10.842 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T21:03:10.842 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T21:03:10.842 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T21:03:10.842 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T21:03:11.110 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:11.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:11 smithi033 ceph-mon[30860]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:03:11.485 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:11 smithi033 ceph-mon[30860]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 -- 172.21.15.33:0/3955141060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450109150 msgr2=0x7f745010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 --2- 172.21.15.33:0/3955141060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450109150 0x7f745010f950 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7448009ca0 tx=0x7f744802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 -- 172.21.15.33:0/3955141060 shutdown_connections 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 --2- 172.21.15.33:0/3955141060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7450109150 0x7f745010f950 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 --2- 172.21.15.33:0/3955141060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74501087d0 0x7f7450108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 --2- 172.21.15.33:0/3955141060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102dc0 0x7f74501031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 -- 172.21.15.33:0/3955141060 >> 172.21.15.33:0/3955141060 conn(0x7f74500fe830 msgr2=0x7f7450100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 -- 172.21.15.33:0/3955141060 shutdown_connections 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.558+0000 7f7457018640 1 -- 172.21.15.33:0/3955141060 wait complete. 2024-09-10T21:03:11.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 Processor -- start 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 -- start start 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102dc0 0x7f74501992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74501087d0 0x7f74501997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450109150 0x7f745019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7450112e60 con 0x7f74501087d0 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7450112ce0 con 0x7f7450102dc0 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7457018640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7450112fe0 con 0x7f7450109150 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7447fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74501087d0 0x7f74501997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7447fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74501087d0 0x7f74501997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56666/0 (socket says 172.21.15.33:56666) 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7447fff640 1 -- 172.21.15.33:0/216298847 learned_addr learned my addr 172.21.15.33:0/216298847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.559+0000 7f7454db4640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102dc0 0x7f74501992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f74555b5640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450109150 0x7f745019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:11.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7447fff640 1 -- 172.21.15.33:0/216298847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450109150 msgr2=0x7f745019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7447fff640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450109150 0x7f745019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7447fff640 1 -- 172.21.15.33:0/216298847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102dc0 msgr2=0x7f74501992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7447fff640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102dc0 0x7f74501992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7447fff640 1 -- 172.21.15.33:0/216298847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745019f360 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7447fff640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74501087d0 0x7f74501997f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f744000d950 tx=0x7f744000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7445ffb640 1 -- 172.21.15.33:0/216298847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440017070 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745019f650 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7445ffb640 1 -- 172.21.15.33:0/216298847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7440009c30 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7445ffb640 1 -- 172.21.15.33:0/216298847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744000f620 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.560+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74501a5d10 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.561+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74501031a0 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.563+0000 7f7445ffb640 1 -- 172.21.15.33:0/216298847 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f7440010460 con 0x7f74501087d0 2024-09-10T21:03:11.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.563+0000 7f7445ffb640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b5370 0x7f74380b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.563+0000 7f7454db4640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b5370 0x7f74380b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.563+0000 7f7445ffb640 1 -- 172.21.15.33:0/216298847 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f74400e58c0 con 0x7f74501087d0 2024-09-10T21:03:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.564+0000 7f7454db4640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b5370 0x7f74380b7830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f744c00bbe0 tx=0x7f744c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:11.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.567+0000 7f7445ffb640 1 -- 172.21.15.33:0/216298847 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7440009de0 con 0x7f74501087d0 2024-09-10T21:03:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:11 smithi174 ceph-mon[35635]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:03:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:11 smithi174 ceph-mon[35635]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:11.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.793+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f745019a3e0 con 0x7f74501087d0 2024-09-10T21:03:11.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.794+0000 7f7445ffb640 1 -- 172.21.15.33:0/216298847 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f74400abfd0 con 0x7f74501087d0 2024-09-10T21:03:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:11.798 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b5370 msgr2=0x7f74380b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b5370 0x7f74380b7830 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f744c00bbe0 tx=0x7f744c00b390 comp rx=0 tx=0).stop 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74501087d0 msgr2=0x7f74501997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74501087d0 0x7f74501997f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f744000d950 tx=0x7f744000de10 comp rx=0 tx=0).stop 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 shutdown_connections 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74380b5370 0x7f74380b7830 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7450109150 0x7f745019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74501087d0 0x7f74501997f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 --2- 172.21.15.33:0/216298847 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7450102dc0 0x7f74501992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 >> 172.21.15.33:0/216298847 conn(0x7f74500fe830 msgr2=0x7f7450100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 shutdown_connections 2024-09-10T21:03:11.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:11.797+0000 7f7457018640 1 -- 172.21.15.33:0/216298847 wait complete. 2024-09-10T21:03:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:11 smithi104 ceph-mon[35325]: from='client.15027 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:03:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:11 smithi104 ceph-mon[35325]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:12.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:12 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/216298847' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T21:03:12.476 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T21:03:12.477 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:03:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:12 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/216298847' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T21:03:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:12.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:12 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/216298847' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T21:03:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 -- 172.21.15.33:0/2046800319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4181076e0 msgr2=0x7fc418107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2046800319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4181076e0 0x7fc418107b20 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc400009900 tx=0x7fc40002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 -- 172.21.15.33:0/2046800319 shutdown_connections 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2046800319 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc418108060 0x7fc41810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2046800319 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4181076e0 0x7fc418107b20 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2046800319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc418101cd0 0x7fc4181020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 -- 172.21.15.33:0/2046800319 >> 172.21.15.33:0/2046800319 conn(0x7fc4180fd700 msgr2=0x7fc4180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 -- 172.21.15.33:0/2046800319 shutdown_connections 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.228+0000 7fc41cf36640 1 -- 172.21.15.33:0/2046800319 wait complete. 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 Processor -- start 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 -- start start 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc418101cd0 0x7fc4181993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4181076e0 0x7fc4181998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418108060 0x7fc41819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc418111d20 con 0x7fc418108060 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc418111ba0 con 0x7fc418101cd0 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc41cf36640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc418111ea0 con 0x7fc4181076e0 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc416d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418108060 0x7fc41819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc416d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418108060 0x7fc41819ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56686/0 (socket says 172.21.15.33:56686) 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.229+0000 7fc416d9d640 1 -- 172.21.15.33:0/2566743852 learned_addr learned my addr 172.21.15.33:0/2566743852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc40ffff640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4181076e0 0x7fc4181998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc41659c640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc418101cd0 0x7fc4181993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc416d9d640 1 -- 172.21.15.33:0/2566743852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4181076e0 msgr2=0x7fc4181998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc416d9d640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4181076e0 0x7fc4181998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc416d9d640 1 -- 172.21.15.33:0/2566743852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc418101cd0 msgr2=0x7fc4181993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc416d9d640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc418101cd0 0x7fc4181993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc416d9d640 1 -- 172.21.15.33:0/2566743852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41819f460 con 0x7fc418108060 2024-09-10T21:03:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc41659c640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc418101cd0 0x7fc4181993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc416d9d640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418108060 0x7fc41819ec90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc41000b970 tx=0x7fc41000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc40f7fe640 1 -- 172.21.15.33:0/2566743852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc410019070 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc40f7fe640 1 -- 172.21.15.33:0/2566743852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc410014440 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41819f750 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc40f7fe640 1 -- 172.21.15.33:0/2566743852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4100075d0 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.230+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4181a5e10 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.231+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4181020b0 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.232+0000 7fc40f7fe640 1 -- 172.21.15.33:0/2566743852 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fc41000cb40 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.234+0000 7fc40f7fe640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4040b53f0 0x7fc4040b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.234+0000 7fc41659c640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4040b53f0 0x7fc4040b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.234+0000 7fc40f7fe640 1 -- 172.21.15.33:0/2566743852 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc4100e5590 con 0x7fc418108060 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.234+0000 7fc41659c640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4040b53f0 0x7fc4040b78b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4080058c0 tx=0x7fc408005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.238+0000 7fc40f7fe640 1 -- 172.21.15.33:0/2566743852 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc4100ac2b0 con 0x7fc418108060 2024-09-10T21:03:13.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:13 smithi033 ceph-mon[30860]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.462+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4181a6170 con 0x7fc418108060 2024-09-10T21:03:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.463+0000 7fc40f7fe640 1 -- 172.21.15.33:0/2566743852 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11973 (secure 0 0 0) 0x7fc4100abbb0 con 0x7fc418108060 2024-09-10T21:03:13.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:13.468 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"btime":"2024-09-10T21:02:12:865816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:02:12.865787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:03:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-09-10T21:03:13.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4040b53f0 msgr2=0x7fc4040b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4040b53f0 0x7fc4040b78b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc4080058c0 tx=0x7fc408005d30 comp rx=0 tx=0).stop 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418108060 msgr2=0x7fc41819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418108060 0x7fc41819ec90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc41000b970 tx=0x7fc41000be30 comp rx=0 tx=0).stop 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 shutdown_connections 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4040b53f0 0x7fc4040b78b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc418108060 0x7fc41819ec90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4181076e0 0x7fc4181998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 --2- 172.21.15.33:0/2566743852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc418101cd0 0x7fc4181993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 >> 172.21.15.33:0/2566743852 conn(0x7fc4180fd700 msgr2=0x7fc41806b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:13.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.465+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 shutdown_connections 2024-09-10T21:03:13.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:13.466+0000 7fc41cf36640 1 -- 172.21.15.33:0/2566743852 wait complete. 2024-09-10T21:03:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:13 smithi174 ceph-mon[35635]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:13 smithi104 ceph-mon[35325]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:14.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:03:14.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:14 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2566743852' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:14.436 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:14 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2566743852' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:14.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:14 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2566743852' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 -- 172.21.15.33:0/2446317776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10a920 msgr2=0x7fe00c10cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:14.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 --2- 172.21.15.33:0/2446317776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10a920 0x7fe00c10cd10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe000009900 tx=0x7fe00002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 -- 172.21.15.33:0/2446317776 shutdown_connections 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 --2- 172.21.15.33:0/2446317776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c10d250 0x7fe00c10f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 --2- 172.21.15.33:0/2446317776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10a920 0x7fe00c10cd10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 --2- 172.21.15.33:0/2446317776 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe00c101810 0x7fe00c101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 -- 172.21.15.33:0/2446317776 >> 172.21.15.33:0/2446317776 conn(0x7fe00c076f50 msgr2=0x7fe00c077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 -- 172.21.15.33:0/2446317776 shutdown_connections 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 -- 172.21.15.33:0/2446317776 wait complete. 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.937+0000 7fe011755640 1 Processor -- start 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe011755640 1 -- start start 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe011755640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c101810 0x7fe00c1027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe011755640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe00c10a920 0x7fe00c102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:14.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe011755640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10d250 0x7fe00c103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe011755640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c112d50 con 0x7fe00c10d250 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe011755640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c112bd0 con 0x7fe00c10a920 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe011755640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe00c112ed0 con 0x7fe00c101810 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10d250 0x7fe00c103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10d250 0x7fe00c103260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56714/0 (socket says 172.21.15.33:56714) 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 -- 172.21.15.33:0/4277438014 learned_addr learned my addr 172.21.15.33:0/4277438014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00a7fc640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe00c10a920 0x7fe00c102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00affd640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c101810 0x7fe00c1027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 -- 172.21.15.33:0/4277438014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c101810 msgr2=0x7fe00c1027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c101810 0x7fe00c1027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 -- 172.21.15.33:0/4277438014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe00c10a920 msgr2=0x7fe00c102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe00c10a920 0x7fe00c102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:14.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 -- 172.21.15.33:0/4277438014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe00c1a77e0 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00affd640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c101810 0x7fe00c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fe00b7fe640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10d250 0x7fe00c103260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe00400d750 tx=0x7fe00400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.938+0000 7fdff3fff640 1 -- 172.21.15.33:0/4277438014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004017070 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.939+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe00c1a7ad0 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.939+0000 7fdff3fff640 1 -- 172.21.15.33:0/4277438014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0040054d0 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.939+0000 7fdff3fff640 1 -- 172.21.15.33:0/4277438014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe004004c60 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.939+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe00c1a7fe0 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.940+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe00c06ae10 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.941+0000 7fdff3fff640 1 -- 172.21.15.33:0/4277438014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fe004004e00 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.942+0000 7fdff3fff640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdff40b5370 0x7fdff40b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.942+0000 7fdff3fff640 1 -- 172.21.15.33:0/4277438014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe0040e5b20 con 0x7fe00c10d250 2024-09-10T21:03:14.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.942+0000 7fe00affd640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdff40b5370 0x7fdff40b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:14.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.942+0000 7fe00affd640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdff40b5370 0x7fdff40b7830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005c40 tx=0x7fdffc0082f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:14.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:14.945+0000 7fdff3fff640 1 -- 172.21.15.33:0/4277438014 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe0040ac880 con 0x7fe00c10d250 2024-09-10T21:03:15.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.170+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe00c101bf0 con 0x7fe00c10d250 2024-09-10T21:03:15.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.171+0000 7fdff3fff640 1 -- 172.21.15.33:0/4277438014 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11973 (secure 0 0 0) 0x7fe00c101bf0 con 0x7fe00c10d250 2024-09-10T21:03:15.176 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:15.176 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"btime":"2024-09-10T21:02:12:865816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:02:12.865787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:03:15.177 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-09-10T21:03:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.173+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdff40b5370 msgr2=0x7fdff40b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:15.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.173+0000 7fe011755640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdff40b5370 0x7fdff40b7830 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fdffc005c40 tx=0x7fdffc0082f0 comp rx=0 tx=0).stop 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.173+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10d250 msgr2=0x7fe00c103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.173+0000 7fe011755640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10d250 0x7fe00c103260 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe00400d750 tx=0x7fe00400dc10 comp rx=0 tx=0).stop 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 shutdown_connections 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdff40b5370 0x7fdff40b7830 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe00c10d250 0x7fe00c103260 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe00c10a920 0x7fe00c102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 --2- 172.21.15.33:0/4277438014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe00c101810 0x7fe00c1027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 >> 172.21.15.33:0/4277438014 conn(0x7fe00c076f50 msgr2=0x7fe00c1005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 shutdown_connections 2024-09-10T21:03:15.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:15.174+0000 7fe011755640 1 -- 172.21.15.33:0/4277438014 wait complete. 2024-09-10T21:03:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:15 smithi033 ceph-mon[30860]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:15 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4277438014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:15.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:15 smithi174 ceph-mon[35635]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:15.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:15 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4277438014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:15 smithi104 ceph-mon[35325]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:15 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4277438014' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:15.871 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:03:16.130 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 -- 172.21.15.33:0/1085085345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 msgr2=0x7f713810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 --2- 172.21.15.33:0/1085085345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713810f950 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7130009ca0 tx=0x7f713002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 -- 172.21.15.33:0/1085085345 shutdown_connections 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 --2- 172.21.15.33:0/1085085345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713810f950 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 --2- 172.21.15.33:0/1085085345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f71381087d0 0x7f7138108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 --2- 172.21.15.33:0/1085085345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7138102dc0 0x7f71381031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 -- 172.21.15.33:0/1085085345 >> 172.21.15.33:0/1085085345 conn(0x7f71380fe830 msgr2=0x7f7138100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 -- 172.21.15.33:0/1085085345 shutdown_connections 2024-09-10T21:03:16.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.621+0000 7f713df9f640 1 -- 172.21.15.33:0/1085085345 wait complete. 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.622+0000 7f713df9f640 1 Processor -- start 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.622+0000 7f713df9f640 1 -- start start 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f713df9f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7138102dc0 0x7f71381993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f713df9f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71381087d0 0x7f7138199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f713df9f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f713df9f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7138112e50 con 0x7f7138109150 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f713df9f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7138112cd0 con 0x7f7138102dc0 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f713df9f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7138112fd0 con 0x7f71381087d0 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713819eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56730/0 (socket says 172.21.15.33:56730) 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 -- 172.21.15.33:0/1754870833 learned_addr learned my addr 172.21.15.33:0/1754870833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:16.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f71377fe640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7138102dc0 0x7f71381993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 -- 172.21.15.33:0/1754870833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71381087d0 msgr2=0x7f7138199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f712ffff640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71381087d0 0x7f7138199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71381087d0 0x7f7138199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 -- 172.21.15.33:0/1754870833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7138102dc0 msgr2=0x7f71381993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7138102dc0 0x7f71381993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 -- 172.21.15.33:0/1754870833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713819f470 con 0x7f7138109150 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f71377fe640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7138102dc0 0x7f71381993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f7137fff640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713819eca0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7130005cc0 tx=0x7f713002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f71357fa640 1 -- 172.21.15.33:0/1754870833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7130003cf0 con 0x7f7138109150 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f71357fa640 1 -- 172.21.15.33:0/1754870833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71300044f0 con 0x7f7138109150 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f71357fa640 1 -- 172.21.15.33:0/1754870833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f713002fd10 con 0x7f7138109150 2024-09-10T21:03:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.623+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713819f700 con 0x7f7138109150 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.624+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71381a5ea0 con 0x7f7138109150 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.625+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7138103220 con 0x7f7138109150 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.626+0000 7f71357fa640 1 -- 172.21.15.33:0/1754870833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f7130004040 con 0x7f7138109150 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.627+0000 7f71357fa640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71240b53f0 0x7f71240b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.627+0000 7f71377fe640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71240b53f0 0x7f71240b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.627+0000 7f71357fa640 1 -- 172.21.15.33:0/1754870833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7130107510 con 0x7f7138109150 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.627+0000 7f71377fe640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71240b53f0 0x7f71240b78b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7128009930 tx=0x7f7128008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:16.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.630+0000 7f71357fa640 1 -- 172.21.15.33:0/1754870833 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f71300ce270 con 0x7f7138109150 2024-09-10T21:03:16.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.818+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f713802e5c0 con 0x7f7138109150 2024-09-10T21:03:16.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.820+0000 7f71357fa640 1 -- 172.21.15.33:0/1754870833 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22763 (secure 0 0 0) 0x7f71300cdb70 con 0x7f7138109150 2024-09-10T21:03:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:16.825 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:41.855317+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:03:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71240b53f0 msgr2=0x7f71240b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71240b53f0 0x7f71240b78b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7128009930 tx=0x7f7128008040 comp rx=0 tx=0).stop 2024-09-10T21:03:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 msgr2=0x7f713819eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713819eca0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7130005cc0 tx=0x7f713002f470 comp rx=0 tx=0).stop 2024-09-10T21:03:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 shutdown_connections 2024-09-10T21:03:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f71240b53f0 0x7f71240b78b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7138109150 0x7f713819eca0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71381087d0 0x7f7138199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 --2- 172.21.15.33:0/1754870833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7138102dc0 0x7f71381993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 >> 172.21.15.33:0/1754870833 conn(0x7f71380fe830 msgr2=0x7f7138100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 shutdown_connections 2024-09-10T21:03:16.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:16.822+0000 7f713df9f640 1 -- 172.21.15.33:0/1754870833 wait complete. 2024-09-10T21:03:17.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:17 smithi033 ceph-mon[30860]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:17.507 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:17 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1754870833' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:17.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:03:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:17 smithi174 ceph-mon[35635]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:17.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:17 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1754870833' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:17.799 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:17 smithi104 ceph-mon[35325]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:17 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1754870833' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.297+0000 7f595795c640 1 -- 172.21.15.33:0/974541952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 msgr2=0x7f59501020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.297+0000 7f595795c640 1 --2- 172.21.15.33:0/974541952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f59501020b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f594c009ca0 tx=0x7f594c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.297+0000 7f595795c640 1 -- 172.21.15.33:0/974541952 shutdown_connections 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.297+0000 7f595795c640 1 --2- 172.21.15.33:0/974541952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5950108060 0x7f595010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.297+0000 7f595795c640 1 --2- 172.21.15.33:0/974541952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59501076e0 0x7f5950107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.297+0000 7f595795c640 1 --2- 172.21.15.33:0/974541952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f59501020b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.297+0000 7f595795c640 1 -- 172.21.15.33:0/974541952 >> 172.21.15.33:0/974541952 conn(0x7f59500fd720 msgr2=0x7f59500ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:18.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 -- 172.21.15.33:0/974541952 shutdown_connections 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 -- 172.21.15.33:0/974541952 wait complete. 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 Processor -- start 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 -- start start 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f5950199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59501076e0 0x7f5950199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5950108060 0x7f595019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5950111d20 con 0x7f5950101cd0 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5950111ba0 con 0x7f5950108060 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.298+0000 7f595795c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5950111ea0 con 0x7f59501076e0 2024-09-10T21:03:18.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f5950199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f5950199350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56758/0 (socket says 172.21.15.33:56758) 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 -- 172.21.15.33:0/118244954 learned_addr learned my addr 172.21.15.33:0/118244954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f5955ef9640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5950108060 0x7f595019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f5954ef7640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59501076e0 0x7f5950199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 -- 172.21.15.33:0/118244954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59501076e0 msgr2=0x7f5950199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59501076e0 0x7f5950199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 -- 172.21.15.33:0/118244954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5950108060 msgr2=0x7f595019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5950108060 0x7f595019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 -- 172.21.15.33:0/118244954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595019f400 con 0x7f5950101cd0 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f5955ef9640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5950108060 0x7f595019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f5954ef7640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59501076e0 0x7f5950199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59556f8640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f5950199350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f594c02ade0 tx=0x7f594c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:18.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59467fc640 1 -- 172.21.15.33:0/118244954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c003d60 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59467fc640 1 -- 172.21.15.33:0/118244954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f594c0044f0 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595019f690 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f59467fc640 1 -- 172.21.15.33:0/118244954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c02fd10 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.299+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59501a5db0 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.300+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59501020b0 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.302+0000 7f59467fc640 1 -- 172.21.15.33:0/118244954 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f594c004040 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.302+0000 7f59467fc640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f593c0b5370 0x7f593c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.302+0000 7f5954ef7640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f593c0b5370 0x7f593c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.302+0000 7f59467fc640 1 -- 172.21.15.33:0/118244954 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f594c107480 con 0x7f5950101cd0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.303+0000 7f5954ef7640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f593c0b5370 0x7f593c0b7830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5940006fd0 tx=0x7f5940008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:18.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.305+0000 7f59467fc640 1 -- 172.21.15.33:0/118244954 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f594c0ce1e0 con 0x7f5950101cd0 2024-09-10T21:03:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.528+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f595019a480 con 0x7f5950101cd0 2024-09-10T21:03:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.529+0000 7f59467fc640 1 -- 172.21.15.33:0/118244954 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11973 (secure 0 0 0) 0x7f594c0cdae0 con 0x7f5950101cd0 2024-09-10T21:03:18.534 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:18.534 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"btime":"2024-09-10T21:02:12:865816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:02:12.865787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:03:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-09-10T21:03:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f593c0b5370 msgr2=0x7f593c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f593c0b5370 0x7f593c0b7830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5940006fd0 tx=0x7f5940008040 comp rx=0 tx=0).stop 2024-09-10T21:03:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 msgr2=0x7f5950199350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f5950199350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f594c02ade0 tx=0x7f594c003970 comp rx=0 tx=0).stop 2024-09-10T21:03:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 shutdown_connections 2024-09-10T21:03:18.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f593c0b5370 0x7f593c0b7830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5950108060 0x7f595019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59501076e0 0x7f5950199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 --2- 172.21.15.33:0/118244954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5950101cd0 0x7f5950199350 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 >> 172.21.15.33:0/118244954 conn(0x7f59500fd720 msgr2=0x7f595006b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 shutdown_connections 2024-09-10T21:03:18.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:18.531+0000 7f595795c640 1 -- 172.21.15.33:0/118244954 wait complete. 2024-09-10T21:03:19.206 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T21:02:01.445404+0000', 'modified': '2024-09-10T21:02:12.865787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24547, 'mds_2': 24539, 'mds_3': 24532, 'mds_4': 24526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14697': {'gid': 14697, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6839/295723844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 295723844}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 295723844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14709': {'gid': 14709, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6841/1476321443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6840', 'nonce': 1476321443}, {'type': 'v1', 'addr': '172.21.15.33:6841', 'nonce': 1476321443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/2049405106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 2049405106}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 2049405106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/34665907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 34665907}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 34665907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/1051931570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 1051931570}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 1051931570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6835/982304250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 982304250}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 982304250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/1592763439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1592763439}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1592763439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.104:6837/2894143081', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2894143081}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2894143081}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/3620064873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3620064873}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3620064873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.104:6839/2034440949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 2034440949}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 2034440949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24566, 'qdb_cluster': [24566, 24547, 24532, 24539, 24526]} 2024-09-10T21:03:19.207 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T21:03:19.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T21:03:19.466 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:19.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:19 smithi033 ceph-mon[30860]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:19.587 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:19 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/118244954' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:19.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:19 smithi174 ceph-mon[35635]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:19.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:19 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/118244954' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:19 smithi104 ceph-mon[35325]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:19 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/118244954' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:19.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 -- 172.21.15.33:0/2490250813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 msgr2=0x7fdcfc101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:19.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 --2- 172.21.15.33:0/2490250813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc101c10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8009900 tx=0x7fdcf802a480 comp rx=0 tx=0).stop 2024-09-10T21:03:19.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 -- 172.21.15.33:0/2490250813 shutdown_connections 2024-09-10T21:03:19.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 --2- 172.21.15.33:0/2490250813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcfc10f3e0 0x7fdcfc1117d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 --2- 172.21.15.33:0/2490250813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcfc102150 0x7fdcfc10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 --2- 172.21.15.33:0/2490250813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc101c10 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 -- 172.21.15.33:0/2490250813 >> 172.21.15.33:0/2490250813 conn(0x7fdcfc0fd700 msgr2=0x7fdcfc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 -- 172.21.15.33:0/2490250813 shutdown_connections 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 -- 172.21.15.33:0/2490250813 wait complete. 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 Processor -- start 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.966+0000 7fdd0367c640 1 -- start start 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd0367c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc19b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd0367c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcfc102150 0x7fdcfc19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd0367c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcfc10f3e0 0x7fdcfc1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd0367c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc114fa0 con 0x7fdcfc101830 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd0367c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc114e20 con 0x7fdcfc10f3e0 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd0367c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcfc115120 con 0x7fdcfc102150 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc19b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc19b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34138/0 (socket says 172.21.15.33:34138) 2024-09-10T21:03:19.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 -- 172.21.15.33:0/3536212134 learned_addr learned my addr 172.21.15.33:0/3536212134 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd00c17640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcfc102150 0x7fdcfc19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01c19640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcfc10f3e0 0x7fdcfc1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 -- 172.21.15.33:0/3536212134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcfc102150 msgr2=0x7fdcfc19ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcfc102150 0x7fdcfc19ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 -- 172.21.15.33:0/3536212134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcfc10f3e0 msgr2=0x7fdcfc1a0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcfc10f3e0 0x7fdcfc1a0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 -- 172.21.15.33:0/3536212134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcfc1a15b0 con 0x7fdcfc101830 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdd01418640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc19b500 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005940 tx=0x7fdcf80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.967+0000 7fdcf27fc640 1 -- 172.21.15.33:0/3536212134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf803f030 con 0x7fdcfc101830 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.968+0000 7fdcf27fc640 1 -- 172.21.15.33:0/3536212134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf8003cb0 con 0x7fdcfc101830 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.968+0000 7fdcf27fc640 1 -- 172.21.15.33:0/3536212134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8035070 con 0x7fdcfc101830 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.968+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcfc1a1840 con 0x7fdcfc101830 2024-09-10T21:03:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.968+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcfc1a7f60 con 0x7fdcfc101830 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.970+0000 7fdcf27fc640 1 -- 172.21.15.33:0/3536212134 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fdcf8003800 con 0x7fdcfc101830 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.970+0000 7fdd05a51740 1 Processor -- start 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.970+0000 7fdd05a51740 1 -- start start 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597dd30 0x564ff597e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff598a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff598c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd05a51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ff58f5c10 con 0x564ff597e650 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd05a51740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ff58e5040 con 0x564ff598a5e0 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd05a51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ff58e8e50 con 0x564ff597dd30 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff598a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff598a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34140/0 (socket says 172.21.15.33:34140) 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01c19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597dd30 0x564ff597e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 -- 172.21.15.33:0/1187836334 learned_addr learned my addr 172.21.15.33:0/1187836334 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd00c17640 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff598c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 -- 172.21.15.33:0/1187836334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597dd30 msgr2=0x564ff597e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597dd30 0x564ff597e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 -- 172.21.15.33:0/1187836334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 msgr2=0x564ff598c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff598c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 -- 172.21.15.33:0/1187836334 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff5643540 con 0x564ff597e650 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdcf27fc640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdce80b5370 0x7fdce80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01c19640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdce80b5370 0x7fdce80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.971+0000 7fdd01418640 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff598a0a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdcf810cbf0 tx=0x7fdcf810cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62385080b57ab3a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdcf27fc640 1 -- 172.21.15.33:0/3536212134 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdcf8106800 con 0x7fdcfc101830 2024-09-10T21:03:19.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdce77fe640 1 -- 172.21.15.33:0/1187836334 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8109e60 con 0x564ff597e650 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdce77fe640 1 -- 172.21.15.33:0/1187836334 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf80cc970 con 0x564ff597e650 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdce77fe640 1 -- 172.21.15.33:0/1187836334 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80c8d00 con 0x564ff597e650 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd05a51740 1 -- 172.21.15.33:0/1187836334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 msgr2=0x564ff598a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd01c19640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdce80b5370 0x7fdce80b7830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4005fd0 tx=0x7fdcf4009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd05a51740 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff598a0a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdcf810cbf0 tx=0x7fdcf810cce0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd05a51740 1 -- 172.21.15.33:0/1187836334 shutdown_connections 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd05a51740 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff598c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd05a51740 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff598a0a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd05a51740 1 --2- 172.21.15.33:0/1187836334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597dd30 0x564ff597e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.972+0000 7fdd05a51740 1 -- 172.21.15.33:0/1187836334 >> 172.21.15.33:0/1187836334 conn(0x564ff597bb60 msgr2=0x564ff597cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 -- 172.21.15.33:0/1187836334 shutdown_connections 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 -- 172.21.15.33:0/1187836334 wait complete. 2024-09-10T21:03:19.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 -- start start 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 Processor -- start 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff59875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597e650 0x564ff5987b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff5988040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ff57465f0 con 0x564ff597dd30 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ff5880010 con 0x564ff598a5e0 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ff598d050 con 0x564ff597e650 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd05a51740 10 client.0 _ll_register_callbacks cb 0x564ff58df340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff59875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff59875c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34144/0 (socket says 172.21.15.33:34144) 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.973+0000 7fdd01418640 1 -- 172.21.15.33:0/2863465804 learned_addr learned my addr 172.21.15.33:0/2863465804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01c19640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff5988040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd00c17640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597e650 0x564ff5987b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01418640 1 -- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597e650 msgr2=0x564ff5987b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01418640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597e650 0x564ff5987b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01418640 1 -- 172.21.15.33:0/2863465804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 msgr2=0x564ff5988040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01418640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff5988040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01418640 1 -- 172.21.15.33:0/2863465804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff58ada00 con 0x564ff597dd30 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01c19640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff5988040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd00c17640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597e650 0x564ff5987b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd01418640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff59875c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8104230 tx=0x7fdcf802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d220878162cde57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdce4ff9640 1 -- 172.21.15.33:0/2863465804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8109e60 con 0x564ff597dd30 2024-09-10T21:03:19.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdce4ff9640 1 -- 172.21.15.33:0/2863465804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdcf80b42d0 con 0x564ff597dd30 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcf80cc970 con 0x564ff597dd30 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdce4ff9640 1 -- 172.21.15.33:0/2863465804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcf8049070 con 0x564ff597dd30 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd05a51740 10 client.15069 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.974+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ff5998bb0 con 0x564ff597dd30 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdce4ff9640 1 -- 172.21.15.33:0/2863465804 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf80cc970 con 0x564ff597dd30 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdce4ff9640 1 -- 172.21.15.33:0/2863465804 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdcf80cd0c0 con 0x564ff597dd30 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdd05a51740 10 client.15069 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdd05a51740 10 client.15069 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T21:03:19.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdd05a51740 20 client.15069 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:19.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdd05a51740 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff599afb0 0x564ff599c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:19.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdd05a51740 4 client.15069 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:19.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ff58f7b60 con 0x564ff599afb0 2024-09-10T21:03:19.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.975+0000 7fdd00c17640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff599afb0 0x564ff599c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:19.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.976+0000 7fdd00c17640 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff599afb0 0x564ff599c420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f47d96d57080a723 server_cookie=f5d4d8647a45c8d9 in_seq=0 out_seq=0 2024-09-10T21:03:19.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.976+0000 7fdce4ff9640 10 client.15069 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:19.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdce4ff9640 1 -- 172.21.15.33:0/2863465804 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ff58f7b60 con 0x564ff599afb0 2024-09-10T21:03:19.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdce4ff9640 10 client.15069 handle_command_reply: tid=0 2024-09-10T21:03:19.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 client.15069 shutdown 2024-09-10T21:03:19.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 msgr2=0x564ff59875c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:19.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff59875c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8104230 tx=0x7fdcf802a480 comp rx=0 tx=0).stop 2024-09-10T21:03:19.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 shutdown_connections 2024-09-10T21:03:19.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff599afb0 0x564ff599c420 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff598a5e0 0x564ff5988040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff597e650 0x564ff5987b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 --2- 172.21.15.33:0/2863465804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff59875c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:19.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 >> 172.21.15.33:0/2863465804 conn(0x564ff597bb60 msgr2=0x564ff5981700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:19.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdce4ff9640 0 client.15069 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:19.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 shutdown_connections 2024-09-10T21:03:19.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 1 -- 172.21.15.33:0/2863465804 wait complete. 2024-09-10T21:03:19.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.977+0000 7fdd05a51740 20 client.15069 trim_cache size 0 max 16384 2024-09-10T21:03:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.995+0000 7fdd05a51740 1 Processor -- start 2024-09-10T21:03:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.995+0000 7fdd05a51740 1 -- start start 2024-09-10T21:03:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff5986ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597e650 0x564ff5981c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:20.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff5982840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd05a51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ff599a760 con 0x564ff597dd30 2024-09-10T21:03:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd05a51740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ff599ae30 con 0x564ff597e650 2024-09-10T21:03:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd05a51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ff599b500 con 0x564ff598a5e0 2024-09-10T21:03:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff5986ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:20.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff5986ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34158/0 (socket says 172.21.15.33:34158) 2024-09-10T21:03:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 -- 172.21.15.33:0/4015422145 learned_addr learned my addr 172.21.15.33:0/4015422145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd00c17640 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597e650 0x564ff5981c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01c19640 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff5982840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 -- 172.21.15.33:0/4015422145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 msgr2=0x564ff5982840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff5982840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 -- 172.21.15.33:0/4015422145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597e650 msgr2=0x564ff5981c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597e650 0x564ff5981c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 -- 172.21.15.33:0/4015422145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ff59774f0 con 0x564ff597dd30 2024-09-10T21:03:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01c19640 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff5982840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdd01418640 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff5986ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80ccb60 tx=0x7fdcf8032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd86c14f5de4be6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdce6ffd640 1 -- 172.21.15.33:0/4015422145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8005d50 con 0x564ff597dd30 2024-09-10T21:03:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdce6ffd640 1 -- 172.21.15.33:0/4015422145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf810ca70 con 0x564ff597dd30 2024-09-10T21:03:20.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.996+0000 7fdce6ffd640 1 -- 172.21.15.33:0/4015422145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf80cc5d0 con 0x564ff597dd30 2024-09-10T21:03:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 -- 172.21.15.33:0/4015422145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 msgr2=0x564ff5986ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff5986ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80ccb60 tx=0x7fdcf8032f60 comp rx=0 tx=0).stop 2024-09-10T21:03:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 -- 172.21.15.33:0/4015422145 shutdown_connections 2024-09-10T21:03:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff5982840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597e650 0x564ff5981c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 --2- 172.21.15.33:0/4015422145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597dd30 0x564ff5986ab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 -- 172.21.15.33:0/4015422145 >> 172.21.15.33:0/4015422145 conn(0x564ff597bb60 msgr2=0x564ff5999f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 -- 172.21.15.33:0/4015422145 shutdown_connections 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 -- 172.21.15.33:0/4015422145 wait complete. 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.997+0000 7fdd05a51740 1 -- start start 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 1 Processor -- start 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597dd30 0x564ff59a4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff599e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff599e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ff599c2a0 con 0x564ff597e650 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ff599bbd0 con 0x564ff597dd30 2024-09-10T21:03:20.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ff599c970 con 0x564ff598a5e0 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd05a51740 10 client.0 _ll_register_callbacks cb 0x7fdcec0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff599e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff599e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34164/0 (socket says 172.21.15.33:34164) 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 -- 172.21.15.33:0/714165403 learned_addr learned my addr 172.21.15.33:0/714165403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd01418640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597dd30 0x564ff59a4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd01c19640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff599e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 -- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 msgr2=0x564ff599e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff599e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 -- 172.21.15.33:0/714165403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597dd30 msgr2=0x564ff59a4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597dd30 0x564ff59a4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd00c17640 1 -- 172.21.15.33:0/714165403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ff599ee70 con 0x564ff597e650 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.998+0000 7fdd01c19640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff599e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:20.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdd00c17640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff599e220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x564ff599cbc0 tx=0x7fdcec016800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ca3d6947e42f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdce4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdce4ff9640 1 -- 172.21.15.33:0/714165403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec0155d0 con 0x564ff597e650 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcf810ca70 con 0x564ff597e650 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdce4ff9640 1 -- 172.21.15.33:0/714165403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdcec015770 con 0x564ff597e650 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdce4ff9640 1 -- 172.21.15.33:0/714165403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdcec015970 con 0x564ff597e650 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdd05a51740 10 client.15075 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ff5a078f0 con 0x564ff597e650 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdce4ff9640 1 -- 172.21.15.33:0/714165403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf810ca70 con 0x564ff597e650 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdce4ff9640 1 -- 172.21.15.33:0/714165403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fdcec026c30 con 0x564ff597e650 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:19.999+0000 7fdd05a51740 10 client.15075 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd05a51740 10 client.15075 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd05a51740 20 client.15075 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:20.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd05a51740 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff5a0c380 0x564ff5a0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd05a51740 4 client.15075 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "status"}) -- 0x564ff58f3610 con 0x564ff5a0c380 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd01418640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff5a0c380 0x564ff5a0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd01418640 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff5a0c380 0x564ff5a0b090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fa6fc782976b87e4 server_cookie=50590e6c4c2109e1 in_seq=0 out_seq=0 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdce4ff9640 10 client.15075 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdce4ff9640 1 -- 172.21.15.33:0/714165403 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x564ff58f3610 con 0x564ff5a0c380 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdce4ff9640 10 client.15075 handle_command_reply: tid=0 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.000+0000 7fdd05a51740 1 client.15075 shutdown 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 msgr2=0x564ff599e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff599e220 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x564ff599cbc0 tx=0x7fdcec016800 comp rx=0 tx=0).stop 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 shutdown_connections 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x564ff5a0c380 0x564ff5a0b090 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ff598a5e0 0x564ff599e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ff597e650 0x564ff599e220 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdce4ff9640 0 client.15075 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 --2- 172.21.15.33:0/714165403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ff597dd30 0x564ff59a4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 >> 172.21.15.33:0/714165403 conn(0x564ff597bb60 msgr2=0x564ff5982840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 shutdown_connections 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 1 -- 172.21.15.33:0/714165403 wait complete. 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.001+0000 7fdd05a51740 20 client.15075 trim_cache size 0 max 16384 2024-09-10T21:03:20.019 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:20.020 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:20.020 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 4, 2024-09-10T21:03:20.020 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24526, 2024-09-10T21:03:20.020 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:20.020 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:20.020 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:20.020 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 68.143521617000005, 2024-09-10T21:03:20.021 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:20.021 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:20.021 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:20.021 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 105.94707163 2024-09-10T21:03:20.021 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdce80b5370 msgr2=0x7fdce80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdce80b5370 0x7fdce80b7830 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdcf4005fd0 tx=0x7fdcf4009600 comp rx=0 tx=0).stop 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 msgr2=0x7fdcfc19b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc19b500 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdcf8005940 tx=0x7fdcf80042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 shutdown_connections 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdce80b5370 0x7fdce80b7830 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdcfc10f3e0 0x7fdcfc1a0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcfc102150 0x7fdcfc19ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 --2- 172.21.15.33:0/3536212134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcfc101830 0x7fdcfc19b500 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:20.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 >> 172.21.15.33:0/3536212134 conn(0x7fdcfc0fd700 msgr2=0x7fdcfc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:20.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 shutdown_connections 2024-09-10T21:03:20.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:20.002+0000 7fdd0367c640 1 -- 172.21.15.33:0/3536212134 wait complete. 2024-09-10T21:03:20.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T21:03:20.972 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:21.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 -- 172.21.15.33:0/2560327994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c10d140 msgr2=0x7fb17c10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 --2- 172.21.15.33:0/2560327994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c10d140 0x7fb17c10f5a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb174009ca0 tx=0x7fb17402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 -- 172.21.15.33:0/2560327994 shutdown_connections 2024-09-10T21:03:21.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 --2- 172.21.15.33:0/2560327994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c10d140 0x7fb17c10f5a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 --2- 172.21.15.33:0/2560327994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb17c104600 0x7fb17c10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 --2- 172.21.15.33:0/2560327994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c103ce0 0x7fb17c1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 -- 172.21.15.33:0/2560327994 >> 172.21.15.33:0/2560327994 conn(0x7fb17c0fd700 msgr2=0x7fb17c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 -- 172.21.15.33:0/2560327994 shutdown_connections 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.399+0000 7fb183524640 1 -- 172.21.15.33:0/2560327994 wait complete. 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 Processor -- start 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 -- start start 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c103ce0 0x7fb17c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c104600 0x7fb17c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb17c10d140 0x7fb17c19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c112db0 con 0x7fb17c104600 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c112c30 con 0x7fb17c10d140 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb183524640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb17c112f30 con 0x7fb17c103ce0 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb180abf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c104600 0x7fb17c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb180abf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c104600 0x7fb17c1997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34186/0 (socket says 172.21.15.33:34186) 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb180abf640 1 -- 172.21.15.33:0/798823902 learned_addr learned my addr 172.21.15.33:0/798823902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.400+0000 7fb1812c0640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c103ce0 0x7fb17c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb181ac1640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb17c10d140 0x7fb17c19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb180abf640 1 -- 172.21.15.33:0/798823902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c103ce0 msgr2=0x7fb17c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb180abf640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c103ce0 0x7fb17c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb180abf640 1 -- 172.21.15.33:0/798823902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb17c10d140 msgr2=0x7fb17c19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb180abf640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb17c10d140 0x7fb17c19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb180abf640 1 -- 172.21.15.33:0/798823902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb17c19f330 con 0x7fb17c104600 2024-09-10T21:03:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb180abf640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c104600 0x7fb17c1997c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00d740 tx=0x7fb16c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb1727fc640 1 -- 172.21.15.33:0/798823902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c017020 con 0x7fb17c104600 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb1727fc640 1 -- 172.21.15.33:0/798823902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb16c00de50 con 0x7fb17c104600 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb17c19f620 con 0x7fb17c104600 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb1727fc640 1 -- 172.21.15.33:0/798823902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c00f620 con 0x7fb17c104600 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.401+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb17c1a5eb0 con 0x7fb17c104600 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 Processor -- start 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1727fc640 1 -- 172.21.15.33:0/798823902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb16c010460 con 0x7fb17c104600 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 -- start start 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570e0650 0x55d7570f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570f09f0 0x55d7570f2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d757057c10 con 0x55d7570dfd30 2024-09-10T21:03:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d757047040 con 0x55d7570e0650 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb1858f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d75704ae50 con 0x55d7570f09f0 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb181ac1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb181ac1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34188/0 (socket says 172.21.15.33:34188) 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.404+0000 7fb181ac1640 1 -- 172.21.15.33:0/3734878398 learned_addr learned my addr 172.21.15.33:0/3734878398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1812c0640 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570e0650 0x55d7570f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb180abf640 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570f09f0 0x55d7570f2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb181ac1640 1 -- 172.21.15.33:0/3734878398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570f09f0 msgr2=0x55d7570f2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb181ac1640 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570f09f0 0x55d7570f2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb181ac1640 1 -- 172.21.15.33:0/3734878398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570e0650 msgr2=0x55d7570f04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb181ac1640 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570e0650 0x55d7570f04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb181ac1640 1 -- 172.21.15.33:0/3734878398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d756da5540 con 0x55d7570dfd30 2024-09-10T21:03:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1812c0640 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570e0650 0x55d7570f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1727fc640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1680b53f0 0x7fb1680b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb181ac1640 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e0110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb174005f70 tx=0x7fb174009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9d386a0d539f00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1812c0640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1680b53f0 0x7fb1680b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1677fe640 1 -- 172.21.15.33:0/3734878398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174031cf0 con 0x55d7570dfd30 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1727fc640 1 -- 172.21.15.33:0/798823902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb16c0e5860 con 0x7fb17c104600 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1677fe640 1 -- 172.21.15.33:0/3734878398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb174040070 con 0x55d7570dfd30 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.405+0000 7fb1677fe640 1 -- 172.21.15.33:0/3734878398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174004040 con 0x55d7570dfd30 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 -- 172.21.15.33:0/3734878398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 msgr2=0x55d7570e0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1812c0640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1680b53f0 0x7fb1680b78b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55d7570e2eb0 tx=0x7fb178005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e0110 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb174005f70 tx=0x7fb174009360 comp rx=0 tx=0).stop 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 -- 172.21.15.33:0/3734878398 shutdown_connections 2024-09-10T21:03:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570f09f0 0x55d7570f2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570e0650 0x55d7570f04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 --2- 172.21.15.33:0/3734878398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e0110 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 -- 172.21.15.33:0/3734878398 >> 172.21.15.33:0/3734878398 conn(0x55d7570ddb60 msgr2=0x55d7570ded00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 -- 172.21.15.33:0/3734878398 shutdown_connections 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 -- 172.21.15.33:0/3734878398 wait complete. 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.406+0000 7fb1858f9740 1 -- start start 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 1 Processor -- start 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570dfd30 0x55d7570eda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570edf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570f09f0 0x55d7570ee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d756ea85f0 con 0x55d7570f09f0 2024-09-10T21:03:21.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d756fe2010 con 0x55d7570dfd30 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d7570f3640 con 0x55d7570e0650 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1858f9740 10 client.0 _ll_register_callbacks cb 0x55d757041340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570f09f0 0x55d7570ee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570f09f0 0x55d7570ee490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34192/0 (socket says 172.21.15.33:34192) 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 -- 172.21.15.33:0/2127774663 learned_addr learned my addr 172.21.15.33:0/2127774663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb1812c0640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570dfd30 0x55d7570eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb180abf640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570edf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 -- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 msgr2=0x55d7570edf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570edf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 -- 172.21.15.33:0/2127774663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570dfd30 msgr2=0x55d7570eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570dfd30 0x55d7570eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb181ac1640 1 -- 172.21.15.33:0/2127774663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d75700fa00 con 0x55d7570f09f0 2024-09-10T21:03:21.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.407+0000 7fb180abf640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb181ac1640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570f09f0 0x55d7570ee490 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb17402ade0 tx=0x7fb174003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23916365ade518a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb164ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb164ff9640 1 -- 172.21.15.33:0/2127774663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17402f860 con 0x55d7570f09f0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb164ff9640 1 -- 172.21.15.33:0/2127774663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb17402fa00 con 0x55d7570f09f0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb174040070 con 0x55d7570f09f0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb164ff9640 1 -- 172.21.15.33:0/2127774663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb17402fc00 con 0x55d7570f09f0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 10 client.15087 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d7570ff260 con 0x55d7570f09f0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb164ff9640 1 -- 172.21.15.33:0/2127774663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb174040070 con 0x55d7570f09f0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb164ff9640 1 -- 172.21.15.33:0/2127774663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb174030ca0 con 0x55d7570f09f0 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 10 client.15087 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 10 client.15087 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:03:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 20 client.15087 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d757101420 0x55d757102890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 4 client.15087 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.408+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d757059b60 con 0x55d757101420 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.409+0000 7fb1812c0640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d757101420 0x55d757102890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.409+0000 7fb164ff9640 10 client.15087 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.409+0000 7fb1812c0640 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d757101420 0x55d757102890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e5770db2c0700a9b server_cookie=bb8e6d69192b6a93 in_seq=0 out_seq=0 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb164ff9640 1 -- 172.21.15.33:0/2127774663 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d757059b60 con 0x55d757101420 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb164ff9640 10 client.15087 handle_command_reply: tid=0 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 client.15087 shutdown 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570f09f0 msgr2=0x55d7570ee490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570f09f0 0x55d7570ee490 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb17402ade0 tx=0x7fb174003b90 comp rx=0 tx=0).stop 2024-09-10T21:03:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 shutdown_connections 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570f09f0 0x55d7570ee490 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d757101420 0x55d757102890 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570edf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2127774663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570dfd30 0x55d7570eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb164ff9640 0 client.15087 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 >> 172.21.15.33:0/2127774663 conn(0x55d7570ddb60 msgr2=0x55d7570dcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 shutdown_connections 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 1 -- 172.21.15.33:0/2127774663 wait complete. 2024-09-10T21:03:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.410+0000 7fb1858f9740 20 client.15087 trim_cache size 0 max 16384 2024-09-10T21:03:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 Processor -- start 2024-09-10T21:03:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 -- start start 2024-09-10T21:03:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570dfd30 0x55d7570dd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570e0650 0x55d7570eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d757100bd0 con 0x55d7570e0650 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d7571012a0 con 0x55d7570f09f0 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1858f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d757101970 con 0x55d7570dfd30 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570e0650 0x55d7570eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570e0650 0x55d7570eb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34202/0 (socket says 172.21.15.33:34202) 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 -- 172.21.15.33:0/1133285703 learned_addr learned my addr 172.21.15.33:0/1133285703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb181ac1640 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb1812c0640 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570dfd30 0x55d7570dd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 -- 172.21.15.33:0/1133285703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570dfd30 msgr2=0x55d7570dd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570dfd30 0x55d7570dd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 -- 172.21.15.33:0/1133285703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 msgr2=0x55d7570f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 -- 172.21.15.33:0/1133285703 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7570f3150 con 0x55d7570e0650 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.429+0000 7fb180abf640 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570e0650 0x55d7570eb480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0a8490 tx=0x7fb16c010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508c458f102938b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb166ffd640 1 -- 172.21.15.33:0/1133285703 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c0abec0 con 0x55d7570e0650 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb166ffd640 1 -- 172.21.15.33:0/1133285703 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb16c0b09a0 con 0x55d7570e0650 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb166ffd640 1 -- 172.21.15.33:0/1133285703 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16c093350 con 0x55d7570e0650 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 -- 172.21.15.33:0/1133285703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570e0650 msgr2=0x55d7570eb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570e0650 0x55d7570eb480 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0a8490 tx=0x7fb16c010c70 comp rx=0 tx=0).stop 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 -- 172.21.15.33:0/1133285703 shutdown_connections 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570e0650 0x55d7570eb480 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 --2- 172.21.15.33:0/1133285703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570dfd30 0x55d7570dd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 -- 172.21.15.33:0/1133285703 >> 172.21.15.33:0/1133285703 conn(0x55d7570ddb60 msgr2=0x55d7570f0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:21.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 -- 172.21.15.33:0/1133285703 shutdown_connections 2024-09-10T21:03:21.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.430+0000 7fb1858f9740 1 -- 172.21.15.33:0/1133285703 wait complete. 2024-09-10T21:03:21.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 -- start start 2024-09-10T21:03:21.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 Processor -- start 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d757102710 con 0x55d7570dfd30 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d757102040 con 0x55d7570f09f0 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d757102de0 con 0x55d7570e0650 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1858f9740 10 client.0 _ll_register_callbacks cb 0x7fb174009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1812c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1812c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34216/0 (socket says 172.21.15.33:34216) 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb1812c0640 1 -- 172.21.15.33:0/2669599394 learned_addr learned my addr 172.21.15.33:0/2669599394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:21.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.431+0000 7fb180abf640 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb181ac1640 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1812c0640 1 -- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 msgr2=0x55d7570e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1812c0640 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1812c0640 1 -- 172.21.15.33:0/2669599394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 msgr2=0x55d7570e12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1812c0640 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570e12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1812c0640 1 -- 172.21.15.33:0/2669599394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7570e1930 con 0x55d7570dfd30 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1812c0640 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e2240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb178012cb0 tx=0x7fb1780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3fbc85dde76b1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb164ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb164ff9640 1 -- 172.21.15.33:0/2669599394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17801db50 con 0x55d7570dfd30 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb164ff9640 1 -- 172.21.15.33:0/2669599394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb17801dcf0 con 0x55d7570dfd30 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb16c0b09a0 con 0x55d7570dfd30 2024-09-10T21:03:21.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb164ff9640 1 -- 172.21.15.33:0/2669599394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb178004020 con 0x55d7570dfd30 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1858f9740 10 client.15093 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d75716ec70 con 0x55d7570dfd30 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.432+0000 7fb164ff9640 1 -- 172.21.15.33:0/2669599394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb16c0b09a0 con 0x55d7570dfd30 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb164ff9640 1 -- 172.21.15.33:0/2669599394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb17802f070 con 0x55d7570dfd30 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb1858f9740 10 client.15093 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb1858f9740 10 client.15093 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb1858f9740 20 client.15093 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d7571738b0 0x55d7571725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb1858f9740 4 client.15093 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "status"}) -- 0x55d757055610 con 0x55d7571738b0 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb180abf640 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d7571738b0 0x55d7571725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb180abf640 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d7571738b0 0x55d7571725c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6145c4cad1e1b80 server_cookie=43ec8ca5ae5ae65 in_seq=0 out_seq=0 2024-09-10T21:03:21.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.433+0000 7fb164ff9640 10 client.15093 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb164ff9640 1 -- 172.21.15.33:0/2669599394 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55d757055610 con 0x55d7571738b0 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb164ff9640 10 client.15093 handle_command_reply: tid=0 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 client.15093 shutdown 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 msgr2=0x55d7570e2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e2240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb178012cb0 tx=0x7fb1780046c0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 shutdown_connections 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d7570f09f0 0x55d7570e12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d7571738b0 0x55d7571725c0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d7570e0650 0x55d7570e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 --2- 172.21.15.33:0/2669599394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d7570dfd30 0x55d7570e2240 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 >> 172.21.15.33:0/2669599394 conn(0x55d7570ddb60 msgr2=0x55d7570f1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb164ff9640 0 client.15093 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:21.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 shutdown_connections 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 1 -- 172.21.15.33:0/2669599394 wait complete. 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.434+0000 7fb1858f9740 20 client.15093 trim_cache size 0 max 16384 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 3, 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24532, 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:21.445 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 70.583392731999993, 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 104.69165700800001 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1680b53f0 msgr2=0x7fb1680b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1680b53f0 0x7fb1680b78b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55d7570e2eb0 tx=0x7fb178005f90 comp rx=0 tx=0).stop 2024-09-10T21:03:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c104600 msgr2=0x7fb17c1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c104600 0x7fb17c1997c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb16c00d740 tx=0x7fb16c00dc00 comp rx=0 tx=0).stop 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 shutdown_connections 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1680b53f0 0x7fb1680b78b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb17c10d140 0x7fb17c19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb17c104600 0x7fb17c1997c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 --2- 172.21.15.33:0/798823902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb17c103ce0 0x7fb17c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 >> 172.21.15.33:0/798823902 conn(0x7fb17c0fd700 msgr2=0x7fb17c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 shutdown_connections 2024-09-10T21:03:21.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:21.435+0000 7fb183524640 1 -- 172.21.15.33:0/798823902 wait complete. 2024-09-10T21:03:21.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:21 smithi174 ceph-mon[35635]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:21 smithi033 ceph-mon[30860]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:21 smithi104 ceph-mon[35325]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:22.204 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T21:03:22.467 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 -- 172.21.15.33:0/1806150270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 msgr2=0x7fc1a810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/1806150270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a810f950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0009ca0 tx=0x7fc1a002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 -- 172.21.15.33:0/1806150270 shutdown_connections 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/1806150270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a810f950 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/1806150270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1a81087d0 0x7fc1a8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/1806150270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a8102dc0 0x7fc1a81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 -- 172.21.15.33:0/1806150270 >> 172.21.15.33:0/1806150270 conn(0x7fc1a80fe830 msgr2=0x7fc1a8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:22.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 -- 172.21.15.33:0/1806150270 shutdown_connections 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.942+0000 7fc1ac81b640 1 -- 172.21.15.33:0/1806150270 wait complete. 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 Processor -- start 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 -- start start 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a8102dc0 0x7fc1a80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1a81087d0 0x7fc1a80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8112e50 con 0x7fc1a8109150 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8112cd0 con 0x7fc1a8102dc0 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1ac81b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8112fd0 con 0x7fc1a81087d0 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1a6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a8075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34232/0 (socket says 172.21.15.33:34232) 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1a6d9d640 1 -- 172.21.15.33:0/2496158386 learned_addr learned my addr 172.21.15.33:0/2496158386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:22.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.943+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a8102dc0 0x7fc1a80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1a81087d0 0x7fc1a80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a6d9d640 1 -- 172.21.15.33:0/2496158386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1a81087d0 msgr2=0x7fc1a80759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a6d9d640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1a81087d0 0x7fc1a80759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a6d9d640 1 -- 172.21.15.33:0/2496158386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a8102dc0 msgr2=0x7fc1a80754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a6d9d640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a8102dc0 0x7fc1a80754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a6d9d640 1 -- 172.21.15.33:0/2496158386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a81a7800 con 0x7fc1a8109150 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1a81087d0 0x7fc1a80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1a6d9d640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a8075f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005cc0 tx=0x7fc1a002f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1977fe640 1 -- 172.21.15.33:0/2496158386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0003cf0 con 0x7fc1a8109150 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1977fe640 1 -- 172.21.15.33:0/2496158386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a00044f0 con 0x7fc1a8109150 2024-09-10T21:03:22.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81a7a90 con 0x7fc1a8109150 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1977fe640 1 -- 172.21.15.33:0/2496158386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a002fc30 con 0x7fc1a8109150 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.944+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81a8050 con 0x7fc1a8109150 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.946+0000 7fc1977fe640 1 -- 172.21.15.33:0/2496158386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fc1a0004040 con 0x7fc1a8109150 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 Processor -- start 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 -- start start 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82ba75e0 0x55ab82ba99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab82b12c10 con 0x55ab82b9ad30 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab82b02040 con 0x55ab82b9b650 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab82b05e50 con 0x55ab82ba75e0 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.947+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34240/0 (socket says 172.21.15.33:34240) 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2496380268 learned_addr learned my addr 172.21.15.33:0/2496380268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82ba75e0 0x55ab82ba99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1977fe640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc1900b5370 0x7fc1900b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2496380268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82ba75e0 msgr2=0x55ab82ba99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82ba75e0 0x55ab82ba99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2496380268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 msgr2=0x55ab82ba70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2496380268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab82860540 con 0x55ab82b9ad30 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a6d9d640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc1900b5370 0x7fc1900b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9b110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009cf0 tx=0x7fc19c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aa50e62ca3c887 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc1977fe640 1 -- 172.21.15.33:0/2496158386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc1a0107240 con 0x7fc1a8109150 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc194ff9640 1 -- 172.21.15.33:0/2496380268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c009ea0 con 0x55ab82b9ad30 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc194ff9640 1 -- 172.21.15.33:0/2496380268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c010040 con 0x55ab82b9ad30 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.948+0000 7fc194ff9640 1 -- 172.21.15.33:0/2496380268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c006760 con 0x55ab82b9ad30 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2496380268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 msgr2=0x55ab82b9b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9b110 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009cf0 tx=0x7fc19c0094c0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1a6d9d640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc1900b5370 0x7fc1900b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00cd560 tx=0x7fc1a0097a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2496380268 shutdown_connections 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82ba75e0 0x55ab82ba99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2496380268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9b110 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2496380268 >> 172.21.15.33:0/2496380268 conn(0x55ab82b98b60 msgr2=0x55ab82b99d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2496380268 shutdown_connections 2024-09-10T21:03:22.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2496380268 wait complete. 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.949+0000 7fc1aebf0740 1 -- start start 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 1 Processor -- start 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82ba45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82ba75e0 0x55ab82ba5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab829635f0 con 0x55ab82ba75e0 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab82a9d010 con 0x55ab82b9b650 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab82baa230 con 0x55ab82b9ad30 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1aebf0740 10 client.0 _ll_register_callbacks cb 0x55ab82afc340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82ba75e0 0x55ab82ba5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82ba75e0 0x55ab82ba5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34248/0 (socket says 172.21.15.33:34248) 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2259520048 learned_addr learned my addr 172.21.15.33:0/2259520048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:22.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82ba45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2259520048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 msgr2=0x55ab82ba45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82ba45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 msgr2=0x55ab82ba4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2259520048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab82acaa00 con 0x55ab82ba75e0 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.950+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82ba75e0 0x55ab82ba5020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009170 tx=0x7fc19c006340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=700b9cfdd4b31699 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc18dffb640 1 -- 172.21.15.33:0/2259520048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c028070 con 0x55ab82ba75e0 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc18dffb640 1 -- 172.21.15.33:0/2259520048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc19c005bd0 con 0x55ab82ba75e0 2024-09-10T21:03:22.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19c010040 con 0x55ab82ba75e0 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc18dffb640 1 -- 172.21.15.33:0/2259520048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc19c024070 con 0x55ab82ba75e0 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 10 client.15105 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab82bb5e20 con 0x55ab82ba75e0 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc18dffb640 1 -- 172.21.15.33:0/2259520048 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c010040 con 0x55ab82ba75e0 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc18dffb640 1 -- 172.21.15.33:0/2259520048 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc19c02b030 con 0x55ab82ba75e0 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 10 client.15105 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 10 client.15105 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 20 client.15105 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:22.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82bb80e0 0x55ab82bb9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 4 client.15105 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.951+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab82b14b60 con 0x55ab82bb80e0 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.952+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82bb80e0 0x55ab82bb9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.952+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82bb80e0 0x55ab82bb9550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bf9b23d946337b1a server_cookie=e54c7790e94d3e9b in_seq=0 out_seq=0 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.952+0000 7fc18dffb640 10 client.15105 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc18dffb640 1 -- 172.21.15.33:0/2259520048 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ab82b14b60 con 0x55ab82bb80e0 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc18dffb640 10 client.15105 handle_command_reply: tid=0 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 client.15105 shutdown 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82ba75e0 msgr2=0x55ab82ba5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82ba75e0 0x55ab82ba5020 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc19c009170 tx=0x7fc19c006340 comp rx=0 tx=0).stop 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 shutdown_connections 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82bb80e0 0x55ab82bb9550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82ba75e0 0x55ab82ba5020 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82b9b650 0x55ab82ba4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc18dffb640 0 client.15105 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2259520048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82ba45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 >> 172.21.15.33:0/2259520048 conn(0x55ab82b98b60 msgr2=0x55ab82b9e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 shutdown_connections 2024-09-10T21:03:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2259520048 wait complete. 2024-09-10T21:03:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.953+0000 7fc1aebf0740 20 client.15105 trim_cache size 0 max 16384 2024-09-10T21:03:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 Processor -- start 2024-09-10T21:03:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 -- start start 2024-09-10T21:03:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9b650 0x55ab82ba2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82ba3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab82bb7890 con 0x55ab82b9ad30 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab82bb7f60 con 0x55ab82ba75e0 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab82bb8630 con 0x55ab82b9b650 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.972+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34254/0 (socket says 172.21.15.33:34254) 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2929095331 learned_addr learned my addr 172.21.15.33:0/2929095331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a659c640 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9b650 0x55ab82ba2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82ba3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2929095331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9b650 msgr2=0x55ab82ba2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9b650 0x55ab82ba2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2929095331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 msgr2=0x55ab82ba3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82ba3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 -- 172.21.15.33:0/2929095331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab82b995e0 con 0x55ab82b9ad30 2024-09-10T21:03:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9eef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00ff50 tx=0x7fc19c009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81509fc62312d16c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc18ffff640 1 -- 172.21.15.33:0/2929095331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c028030 con 0x55ab82b9ad30 2024-09-10T21:03:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc18ffff640 1 -- 172.21.15.33:0/2929095331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c0091e0 con 0x55ab82b9ad30 2024-09-10T21:03:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.973+0000 7fc18ffff640 1 -- 172.21.15.33:0/2929095331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c005bd0 con 0x55ab82b9ad30 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2929095331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 msgr2=0x55ab82b9eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9eef0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00ff50 tx=0x7fc19c009910 comp rx=0 tx=0).stop 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2929095331 shutdown_connections 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82ba3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9b650 0x55ab82ba2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/2929095331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9ad30 0x55ab82b9eef0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2929095331 >> 172.21.15.33:0/2929095331 conn(0x55ab82b98b60 msgr2=0x55ab82b99a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2929095331 shutdown_connections 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 -- 172.21.15.33:0/2929095331 wait complete. 2024-09-10T21:03:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 -- start start 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.974+0000 7fc1aebf0740 1 Processor -- start 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82bbc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9b650 0x55ab82bbce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1aebf0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82bc1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ab82bb93d0 con 0x55ab82b9b650 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ab82bb8d00 con 0x55ab82ba75e0 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1aebf0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab82bb9aa0 con 0x55ab82b9ad30 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1aebf0740 10 client.0 _ll_register_callbacks cb 0x7fc19c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9b650 0x55ab82bbce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9b650 0x55ab82bbce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34260/0 (socket says 172.21.15.33:34260) 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 -- 172.21.15.33:0/1641041033 learned_addr learned my addr 172.21.15.33:0/1641041033 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82bc1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82bbc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 -- 172.21.15.33:0/1641041033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 msgr2=0x55ab82bbc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82bbc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 -- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 msgr2=0x55ab82bc1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82bc1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 -- 172.21.15.33:0/1641041033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab82bc1dd0 con 0x55ab82b9b650 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc1a659c640 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9b650 0x55ab82bbce30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc198004810 tx=0x7fc198009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9066f5dd6d95ca31 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc18dffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.975+0000 7fc18dffb640 1 -- 172.21.15.33:0/1641041033 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1980155d0 con 0x55ab82b9b650 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc18dffb640 1 -- 172.21.15.33:0/1641041033 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc198015770 con 0x55ab82b9b650 2024-09-10T21:03:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc19c0091e0 con 0x55ab82b9b650 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc18dffb640 1 -- 172.21.15.33:0/1641041033 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc198015970 con 0x55ab82b9b650 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 10 client.15111 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ab82c25390 con 0x55ab82b9b650 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc18dffb640 1 -- 172.21.15.33:0/1641041033 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc19c0091e0 con 0x55ab82b9b650 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc18dffb640 1 -- 172.21.15.33:0/1641041033 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fc198025ac0 con 0x55ab82b9b650 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 10 client.15111 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 10 client.15111 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 20 client.15111 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82c29f40 0x55ab82c28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 4 client.15111 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "status"}) -- 0x55ab82b10610 con 0x55ab82c29f40 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.976+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82c29f40 0x55ab82c28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1a5d9b640 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82c29f40 0x55ab82c28c50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dba5af1846819860 server_cookie=a7a688c92e251e42 in_seq=0 out_seq=0 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc18dffb640 10 client.15111 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc18dffb640 1 -- 172.21.15.33:0/1641041033 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ab82b10610 con 0x55ab82c29f40 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc18dffb640 10 client.15111 handle_command_reply: tid=0 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 client.15111 shutdown 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9b650 msgr2=0x55ab82bbce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9b650 0x55ab82bbce30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc198004810 tx=0x7fc198009c70 comp rx=0 tx=0).stop 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 shutdown_connections 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ab82ba75e0 0x55ab82bc1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ab82c29f40 0x55ab82c28c50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ab82b9b650 0x55ab82bbce30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 --2- 172.21.15.33:0/1641041033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab82b9ad30 0x55ab82bbc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 >> 172.21.15.33:0/1641041033 conn(0x55ab82b98b60 msgr2=0x55ab82ba2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc18dffb640 0 client.15111 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 shutdown_connections 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 1 -- 172.21.15.33:0/1641041033 wait complete. 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.977+0000 7fc1aebf0740 20 client.15111 trim_cache size 0 max 16384 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24539, 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:22.989 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 73.138828480000001, 2024-09-10T21:03:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:22.990 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 103.534819063 2024-09-10T21:03:22.990 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc1900b5370 msgr2=0x7fc1900b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc1900b5370 0x7fc1900b7830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc1a00cd560 tx=0x7fc1a0097a40 comp rx=0 tx=0).stop 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 msgr2=0x7fc1a8075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a8075f20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005cc0 tx=0x7fc1a002f470 comp rx=0 tx=0).stop 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 shutdown_connections 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc1900b5370 0x7fc1900b7830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc1a8109150 0x7fc1a8075f20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1a81087d0 0x7fc1a80759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 --2- 172.21.15.33:0/2496158386 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc1a8102dc0 0x7fc1a80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 >> 172.21.15.33:0/2496158386 conn(0x7fc1a80fe830 msgr2=0x7fc1a8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:22.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 shutdown_connections 2024-09-10T21:03:22.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:22.978+0000 7fc1ac81b640 1 -- 172.21.15.33:0/2496158386 wait complete. 2024-09-10T21:03:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:23 smithi174 ceph-mon[35635]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:23.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:23 smithi033 ceph-mon[30860]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:23.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T21:03:23.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:23 smithi104 ceph-mon[35325]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:24.005 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 -- 172.21.15.33:0/2055045311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb914108020 msgr2=0x7fb91410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/2055045311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb914108020 0x7fb91410e820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009ca0 tx=0x7fb90c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 -- 172.21.15.33:0/2055045311 shutdown_connections 2024-09-10T21:03:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/2055045311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb914108020 0x7fb91410e820 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/2055045311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9141076a0 0x7fb914107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/2055045311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb914101c90 0x7fb914102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 -- 172.21.15.33:0/2055045311 >> 172.21.15.33:0/2055045311 conn(0x7fb9140fd700 msgr2=0x7fb9140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 -- 172.21.15.33:0/2055045311 shutdown_connections 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 -- 172.21.15.33:0/2055045311 wait complete. 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 Processor -- start 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.477+0000 7fb91b5f5640 1 -- start start 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb914101c90 0x7fb9140fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9141076a0 0x7fb9140fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb914108020 0x7fb9140fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb914111ce0 con 0x7fb9141076a0 2024-09-10T21:03:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb914111b60 con 0x7fb914108020 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb914111e60 con 0x7fb914101c90 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9141076a0 0x7fb9140fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9141076a0 0x7fb9140fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34280/0 (socket says 172.21.15.33:34280) 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 -- 172.21.15.33:0/1787429283 learned_addr learned my addr 172.21.15.33:0/1787429283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb919391640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb914101c90 0x7fb9140fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb919b92640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb914108020 0x7fb9140fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 -- 172.21.15.33:0/1787429283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb914101c90 msgr2=0x7fb9140fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb914101c90 0x7fb9140fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 -- 172.21.15.33:0/1787429283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb914108020 msgr2=0x7fb9140fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb914108020 0x7fb9140fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 -- 172.21.15.33:0/1787429283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9141a76e0 con 0x7fb9141076a0 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb919391640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb914101c90 0x7fb9140fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb918b90640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9141076a0 0x7fb9140fe140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c890 tx=0x7fb90400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb90a7fc640 1 -- 172.21.15.33:0/1787429283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904004220 con 0x7fb9141076a0 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb90a7fc640 1 -- 172.21.15.33:0/1787429283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9040043c0 con 0x7fb9141076a0 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb90a7fc640 1 -- 172.21.15.33:0/1787429283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb904004a00 con 0x7fb9141076a0 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9141a79d0 con 0x7fb9141076a0 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.478+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9141a7fe0 con 0x7fb9141076a0 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 Processor -- start 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 -- start start 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e8d30 0x55c9488e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb90a7fc640 1 -- 172.21.15.33:0/1787429283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb9040079b0 con 0x7fb9141076a0 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e9650 0x55c9488f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488f55e0 0x55c9488f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c948860c10 con 0x55c9488e9650 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c948850040 con 0x55c9488f55e0 2024-09-10T21:03:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c948853e50 con 0x55c9488e8d30 2024-09-10T21:03:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb919391640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e9650 0x55c9488f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb919391640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e9650 0x55c9488f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34284/0 (socket says 172.21.15.33:34284) 2024-09-10T21:03:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.481+0000 7fb919391640 1 -- 172.21.15.33:0/2621319365 learned_addr learned my addr 172.21.15.33:0/2621319365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb918b90640 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488f55e0 0x55c9488f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919b92640 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e8d30 0x55c9488e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919b92640 1 -- 172.21.15.33:0/2621319365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488f55e0 msgr2=0x55c9488f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919b92640 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488f55e0 0x55c9488f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919b92640 1 -- 172.21.15.33:0/2621319365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e9650 msgr2=0x55c9488f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919b92640 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e9650 0x55c9488f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919b92640 1 -- 172.21.15.33:0/2621319365 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9485ae540 con 0x55c9488e8d30 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb918b90640 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488f55e0 0x55c9488f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919391640 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e9650 0x55c9488f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919b92640 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e8d30 0x55c9488e9110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c009930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47defe52629b0bab server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb90a7fc640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9000b53f0 0x7fb9000b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb8ff7fe640 1 -- 172.21.15.33:0/2621319365 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c031650 con 0x55c9488e8d30 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb919391640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9000b53f0 0x7fb9000b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb8ff7fe640 1 -- 172.21.15.33:0/2621319365 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90c040070 con 0x55c9488e8d30 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.482+0000 7fb8ff7fe640 1 -- 172.21.15.33:0/2621319365 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0045f0 con 0x55c9488e8d30 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb90a7fc640 1 -- 172.21.15.33:0/1787429283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb9040e5b30 con 0x7fb9141076a0 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2621319365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e8d30 msgr2=0x55c9488e9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb919391640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9000b53f0 0x7fb9000b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55c9488ebeb0 tx=0x7fb910008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e8d30 0x55c9488e9110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c009930 comp rx=0 tx=0).stop 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2621319365 shutdown_connections 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488f55e0 0x55c9488f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e9650 0x55c9488f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2621319365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e8d30 0x55c9488e9110 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2621319365 >> 172.21.15.33:0/2621319365 conn(0x55c9488e6b60 msgr2=0x55c9488e7d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2621319365 shutdown_connections 2024-09-10T21:03:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.483+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2621319365 wait complete. 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 -- start start 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 Processor -- start 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488f2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 0x55c9488f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9486b15f0 con 0x55c9488f55e0 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c9487eb010 con 0x55c9488e8d30 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9488f8230 con 0x55c9488e9650 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb91d9ca740 10 client.0 _ll_register_callbacks cb 0x55c94884a340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb919b92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.484+0000 7fb919b92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34300/0 (socket says 172.21.15.33:34300) 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919b92640 1 -- 172.21.15.33:0/2362423020 learned_addr learned my addr 172.21.15.33:0/2362423020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb918b90640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 0x55c9488f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919391640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488f2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919b92640 1 -- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 msgr2=0x55c9488f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919b92640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 0x55c9488f2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919b92640 1 -- 172.21.15.33:0/2362423020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 msgr2=0x55c9488f2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919b92640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488f2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919b92640 1 -- 172.21.15.33:0/2362423020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c948818a00 con 0x55c9488f55e0 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919391640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb919b92640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f2fc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009a60 tx=0x7fb90c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8247ebdc79f289 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb8fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2362423020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c04c020 con 0x55c9488f55e0 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb90c040070 con 0x55c9488f55e0 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2362423020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb90c003b70 con 0x55c9488f55e0 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2362423020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb90c0318a0 con 0x55c9488f55e0 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb91d9ca740 10 client.15123 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c948903df0 con 0x55c9488f55e0 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.485+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2362423020 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90c040070 con 0x55c9488f55e0 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2362423020 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb90c0305c0 con 0x55c9488f55e0 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb91d9ca740 10 client.15123 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb91d9ca740 10 client.15123 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb91d9ca740 20 client.15123 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c948906010 0x55c948907480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb91d9ca740 4 client.15123 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c948862b60 con 0x55c948906010 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb919391640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c948906010 0x55c948907480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb919391640 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c948906010 0x55c948907480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fe50f4dcf0ba881 server_cookie=7aa8c9b75404fb9b in_seq=0 out_seq=0 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.486+0000 7fb8fcff9640 10 client.15123 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.487+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2362423020 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c948862b60 con 0x55c948906010 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.487+0000 7fb8fcff9640 10 client.15123 handle_command_reply: tid=0 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.487+0000 7fb91d9ca740 1 client.15123 shutdown 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.487+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 msgr2=0x55c9488f2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.487+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f2fc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009a60 tx=0x7fb90c0045e0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 shutdown_connections 2024-09-10T21:03:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c948906010 0x55c948907480 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f2fc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 0x55c9488f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb8fcff9640 0 client.15123 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2362423020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488f2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 >> 172.21.15.33:0/2362423020 conn(0x55c9488e6b60 msgr2=0x55c9488ec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 shutdown_connections 2024-09-10T21:03:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2362423020 wait complete. 2024-09-10T21:03:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.488+0000 7fb91d9ca740 20 client.15123 trim_cache size 0 max 16384 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.506+0000 7fb91d9ca740 1 Processor -- start 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.506+0000 7fb91d9ca740 1 -- start start 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.506+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 0x55c9488e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c9489057c0 con 0x55c9488f55e0 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c948905e90 con 0x55c9488e8d30 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c948906560 con 0x55c9488e9650 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34302/0 (socket says 172.21.15.33:34302) 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 -- 172.21.15.33:0/1309082240 learned_addr learned my addr 172.21.15.33:0/1309082240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 -- 172.21.15.33:0/1309082240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 msgr2=0x55c9488e1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 0x55c9488e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 -- 172.21.15.33:0/1309082240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 msgr2=0x55c9488ecef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919391640 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488ecef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 -- 172.21.15.33:0/1309082240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9488e75e0 con 0x55c9488f55e0 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919391640 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb919b92640 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f3300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb90c02f590 tx=0x7fb90c009930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5690e7a5ac6e4761 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb8feffd640 1 -- 172.21.15.33:0/1309082240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c04c070 con 0x55c9488f55e0 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb8feffd640 1 -- 172.21.15.33:0/1309082240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90c040070 con 0x55c9488f55e0 2024-09-10T21:03:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.507+0000 7fb8feffd640 1 -- 172.21.15.33:0/1309082240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c004040 con 0x55c9488f55e0 2024-09-10T21:03:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 -- 172.21.15.33:0/1309082240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 msgr2=0x55c9488f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f3300 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb90c02f590 tx=0x7fb90c009930 comp rx=0 tx=0).stop 2024-09-10T21:03:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 -- 172.21.15.33:0/1309082240 shutdown_connections 2024-09-10T21:03:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488f55e0 0x55c9488f3300 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488e9650 0x55c9488e1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/1309082240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e8d30 0x55c9488ecef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 -- 172.21.15.33:0/1309082240 >> 172.21.15.33:0/1309082240 conn(0x55c9488e6b60 msgr2=0x55c9488ed830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 -- 172.21.15.33:0/1309082240 shutdown_connections 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 -- 172.21.15.33:0/1309082240 wait complete. 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.508+0000 7fb91d9ca740 1 -- start start 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 1 Processor -- start 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e8d30 0x55c94890ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e9650 0x55c948909450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488f55e0 0x55c9489099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c948907300 con 0x55c9488e8d30 2024-09-10T21:03:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c948906c30 con 0x55c9488e9650 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9489079d0 con 0x55c9488f55e0 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb91d9ca740 10 client.0 _ll_register_callbacks cb 0x7fb9100072a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e8d30 0x55c94890ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e8d30 0x55c94890ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34310/0 (socket says 172.21.15.33:34310) 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 -- 172.21.15.33:0/2483610601 learned_addr learned my addr 172.21.15.33:0/2483610601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb918b90640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e9650 0x55c948909450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919b92640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488f55e0 0x55c9489099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 -- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488f55e0 msgr2=0x55c9489099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488f55e0 0x55c9489099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 -- 172.21.15.33:0/2483610601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e9650 msgr2=0x55c948909450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e9650 0x55c948909450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb919391640 1 -- 172.21.15.33:0/2483610601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c94890a220 con 0x55c9488e8d30 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.509+0000 7fb918b90640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e9650 0x55c948909450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb919391640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e8d30 0x55c94890ff30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb910006f40 tx=0x7fb9100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4ec38f7ad9b6fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb8fcff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2483610601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb91001c040 con 0x55c9488e8d30 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2483610601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb91002a480 con 0x55c9488e8d30 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb90c040070 con 0x55c9488e8d30 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2483610601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb91002a680 con 0x55c9488e8d30 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 10 client.15129 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c9489729b0 con 0x55c9488e8d30 2024-09-10T21:03:24.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2483610601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90c040070 con 0x55c9488e8d30 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2483610601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fb9100238c0 con 0x55c9488e8d30 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 10 client.15129 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 20 client.15129 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c9489775e0 0x55c9489762f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 4 client.15129 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.510+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "status"}) -- 0x55c94885e610 con 0x55c9489775e0 2024-09-10T21:03:24.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb918b90640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c9489775e0 0x55c9489762f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb918b90640 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c9489775e0 0x55c9489762f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5c853f02edc4255e server_cookie=314090e7c6d150e2 in_seq=0 out_seq=0 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb8fcff9640 10 client.15129 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb8fcff9640 1 -- 172.21.15.33:0/2483610601 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c94885e610 con 0x55c9489775e0 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb8fcff9640 10 client.15129 handle_command_reply: tid=0 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 client.15129 shutdown 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e8d30 msgr2=0x55c94890ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e8d30 0x55c94890ff30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb910006f40 tx=0x7fb9100046c0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 shutdown_connections 2024-09-10T21:03:24.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9488f55e0 0x55c9489099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c9488e9650 0x55c948909450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c9489775e0 0x55c9489762f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 --2- 172.21.15.33:0/2483610601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c9488e8d30 0x55c94890ff30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 >> 172.21.15.33:0/2483610601 conn(0x55c9488e6b60 msgr2=0x55c9488ed830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.511+0000 7fb8fcff9640 0 client.15129 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 shutdown_connections 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91d9ca740 1 -- 172.21.15.33:0/2483610601 wait complete. 2024-09-10T21:03:24.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91d9ca740 20 client.15129 trim_cache size 0 max 16384 2024-09-10T21:03:24.536 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-09-10T21:03:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24547, 2024-09-10T21:03:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:24.536 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 75.679202212000007, 2024-09-10T21:03:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:24.537 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 102.439648324 2024-09-10T21:03:24.537 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:24.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9000b53f0 msgr2=0x7fb9000b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9000b53f0 0x7fb9000b78b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55c9488ebeb0 tx=0x7fb910008040 comp rx=0 tx=0).stop 2024-09-10T21:03:24.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9141076a0 msgr2=0x7fb9140fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9141076a0 0x7fb9140fe140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb90400c890 tx=0x7fb90400cd50 comp rx=0 tx=0).stop 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 shutdown_connections 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9000b53f0 0x7fb9000b78b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb914108020 0x7fb9140fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9141076a0 0x7fb9140fe140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 --2- 172.21.15.33:0/1787429283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb914101c90 0x7fb9140fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 >> 172.21.15.33:0/1787429283 conn(0x7fb9140fd700 msgr2=0x7fb91406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 shutdown_connections 2024-09-10T21:03:24.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:24.512+0000 7fb91b5f5640 1 -- 172.21.15.33:0/1787429283 wait complete. 2024-09-10T21:03:25.269 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T21:03:25.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:25 smithi033 ceph-mon[30860]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:25.528 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:25 smithi174 ceph-mon[35635]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:25.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:25 smithi104 ceph-mon[35325]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 -- 172.21.15.33:0/2899144058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875810a920 msgr2=0x7f875810cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 --2- 172.21.15.33:0/2899144058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875810a920 0x7f875810cd10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f8748009900 tx=0x7f874802a410 comp rx=0 tx=0).stop 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 -- 172.21.15.33:0/2899144058 shutdown_connections 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 --2- 172.21.15.33:0/2899144058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f875810d250 0x7f875810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 --2- 172.21.15.33:0/2899144058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f875810a920 0x7f875810cd10 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 --2- 172.21.15.33:0/2899144058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8758101810 0x7f8758101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 -- 172.21.15.33:0/2899144058 >> 172.21.15.33:0/2899144058 conn(0x7f8758076f50 msgr2=0x7f8758077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 -- 172.21.15.33:0/2899144058 shutdown_connections 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.049+0000 7f875f7bb640 1 -- 172.21.15.33:0/2899144058 wait complete. 2024-09-10T21:03:26.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 Processor -- start 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 -- start start 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8758101810 0x7f8758199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f875810a920 0x7f8758199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f875810d250 0x7f875819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8758112db0 con 0x7f8758101810 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8758112c30 con 0x7f875810d250 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875f7bb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8758112f30 con 0x7f875810a920 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875d557640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8758101810 0x7f8758199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875d557640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8758101810 0x7f8758199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34336/0 (socket says 172.21.15.33:34336) 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875d557640 1 -- 172.21.15.33:0/980179383 learned_addr learned my addr 172.21.15.33:0/980179383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875cd56640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f875810a920 0x7f8758199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.050+0000 7f875dd58640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f875810d250 0x7f875819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875d557640 1 -- 172.21.15.33:0/980179383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f875810a920 msgr2=0x7f8758199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875d557640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f875810a920 0x7f8758199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875d557640 1 -- 172.21.15.33:0/980179383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f875810d250 msgr2=0x7f875819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875d557640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f875810d250 0x7f875819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875d557640 1 -- 172.21.15.33:0/980179383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f875819f2f0 con 0x7f8758101810 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875cd56640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f875810a920 0x7f8758199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875d557640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8758101810 0x7f8758199240 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f875400b4a0 tx=0x7f875400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f874e7fc640 1 -- 172.21.15.33:0/980179383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754014070 con 0x7f8758101810 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f874e7fc640 1 -- 172.21.15.33:0/980179383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8754010ac0 con 0x7f8758101810 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f875819f5e0 con 0x7f8758101810 2024-09-10T21:03:26.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f874e7fc640 1 -- 172.21.15.33:0/980179383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8754017c30 con 0x7f8758101810 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.051+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581a5d20 con 0x7f8758101810 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.054+0000 7f8761b90740 1 Processor -- start 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.054+0000 7f8761b90740 1 -- start start 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.054+0000 7f874e7fc640 1 -- 172.21.15.33:0/980179383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f8754010610 con 0x7f8758101810 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.054+0000 7f8761b90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff19d30 0x55dc4ff1a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.054+0000 7f8761b90740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff260a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f8761b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff289d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f8761b90740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc4fe91c10 con 0x55dc4ff1a650 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f8761b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dc4fe81040 con 0x55dc4ff265e0 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f8761b90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc4fe84e50 con 0x55dc4ff19d30 2024-09-10T21:03:26.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff260a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff260a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34350/0 (socket says 172.21.15.33:34350) 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 -- 172.21.15.33:0/4032528561 learned_addr learned my addr 172.21.15.33:0/4032528561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875cd56640 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff289d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875dd58640 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff19d30 0x55dc4ff1a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 -- 172.21.15.33:0/4032528561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff19d30 msgr2=0x55dc4ff1a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff19d30 0x55dc4ff1a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 -- 172.21.15.33:0/4032528561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 msgr2=0x55dc4ff289d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff289d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 -- 172.21.15.33:0/4032528561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4fbdf540 con 0x55dc4ff1a650 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875d557640 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff260a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87540ed5a0 tx=0x7f87540eda60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e420bc98b54b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f874e7fc640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87440b5370 0x7f87440b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.055+0000 7f875dd58640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87440b5370 0x7f87440b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f87437fe640 1 -- 172.21.15.33:0/4032528561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87540f9070 con 0x55dc4ff1a650 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f87437fe640 1 -- 172.21.15.33:0/4032528561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8754017360 con 0x55dc4ff1a650 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f87437fe640 1 -- 172.21.15.33:0/4032528561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87540a7860 con 0x55dc4ff1a650 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f874e7fc640 1 -- 172.21.15.33:0/980179383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87540e5360 con 0x7f8758101810 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f8761b90740 1 -- 172.21.15.33:0/4032528561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 msgr2=0x55dc4ff260a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f875dd58640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87440b5370 0x7f87440b7830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8750007e30 tx=0x7f8750007400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f8761b90740 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff260a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f87540ed5a0 tx=0x7f87540eda60 comp rx=0 tx=0).stop 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f8761b90740 1 -- 172.21.15.33:0/4032528561 shutdown_connections 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f8761b90740 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff289d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f8761b90740 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff260a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f8761b90740 1 --2- 172.21.15.33:0/4032528561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff19d30 0x55dc4ff1a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.056+0000 7f8761b90740 1 -- 172.21.15.33:0/4032528561 >> 172.21.15.33:0/4032528561 conn(0x55dc4ff17b60 msgr2=0x55dc4ff18d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 -- 172.21.15.33:0/4032528561 shutdown_connections 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 -- 172.21.15.33:0/4032528561 wait complete. 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 -- start start 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 Processor -- start 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff23550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff23a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff23fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc4fce25f0 con 0x55dc4ff1a650 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dc4fe1c010 con 0x55dc4ff19d30 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ff29290 con 0x55dc4ff265e0 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f8761b90740 10 client.0 _ll_register_callbacks cb 0x55dc4fe7b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f875cd56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff23a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f875cd56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff23a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34362/0 (socket says 172.21.15.33:34362) 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.057+0000 7f875cd56640 1 -- 172.21.15.33:0/704938868 learned_addr learned my addr 172.21.15.33:0/704938868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875d557640 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff23550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875dd58640 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff23fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875cd56640 1 -- 172.21.15.33:0/704938868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 msgr2=0x55dc4ff23fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875cd56640 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff23fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875cd56640 1 -- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 msgr2=0x55dc4ff23550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875cd56640 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff23550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875cd56640 1 -- 172.21.15.33:0/704938868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4fe49a00 con 0x55dc4ff1a650 2024-09-10T21:03:26.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f875cd56640 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff23a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f874802a930 tx=0x7f8748031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4700b0ede122c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f8740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f8740ff9640 1 -- 172.21.15.33:0/704938868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748035040 con 0x55dc4ff1a650 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f8740ff9640 1 -- 172.21.15.33:0/704938868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8748004300 con 0x55dc4ff1a650 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dc4fe93b60 con 0x55dc4ff1a650 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f8740ff9640 1 -- 172.21.15.33:0/704938868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8748005020 con 0x55dc4ff1a650 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f8761b90740 10 client.15141 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.058+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc4ff34e20 con 0x55dc4ff1a650 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8740ff9640 1 -- 172.21.15.33:0/704938868 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dc4fe93b60 con 0x55dc4ff1a650 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8740ff9640 1 -- 172.21.15.33:0/704938868 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f87480035f0 con 0x55dc4ff1a650 2024-09-10T21:03:26.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8761b90740 10 client.15141 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8761b90740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8761b90740 20 client.15141 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8761b90740 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ff37490 0x55dc4ff38900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8761b90740 4 client.15141 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc4fe8f610 con 0x55dc4ff37490 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f875d557640 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ff37490 0x55dc4ff38900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f875d557640 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ff37490 0x55dc4ff38900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8dc15154e6b87a6 server_cookie=271c7762df529563 in_seq=0 out_seq=0 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.059+0000 7f8740ff9640 10 client.15141 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8740ff9640 1 -- 172.21.15.33:0/704938868 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dc4fe8f610 con 0x55dc4ff37490 2024-09-10T21:03:26.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8740ff9640 10 client.15141 handle_command_reply: tid=0 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 client.15141 shutdown 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 msgr2=0x55dc4ff23a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff23a90 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f874802a930 tx=0x7f8748031950 comp rx=0 tx=0).stop 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 shutdown_connections 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ff37490 0x55dc4ff38900 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff23fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff23a90 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 --2- 172.21.15.33:0/704938868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff23550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8740ff9640 0 client.15141 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.060+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 >> 172.21.15.33:0/704938868 conn(0x55dc4ff17b60 msgr2=0x55dc4ff1d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.061+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 shutdown_connections 2024-09-10T21:03:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.061+0000 7f8761b90740 1 -- 172.21.15.33:0/704938868 wait complete. 2024-09-10T21:03:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.061+0000 7f8761b90740 20 client.15141 trim_cache size 0 max 16384 2024-09-10T21:03:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 Processor -- start 2024-09-10T21:03:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 -- start start 2024-09-10T21:03:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff22a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff1e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff1ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ff36c40 con 0x55dc4ff1a650 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ff37310 con 0x55dc4ff19d30 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f8761b90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ff379e0 con 0x55dc4ff265e0 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff1e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff1e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34366/0 (socket says 172.21.15.33:34366) 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 -- 172.21.15.33:0/4089812798 learned_addr learned my addr 172.21.15.33:0/4089812798 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875d557640 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff22a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875dd58640 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff1ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 -- 172.21.15.33:0/4089812798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 msgr2=0x55dc4ff1ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff1ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 -- 172.21.15.33:0/4089812798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 msgr2=0x55dc4ff22a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff22a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.079+0000 7f875cd56640 1 -- 172.21.15.33:0/4089812798 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4ff185e0 con 0x55dc4ff1a650 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f875cd56640 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff1e820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8748003a40 tx=0x7f8748009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f9a209b0177d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8742ffd640 1 -- 172.21.15.33:0/4089812798 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8748035030 con 0x55dc4ff1a650 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8742ffd640 1 -- 172.21.15.33:0/4089812798 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8748003c30 con 0x55dc4ff1a650 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8742ffd640 1 -- 172.21.15.33:0/4089812798 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87480055e0 con 0x55dc4ff1a650 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8761b90740 1 -- 172.21.15.33:0/4089812798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 msgr2=0x55dc4ff1e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8761b90740 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff1e820 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f8748003a40 tx=0x7f8748009520 comp rx=0 tx=0).stop 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8761b90740 1 -- 172.21.15.33:0/4089812798 shutdown_connections 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8761b90740 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff265e0 0x55dc4ff1ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.080+0000 7f8761b90740 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff1a650 0x55dc4ff1e820 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 --2- 172.21.15.33:0/4089812798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff19d30 0x55dc4ff22a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 -- 172.21.15.33:0/4089812798 >> 172.21.15.33:0/4089812798 conn(0x55dc4ff17b60 msgr2=0x55dc4ff362e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 -- 172.21.15.33:0/4089812798 shutdown_connections 2024-09-10T21:03:26.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 -- 172.21.15.33:0/4089812798 wait complete. 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 -- start start 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 Processor -- start 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff19d30 0x55dc4ff40b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff1a650 0x55dc4ff410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff415e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ff38780 con 0x55dc4ff19d30 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ff380b0 con 0x55dc4ff265e0 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc4ff38e50 con 0x55dc4ff1a650 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f8761b90740 10 client.0 _ll_register_callbacks cb 0x7f8748031b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.081+0000 7f875d557640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff19d30 0x55dc4ff40b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff19d30 0x55dc4ff40b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34372/0 (socket says 172.21.15.33:34372) 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 -- 172.21.15.33:0/4043489408 learned_addr learned my addr 172.21.15.33:0/4043489408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875dd58640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875cd56640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff1a650 0x55dc4ff410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 -- 172.21.15.33:0/4043489408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff1a650 msgr2=0x55dc4ff410a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff1a650 0x55dc4ff410a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 -- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 msgr2=0x55dc4ff415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 -- 172.21.15.33:0/4043489408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4ff3bcc0 con 0x55dc4ff19d30 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875dd58640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f875d557640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff19d30 0x55dc4ff40b60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f87540a8c30 tx=0x7f87540175c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2525cc2a14672b8b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f8740ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f8740ff9640 1 -- 172.21.15.33:0/4043489408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875405fb60 con 0x55dc4ff19d30 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f8740ff9640 1 -- 172.21.15.33:0/4043489408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87540e2f60 con 0x55dc4ff19d30 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8748003c30 con 0x55dc4ff19d30 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f8740ff9640 1 -- 172.21.15.33:0/4043489408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f87540607e0 con 0x55dc4ff19d30 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.082+0000 7f8761b90740 10 client.15147 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dc4ffa4270 con 0x55dc4ff19d30 2024-09-10T21:03:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8740ff9640 1 -- 172.21.15.33:0/4043489408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8748003c30 con 0x55dc4ff19d30 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8740ff9640 1 -- 172.21.15.33:0/4043489408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f87540a7c30 con 0x55dc4ff19d30 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8761b90740 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8761b90740 10 client.15147 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8761b90740 20 client.15147 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8761b90740 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ffa88b0 0x55dc4ffa75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8761b90740 4 client.15147 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "status"}) -- 0x55dc4fe8ca40 con 0x55dc4ffa88b0 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.083+0000 7f875cd56640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ffa88b0 0x55dc4ffa75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8740ff9640 10 client.15147 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f875cd56640 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ffa88b0 0x55dc4ffa75c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=445cf27452f0be40 server_cookie=4d060662d903d739 in_seq=0 out_seq=0 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8740ff9640 1 -- 172.21.15.33:0/4043489408 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dc4fe8ca40 con 0x55dc4ffa88b0 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8740ff9640 10 client.15147 handle_command_reply: tid=0 2024-09-10T21:03:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 client.15147 shutdown 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff19d30 msgr2=0x55dc4ff40b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff19d30 0x55dc4ff40b60 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f87540a8c30 tx=0x7f87540175c0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 shutdown_connections 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55dc4ffa88b0 0x55dc4ffa75c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55dc4ff265e0 0x55dc4ff415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc4ff1a650 0x55dc4ff410a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8740ff9640 0 client.15147 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 --2- 172.21.15.33:0/4043489408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55dc4ff19d30 0x55dc4ff40b60 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 >> 172.21.15.33:0/4043489408 conn(0x55dc4ff17b60 msgr2=0x55dc4ff1e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 shutdown_connections 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 1 -- 172.21.15.33:0/4043489408 wait complete. 2024-09-10T21:03:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.084+0000 7f8761b90740 20 client.15147 trim_cache size 0 max 16384 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24566, 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 84.297963663000004, 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:26.098 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:26.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 95.111541418000002 2024-09-10T21:03:26.099 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87440b5370 msgr2=0x7f87440b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87440b5370 0x7f87440b7830 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8750007e30 tx=0x7f8750007400 comp rx=0 tx=0).stop 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8758101810 msgr2=0x7f8758199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8758101810 0x7f8758199240 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f875400b4a0 tx=0x7f875400b960 comp rx=0 tx=0).stop 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 shutdown_connections 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87440b5370 0x7f87440b7830 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f875810d250 0x7f875819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f875810a920 0x7f8758199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 --2- 172.21.15.33:0/980179383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8758101810 0x7f8758199240 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 >> 172.21.15.33:0/980179383 conn(0x7f8758076f50 msgr2=0x7f87581005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 shutdown_connections 2024-09-10T21:03:26.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:26.085+0000 7f875f7bb640 1 -- 172.21.15.33:0/980179383 wait complete. 2024-09-10T21:03:26.854 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T21:03:26.854 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T21:03:26.866 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-09-10T21:03:26.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T21:03:27.126 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:27.530 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:27 smithi033 ceph-mon[30860]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:27.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.609+0000 7f9f04999640 1 -- 172.21.15.33:0/1679427086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f00102220 msgr2=0x7f9f001026a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:27.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.609+0000 7f9f04999640 1 --2- 172.21.15.33:0/1679427086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f00102220 0x7f9f001026a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009900 tx=0x7f9ef402a410 comp rx=0 tx=0).stop 2024-09-10T21:03:27.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.609+0000 7f9f04999640 1 -- 172.21.15.33:0/1679427086 shutdown_connections 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.609+0000 7f9f04999640 1 --2- 172.21.15.33:0/1679427086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f00102be0 0x7f9f00109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.609+0000 7f9f04999640 1 --2- 172.21.15.33:0/1679427086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f00102220 0x7f9f001026a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.609+0000 7f9f04999640 1 --2- 172.21.15.33:0/1679427086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f001010c0 0x7f9f001014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.609+0000 7f9f04999640 1 -- 172.21.15.33:0/1679427086 >> 172.21.15.33:0/1679427086 conn(0x7f9f000fc990 msgr2=0x7f9f000fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.610+0000 7f9f04999640 1 -- 172.21.15.33:0/1679427086 shutdown_connections 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.610+0000 7f9f04999640 1 -- 172.21.15.33:0/1679427086 wait complete. 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.610+0000 7f9f04999640 1 Processor -- start 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.610+0000 7f9f04999640 1 -- start start 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.610+0000 7f9f04999640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f001010c0 0x7f9f00198e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.610+0000 7f9f04999640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f00102220 0x7f9f00199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:27.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9f04999640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f00102be0 0x7f9f0019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9f04999640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0010e740 con 0x7f9f001010c0 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9f04999640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0010e5c0 con 0x7f9f00102be0 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9f04999640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f0010e8c0 con 0x7f9f00102220 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f001010c0 0x7f9f00198e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f001010c0 0x7f9f00198e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34396/0 (socket says 172.21.15.33:34396) 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 -- 172.21.15.33:0/281016536 learned_addr learned my addr 172.21.15.33:0/281016536 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efdd9b640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f00102220 0x7f9f00199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efed9d640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f00102be0 0x7f9f0019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 -- 172.21.15.33:0/281016536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f00102220 msgr2=0x7f9f00199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:27.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f00102220 0x7f9f00199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 -- 172.21.15.33:0/281016536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f00102be0 msgr2=0x7f9f0019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f00102be0 0x7f9f0019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 -- 172.21.15.33:0/281016536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0019a940 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9efe59c640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f001010c0 0x7f9f00198e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000c970 tx=0x7f9ef000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9eef7fe640 1 -- 172.21.15.33:0/281016536 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0007bb0 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9eef7fe640 1 -- 172.21.15.33:0/281016536 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef0007d50 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9eef7fe640 1 -- 172.21.15.33:0/281016536 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000f450 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0019ac30 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.611+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f001a1310 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.612+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f001014c0 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.614+0000 7f9eef7fe640 1 -- 172.21.15.33:0/281016536 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f9ef0004f00 con 0x7f9f001010c0 2024-09-10T21:03:27.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.614+0000 7f9eef7fe640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ee80b5370 0x7f9ee80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.615+0000 7f9efdd9b640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ee80b5370 0x7f9ee80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.615+0000 7f9eef7fe640 1 -- 172.21.15.33:0/281016536 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9ef00e5f00 con 0x7f9f001010c0 2024-09-10T21:03:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.615+0000 7f9efdd9b640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ee80b5370 0x7f9ee80b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000c00 tx=0x7f9ef4031430 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:27.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.617+0000 7f9eef7fe640 1 -- 172.21.15.33:0/281016536 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9ef0014030 con 0x7f9f001010c0 2024-09-10T21:03:27.634 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:27 smithi104 ceph-mon[35325]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:27 smithi174 ceph-mon[35635]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:27.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.820+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9f00063590 con 0x7f9ee80b5370 2024-09-10T21:03:27.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.821+0000 7f9eef7fe640 1 -- 172.21.15.33:0/281016536 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9f00063590 con 0x7f9ee80b5370 2024-09-10T21:03:27.825 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-09-10T21:03:27.825 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-09-10T21:03:27.825 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-09-10T21:03:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ee80b5370 msgr2=0x7f9ee80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ee80b5370 0x7f9ee80b7830 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4000c00 tx=0x7f9ef4031430 comp rx=0 tx=0).stop 2024-09-10T21:03:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f001010c0 msgr2=0x7f9f00198e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:27.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f001010c0 0x7f9f00198e10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000c970 tx=0x7f9ef000ce30 comp rx=0 tx=0).stop 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 shutdown_connections 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ee80b5370 0x7f9ee80b7830 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f00102be0 0x7f9f0019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f00102220 0x7f9f00199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 --2- 172.21.15.33:0/281016536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f001010c0 0x7f9f00198e10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 >> 172.21.15.33:0/281016536 conn(0x7f9f000fc990 msgr2=0x7f9f00105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 shutdown_connections 2024-09-10T21:03:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:27.823+0000 7f9f04999640 1 -- 172.21.15.33:0/281016536 wait complete. 2024-09-10T21:03:28.629 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:28 smithi033 ceph-mon[30860]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:28.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:28 smithi174 ceph-mon[35635]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:28.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T21:03:28.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:28 smithi104 ceph-mon[35325]: from='client.15153 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:28.929 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:29.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 -- 172.21.15.33:0/1231344802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 msgr2=0x7f123c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 --2- 172.21.15.33:0/1231344802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c102e30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f122c009930 tx=0x7f122c02a420 comp rx=0 tx=0).stop 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 -- 172.21.15.33:0/1231344802 shutdown_connections 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 --2- 172.21.15.33:0/1231344802 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f123c104550 0x7f123c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 --2- 172.21.15.33:0/1231344802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c103b90 0x7f123c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 --2- 172.21.15.33:0/1231344802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c102e30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 -- 172.21.15.33:0/1231344802 >> 172.21.15.33:0/1231344802 conn(0x7f123c0fe320 msgr2=0x7f123c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 -- 172.21.15.33:0/1231344802 shutdown_connections 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.394+0000 7f12428b0640 1 -- 172.21.15.33:0/1231344802 wait complete. 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 Processor -- start 2024-09-10T21:03:29.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 -- start start 2024-09-10T21:03:29.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:29.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f123c103b90 0x7f123c075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:29.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c104550 0x7f123c075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:29.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10e6b0 con 0x7f123c102a30 2024-09-10T21:03:29.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10e530 con 0x7f123c103b90 2024-09-10T21:03:29.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f12428b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f123c10e830 con 0x7f123c104550 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c076cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34416/0 (socket says 172.21.15.33:34416) 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 -- 172.21.15.33:0/1152620734 learned_addr learned my addr 172.21.15.33:0/1152620734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f1240e4d640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c104550 0x7f123c075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123b7fe640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f123c103b90 0x7f123c075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 -- 172.21.15.33:0/1152620734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c104550 msgr2=0x7f123c075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c104550 0x7f123c075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 -- 172.21.15.33:0/1152620734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f123c103b90 msgr2=0x7f123c075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f123c103b90 0x7f123c075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.395+0000 7f123bfff640 1 -- 172.21.15.33:0/1152620734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c076340 con 0x7f123c102a30 2024-09-10T21:03:29.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.396+0000 7f123bfff640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c076cd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f122c005740 tx=0x7f122c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:29.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.396+0000 7f12397fa640 1 -- 172.21.15.33:0/1152620734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c03f030 con 0x7f123c102a30 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.396+0000 7f12397fa640 1 -- 172.21.15.33:0/1152620734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f122c003cb0 con 0x7f123c102a30 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.396+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c1a2ed0 con 0x7f123c102a30 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.396+0000 7f12397fa640 1 -- 172.21.15.33:0/1152620734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c035070 con 0x7f123c102a30 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.396+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1a32c0 con 0x7f123c102a30 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.397+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123c109db0 con 0x7f123c102a30 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.398+0000 7f12397fa640 1 -- 172.21.15.33:0/1152620734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f122c003800 con 0x7f123c102a30 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.400+0000 7f12397fa640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f12240b5370 0x7f12240b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.400+0000 7f123b7fe640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f12240b5370 0x7f12240b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.400+0000 7f12397fa640 1 -- 172.21.15.33:0/1152620734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f122c1065e0 con 0x7f123c102a30 2024-09-10T21:03:29.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.400+0000 7f123b7fe640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f12240b5370 0x7f12240b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1230009b60 tx=0x7f1230005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.404+0000 7f12397fa640 1 -- 172.21.15.33:0/1152620734 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f122c0cd340 con 0x7f123c102a30 2024-09-10T21:03:29.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.597+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f123c063590 con 0x7f12240b5370 2024-09-10T21:03:29.602 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:29 smithi033 ceph-mon[30860]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:29.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.610+0000 7f12397fa640 1 -- 172.21.15.33:0/1152620734 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f123c063590 con 0x7f12240b5370 2024-09-10T21:03:29.613 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.a smithi033 running (101s) 21s ago 101s 17.2M - 19.3.0-4854-g8f219c47 589dd18248e4 be3eb55e3891 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.b smithi104 running (99s) 31s ago 98s 26.7M - 19.3.0-4854-g8f219c47 589dd18248e4 da5a9bb67f8f 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.c smithi174 running (113s) 31s ago 112s 18.2M - 19.3.0-4854-g8f219c47 589dd18248e4 b261a3a6c7b8 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.d smithi033 running (2m) 21s ago 2m 16.1M - 19.3.0-4854-g8f219c47 589dd18248e4 b73e89ef5e36 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.e smithi104 running (118s) 31s ago 118s 18.2M - 19.3.0-4854-g8f219c47 589dd18248e4 97bde8c1f2ee 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.f smithi174 running (115s) 31s ago 115s 18.3M - 19.3.0-4854-g8f219c47 589dd18248e4 121a532764b9 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.g smithi033 running (105s) 21s ago 104s 17.1M - 19.3.0-4854-g8f219c47 589dd18248e4 dd8736cf1487 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.h smithi104 running (110s) 31s ago 110s 18.2M - 19.3.0-4854-g8f219c47 589dd18248e4 4f038929f650 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.i smithi174 running (107s) 31s ago 107s 20.5M - 19.3.0-4854-g8f219c47 589dd18248e4 d48521b7c06e 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.j smithi033 running (2m) 21s ago 2m 16.1M - 19.3.0-4854-g8f219c47 589dd18248e4 99e5230000fc 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.k smithi104 running (2m) 31s ago 2m 17.3M - 19.3.0-4854-g8f219c47 589dd18248e4 f6c1b5f4684a 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mds.l smithi174 running (2m) 31s ago 2m 16.2M - 19.3.0-4854-g8f219c47 589dd18248e4 6a4a62d2e96a 2024-09-10T21:03:29.614 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.x smithi033 *:9283,8765 running (9m) 21s ago 9m 523M - 19.3.0-4854-g8f219c47 589dd18248e4 72b7c416be32 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.y smithi104 *:8443,8765 running (7m) 31s ago 7m 456M - 19.3.0-4854-g8f219c47 589dd18248e4 8307e8746dc8 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.z smithi174 *:8443,8765 running (7m) 31s ago 7m 455M - 19.3.0-4854-g8f219c47 589dd18248e4 2d5a591f6603 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:mon.a smithi033 running (9m) 21s ago 9m 52.3M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 9658efd159e1 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:mon.b smithi104 running (8m) 31s ago 8m 42.4M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 a799043be3ba 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:mon.c smithi174 running (8m) 31s ago 8m 42.7M 2048M 19.3.0-4854-g8f219c47 589dd18248e4 9dea2b2e5d34 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 21s ago 7m 44.7M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 b45e15d32b9b 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi104 running (6m) 31s ago 6m 57.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 b9e81727d449 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi174 running (6m) 31s ago 6m 46.8M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 c4340f3b9e8c 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi033 running (5m) 21s ago 5m 43.7M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 4a332492049f 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi104 running (5m) 31s ago 5m 43.0M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 e2045cce0b83 2024-09-10T21:03:29.615 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi174 running (5m) 31s ago 5m 43.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 85ad21d70817 2024-09-10T21:03:29.616 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 smithi033 running (4m) 21s ago 4m 52.4M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 4b6d04befc55 2024-09-10T21:03:29.616 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 smithi104 running (4m) 31s ago 4m 53.3M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 4e8538348525 2024-09-10T21:03:29.616 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 smithi174 running (3m) 31s ago 3m 42.7M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 601c9c1d1199 2024-09-10T21:03:29.616 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 smithi033 running (3m) 21s ago 3m 40.8M 4273M 19.3.0-4854-g8f219c47 589dd18248e4 9715be0d846f 2024-09-10T21:03:29.616 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 smithi104 running (2m) 31s ago 2m 54.4M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 91dc5f956bae 2024-09-10T21:03:29.616 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 smithi174 running (2m) 31s ago 2m 40.2M 4272M 19.3.0-4854-g8f219c47 589dd18248e4 d8d215bfd1d4 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f12240b5370 msgr2=0x7f12240b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f12240b5370 0x7f12240b7830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1230009b60 tx=0x7f1230005f90 comp rx=0 tx=0).stop 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 msgr2=0x7f123c076cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c076cd0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f122c005740 tx=0x7f122c0042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 shutdown_connections 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f12240b5370 0x7f12240b7830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f123c104550 0x7f123c075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f123c103b90 0x7f123c075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 --2- 172.21.15.33:0/1152620734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f123c102a30 0x7f123c076cd0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:29.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 >> 172.21.15.33:0/1152620734 conn(0x7f123c0fe320 msgr2=0x7f123c106f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:29.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 shutdown_connections 2024-09-10T21:03:29.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:29.612+0000 7f12428b0640 1 -- 172.21.15.33:0/1152620734 wait complete. 2024-09-10T21:03:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:29 smithi174 ceph-mon[35635]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:29 smithi104 ceph-mon[35325]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:30.346 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T21:03:30.611 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:30.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:30 smithi033 ceph-mon[30860]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:30.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:30 smithi104 ceph-mon[35325]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:30.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:30 smithi174 ceph-mon[35635]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 -- 172.21.15.33:0/75671470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40103470 msgr2=0x7f0d40109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 --2- 172.21.15.33:0/75671470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40103470 0x7f0d40109df0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009cd0 tx=0x7f0d3802a8e0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 -- 172.21.15.33:0/75671470 shutdown_connections 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 --2- 172.21.15.33:0/75671470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40103470 0x7f0d40109df0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 --2- 172.21.15.33:0/75671470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40102ab0 0x7f0d40102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 --2- 172.21.15.33:0/75671470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d40101950 0x7f0d40101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 -- 172.21.15.33:0/75671470 >> 172.21.15.33:0/75671470 conn(0x7f0d400fd200 msgr2=0x7f0d400ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 -- 172.21.15.33:0/75671470 shutdown_connections 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 -- 172.21.15.33:0/75671470 wait complete. 2024-09-10T21:03:31.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.074+0000 7f0d481f7640 1 Processor -- start 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d481f7640 1 -- start start 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d481f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40101950 0x7f0d40197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d481f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40102ab0 0x7f0d40198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d481f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d40103470 0x7f0d40199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d481f7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d630 con 0x7f0d40102ab0 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d481f7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d4b0 con 0x7f0d40101950 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d481f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4010d7b0 con 0x7f0d40103470 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40102ab0 0x7f0d40198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40102ab0 0x7f0d40198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46968/0 (socket says 172.21.15.33:46968) 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 -- 172.21.15.33:0/1390393890 learned_addr learned my addr 172.21.15.33:0/1390393890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45f93640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40101950 0x7f0d40197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d46794640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d40103470 0x7f0d40199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:31.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 -- 172.21.15.33:0/1390393890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d40103470 msgr2=0x7f0d40199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d40103470 0x7f0d40199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 -- 172.21.15.33:0/1390393890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40101950 msgr2=0x7f0d40197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40101950 0x7f0d40197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 -- 172.21.15.33:0/1390393890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d40199920 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.075+0000 7f0d45792640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40102ab0 0x7f0d40198440 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000b4a0 tx=0x7f0d3000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.076+0000 7f0d36ffd640 1 -- 172.21.15.33:0/1390393890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30014070 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.076+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d40199c10 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.076+0000 7f0d36ffd640 1 -- 172.21.15.33:0/1390393890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30010ac0 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.076+0000 7f0d36ffd640 1 -- 172.21.15.33:0/1390393890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30017c30 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.076+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401a02f0 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.077+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d40108c70 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.078+0000 7f0d36ffd640 1 -- 172.21.15.33:0/1390393890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f0d30010610 con 0x7f0d40102ab0 2024-09-10T21:03:31.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.079+0000 7f0d36ffd640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b5370 0x7f0d2c0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.079+0000 7f0d45f93640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b5370 0x7f0d2c0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.079+0000 7f0d36ffd640 1 -- 172.21.15.33:0/1390393890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0d300e57b0 con 0x7f0d40102ab0 2024-09-10T21:03:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.080+0000 7f0d45f93640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b5370 0x7f0d2c0b7830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00bbe0 tx=0x7f0d3c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.082+0000 7f0d36ffd640 1 -- 172.21.15.33:0/1390393890 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0d300ac4d0 con 0x7f0d40102ab0 2024-09-10T21:03:31.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.276+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f0d40063590 con 0x7f0d2c0b5370 2024-09-10T21:03:31.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.283+0000 7f0d36ffd640 1 -- 172.21.15.33:0/1390393890 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f0d40063590 con 0x7f0d2c0b5370 2024-09-10T21:03:31.288 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T21:03:31.288 INFO:teuthology.orchestra.run.smithi033.stdout:mds.all 12/12 33s ago 2m smithi033=a;smithi033=d;smithi033=g;smithi033=j;smithi104=b;smithi104=e;smithi104=h;smithi104=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-09-10T21:03:31.288 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 3/3 33s ago 7m smithi033=x;smithi104=y;smithi174=z;count:3 2024-09-10T21:03:31.288 INFO:teuthology.orchestra.run.smithi033.stdout:mon 3/3 33s ago 8m smithi033:172.21.15.33=a;smithi104:172.21.15.104=b;smithi174:172.21.15.174=c;count:3 2024-09-10T21:03:31.288 INFO:teuthology.orchestra.run.smithi033.stdout:osd 12 33s ago - 2024-09-10T21:03:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b5370 msgr2=0x7f0d2c0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b5370 0x7f0d2c0b7830 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00bbe0 tx=0x7f0d3c00b390 comp rx=0 tx=0).stop 2024-09-10T21:03:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40102ab0 msgr2=0x7f0d40198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40102ab0 0x7f0d40198440 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000b4a0 tx=0x7f0d3000b960 comp rx=0 tx=0).stop 2024-09-10T21:03:31.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 shutdown_connections 2024-09-10T21:03:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b5370 0x7f0d2c0b7830 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d40103470 0x7f0d40199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d40102ab0 0x7f0d40198440 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 --2- 172.21.15.33:0/1390393890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40101950 0x7f0d40197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 >> 172.21.15.33:0/1390393890 conn(0x7f0d400fd200 msgr2=0x7f0d40105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 shutdown_connections 2024-09-10T21:03:31.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:31.285+0000 7f0d481f7640 1 -- 172.21.15.33:0/1390393890 wait complete. 2024-09-10T21:03:31.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:31 smithi033 ceph-mon[30860]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:31 smithi104 ceph-mon[35325]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:31 smithi174 ceph-mon[35635]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:32.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T21:03:32.268 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:32.655 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:32 smithi033 ceph-mon[30860]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:32.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 -- 172.21.15.33:0/3127630175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 msgr2=0x7f0c6810a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:32.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 --2- 172.21.15.33:0/3127630175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6810a360 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009900 tx=0x7f0c5802a410 comp rx=0 tx=0).stop 2024-09-10T21:03:32.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 -- 172.21.15.33:0/3127630175 shutdown_connections 2024-09-10T21:03:32.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 --2- 172.21.15.33:0/3127630175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6810a980 0x7f0c6810ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 --2- 172.21.15.33:0/3127630175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6810a360 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 --2- 172.21.15.33:0/3127630175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c6806bfb0 0x7f0c68107a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 -- 172.21.15.33:0/3127630175 >> 172.21.15.33:0/3127630175 conn(0x7f0c680fd200 msgr2=0x7f0c680ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 -- 172.21.15.33:0/3127630175 shutdown_connections 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 -- 172.21.15.33:0/3127630175 wait complete. 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.736+0000 7f0c706ff640 1 Processor -- start 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c706ff640 1 -- start start 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c706ff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c6806bfb0 0x7f0c6819afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c706ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c706ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6810a980 0x7f0c6819c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c706ff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0c681108d0 con 0x7f0c68107f70 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c706ff640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68110750 con 0x7f0c6806bfb0 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c706ff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c68110a50 con 0x7f0c6810a980 2024-09-10T21:03:32.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6819b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46990/0 (socket says 172.21.15.33:46990) 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 -- 172.21.15.33:0/2143399138 learned_addr learned my addr 172.21.15.33:0/2143399138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6e49b640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c6806bfb0 0x7f0c6819afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 -- 172.21.15.33:0/2143399138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6810a980 msgr2=0x7f0c6819c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6810a980 0x7f0c6819c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 -- 172.21.15.33:0/2143399138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c6806bfb0 msgr2=0x7f0c6819afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c6806bfb0 0x7f0c6819afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 -- 172.21.15.33:0/2143399138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c6819caf0 con 0x7f0c68107f70 2024-09-10T21:03:32.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.737+0000 7f0c6dc9a640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6819b610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c58006010 tx=0x7f0c580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.738+0000 7f0c5f7fe640 1 -- 172.21.15.33:0/2143399138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5803f030 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.738+0000 7f0c5f7fe640 1 -- 172.21.15.33:0/2143399138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c58003cb0 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.738+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c6819cd80 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.738+0000 7f0c5f7fe640 1 -- 172.21.15.33:0/2143399138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c58035070 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.738+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c681a34c0 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.739+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c680fecf0 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.740+0000 7f0c5f7fe640 1 -- 172.21.15.33:0/2143399138 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f0c5803bb60 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.741+0000 7f0c5f7fe640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c540b53f0 0x7f0c540b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.741+0000 7f0c6e49b640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c540b53f0 0x7f0c540b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.741+0000 7f0c5f7fe640 1 -- 172.21.15.33:0/2143399138 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0c58106ab0 con 0x7f0c68107f70 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.741+0000 7f0c6e49b640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c540b53f0 0x7f0c540b78b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005e10 tx=0x7f0c64005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:32.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.744+0000 7f0c5f7fe640 1 -- 172.21.15.33:0/2143399138 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f0c580cd810 con 0x7f0c68107f70 2024-09-10T21:03:32.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:32 smithi104 ceph-mon[35325]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:32.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:32 smithi174 ceph-mon[35635]: from='client.15165 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.935+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f0c68063590 con 0x7f0c540b53f0 2024-09-10T21:03:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.936+0000 7f0c5f7fe640 1 -- 172.21.15.33:0/2143399138 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f0c68063590 con 0x7f0c540b53f0 2024-09-10T21:03:32.940 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-09-10T21:03:32.940 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-09-10T21:03:32.940 INFO:teuthology.orchestra.run.smithi033.stdout:smithi104 172.21.15.104 2024-09-10T21:03:32.940 INFO:teuthology.orchestra.run.smithi033.stdout:smithi174 172.21.15.174 2024-09-10T21:03:32.940 INFO:teuthology.orchestra.run.smithi033.stdout:3 hosts in cluster 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c540b53f0 msgr2=0x7f0c540b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c540b53f0 0x7f0c540b78b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c64005e10 tx=0x7f0c64005da0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 msgr2=0x7f0c6819b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6819b610 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c58006010 tx=0x7f0c580042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 shutdown_connections 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0c540b53f0 0x7f0c540b78b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c6810a980 0x7f0c6819c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c68107f70 0x7f0c6819b610 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 --2- 172.21.15.33:0/2143399138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c6806bfb0 0x7f0c6819afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 >> 172.21.15.33:0/2143399138 conn(0x7f0c680fd200 msgr2=0x7f0c68108b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 shutdown_connections 2024-09-10T21:03:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:32.939+0000 7f0c706ff640 1 -- 172.21.15.33:0/2143399138 wait complete. 2024-09-10T21:03:33.603 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:33 smithi033 ceph-mon[30860]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:33.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T21:03:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:33 smithi104 ceph-mon[35325]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:33.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:33 smithi174 ceph-mon[35635]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:03:33.897 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 -- 172.21.15.33:0/1596660124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 msgr2=0x7f46b410aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/1596660124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b410aee0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009cd0 tx=0x7f46ac02a8e0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 -- 172.21.15.33:0/1596660124 shutdown_connections 2024-09-10T21:03:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/1596660124 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b410aee0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/1596660124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b4103ba0 0x7f46b4104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/1596660124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b4102a40 0x7f46b4102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 -- 172.21.15.33:0/1596660124 >> 172.21.15.33:0/1596660124 conn(0x7f46b40fe2f0 msgr2=0x7f46b4100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 -- 172.21.15.33:0/1596660124 shutdown_connections 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.386+0000 7f46ba0e1640 1 -- 172.21.15.33:0/1596660124 wait complete. 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 Processor -- start 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 -- start start 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b4102a40 0x7f46b4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b4103ba0 0x7f46b41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46b410e720 con 0x7f46b4104560 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46b410e5a0 con 0x7f46b4102a40 2024-09-10T21:03:34.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46ba0e1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46b410e8a0 con 0x7f46b4103ba0 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b419a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47008/0 (socket says 172.21.15.33:47008) 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 -- 172.21.15.33:0/3620260925 learned_addr learned my addr 172.21.15.33:0/3620260925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b37fe640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b4102a40 0x7f46b4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 -- 172.21.15.33:0/3620260925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b4103ba0 msgr2=0x7f46b41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b4103ba0 0x7f46b41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 -- 172.21.15.33:0/3620260925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b4102a40 msgr2=0x7f46b4198e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b4102a40 0x7f46b4198e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.387+0000 7f46b3fff640 1 -- 172.21.15.33:0/3620260925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b419a990 con 0x7f46b4104560 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.388+0000 7f46b3fff640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b419a0f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0317d0 tx=0x7f46ac003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.388+0000 7f46b0ff9640 1 -- 172.21.15.33:0/3620260925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac004450 con 0x7f46b4104560 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.388+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b419ac20 con 0x7f46b4104560 2024-09-10T21:03:34.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.388+0000 7f46b0ff9640 1 -- 172.21.15.33:0/3620260925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46ac0045f0 con 0x7f46b4104560 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.388+0000 7f46b0ff9640 1 -- 172.21.15.33:0/3620260925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac045670 con 0x7f46b4104560 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.388+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b41a1360 con 0x7f46b4104560 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.389+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b4109cf0 con 0x7f46b4104560 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.390+0000 7f46b0ff9640 1 -- 172.21.15.33:0/3620260925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f46ac04d070 con 0x7f46b4104560 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.391+0000 7f46b0ff9640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46a00b5370 0x7f46a00b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.391+0000 7f46b37fe640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46a00b5370 0x7f46a00b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.391+0000 7f46b0ff9640 1 -- 172.21.15.33:0/3620260925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f46ac02e030 con 0x7f46b4104560 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.391+0000 7f46b37fe640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46a00b5370 0x7f46a00b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f469c009890 tx=0x7f469c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.394+0000 7f46b0ff9640 1 -- 172.21.15.33:0/3620260925 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f46ac04a060 con 0x7f46b4104560 2024-09-10T21:03:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.588+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f46b4063590 con 0x7f46a00b5370 2024-09-10T21:03:34.593 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:34 smithi033 ceph-mon[30860]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.590+0000 7f46b0ff9640 1 -- 172.21.15.33:0/3620260925 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f46b4063590 con 0x7f46a00b5370 2024-09-10T21:03:34.594 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T21:03:34.594 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-10T21:03:34.594 INFO:teuthology.orchestra.run.smithi033.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T21:03:34.594 INFO:teuthology.orchestra.run.smithi033.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-10T21:03:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46a00b5370 msgr2=0x7f46a00b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46a00b5370 0x7f46a00b7830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f469c009890 tx=0x7f469c008040 comp rx=0 tx=0).stop 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 msgr2=0x7f46b419a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b419a0f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0317d0 tx=0x7f46ac003ce0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 shutdown_connections 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46a00b5370 0x7f46a00b7830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46b4104560 0x7f46b419a0f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46b4103ba0 0x7f46b41994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 --2- 172.21.15.33:0/3620260925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46b4102a40 0x7f46b4198e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.592+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 >> 172.21.15.33:0/3620260925 conn(0x7f46b40fe2f0 msgr2=0x7f46b4106ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.593+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 shutdown_connections 2024-09-10T21:03:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:34.593+0000 7f46ba0e1640 1 -- 172.21.15.33:0/3620260925 wait complete. 2024-09-10T21:03:34.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:34 smithi104 ceph-mon[35325]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:34.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:34 smithi174 ceph-mon[35635]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:35.433 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T21:03:35.442 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-09-10T21:03:35.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T21:03:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:35 smithi033 ceph-mon[30860]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:35 smithi033 ceph-mon[30860]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:35.708 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:35 smithi104 ceph-mon[35325]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:35 smithi104 ceph-mon[35325]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:35 smithi174 ceph-mon[35635]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:03:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:35 smithi174 ceph-mon[35635]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T21:03:36.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 -- 172.21.15.33:0/2690742231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 msgr2=0x7f1fc4101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:36.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 --2- 172.21.15.33:0/2690742231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4101d40 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009930 tx=0x7f1fb402a420 comp rx=0 tx=0).stop 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 -- 172.21.15.33:0/2690742231 shutdown_connections 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 --2- 172.21.15.33:0/2690742231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc4103460 0x7f1fc4109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 --2- 172.21.15.33:0/2690742231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc4102aa0 0x7f1fc4102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 --2- 172.21.15.33:0/2690742231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4101d40 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 -- 172.21.15.33:0/2690742231 >> 172.21.15.33:0/2690742231 conn(0x7f1fc40fd210 msgr2=0x7f1fc40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 -- 172.21.15.33:0/2690742231 shutdown_connections 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.197+0000 7f1fcb687640 1 -- 172.21.15.33:0/2690742231 wait complete. 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 Processor -- start 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 -- start start 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4198e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc4102aa0 0x7f1fc41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:36.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc4103460 0x7f1fc419a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc410e6b0 con 0x7f1fc4101940 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc410e530 con 0x7f1fc4103460 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fcb687640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc410e830 con 0x7f1fc4102aa0 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4198e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4198e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47030/0 (socket says 172.21.15.33:47030) 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 -- 172.21.15.33:0/1809221269 learned_addr learned my addr 172.21.15.33:0/1809221269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9c24640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc4103460 0x7f1fc419a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc8c22640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc4102aa0 0x7f1fc41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 -- 172.21.15.33:0/1809221269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc4102aa0 msgr2=0x7f1fc41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc4102aa0 0x7f1fc41994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 -- 172.21.15.33:0/1809221269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc4103460 msgr2=0x7f1fc419a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc4103460 0x7f1fc419a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.198+0000 7f1fc9423640 1 -- 172.21.15.33:0/1809221269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc419a990 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.199+0000 7f1fc9423640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4198e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4005ab0 tx=0x7f1fb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.199+0000 7f1fc27fc640 1 -- 172.21.15.33:0/1809221269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb403f030 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.199+0000 7f1fc27fc640 1 -- 172.21.15.33:0/1809221269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb4003cb0 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.199+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc419ac20 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.199+0000 7f1fc27fc640 1 -- 172.21.15.33:0/1809221269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4035070 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.199+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc41a1360 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.200+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc4101d40 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.201+0000 7f1fc27fc640 1 -- 172.21.15.33:0/1809221269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f1fb403bc20 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.202+0000 7f1fc27fc640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1fac0b53f0 0x7f1fac0b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.202+0000 7f1fc8c22640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1fac0b53f0 0x7f1fac0b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.202+0000 7f1fc27fc640 1 -- 172.21.15.33:0/1809221269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1fb4106930 con 0x7f1fc4101940 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.203+0000 7f1fc8c22640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1fac0b53f0 0x7f1fac0b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005d00 tx=0x7f1fb800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:36.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.205+0000 7f1fc27fc640 1 -- 172.21.15.33:0/1809221269 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1fb40cd000 con 0x7f1fc4101940 2024-09-10T21:03:36.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.429+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f1fc402e5c0 con 0x7f1fc4101940 2024-09-10T21:03:36.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.429+0000 7f1fc27fc640 1 -- 172.21.15.33:0/1809221269 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3128 (secure 0 0 0) 0x7f1fb40d1a30 con 0x7f1fc4101940 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:e27 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:btime 2024-09-10T21:02:12:865816+0000 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 27 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-09-10T21:02:01.445404+0000 2024-09-10T21:03:36.434 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-09-10T21:02:12.865787+0000 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 300 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 5 2024-09-10T21:03:36.435 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1,2,3,4 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=24566,1=24547,2=24539,3=24532,4=24526} 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 24566 members: 24566,24547,24532,24539,24526 2024-09-10T21:03:36.436 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.b{0:24566} state up:active seq 4 addr [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.e{0:24518} state up:standby-replay seq 1 addr [v2:172.21.15.104:6834/1051931570,v1:172.21.15.104:6835/1051931570] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.i{1:24547} state up:active seq 8 addr [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.l{1:24505} state up:standby-replay seq 1 addr [v2:172.21.15.174:6832/34665907,v1:172.21.15.174:6833/34665907] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.h{2:24539} state up:active seq 9 addr [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.k{2:24494} state up:standby-replay seq 1 addr [v2:172.21.15.104:6832/2049405106,v1:172.21.15.104:6833/2049405106] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.c{3:24532} state up:active seq 10 addr [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.a{3:14709} state up:standby-replay seq 1 addr [v2:172.21.15.33:6840/1476321443,v1:172.21.15.33:6841/1476321443] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.f{4:24526} state up:active seq 11 addr [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.g{4:14697} state up:standby-replay seq 1 addr [v2:172.21.15.33:6838/295723844,v1:172.21.15.33:6839/295723844] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-09-10T21:03:36.437 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.j{-1:14637} state up:standby seq 1 addr [v2:172.21.15.33:6834/758676630,v1:172.21.15.33:6835/758676630] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.438 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.d{-1:14649} state up:standby seq 1 addr [v2:172.21.15.33:6836/1550204890,v1:172.21.15.33:6837/1550204890] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T21:03:36.438 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.431+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1fac0b53f0 msgr2=0x7f1fac0b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.431+0000 7f1fcb687640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1fac0b53f0 0x7f1fac0b78b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8005d00 tx=0x7f1fb800b410 comp rx=0 tx=0).stop 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.431+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 msgr2=0x7f1fc4198e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.431+0000 7f1fcb687640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4198e60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4005ab0 tx=0x7f1fb40042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 shutdown_connections 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1fac0b53f0 0x7f1fac0b78b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1fc4103460 0x7f1fc419a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc4102aa0 0x7f1fc41994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 --2- 172.21.15.33:0/1809221269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fc4101940 0x7f1fc4198e60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 >> 172.21.15.33:0/1809221269 conn(0x7f1fc40fd210 msgr2=0x7f1fc4105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:36.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 shutdown_connections 2024-09-10T21:03:36.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:36.432+0000 7f1fcb687640 1 -- 172.21.15.33:0/1809221269 wait complete. 2024-09-10T21:03:37.250 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-10T21:03:37.514 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:37.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:37 smithi033 ceph-mon[30860]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:37.635 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:37 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1809221269' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T21:03:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:37 smithi104 ceph-mon[35325]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:37 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1809221269' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T21:03:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:37 smithi174 ceph-mon[35635]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:37 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1809221269' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-10T21:03:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 -- 172.21.15.33:0/170846480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc104560 msgr2=0x7f59bc10aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 --2- 172.21.15.33:0/170846480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc104560 0x7f59bc10aee0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009cd0 tx=0x7f59b402a8e0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 -- 172.21.15.33:0/170846480 shutdown_connections 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 --2- 172.21.15.33:0/170846480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc104560 0x7f59bc10aee0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 --2- 172.21.15.33:0/170846480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59bc103ba0 0x7f59bc104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 --2- 172.21.15.33:0/170846480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59bc102a40 0x7f59bc102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 -- 172.21.15.33:0/170846480 >> 172.21.15.33:0/170846480 conn(0x7f59bc0fe2f0 msgr2=0x7f59bc100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 -- 172.21.15.33:0/170846480 shutdown_connections 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.007+0000 7f59c42cd640 1 -- 172.21.15.33:0/170846480 wait complete. 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.008+0000 7f59c42cd640 1 Processor -- start 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.008+0000 7f59c42cd640 1 -- start start 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.008+0000 7f59c42cd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59bc102a40 0x7f59bc076cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:38.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c42cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc103ba0 0x7f59bc075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c42cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59bc104560 0x7f59bc075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c42cd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10e710 con 0x7f59bc103ba0 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c42cd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10e590 con 0x7f59bc102a40 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c42cd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59bc10e890 con 0x7f59bc104560 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc103ba0 0x7f59bc075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc103ba0 0x7f59bc075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47042/0 (socket says 172.21.15.33:47042) 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 -- 172.21.15.33:0/4268795467 learned_addr learned my addr 172.21.15.33:0/4268795467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c2069640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59bc102a40 0x7f59bc076cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c286a640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59bc104560 0x7f59bc075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 -- 172.21.15.33:0/4268795467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59bc104560 msgr2=0x7f59bc075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59bc104560 0x7f59bc075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 -- 172.21.15.33:0/4268795467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59bc102a40 msgr2=0x7f59bc076cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59bc102a40 0x7f59bc076cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 -- 172.21.15.33:0/4268795467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59bc076340 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c286a640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59bc104560 0x7f59bc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c1868640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc103ba0 0x7f59bc075540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f59b000d770 tx=0x7f59b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59af7fe640 1 -- 172.21.15.33:0/4268795467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0017070 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59bc1a2e60 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59af7fe640 1 -- 172.21.15.33:0/4268795467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b00054d0 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59af7fe640 1 -- 172.21.15.33:0/4268795467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0004c30 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.009+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59bc1a32c0 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.010+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59bc109d60 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.012+0000 7f59af7fe640 1 -- 172.21.15.33:0/4268795467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7f59b0004dd0 con 0x7f59bc103ba0 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.013+0000 7f59af7fe640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f59a80b5370 0x7f59a80b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:38.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.013+0000 7f59c2069640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f59a80b5370 0x7f59a80b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.013+0000 7f59af7fe640 1 -- 172.21.15.33:0/4268795467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f59b00e5c00 con 0x7f59bc103ba0 2024-09-10T21:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.013+0000 7f59c2069640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f59a80b5370 0x7f59a80b7830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f59b800bbe0 tx=0x7f59b800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:38.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.015+0000 7f59af7fe640 1 -- 172.21.15.33:0/4268795467 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f59b00ac960 con 0x7f59bc103ba0 2024-09-10T21:03:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.206+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f59bc02e7c0 con 0x7f59bc103ba0 2024-09-10T21:03:38.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.207+0000 7f59af7fe640 1 -- 172.21.15.33:0/4268795467 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5309 (secure 0 0 0) 0x7f59b00ac260 con 0x7f59bc103ba0 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 77 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-09-10T20:53:48.210364+0000 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-09-10T21:02:41.855317+0000 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:crush_version 26 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:full_ratio 0.95 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:backfillfull_ratio 0.9 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:nearfull_ratio 0.85 2024-09-10T21:03:38.211 INFO:teuthology.orchestra.run.smithi033.stdout:require_min_compat_client luminous 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:min_compat_client luminous 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:require_osd_release squid 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:stretch_mode_enabled false 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:max_osd 12 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 up in weight 1 up_from 8 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6802/3008534631,v1:172.21.15.33:6803/3008534631] [v2:172.21.15.33:6804/3008534631,v1:172.21.15.33:6805/3008534631] exists,up 343a51e3-d26a-4f11-9714-d6464a31c878 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 up in weight 1 up_from 13 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6800/2112783762,v1:172.21.15.104:6801/2112783762] [v2:172.21.15.104:6802/2112783762,v1:172.21.15.104:6803/2112783762] exists,up a51a4510-302a-4343-8cce-ee4b0c562880 2024-09-10T21:03:38.212 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 up in weight 1 up_from 17 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/4030111255,v1:172.21.15.174:6801/4030111255] [v2:172.21.15.174:6802/4030111255,v1:172.21.15.174:6803/4030111255] exists,up ae175f96-8529-4db0-83bd-7ee10b49dddf 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 up in weight 1 up_from 24 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6810/2053264902,v1:172.21.15.33:6811/2053264902] [v2:172.21.15.33:6812/2053264902,v1:172.21.15.33:6813/2053264902] exists,up 87f711da-d77c-452d-9a29-3418de2ba617 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 up in weight 1 up_from 30 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6808/3963281579,v1:172.21.15.104:6809/3963281579] [v2:172.21.15.104:6810/3963281579,v1:172.21.15.104:6811/3963281579] exists,up 02b1eb40-cf26-4cee-9c4b-c5a353a8537a 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 up in weight 1 up_from 35 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/1209422453,v1:172.21.15.174:6809/1209422453] [v2:172.21.15.174:6810/1209422453,v1:172.21.15.174:6811/1209422453] exists,up e7d7361f-8c71-472e-84a7-bedf288dc9bd 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.6 up in weight 1 up_from 41 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6818/3359699940,v1:172.21.15.33:6819/3359699940] [v2:172.21.15.33:6820/3359699940,v1:172.21.15.33:6821/3359699940] exists,up 03e0706e-6a19-42c7-97c4-54dbe8877a69 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.7 up in weight 1 up_from 46 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6816/947117996,v1:172.21.15.104:6817/947117996] [v2:172.21.15.104:6818/947117996,v1:172.21.15.104:6819/947117996] exists,up 5f9abb9d-783e-4ded-93ea-4604eb6df315 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.8 up in weight 1 up_from 52 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/758997036,v1:172.21.15.174:6817/758997036] [v2:172.21.15.174:6818/758997036,v1:172.21.15.174:6819/758997036] exists,up f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.9 up in weight 1 up_from 58 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.33:6826/3501032223,v1:172.21.15.33:6827/3501032223] [v2:172.21.15.33:6828/3501032223,v1:172.21.15.33:6829/3501032223] exists,up dbae656c-994b-4a32-a22d-de19664248e9 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.10 up in weight 1 up_from 63 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.104:6824/575213460,v1:172.21.15.104:6825/575213460] [v2:172.21.15.104:6826/575213460,v1:172.21.15.104:6827/575213460] exists,up f020be25-441a-4f54-8077-3d0ea98e1181 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/117399142,v1:172.21.15.174:6825/117399142] [v2:172.21.15.174:6826/117399142,v1:172.21.15.174:6827/117399142] exists,up 9b6009d5-69cd-4c9c-9387-bc1fdceb041d 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.5 [8,11] 2024-09-10T21:03:38.213 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.10 [2,1] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1b [2,4] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 2.1d [8,11] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.14 [4,6] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.1b [5,9] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.1d [4,8] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.2d [3,9] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:pg_upmap_items 3.34 [4,9] 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2096552792 expires 2024-09-11T20:54:39.992584+0000 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/2468168068 expires 2024-09-11T20:54:39.992584+0000 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/2468168068 expires 2024-09-11T20:54:39.992584+0000 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2596082174 expires 2024-09-11T20:54:14.879196+0000 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2930837239 expires 2024-09-11T20:54:39.992584+0000 2024-09-10T21:03:38.214 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6801/1151081438 expires 2024-09-11T20:54:14.879196+0000 2024-09-10T21:03:38.215 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2648657406 expires 2024-09-11T20:54:39.992584+0000 2024-09-10T21:03:38.215 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:6800/1151081438 expires 2024-09-11T20:54:14.879196+0000 2024-09-10T21:03:38.215 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/4287278386 expires 2024-09-11T20:54:14.879196+0000 2024-09-10T21:03:38.215 INFO:teuthology.orchestra.run.smithi033.stdout:blocklist 172.21.15.33:0/2461327124 expires 2024-09-11T20:54:14.879196+0000 2024-09-10T21:03:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f59a80b5370 msgr2=0x7f59a80b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f59a80b5370 0x7f59a80b7830 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f59b800bbe0 tx=0x7f59b800b390 comp rx=0 tx=0).stop 2024-09-10T21:03:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc103ba0 msgr2=0x7f59bc075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc103ba0 0x7f59bc075540 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f59b000d770 tx=0x7f59b000dc30 comp rx=0 tx=0).stop 2024-09-10T21:03:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 shutdown_connections 2024-09-10T21:03:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f59a80b5370 0x7f59a80b7830 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59bc104560 0x7f59bc075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59bc103ba0 0x7f59bc075540 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 --2- 172.21.15.33:0/4268795467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59bc102a40 0x7f59bc076cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 >> 172.21.15.33:0/4268795467 conn(0x7f59bc0fe2f0 msgr2=0x7f59bc106f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 shutdown_connections 2024-09-10T21:03:38.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:38.209+0000 7f59c42cd640 1 -- 172.21.15.33:0/4268795467 wait complete. 2024-09-10T21:03:39.049 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-10T21:03:39.061 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:03:39.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:39 smithi104 ceph-mon[35325]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:39.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:39 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4268795467' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T21:03:39.325 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:39.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:39 smithi174 ceph-mon[35635]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:39.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:39 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4268795467' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T21:03:39.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:39 smithi033 ceph-mon[30860]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:03:39.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:39 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4268795467' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-10T21:03:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 -- 172.21.15.33:0/1591764458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0101c90 msgr2=0x7fcfc0102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:39.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 --2- 172.21.15.33:0/1591764458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0101c90 0x7fcfc0102070 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0009900 tx=0x7fcfb002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 -- 172.21.15.33:0/1591764458 shutdown_connections 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 --2- 172.21.15.33:0/1591764458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc0108020 0x7fcfc010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 --2- 172.21.15.33:0/1591764458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfc01076a0 0x7fcfc0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 --2- 172.21.15.33:0/1591764458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0101c90 0x7fcfc0102070 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 -- 172.21.15.33:0/1591764458 >> 172.21.15.33:0/1591764458 conn(0x7fcfc00fd700 msgr2=0x7fcfc00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 -- 172.21.15.33:0/1591764458 shutdown_connections 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.818+0000 7fcfc6511640 1 -- 172.21.15.33:0/1591764458 wait complete. 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 Processor -- start 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 -- start start 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc0101c90 0x7fcfc019a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfc01076a0 0x7fcfc019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:39.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0108020 0x7fcfc01a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0111ce0 con 0x7fcfc0108020 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0111b60 con 0x7fcfc0101c90 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc6511640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0111e60 con 0x7fcfc01076a0 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc4aae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0108020 0x7fcfc01a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc4aae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0108020 0x7fcfc01a5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40192/0 (socket says 172.21.15.33:40192) 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.819+0000 7fcfc4aae640 1 -- 172.21.15.33:0/3938494442 learned_addr learned my addr 172.21.15.33:0/3938494442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfb7fff640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfc01076a0 0x7fcfc019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfbffff640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc0101c90 0x7fcfc019a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc4aae640 1 -- 172.21.15.33:0/3938494442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfc01076a0 msgr2=0x7fcfc019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc4aae640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfc01076a0 0x7fcfc019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc4aae640 1 -- 172.21.15.33:0/3938494442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc0101c90 msgr2=0x7fcfc019a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:39.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc4aae640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc0101c90 0x7fcfc019a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc4aae640 1 -- 172.21.15.33:0/3938494442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc01a5670 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc4aae640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0108020 0x7fcfc01a5130 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800eaa0 tx=0x7fcfb800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfbdffb640 1 -- 172.21.15.33:0/3938494442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800cdb0 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc01a5840 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfbdffb640 1 -- 172.21.15.33:0/3938494442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfb8005020 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfbdffb640 1 -- 172.21.15.33:0/3938494442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80116d0 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.820+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc01a5cd0 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.821+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc0102070 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.823+0000 7fcfbdffb640 1 -- 172.21.15.33:0/3938494442 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fcfb800f040 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.823+0000 7fcfbdffb640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcfac0b5370 0x7fcfac0b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.823+0000 7fcfbffff640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcfac0b5370 0x7fcfac0b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.823+0000 7fcfbdffb640 1 -- 172.21.15.33:0/3938494442 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcfb80e5ec0 con 0x7fcfc0108020 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.824+0000 7fcfbffff640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcfac0b5370 0x7fcfac0b7830 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0031440 tx=0x7fcfb0005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:39.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:39.826+0000 7fcfbdffb640 1 -- 172.21.15.33:0/3938494442 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcfb8016030 con 0x7fcfc0108020 2024-09-10T21:03:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.053+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfc006af20 con 0x7fcfc0108020 2024-09-10T21:03:40.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.053+0000 7fcfbdffb640 1 -- 172.21.15.33:0/3938494442 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11973 (secure 0 0 0) 0x7fcfb80ac5d0 con 0x7fcfc0108020 2024-09-10T21:03:40.057 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:40.058 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"btime":"2024-09-10T21:02:12:865816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:02:12.865787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:03:40.058 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-09-10T21:03:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcfac0b5370 msgr2=0x7fcfac0b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcfac0b5370 0x7fcfac0b7830 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0031440 tx=0x7fcfb0005b50 comp rx=0 tx=0).stop 2024-09-10T21:03:40.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0108020 msgr2=0x7fcfc01a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0108020 0x7fcfc01a5130 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800eaa0 tx=0x7fcfb800ef60 comp rx=0 tx=0).stop 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 shutdown_connections 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcfac0b5370 0x7fcfac0b7830 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfc0108020 0x7fcfc01a5130 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfc01076a0 0x7fcfc019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 --2- 172.21.15.33:0/3938494442 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcfc0101c90 0x7fcfc019a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 >> 172.21.15.33:0/3938494442 conn(0x7fcfc00fd700 msgr2=0x7fcfc00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 shutdown_connections 2024-09-10T21:03:40.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:40.056+0000 7fcfc6511640 1 -- 172.21.15.33:0/3938494442 wait complete. 2024-09-10T21:03:40.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:40 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3938494442' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:40.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:40 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3938494442' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:40.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:40 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3938494442' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:40.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:03:41.034 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:41.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:41 smithi174 ceph-mon[35635]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:41.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:41 smithi033 ceph-mon[30860]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2037869097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb33410d0e0 msgr2=0x7fb33410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:41.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2037869097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb33410d0e0 0x7fb33410f550 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009ca0 tx=0x7fb32c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2037869097 shutdown_connections 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2037869097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb33410d0e0 0x7fb33410f550 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2037869097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb334104530 0x7fb33410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2037869097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb334103b30 0x7fb334103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2037869097 >> 172.21.15.33:0/2037869097 conn(0x7fb3340fd700 msgr2=0x7fb3340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2037869097 shutdown_connections 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.509+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2037869097 wait complete. 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 Processor -- start 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 -- start start 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334103b30 0x7fb334199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb334104530 0x7fb334199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:41.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb33410d0e0 0x7fb33419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb334112d60 con 0x7fb334103b30 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb334112be0 con 0x7fb334104530 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33b5f2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb334112ee0 con 0x7fb33410d0e0 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334103b30 0x7fb334199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334103b30 0x7fb334199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40216/0 (socket says 172.21.15.33:40216) 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 -- 172.21.15.33:0/2306253988 learned_addr learned my addr 172.21.15.33:0/2306253988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb339b8f640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb33410d0e0 0x7fb33419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb338b8d640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb334104530 0x7fb334199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 -- 172.21.15.33:0/2306253988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb33410d0e0 msgr2=0x7fb33419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb33410d0e0 0x7fb33419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 -- 172.21.15.33:0/2306253988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb334104530 msgr2=0x7fb334199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:41.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb334104530 0x7fb334199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb33938e640 1 -- 172.21.15.33:0/2306253988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33419f2c0 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb339b8f640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb33410d0e0 0x7fb33419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.510+0000 7fb338b8d640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb334104530 0x7fb334199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.511+0000 7fb33938e640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334103b30 0x7fb334199210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e9f0 tx=0x7fb33000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.511+0000 7fb32a7fc640 1 -- 172.21.15.33:0/2306253988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000cd80 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.511+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33419f5b0 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.511+0000 7fb32a7fc640 1 -- 172.21.15.33:0/2306253988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3300054d0 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.511+0000 7fb32a7fc640 1 -- 172.21.15.33:0/2306253988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330010690 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.511+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3341a5c70 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.512+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb334103f10 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.514+0000 7fb32a7fc640 1 -- 172.21.15.33:0/2306253988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) ==== 153301+0+0 (secure 0 0 0) 0x7fb330004e50 con 0x7fb334103b30 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.515+0000 7fb32a7fc640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3200b5370 0x7fb3200b7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.515+0000 7fb338b8d640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3200b5370 0x7fb3200b7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:41.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.515+0000 7fb32a7fc640 1 -- 172.21.15.33:0/2306253988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb3300e64e0 con 0x7fb334103b30 2024-09-10T21:03:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.516+0000 7fb338b8d640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3200b5370 0x7fb3200b7830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3340fff70 tx=0x7fb324005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:41.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.518+0000 7fb32a7fc640 1 -- 172.21.15.33:0/2306253988 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb330056220 con 0x7fb334103b30 2024-09-10T21:03:41.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:41 smithi104 ceph-mon[35325]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:03:41.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.738+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb33402e5c0 con 0x7fb334103b30 2024-09-10T21:03:41.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.739+0000 7fb32a7fc640 1 -- 172.21.15.33:0/2306253988 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11973 (secure 0 0 0) 0x7fb3300ad200 con 0x7fb334103b30 2024-09-10T21:03:41.743 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:41.743 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"btime":"2024-09-10T21:02:12:865816+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:02:12.865787+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:03:41.743 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-09-10T21:03:41.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3200b5370 msgr2=0x7fb3200b7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:41.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3200b5370 0x7fb3200b7830 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb3340fff70 tx=0x7fb324005d20 comp rx=0 tx=0).stop 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334103b30 msgr2=0x7fb334199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334103b30 0x7fb334199210 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb33000e9f0 tx=0x7fb33000edf0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 shutdown_connections 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3200b5370 0x7fb3200b7830 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb33410d0e0 0x7fb33419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb334104530 0x7fb334199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 --2- 172.21.15.33:0/2306253988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb334103b30 0x7fb334199210 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 >> 172.21.15.33:0/2306253988 conn(0x7fb3340fd700 msgr2=0x7fb3340fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 shutdown_connections 2024-09-10T21:03:41.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:41.741+0000 7fb33b5f2640 1 -- 172.21.15.33:0/2306253988 wait complete. 2024-09-10T21:03:42.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:42 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2306253988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:42.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:42 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2306253988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:42.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:03:42.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:42 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2306253988' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:03:42.753 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:43.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 -- 172.21.15.33:0/1369081384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 msgr2=0x7f56d8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:43.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/1369081384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8108c10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f56c0009900 tx=0x7f56c002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:43.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 -- 172.21.15.33:0/1369081384 shutdown_connections 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/1369081384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d8109150 0x7f56d810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/1369081384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8108c10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/1369081384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d8102dc0 0x7f56d81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 -- 172.21.15.33:0/1369081384 >> 172.21.15.33:0/1369081384 conn(0x7f56d80fe830 msgr2=0x7f56d8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 -- 172.21.15.33:0/1369081384 shutdown_connections 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.218+0000 7f56dcdb0640 1 -- 172.21.15.33:0/1369081384 wait complete. 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 Processor -- start 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 -- start start 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d8102dc0 0x7f56d81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d8109150 0x7f56d819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56d8112e10 con 0x7f56d81087d0 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56d8112c90 con 0x7f56d8109150 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56dcdb0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56d8112f90 con 0x7f56d8102dc0 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56cdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56cdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40228/0 (socket says 172.21.15.33:40228) 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56cdd9b640 1 -- 172.21.15.33:0/2183429418 learned_addr learned my addr 172.21.15.33:0/2183429418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56d6d9d640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d8109150 0x7f56d819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56d659c640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d8102dc0 0x7f56d81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.219+0000 7f56cdd9b640 1 -- 172.21.15.33:0/2183429418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d8102dc0 msgr2=0x7f56d81992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cdd9b640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d8102dc0 0x7f56d81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cdd9b640 1 -- 172.21.15.33:0/2183429418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d8109150 msgr2=0x7f56d819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cdd9b640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d8109150 0x7f56d819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cdd9b640 1 -- 172.21.15.33:0/2183429418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56d819f390 con 0x7f56d81087d0 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56d6d9d640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d8109150 0x7f56d819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cdd9b640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8199820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006010 tx=0x7f56c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cffff640 1 -- 172.21.15.33:0/2183429418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c003f030 con 0x7f56d81087d0 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cffff640 1 -- 172.21.15.33:0/2183429418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56c0003800 con 0x7f56d81087d0 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56d819f620 con 0x7f56d81087d0 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56cffff640 1 -- 172.21.15.33:0/2183429418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56c0035070 con 0x7f56d81087d0 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.220+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56d81a5d40 con 0x7f56d81087d0 2024-09-10T21:03:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.221+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56d81031a0 con 0x7f56d81087d0 2024-09-10T21:03:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.223+0000 7f56cffff640 1 -- 172.21.15.33:0/2183429418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f56c0031b10 con 0x7f56d81087d0 2024-09-10T21:03:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.223+0000 7f56cffff640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56c40b5360 0x7f56c40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.223+0000 7f56d659c640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56c40b5360 0x7f56c40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.223+0000 7f56cffff640 1 -- 172.21.15.33:0/2183429418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f56c01067f0 con 0x7f56d81087d0 2024-09-10T21:03:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.224+0000 7f56d659c640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56c40b5360 0x7f56c40b7820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56c80058c0 tx=0x7f56c8005c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.227+0000 7f56cffff640 1 -- 172.21.15.33:0/2183429418 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f56c00cd6b0 con 0x7f56d81087d0 2024-09-10T21:03:43.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:43 smithi174 ceph-mon[35635]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T21:03:43.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.415+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56d819a410 con 0x7f56d81087d0 2024-09-10T21:03:43.419 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:43 smithi033 ceph-mon[30860]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T21:03:43.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.416+0000 7f56cffff640 1 -- 172.21.15.33:0/2183429418 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22763 (secure 0 0 0) 0x7f56c00ccfb0 con 0x7f56d81087d0 2024-09-10T21:03:43.420 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:43.420 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:41.855317+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56c40b5360 msgr2=0x7f56c40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56c40b5360 0x7f56c40b7820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f56c80058c0 tx=0x7f56c8005c60 comp rx=0 tx=0).stop 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 msgr2=0x7f56d8199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8199820 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f56c0006010 tx=0x7f56c00042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 shutdown_connections 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56c40b5360 0x7f56c40b7820 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56d8109150 0x7f56d819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56d81087d0 0x7f56d8199820 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 --2- 172.21.15.33:0/2183429418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56d8102dc0 0x7f56d81992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.418+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 >> 172.21.15.33:0/2183429418 conn(0x7f56d80fe830 msgr2=0x7f56d8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.419+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 shutdown_connections 2024-09-10T21:03:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:43.419+0000 7f56dcdb0640 1 -- 172.21.15.33:0/2183429418 wait complete. 2024-09-10T21:03:43.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:43 smithi104 ceph-mon[35325]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-09-10T21:03:44.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-10T21:02:01.445404+0000', 'modified': '2024-09-10T21:02:12.865787+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24566, 'mds_1': 24547, 'mds_2': 24539, 'mds_3': 24532, 'mds_4': 24526}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14697': {'gid': 14697, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6839/295723844', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6838', 'nonce': 295723844}, {'type': 'v1', 'addr': '172.21.15.33:6839', 'nonce': 295723844}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14709': {'gid': 14709, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.33:6841/1476321443', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.33:6840', 'nonce': 1476321443}, {'type': 'v1', 'addr': '172.21.15.33:6841', 'nonce': 1476321443}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6833/2049405106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6832', 'nonce': 2049405106}, {'type': 'v1', 'addr': '172.21.15.104:6833', 'nonce': 2049405106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24505': {'gid': 24505, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/34665907', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 34665907}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 34665907}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24518': {'gid': 24518, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.104:6835/1051931570', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6834', 'nonce': 1051931570}, {'type': 'v1', 'addr': '172.21.15.104:6835', 'nonce': 1051931570}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24526': {'gid': 24526, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6835/982304250', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 982304250}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 982304250}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24532': {'gid': 24532, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.174:6837/1592763439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1592763439}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1592763439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24539': {'gid': 24539, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.104:6837/2894143081', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6836', 'nonce': 2894143081}, {'type': 'v1', 'addr': '172.21.15.104:6837', 'nonce': 2894143081}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24547': {'gid': 24547, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.174:6839/3620064873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 3620064873}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 3620064873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24566': {'gid': 24566, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.104:6839/2034440949', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.104:6838', 'nonce': 2034440949}, {'type': 'v1', 'addr': '172.21.15.104:6839', 'nonce': 2034440949}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24566, 'qdb_cluster': [24566, 24547, 24532, 24539, 24526]} 2024-09-10T21:03:44.052 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-10T21:03:44.052 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-10T21:03:44.313 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:44.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:44 smithi174 ceph-mon[35635]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T21:03:44.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:44 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2183429418' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:44.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:44 smithi033 ceph-mon[30860]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T21:03:44.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:44 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2183429418' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:44.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:44 smithi104 ceph-mon[35325]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-10T21:03:44.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:44 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2183429418' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:44.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 -- 172.21.15.33:0/1497851857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 msgr2=0x7feeb41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/1497851857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb41031a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7feea4009900 tx=0x7feea402a410 comp rx=0 tx=0).stop 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 -- 172.21.15.33:0/1497851857 shutdown_connections 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/1497851857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeb4109150 0x7feeb410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/1497851857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb41087d0 0x7feeb4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/1497851857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb41031a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 -- 172.21.15.33:0/1497851857 >> 172.21.15.33:0/1497851857 conn(0x7feeb40fe830 msgr2=0x7feeb4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 -- 172.21.15.33:0/1497851857 shutdown_connections 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.770+0000 7feeb9b7a640 1 -- 172.21.15.33:0/1497851857 wait complete. 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 Processor -- start 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 -- start start 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb4199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeb41087d0 0x7feeb4199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb4109150 0x7feeb419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4112e60 con 0x7feeb4102dc0 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4112ce0 con 0x7feeb4109150 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb9b7a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feeb4112fe0 con 0x7feeb41087d0 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb4199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb4199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40252/0 (socket says 172.21.15.33:40252) 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb37fe640 1 -- 172.21.15.33:0/3953609886 learned_addr learned my addr 172.21.15.33:0/3953609886 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeb41087d0 0x7feeb4199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb37fe640 1 -- 172.21.15.33:0/3953609886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeb41087d0 msgr2=0x7feeb4199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb37fe640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeb41087d0 0x7feeb4199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.771+0000 7feeb37fe640 1 -- 172.21.15.33:0/3953609886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb4109150 msgr2=0x7feeb419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb3fff640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb4109150 0x7feeb419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb37fe640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb4109150 0x7feeb419ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb37fe640 1 -- 172.21.15.33:0/3953609886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb419f3c0 con 0x7feeb4102dc0 2024-09-10T21:03:44.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb37fe640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb4199310 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feea4005ab0 tx=0x7feea4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb0ff9640 1 -- 172.21.15.33:0/3953609886 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea403f070 con 0x7feeb4102dc0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb0ff9640 1 -- 172.21.15.33:0/3953609886 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feea4031d60 con 0x7feeb4102dc0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb419f650 con 0x7feeb4102dc0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb0ff9640 1 -- 172.21.15.33:0/3953609886 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea4035030 con 0x7feeb4102dc0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.772+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb41a5d70 con 0x7feeb4102dc0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 Processor -- start 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb0ff9640 1 -- 172.21.15.33:0/3953609886 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7feea403b580 con 0x7feeb4102dc0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 -- start start 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2c3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2d34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f0f23ac10 con 0x556f0f2c2d30 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f0f22a040 con 0x556f0f2c3650 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feebbf4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f0f22de50 con 0x556f0f2d39f0 2024-09-10T21:03:44.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2c3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2c3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40256/0 (socket says 172.21.15.33:40256) 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb3fff640 1 -- 172.21.15.33:0/333610129 learned_addr learned my addr 172.21.15.33:0/333610129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb37fe640 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2d34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb3fff640 1 -- 172.21.15.33:0/333610129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 msgr2=0x556f0f2d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.775+0000 7feeb3fff640 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feeb3fff640 1 -- 172.21.15.33:0/333610129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 msgr2=0x556f0f2d34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feeb3fff640 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2d34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feeb3fff640 1 -- 172.21.15.33:0/333610129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0ef88540 con 0x556f0f2c2d30 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feeb3fff640 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2c3110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feeac009da0 tx=0x7feeac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8731d0bd9239224 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feeb0ff9640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feea00b52e0 0x7feea00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feea9ffb640 1 -- 172.21.15.33:0/333610129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac003970 con 0x556f0f2c2d30 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feea9ffb640 1 -- 172.21.15.33:0/333610129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeac004030 con 0x556f0f2c2d30 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feea9ffb640 1 -- 172.21.15.33:0/333610129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac011750 con 0x556f0f2c2d30 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feeb37fe640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feea00b52e0 0x7feea00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.776+0000 7feeb0ff9640 1 -- 172.21.15.33:0/3953609886 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feea41068c0 con 0x7feeb4102dc0 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 -- 172.21.15.33:0/333610129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 msgr2=0x556f0f2c3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feeb37fe640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feea00b52e0 0x7feea00b77a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feea40ca2f0 tx=0x7feea402a410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2c3110 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7feeac009da0 tx=0x7feeac00f730 comp rx=0 tx=0).stop 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 -- 172.21.15.33:0/333610129 shutdown_connections 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2d34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 --2- 172.21.15.33:0/333610129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2c3110 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 -- 172.21.15.33:0/333610129 >> 172.21.15.33:0/333610129 conn(0x556f0f2c0b60 msgr2=0x556f0f2c1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 -- 172.21.15.33:0/333610129 shutdown_connections 2024-09-10T21:03:44.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 -- 172.21.15.33:0/333610129 wait complete. 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 -- start start 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.777+0000 7feebbf4f740 1 Processor -- start 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c3650 0x556f0f2d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f0f08b5f0 con 0x556f0f2c2d30 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f0f1c5010 con 0x556f0f2d39f0 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f0f2d66a0 con 0x556f0f2c3650 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 10 client.0 _ll_register_callbacks cb 0x556f0f224340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2d0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40268/0 (socket says 172.21.15.33:40268) 2024-09-10T21:03:44.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 -- 172.21.15.33:0/3443369719 learned_addr learned my addr 172.21.15.33:0/3443369719 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c3650 0x556f0f2d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 -- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c3650 msgr2=0x556f0f2d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c3650 0x556f0f2d0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 -- 172.21.15.33:0/3443369719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 msgr2=0x556f0f2d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 -- 172.21.15.33:0/3443369719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0f1f2a00 con 0x556f0f2c2d30 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c3650 0x556f0f2d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feeb3fff640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2d0980 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feeac0048d0 tx=0x7feeac005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cc8b8e268bac95 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7fee92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:44.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7fee92ffd640 1 -- 172.21.15.33:0/3443369719 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac004e50 con 0x556f0f2c2d30 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7fee92ffd640 1 -- 172.21.15.33:0/3443369719 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feeac003970 con 0x556f0f2c2d30 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.778+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeac004030 con 0x556f0f2c2d30 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7fee92ffd640 1 -- 172.21.15.33:0/3443369719 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feeac00fe80 con 0x556f0f2c2d30 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 10 client.15219 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f0f2e2210 con 0x556f0f2c2d30 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7fee92ffd640 1 -- 172.21.15.33:0/3443369719 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feeac004030 con 0x556f0f2c2d30 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7fee92ffd640 1 -- 172.21.15.33:0/3443369719 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feeac028070 con 0x556f0f2c2d30 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 10 client.15219 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 10 client.15219 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 20 client.15219 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f2e4470 0x556f0f2e58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 4 client.15219 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:44.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f0f23cb60 con 0x556f0f2e4470 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.779+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f2e4470 0x556f0f2e58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.780+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f2e4470 0x556f0f2e58e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4fb7e103a34d2de1 server_cookie=1e52b8051815698 in_seq=0 out_seq=0 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.780+0000 7fee92ffd640 10 client.15219 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7fee92ffd640 1 -- 172.21.15.33:0/3443369719 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f0f23cb60 con 0x556f0f2e4470 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7fee92ffd640 10 client.15219 handle_command_reply: tid=0 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 client.15219 shutdown 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 msgr2=0x556f0f2d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2d0980 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7feeac0048d0 tx=0x7feeac005bf0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 shutdown_connections 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f2e4470 0x556f0f2e58e0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c3650 0x556f0f2d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3443369719 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2d0980 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 >> 172.21.15.33:0/3443369719 conn(0x556f0f2c0b60 msgr2=0x556f0f2bfd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:44.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7fee92ffd640 0 client.15219 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:44.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 shutdown_connections 2024-09-10T21:03:44.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 1 -- 172.21.15.33:0/3443369719 wait complete. 2024-09-10T21:03:44.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.781+0000 7feebbf4f740 20 client.15219 trim_cache size 0 max 16384 2024-09-10T21:03:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 Processor -- start 2024-09-10T21:03:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 -- start start 2024-09-10T21:03:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c2d30 0x556f0f2e4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c3650 0x556f0f2e35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f0f2e3c20 con 0x556f0f2c3650 2024-09-10T21:03:44.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f0f2e42f0 con 0x556f0f2d39f0 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feebbf4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f0f2e49c0 con 0x556f0f2c2d30 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feeb2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c3650 0x556f0f2e35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feeb2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c3650 0x556f0f2e35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40280/0 (socket says 172.21.15.33:40280) 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feeb2ffd640 1 -- 172.21.15.33:0/3639555422 learned_addr learned my addr 172.21.15.33:0/3639555422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.800+0000 7feeb3fff640 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feeb3fff640 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c2d30 0x556f0f2e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feeb2ffd640 1 -- 172.21.15.33:0/3639555422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c2d30 msgr2=0x556f0f2e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c2d30 0x556f0f2e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feeb2ffd640 1 -- 172.21.15.33:0/3639555422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 msgr2=0x556f0f2d59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feeb2ffd640 1 -- 172.21.15.33:0/3639555422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0f2c15e0 con 0x556f0f2c3650 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c3650 0x556f0f2e35c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00da40 tx=0x7fee9c00d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8edbf960dc35171f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:44.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feea97fa640 1 -- 172.21.15.33:0/3639555422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c01e070 con 0x556f0f2c3650 2024-09-10T21:03:44.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feea97fa640 1 -- 172.21.15.33:0/3639555422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c004230 con 0x556f0f2c3650 2024-09-10T21:03:44.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.801+0000 7feea97fa640 1 -- 172.21.15.33:0/3639555422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c0116b0 con 0x556f0f2c3650 2024-09-10T21:03:44.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 -- 172.21.15.33:0/3639555422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c3650 msgr2=0x556f0f2e35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c3650 0x556f0f2e35c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00da40 tx=0x7fee9c00d340 comp rx=0 tx=0).stop 2024-09-10T21:03:44.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 -- 172.21.15.33:0/3639555422 shutdown_connections 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2d39f0 0x556f0f2d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c3650 0x556f0f2e35c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 --2- 172.21.15.33:0/3639555422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2c2d30 0x556f0f2e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 -- 172.21.15.33:0/3639555422 >> 172.21.15.33:0/3639555422 conn(0x556f0f2c0b60 msgr2=0x556f0f2d3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 -- 172.21.15.33:0/3639555422 shutdown_connections 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 -- 172.21.15.33:0/3639555422 wait complete. 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.802+0000 7feebbf4f740 1 -- start start 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 1 Processor -- start 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2cc240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2cc780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f0f2e5760 con 0x556f0f2c2d30 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f0f2e5090 con 0x556f0f2c3650 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f0f2e5e30 con 0x556f0f2d39f0 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feebbf4f740 10 client.0 _ll_register_callbacks cb 0x7feeac009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feeb3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40292/0 (socket says 172.21.15.33:40292) 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feeb3fff640 1 -- 172.21.15.33:0/2667491556 learned_addr learned my addr 172.21.15.33:0/2667491556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.803+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb3fff640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb3fff640 1 -- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 msgr2=0x556f0f2cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb3fff640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb3fff640 1 -- 172.21.15.33:0/2667491556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 msgr2=0x556f0f2cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb3fff640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2cc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb3fff640 1 -- 172.21.15.33:0/2667491556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0f2ccd70 con 0x556f0f2c2d30 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feeb3fff640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2ce260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feeac00fc30 tx=0x7feeac009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9ac23902220baa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7fee92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7fee92ffd640 1 -- 172.21.15.33:0/2667491556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeac03d060 con 0x556f0f2c2d30 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7fee92ffd640 1 -- 172.21.15.33:0/2667491556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feeac011530 con 0x556f0f2c2d30 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee9c004230 con 0x556f0f2c2d30 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7fee92ffd640 1 -- 172.21.15.33:0/2667491556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7feeac011730 con 0x556f0f2c2d30 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feebbf4f740 10 client.15225 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:44.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.804+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x556f0f2ee800 con 0x556f0f2c2d30 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7fee92ffd640 1 -- 172.21.15.33:0/2667491556 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c004230 con 0x556f0f2c2d30 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7fee92ffd640 1 -- 172.21.15.33:0/2667491556 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7feeac028040 con 0x556f0f2c2d30 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7feebbf4f740 10 client.15225 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7feebbf4f740 10 client.15225 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7feebbf4f740 20 client.15225 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7feebbf4f740 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f359ef0 0x556f0f358c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7feebbf4f740 4 client.15225 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "status"}) -- 0x556f0f238610 con 0x556f0f359ef0 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.805+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f359ef0 0x556f0f358c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7fee92ffd640 10 client.15225 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:44.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feeb2ffd640 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f359ef0 0x556f0f358c00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6cf669e997a21bc server_cookie=9ec64ebb5f9f231b in_seq=0 out_seq=0 2024-09-10T21:03:44.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7fee92ffd640 1 -- 172.21.15.33:0/2667491556 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556f0f238610 con 0x556f0f359ef0 2024-09-10T21:03:44.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7fee92ffd640 10 client.15225 handle_command_reply: tid=0 2024-09-10T21:03:44.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 client.15225 shutdown 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 msgr2=0x556f0f2ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2ce260 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7feeac00fc30 tx=0x7feeac009900 comp rx=0 tx=0).stop 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 shutdown_connections 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x556f0f359ef0 0x556f0f358c00 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f0f2d39f0 0x556f0f2cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f0f2c3650 0x556f0f2cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 --2- 172.21.15.33:0/2667491556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f0f2c2d30 0x556f0f2ce260 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 >> 172.21.15.33:0/2667491556 conn(0x556f0f2c0b60 msgr2=0x556f0f2d59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7fee92ffd640 0 client.15225 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 shutdown_connections 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.806+0000 7feebbf4f740 1 -- 172.21.15.33:0/2667491556 wait complete. 2024-09-10T21:03:44.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feebbf4f740 20 client.15225 trim_cache size 0 max 16384 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 4, 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24526, 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 92.948109236999997, 2024-09-10T21:03:44.816 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:44.817 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:44.817 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:44.817 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 130.751661603 2024-09-10T21:03:44.817 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:44.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feea00b52e0 msgr2=0x7feea00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feea00b52e0 0x7feea00b77a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7feea40ca2f0 tx=0x7feea402a410 comp rx=0 tx=0).stop 2024-09-10T21:03:44.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 msgr2=0x7feeb4199310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb4199310 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feea4005ab0 tx=0x7feea4003cd0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 shutdown_connections 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feea00b52e0 0x7feea00b77a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feeb4109150 0x7feeb419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feeb41087d0 0x7feeb4199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 --2- 172.21.15.33:0/3953609886 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feeb4102dc0 0x7feeb4199310 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 >> 172.21.15.33:0/3953609886 conn(0x7feeb40fe830 msgr2=0x7feeb4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 shutdown_connections 2024-09-10T21:03:44.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:44.807+0000 7feeb9b7a640 1 -- 172.21.15.33:0/3953609886 wait complete. 2024-09-10T21:03:45.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:45 smithi033 ceph-mon[30860]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T21:03:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:45 smithi104 ceph-mon[35325]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T21:03:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:45 smithi174 ceph-mon[35635]: pgmap v339: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-09-10T21:03:45.686 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-10T21:03:45.946 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:46.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 -- 172.21.15.33:0/915778434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe974108060 msgr2=0x7fe97410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/915778434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe974108060 0x7fe97410e860 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe96c009ca0 tx=0x7fe96c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 -- 172.21.15.33:0/915778434 shutdown_connections 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/915778434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe974108060 0x7fe97410e860 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/915778434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe9741076e0 0x7fe974107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/915778434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe974101cd0 0x7fe9741020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 -- 172.21.15.33:0/915778434 >> 172.21.15.33:0/915778434 conn(0x7fe9740fd720 msgr2=0x7fe9740ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 -- 172.21.15.33:0/915778434 shutdown_connections 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 -- 172.21.15.33:0/915778434 wait complete. 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.424+0000 7fe97a0b5640 1 Processor -- start 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe97a0b5640 1 -- start start 2024-09-10T21:03:46.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe97a0b5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe974101cd0 0x7fe97410f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe97a0b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9741076e0 0x7fe97410f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe97a0b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe974108060 0x7fe97410fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe97a0b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe974112e70 con 0x7fe9741076e0 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe97a0b5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe974112cf0 con 0x7fe974101cd0 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe97a0b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe974112ff0 con 0x7fe974108060 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9741076e0 0x7fe97410f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9741076e0 0x7fe97410f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40300/0 (socket says 172.21.15.33:40300) 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 -- 172.21.15.33:0/221960607 learned_addr learned my addr 172.21.15.33:0/221960607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe9737fe640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe974101cd0 0x7fe97410f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 -- 172.21.15.33:0/221960607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe974108060 msgr2=0x7fe97410fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe974108060 0x7fe97410fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 -- 172.21.15.33:0/221960607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe974101cd0 msgr2=0x7fe97410f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe974101cd0 0x7fe97410f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.425+0000 7fe972ffd640 1 -- 172.21.15.33:0/221960607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9741a7640 con 0x7fe9741076e0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.426+0000 7fe972ffd640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9741076e0 0x7fe97410f8b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00b4a0 tx=0x7fe95c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.426+0000 7fe970ff9640 1 -- 172.21.15.33:0/221960607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c014070 con 0x7fe9741076e0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.426+0000 7fe970ff9640 1 -- 172.21.15.33:0/221960607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c010ac0 con 0x7fe9741076e0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.426+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9741a7930 con 0x7fe9741076e0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.426+0000 7fe970ff9640 1 -- 172.21.15.33:0/221960607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c017c30 con 0x7fe9741076e0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.426+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9741a7f40 con 0x7fe9741076e0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.428+0000 7fe970ff9640 1 -- 172.21.15.33:0/221960607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe95c010610 con 0x7fe9741076e0 2024-09-10T21:03:46.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.428+0000 7fe97c48a740 1 Processor -- start 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.428+0000 7fe97c48a740 1 -- start start 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d544110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 0x55f16d5529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe97c48a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f16d4bbc10 con 0x55f16d544650 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe97c48a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f16d4ab040 con 0x55f16d543d30 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe97c48a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f16d4aee50 con 0x55f16d5505e0 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40306/0 (socket says 172.21.15.33:40306) 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 -- 172.21.15.33:0/1900439996 learned_addr learned my addr 172.21.15.33:0/1900439996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe973fff640 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d544110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 -- 172.21.15.33:0/1900439996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 msgr2=0x55f16d5529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 0x55f16d5529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 -- 172.21.15.33:0/1900439996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 msgr2=0x55f16d544110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d544110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 -- 172.21.15.33:0/1900439996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f16d209540 con 0x55f16d544650 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe970ff9640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9600b5360 0x7fe9600b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe9737fe640 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5500a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe96400bb40 tx=0x7fe96400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd5cd87b5fa12957 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe973fff640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9600b5360 0x7fe9600b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe970ff9640 1 -- 172.21.15.33:0/221960607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe95c0e5520 con 0x7fe9741076e0 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe969ffb640 1 -- 172.21.15.33:0/1900439996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964017420 con 0x55f16d544650 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe969ffb640 1 -- 172.21.15.33:0/1900439996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe964017a70 con 0x55f16d544650 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.429+0000 7fe969ffb640 1 -- 172.21.15.33:0/1900439996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96400f9c0 con 0x55f16d544650 2024-09-10T21:03:46.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 -- 172.21.15.33:0/1900439996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 msgr2=0x55f16d5500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe973fff640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9600b5360 0x7fe9600b7820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe96c02ade0 tx=0x7fe96c005df0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5500a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe96400bb40 tx=0x7fe96400d340 comp rx=0 tx=0).stop 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 -- 172.21.15.33:0/1900439996 shutdown_connections 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 0x55f16d5529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5500a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1900439996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d544110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 -- 172.21.15.33:0/1900439996 >> 172.21.15.33:0/1900439996 conn(0x55f16d541b60 msgr2=0x55f16d542d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 -- 172.21.15.33:0/1900439996 shutdown_connections 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.430+0000 7fe97c48a740 1 -- 172.21.15.33:0/1900439996 wait complete. 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 -- start start 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 Processor -- start 2024-09-10T21:03:46.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d543d30 0x55f16d54d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 0x55f16d54db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d5505e0 0x55f16d54e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f16d30c5f0 con 0x55f16d543d30 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f16d446010 con 0x55f16d5505e0 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f16d553290 con 0x55f16d544650 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe97c48a740 10 client.0 _ll_register_callbacks cb 0x55f16d4a5340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d543d30 0x55f16d54d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d543d30 0x55f16d54d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40316/0 (socket says 172.21.15.33:40316) 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 -- 172.21.15.33:0/2053586491 learned_addr learned my addr 172.21.15.33:0/2053586491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe973fff640 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d5505e0 0x55f16d54e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 -- 172.21.15.33:0/2053586491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 msgr2=0x55f16d54db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 0x55f16d54db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 -- 172.21.15.33:0/2053586491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d5505e0 msgr2=0x55f16d54e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d5505e0 0x55f16d54e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe9737fe640 1 -- 172.21.15.33:0/2053586491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f16d473a00 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.431+0000 7fe973fff640 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d5505e0 0x55f16d54e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe9737fe640 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d543d30 0x55f16d54d630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe964019040 tx=0x7fe964017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6717ad966aade2ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe952ffd640 1 -- 172.21.15.33:0/2053586491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964026070 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe952ffd640 1 -- 172.21.15.33:0/2053586491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe96400f700 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe964017a70 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe952ffd640 1 -- 172.21.15.33:0/2053586491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe964032050 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 10 client.15237 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f16d55ef00 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe952ffd640 1 -- 172.21.15.33:0/2053586491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe964017a70 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe952ffd640 1 -- 172.21.15.33:0/2053586491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe9640229e0 con 0x55f16d543d30 2024-09-10T21:03:46.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 10 client.15237 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 10 client.15237 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 20 client.15237 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d561230 0x55f16d5626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 4 client.15237 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.432+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f16d4bdb60 con 0x55f16d561230 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.433+0000 7fe972ffd640 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d561230 0x55f16d5626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.433+0000 7fe972ffd640 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d561230 0x55f16d5626a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=282d3171bd5e0bd1 server_cookie=8ca066dc3b02ab62 in_seq=0 out_seq=0 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.433+0000 7fe952ffd640 10 client.15237 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe952ffd640 1 -- 172.21.15.33:0/2053586491 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f16d4bdb60 con 0x55f16d561230 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe952ffd640 10 client.15237 handle_command_reply: tid=0 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 client.15237 shutdown 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d543d30 msgr2=0x55f16d54d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d543d30 0x55f16d54d630 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe964019040 tx=0x7fe964017f90 comp rx=0 tx=0).stop 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 shutdown_connections 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d5505e0 0x55f16d54e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d561230 0x55f16d5626a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 0x55f16d54db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2053586491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d543d30 0x55f16d54d630 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe952ffd640 0 client.15237 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 >> 172.21.15.33:0/2053586491 conn(0x55f16d541b60 msgr2=0x55f16d547700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 shutdown_connections 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 1 -- 172.21.15.33:0/2053586491 wait complete. 2024-09-10T21:03:46.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.434+0000 7fe97c48a740 20 client.15237 trim_cache size 0 max 16384 2024-09-10T21:03:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 Processor -- start 2024-09-10T21:03:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 -- start start 2024-09-10T21:03:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d54e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 0x55f16d547da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d5505e0 0x55f16d54a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f16d5609e0 con 0x55f16d5505e0 2024-09-10T21:03:46.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f16d5610b0 con 0x55f16d543d30 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe97c48a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f16d561780 con 0x55f16d544650 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d5505e0 0x55f16d54a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d5505e0 0x55f16d54a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40328/0 (socket says 172.21.15.33:40328) 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 -- 172.21.15.33:0/1581795441 learned_addr learned my addr 172.21.15.33:0/1581795441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe9737fe640 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d54e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe972ffd640 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 0x55f16d547da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 -- 172.21.15.33:0/1581795441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 msgr2=0x55f16d547da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 0x55f16d547da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 -- 172.21.15.33:0/1581795441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 msgr2=0x55f16d54e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d54e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 -- 172.21.15.33:0/1581795441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f16d5425e0 con 0x55f16d5505e0 2024-09-10T21:03:46.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.453+0000 7fe973fff640 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d5505e0 0x55f16d54a840 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe96c031f00 tx=0x7fe96c03d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef99f4770aa6510 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe9697fa640 1 -- 172.21.15.33:0/1581795441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c004240 con 0x55f16d5505e0 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe9697fa640 1 -- 172.21.15.33:0/1581795441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe96c04d070 con 0x55f16d5505e0 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe9697fa640 1 -- 172.21.15.33:0/1581795441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe96c02fe40 con 0x55f16d5505e0 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 -- 172.21.15.33:0/1581795441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d5505e0 msgr2=0x55f16d54a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d5505e0 0x55f16d54a840 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fe96c031f00 tx=0x7fe96c03d000 comp rx=0 tx=0).stop 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 -- 172.21.15.33:0/1581795441 shutdown_connections 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d5505e0 0x55f16d54a840 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d544650 0x55f16d547da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 --2- 172.21.15.33:0/1581795441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d54e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 -- 172.21.15.33:0/1581795441 >> 172.21.15.33:0/1581795441 conn(0x55f16d541b60 msgr2=0x55f16d54beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 -- 172.21.15.33:0/1581795441 shutdown_connections 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.454+0000 7fe97c48a740 1 -- 172.21.15.33:0/1581795441 wait complete. 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 -- start start 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 Processor -- start 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d564db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5652f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 0x55f16d56a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f16d562520 con 0x55f16d544650 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f16d561e50 con 0x55f16d543d30 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f16d562bf0 con 0x55f16d5505e0 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe97c48a740 10 client.0 _ll_register_callbacks cb 0x7fe96400b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:46.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe972ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5652f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40338/0 (socket says 172.21.15.33:40338) 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.455+0000 7fe972ffd640 1 -- 172.21.15.33:0/2812790705 learned_addr learned my addr 172.21.15.33:0/2812790705 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe9737fe640 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d564db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe973fff640 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 0x55f16d56a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe972ffd640 1 -- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 msgr2=0x55f16d56a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe972ffd640 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 0x55f16d56a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe972ffd640 1 -- 172.21.15.33:0/2812790705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 msgr2=0x55f16d564db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe972ffd640 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d564db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe972ffd640 1 -- 172.21.15.33:0/2812790705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f16d56a9a0 con 0x55f16d544650 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe972ffd640 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5652f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0e3020 tx=0x7fe95c017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e52c0863ca80e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe952ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:46.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe952ffd640 1 -- 172.21.15.33:0/2812790705 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c01e040 con 0x55f16d544650 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe952ffd640 1 -- 172.21.15.33:0/2812790705 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe95c010c70 con 0x55f16d544650 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe96c04d070 con 0x55f16d544650 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe952ffd640 1 -- 172.21.15.33:0/2812790705 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe95c0ab7f0 con 0x55f16d544650 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe97c48a740 10 client.15243 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f16d5ce2e0 con 0x55f16d544650 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.456+0000 7fe952ffd640 1 -- 172.21.15.33:0/2812790705 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe96c04d070 con 0x55f16d544650 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe952ffd640 1 -- 172.21.15.33:0/2812790705 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe95c0a7f50 con 0x55f16d544650 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe97c48a740 10 client.15243 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe97c48a740 10 client.15243 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe97c48a740 20 client.15243 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d5d2eb0 0x55f16d5d1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe97c48a740 4 client.15243 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:46.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "status"}) -- 0x55f16d4b9610 con 0x55f16d5d2eb0 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe9737fe640 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d5d2eb0 0x55f16d5d1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe9737fe640 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d5d2eb0 0x55f16d5d1bc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48544bb00b1a41aa server_cookie=3d1bf5d378e10b9e in_seq=0 out_seq=0 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.457+0000 7fe952ffd640 10 client.15243 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe952ffd640 1 -- 172.21.15.33:0/2812790705 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f16d4b9610 con 0x55f16d5d2eb0 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe952ffd640 10 client.15243 handle_command_reply: tid=0 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 client.15243 shutdown 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 msgr2=0x55f16d5652f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5652f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe95c0e3020 tx=0x7fe95c017650 comp rx=0 tx=0).stop 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 shutdown_connections 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f16d5d2eb0 0x55f16d5d1bc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f16d5505e0 0x55f16d56a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe952ffd640 0 client.15243 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:03:46.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f16d544650 0x55f16d5652f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 --2- 172.21.15.33:0/2812790705 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f16d543d30 0x55f16d564db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 >> 172.21.15.33:0/2812790705 conn(0x55f16d541b60 msgr2=0x55f16d55b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 shutdown_connections 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 1 -- 172.21.15.33:0/2812790705 wait complete. 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.458+0000 7fe97c48a740 20 client.15243 trim_cache size 0 max 16384 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 3, 2024-09-10T21:03:46.467 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24532, 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 95.606576406000002, 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 129.71484016299999 2024-09-10T21:03:46.468 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9600b5360 msgr2=0x7fe9600b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9600b5360 0x7fe9600b7820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe96c02ade0 tx=0x7fe96c005df0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9741076e0 msgr2=0x7fe97410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9741076e0 0x7fe97410f8b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00b4a0 tx=0x7fe95c00b960 comp rx=0 tx=0).stop 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 shutdown_connections 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9600b5360 0x7fe9600b7820 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe974108060 0x7fe97410fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9741076e0 0x7fe97410f8b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 --2- 172.21.15.33:0/221960607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe974101cd0 0x7fe97410f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:46.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 >> 172.21.15.33:0/221960607 conn(0x7fe9740fd720 msgr2=0x7fe9740ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:46.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 shutdown_connections 2024-09-10T21:03:46.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:46.459+0000 7fe97a0b5640 1 -- 172.21.15.33:0/221960607 wait complete. 2024-09-10T21:03:47.300 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-10T21:03:47.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:47 smithi033 ceph-mon[30860]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T21:03:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:47 smithi104 ceph-mon[35325]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T21:03:47.563 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:47.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:47 smithi174 ceph-mon[35635]: pgmap v340: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T21:03:48.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.149+0000 7eff0da9a640 1 -- 172.21.15.33:0/3139652515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 msgr2=0x7eff0810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.149+0000 7eff0da9a640 1 --2- 172.21.15.33:0/3139652515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0810f5a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7eff00009ca0 tx=0x7eff0002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.149+0000 7eff0da9a640 1 -- 172.21.15.33:0/3139652515 shutdown_connections 2024-09-10T21:03:48.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.149+0000 7eff0da9a640 1 --2- 172.21.15.33:0/3139652515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0810f5a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.149+0000 7eff0da9a640 1 --2- 172.21.15.33:0/3139652515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff08104600 0x7eff0810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.149+0000 7eff0da9a640 1 --2- 172.21.15.33:0/3139652515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff08103ce0 0x7eff081040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.149+0000 7eff0da9a640 1 -- 172.21.15.33:0/3139652515 >> 172.21.15.33:0/3139652515 conn(0x7eff080fd700 msgr2=0x7eff080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.150+0000 7eff0da9a640 1 -- 172.21.15.33:0/3139652515 shutdown_connections 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.150+0000 7eff0da9a640 1 -- 172.21.15.33:0/3139652515 wait complete. 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.150+0000 7eff0da9a640 1 Processor -- start 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.150+0000 7eff0da9a640 1 -- start start 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff0da9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff08103ce0 0x7eff08199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff0da9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff08104600 0x7eff081997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff0da9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff0da9a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7eff08112db0 con 0x7eff0810d140 2024-09-10T21:03:48.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff0da9a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7eff08112c30 con 0x7eff08103ce0 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff0da9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff08112f30 con 0x7eff08104600 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40362/0 (socket says 172.21.15.33:40362) 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 -- 172.21.15.33:0/1855046252 learned_addr learned my addr 172.21.15.33:0/1855046252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff077fe640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff08103ce0 0x7eff08199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7efefeffd640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff08104600 0x7eff081997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 -- 172.21.15.33:0/1855046252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff08104600 msgr2=0x7eff081997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff08104600 0x7eff081997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 -- 172.21.15.33:0/1855046252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff08103ce0 msgr2=0x7eff08199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff08103ce0 0x7eff08199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 -- 172.21.15.33:0/1855046252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0819f310 con 0x7eff0810d140 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.151+0000 7eff07fff640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0819eb40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff00005cc0 tx=0x7eff0002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:48.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.152+0000 7eff057fa640 1 -- 172.21.15.33:0/1855046252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff00003bf0 con 0x7eff0810d140 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.152+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0819f5a0 con 0x7eff0810d140 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.152+0000 7eff057fa640 1 -- 172.21.15.33:0/1855046252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff00003d90 con 0x7eff0810d140 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.152+0000 7eff057fa640 1 -- 172.21.15.33:0/1855046252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0002fbc0 con 0x7eff0810d140 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.152+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff081a5cc0 con 0x7eff0810d140 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff057fa640 1 -- 172.21.15.33:0/1855046252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7eff00004610 con 0x7eff0810d140 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 Processor -- start 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 -- start start 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bfe0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562042b69c10 con 0x562042bfe5e0 2024-09-10T21:03:48.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562042b59040 con 0x562042bf2650 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.154+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562042b5ce50 con 0x562042bf1d30 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40372/0 (socket says 172.21.15.33:40372) 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 -- 172.21.15.33:0/610943274 learned_addr learned my addr 172.21.15.33:0/610943274 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7eff077fe640 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bfe0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 -- 172.21.15.33:0/610943274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 msgr2=0x562042bf2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 -- 172.21.15.33:0/610943274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 msgr2=0x562042bfe0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bfe0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 -- 172.21.15.33:0/610943274 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620428b7540 con 0x562042bfe5e0 2024-09-10T21:03:48.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7eff057fa640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efef40b5360 0x7efef40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefeffd640 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c009d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efef00125a0 tx=0x7efef000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596d4a8ec6c01caa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7eff077fe640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efef40b5360 0x7efef40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7eff057fa640 1 -- 172.21.15.33:0/1855046252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7eff00106fc0 con 0x7eff0810d140 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefdffb640 1 -- 172.21.15.33:0/610943274 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0004220 con 0x562042bfe5e0 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefdffb640 1 -- 172.21.15.33:0/610943274 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef00043c0 con 0x562042bfe5e0 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.155+0000 7efefdffb640 1 -- 172.21.15.33:0/610943274 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef00109c0 con 0x562042bfe5e0 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 -- 172.21.15.33:0/610943274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 msgr2=0x562042c009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff077fe640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efef40b5360 0x7efef40b7820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x562042bf4eb0 tx=0x7efef800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c009d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7efef00125a0 tx=0x7efef000b9c0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 -- 172.21.15.33:0/610943274 shutdown_connections 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c009d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bfe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/610943274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 -- 172.21.15.33:0/610943274 >> 172.21.15.33:0/610943274 conn(0x562042befb60 msgr2=0x562042bf0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 -- 172.21.15.33:0/610943274 shutdown_connections 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 -- 172.21.15.33:0/610943274 wait complete. 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.156+0000 7eff0fe6f740 1 -- start start 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 1 Processor -- start 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bfb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bf2650 0x562042bfbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bfe5e0 0x562042bfc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5620429ba5f0 con 0x562042bf2650 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562042af4010 con 0x562042bfe5e0 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562042c01290 con 0x562042bf1d30 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff0fe6f740 10 client.0 _ll_register_callbacks cb 0x562042b53340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bf2650 0x562042bfbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bf2650 0x562042bfbb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40380/0 (socket says 172.21.15.33:40380) 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 -- 172.21.15.33:0/3994376578 learned_addr learned my addr 172.21.15.33:0/3994376578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff07fff640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bfe5e0 0x562042bfc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7eff077fe640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bfb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 -- 172.21.15.33:0/3994376578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 msgr2=0x562042bfb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bfb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 -- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bfe5e0 msgr2=0x562042bfc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bfe5e0 0x562042bfc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.157+0000 7efefeffd640 1 -- 172.21.15.33:0/3994376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562042b21a00 con 0x562042bf2650 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7eff077fe640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bfb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7efefeffd640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bf2650 0x562042bfbb70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efef0001eb0 tx=0x7efef00042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddc6d906dc8fd8d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7efee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7efee6ffd640 1 -- 172.21.15.33:0/3994376578 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0005450 con 0x562042bf2650 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7efee6ffd640 1 -- 172.21.15.33:0/3994376578 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efef00055f0 con 0x562042bf2650 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efef00043c0 con 0x562042bf2650 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7efee6ffd640 1 -- 172.21.15.33:0/3994376578 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7efef00046c0 con 0x562042bf2650 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7eff0fe6f740 10 client.15255 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:48.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562042c0ced0 con 0x562042bf2650 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7efee6ffd640 1 -- 172.21.15.33:0/3994376578 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efef00043c0 con 0x562042bf2650 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7efee6ffd640 1 -- 172.21.15.33:0/3994376578 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7efef001c030 con 0x562042bf2650 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.158+0000 7eff0fe6f740 10 client.15255 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7eff0fe6f740 10 client.15255 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7eff0fe6f740 20 client.15255 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c0f2c0 0x562042c10730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7eff0fe6f740 4 client.15255 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562042b6bb60 con 0x562042c0f2c0 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7eff077fe640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c0f2c0 0x562042c10730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7efee6ffd640 10 client.15255 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:48.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.159+0000 7eff077fe640 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c0f2c0 0x562042c10730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b02b1aa17b191914 server_cookie=2e0fcb5a65a0ca35 in_seq=0 out_seq=0 2024-09-10T21:03:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7efee6ffd640 1 -- 172.21.15.33:0/3994376578 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562042b6bb60 con 0x562042c0f2c0 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7efee6ffd640 10 client.15255 handle_command_reply: tid=0 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 client.15255 shutdown 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bf2650 msgr2=0x562042bfbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bf2650 0x562042bfbb70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7efef0001eb0 tx=0x7efef00042b0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 shutdown_connections 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bfe5e0 0x562042bfc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c0f2c0 0x562042c10730 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bf2650 0x562042bfbb70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/3994376578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bfb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7efee6ffd640 0 client.15255 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 >> 172.21.15.33:0/3994376578 conn(0x562042befb60 msgr2=0x562042bf5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.160+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 shutdown_connections 2024-09-10T21:03:48.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.161+0000 7eff0fe6f740 1 -- 172.21.15.33:0/3994376578 wait complete. 2024-09-10T21:03:48.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.161+0000 7eff0fe6f740 20 client.15255 trim_cache size 0 max 16384 2024-09-10T21:03:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 Processor -- start 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 -- start start 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042bf6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562042c0ea70 con 0x562042bfe5e0 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562042c0f140 con 0x562042bf2650 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562042c0f810 con 0x562042bf1d30 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042bf6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042bf6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40394/0 (socket says 172.21.15.33:40394) 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 -- 172.21.15.33:0/1833745823 learned_addr learned my addr 172.21.15.33:0/1833745823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7efefeffd640 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff077fe640 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 -- 172.21.15.33:0/1833745823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 msgr2=0x562042bf5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 -- 172.21.15.33:0/1833745823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 msgr2=0x562042bf68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 -- 172.21.15.33:0/1833745823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562042c10a60 con 0x562042bfe5e0 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7efefeffd640 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff077fe640 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.179+0000 7eff07fff640 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042bf6e10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7eff00006010 tx=0x7eff000cd190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305ca11be1389b00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7efefd7fa640 1 -- 172.21.15.33:0/1833745823 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000cde80 con 0x562042bfe5e0 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7efefd7fa640 1 -- 172.21.15.33:0/1833745823 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff00036040 con 0x562042bfe5e0 2024-09-10T21:03:48.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7efefd7fa640 1 -- 172.21.15.33:0/1833745823 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000d6d10 con 0x562042bfe5e0 2024-09-10T21:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 -- 172.21.15.33:0/1833745823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 msgr2=0x562042bf6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042bf6e10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7eff00006010 tx=0x7eff000cd190 comp rx=0 tx=0).stop 2024-09-10T21:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 -- 172.21.15.33:0/1833745823 shutdown_connections 2024-09-10T21:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042bf6e10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/1833745823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 -- 172.21.15.33:0/1833745823 >> 172.21.15.33:0/1833745823 conn(0x562042befb60 msgr2=0x562042c0e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 -- 172.21.15.33:0/1833745823 shutdown_connections 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.180+0000 7eff0fe6f740 1 -- 172.21.15.33:0/1833745823 wait complete. 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 -- start start 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 Processor -- start 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c79bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562042c105b0 con 0x562042bfe5e0 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562042c0fee0 con 0x562042bf2650 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562042c10c80 con 0x562042bf1d30 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff0fe6f740 10 client.0 _ll_register_callbacks cb 0x7efef800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c79bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c79bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40410/0 (socket says 172.21.15.33:40410) 2024-09-10T21:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff07fff640 1 -- 172.21.15.33:0/263011167 learned_addr learned my addr 172.21.15.33:0/263011167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.181+0000 7eff077fe640 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7efefeffd640 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff07fff640 1 -- 172.21.15.33:0/263011167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 msgr2=0x562042bf8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff07fff640 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff07fff640 1 -- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 msgr2=0x562042bf86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff07fff640 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff07fff640 1 -- 172.21.15.33:0/263011167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562042c7a2e0 con 0x562042bfe5e0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff07fff640 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c79bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7eff00005e30 tx=0x7eff000cd200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d56adfef6543aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7efee6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7efee6ffd640 1 -- 172.21.15.33:0/263011167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff000d6a80 con 0x562042bfe5e0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7efee6ffd640 1 -- 172.21.15.33:0/263011167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7eff00004b40 con 0x562042bfe5e0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff00036040 con 0x562042bfe5e0 2024-09-10T21:03:48.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7efee6ffd640 1 -- 172.21.15.33:0/263011167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7eff000098b0 con 0x562042bfe5e0 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff0fe6f740 10 client.15261 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562042c7b530 con 0x562042bfe5e0 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.182+0000 7efee6ffd640 1 -- 172.21.15.33:0/263011167 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff00036040 con 0x562042bfe5e0 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7efee6ffd640 1 -- 172.21.15.33:0/263011167 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7eff000c90d0 con 0x562042bfe5e0 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff0fe6f740 10 client.15261 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff0fe6f740 10 client.15261 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff0fe6f740 20 client.15261 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c800b0 0x562042c7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff0fe6f740 4 client.15261 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "status"}) -- 0x562042b67610 con 0x562042c800b0 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff077fe640 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c800b0 0x562042c7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7eff077fe640 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c800b0 0x562042c7edc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e9169af9f1fb903 server_cookie=f93b4bd4e2f5bc5b in_seq=0 out_seq=0 2024-09-10T21:03:48.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7efee6ffd640 10 client.15261 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.183+0000 7efee6ffd640 1 -- 172.21.15.33:0/263011167 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562042b67610 con 0x562042c800b0 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7efee6ffd640 10 client.15261 handle_command_reply: tid=0 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 client.15261 shutdown 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 msgr2=0x562042c79bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c79bf0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7eff00005e30 tx=0x7eff000cd200 comp rx=0 tx=0).stop 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 shutdown_connections 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x562042c800b0 0x562042c7edc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562042bfe5e0 0x562042c79bf0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562042bf2650 0x562042bf86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7efee6ffd640 0 client.15261 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 --2- 172.21.15.33:0/263011167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562042bf1d30 0x562042bf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 >> 172.21.15.33:0/263011167 conn(0x562042befb60 msgr2=0x562042bf68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 shutdown_connections 2024-09-10T21:03:48.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 1 -- 172.21.15.33:0/263011167 wait complete. 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0fe6f740 20 client.15261 trim_cache size 0 max 16384 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 2, 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24539, 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 98.344333333999998, 2024-09-10T21:03:48.194 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 128.74032473899999 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efef40b5360 msgr2=0x7efef40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0da9a640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efef40b5360 0x7efef40b7820 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x562042bf4eb0 tx=0x7efef800b390 comp rx=0 tx=0).stop 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.184+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 msgr2=0x7eff0819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:48.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0819eb40 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7eff00005cc0 tx=0x7eff0002f380 comp rx=0 tx=0).stop 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 shutdown_connections 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efef40b5360 0x7efef40b7820 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff0810d140 0x7eff0819eb40 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff08104600 0x7eff081997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 --2- 172.21.15.33:0/1855046252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7eff08103ce0 0x7eff08199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 >> 172.21.15.33:0/1855046252 conn(0x7eff080fd700 msgr2=0x7eff080fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 shutdown_connections 2024-09-10T21:03:48.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:48.185+0000 7eff0da9a640 1 -- 172.21.15.33:0/1855046252 wait complete. 2024-09-10T21:03:48.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-10T21:03:49.240 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:49.362 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:49 smithi033 ceph-mon[30860]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T21:03:49.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:49 smithi104 ceph-mon[35325]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T21:03:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:49 smithi174 ceph-mon[35635]: pgmap v341: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T21:03:49.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 -- 172.21.15.33:0/1909762683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa8104570 msgr2=0x7f0aa810cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 --2- 172.21.15.33:0/1909762683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa8104570 0x7f0aa810cb50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009900 tx=0x7f0a9002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 -- 172.21.15.33:0/1909762683 shutdown_connections 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 --2- 172.21.15.33:0/1909762683 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa810d120 0x7f0aa810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 --2- 172.21.15.33:0/1909762683 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa8104570 0x7f0aa810cb50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 --2- 172.21.15.33:0/1909762683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa8103b70 0x7f0aa8103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 -- 172.21.15.33:0/1909762683 >> 172.21.15.33:0/1909762683 conn(0x7f0aa80fd700 msgr2=0x7f0aa80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 -- 172.21.15.33:0/1909762683 shutdown_connections 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 -- 172.21.15.33:0/1909762683 wait complete. 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 Processor -- start 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.761+0000 7f0aae778640 1 -- start start 2024-09-10T21:03:49.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aae778640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa8103b70 0x7f0aa8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aae778640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa8104570 0x7f0aa81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aae778640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa810d120 0x7f0aa819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aae778640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8112da0 con 0x7f0aa810d120 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aae778640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8112c20 con 0x7f0aa8103b70 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aae778640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8112f20 con 0x7f0aa8104570 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa810d120 0x7f0aa819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa810d120 0x7f0aa819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42508/0 (socket says 172.21.15.33:42508) 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 -- 172.21.15.33:0/1917822007 learned_addr learned my addr 172.21.15.33:0/1917822007 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa8103b70 0x7f0aa8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa8104570 0x7f0aa81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 -- 172.21.15.33:0/1917822007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa8104570 msgr2=0x7f0aa81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa8104570 0x7f0aa81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 -- 172.21.15.33:0/1917822007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa8103b70 msgr2=0x7f0aa8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa8103b70 0x7f0aa8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 -- 172.21.15.33:0/1917822007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa819f340 con 0x7f0aa810d120 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.762+0000 7f0aacd15640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa810d120 0x7f0aa819eb70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000eaa0 tx=0x7f0aa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.763+0000 7f0aa5ffb640 1 -- 172.21.15.33:0/1917822007 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa000cdb0 con 0x7f0aa810d120 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.763+0000 7f0aa5ffb640 1 -- 172.21.15.33:0/1917822007 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0aa0005020 con 0x7f0aa810d120 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.763+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa819f630 con 0x7f0aa810d120 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.763+0000 7f0aa5ffb640 1 -- 172.21.15.33:0/1917822007 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00116d0 con 0x7f0aa810d120 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.763+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81a5cf0 con 0x7f0aa810d120 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.765+0000 7f0aa5ffb640 1 -- 172.21.15.33:0/1917822007 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0aa000f040 con 0x7f0aa810d120 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.765+0000 7f0ab0b4d740 1 Processor -- start 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.765+0000 7f0ab0b4d740 1 -- start start 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b0e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9a85c10 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9a75040 con 0x55ffd9b0e650 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9a78e50 con 0x55ffd9b0dd30 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42512/0 (socket says 172.21.15.33:42512) 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 -- 172.21.15.33:0/2561045473 learned_addr learned my addr 172.21.15.33:0/2561045473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b0e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 -- 172.21.15.33:0/2561045473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0dd30 msgr2=0x55ffd9b0e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b0e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 -- 172.21.15.33:0/2561045473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 msgr2=0x55ffd9b1e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 -- 172.21.15.33:0/2561045473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd97d3540 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20de0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0a900096c0 tx=0x7f0a90005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3509b4404c0bd5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.766+0000 7f0aa5ffb640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a940b53e0 0x7f0a940b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a940b53e0 0x7f0a940b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0a9e7fc640 1 -- 172.21.15.33:0/2561045473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a90035070 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0a9e7fc640 1 -- 172.21.15.33:0/2561045473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a900035f0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0a9e7fc640 1 -- 172.21.15.33:0/2561045473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a900044e0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0aa5ffb640 1 -- 172.21.15.33:0/1917822007 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0aa00e5950 con 0x7f0aa810d120 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2561045473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 msgr2=0x55ffd9b20de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a940b53e0 0x7f0a940b78a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55ffd9b0d810 tx=0x7f0a9800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20de0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f0a900096c0 tx=0x7f0a90005b50 comp rx=0 tx=0).stop 2024-09-10T21:03:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2561045473 shutdown_connections 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20de0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2561045473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b0e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2561045473 >> 172.21.15.33:0/2561045473 conn(0x55ffd9b0bb60 msgr2=0x55ffd9b0cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2561045473 shutdown_connections 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.767+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2561045473 wait complete. 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 -- start start 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 Processor -- start 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b1b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b1c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffd98d65f0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9a10010 con 0x55ffd9b0dd30 2024-09-10T21:03:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9b21640 con 0x55ffd9b0e650 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0ab0b4d740 10 client.0 _ll_register_callbacks cb 0x55ffd9a6f340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0aacd15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b1c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0aacd15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b1c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42528/0 (socket says 172.21.15.33:42528) 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.768+0000 7f0aacd15640 1 -- 172.21.15.33:0/2456603983 learned_addr learned my addr 172.21.15.33:0/2456603983 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b1b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0aacd15640 1 -- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 msgr2=0x55ffd9b1bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0aacd15640 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0aacd15640 1 -- 172.21.15.33:0/2456603983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 msgr2=0x55ffd9b1b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0aacd15640 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b1b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0aacd15640 1 -- 172.21.15.33:0/2456603983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd9a3da00 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0aacd15640 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b1c400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000efd0 tx=0x7f0aa00abd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2e9d6aea08fa2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0a877fe640 1 -- 172.21.15.33:0/2456603983 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa001f040 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0a877fe640 1 -- 172.21.15.33:0/2456603983 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0aa00054f0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a900035f0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0a877fe640 1 -- 172.21.15.33:0/2456603983 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0aa00b5660 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0ab0b4d740 10 client.15273 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.769+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffd9b2d2e0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0a877fe640 1 -- 172.21.15.33:0/2456603983 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a900035f0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0a877fe640 1 -- 172.21.15.33:0/2456603983 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0aa00cdab0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0ab0b4d740 10 client.15273 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0ab0b4d740 10 client.15273 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0ab0b4d740 20 client.15273 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9b2f8a0 0x55ffd9b30d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0ab0b4d740 4 client.15273 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffd9a87b60 con 0x55ffd9b2f8a0 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9b2f8a0 0x55ffd9b30d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9b2f8a0 0x55ffd9b30d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=958b0f76631e80b1 server_cookie=4e3a774287eed790 in_seq=0 out_seq=0 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.770+0000 7f0a877fe640 10 client.15273 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.771+0000 7f0a877fe640 1 -- 172.21.15.33:0/2456603983 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffd9a87b60 con 0x55ffd9b2f8a0 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.771+0000 7f0a877fe640 10 client.15273 handle_command_reply: tid=0 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.771+0000 7f0ab0b4d740 1 client.15273 shutdown 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.771+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 msgr2=0x55ffd9b1c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.771+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b1c400 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000efd0 tx=0x7f0aa00abd10 comp rx=0 tx=0).stop 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 shutdown_connections 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9b2f8a0 0x55ffd9b30d10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b1c400 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b1bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0a877fe640 0 client.15273 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/2456603983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b1b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 >> 172.21.15.33:0/2456603983 conn(0x55ffd9b0bb60 msgr2=0x55ffd9b0ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 shutdown_connections 2024-09-10T21:03:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/2456603983 wait complete. 2024-09-10T21:03:49.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.772+0000 7f0ab0b4d740 20 client.15273 trim_cache size 0 max 16384 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 Processor -- start 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 -- start start 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b19480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b0cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9b2f050 con 0x55ffd9b0dd30 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9b2f720 con 0x55ffd9b0e650 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9b2fdf0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0aa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b19480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0aa7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b19480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42542/0 (socket says 172.21.15.33:42542) 2024-09-10T21:03:49.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.790+0000 7f0aa7fff640 1 -- 172.21.15.33:0/3798164962 learned_addr learned my addr 172.21.15.33:0/3798164962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b0cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0aacd15640 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0aa7fff640 1 -- 172.21.15.33:0/3798164962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b1e9f0 msgr2=0x55ffd9b20a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0aa7fff640 1 -- 172.21.15.33:0/3798164962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 msgr2=0x55ffd9b0cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b0cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0aa7fff640 1 -- 172.21.15.33:0/3798164962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffd9b0e480 con 0x55ffd9b0dd30 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b19480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55ffd9b1e900 tx=0x7f0a98004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156c68d12af9d8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0a9dffb640 1 -- 172.21.15.33:0/3798164962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9801ea10 con 0x55ffd9b0dd30 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0a9dffb640 1 -- 172.21.15.33:0/3798164962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a9801ebb0 con 0x55ffd9b0dd30 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.791+0000 7f0a9dffb640 1 -- 172.21.15.33:0/3798164962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9801d670 con 0x55ffd9b0dd30 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/3798164962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b0dd30 msgr2=0x55ffd9b19480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b19480 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x55ffd9b1e900 tx=0x7f0a98004610 comp rx=0 tx=0).stop 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/3798164962 shutdown_connections 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b20a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0e650 0x55ffd9b0cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/3798164962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b19480 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/3798164962 >> 172.21.15.33:0/3798164962 conn(0x55ffd9b0bb60 msgr2=0x55ffd9b0c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/3798164962 shutdown_connections 2024-09-10T21:03:49.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/3798164962 wait complete. 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 -- start start 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.792+0000 7f0ab0b4d740 1 Processor -- start 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b393b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0ab0b4d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b34090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9b30b90 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9b304c0 con 0x55ffd9b0dd30 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0ab0b4d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffd9b31260 con 0x55ffd9b0e650 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b34090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b34090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42546/0 (socket says 172.21.15.33:42546) 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0ab0b4d740 10 client.0 _ll_register_callbacks cb 0x7f0aa00a87d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 -- 172.21.15.33:0/1802758508 learned_addr learned my addr 172.21.15.33:0/1802758508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b393b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 -- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 msgr2=0x55ffd9b39930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b39930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 -- 172.21.15.33:0/1802758508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 msgr2=0x55ffd9b393b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b393b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 -- 172.21.15.33:0/1802758508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffd9b348c0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0a9ffff640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0aacd15640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b34090 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009270 tx=0x7f0aa0005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddb3bd1d0fe2e2b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0a877fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0a877fe640 1 -- 172.21.15.33:0/1802758508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa005b070 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.793+0000 7f0a877fe640 1 -- 172.21.15.33:0/1802758508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0aa00ab3e0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a9801ebb0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0a877fe640 1 -- 172.21.15.33:0/1802758508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0aa001f070 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 10 client.15279 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55ffd9b9ce70 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0a877fe640 1 -- 172.21.15.33:0/1802758508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a9801ebb0 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0a877fe640 1 -- 172.21.15.33:0/1802758508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7f0aa00aae50 con 0x55ffd9b1e9f0 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 10 client.15279 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 10 client.15279 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 20 client.15279 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9ba1960 0x55ffd9ba0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 4 client.15279 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "status"}) -- 0x55ffd9a83610 con 0x55ffd9ba1960 2024-09-10T21:03:49.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.794+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9ba1960 0x55ffd9ba0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:49.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0aa7fff640 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9ba1960 0x55ffd9ba0670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=efa48e617a70d4a2 server_cookie=c4c4b9b78be9efde in_seq=0 out_seq=0 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0a877fe640 10 client.15279 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0a877fe640 1 -- 172.21.15.33:0/1802758508 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55ffd9a83610 con 0x55ffd9ba1960 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0a877fe640 10 client.15279 handle_command_reply: tid=0 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 client.15279 shutdown 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 msgr2=0x55ffd9b34090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b34090 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009270 tx=0x7f0aa0005400 comp rx=0 tx=0).stop 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 shutdown_connections 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffd9ba1960 0x55ffd9ba0670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffd9b1e9f0 0x55ffd9b34090 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffd9b0e650 0x55ffd9b39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 --2- 172.21.15.33:0/1802758508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffd9b0dd30 0x55ffd9b393b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0a877fe640 0 client.15279 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:03:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 >> 172.21.15.33:0/1802758508 conn(0x55ffd9b0bb60 msgr2=0x55ffd9b1cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 shutdown_connections 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 1 -- 172.21.15.33:0/1802758508 wait complete. 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.795+0000 7f0ab0b4d740 20 client.15279 trim_cache size 0 max 16384 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 1, 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24547, 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:49.806 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 100.96204476699999, 2024-09-10T21:03:49.807 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:49.807 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:49.807 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:49.807 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 127.722491828 2024-09-10T21:03:49.807 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a940b53e0 msgr2=0x7f0a940b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a940b53e0 0x7f0a940b78a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x55ffd9b0d810 tx=0x7f0a9800b390 comp rx=0 tx=0).stop 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa810d120 msgr2=0x7f0aa819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa810d120 0x7f0aa819eb70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000eaa0 tx=0x7f0aa000ef60 comp rx=0 tx=0).stop 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 shutdown_connections 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a940b53e0 0x7f0a940b78a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa810d120 0x7f0aa819eb70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa8104570 0x7f0aa81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 --2- 172.21.15.33:0/1917822007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa8103b70 0x7f0aa8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 >> 172.21.15.33:0/1917822007 conn(0x7f0aa80fd700 msgr2=0x7f0aa80fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 shutdown_connections 2024-09-10T21:03:49.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:49.796+0000 7f0aae778640 1 -- 172.21.15.33:0/1917822007 wait complete. 2024-09-10T21:03:50.612 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-10T21:03:50.878 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:51.282 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:51 smithi033 ceph-mon[30860]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/2676172819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 msgr2=0x7fe7b0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/2676172819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0107ae0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009ca0 tx=0x7fe7a002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/2676172819 shutdown_connections 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/2676172819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0108020 0x7fe7b010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/2676172819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0107ae0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/2676172819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7b0101c90 0x7fe7b0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/2676172819 >> 172.21.15.33:0/2676172819 conn(0x7fe7b00fd700 msgr2=0x7fe7b00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/2676172819 shutdown_connections 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.362+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/2676172819 wait complete. 2024-09-10T21:03:51.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 Processor -- start 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 -- start start 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0101c90 0x7fe7b0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7b0108020 0x7fe7b019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0111d30 con 0x7fe7b01076a0 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0111bb0 con 0x7fe7b0108020 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b6ca5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b0111eb0 con 0x7fe7b0101c90 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42568/0 (socket says 172.21.15.33:42568) 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7affff640 1 -- 172.21.15.33:0/1227524909 learned_addr learned my addr 172.21.15.33:0/1227524909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0101c90 0x7fe7b0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.363+0000 7fe7b5242640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7b0108020 0x7fe7b019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7affff640 1 -- 172.21.15.33:0/1227524909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0101c90 msgr2=0x7fe7b0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7affff640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0101c90 0x7fe7b0199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7affff640 1 -- 172.21.15.33:0/1227524909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7b0108020 msgr2=0x7fe7b019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7affff640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7b0108020 0x7fe7b019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7affff640 1 -- 172.21.15.33:0/1227524909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b019f3e0 con 0x7fe7b01076a0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7b5242640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7b0108020 0x7fe7b019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7affff640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0199870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005e30 tx=0x7fe7a002f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7adffb640 1 -- 172.21.15.33:0/1227524909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a0003bf0 con 0x7fe7b01076a0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7adffb640 1 -- 172.21.15.33:0/1227524909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0003d90 con 0x7fe7b01076a0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7adffb640 1 -- 172.21.15.33:0/1227524909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a002fb10 con 0x7fe7b01076a0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b019f670 con 0x7fe7b01076a0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.364+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b01a5d90 con 0x7fe7b01076a0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 Processor -- start 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7adffb640 1 -- 172.21.15.33:0/1227524909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe7a0004680 con 0x7fe7b01076a0 2024-09-10T21:03:51.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 -- start start 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d21d30 0x564ce1d22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d22650 0x564ce1d2e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ce1c99c10 con 0x564ce1d21d30 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ce1c89040 con 0x564ce1d2e5e0 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b907a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ce1c8ce50 con 0x564ce1d22650 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d21d30 0x564ce1d22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d21d30 0x564ce1d22110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42580/0 (socket says 172.21.15.33:42580) 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 -- 172.21.15.33:0/81463677 learned_addr learned my addr 172.21.15.33:0/81463677 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7affff640 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d22650 0x564ce1d2e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 -- 172.21.15.33:0/81463677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d22650 msgr2=0x564ce1d2e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d22650 0x564ce1d2e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 -- 172.21.15.33:0/81463677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 msgr2=0x564ce1d309d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d309d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.367+0000 7fe7b5242640 1 -- 172.21.15.33:0/81463677 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce19e7540 con 0x564ce1d21d30 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.368+0000 7fe7adffb640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe79c0b52e0 0x7fe79c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.368+0000 7fe7b5242640 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d21d30 0x564ce1d22110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8007830 tx=0x7fe7a8007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09e6d379fb2c17e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.368+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe79c0b52e0 0x7fe79c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.368+0000 7fe7adffb640 1 -- 172.21.15.33:0/1227524909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe7a0106f30 con 0x7fe7b01076a0 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.368+0000 7fe7a6ffd640 1 -- 172.21.15.33:0/81463677 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a800d950 con 0x564ce1d21d30 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.368+0000 7fe7a6ffd640 1 -- 172.21.15.33:0/81463677 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a8018070 con 0x564ce1d21d30 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.368+0000 7fe7a6ffd640 1 -- 172.21.15.33:0/81463677 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a8013d70 con 0x564ce1d21d30 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 -- 172.21.15.33:0/81463677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d21d30 msgr2=0x564ce1d22110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe79c0b52e0 0x7fe79c0b77a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564ce1d24eb0 tx=0x7fe79800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d21d30 0x564ce1d22110 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe7a8007830 tx=0x7fe7a8007cf0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 -- 172.21.15.33:0/81463677 shutdown_connections 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d22650 0x564ce1d2e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 --2- 172.21.15.33:0/81463677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d21d30 0x564ce1d22110 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 -- 172.21.15.33:0/81463677 >> 172.21.15.33:0/81463677 conn(0x564ce1d1fb60 msgr2=0x564ce1d20d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 -- 172.21.15.33:0/81463677 shutdown_connections 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 -- 172.21.15.33:0/81463677 wait complete. 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 -- start start 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.369+0000 7fe7b907a740 1 Processor -- start 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d2b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d2bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b907a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ce1aea5f0 con 0x564ce1d22650 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b907a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ce1c24010 con 0x564ce1d21d30 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b907a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ce1d31290 con 0x564ce1d2e5e0 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b907a740 10 client.0 _ll_register_callbacks cb 0x564ce1c83340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d2baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42594/0 (socket says 172.21.15.33:42594) 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 -- 172.21.15.33:0/2996956184 learned_addr learned my addr 172.21.15.33:0/2996956184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d2b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7b5242640 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d2bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 -- 172.21.15.33:0/2996956184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 msgr2=0x564ce1d2bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d2bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 -- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 msgr2=0x564ce1d2b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d2b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 -- 172.21.15.33:0/2996956184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce1c51a00 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe7affff640 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d2baa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00cd680 tx=0x7fe7a00c9d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167553d487d1db9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.370+0000 7fe78ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe78ffff640 1 -- 172.21.15.33:0/2996956184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00cd040 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe78ffff640 1 -- 172.21.15.33:0/2996956184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe7a00cd950 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a8018070 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe78ffff640 1 -- 172.21.15.33:0/2996956184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7a00cce60 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 10 client.15291 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ce1d41210 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe78ffff640 1 -- 172.21.15.33:0/2996956184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a8018070 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe78ffff640 1 -- 172.21.15.33:0/2996956184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe7a00d67f0 con 0x564ce1d22650 2024-09-10T21:03:51.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 10 client.15291 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 10 client.15291 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 20 client.15291 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1d44c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 4 client.15291 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.371+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ce1c9bb60 con 0x564ce1d43820 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.372+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1d44c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.372+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1d44c90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4fe0c8cd2033a6 server_cookie=a56ebbc7fc1c56b2 in_seq=0 out_seq=0 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.372+0000 7fe78ffff640 10 client.15291 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe78ffff640 1 -- 172.21.15.33:0/2996956184 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ce1c9bb60 con 0x564ce1d43820 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe78ffff640 10 client.15291 handle_command_reply: tid=0 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 client.15291 shutdown 2024-09-10T21:03:51.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 msgr2=0x564ce1d2baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d2baa0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00cd680 tx=0x7fe7a00c9d40 comp rx=0 tx=0).stop 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 shutdown_connections 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1d44c90 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d2bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d2baa0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe78ffff640 0 client.15291 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 --2- 172.21.15.33:0/2996956184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d2b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 >> 172.21.15.33:0/2996956184 conn(0x564ce1d1fb60 msgr2=0x564ce1d25700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 shutdown_connections 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 1 -- 172.21.15.33:0/2996956184 wait complete. 2024-09-10T21:03:51.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.373+0000 7fe7b907a740 20 client.15291 trim_cache size 0 max 16384 2024-09-10T21:03:51.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.397+0000 7fe7b907a740 1 Processor -- start 2024-09-10T21:03:51.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.397+0000 7fe7b907a740 1 -- start start 2024-09-10T21:03:51.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d21d30 0x564ce1d43ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d26840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b907a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ce1d42fd0 con 0x564ce1d22650 2024-09-10T21:03:51.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b907a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ce1d436a0 con 0x564ce1d2e5e0 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b907a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ce1d43d70 con 0x564ce1d21d30 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d26840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d26840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42596/0 (socket says 172.21.15.33:42596) 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 -- 172.21.15.33:0/4280570557 learned_addr learned my addr 172.21.15.33:0/4280570557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d21d30 0x564ce1d43ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7b5242640 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 -- 172.21.15.33:0/4280570557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d21d30 msgr2=0x564ce1d43ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d21d30 0x564ce1d43ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 -- 172.21.15.33:0/4280570557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 msgr2=0x564ce1d26d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d26d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 -- 172.21.15.33:0/4280570557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ce1d205e0 con 0x564ce1d22650 2024-09-10T21:03:51.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7affff640 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d26840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005fa0 tx=0x7fe7a00c9e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07d2cbf9217b014 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7a67fc640 1 -- 172.21.15.33:0/4280570557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00cd040 con 0x564ce1d22650 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7a67fc640 1 -- 172.21.15.33:0/4280570557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0036070 con 0x564ce1d22650 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.398+0000 7fe7a67fc640 1 -- 172.21.15.33:0/4280570557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00c9590 con 0x564ce1d22650 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 -- 172.21.15.33:0/4280570557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 msgr2=0x564ce1d26840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d26840 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005fa0 tx=0x7fe7a00c9e70 comp rx=0 tx=0).stop 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 -- 172.21.15.33:0/4280570557 shutdown_connections 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d2e5e0 0x564ce1d26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d26840 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 --2- 172.21.15.33:0/4280570557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d21d30 0x564ce1d43ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 -- 172.21.15.33:0/4280570557 >> 172.21.15.33:0/4280570557 conn(0x564ce1d1fb60 msgr2=0x564ce1d42560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 -- 172.21.15.33:0/4280570557 shutdown_connections 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 -- 172.21.15.33:0/4280570557 wait complete. 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.399+0000 7fe7b907a740 1 -- start start 2024-09-10T21:03:51.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 1 Processor -- start 2024-09-10T21:03:51.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d4cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d4d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d4bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ce1d44b10 con 0x564ce1d22650 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ce1d44440 con 0x564ce1d21d30 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ce1d451e0 con 0x564ce1d2e5e0 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b907a740 10 client.0 _ll_register_callbacks cb 0x7fe7a0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d4d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d4d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42600/0 (socket says 172.21.15.33:42600) 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 -- 172.21.15.33:0/3441253927 learned_addr learned my addr 172.21.15.33:0/3441253927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d4cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7b5242640 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d4bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 -- 172.21.15.33:0/3441253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 msgr2=0x564ce1d4bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d4bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 -- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 msgr2=0x564ce1d4cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d4cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.400+0000 7fe7affff640 1 -- 172.21.15.33:0/3441253927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ce1d26330 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7affff640 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d4d4b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006010 tx=0x7fe7a002f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7276b142d3064406 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe78ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe78ffff640 1 -- 172.21.15.33:0/3441253927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00c9590 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe78ffff640 1 -- 172.21.15.33:0/3441253927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe7a00cd950 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7a0036070 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe78ffff640 1 -- 172.21.15.33:0/3441253927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe7a00cd040 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 10 client.15297 fetch_fsmap learned FSMap version 27 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564ce1d26330 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe78ffff640 1 -- 172.21.15.33:0/3441253927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7a0036070 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe78ffff640 1 -- 172.21.15.33:0/3441253927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 27) ==== 8362+0+0 (secure 0 0 0) 0x7fe7a0092200 con 0x564ce1d22650 2024-09-10T21:03:51.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 10 client.15297 fetch_fsmap finished waiting for FSMap version 27 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 10 client.15297 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 20 client.15297 populate_metadata read hostname 'smithi033' 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1db3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 4 client.15297 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.401+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "status"}) -- 0x564ce1c97610 con 0x564ce1d43820 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1db3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe78ffff640 10 client.15297 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe7b4a41640 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1db3b80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2a4d297db6c20e8 server_cookie=9cdeb076ccae53da in_seq=0 out_seq=0 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe78ffff640 1 -- 172.21.15.33:0/3441253927 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x564ce1c97610 con 0x564ce1d43820 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe78ffff640 10 client.15297 handle_command_reply: tid=0 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe7b907a740 1 client.15297 shutdown 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 msgr2=0x564ce1d4d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.402+0000 7fe7b907a740 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d4d4b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0006010 tx=0x7fe7a002f610 comp rx=0 tx=0).stop 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 shutdown_connections 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564ce1d43820 0x564ce1db3b80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ce1d2e5e0 0x564ce1d4bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ce1d22650 0x564ce1d4d4b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 --2- 172.21.15.33:0/3441253927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ce1d21d30 0x564ce1d4cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 >> 172.21.15.33:0/3441253927 conn(0x564ce1d1fb60 msgr2=0x564ce1d26840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe78ffff640 0 client.15297 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 shutdown_connections 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 1 -- 172.21.15.33:0/3441253927 wait complete. 2024-09-10T21:03:51.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b907a740 20 client.15297 trim_cache size 0 max 16384 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "cluster_fsid": "a07b33bc-6fb6-11ef-bcea-c7b262605968", 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "whoami": 0, 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 24566, 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "want_state": "up:active", 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "state": "up:active", 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs", 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "rank_uptime": 109.616542982, 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "mdsmap_epoch": 27, 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch": 77, 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_epoch_barrier": 75, 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout: "uptime": 120.430123624 2024-09-10T21:03:51.413 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe79c0b52e0 msgr2=0x7fe79c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe79c0b52e0 0x7fe79c0b77a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x564ce1d24eb0 tx=0x7fe79800b390 comp rx=0 tx=0).stop 2024-09-10T21:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 msgr2=0x7fe7b0199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0199870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0005e30 tx=0x7fe7a002f3e0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 shutdown_connections 2024-09-10T21:03:51.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe79c0b52e0 0x7fe79c0b77a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7b0108020 0x7fe7b019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7b01076a0 0x7fe7b0199870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 --2- 172.21.15.33:0/1227524909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7b0101c90 0x7fe7b0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.403+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 >> 172.21.15.33:0/1227524909 conn(0x7fe7b00fd700 msgr2=0x7fe7b0104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.404+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 shutdown_connections 2024-09-10T21:03:51.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:51.404+0000 7fe7b6ca5640 1 -- 172.21.15.33:0/1227524909 wait complete. 2024-09-10T21:03:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:51 smithi104 ceph-mon[35325]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T21:03:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:51 smithi174 ceph-mon[35635]: pgmap v342: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-10T21:03:52.171 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T21:03:52.200 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T21:03:52.200 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-10T21:03:52.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:52.200 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-09-10T21:03:52.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:52.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-09-10T21:03:52.289 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-09-10T21:03:52.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:03:52.291 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-10T21:03:52.291 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:03:52.291 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:03:52.291 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T21:03:52.589 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:53.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 -- 172.21.15.33:0/1090899404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 msgr2=0x7f2f2810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:53.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1090899404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2810f950 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2f20009ca0 tx=0x7f2f2002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 -- 172.21.15.33:0/1090899404 shutdown_connections 2024-09-10T21:03:53.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1090899404 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2810f950 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1090899404 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f281087d0 0x7f2f28108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1090899404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f28102dc0 0x7f2f281031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 -- 172.21.15.33:0/1090899404 >> 172.21.15.33:0/1090899404 conn(0x7f2f280fe830 msgr2=0x7f2f28100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 -- 172.21.15.33:0/1090899404 shutdown_connections 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 -- 172.21.15.33:0/1090899404 wait complete. 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 Processor -- start 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.055+0000 7f2f2e547640 1 -- start start 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2e547640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f28102dc0 0x7f2f281992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2e547640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f281087d0 0x7f2f28199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2e547640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2e547640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28112e10 con 0x7f2f28109150 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2e547640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28112c90 con 0x7f2f28102dc0 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2e547640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f28112f90 con 0x7f2f281087d0 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:53.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f27fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f28102dc0 0x7f2f281992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42628/0 (socket says 172.21.15.33:42628) 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 -- 172.21.15.33:0/1854917161 learned_addr learned my addr 172.21.15.33:0/1854917161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f277fe640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f281087d0 0x7f2f28199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 -- 172.21.15.33:0/1854917161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f281087d0 msgr2=0x7f2f28199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f281087d0 0x7f2f28199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 -- 172.21.15.33:0/1854917161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f28102dc0 msgr2=0x7f2f281992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f28102dc0 0x7f2f281992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f2cae4640 1 -- 172.21.15.33:0/1854917161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2819f370 con 0x7f2f28109150 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.056+0000 7f2f27fff640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f28102dc0 0x7f2f281992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.057+0000 7f2f2cae4640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2819eba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005cc0 tx=0x7f2f2002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.057+0000 7f2f257fa640 1 -- 172.21.15.33:0/1854917161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003bf0 con 0x7f2f28109150 2024-09-10T21:03:53.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.057+0000 7f2f257fa640 1 -- 172.21.15.33:0/1854917161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f20003d90 con 0x7f2f28109150 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.057+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2819f600 con 0x7f2f28109150 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.057+0000 7f2f257fa640 1 -- 172.21.15.33:0/1854917161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f2002fbc0 con 0x7f2f28109150 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.057+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f281a5d20 con 0x7f2f28109150 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.058+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f281031a0 con 0x7f2f28109150 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.060+0000 7f2f257fa640 1 -- 172.21.15.33:0/1854917161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2f20004610 con 0x7f2f28109150 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.061+0000 7f2f257fa640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2f100b5360 0x7f2f100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.061+0000 7f2f27fff640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2f100b5360 0x7f2f100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.061+0000 7f2f257fa640 1 -- 172.21.15.33:0/1854917161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2f201071c0 con 0x7f2f28109150 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.061+0000 7f2f27fff640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2f100b5360 0x7f2f100b7820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800bfd0 tx=0x7f2f1800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:53.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.064+0000 7f2f257fa640 1 -- 172.21.15.33:0/1854917161 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2f200ce080 con 0x7f2f28109150 2024-09-10T21:03:53.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.290+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2f2819a3f0 con 0x7f2f28109150 2024-09-10T21:03:53.294 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:53 smithi033 ceph-mon[30860]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:03:53.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.290+0000 7f2f257fa640 1 -- 172.21.15.33:0/1854917161 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f2f200cd980 con 0x7f2f28109150 2024-09-10T21:03:53.295 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2f100b5360 msgr2=0x7f2f100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2f100b5360 0x7f2f100b7820 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f1800bfd0 tx=0x7f2f1800b390 comp rx=0 tx=0).stop 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 msgr2=0x7f2f2819eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2819eba0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2f20005cc0 tx=0x7f2f2002f380 comp rx=0 tx=0).stop 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 shutdown_connections 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2f100b5360 0x7f2f100b7820 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f28109150 0x7f2f2819eba0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f281087d0 0x7f2f28199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 --2- 172.21.15.33:0/1854917161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2f28102dc0 0x7f2f281992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 >> 172.21.15.33:0/1854917161 conn(0x7f2f280fe830 msgr2=0x7f2f28100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 shutdown_connections 2024-09-10T21:03:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:53.294+0000 7f2f2e547640 1 -- 172.21.15.33:0/1854917161 wait complete. 2024-09-10T21:03:53.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:53 smithi104 ceph-mon[35325]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:03:53.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:53 smithi174 ceph-mon[35635]: pgmap v343: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:03:54.134 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:03:54.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:54.135 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-09-10T21:03:54.153 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-09-10T21:03:54.154 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42184sec preferred_lft 42184sec 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:03:54.155 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-09-10T21:03:54.155 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:03:54.156 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:03:54.156 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-10T21:03:54.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:03:54.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-09-10T21:03:54.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-09-10T21:03:54.156 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:03:54.156 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-10T21:03:54.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:03:54.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:54 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1854917161' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T21:03:54.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:54.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:54.383 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:03:54.410 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-09-10T21:03:54.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:54.412 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-09-10T21:03:54.496 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-09-10T21:03:54.496 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:03:54.496 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:03:54.496 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:03:54.496 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:03:54.497 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:03:54.497 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-10T21:03:54.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:03:54.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:03:54.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:03:54.497 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-10T21:03:54.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:54 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1854917161' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T21:03:54.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:03:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:54 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1854917161' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-10T21:03:54.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:54.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:54.817 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-09-10T21:03:54.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:54.834 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-09-10T21:03:54.890 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:03:54.890 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-10T21:03:54.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:03:54.890 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T21:03:54.890 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-10T21:03:54.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:03:55.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:55.009 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T21:03:55.010 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-10T21:03:55.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:03:55.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:55.138 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:03:55.138 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-09-10T21:03:55.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-09-10T21:03:55.138 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:03:55.138 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-09-10T21:03:55.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:03:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:55.218 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:03:55.218 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:03:55.234 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:03:55.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:03:55.234 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:03:55.290 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-10T21:03:55.290 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-10T21:03:55.290 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/44306e77-b631-4a07-b5a9-d08fc1590f9a 2024-09-10T21:03:55.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:55.290 DEBUG:teuthology.orchestra.run.smithi033:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/44306e77-b631-4a07-b5a9-d08fc1590f9a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-10T21:03:55.353 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:55 smithi033 ceph-mon[30860]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T21:03:55.402 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-10T21:03:55.402 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-10T21:03:55.402 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2024-09-10T21:03:55.538 INFO:teuthology.orchestra.run.smithi033.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-10T21:03:55.538 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-10T21:03:55.538 INFO:teuthology.orchestra.run.smithi033.stdout:invalid new device string format 2024-09-10T21:03:55.538 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: resolved to: "172.21.15.33:3300,172.21.15.104:3300,172.21.15.174:3300" 2024-09-10T21:03:55.538 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.33:3300,172.21.15.104:3300,172.21.15.174:3300:/volumes/qa/sv_1/44306e77-b631-4a07-b5a9-d08fc1590f9a 2024-09-10T21:03:55.539 INFO:teuthology.orchestra.run.smithi033.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=a07b33bc-6fb6-11ef-bcea-c7b262605968" will pass to kernel 2024-09-10T21:03:55.539 INFO:teuthology.orchestra.run.smithi033.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:03:55.539 INFO:teuthology.orchestra.run.smithi033.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:03:55.539 INFO:teuthology.orchestra.run.smithi033.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:03:55.539 INFO:teuthology.orchestra.run.smithi033.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:03:55.539 INFO:teuthology.orchestra.run.smithi033.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:03:55.541 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:03:55.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:03:55.542 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:03:55.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:55 smithi104 ceph-mon[35325]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T21:03:55.614 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:03:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:55 smithi174 ceph-mon[35635]: pgmap v344: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T21:03:55.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:03:55.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:55.715 INFO:teuthology.orchestra.run.smithi033.stdout:client24947 2024-09-10T21:03:55.715 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T21:03:55.716 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:03:55.734 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:03:55.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:03:55.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:55.833 INFO:teuthology.orchestra.run.smithi033.stdout:client24947 2024-09-10T21:03:55.833 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:03:55.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:03:55.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:55.891 INFO:teuthology.orchestra.run.smithi033.stdout:client24947 2024-09-10T21:03:55.891 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968.client24947/status') 2024-09-10T21:03:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:03:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:instance: client.24947 (3)172.21.15.33:0/1212617070 2024-09-10T21:03:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:blocklisted: false 2024-09-10T21:03:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records in 2024-09-10T21:03:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:0+1 records out 2024-09-10T21:03:55.952 INFO:teuthology.orchestra.run.smithi033.stderr:71 bytes copied, 2.4163e-05 s, 2.9 MB/s 2024-09-10T21:03:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:03:55.959 INFO:teuthology.run_tasks:Running task exec... 2024-09-10T21:03:55.970 INFO:teuthology.task.exec:Executing custom commands... 2024-09-10T21:03:55.970 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi033.front.sepia.ceph.com 2024-09-10T21:03:55.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-09-10T21:03:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 -- 172.21.15.33:0/1158061988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d006bf10 msgr2=0x7f80d0105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 --2- 172.21.15.33:0/1158061988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d006bf10 0x7f80d0105620 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009900 tx=0x7f80c002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:56.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 -- 172.21.15.33:0/1158061988 shutdown_connections 2024-09-10T21:03:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 --2- 172.21.15.33:0/1158061988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d01086d0 0x7f80d010ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 --2- 172.21.15.33:0/1158061988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d0105b60 0x7f80d0107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 --2- 172.21.15.33:0/1158061988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d006bf10 0x7f80d0105620 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 -- 172.21.15.33:0/1158061988 >> 172.21.15.33:0/1158061988 conn(0x7f80d00fd150 msgr2=0x7f80d00ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 -- 172.21.15.33:0/1158061988 shutdown_connections 2024-09-10T21:03:56.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.071+0000 7f80d4f05640 1 -- 172.21.15.33:0/1158061988 wait complete. 2024-09-10T21:03:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 Processor -- start 2024-09-10T21:03:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 -- start start 2024-09-10T21:03:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d006bf10 0x7f80d0198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d0105b60 0x7f80d0199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d01086d0 0x7f80d019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:56.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010e620 con 0x7f80d01086d0 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010e4a0 con 0x7f80d006bf10 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80d4f05640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80d010e7a0 con 0x7f80d0105b60 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80ced9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d01086d0 0x7f80d019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80ced9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d01086d0 0x7f80d019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42652/0 (socket says 172.21.15.33:42652) 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.072+0000 7f80ced9d640 1 -- 172.21.15.33:0/2738736259 learned_addr learned my addr 172.21.15.33:0/2738736259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80ce59c640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d006bf10 0x7f80d0198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80cdd9b640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d0105b60 0x7f80d0199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80ced9d640 1 -- 172.21.15.33:0/2738736259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d0105b60 msgr2=0x7f80d0199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80ced9d640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d0105b60 0x7f80d0199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80ced9d640 1 -- 172.21.15.33:0/2738736259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d006bf10 msgr2=0x7f80d0198dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80ced9d640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d006bf10 0x7f80d0198dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80ced9d640 1 -- 172.21.15.33:0/2738736259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80d019a910 con 0x7f80d01086d0 2024-09-10T21:03:56.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80ced9d640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d01086d0 0x7f80d019a050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009fd0 tx=0x7f80c800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:56.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80c77fe640 1 -- 172.21.15.33:0/2738736259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8017070 con 0x7f80d01086d0 2024-09-10T21:03:56.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80c77fe640 1 -- 172.21.15.33:0/2738736259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80c8003e20 con 0x7f80d01086d0 2024-09-10T21:03:56.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80c77fe640 1 -- 172.21.15.33:0/2738736259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c8010460 con 0x7f80d01086d0 2024-09-10T21:03:56.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80d019ac00 con 0x7f80d01086d0 2024-09-10T21:03:56.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.073+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80d01a12d0 con 0x7f80d01086d0 2024-09-10T21:03:56.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.074+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80d00fec40 con 0x7f80d01086d0 2024-09-10T21:03:56.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.075+0000 7f80c77fe640 1 -- 172.21.15.33:0/2738736259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f80c8003970 con 0x7f80d01086d0 2024-09-10T21:03:56.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.076+0000 7f80c77fe640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80bc0b52e0 0x7f80bc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:56.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.077+0000 7f80ce59c640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80bc0b52e0 0x7f80bc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:56.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.077+0000 7f80c77fe640 1 -- 172.21.15.33:0/2738736259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f80c80e59b0 con 0x7f80d01086d0 2024-09-10T21:03:56.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.077+0000 7f80ce59c640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80bc0b52e0 0x7f80bc0b77a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c00314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:56.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.081+0000 7f80c77fe640 1 -- 172.21.15.33:0/2738736259 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f80c80ac830 con 0x7f80d01086d0 2024-09-10T21:03:56.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:56.305+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f80d002e5c0 con 0x7f80d01086d0 2024-09-10T21:03:57.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.172+0000 7f80c77fe640 1 -- 172.21.15.33:0/2738736259 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f80c80ac130 con 0x7f80d01086d0 2024-09-10T21:03:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80bc0b52e0 msgr2=0x7f80bc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80bc0b52e0 0x7f80bc0b77a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f80c0000c00 tx=0x7f80c00314a0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d01086d0 msgr2=0x7f80d019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d01086d0 0x7f80d019a050 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009fd0 tx=0x7f80c800b260 comp rx=0 tx=0).stop 2024-09-10T21:03:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 shutdown_connections 2024-09-10T21:03:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f80bc0b52e0 0x7f80bc0b77a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80d01086d0 0x7f80d019a050 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80d0105b60 0x7f80d0199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 --2- 172.21.15.33:0/2738736259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f80d006bf10 0x7f80d0198dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 >> 172.21.15.33:0/2738736259 conn(0x7f80d00fd150 msgr2=0x7f80d0103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 shutdown_connections 2024-09-10T21:03:57.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.175+0000 7f80d4f05640 1 -- 172.21.15.33:0/2738736259 wait complete. 2024-09-10T21:03:57.193 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-10T21:03:57.209 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-10T21:03:57.227 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:03:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 -- 172.21.15.33:0/62626525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 msgr2=0x7fd964103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 --2- 172.21.15.33:0/62626525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964103f60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd958009930 tx=0x7fd95802a420 comp rx=0 tx=0).stop 2024-09-10T21:03:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 -- 172.21.15.33:0/62626525 shutdown_connections 2024-09-10T21:03:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 --2- 172.21.15.33:0/62626525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9641044a0 0x7fd96410ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 --2- 172.21.15.33:0/62626525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964103f60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 --2- 172.21.15.33:0/62626525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd964102990 0x7fd964102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 -- 172.21.15.33:0/62626525 >> 172.21.15.33:0/62626525 conn(0x7fd9640fe270 msgr2=0x7fd964100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:57.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 -- 172.21.15.33:0/62626525 shutdown_connections 2024-09-10T21:03:57.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 -- 172.21.15.33:0/62626525 wait complete. 2024-09-10T21:03:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.336+0000 7fd969016640 1 Processor -- start 2024-09-10T21:03:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd969016640 1 -- start start 2024-09-10T21:03:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd969016640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd964102990 0x7fd964076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd969016640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd969016640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9641044a0 0x7fd964075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd969016640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd96410e600 con 0x7fd964103ae0 2024-09-10T21:03:57.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd969016640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd96410e480 con 0x7fd9641044a0 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd969016640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd96410e780 con 0x7fd964102990 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964075540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42664/0 (socket says 172.21.15.33:42664) 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 -- 172.21.15.33:0/143770535 learned_addr learned my addr 172.21.15.33:0/143770535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd962d9d640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd964102990 0x7fd964076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96359e640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9641044a0 0x7fd964075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 -- 172.21.15.33:0/143770535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd964102990 msgr2=0x7fd964076ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd964102990 0x7fd964076ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 -- 172.21.15.33:0/143770535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9641044a0 msgr2=0x7fd964075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9641044a0 0x7fd964075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.337+0000 7fd96259c640 1 -- 172.21.15.33:0/143770535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9640762f0 con 0x7fd964103ae0 2024-09-10T21:03:57.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.338+0000 7fd96259c640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964075540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd958009a50 tx=0x7fd9580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:57.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.338+0000 7fd94bfff640 1 -- 172.21.15.33:0/143770535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95803f030 con 0x7fd964103ae0 2024-09-10T21:03:57.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.338+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9641a2de0 con 0x7fd964103ae0 2024-09-10T21:03:57.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.338+0000 7fd94bfff640 1 -- 172.21.15.33:0/143770535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd958003cb0 con 0x7fd964103ae0 2024-09-10T21:03:57.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.338+0000 7fd94bfff640 1 -- 172.21.15.33:0/143770535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd958035070 con 0x7fd964103ae0 2024-09-10T21:03:57.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.338+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9641a3240 con 0x7fd964103ae0 2024-09-10T21:03:57.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.339+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd964109ce0 con 0x7fd964103ae0 2024-09-10T21:03:57.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.341+0000 7fd94bfff640 1 -- 172.21.15.33:0/143770535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd95803bb90 con 0x7fd964103ae0 2024-09-10T21:03:57.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.342+0000 7fd94bfff640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd94c0b52e0 0x7fd94c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.342+0000 7fd962d9d640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd94c0b52e0 0x7fd94c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.342+0000 7fd94bfff640 1 -- 172.21.15.33:0/143770535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd958106820 con 0x7fd964103ae0 2024-09-10T21:03:57.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.342+0000 7fd962d9d640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd94c0b52e0 0x7fd94c0b77a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd954005fd0 tx=0x7fd954009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:57.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.345+0000 7fd94bfff640 1 -- 172.21.15.33:0/143770535 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd9580cd090 con 0x7fd964103ae0 2024-09-10T21:03:57.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:57 smithi033 ceph-mon[30860]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T21:03:57.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:57 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2738736259' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T21:03:57.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:57 smithi174 ceph-mon[35635]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T21:03:57.446 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:57 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2738736259' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T21:03:57.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.529+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd96402e810 con 0x7fd964103ae0 2024-09-10T21:03:57.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.531+0000 7fd94bfff640 1 -- 172.21.15.33:0/143770535 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22763 (secure 0 0 0) 0x7fd9580d1ac0 con 0x7fd964103ae0 2024-09-10T21:03:57.535 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:57.535 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:41.855317+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:03:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd94c0b52e0 msgr2=0x7fd94c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd94c0b52e0 0x7fd94c0b77a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd954005fd0 tx=0x7fd954009680 comp rx=0 tx=0).stop 2024-09-10T21:03:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 msgr2=0x7fd964075540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964075540 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd958009a50 tx=0x7fd9580042f0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 shutdown_connections 2024-09-10T21:03:57.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd94c0b52e0 0x7fd94c0b77a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd9641044a0 0x7fd964075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd964103ae0 0x7fd964075540 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 --2- 172.21.15.33:0/143770535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd964102990 0x7fd964076ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 >> 172.21.15.33:0/143770535 conn(0x7fd9640fe270 msgr2=0x7fd964106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 shutdown_connections 2024-09-10T21:03:57.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.533+0000 7fd969016640 1 -- 172.21.15.33:0/143770535 wait complete. 2024-09-10T21:03:57.548 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T20:57:24.470902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T21:01:57.440910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T21:01:59.340474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:03:57.548 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:03:57.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:57 smithi104 ceph-mon[35325]: pgmap v345: 97 pgs: 97 active+clean; 645 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-10T21:03:57.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:57 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2738736259' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} : dispatch 2024-09-10T21:03:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 -- 172.21.15.33:0/3153077632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010e4b0 msgr2=0x7f24f010e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 --2- 172.21.15.33:0/3153077632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010e4b0 0x7f24f010e930 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f24d8009930 tx=0x7f24d802a420 comp rx=0 tx=0).stop 2024-09-10T21:03:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 -- 172.21.15.33:0/3153077632 shutdown_connections 2024-09-10T21:03:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 --2- 172.21.15.33:0/3153077632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f010ee70 0x7f24f01157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 --2- 172.21.15.33:0/3153077632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010e4b0 0x7f24f010e930 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 --2- 172.21.15.33:0/3153077632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24f010d410 0x7f24f010d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 -- 172.21.15.33:0/3153077632 >> 172.21.15.33:0/3153077632 conn(0x7f24f006d1f0 msgr2=0x7f24f006d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 -- 172.21.15.33:0/3153077632 shutdown_connections 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.652+0000 7f24f6862640 1 -- 172.21.15.33:0/3153077632 wait complete. 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 Processor -- start 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 -- start start 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f010d410 0x7f24f01a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24f010e4b0 0x7f24f01a3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010ee70 0x7f24f01a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0119080 con 0x7f24f010ee70 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0118f00 con 0x7f24f010e4b0 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f6862640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24f0119200 con 0x7f24f010d410 2024-09-10T21:03:57.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010ee70 0x7f24f01a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010ee70 0x7f24f01a4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42680/0 (socket says 172.21.15.33:42680) 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 -- 172.21.15.33:0/3793522233 learned_addr learned my addr 172.21.15.33:0/3793522233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 -- 172.21.15.33:0/3793522233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f010d410 msgr2=0x7f24f01a37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24e7fff640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24f010e4b0 0x7f24f01a3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f010d410 0x7f24f01a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 -- 172.21.15.33:0/3793522233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24f010e4b0 msgr2=0x7f24f01a3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24f010e4b0 0x7f24f01a3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24f4dff640 1 -- 172.21.15.33:0/3793522233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24f01a5320 con 0x7f24f010ee70 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.653+0000 7f24e7fff640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24f010e4b0 0x7f24f01a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.654+0000 7f24f4dff640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010ee70 0x7f24f01a4a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b970 tx=0x7f24e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.654+0000 7f24edffb640 1 -- 172.21.15.33:0/3793522233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e8019070 con 0x7f24f010ee70 2024-09-10T21:03:57.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.654+0000 7f24edffb640 1 -- 172.21.15.33:0/3793522233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24e8014440 con 0x7f24f010ee70 2024-09-10T21:03:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.654+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24f01a5610 con 0x7f24f010ee70 2024-09-10T21:03:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.654+0000 7f24edffb640 1 -- 172.21.15.33:0/3793522233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24e80075d0 con 0x7f24f010ee70 2024-09-10T21:03:57.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.654+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24f01abce0 con 0x7f24f010ee70 2024-09-10T21:03:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.655+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f0114670 con 0x7f24f010ee70 2024-09-10T21:03:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.656+0000 7f24edffb640 1 -- 172.21.15.33:0/3793522233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f24e800cb40 con 0x7f24f010ee70 2024-09-10T21:03:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.657+0000 7f24edffb640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24dc0b5360 0x7f24dc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.657+0000 7f24effff640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24dc0b5360 0x7f24dc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.657+0000 7f24edffb640 1 -- 172.21.15.33:0/3793522233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f24e80e53c0 con 0x7f24f010ee70 2024-09-10T21:03:57.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.657+0000 7f24effff640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24dc0b5360 0x7f24dc0b7820 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e0005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:57.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.661+0000 7f24edffb640 1 -- 172.21.15.33:0/3793522233 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f24e80ac280 con 0x7f24f010ee70 2024-09-10T21:03:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.847+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f24f0063d50 con 0x7f24f010ee70 2024-09-10T21:03:57.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.847+0000 7f24edffb640 1 -- 172.21.15.33:0/3793522233 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f24e80abb80 con 0x7f24f010ee70 2024-09-10T21:03:57.851 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-09-10T21:03:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24dc0b5360 msgr2=0x7f24dc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24dc0b5360 0x7f24dc0b7820 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f24e0005fd0 tx=0x7f24e0005850 comp rx=0 tx=0).stop 2024-09-10T21:03:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010ee70 msgr2=0x7f24f01a4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010ee70 0x7f24f01a4a60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b970 tx=0x7f24e800be30 comp rx=0 tx=0).stop 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 shutdown_connections 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24dc0b5360 0x7f24dc0b7820 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24f010ee70 0x7f24f01a4a60 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f24f010e4b0 0x7f24f01a3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 --2- 172.21.15.33:0/3793522233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24f010d410 0x7f24f01a37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 >> 172.21.15.33:0/3793522233 conn(0x7f24f006d1f0 msgr2=0x7f24f0111840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 shutdown_connections 2024-09-10T21:03:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.849+0000 7f24f6862640 1 -- 172.21.15.33:0/3793522233 wait complete. 2024-09-10T21:03:57.865 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T21:03:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.967+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/3914133042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc541029b0 msgr2=0x7fcc54102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.967+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/3914133042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc541029b0 0x7fcc54102e30 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009930 tx=0x7fcc4802a420 comp rx=0 tx=0).stop 2024-09-10T21:03:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.967+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/3914133042 shutdown_connections 2024-09-10T21:03:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.967+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/3914133042 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc54103370 0x7fcc54109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.967+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/3914133042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc541029b0 0x7fcc54102e30 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.967+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/3914133042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54101860 0x7fcc54101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.967+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/3914133042 >> 172.21.15.33:0/3914133042 conn(0x7fcc540fd140 msgr2=0x7fcc540ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/3914133042 shutdown_connections 2024-09-10T21:03:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/3914133042 wait complete. 2024-09-10T21:03:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 Processor -- start 2024-09-10T21:03:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 -- start start 2024-09-10T21:03:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc54101860 0x7fcc54105600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc541029b0 0x7fcc54103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54103370 0x7fcc541043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410d4d0 con 0x7fcc54101860 2024-09-10T21:03:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410d350 con 0x7fcc541029b0 2024-09-10T21:03:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.968+0000 7fcc5a4b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5410d650 con 0x7fcc54103370 2024-09-10T21:03:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc54101860 0x7fcc54105600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc54101860 0x7fcc54105600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42690/0 (socket says 172.21.15.33:42690) 2024-09-10T21:03:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 -- 172.21.15.33:0/2726859601 learned_addr learned my addr 172.21.15.33:0/2726859601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc58a55640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54103370 0x7fcc541043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc537fe640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc541029b0 0x7fcc54103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 -- 172.21.15.33:0/2726859601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54103370 msgr2=0x7fcc541043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54103370 0x7fcc541043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 -- 172.21.15.33:0/2726859601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc541029b0 msgr2=0x7fcc54103e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc541029b0 0x7fcc54103e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 -- 172.21.15.33:0/2726859601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc54104c10 con 0x7fcc54101860 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc58a55640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54103370 0x7fcc541043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc53fff640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc54101860 0x7fcc54105600 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400d770 tx=0x7fcc4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc517fa640 1 -- 172.21.15.33:0/2726859601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44017070 con 0x7fcc54101860 2024-09-10T21:03:57.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc517fa640 1 -- 172.21.15.33:0/2726859601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc440054d0 con 0x7fcc54101860 2024-09-10T21:03:57.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc541a2e80 con 0x7fcc54101860 2024-09-10T21:03:57.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc517fa640 1 -- 172.21.15.33:0/2726859601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc44004c30 con 0x7fcc54101860 2024-09-10T21:03:57.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.969+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc541a32e0 con 0x7fcc54101860 2024-09-10T21:03:57.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.970+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc541a3510 con 0x7fcc54101860 2024-09-10T21:03:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.972+0000 7fcc517fa640 1 -- 172.21.15.33:0/2726859601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fcc44004dd0 con 0x7fcc54101860 2024-09-10T21:03:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.972+0000 7fcc517fa640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcc3c0b5360 0x7fcc3c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.972+0000 7fcc517fa640 1 -- 172.21.15.33:0/2726859601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcc440e5ab0 con 0x7fcc54101860 2024-09-10T21:03:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.972+0000 7fcc537fe640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcc3c0b5360 0x7fcc3c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.973+0000 7fcc537fe640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcc3c0b5360 0x7fcc3c0b7820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc480314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:57.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:57.975+0000 7fcc517fa640 1 -- 172.21.15.33:0/2726859601 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fcc440ac9f0 con 0x7fcc54101860 2024-09-10T21:03:58.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.161+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcc541a36e0 con 0x7fcc54101860 2024-09-10T21:03:58.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.161+0000 7fcc517fa640 1 -- 172.21.15.33:0/2726859601 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7fcc440ac2f0 con 0x7fcc54101860 2024-09-10T21:03:58.165 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-09-10T21:03:58.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcc3c0b5360 msgr2=0x7fcc3c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcc3c0b5360 0x7fcc3c0b7820 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc48000c00 tx=0x7fcc480314a0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc54101860 msgr2=0x7fcc54105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc54101860 0x7fcc54105600 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcc4400d770 tx=0x7fcc4400dc30 comp rx=0 tx=0).stop 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 shutdown_connections 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcc3c0b5360 0x7fcc3c0b7820 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc54103370 0x7fcc541043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcc541029b0 0x7fcc54103e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 --2- 172.21.15.33:0/2726859601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc54101860 0x7fcc54105600 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 >> 172.21.15.33:0/2726859601 conn(0x7fcc540fd140 msgr2=0x7fcc540fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 shutdown_connections 2024-09-10T21:03:58.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.164+0000 7fcc5a4b8640 1 -- 172.21.15.33:0/2726859601 wait complete. 2024-09-10T21:03:58.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T21:03:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.281+0000 7f8469495640 1 -- 172.21.15.33:0/3555267383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8464105ac0 msgr2=0x7f8464107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 --2- 172.21.15.33:0/3555267383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8464105ac0 0x7f8464107f30 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8458009900 tx=0x7f845802a410 comp rx=0 tx=0).stop 2024-09-10T21:03:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 -- 172.21.15.33:0/3555267383 shutdown_connections 2024-09-10T21:03:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 --2- 172.21.15.33:0/3555267383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8464108670 0x7f846410ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 --2- 172.21.15.33:0/3555267383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8464105ac0 0x7f8464107f30 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 --2- 172.21.15.33:0/3555267383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f846406afc0 0x7f8464105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 -- 172.21.15.33:0/3555267383 >> 172.21.15.33:0/3555267383 conn(0x7f84640fd140 msgr2=0x7f84640ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:58.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 -- 172.21.15.33:0/3555267383 shutdown_connections 2024-09-10T21:03:58.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 -- 172.21.15.33:0/3555267383 wait complete. 2024-09-10T21:03:58.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 Processor -- start 2024-09-10T21:03:58.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.282+0000 7f8469495640 1 -- start start 2024-09-10T21:03:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846406afc0 0x7f8464198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8464105ac0 0x7f8464199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8464108670 0x7f846419a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f846410e5c0 con 0x7f846406afc0 2024-09-10T21:03:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f846410e440 con 0x7f8464108670 2024-09-10T21:03:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f846410e740 con 0x7f8464105ac0 2024-09-10T21:03:58.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846406afc0 0x7f8464198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846406afc0 0x7f8464198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42710/0 (socket says 172.21.15.33:42710) 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 -- 172.21.15.33:0/187105459 learned_addr learned my addr 172.21.15.33:0/187105459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f84627fc640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8464105ac0 0x7f8464199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f84637fe640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8464108670 0x7f846419a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 -- 172.21.15.33:0/187105459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8464105ac0 msgr2=0x7f8464199420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8464105ac0 0x7f8464199420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 -- 172.21.15.33:0/187105459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8464108670 msgr2=0x7f846419a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8464108670 0x7f846419a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 -- 172.21.15.33:0/187105459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846419a910 con 0x7f846406afc0 2024-09-10T21:03:58.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f84637fe640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8464108670 0x7f846419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f84627fc640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8464105ac0 0x7f8464199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8462ffd640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846406afc0 0x7f8464198dd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f845400d950 tx=0x7f845400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f844bfff640 1 -- 172.21.15.33:0/187105459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454017030 con 0x7f846406afc0 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f844bfff640 1 -- 172.21.15.33:0/187105459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8454004020 con 0x7f846406afc0 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846419ac00 con 0x7f846406afc0 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f844bfff640 1 -- 172.21.15.33:0/187105459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540047a0 con 0x7f846406afc0 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.283+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84641a12d0 con 0x7f846406afc0 2024-09-10T21:03:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.284+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84640fec30 con 0x7f846406afc0 2024-09-10T21:03:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.286+0000 7f844bfff640 1 -- 172.21.15.33:0/187105459 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8454009a30 con 0x7f846406afc0 2024-09-10T21:03:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.287+0000 7f844bfff640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f844c0b52e0 0x7f844c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.287+0000 7f844bfff640 1 -- 172.21.15.33:0/187105459 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f84540e5cf0 con 0x7f846406afc0 2024-09-10T21:03:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.287+0000 7f84627fc640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f844c0b52e0 0x7f844c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.287+0000 7f84627fc640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f844c0b52e0 0x7f844c0b77a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8458000c00 tx=0x7f84580314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.290+0000 7f844bfff640 1 -- 172.21.15.33:0/187105459 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f84540acbb0 con 0x7f846406afc0 2024-09-10T21:03:58.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.476+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8464063d50 con 0x7f846406afc0 2024-09-10T21:03:58.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.476+0000 7f844bfff640 1 -- 172.21.15.33:0/187105459 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7f84540ac4b0 con 0x7f846406afc0 2024-09-10T21:03:58.480 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-09-10T21:03:58.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f844c0b52e0 msgr2=0x7f844c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f844c0b52e0 0x7f844c0b77a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8458000c00 tx=0x7f84580314a0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846406afc0 msgr2=0x7f8464198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846406afc0 0x7f8464198dd0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f845400d950 tx=0x7f845400de10 comp rx=0 tx=0).stop 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 shutdown_connections 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f844c0b52e0 0x7f844c0b77a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8464108670 0x7f846419a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8464105ac0 0x7f8464199420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 --2- 172.21.15.33:0/187105459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f846406afc0 0x7f8464198dd0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.478+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 >> 172.21.15.33:0/187105459 conn(0x7f84640fd140 msgr2=0x7f8464103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.479+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 shutdown_connections 2024-09-10T21:03:58.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:58.479+0000 7f8469495640 1 -- 172.21.15.33:0/187105459 wait complete. 2024-09-10T21:03:58.492 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2738736259' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T21:03:58.492 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:58 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:03:58.492 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/143770535' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:58.492 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3793522233' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:03:58.492 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:58 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2726859601' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T21:03:58.493 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-10T21:03:58.494 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:03:58.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2738736259' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T21:03:58.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:58 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:03:58.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/143770535' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:58.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3793522233' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:03:58.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:58 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2726859601' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T21:03:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2738736259' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-09-10T21:03:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:58 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:03:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/143770535' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:03:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3793522233' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:03:58.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:58 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2726859601' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T21:03:58.752 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.255+0000 7f885bb56640 1 -- 172.21.15.33:0/387008343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 msgr2=0x7f88541020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.255+0000 7f885bb56640 1 --2- 172.21.15.33:0/387008343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f88541020b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8850009900 tx=0x7f885002a410 comp rx=0 tx=0).stop 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.255+0000 7f885bb56640 1 -- 172.21.15.33:0/387008343 shutdown_connections 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.255+0000 7f885bb56640 1 --2- 172.21.15.33:0/387008343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8854108060 0x7f885410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.255+0000 7f885bb56640 1 --2- 172.21.15.33:0/387008343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88541076e0 0x7f8854107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.255+0000 7f885bb56640 1 --2- 172.21.15.33:0/387008343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f88541020b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.255+0000 7f885bb56640 1 -- 172.21.15.33:0/387008343 >> 172.21.15.33:0/387008343 conn(0x7f88540fd700 msgr2=0x7f88540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:59.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 -- 172.21.15.33:0/387008343 shutdown_connections 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 -- 172.21.15.33:0/387008343 wait complete. 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 Processor -- start 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 -- start start 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f885419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88541076e0 0x7f885419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8854108060 0x7f88541a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8854111d20 con 0x7f8854101cd0 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8854111ba0 con 0x7f8854108060 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.256+0000 7f885bb56640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8854111ea0 con 0x7f88541076e0 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f885419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f885419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42736/0 (socket says 172.21.15.33:42736) 2024-09-10T21:03:59.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 -- 172.21.15.33:0/2562957934 learned_addr learned my addr 172.21.15.33:0/2562957934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88590f1640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88541076e0 0x7f885419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 -- 172.21.15.33:0/2562957934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88541076e0 msgr2=0x7f885419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f885a0f3640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8854108060 0x7f88541a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88541076e0 0x7f885419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 -- 172.21.15.33:0/2562957934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8854108060 msgr2=0x7f88541a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8854108060 0x7f88541a5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 -- 172.21.15.33:0/2562957934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88541a5690 con 0x7f8854101cd0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f88598f2640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f885419a0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8850005940 tx=0x7f8850004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f884affd640 1 -- 172.21.15.33:0/2562957934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885003f030 con 0x7f8854101cd0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f884affd640 1 -- 172.21.15.33:0/2562957934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8850003aa0 con 0x7f8854101cd0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88541a5860 con 0x7f8854101cd0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f884affd640 1 -- 172.21.15.33:0/2562957934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8850035070 con 0x7f8854101cd0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.257+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88541a5cc0 con 0x7f8854101cd0 2024-09-10T21:03:59.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.258+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88541020b0 con 0x7f8854101cd0 2024-09-10T21:03:59.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.260+0000 7f884affd640 1 -- 172.21.15.33:0/2562957934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f885003bce0 con 0x7f8854101cd0 2024-09-10T21:03:59.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.261+0000 7f884affd640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88400b5360 0x7f88400b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:03:59.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.261+0000 7f88590f1640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88400b5360 0x7f88400b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:03:59.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.261+0000 7f884affd640 1 -- 172.21.15.33:0/2562957934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8850106620 con 0x7f8854101cd0 2024-09-10T21:03:59.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.261+0000 7f88590f1640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88400b5360 0x7f88400b7820 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f885406c0f0 tx=0x7f8844005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:03:59.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.265+0000 7f884affd640 1 -- 172.21.15.33:0/2562957934 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f88500cd4e0 con 0x7f8854101cd0 2024-09-10T21:03:59.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:59 smithi033 ceph-mon[30860]: pgmap v346: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T21:03:59.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:03:59 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/187105459' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T21:03:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.488+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f885402e5c0 con 0x7f8854101cd0 2024-09-10T21:03:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.489+0000 7f884affd640 1 -- 172.21.15.33:0/2562957934 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11973 (secure 0 0 0) 0x7f88500ccde0 con 0x7f8854101cd0 2024-09-10T21:03:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:03:59.493 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"btime":"2024-09-10T21:03:57:162551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:03:56.306223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:03:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-09-10T21:03:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88400b5360 msgr2=0x7f88400b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88400b5360 0x7f88400b7820 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f885406c0f0 tx=0x7f8844005f90 comp rx=0 tx=0).stop 2024-09-10T21:03:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 msgr2=0x7f885419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:03:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f885419a0d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8850005940 tx=0x7f8850004270 comp rx=0 tx=0).stop 2024-09-10T21:03:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 shutdown_connections 2024-09-10T21:03:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88400b5360 0x7f88400b7820 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8854108060 0x7f88541a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88541076e0 0x7f885419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 --2- 172.21.15.33:0/2562957934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8854101cd0 0x7f885419a0d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:03:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 >> 172.21.15.33:0/2562957934 conn(0x7f88540fd700 msgr2=0x7f88540ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:03:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 shutdown_connections 2024-09-10T21:03:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:03:59.491+0000 7f885bb56640 1 -- 172.21.15.33:0/2562957934 wait complete. 2024-09-10T21:03:59.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:59 smithi104 ceph-mon[35325]: pgmap v346: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T21:03:59.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:03:59 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/187105459' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T21:03:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:59 smithi174 ceph-mon[35635]: pgmap v346: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-10T21:03:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:03:59 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/187105459' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T21:04:00.305 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-10T21:04:00.305 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-10T21:04:00.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:04:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 -- 172.21.15.33:0/2745600364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1018b0 msgr2=0x7f394c101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2745600364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1018b0 0x7f394c101cb0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3940009930 tx=0x7f394002a420 comp rx=0 tx=0).stop 2024-09-10T21:04:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 -- 172.21.15.33:0/2745600364 shutdown_connections 2024-09-10T21:04:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2745600364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c1033c0 0x7f394c109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2745600364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f394c102a00 0x7f394c102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2745600364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1018b0 0x7f394c101cb0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 -- 172.21.15.33:0/2745600364 >> 172.21.15.33:0/2745600364 conn(0x7f394c0fd190 msgr2=0x7f394c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 -- 172.21.15.33:0/2745600364 shutdown_connections 2024-09-10T21:04:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.412+0000 7f3952d8a640 1 -- 172.21.15.33:0/2745600364 wait complete. 2024-09-10T21:04:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 Processor -- start 2024-09-10T21:04:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 -- start start 2024-09-10T21:04:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f394c1018b0 0x7f394c06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c102a00 0x7f394c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1033c0 0x7f394c1a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f394c10d520 con 0x7f394c1033c0 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f394c10d3a0 con 0x7f394c102a00 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3952d8a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f394c10d6a0 con 0x7f394c1018b0 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3951327640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1033c0 0x7f394c1a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3951327640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1033c0 0x7f394c1a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52740/0 (socket says 172.21.15.33:52740) 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.413+0000 7f3951327640 1 -- 172.21.15.33:0/2464902922 learned_addr learned my addr 172.21.15.33:0/2464902922 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3950b26640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f394c1018b0 0x7f394c06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3951327640 1 -- 172.21.15.33:0/2464902922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f394c1018b0 msgr2=0x7f394c06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f394bfff640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c102a00 0x7f394c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3951327640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f394c1018b0 0x7f394c06bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3951327640 1 -- 172.21.15.33:0/2464902922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c102a00 msgr2=0x7f394c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3951327640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c102a00 0x7f394c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3951327640 1 -- 172.21.15.33:0/2464902922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c1a0a00 con 0x7f394c1033c0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f394bfff640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c102a00 0x7f394c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3951327640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1033c0 0x7f394c1a00f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f394400eaa0 tx=0x7f394400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3949ffb640 1 -- 172.21.15.33:0/2464902922 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394400cdb0 con 0x7f394c1033c0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3949ffb640 1 -- 172.21.15.33:0/2464902922 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3944005020 con 0x7f394c1033c0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c1a0c90 con 0x7f394c1033c0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3949ffb640 1 -- 172.21.15.33:0/2464902922 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39440116d0 con 0x7f394c1033c0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.414+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c1a1190 con 0x7f394c1033c0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.415+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c108ca0 con 0x7f394c1033c0 2024-09-10T21:04:00.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.416+0000 7f3949ffb640 1 -- 172.21.15.33:0/2464902922 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f394400f040 con 0x7f394c1033c0 2024-09-10T21:04:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.417+0000 7f3949ffb640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f393c0b52e0 0x7f393c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.418+0000 7f3950b26640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f393c0b52e0 0x7f393c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.418+0000 7f3949ffb640 1 -- 172.21.15.33:0/2464902922 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f39440e5af0 con 0x7f394c1033c0 2024-09-10T21:04:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.418+0000 7f3950b26640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f393c0b52e0 0x7f393c0b77a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3940031500 tx=0x7f3940000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.421+0000 7f3949ffb640 1 -- 172.21.15.33:0/2464902922 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3944016030 con 0x7f394c1033c0 2024-09-10T21:04:00.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:00 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2562957934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:00.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.604+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f394c063590 con 0x7f393c0b52e0 2024-09-10T21:04:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.608+0000 7f3949ffb640 1 -- 172.21.15.33:0/2464902922 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f394c063590 con 0x7f393c0b52e0 2024-09-10T21:04:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:04:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f393c0b52e0 msgr2=0x7f393c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f393c0b52e0 0x7f393c0b77a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3940031500 tx=0x7f3940000bc0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1033c0 msgr2=0x7f394c1a00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1033c0 0x7f394c1a00f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f394400eaa0 tx=0x7f394400ef60 comp rx=0 tx=0).stop 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 shutdown_connections 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f393c0b52e0 0x7f393c0b77a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f394c1033c0 0x7f394c1a00f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f394c102a00 0x7f394c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 --2- 172.21.15.33:0/2464902922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f394c1018b0 0x7f394c06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 >> 172.21.15.33:0/2464902922 conn(0x7f394c0fd190 msgr2=0x7f394c105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 shutdown_connections 2024-09-10T21:04:00.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.612+0000 7f3952d8a640 1 -- 172.21.15.33:0/2464902922 wait complete. 2024-09-10T21:04:00.625 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:00 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2562957934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:00.627 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T21:04:00.113197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665501,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1171,"num_read_kb":695,"num_write":392,"num_write_kb":2311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":2953077,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":824,"ondisk_log_size":824,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355516,"kb_used_data":10428,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586308,"statfs":{"total":1151940427776,"available":1151576379392,"internally_reserved":0,"allocated":10678272,"data_stored":6866523,"data_compressed":176985,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4930,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":62,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003095"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498746+0000","last_change":"2024-09-10T21:02:00.805421+0000","last_active":"2024-09-10T21:03:40.498746+0000","last_peered":"2024-09-10T21:03:40.498746+0000","last_clean":"2024-09-10T21:03:40.498746+0000","last_became_active":"2024-09-10T21:02:00.804274+0000","last_became_peered":"2024-09-10T21:02:00.804274+0000","last_unstale":"2024-09-10T21:03:40.498746+0000","last_undegraded":"2024-09-10T21:03:40.498746+0000","last_fullsized":"2024-09-10T21:03:40.498746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:12.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.497068+0000","last_change":"2024-09-10T21:02:00.806104+0000","last_active":"2024-09-10T21:03:40.497068+0000","last_peered":"2024-09-10T21:03:40.497068+0000","last_clean":"2024-09-10T21:03:40.497068+0000","last_became_active":"2024-09-10T21:02:00.805510+0000","last_became_peered":"2024-09-10T21:02:00.805510+0000","last_unstale":"2024-09-10T21:03:40.497068+0000","last_undegraded":"2024-09-10T21:03:40.497068+0000","last_fullsized":"2024-09-10T21:03:40.497068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:37.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503426+0000","last_change":"2024-09-10T21:02:00.807691+0000","last_active":"2024-09-10T21:03:40.503426+0000","last_peered":"2024-09-10T21:03:40.503426+0000","last_clean":"2024-09-10T21:03:40.503426+0000","last_became_active":"2024-09-10T21:02:00.807364+0000","last_became_peered":"2024-09-10T21:02:00.807364+0000","last_unstale":"2024-09-10T21:03:40.503426+0000","last_undegraded":"2024-09-10T21:03:40.503426+0000","last_fullsized":"2024-09-10T21:03:40.503426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:27.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499586+0000","last_change":"2024-09-10T21:02:00.809542+0000","last_active":"2024-09-10T21:03:40.499586+0000","last_peered":"2024-09-10T21:03:40.499586+0000","last_clean":"2024-09-10T21:03:40.499586+0000","last_became_active":"2024-09-10T21:02:00.809034+0000","last_became_peered":"2024-09-10T21:02:00.809034+0000","last_unstale":"2024-09-10T21:03:40.499586+0000","last_undegraded":"2024-09-10T21:03:40.499586+0000","last_fullsized":"2024-09-10T21:03:40.499586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:29.923532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491227+0000","last_change":"2024-09-10T21:02:00.809355+0000","last_active":"2024-09-10T21:03:40.491227+0000","last_peered":"2024-09-10T21:03:40.491227+0000","last_clean":"2024-09-10T21:03:40.491227+0000","last_became_active":"2024-09-10T21:02:00.808863+0000","last_became_peered":"2024-09-10T21:02:00.808863+0000","last_unstale":"2024-09-10T21:03:40.491227+0000","last_undegraded":"2024-09-10T21:03:40.491227+0000","last_fullsized":"2024-09-10T21:03:40.491227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:29.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502481+0000","last_change":"2024-09-10T21:02:00.803756+0000","last_active":"2024-09-10T21:03:40.502481+0000","last_peered":"2024-09-10T21:03:40.502481+0000","last_clean":"2024-09-10T21:03:40.502481+0000","last_became_active":"2024-09-10T21:02:00.803406+0000","last_became_peered":"2024-09-10T21:02:00.803406+0000","last_unstale":"2024-09-10T21:03:40.502481+0000","last_undegraded":"2024-09-10T21:03:40.502481+0000","last_fullsized":"2024-09-10T21:03:40.502481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:53.437629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504748+0000","last_change":"2024-09-10T21:02:00.808845+0000","last_active":"2024-09-10T21:03:40.504748+0000","last_peered":"2024-09-10T21:03:40.504748+0000","last_clean":"2024-09-10T21:03:40.504748+0000","last_became_active":"2024-09-10T21:02:00.807759+0000","last_became_peered":"2024-09-10T21:02:00.807759+0000","last_unstale":"2024-09-10T21:03:40.504748+0000","last_undegraded":"2024-09-10T21:03:40.504748+0000","last_fullsized":"2024-09-10T21:03:40.504748+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.809389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.487348+0000","last_change":"2024-09-10T21:02:00.811199+0000","last_active":"2024-09-10T21:03:40.487348+0000","last_peered":"2024-09-10T21:03:40.487348+0000","last_clean":"2024-09-10T21:03:40.487348+0000","last_became_active":"2024-09-10T21:02:00.810692+0000","last_became_peered":"2024-09-10T21:02:00.810692+0000","last_unstale":"2024-09-10T21:03:40.487348+0000","last_undegraded":"2024-09-10T21:03:40.487348+0000","last_fullsized":"2024-09-10T21:03:40.487348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:59.557973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509702+0000","last_change":"2024-09-10T21:02:00.812686+0000","last_active":"2024-09-10T21:03:40.509702+0000","last_peered":"2024-09-10T21:03:40.509702+0000","last_clean":"2024-09-10T21:03:40.509702+0000","last_became_active":"2024-09-10T21:02:00.812363+0000","last_became_peered":"2024-09-10T21:02:00.812363+0000","last_unstale":"2024-09-10T21:03:40.509702+0000","last_undegraded":"2024-09-10T21:03:40.509702+0000","last_fullsized":"2024-09-10T21:03:40.509702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:37.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503735+0000","last_change":"2024-09-10T21:02:00.812058+0000","last_active":"2024-09-10T21:03:40.503735+0000","last_peered":"2024-09-10T21:03:40.503735+0000","last_clean":"2024-09-10T21:03:40.503735+0000","last_became_active":"2024-09-10T21:02:00.811490+0000","last_became_peered":"2024-09-10T21:02:00.811490+0000","last_unstale":"2024-09-10T21:03:40.503735+0000","last_undegraded":"2024-09-10T21:03:40.503735+0000","last_fullsized":"2024-09-10T21:03:40.503735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:31.557485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.507421+0000","last_change":"2024-09-10T21:02:00.809991+0000","last_active":"2024-09-10T21:03:40.507421+0000","last_peered":"2024-09-10T21:03:40.507421+0000","last_clean":"2024-09-10T21:03:40.507421+0000","last_became_active":"2024-09-10T21:02:00.809197+0000","last_became_peered":"2024-09-10T21:02:00.809197+0000","last_unstale":"2024-09-10T21:03:40.507421+0000","last_undegraded":"2024-09-10T21:03:40.507421+0000","last_fullsized":"2024-09-10T21:03:40.507421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490787+0000","last_change":"2024-09-10T21:02:41.918484+0000","last_active":"2024-09-10T21:03:40.490787+0000","last_peered":"2024-09-10T21:03:40.490787+0000","last_clean":"2024-09-10T21:03:40.490787+0000","last_became_active":"2024-09-10T21:02:41.918068+0000","last_became_peered":"2024-09-10T21:02:41.918068+0000","last_unstale":"2024-09-10T21:03:40.490787+0000","last_undegraded":"2024-09-10T21:03:40.490787+0000","last_fullsized":"2024-09-10T21:03:40.490787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:08.954072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.487079+0000","last_change":"2024-09-10T21:02:00.807280+0000","last_active":"2024-09-10T21:03:40.487079+0000","last_peered":"2024-09-10T21:03:40.487079+0000","last_clean":"2024-09-10T21:03:40.487079+0000","last_became_active":"2024-09-10T21:02:00.804247+0000","last_became_peered":"2024-09-10T21:02:00.804247+0000","last_unstale":"2024-09-10T21:03:40.487079+0000","last_undegraded":"2024-09-10T21:03:40.487079+0000","last_fullsized":"2024-09-10T21:03:40.487079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:28.163960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498116+0000","last_change":"2024-09-10T21:02:00.805723+0000","last_active":"2024-09-10T21:03:40.498116+0000","last_peered":"2024-09-10T21:03:40.498116+0000","last_clean":"2024-09-10T21:03:40.498116+0000","last_became_active":"2024-09-10T21:02:00.805104+0000","last_became_peered":"2024-09-10T21:02:00.805104+0000","last_unstale":"2024-09-10T21:03:40.498116+0000","last_undegraded":"2024-09-10T21:03:40.498116+0000","last_fullsized":"2024-09-10T21:03:40.498116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.056884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500888+0000","last_change":"2024-09-10T21:02:00.810497+0000","last_active":"2024-09-10T21:03:40.500888+0000","last_peered":"2024-09-10T21:03:40.500888+0000","last_clean":"2024-09-10T21:03:40.500888+0000","last_became_active":"2024-09-10T21:02:00.809392+0000","last_became_peered":"2024-09-10T21:02:00.809392+0000","last_unstale":"2024-09-10T21:03:40.500888+0000","last_undegraded":"2024-09-10T21:03:40.500888+0000","last_fullsized":"2024-09-10T21:03:40.500888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:51.214771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509646+0000","last_change":"2024-09-10T21:02:00.813973+0000","last_active":"2024-09-10T21:03:40.509646+0000","last_peered":"2024-09-10T21:03:40.509646+0000","last_clean":"2024-09-10T21:03:40.509646+0000","last_became_active":"2024-09-10T21:02:00.813580+0000","last_became_peered":"2024-09-10T21:02:00.813580+0000","last_unstale":"2024-09-10T21:03:40.509646+0000","last_undegraded":"2024-09-10T21:03:40.509646+0000","last_fullsized":"2024-09-10T21:03:40.509646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:48.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498592+0000","last_change":"2024-09-10T21:02:00.804844+0000","last_active":"2024-09-10T21:03:40.498592+0000","last_peered":"2024-09-10T21:03:40.498592+0000","last_clean":"2024-09-10T21:03:40.498592+0000","last_became_active":"2024-09-10T21:02:00.803866+0000","last_became_peered":"2024-09-10T21:02:00.803866+0000","last_unstale":"2024-09-10T21:03:40.498592+0000","last_undegraded":"2024-09-10T21:03:40.498592+0000","last_fullsized":"2024-09-10T21:03:40.498592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:05.231877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.506529+0000","last_change":"2024-09-10T21:02:00.801343+0000","last_active":"2024-09-10T21:03:40.506529+0000","last_peered":"2024-09-10T21:03:40.506529+0000","last_clean":"2024-09-10T21:03:40.506529+0000","last_became_active":"2024-09-10T21:02:00.801008+0000","last_became_peered":"2024-09-10T21:02:00.801008+0000","last_unstale":"2024-09-10T21:03:40.506529+0000","last_undegraded":"2024-09-10T21:03:40.506529+0000","last_fullsized":"2024-09-10T21:03:40.506529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:31.167925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498608+0000","last_change":"2024-09-10T21:02:41.917997+0000","last_active":"2024-09-10T21:03:40.498608+0000","last_peered":"2024-09-10T21:03:40.498608+0000","last_clean":"2024-09-10T21:03:40.498608+0000","last_became_active":"2024-09-10T21:02:41.917621+0000","last_became_peered":"2024-09-10T21:02:41.917621+0000","last_unstale":"2024-09-10T21:03:40.498608+0000","last_undegraded":"2024-09-10T21:03:40.498608+0000","last_fullsized":"2024-09-10T21:03:40.498608+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:12.742475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491709+0000","last_change":"2024-09-10T21:02:00.810502+0000","last_active":"2024-09-10T21:03:40.491709+0000","last_peered":"2024-09-10T21:03:40.491709+0000","last_clean":"2024-09-10T21:03:40.491709+0000","last_became_active":"2024-09-10T21:02:00.810209+0000","last_became_peered":"2024-09-10T21:02:00.810209+0000","last_unstale":"2024-09-10T21:03:40.491709+0000","last_undegraded":"2024-09-10T21:03:40.491709+0000","last_fullsized":"2024-09-10T21:03:40.491709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:51.311272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502162+0000","last_change":"2024-09-10T21:02:00.797119+0000","last_active":"2024-09-10T21:03:40.502162+0000","last_peered":"2024-09-10T21:03:40.502162+0000","last_clean":"2024-09-10T21:03:40.502162+0000","last_became_active":"2024-09-10T21:02:00.796788+0000","last_became_peered":"2024-09-10T21:02:00.796788+0000","last_unstale":"2024-09-10T21:03:40.502162+0000","last_undegraded":"2024-09-10T21:03:40.502162+0000","last_fullsized":"2024-09-10T21:03:40.502162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:07.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510423+0000","last_change":"2024-09-10T21:02:00.802284+0000","last_active":"2024-09-10T21:03:40.510423+0000","last_peered":"2024-09-10T21:03:40.510423+0000","last_clean":"2024-09-10T21:03:40.510423+0000","last_became_active":"2024-09-10T21:02:00.801964+0000","last_became_peered":"2024-09-10T21:02:00.801964+0000","last_unstale":"2024-09-10T21:03:40.510423+0000","last_undegraded":"2024-09-10T21:03:40.510423+0000","last_fullsized":"2024-09-10T21:03:40.510423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:45.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490488+0000","last_change":"2024-09-10T21:02:00.809196+0000","last_active":"2024-09-10T21:03:40.490488+0000","last_peered":"2024-09-10T21:03:40.490488+0000","last_clean":"2024-09-10T21:03:40.490488+0000","last_became_active":"2024-09-10T21:02:00.808903+0000","last_became_peered":"2024-09-10T21:02:00.808903+0000","last_unstale":"2024-09-10T21:03:40.490488+0000","last_undegraded":"2024-09-10T21:03:40.490488+0000","last_fullsized":"2024-09-10T21:03:40.490488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:01.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.488110+0000","last_change":"2024-09-10T21:02:00.808195+0000","last_active":"2024-09-10T21:03:40.488110+0000","last_peered":"2024-09-10T21:03:40.488110+0000","last_clean":"2024-09-10T21:03:40.488110+0000","last_became_active":"2024-09-10T21:02:00.807670+0000","last_became_peered":"2024-09-10T21:02:00.807670+0000","last_unstale":"2024-09-10T21:03:40.488110+0000","last_undegraded":"2024-09-10T21:03:40.488110+0000","last_fullsized":"2024-09-10T21:03:40.488110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:12.549344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.479659+0000","last_change":"2024-09-10T21:02:00.811401+0000","last_active":"2024-09-10T21:03:40.479659+0000","last_peered":"2024-09-10T21:03:40.479659+0000","last_clean":"2024-09-10T21:03:40.479659+0000","last_became_active":"2024-09-10T21:02:00.811055+0000","last_became_peered":"2024-09-10T21:02:00.811055+0000","last_unstale":"2024-09-10T21:03:40.479659+0000","last_undegraded":"2024-09-10T21:03:40.479659+0000","last_fullsized":"2024-09-10T21:03:40.479659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:58.641873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510083+0000","last_change":"2024-09-10T21:02:00.813796+0000","last_active":"2024-09-10T21:03:40.510083+0000","last_peered":"2024-09-10T21:03:40.510083+0000","last_clean":"2024-09-10T21:03:40.510083+0000","last_became_active":"2024-09-10T21:02:00.813468+0000","last_became_peered":"2024-09-10T21:02:00.813468+0000","last_unstale":"2024-09-10T21:03:40.510083+0000","last_undegraded":"2024-09-10T21:03:40.510083+0000","last_fullsized":"2024-09-10T21:03:40.510083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:25.964890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.488390+0000","last_change":"2024-09-10T21:02:00.811436+0000","last_active":"2024-09-10T21:03:40.488390+0000","last_peered":"2024-09-10T21:03:40.488390+0000","last_clean":"2024-09-10T21:03:40.488390+0000","last_became_active":"2024-09-10T21:02:00.810956+0000","last_became_peered":"2024-09-10T21:02:00.810956+0000","last_unstale":"2024-09-10T21:03:40.488390+0000","last_undegraded":"2024-09-10T21:03:40.488390+0000","last_fullsized":"2024-09-10T21:03:40.488390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.306832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.486792+0000","last_change":"2024-09-10T21:02:00.799370+0000","last_active":"2024-09-10T21:03:40.486792+0000","last_peered":"2024-09-10T21:03:40.486792+0000","last_clean":"2024-09-10T21:03:40.486792+0000","last_became_active":"2024-09-10T21:02:00.799044+0000","last_became_peered":"2024-09-10T21:02:00.799044+0000","last_unstale":"2024-09-10T21:03:40.486792+0000","last_undegraded":"2024-09-10T21:03:40.486792+0000","last_fullsized":"2024-09-10T21:03:40.486792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:40.332088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498868+0000","last_change":"2024-09-10T21:02:00.799830+0000","last_active":"2024-09-10T21:03:40.498868+0000","last_peered":"2024-09-10T21:03:40.498868+0000","last_clean":"2024-09-10T21:03:40.498868+0000","last_became_active":"2024-09-10T21:02:00.799533+0000","last_became_peered":"2024-09-10T21:02:00.799533+0000","last_unstale":"2024-09-10T21:03:40.498868+0000","last_undegraded":"2024-09-10T21:03:40.498868+0000","last_fullsized":"2024-09-10T21:03:40.498868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:23.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499851+0000","last_change":"2024-09-10T21:02:00.812739+0000","last_active":"2024-09-10T21:03:40.499851+0000","last_peered":"2024-09-10T21:03:40.499851+0000","last_clean":"2024-09-10T21:03:40.499851+0000","last_became_active":"2024-09-10T21:02:00.812395+0000","last_became_peered":"2024-09-10T21:02:00.812395+0000","last_unstale":"2024-09-10T21:03:40.499851+0000","last_undegraded":"2024-09-10T21:03:40.499851+0000","last_fullsized":"2024-09-10T21:03:40.499851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:55.727155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503421+0000","last_change":"2024-09-10T21:02:00.806291+0000","last_active":"2024-09-10T21:03:40.503421+0000","last_peered":"2024-09-10T21:03:40.503421+0000","last_clean":"2024-09-10T21:03:40.503421+0000","last_became_active":"2024-09-10T21:02:00.805943+0000","last_became_peered":"2024-09-10T21:02:00.805943+0000","last_unstale":"2024-09-10T21:03:40.503421+0000","last_undegraded":"2024-09-10T21:03:40.503421+0000","last_fullsized":"2024-09-10T21:03:40.503421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:21.354105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500561+0000","last_change":"2024-09-10T21:02:00.808774+0000","last_active":"2024-09-10T21:03:40.500561+0000","last_peered":"2024-09-10T21:03:40.500561+0000","last_clean":"2024-09-10T21:03:40.500561+0000","last_became_active":"2024-09-10T21:02:00.808027+0000","last_became_peered":"2024-09-10T21:02:00.808027+0000","last_unstale":"2024-09-10T21:03:40.500561+0000","last_undegraded":"2024-09-10T21:03:40.500561+0000","last_fullsized":"2024-09-10T21:03:40.500561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.486316+0000","last_change":"2024-09-10T21:02:00.808349+0000","last_active":"2024-09-10T21:03:40.486316+0000","last_peered":"2024-09-10T21:03:40.486316+0000","last_clean":"2024-09-10T21:03:40.486316+0000","last_became_active":"2024-09-10T21:02:00.807839+0000","last_became_peered":"2024-09-10T21:02:00.807839+0000","last_unstale":"2024-09-10T21:03:40.486316+0000","last_undegraded":"2024-09-10T21:03:40.486316+0000","last_fullsized":"2024-09-10T21:03:40.486316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:53.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":151,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:55.977242+0000","last_change":"2024-09-10T21:02:41.877376+0000","last_active":"2024-09-10T21:03:55.977242+0000","last_peered":"2024-09-10T21:03:55.977242+0000","last_clean":"2024-09-10T21:03:55.977242+0000","last_became_active":"2024-09-10T21:02:41.871397+0000","last_became_peered":"2024-09-10T21:02:41.871397+0000","last_unstale":"2024-09-10T21:03:55.977242+0000","last_undegraded":"2024-09-10T21:03:55.977242+0000","last_fullsized":"2024-09-10T21:03:55.977242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:59.961597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.489060+0000","last_change":"2024-09-10T21:02:41.979196+0000","last_active":"2024-09-10T21:03:40.489060+0000","last_peered":"2024-09-10T21:03:40.489060+0000","last_clean":"2024-09-10T21:03:40.489060+0000","last_became_active":"2024-09-10T21:02:41.978781+0000","last_became_peered":"2024-09-10T21:02:41.978781+0000","last_unstale":"2024-09-10T21:03:40.489060+0000","last_undegraded":"2024-09-10T21:03:40.489060+0000","last_fullsized":"2024-09-10T21:03:40.489060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:15.100055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917027+0000","last_change":"2024-09-10T21:01:58.787619+0000","last_active":"2024-09-10T21:02:41.917027+0000","last_peered":"2024-09-10T21:02:41.917027+0000","last_clean":"2024-09-10T21:02:41.917027+0000","last_became_active":"2024-09-10T21:01:58.787138+0000","last_became_peered":"2024-09-10T21:01:58.787138+0000","last_unstale":"2024-09-10T21:02:41.917027+0000","last_undegraded":"2024-09-10T21:02:41.917027+0000","last_fullsized":"2024-09-10T21:02:41.917027+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:33.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916753+0000","last_change":"2024-09-10T21:01:58.780115+0000","last_active":"2024-09-10T21:02:41.916753+0000","last_peered":"2024-09-10T21:02:41.916753+0000","last_clean":"2024-09-10T21:02:41.916753+0000","last_became_active":"2024-09-10T21:01:58.779757+0000","last_became_peered":"2024-09-10T21:01:58.779757+0000","last_unstale":"2024-09-10T21:02:41.916753+0000","last_undegraded":"2024-09-10T21:02:41.916753+0000","last_fullsized":"2024-09-10T21:02:41.916753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:29.337978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491520+0000","last_change":"2024-09-10T21:02:00.800925+0000","last_active":"2024-09-10T21:03:40.491520+0000","last_peered":"2024-09-10T21:03:40.491520+0000","last_clean":"2024-09-10T21:03:40.491520+0000","last_became_active":"2024-09-10T21:02:00.798567+0000","last_became_peered":"2024-09-10T21:02:00.798567+0000","last_unstale":"2024-09-10T21:03:40.491520+0000","last_undegraded":"2024-09-10T21:03:40.491520+0000","last_fullsized":"2024-09-10T21:03:40.491520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:01.873716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867127+0000","last_change":"2024-09-10T21:01:58.786040+0000","last_active":"2024-09-10T21:02:41.867127+0000","last_peered":"2024-09-10T21:02:41.867127+0000","last_clean":"2024-09-10T21:02:41.867127+0000","last_became_active":"2024-09-10T21:01:58.785703+0000","last_became_peered":"2024-09-10T21:01:58.785703+0000","last_unstale":"2024-09-10T21:02:41.867127+0000","last_undegraded":"2024-09-10T21:02:41.867127+0000","last_fullsized":"2024-09-10T21:02:41.867127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:39.741614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.497664+0000","last_change":"2024-09-10T21:02:00.798368+0000","last_active":"2024-09-10T21:03:40.497664+0000","last_peered":"2024-09-10T21:03:40.497664+0000","last_clean":"2024-09-10T21:03:40.497664+0000","last_became_active":"2024-09-10T21:02:00.798080+0000","last_became_peered":"2024-09-10T21:02:00.798080+0000","last_unstale":"2024-09-10T21:03:40.497664+0000","last_undegraded":"2024-09-10T21:03:40.497664+0000","last_fullsized":"2024-09-10T21:03:40.497664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:31.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":238,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.978268+0000","last_change":"2024-09-10T21:01:58.788119+0000","last_active":"2024-09-10T21:03:56.978268+0000","last_peered":"2024-09-10T21:03:56.978268+0000","last_clean":"2024-09-10T21:03:56.978268+0000","last_became_active":"2024-09-10T21:01:58.787724+0000","last_became_peered":"2024-09-10T21:01:58.787724+0000","last_unstale":"2024-09-10T21:03:56.978268+0000","last_undegraded":"2024-09-10T21:03:56.978268+0000","last_fullsized":"2024-09-10T21:03:56.978268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:42.872858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":209,"num_read_kb":209,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510209+0000","last_change":"2024-09-10T21:02:00.804425+0000","last_active":"2024-09-10T21:03:40.510209+0000","last_peered":"2024-09-10T21:03:40.510209+0000","last_clean":"2024-09-10T21:03:40.510209+0000","last_became_active":"2024-09-10T21:02:00.804093+0000","last_became_peered":"2024-09-10T21:02:00.804093+0000","last_unstale":"2024-09-10T21:03:40.510209+0000","last_undegraded":"2024-09-10T21:03:40.510209+0000","last_fullsized":"2024-09-10T21:03:40.510209+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:46.183326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870381+0000","last_change":"2024-09-10T21:01:58.791323+0000","last_active":"2024-09-10T21:02:41.870381+0000","last_peered":"2024-09-10T21:02:41.870381+0000","last_clean":"2024-09-10T21:02:41.870381+0000","last_became_active":"2024-09-10T21:01:58.790551+0000","last_became_peered":"2024-09-10T21:01:58.790551+0000","last_unstale":"2024-09-10T21:02:41.870381+0000","last_undegraded":"2024-09-10T21:02:41.870381+0000","last_fullsized":"2024-09-10T21:02:41.870381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:18.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499172+0000","last_change":"2024-09-10T21:02:00.804025+0000","last_active":"2024-09-10T21:03:40.499172+0000","last_peered":"2024-09-10T21:03:40.499172+0000","last_clean":"2024-09-10T21:03:40.499172+0000","last_became_active":"2024-09-10T21:02:00.803663+0000","last_became_peered":"2024-09-10T21:02:00.803663+0000","last_unstale":"2024-09-10T21:03:40.499172+0000","last_undegraded":"2024-09-10T21:03:40.499172+0000","last_fullsized":"2024-09-10T21:03:40.499172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:30.609020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868035+0000","last_change":"2024-09-10T21:01:58.887067+0000","last_active":"2024-09-10T21:02:41.868035+0000","last_peered":"2024-09-10T21:02:41.868035+0000","last_clean":"2024-09-10T21:02:41.868035+0000","last_became_active":"2024-09-10T21:01:58.886611+0000","last_became_peered":"2024-09-10T21:01:58.886611+0000","last_unstale":"2024-09-10T21:02:41.868035+0000","last_undegraded":"2024-09-10T21:02:41.868035+0000","last_fullsized":"2024-09-10T21:02:41.868035+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:06.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.508877+0000","last_change":"2024-09-10T21:02:41.917998+0000","last_active":"2024-09-10T21:03:40.508877+0000","last_peered":"2024-09-10T21:03:40.508877+0000","last_clean":"2024-09-10T21:03:40.508877+0000","last_became_active":"2024-09-10T21:02:41.917665+0000","last_became_peered":"2024-09-10T21:02:41.917665+0000","last_unstale":"2024-09-10T21:03:40.508877+0000","last_undegraded":"2024-09-10T21:03:40.508877+0000","last_fullsized":"2024-09-10T21:03:40.508877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:15.147379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'37","reported_seq":280,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.978810+0000","last_change":"2024-09-10T21:01:58.790966+0000","last_active":"2024-09-10T21:03:56.978810+0000","last_peered":"2024-09-10T21:03:56.978810+0000","last_clean":"2024-09-10T21:03:56.978810+0000","last_became_active":"2024-09-10T21:01:58.790636+0000","last_became_peered":"2024-09-10T21:01:58.790636+0000","last_unstale":"2024-09-10T21:03:56.978810+0000","last_undegraded":"2024-09-10T21:03:56.978810+0000","last_fullsized":"2024-09-10T21:03:56.978810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:06.209074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66041,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.501096+0000","last_change":"2024-09-10T21:02:00.810610+0000","last_active":"2024-09-10T21:03:40.501096+0000","last_peered":"2024-09-10T21:03:40.501096+0000","last_clean":"2024-09-10T21:03:40.501096+0000","last_became_active":"2024-09-10T21:02:00.810224+0000","last_became_peered":"2024-09-10T21:02:00.810224+0000","last_unstale":"2024-09-10T21:03:40.501096+0000","last_undegraded":"2024-09-10T21:03:40.501096+0000","last_fullsized":"2024-09-10T21:03:40.501096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:32.785312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870285+0000","last_change":"2024-09-10T21:01:58.786476+0000","last_active":"2024-09-10T21:02:41.870285+0000","last_peered":"2024-09-10T21:02:41.870285+0000","last_clean":"2024-09-10T21:02:41.870285+0000","last_became_active":"2024-09-10T21:01:58.786153+0000","last_became_peered":"2024-09-10T21:01:58.786153+0000","last_unstale":"2024-09-10T21:02:41.870285+0000","last_undegraded":"2024-09-10T21:02:41.870285+0000","last_fullsized":"2024-09-10T21:02:41.870285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:00.118591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.501464+0000","last_change":"2024-09-10T21:02:00.796138+0000","last_active":"2024-09-10T21:03:40.501464+0000","last_peered":"2024-09-10T21:03:40.501464+0000","last_clean":"2024-09-10T21:03:40.501464+0000","last_became_active":"2024-09-10T21:02:00.795847+0000","last_became_peered":"2024-09-10T21:02:00.795847+0000","last_unstale":"2024-09-10T21:03:40.501464+0000","last_undegraded":"2024-09-10T21:03:40.501464+0000","last_fullsized":"2024-09-10T21:03:40.501464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:29.462134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867576+0000","last_change":"2024-09-10T21:01:58.789314+0000","last_active":"2024-09-10T21:02:41.867576+0000","last_peered":"2024-09-10T21:02:41.867576+0000","last_clean":"2024-09-10T21:02:41.867576+0000","last_became_active":"2024-09-10T21:01:58.788594+0000","last_became_peered":"2024-09-10T21:01:58.788594+0000","last_unstale":"2024-09-10T21:02:41.867576+0000","last_undegraded":"2024-09-10T21:02:41.867576+0000","last_fullsized":"2024-09-10T21:02:41.867576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:44.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.506541+0000","last_change":"2024-09-10T21:02:00.799828+0000","last_active":"2024-09-10T21:03:40.506541+0000","last_peered":"2024-09-10T21:03:40.506541+0000","last_clean":"2024-09-10T21:03:40.506541+0000","last_became_active":"2024-09-10T21:02:00.799465+0000","last_became_peered":"2024-09-10T21:02:00.799465+0000","last_unstale":"2024-09-10T21:03:40.506541+0000","last_undegraded":"2024-09-10T21:03:40.506541+0000","last_fullsized":"2024-09-10T21:03:40.506541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:56.794326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'8","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:57.029723+0000","last_change":"2024-09-10T21:01:58.788730+0000","last_active":"2024-09-10T21:03:57.029723+0000","last_peered":"2024-09-10T21:03:57.029723+0000","last_clean":"2024-09-10T21:03:57.029723+0000","last_became_active":"2024-09-10T21:01:58.788317+0000","last_became_peered":"2024-09-10T21:01:58.788317+0000","last_unstale":"2024-09-10T21:03:57.029723+0000","last_undegraded":"2024-09-10T21:03:57.029723+0000","last_fullsized":"2024-09-10T21:03:57.029723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:52.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504421+0000","last_change":"2024-09-10T21:02:00.807104+0000","last_active":"2024-09-10T21:03:40.504421+0000","last_peered":"2024-09-10T21:03:40.504421+0000","last_clean":"2024-09-10T21:03:40.504421+0000","last_became_active":"2024-09-10T21:02:00.806815+0000","last_became_peered":"2024-09-10T21:02:00.806815+0000","last_unstale":"2024-09-10T21:03:40.504421+0000","last_undegraded":"2024-09-10T21:03:40.504421+0000","last_fullsized":"2024-09-10T21:03:40.504421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:24.747662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.890722+0000","last_change":"2024-09-10T21:02:41.890657+0000","last_active":"2024-09-10T21:02:41.890722+0000","last_peered":"2024-09-10T21:02:41.890722+0000","last_clean":"2024-09-10T21:02:41.890722+0000","last_became_active":"2024-09-10T21:02:41.873778+0000","last_became_peered":"2024-09-10T21:02:41.873778+0000","last_unstale":"2024-09-10T21:02:41.890722+0000","last_undegraded":"2024-09-10T21:02:41.890722+0000","last_fullsized":"2024-09-10T21:02:41.890722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:58.678679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490766+0000","last_change":"2024-09-10T21:02:00.806796+0000","last_active":"2024-09-10T21:03:40.490766+0000","last_peered":"2024-09-10T21:03:40.490766+0000","last_clean":"2024-09-10T21:03:40.490766+0000","last_became_active":"2024-09-10T21:02:00.805802+0000","last_became_peered":"2024-09-10T21:02:00.805802+0000","last_unstale":"2024-09-10T21:03:40.490766+0000","last_undegraded":"2024-09-10T21:03:40.490766+0000","last_fullsized":"2024-09-10T21:03:40.490766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.962870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871699+0000","last_change":"2024-09-10T21:01:58.781088+0000","last_active":"2024-09-10T21:02:41.871699+0000","last_peered":"2024-09-10T21:02:41.871699+0000","last_clean":"2024-09-10T21:02:41.871699+0000","last_became_active":"2024-09-10T21:01:58.780731+0000","last_became_peered":"2024-09-10T21:01:58.780731+0000","last_unstale":"2024-09-10T21:02:41.871699+0000","last_undegraded":"2024-09-10T21:02:41.871699+0000","last_fullsized":"2024-09-10T21:02:41.871699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:50.636077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491805+0000","last_change":"2024-09-10T21:02:00.806551+0000","last_active":"2024-09-10T21:03:40.491805+0000","last_peered":"2024-09-10T21:03:40.491805+0000","last_clean":"2024-09-10T21:03:40.491805+0000","last_became_active":"2024-09-10T21:02:00.806134+0000","last_became_peered":"2024-09-10T21:02:00.806134+0000","last_unstale":"2024-09-10T21:03:40.491805+0000","last_undegraded":"2024-09-10T21:03:40.491805+0000","last_fullsized":"2024-09-10T21:03:40.491805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.574372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916954+0000","last_change":"2024-09-10T21:01:58.780294+0000","last_active":"2024-09-10T21:02:41.916954+0000","last_peered":"2024-09-10T21:02:41.916954+0000","last_clean":"2024-09-10T21:02:41.916954+0000","last_became_active":"2024-09-10T21:01:58.779767+0000","last_became_peered":"2024-09-10T21:01:58.779767+0000","last_unstale":"2024-09-10T21:02:41.916954+0000","last_undegraded":"2024-09-10T21:02:41.916954+0000","last_fullsized":"2024-09-10T21:02:41.916954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:21.734159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503245+0000","last_change":"2024-09-10T21:02:00.809393+0000","last_active":"2024-09-10T21:03:40.503245+0000","last_peered":"2024-09-10T21:03:40.503245+0000","last_clean":"2024-09-10T21:03:40.503245+0000","last_became_active":"2024-09-10T21:02:00.809012+0000","last_became_peered":"2024-09-10T21:02:00.809012+0000","last_unstale":"2024-09-10T21:03:40.503245+0000","last_undegraded":"2024-09-10T21:03:40.503245+0000","last_fullsized":"2024-09-10T21:03:40.503245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:03.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868189+0000","last_change":"2024-09-10T21:01:58.786507+0000","last_active":"2024-09-10T21:02:41.868189+0000","last_peered":"2024-09-10T21:02:41.868189+0000","last_clean":"2024-09-10T21:02:41.868189+0000","last_became_active":"2024-09-10T21:01:58.786115+0000","last_became_peered":"2024-09-10T21:01:58.786115+0000","last_unstale":"2024-09-10T21:02:41.868189+0000","last_undegraded":"2024-09-10T21:02:41.868189+0000","last_fullsized":"2024-09-10T21:02:41.868189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.985353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504534+0000","last_change":"2024-09-10T21:02:00.805050+0000","last_active":"2024-09-10T21:03:40.504534+0000","last_peered":"2024-09-10T21:03:40.504534+0000","last_clean":"2024-09-10T21:03:40.504534+0000","last_became_active":"2024-09-10T21:02:00.804706+0000","last_became_peered":"2024-09-10T21:02:00.804706+0000","last_unstale":"2024-09-10T21:03:40.504534+0000","last_undegraded":"2024-09-10T21:03:40.504534+0000","last_fullsized":"2024-09-10T21:03:40.504534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:15.463986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":141,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:58.030099+0000","last_change":"2024-09-10T21:01:59.114525+0000","last_active":"2024-09-10T21:03:58.030099+0000","last_peered":"2024-09-10T21:03:58.030099+0000","last_clean":"2024-09-10T21:03:58.030099+0000","last_became_active":"2024-09-10T21:01:59.114076+0000","last_became_peered":"2024-09-10T21:01:59.114076+0000","last_unstale":"2024-09-10T21:03:58.030099+0000","last_undegraded":"2024-09-10T21:03:58.030099+0000","last_fullsized":"2024-09-10T21:03:58.030099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:53.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509798+0000","last_change":"2024-09-10T21:02:00.799803+0000","last_active":"2024-09-10T21:03:40.509798+0000","last_peered":"2024-09-10T21:03:40.509798+0000","last_clean":"2024-09-10T21:03:40.509798+0000","last_became_active":"2024-09-10T21:02:00.799459+0000","last_became_peered":"2024-09-10T21:02:00.799459+0000","last_unstale":"2024-09-10T21:03:40.509798+0000","last_undegraded":"2024-09-10T21:03:40.509798+0000","last_fullsized":"2024-09-10T21:03:40.509798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:57.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870115+0000","last_change":"2024-09-10T21:00:48.381903+0000","last_active":"2024-09-10T21:02:41.870115+0000","last_peered":"2024-09-10T21:02:41.870115+0000","last_clean":"2024-09-10T21:02:41.870115+0000","last_became_active":"2024-09-10T21:00:48.376874+0000","last_became_peered":"2024-09-10T21:00:48.376874+0000","last_unstale":"2024-09-10T21:02:41.870115+0000","last_undegraded":"2024-09-10T21:02:41.870115+0000","last_fullsized":"2024-09-10T21:02:41.870115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_clean_scrub_stamp":"2024-09-10T20:57:24.516525+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:35.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866334+0000","last_change":"2024-09-10T21:01:58.788005+0000","last_active":"2024-09-10T21:02:41.866334+0000","last_peered":"2024-09-10T21:02:41.866334+0000","last_clean":"2024-09-10T21:02:41.866334+0000","last_became_active":"2024-09-10T21:01:58.786471+0000","last_became_peered":"2024-09-10T21:01:58.786471+0000","last_unstale":"2024-09-10T21:02:41.866334+0000","last_undegraded":"2024-09-10T21:02:41.866334+0000","last_fullsized":"2024-09-10T21:02:41.866334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:15.622814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.505266+0000","last_change":"2024-09-10T21:02:00.802573+0000","last_active":"2024-09-10T21:03:40.505266+0000","last_peered":"2024-09-10T21:03:40.505266+0000","last_clean":"2024-09-10T21:03:40.505266+0000","last_became_active":"2024-09-10T21:02:00.802263+0000","last_became_peered":"2024-09-10T21:02:00.802263+0000","last_unstale":"2024-09-10T21:03:40.505266+0000","last_undegraded":"2024-09-10T21:03:40.505266+0000","last_fullsized":"2024-09-10T21:03:40.505266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:54.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870004+0000","last_change":"2024-09-10T21:01:58.785700+0000","last_active":"2024-09-10T21:02:41.870004+0000","last_peered":"2024-09-10T21:02:41.870004+0000","last_clean":"2024-09-10T21:02:41.870004+0000","last_became_active":"2024-09-10T21:01:58.785346+0000","last_became_peered":"2024-09-10T21:01:58.785346+0000","last_unstale":"2024-09-10T21:02:41.870004+0000","last_undegraded":"2024-09-10T21:02:41.870004+0000","last_fullsized":"2024-09-10T21:02:41.870004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:39.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.495838+0000","last_change":"2024-09-10T21:02:00.808045+0000","last_active":"2024-09-10T21:03:40.495838+0000","last_peered":"2024-09-10T21:03:40.495838+0000","last_clean":"2024-09-10T21:03:40.495838+0000","last_became_active":"2024-09-10T21:02:00.807757+0000","last_became_peered":"2024-09-10T21:02:00.807757+0000","last_unstale":"2024-09-10T21:03:40.495838+0000","last_undegraded":"2024-09-10T21:03:40.495838+0000","last_fullsized":"2024-09-10T21:03:40.495838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:16.198290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'7","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:58.030473+0000","last_change":"2024-09-10T21:01:58.884939+0000","last_active":"2024-09-10T21:03:58.030473+0000","last_peered":"2024-09-10T21:03:58.030473+0000","last_clean":"2024-09-10T21:03:58.030473+0000","last_became_active":"2024-09-10T21:01:58.884517+0000","last_became_peered":"2024-09-10T21:01:58.884517+0000","last_unstale":"2024-09-10T21:03:58.030473+0000","last_undegraded":"2024-09-10T21:03:58.030473+0000","last_fullsized":"2024-09-10T21:03:58.030473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.579662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490615+0000","last_change":"2024-09-10T21:02:00.813021+0000","last_active":"2024-09-10T21:03:40.490615+0000","last_peered":"2024-09-10T21:03:40.490615+0000","last_clean":"2024-09-10T21:03:40.490615+0000","last_became_active":"2024-09-10T21:02:00.812622+0000","last_became_peered":"2024-09-10T21:02:00.812622+0000","last_unstale":"2024-09-10T21:03:40.490615+0000","last_undegraded":"2024-09-10T21:03:40.490615+0000","last_fullsized":"2024-09-10T21:03:40.490615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:48.348100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:54.009554+0000","last_change":"2024-09-10T21:01:58.792187+0000","last_active":"2024-09-10T21:03:54.009554+0000","last_peered":"2024-09-10T21:03:54.009554+0000","last_clean":"2024-09-10T21:03:54.009554+0000","last_became_active":"2024-09-10T21:01:58.791806+0000","last_became_peered":"2024-09-10T21:01:58.791806+0000","last_unstale":"2024-09-10T21:03:54.009554+0000","last_undegraded":"2024-09-10T21:03:54.009554+0000","last_fullsized":"2024-09-10T21:03:54.009554+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:48.493396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502741+0000","last_change":"2024-09-10T21:02:00.804073+0000","last_active":"2024-09-10T21:03:40.502741+0000","last_peered":"2024-09-10T21:03:40.502741+0000","last_clean":"2024-09-10T21:03:40.502741+0000","last_became_active":"2024-09-10T21:02:00.803760+0000","last_became_peered":"2024-09-10T21:02:00.803760+0000","last_unstale":"2024-09-10T21:03:40.502741+0000","last_undegraded":"2024-09-10T21:03:40.502741+0000","last_fullsized":"2024-09-10T21:03:40.502741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:01.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865323+0000","last_change":"2024-09-10T21:01:58.885088+0000","last_active":"2024-09-10T21:02:41.865323+0000","last_peered":"2024-09-10T21:02:41.865323+0000","last_clean":"2024-09-10T21:02:41.865323+0000","last_became_active":"2024-09-10T21:01:58.884729+0000","last_became_peered":"2024-09-10T21:01:58.884729+0000","last_unstale":"2024-09-10T21:02:41.865323+0000","last_undegraded":"2024-09-10T21:02:41.865323+0000","last_fullsized":"2024-09-10T21:02:41.865323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:37.451629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490451+0000","last_change":"2024-09-10T21:02:00.810329+0000","last_active":"2024-09-10T21:03:40.490451+0000","last_peered":"2024-09-10T21:03:40.490451+0000","last_clean":"2024-09-10T21:03:40.490451+0000","last_became_active":"2024-09-10T21:02:00.809985+0000","last_became_peered":"2024-09-10T21:02:00.809985+0000","last_unstale":"2024-09-10T21:03:40.490451+0000","last_undegraded":"2024-09-10T21:03:40.490451+0000","last_fullsized":"2024-09-10T21:03:40.490451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:39.631969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.880605+0000","last_change":"2024-09-10T21:02:41.880117+0000","last_active":"2024-09-10T21:02:41.880605+0000","last_peered":"2024-09-10T21:02:41.880605+0000","last_clean":"2024-09-10T21:02:41.880605+0000","last_became_active":"2024-09-10T21:02:41.872494+0000","last_became_peered":"2024-09-10T21:02:41.872494+0000","last_unstale":"2024-09-10T21:02:41.880605+0000","last_undegraded":"2024-09-10T21:02:41.880605+0000","last_fullsized":"2024-09-10T21:02:41.880605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:06.965479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491585+0000","last_change":"2024-09-10T21:02:00.804961+0000","last_active":"2024-09-10T21:03:40.491585+0000","last_peered":"2024-09-10T21:03:40.491585+0000","last_clean":"2024-09-10T21:03:40.491585+0000","last_became_active":"2024-09-10T21:02:00.804147+0000","last_became_peered":"2024-09-10T21:02:00.804147+0000","last_unstale":"2024-09-10T21:03:40.491585+0000","last_undegraded":"2024-09-10T21:03:40.491585+0000","last_fullsized":"2024-09-10T21:03:40.491585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:15.801815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.806420+0000","last_change":"2024-09-10T21:01:58.781292+0000","last_active":"2024-09-10T21:03:40.806420+0000","last_peered":"2024-09-10T21:03:40.806420+0000","last_clean":"2024-09-10T21:03:40.806420+0000","last_became_active":"2024-09-10T21:01:58.780995+0000","last_became_peered":"2024-09-10T21:01:58.780995+0000","last_unstale":"2024-09-10T21:03:40.806420+0000","last_undegraded":"2024-09-10T21:03:40.806420+0000","last_fullsized":"2024-09-10T21:03:40.806420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:56.914723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.493246+0000","last_change":"2024-09-10T21:02:00.805878+0000","last_active":"2024-09-10T21:03:40.493246+0000","last_peered":"2024-09-10T21:03:40.493246+0000","last_clean":"2024-09-10T21:03:40.493246+0000","last_became_active":"2024-09-10T21:02:00.804565+0000","last_became_peered":"2024-09-10T21:02:00.804565+0000","last_unstale":"2024-09-10T21:03:40.493246+0000","last_undegraded":"2024-09-10T21:03:40.493246+0000","last_fullsized":"2024-09-10T21:03:40.493246+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:24.359643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866263+0000","last_change":"2024-09-10T21:01:59.114862+0000","last_active":"2024-09-10T21:02:41.866263+0000","last_peered":"2024-09-10T21:02:41.866263+0000","last_clean":"2024-09-10T21:02:41.866263+0000","last_became_active":"2024-09-10T21:01:59.114411+0000","last_became_peered":"2024-09-10T21:01:59.114411+0000","last_unstale":"2024-09-10T21:02:41.866263+0000","last_undegraded":"2024-09-10T21:02:41.866263+0000","last_fullsized":"2024-09-10T21:02:41.866263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.645829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498781+0000","last_change":"2024-09-10T21:02:00.799230+0000","last_active":"2024-09-10T21:03:40.498781+0000","last_peered":"2024-09-10T21:03:40.498781+0000","last_clean":"2024-09-10T21:03:40.498781+0000","last_became_active":"2024-09-10T21:02:00.798992+0000","last_became_peered":"2024-09-10T21:02:00.798992+0000","last_unstale":"2024-09-10T21:03:40.498781+0000","last_undegraded":"2024-09-10T21:03:40.498781+0000","last_fullsized":"2024-09-10T21:03:40.498781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:40.322066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867967+0000","last_change":"2024-09-10T21:01:58.887042+0000","last_active":"2024-09-10T21:02:41.867967+0000","last_peered":"2024-09-10T21:02:41.867967+0000","last_clean":"2024-09-10T21:02:41.867967+0000","last_became_active":"2024-09-10T21:01:58.886710+0000","last_became_peered":"2024-09-10T21:01:58.886710+0000","last_unstale":"2024-09-10T21:02:41.867967+0000","last_undegraded":"2024-09-10T21:02:41.867967+0000","last_fullsized":"2024-09-10T21:02:41.867967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:42.672899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.480934+0000","last_change":"2024-09-10T21:02:00.809763+0000","last_active":"2024-09-10T21:03:40.480934+0000","last_peered":"2024-09-10T21:03:40.480934+0000","last_clean":"2024-09-10T21:03:40.480934+0000","last_became_active":"2024-09-10T21:02:00.808955+0000","last_became_peered":"2024-09-10T21:02:00.808955+0000","last_unstale":"2024-09-10T21:03:40.480934+0000","last_undegraded":"2024-09-10T21:03:40.480934+0000","last_fullsized":"2024-09-10T21:03:40.480934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:22.622852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977719+0000","last_change":"2024-09-10T21:01:58.786312+0000","last_active":"2024-09-10T21:02:41.977719+0000","last_peered":"2024-09-10T21:02:41.977719+0000","last_clean":"2024-09-10T21:02:41.977719+0000","last_became_active":"2024-09-10T21:01:58.785894+0000","last_became_peered":"2024-09-10T21:01:58.785894+0000","last_unstale":"2024-09-10T21:02:41.977719+0000","last_undegraded":"2024-09-10T21:02:41.977719+0000","last_fullsized":"2024-09-10T21:02:41.977719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:11.682904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500168+0000","last_change":"2024-09-10T21:02:00.808374+0000","last_active":"2024-09-10T21:03:40.500168+0000","last_peered":"2024-09-10T21:03:40.500168+0000","last_clean":"2024-09-10T21:03:40.500168+0000","last_became_active":"2024-09-10T21:02:00.807565+0000","last_became_peered":"2024-09-10T21:02:00.807565+0000","last_unstale":"2024-09-10T21:03:40.500168+0000","last_undegraded":"2024-09-10T21:03:40.500168+0000","last_fullsized":"2024-09-10T21:03:40.500168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:24.227480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870458+0000","last_change":"2024-09-10T21:01:58.782375+0000","last_active":"2024-09-10T21:02:41.870458+0000","last_peered":"2024-09-10T21:02:41.870458+0000","last_clean":"2024-09-10T21:02:41.870458+0000","last_became_active":"2024-09-10T21:01:58.781964+0000","last_became_peered":"2024-09-10T21:01:58.781964+0000","last_unstale":"2024-09-10T21:02:41.870458+0000","last_undegraded":"2024-09-10T21:02:41.870458+0000","last_fullsized":"2024-09-10T21:02:41.870458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.460126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.485903+0000","last_change":"2024-09-10T21:02:00.806612+0000","last_active":"2024-09-10T21:03:40.485903+0000","last_peered":"2024-09-10T21:03:40.485903+0000","last_clean":"2024-09-10T21:03:40.485903+0000","last_became_active":"2024-09-10T21:02:00.803470+0000","last_became_peered":"2024-09-10T21:02:00.803470+0000","last_unstale":"2024-09-10T21:03:40.485903+0000","last_undegraded":"2024-09-10T21:03:40.485903+0000","last_fullsized":"2024-09-10T21:03:40.485903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:08.350931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871476+0000","last_change":"2024-09-10T21:01:58.788987+0000","last_active":"2024-09-10T21:02:41.871476+0000","last_peered":"2024-09-10T21:02:41.871476+0000","last_clean":"2024-09-10T21:02:41.871476+0000","last_became_active":"2024-09-10T21:01:58.788605+0000","last_became_peered":"2024-09-10T21:01:58.788605+0000","last_unstale":"2024-09-10T21:02:41.871476+0000","last_undegraded":"2024-09-10T21:02:41.871476+0000","last_fullsized":"2024-09-10T21:02:41.871476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:45.798350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509907+0000","last_change":"2024-09-10T21:02:00.804200+0000","last_active":"2024-09-10T21:03:40.509907+0000","last_peered":"2024-09-10T21:03:40.509907+0000","last_clean":"2024-09-10T21:03:40.509907+0000","last_became_active":"2024-09-10T21:02:00.803875+0000","last_became_peered":"2024-09-10T21:02:00.803875+0000","last_unstale":"2024-09-10T21:03:40.509907+0000","last_undegraded":"2024-09-10T21:03:40.509907+0000","last_fullsized":"2024-09-10T21:03:40.509907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:15.930163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.507079+0000","last_change":"2024-09-10T21:02:41.874580+0000","last_active":"2024-09-10T21:03:40.507079+0000","last_peered":"2024-09-10T21:03:40.507079+0000","last_clean":"2024-09-10T21:03:40.507079+0000","last_became_active":"2024-09-10T21:02:41.873545+0000","last_became_peered":"2024-09-10T21:02:41.873545+0000","last_unstale":"2024-09-10T21:03:40.507079+0000","last_undegraded":"2024-09-10T21:03:40.507079+0000","last_fullsized":"2024-09-10T21:03:40.507079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:05.278855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870983+0000","last_change":"2024-09-10T21:01:58.784834+0000","last_active":"2024-09-10T21:02:41.870983+0000","last_peered":"2024-09-10T21:02:41.870983+0000","last_clean":"2024-09-10T21:02:41.870983+0000","last_became_active":"2024-09-10T21:01:58.784464+0000","last_became_peered":"2024-09-10T21:01:58.784464+0000","last_unstale":"2024-09-10T21:02:41.870983+0000","last_undegraded":"2024-09-10T21:02:41.870983+0000","last_fullsized":"2024-09-10T21:02:41.870983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:09.864824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510409+0000","last_change":"2024-09-10T21:02:00.805677+0000","last_active":"2024-09-10T21:03:40.510409+0000","last_peered":"2024-09-10T21:03:40.510409+0000","last_clean":"2024-09-10T21:03:40.510409+0000","last_became_active":"2024-09-10T21:02:00.805341+0000","last_became_peered":"2024-09-10T21:02:00.805341+0000","last_unstale":"2024-09-10T21:03:40.510409+0000","last_undegraded":"2024-09-10T21:03:40.510409+0000","last_fullsized":"2024-09-10T21:03:40.510409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.236954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.889813+0000","last_change":"2024-09-10T21:02:41.889336+0000","last_active":"2024-09-10T21:02:41.889813+0000","last_peered":"2024-09-10T21:02:41.889813+0000","last_clean":"2024-09-10T21:02:41.889813+0000","last_became_active":"2024-09-10T21:02:41.874164+0000","last_became_peered":"2024-09-10T21:02:41.874164+0000","last_unstale":"2024-09-10T21:02:41.889813+0000","last_undegraded":"2024-09-10T21:02:41.889813+0000","last_fullsized":"2024-09-10T21:02:41.889813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.799546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504133+0000","last_change":"2024-09-10T21:02:00.801514+0000","last_active":"2024-09-10T21:03:40.504133+0000","last_peered":"2024-09-10T21:03:40.504133+0000","last_clean":"2024-09-10T21:03:40.504133+0000","last_became_active":"2024-09-10T21:02:00.801221+0000","last_became_peered":"2024-09-10T21:02:00.801221+0000","last_unstale":"2024-09-10T21:03:40.504133+0000","last_undegraded":"2024-09-10T21:03:40.504133+0000","last_fullsized":"2024-09-10T21:03:40.504133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:35.564890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978209+0000","last_change":"2024-09-10T21:01:58.884876+0000","last_active":"2024-09-10T21:02:41.978209+0000","last_peered":"2024-09-10T21:02:41.978209+0000","last_clean":"2024-09-10T21:02:41.978209+0000","last_became_active":"2024-09-10T21:01:58.884530+0000","last_became_peered":"2024-09-10T21:01:58.884530+0000","last_unstale":"2024-09-10T21:02:41.978209+0000","last_undegraded":"2024-09-10T21:02:41.978209+0000","last_fullsized":"2024-09-10T21:02:41.978209+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.851017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.479680+0000","last_change":"2024-09-10T21:02:00.811148+0000","last_active":"2024-09-10T21:03:40.479680+0000","last_peered":"2024-09-10T21:03:40.479680+0000","last_clean":"2024-09-10T21:03:40.479680+0000","last_became_active":"2024-09-10T21:02:00.809265+0000","last_became_peered":"2024-09-10T21:02:00.809265+0000","last_unstale":"2024-09-10T21:03:40.479680+0000","last_undegraded":"2024-09-10T21:03:40.479680+0000","last_fullsized":"2024-09-10T21:03:40.479680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:25.269876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":143,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.011599+0000","last_change":"2024-09-10T21:01:58.786917+0000","last_active":"2024-09-10T21:03:56.011599+0000","last_peered":"2024-09-10T21:03:56.011599+0000","last_clean":"2024-09-10T21:03:56.011599+0000","last_became_active":"2024-09-10T21:01:58.786595+0000","last_became_peered":"2024-09-10T21:01:58.786595+0000","last_unstale":"2024-09-10T21:03:56.011599+0000","last_undegraded":"2024-09-10T21:03:56.011599+0000","last_fullsized":"2024-09-10T21:03:56.011599+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:30.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74899,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1095,"num_read_kb":631,"num_write":267,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1237,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":224697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":407671,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1064937,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":406957,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":407223,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495671,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":1001366,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":847872,"data_stored":407206,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855427,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":999672,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":408529,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":406499,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":473646,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":474272,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":408545,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:04:00.631 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:04:00.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:00 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2562957934' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 -- 172.21.15.33:0/2821923352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 msgr2=0x7f51181055e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 --2- 172.21.15.33:0/2821923352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f51181055e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5108009900 tx=0x7f510802a410 comp rx=0 tx=0).stop 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 -- 172.21.15.33:0/2821923352 shutdown_connections 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 --2- 172.21.15.33:0/2821923352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51181086d0 0x7f511810ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 --2- 172.21.15.33:0/2821923352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5118105b20 0x7f5118107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 --2- 172.21.15.33:0/2821923352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f51181055e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 -- 172.21.15.33:0/2821923352 >> 172.21.15.33:0/2821923352 conn(0x7f51180fd190 msgr2=0x7f51180ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 -- 172.21.15.33:0/2821923352 shutdown_connections 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.766+0000 7f511ebda640 1 -- 172.21.15.33:0/2821923352 wait complete. 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 Processor -- start 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 -- start start 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f5118198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5118105b20 0x7f51181994a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51181086d0 0x7f511819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f511810e6a0 con 0x7f511806be60 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f511810e520 con 0x7f5118105b20 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511ebda640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f511810e820 con 0x7f51181086d0 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511c976640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f5118198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511c976640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f5118198e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52752/0 (socket says 172.21.15.33:52752) 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.767+0000 7f511c976640 1 -- 172.21.15.33:0/1211278409 learned_addr learned my addr 172.21.15.33:0/1211278409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f5117fff640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5118105b20 0x7f51181994a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511d177640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51181086d0 0x7f511819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511c976640 1 -- 172.21.15.33:0/1211278409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51181086d0 msgr2=0x7f511819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511c976640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51181086d0 0x7f511819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511c976640 1 -- 172.21.15.33:0/1211278409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5118105b20 msgr2=0x7f51181994a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511c976640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5118105b20 0x7f51181994a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511c976640 1 -- 172.21.15.33:0/1211278409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f511819a990 con 0x7f511806be60 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511d177640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51181086d0 0x7f511819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511c976640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f5118198e50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5108005940 tx=0x7f5108004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f5115ffb640 1 -- 172.21.15.33:0/1211278409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510803f030 con 0x7f511806be60 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f5115ffb640 1 -- 172.21.15.33:0/1211278409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5108003cb0 con 0x7f511806be60 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f5115ffb640 1 -- 172.21.15.33:0/1211278409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5108035070 con 0x7f511806be60 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f511819ac20 con 0x7f511806be60 2024-09-10T21:04:00.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.768+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51181a1350 con 0x7f511806be60 2024-09-10T21:04:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.769+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51180fec80 con 0x7f511806be60 2024-09-10T21:04:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.771+0000 7f5115ffb640 1 -- 172.21.15.33:0/1211278409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f510803b4f0 con 0x7f511806be60 2024-09-10T21:04:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.771+0000 7f5115ffb640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f51000b52e0 0x7f51000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.772+0000 7f5117fff640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f51000b52e0 0x7f51000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.772+0000 7f5115ffb640 1 -- 172.21.15.33:0/1211278409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5108106b10 con 0x7f511806be60 2024-09-10T21:04:00.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.772+0000 7f5117fff640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f51000b52e0 0x7f51000b77a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f511806c360 tx=0x7f510c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:00.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.774+0000 7f5115ffb640 1 -- 172.21.15.33:0/1211278409 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f51080cd9d0 con 0x7f511806be60 2024-09-10T21:04:00.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.960+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5118063590 con 0x7f51000b52e0 2024-09-10T21:04:00.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.964+0000 7f5115ffb640 1 -- 172.21.15.33:0/1211278409 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f5118063590 con 0x7f51000b52e0 2024-09-10T21:04:00.971 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:04:00.972 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-10T21:04:00.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.967+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f51000b52e0 msgr2=0x7f51000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.967+0000 7f511ebda640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f51000b52e0 0x7f51000b77a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f511806c360 tx=0x7f510c008040 comp rx=0 tx=0).stop 2024-09-10T21:04:00.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.967+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 msgr2=0x7f5118198e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:00.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.967+0000 7f511ebda640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f5118198e50 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f5108005940 tx=0x7f5108004430 comp rx=0 tx=0).stop 2024-09-10T21:04:00.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 shutdown_connections 2024-09-10T21:04:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f51000b52e0 0x7f51000b77a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f51181086d0 0x7f511819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5118105b20 0x7f51181994a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 --2- 172.21.15.33:0/1211278409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f511806be60 0x7f5118198e50 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 >> 172.21.15.33:0/1211278409 conn(0x7f51180fd190 msgr2=0x7f5118103e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 shutdown_connections 2024-09-10T21:04:00.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:00.968+0000 7f511ebda640 1 -- 172.21.15.33:0/1211278409 wait complete. 2024-09-10T21:04:00.982 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T21:04:00.113197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665501,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1171,"num_read_kb":695,"num_write":392,"num_write_kb":2311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":2953077,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":824,"ondisk_log_size":824,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355516,"kb_used_data":10428,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586308,"statfs":{"total":1151940427776,"available":1151576379392,"internally_reserved":0,"allocated":10678272,"data_stored":6866523,"data_compressed":176985,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4930,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":62,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003095"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498746+0000","last_change":"2024-09-10T21:02:00.805421+0000","last_active":"2024-09-10T21:03:40.498746+0000","last_peered":"2024-09-10T21:03:40.498746+0000","last_clean":"2024-09-10T21:03:40.498746+0000","last_became_active":"2024-09-10T21:02:00.804274+0000","last_became_peered":"2024-09-10T21:02:00.804274+0000","last_unstale":"2024-09-10T21:03:40.498746+0000","last_undegraded":"2024-09-10T21:03:40.498746+0000","last_fullsized":"2024-09-10T21:03:40.498746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:12.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.497068+0000","last_change":"2024-09-10T21:02:00.806104+0000","last_active":"2024-09-10T21:03:40.497068+0000","last_peered":"2024-09-10T21:03:40.497068+0000","last_clean":"2024-09-10T21:03:40.497068+0000","last_became_active":"2024-09-10T21:02:00.805510+0000","last_became_peered":"2024-09-10T21:02:00.805510+0000","last_unstale":"2024-09-10T21:03:40.497068+0000","last_undegraded":"2024-09-10T21:03:40.497068+0000","last_fullsized":"2024-09-10T21:03:40.497068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:37.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503426+0000","last_change":"2024-09-10T21:02:00.807691+0000","last_active":"2024-09-10T21:03:40.503426+0000","last_peered":"2024-09-10T21:03:40.503426+0000","last_clean":"2024-09-10T21:03:40.503426+0000","last_became_active":"2024-09-10T21:02:00.807364+0000","last_became_peered":"2024-09-10T21:02:00.807364+0000","last_unstale":"2024-09-10T21:03:40.503426+0000","last_undegraded":"2024-09-10T21:03:40.503426+0000","last_fullsized":"2024-09-10T21:03:40.503426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:27.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499586+0000","last_change":"2024-09-10T21:02:00.809542+0000","last_active":"2024-09-10T21:03:40.499586+0000","last_peered":"2024-09-10T21:03:40.499586+0000","last_clean":"2024-09-10T21:03:40.499586+0000","last_became_active":"2024-09-10T21:02:00.809034+0000","last_became_peered":"2024-09-10T21:02:00.809034+0000","last_unstale":"2024-09-10T21:03:40.499586+0000","last_undegraded":"2024-09-10T21:03:40.499586+0000","last_fullsized":"2024-09-10T21:03:40.499586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:29.923532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491227+0000","last_change":"2024-09-10T21:02:00.809355+0000","last_active":"2024-09-10T21:03:40.491227+0000","last_peered":"2024-09-10T21:03:40.491227+0000","last_clean":"2024-09-10T21:03:40.491227+0000","last_became_active":"2024-09-10T21:02:00.808863+0000","last_became_peered":"2024-09-10T21:02:00.808863+0000","last_unstale":"2024-09-10T21:03:40.491227+0000","last_undegraded":"2024-09-10T21:03:40.491227+0000","last_fullsized":"2024-09-10T21:03:40.491227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:29.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502481+0000","last_change":"2024-09-10T21:02:00.803756+0000","last_active":"2024-09-10T21:03:40.502481+0000","last_peered":"2024-09-10T21:03:40.502481+0000","last_clean":"2024-09-10T21:03:40.502481+0000","last_became_active":"2024-09-10T21:02:00.803406+0000","last_became_peered":"2024-09-10T21:02:00.803406+0000","last_unstale":"2024-09-10T21:03:40.502481+0000","last_undegraded":"2024-09-10T21:03:40.502481+0000","last_fullsized":"2024-09-10T21:03:40.502481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:53.437629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504748+0000","last_change":"2024-09-10T21:02:00.808845+0000","last_active":"2024-09-10T21:03:40.504748+0000","last_peered":"2024-09-10T21:03:40.504748+0000","last_clean":"2024-09-10T21:03:40.504748+0000","last_became_active":"2024-09-10T21:02:00.807759+0000","last_became_peered":"2024-09-10T21:02:00.807759+0000","last_unstale":"2024-09-10T21:03:40.504748+0000","last_undegraded":"2024-09-10T21:03:40.504748+0000","last_fullsized":"2024-09-10T21:03:40.504748+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.809389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.487348+0000","last_change":"2024-09-10T21:02:00.811199+0000","last_active":"2024-09-10T21:03:40.487348+0000","last_peered":"2024-09-10T21:03:40.487348+0000","last_clean":"2024-09-10T21:03:40.487348+0000","last_became_active":"2024-09-10T21:02:00.810692+0000","last_became_peered":"2024-09-10T21:02:00.810692+0000","last_unstale":"2024-09-10T21:03:40.487348+0000","last_undegraded":"2024-09-10T21:03:40.487348+0000","last_fullsized":"2024-09-10T21:03:40.487348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:59.557973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509702+0000","last_change":"2024-09-10T21:02:00.812686+0000","last_active":"2024-09-10T21:03:40.509702+0000","last_peered":"2024-09-10T21:03:40.509702+0000","last_clean":"2024-09-10T21:03:40.509702+0000","last_became_active":"2024-09-10T21:02:00.812363+0000","last_became_peered":"2024-09-10T21:02:00.812363+0000","last_unstale":"2024-09-10T21:03:40.509702+0000","last_undegraded":"2024-09-10T21:03:40.509702+0000","last_fullsized":"2024-09-10T21:03:40.509702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:37.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503735+0000","last_change":"2024-09-10T21:02:00.812058+0000","last_active":"2024-09-10T21:03:40.503735+0000","last_peered":"2024-09-10T21:03:40.503735+0000","last_clean":"2024-09-10T21:03:40.503735+0000","last_became_active":"2024-09-10T21:02:00.811490+0000","last_became_peered":"2024-09-10T21:02:00.811490+0000","last_unstale":"2024-09-10T21:03:40.503735+0000","last_undegraded":"2024-09-10T21:03:40.503735+0000","last_fullsized":"2024-09-10T21:03:40.503735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:31.557485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.507421+0000","last_change":"2024-09-10T21:02:00.809991+0000","last_active":"2024-09-10T21:03:40.507421+0000","last_peered":"2024-09-10T21:03:40.507421+0000","last_clean":"2024-09-10T21:03:40.507421+0000","last_became_active":"2024-09-10T21:02:00.809197+0000","last_became_peered":"2024-09-10T21:02:00.809197+0000","last_unstale":"2024-09-10T21:03:40.507421+0000","last_undegraded":"2024-09-10T21:03:40.507421+0000","last_fullsized":"2024-09-10T21:03:40.507421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490787+0000","last_change":"2024-09-10T21:02:41.918484+0000","last_active":"2024-09-10T21:03:40.490787+0000","last_peered":"2024-09-10T21:03:40.490787+0000","last_clean":"2024-09-10T21:03:40.490787+0000","last_became_active":"2024-09-10T21:02:41.918068+0000","last_became_peered":"2024-09-10T21:02:41.918068+0000","last_unstale":"2024-09-10T21:03:40.490787+0000","last_undegraded":"2024-09-10T21:03:40.490787+0000","last_fullsized":"2024-09-10T21:03:40.490787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:08.954072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.487079+0000","last_change":"2024-09-10T21:02:00.807280+0000","last_active":"2024-09-10T21:03:40.487079+0000","last_peered":"2024-09-10T21:03:40.487079+0000","last_clean":"2024-09-10T21:03:40.487079+0000","last_became_active":"2024-09-10T21:02:00.804247+0000","last_became_peered":"2024-09-10T21:02:00.804247+0000","last_unstale":"2024-09-10T21:03:40.487079+0000","last_undegraded":"2024-09-10T21:03:40.487079+0000","last_fullsized":"2024-09-10T21:03:40.487079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:28.163960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498116+0000","last_change":"2024-09-10T21:02:00.805723+0000","last_active":"2024-09-10T21:03:40.498116+0000","last_peered":"2024-09-10T21:03:40.498116+0000","last_clean":"2024-09-10T21:03:40.498116+0000","last_became_active":"2024-09-10T21:02:00.805104+0000","last_became_peered":"2024-09-10T21:02:00.805104+0000","last_unstale":"2024-09-10T21:03:40.498116+0000","last_undegraded":"2024-09-10T21:03:40.498116+0000","last_fullsized":"2024-09-10T21:03:40.498116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.056884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500888+0000","last_change":"2024-09-10T21:02:00.810497+0000","last_active":"2024-09-10T21:03:40.500888+0000","last_peered":"2024-09-10T21:03:40.500888+0000","last_clean":"2024-09-10T21:03:40.500888+0000","last_became_active":"2024-09-10T21:02:00.809392+0000","last_became_peered":"2024-09-10T21:02:00.809392+0000","last_unstale":"2024-09-10T21:03:40.500888+0000","last_undegraded":"2024-09-10T21:03:40.500888+0000","last_fullsized":"2024-09-10T21:03:40.500888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:51.214771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509646+0000","last_change":"2024-09-10T21:02:00.813973+0000","last_active":"2024-09-10T21:03:40.509646+0000","last_peered":"2024-09-10T21:03:40.509646+0000","last_clean":"2024-09-10T21:03:40.509646+0000","last_became_active":"2024-09-10T21:02:00.813580+0000","last_became_peered":"2024-09-10T21:02:00.813580+0000","last_unstale":"2024-09-10T21:03:40.509646+0000","last_undegraded":"2024-09-10T21:03:40.509646+0000","last_fullsized":"2024-09-10T21:03:40.509646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:48.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498592+0000","last_change":"2024-09-10T21:02:00.804844+0000","last_active":"2024-09-10T21:03:40.498592+0000","last_peered":"2024-09-10T21:03:40.498592+0000","last_clean":"2024-09-10T21:03:40.498592+0000","last_became_active":"2024-09-10T21:02:00.803866+0000","last_became_peered":"2024-09-10T21:02:00.803866+0000","last_unstale":"2024-09-10T21:03:40.498592+0000","last_undegraded":"2024-09-10T21:03:40.498592+0000","last_fullsized":"2024-09-10T21:03:40.498592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:05.231877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.506529+0000","last_change":"2024-09-10T21:02:00.801343+0000","last_active":"2024-09-10T21:03:40.506529+0000","last_peered":"2024-09-10T21:03:40.506529+0000","last_clean":"2024-09-10T21:03:40.506529+0000","last_became_active":"2024-09-10T21:02:00.801008+0000","last_became_peered":"2024-09-10T21:02:00.801008+0000","last_unstale":"2024-09-10T21:03:40.506529+0000","last_undegraded":"2024-09-10T21:03:40.506529+0000","last_fullsized":"2024-09-10T21:03:40.506529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:31.167925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498608+0000","last_change":"2024-09-10T21:02:41.917997+0000","last_active":"2024-09-10T21:03:40.498608+0000","last_peered":"2024-09-10T21:03:40.498608+0000","last_clean":"2024-09-10T21:03:40.498608+0000","last_became_active":"2024-09-10T21:02:41.917621+0000","last_became_peered":"2024-09-10T21:02:41.917621+0000","last_unstale":"2024-09-10T21:03:40.498608+0000","last_undegraded":"2024-09-10T21:03:40.498608+0000","last_fullsized":"2024-09-10T21:03:40.498608+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:12.742475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491709+0000","last_change":"2024-09-10T21:02:00.810502+0000","last_active":"2024-09-10T21:03:40.491709+0000","last_peered":"2024-09-10T21:03:40.491709+0000","last_clean":"2024-09-10T21:03:40.491709+0000","last_became_active":"2024-09-10T21:02:00.810209+0000","last_became_peered":"2024-09-10T21:02:00.810209+0000","last_unstale":"2024-09-10T21:03:40.491709+0000","last_undegraded":"2024-09-10T21:03:40.491709+0000","last_fullsized":"2024-09-10T21:03:40.491709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:51.311272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502162+0000","last_change":"2024-09-10T21:02:00.797119+0000","last_active":"2024-09-10T21:03:40.502162+0000","last_peered":"2024-09-10T21:03:40.502162+0000","last_clean":"2024-09-10T21:03:40.502162+0000","last_became_active":"2024-09-10T21:02:00.796788+0000","last_became_peered":"2024-09-10T21:02:00.796788+0000","last_unstale":"2024-09-10T21:03:40.502162+0000","last_undegraded":"2024-09-10T21:03:40.502162+0000","last_fullsized":"2024-09-10T21:03:40.502162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:07.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510423+0000","last_change":"2024-09-10T21:02:00.802284+0000","last_active":"2024-09-10T21:03:40.510423+0000","last_peered":"2024-09-10T21:03:40.510423+0000","last_clean":"2024-09-10T21:03:40.510423+0000","last_became_active":"2024-09-10T21:02:00.801964+0000","last_became_peered":"2024-09-10T21:02:00.801964+0000","last_unstale":"2024-09-10T21:03:40.510423+0000","last_undegraded":"2024-09-10T21:03:40.510423+0000","last_fullsized":"2024-09-10T21:03:40.510423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:45.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490488+0000","last_change":"2024-09-10T21:02:00.809196+0000","last_active":"2024-09-10T21:03:40.490488+0000","last_peered":"2024-09-10T21:03:40.490488+0000","last_clean":"2024-09-10T21:03:40.490488+0000","last_became_active":"2024-09-10T21:02:00.808903+0000","last_became_peered":"2024-09-10T21:02:00.808903+0000","last_unstale":"2024-09-10T21:03:40.490488+0000","last_undegraded":"2024-09-10T21:03:40.490488+0000","last_fullsized":"2024-09-10T21:03:40.490488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:01.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.488110+0000","last_change":"2024-09-10T21:02:00.808195+0000","last_active":"2024-09-10T21:03:40.488110+0000","last_peered":"2024-09-10T21:03:40.488110+0000","last_clean":"2024-09-10T21:03:40.488110+0000","last_became_active":"2024-09-10T21:02:00.807670+0000","last_became_peered":"2024-09-10T21:02:00.807670+0000","last_unstale":"2024-09-10T21:03:40.488110+0000","last_undegraded":"2024-09-10T21:03:40.488110+0000","last_fullsized":"2024-09-10T21:03:40.488110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:12.549344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.479659+0000","last_change":"2024-09-10T21:02:00.811401+0000","last_active":"2024-09-10T21:03:40.479659+0000","last_peered":"2024-09-10T21:03:40.479659+0000","last_clean":"2024-09-10T21:03:40.479659+0000","last_became_active":"2024-09-10T21:02:00.811055+0000","last_became_peered":"2024-09-10T21:02:00.811055+0000","last_unstale":"2024-09-10T21:03:40.479659+0000","last_undegraded":"2024-09-10T21:03:40.479659+0000","last_fullsized":"2024-09-10T21:03:40.479659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:58.641873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510083+0000","last_change":"2024-09-10T21:02:00.813796+0000","last_active":"2024-09-10T21:03:40.510083+0000","last_peered":"2024-09-10T21:03:40.510083+0000","last_clean":"2024-09-10T21:03:40.510083+0000","last_became_active":"2024-09-10T21:02:00.813468+0000","last_became_peered":"2024-09-10T21:02:00.813468+0000","last_unstale":"2024-09-10T21:03:40.510083+0000","last_undegraded":"2024-09-10T21:03:40.510083+0000","last_fullsized":"2024-09-10T21:03:40.510083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:25.964890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.488390+0000","last_change":"2024-09-10T21:02:00.811436+0000","last_active":"2024-09-10T21:03:40.488390+0000","last_peered":"2024-09-10T21:03:40.488390+0000","last_clean":"2024-09-10T21:03:40.488390+0000","last_became_active":"2024-09-10T21:02:00.810956+0000","last_became_peered":"2024-09-10T21:02:00.810956+0000","last_unstale":"2024-09-10T21:03:40.488390+0000","last_undegraded":"2024-09-10T21:03:40.488390+0000","last_fullsized":"2024-09-10T21:03:40.488390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.306832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.486792+0000","last_change":"2024-09-10T21:02:00.799370+0000","last_active":"2024-09-10T21:03:40.486792+0000","last_peered":"2024-09-10T21:03:40.486792+0000","last_clean":"2024-09-10T21:03:40.486792+0000","last_became_active":"2024-09-10T21:02:00.799044+0000","last_became_peered":"2024-09-10T21:02:00.799044+0000","last_unstale":"2024-09-10T21:03:40.486792+0000","last_undegraded":"2024-09-10T21:03:40.486792+0000","last_fullsized":"2024-09-10T21:03:40.486792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:40.332088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498868+0000","last_change":"2024-09-10T21:02:00.799830+0000","last_active":"2024-09-10T21:03:40.498868+0000","last_peered":"2024-09-10T21:03:40.498868+0000","last_clean":"2024-09-10T21:03:40.498868+0000","last_became_active":"2024-09-10T21:02:00.799533+0000","last_became_peered":"2024-09-10T21:02:00.799533+0000","last_unstale":"2024-09-10T21:03:40.498868+0000","last_undegraded":"2024-09-10T21:03:40.498868+0000","last_fullsized":"2024-09-10T21:03:40.498868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:23.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499851+0000","last_change":"2024-09-10T21:02:00.812739+0000","last_active":"2024-09-10T21:03:40.499851+0000","last_peered":"2024-09-10T21:03:40.499851+0000","last_clean":"2024-09-10T21:03:40.499851+0000","last_became_active":"2024-09-10T21:02:00.812395+0000","last_became_peered":"2024-09-10T21:02:00.812395+0000","last_unstale":"2024-09-10T21:03:40.499851+0000","last_undegraded":"2024-09-10T21:03:40.499851+0000","last_fullsized":"2024-09-10T21:03:40.499851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:55.727155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503421+0000","last_change":"2024-09-10T21:02:00.806291+0000","last_active":"2024-09-10T21:03:40.503421+0000","last_peered":"2024-09-10T21:03:40.503421+0000","last_clean":"2024-09-10T21:03:40.503421+0000","last_became_active":"2024-09-10T21:02:00.805943+0000","last_became_peered":"2024-09-10T21:02:00.805943+0000","last_unstale":"2024-09-10T21:03:40.503421+0000","last_undegraded":"2024-09-10T21:03:40.503421+0000","last_fullsized":"2024-09-10T21:03:40.503421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:21.354105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500561+0000","last_change":"2024-09-10T21:02:00.808774+0000","last_active":"2024-09-10T21:03:40.500561+0000","last_peered":"2024-09-10T21:03:40.500561+0000","last_clean":"2024-09-10T21:03:40.500561+0000","last_became_active":"2024-09-10T21:02:00.808027+0000","last_became_peered":"2024-09-10T21:02:00.808027+0000","last_unstale":"2024-09-10T21:03:40.500561+0000","last_undegraded":"2024-09-10T21:03:40.500561+0000","last_fullsized":"2024-09-10T21:03:40.500561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.486316+0000","last_change":"2024-09-10T21:02:00.808349+0000","last_active":"2024-09-10T21:03:40.486316+0000","last_peered":"2024-09-10T21:03:40.486316+0000","last_clean":"2024-09-10T21:03:40.486316+0000","last_became_active":"2024-09-10T21:02:00.807839+0000","last_became_peered":"2024-09-10T21:02:00.807839+0000","last_unstale":"2024-09-10T21:03:40.486316+0000","last_undegraded":"2024-09-10T21:03:40.486316+0000","last_fullsized":"2024-09-10T21:03:40.486316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:53.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":151,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:55.977242+0000","last_change":"2024-09-10T21:02:41.877376+0000","last_active":"2024-09-10T21:03:55.977242+0000","last_peered":"2024-09-10T21:03:55.977242+0000","last_clean":"2024-09-10T21:03:55.977242+0000","last_became_active":"2024-09-10T21:02:41.871397+0000","last_became_peered":"2024-09-10T21:02:41.871397+0000","last_unstale":"2024-09-10T21:03:55.977242+0000","last_undegraded":"2024-09-10T21:03:55.977242+0000","last_fullsized":"2024-09-10T21:03:55.977242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:59.961597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.489060+0000","last_change":"2024-09-10T21:02:41.979196+0000","last_active":"2024-09-10T21:03:40.489060+0000","last_peered":"2024-09-10T21:03:40.489060+0000","last_clean":"2024-09-10T21:03:40.489060+0000","last_became_active":"2024-09-10T21:02:41.978781+0000","last_became_peered":"2024-09-10T21:02:41.978781+0000","last_unstale":"2024-09-10T21:03:40.489060+0000","last_undegraded":"2024-09-10T21:03:40.489060+0000","last_fullsized":"2024-09-10T21:03:40.489060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:15.100055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917027+0000","last_change":"2024-09-10T21:01:58.787619+0000","last_active":"2024-09-10T21:02:41.917027+0000","last_peered":"2024-09-10T21:02:41.917027+0000","last_clean":"2024-09-10T21:02:41.917027+0000","last_became_active":"2024-09-10T21:01:58.787138+0000","last_became_peered":"2024-09-10T21:01:58.787138+0000","last_unstale":"2024-09-10T21:02:41.917027+0000","last_undegraded":"2024-09-10T21:02:41.917027+0000","last_fullsized":"2024-09-10T21:02:41.917027+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:33.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916753+0000","last_change":"2024-09-10T21:01:58.780115+0000","last_active":"2024-09-10T21:02:41.916753+0000","last_peered":"2024-09-10T21:02:41.916753+0000","last_clean":"2024-09-10T21:02:41.916753+0000","last_became_active":"2024-09-10T21:01:58.779757+0000","last_became_peered":"2024-09-10T21:01:58.779757+0000","last_unstale":"2024-09-10T21:02:41.916753+0000","last_undegraded":"2024-09-10T21:02:41.916753+0000","last_fullsized":"2024-09-10T21:02:41.916753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:29.337978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491520+0000","last_change":"2024-09-10T21:02:00.800925+0000","last_active":"2024-09-10T21:03:40.491520+0000","last_peered":"2024-09-10T21:03:40.491520+0000","last_clean":"2024-09-10T21:03:40.491520+0000","last_became_active":"2024-09-10T21:02:00.798567+0000","last_became_peered":"2024-09-10T21:02:00.798567+0000","last_unstale":"2024-09-10T21:03:40.491520+0000","last_undegraded":"2024-09-10T21:03:40.491520+0000","last_fullsized":"2024-09-10T21:03:40.491520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:01.873716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867127+0000","last_change":"2024-09-10T21:01:58.786040+0000","last_active":"2024-09-10T21:02:41.867127+0000","last_peered":"2024-09-10T21:02:41.867127+0000","last_clean":"2024-09-10T21:02:41.867127+0000","last_became_active":"2024-09-10T21:01:58.785703+0000","last_became_peered":"2024-09-10T21:01:58.785703+0000","last_unstale":"2024-09-10T21:02:41.867127+0000","last_undegraded":"2024-09-10T21:02:41.867127+0000","last_fullsized":"2024-09-10T21:02:41.867127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:39.741614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.497664+0000","last_change":"2024-09-10T21:02:00.798368+0000","last_active":"2024-09-10T21:03:40.497664+0000","last_peered":"2024-09-10T21:03:40.497664+0000","last_clean":"2024-09-10T21:03:40.497664+0000","last_became_active":"2024-09-10T21:02:00.798080+0000","last_became_peered":"2024-09-10T21:02:00.798080+0000","last_unstale":"2024-09-10T21:03:40.497664+0000","last_undegraded":"2024-09-10T21:03:40.497664+0000","last_fullsized":"2024-09-10T21:03:40.497664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:31.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":238,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.978268+0000","last_change":"2024-09-10T21:01:58.788119+0000","last_active":"2024-09-10T21:03:56.978268+0000","last_peered":"2024-09-10T21:03:56.978268+0000","last_clean":"2024-09-10T21:03:56.978268+0000","last_became_active":"2024-09-10T21:01:58.787724+0000","last_became_peered":"2024-09-10T21:01:58.787724+0000","last_unstale":"2024-09-10T21:03:56.978268+0000","last_undegraded":"2024-09-10T21:03:56.978268+0000","last_fullsized":"2024-09-10T21:03:56.978268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:42.872858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":209,"num_read_kb":209,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510209+0000","last_change":"2024-09-10T21:02:00.804425+0000","last_active":"2024-09-10T21:03:40.510209+0000","last_peered":"2024-09-10T21:03:40.510209+0000","last_clean":"2024-09-10T21:03:40.510209+0000","last_became_active":"2024-09-10T21:02:00.804093+0000","last_became_peered":"2024-09-10T21:02:00.804093+0000","last_unstale":"2024-09-10T21:03:40.510209+0000","last_undegraded":"2024-09-10T21:03:40.510209+0000","last_fullsized":"2024-09-10T21:03:40.510209+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:46.183326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870381+0000","last_change":"2024-09-10T21:01:58.791323+0000","last_active":"2024-09-10T21:02:41.870381+0000","last_peered":"2024-09-10T21:02:41.870381+0000","last_clean":"2024-09-10T21:02:41.870381+0000","last_became_active":"2024-09-10T21:01:58.790551+0000","last_became_peered":"2024-09-10T21:01:58.790551+0000","last_unstale":"2024-09-10T21:02:41.870381+0000","last_undegraded":"2024-09-10T21:02:41.870381+0000","last_fullsized":"2024-09-10T21:02:41.870381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:18.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499172+0000","last_change":"2024-09-10T21:02:00.804025+0000","last_active":"2024-09-10T21:03:40.499172+0000","last_peered":"2024-09-10T21:03:40.499172+0000","last_clean":"2024-09-10T21:03:40.499172+0000","last_became_active":"2024-09-10T21:02:00.803663+0000","last_became_peered":"2024-09-10T21:02:00.803663+0000","last_unstale":"2024-09-10T21:03:40.499172+0000","last_undegraded":"2024-09-10T21:03:40.499172+0000","last_fullsized":"2024-09-10T21:03:40.499172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:30.609020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868035+0000","last_change":"2024-09-10T21:01:58.887067+0000","last_active":"2024-09-10T21:02:41.868035+0000","last_peered":"2024-09-10T21:02:41.868035+0000","last_clean":"2024-09-10T21:02:41.868035+0000","last_became_active":"2024-09-10T21:01:58.886611+0000","last_became_peered":"2024-09-10T21:01:58.886611+0000","last_unstale":"2024-09-10T21:02:41.868035+0000","last_undegraded":"2024-09-10T21:02:41.868035+0000","last_fullsized":"2024-09-10T21:02:41.868035+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:06.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.508877+0000","last_change":"2024-09-10T21:02:41.917998+0000","last_active":"2024-09-10T21:03:40.508877+0000","last_peered":"2024-09-10T21:03:40.508877+0000","last_clean":"2024-09-10T21:03:40.508877+0000","last_became_active":"2024-09-10T21:02:41.917665+0000","last_became_peered":"2024-09-10T21:02:41.917665+0000","last_unstale":"2024-09-10T21:03:40.508877+0000","last_undegraded":"2024-09-10T21:03:40.508877+0000","last_fullsized":"2024-09-10T21:03:40.508877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:15.147379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'37","reported_seq":280,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.978810+0000","last_change":"2024-09-10T21:01:58.790966+0000","last_active":"2024-09-10T21:03:56.978810+0000","last_peered":"2024-09-10T21:03:56.978810+0000","last_clean":"2024-09-10T21:03:56.978810+0000","last_became_active":"2024-09-10T21:01:58.790636+0000","last_became_peered":"2024-09-10T21:01:58.790636+0000","last_unstale":"2024-09-10T21:03:56.978810+0000","last_undegraded":"2024-09-10T21:03:56.978810+0000","last_fullsized":"2024-09-10T21:03:56.978810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:06.209074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66041,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.501096+0000","last_change":"2024-09-10T21:02:00.810610+0000","last_active":"2024-09-10T21:03:40.501096+0000","last_peered":"2024-09-10T21:03:40.501096+0000","last_clean":"2024-09-10T21:03:40.501096+0000","last_became_active":"2024-09-10T21:02:00.810224+0000","last_became_peered":"2024-09-10T21:02:00.810224+0000","last_unstale":"2024-09-10T21:03:40.501096+0000","last_undegraded":"2024-09-10T21:03:40.501096+0000","last_fullsized":"2024-09-10T21:03:40.501096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:32.785312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870285+0000","last_change":"2024-09-10T21:01:58.786476+0000","last_active":"2024-09-10T21:02:41.870285+0000","last_peered":"2024-09-10T21:02:41.870285+0000","last_clean":"2024-09-10T21:02:41.870285+0000","last_became_active":"2024-09-10T21:01:58.786153+0000","last_became_peered":"2024-09-10T21:01:58.786153+0000","last_unstale":"2024-09-10T21:02:41.870285+0000","last_undegraded":"2024-09-10T21:02:41.870285+0000","last_fullsized":"2024-09-10T21:02:41.870285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:00.118591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.501464+0000","last_change":"2024-09-10T21:02:00.796138+0000","last_active":"2024-09-10T21:03:40.501464+0000","last_peered":"2024-09-10T21:03:40.501464+0000","last_clean":"2024-09-10T21:03:40.501464+0000","last_became_active":"2024-09-10T21:02:00.795847+0000","last_became_peered":"2024-09-10T21:02:00.795847+0000","last_unstale":"2024-09-10T21:03:40.501464+0000","last_undegraded":"2024-09-10T21:03:40.501464+0000","last_fullsized":"2024-09-10T21:03:40.501464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:29.462134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867576+0000","last_change":"2024-09-10T21:01:58.789314+0000","last_active":"2024-09-10T21:02:41.867576+0000","last_peered":"2024-09-10T21:02:41.867576+0000","last_clean":"2024-09-10T21:02:41.867576+0000","last_became_active":"2024-09-10T21:01:58.788594+0000","last_became_peered":"2024-09-10T21:01:58.788594+0000","last_unstale":"2024-09-10T21:02:41.867576+0000","last_undegraded":"2024-09-10T21:02:41.867576+0000","last_fullsized":"2024-09-10T21:02:41.867576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:44.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.506541+0000","last_change":"2024-09-10T21:02:00.799828+0000","last_active":"2024-09-10T21:03:40.506541+0000","last_peered":"2024-09-10T21:03:40.506541+0000","last_clean":"2024-09-10T21:03:40.506541+0000","last_became_active":"2024-09-10T21:02:00.799465+0000","last_became_peered":"2024-09-10T21:02:00.799465+0000","last_unstale":"2024-09-10T21:03:40.506541+0000","last_undegraded":"2024-09-10T21:03:40.506541+0000","last_fullsized":"2024-09-10T21:03:40.506541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:56.794326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'8","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:57.029723+0000","last_change":"2024-09-10T21:01:58.788730+0000","last_active":"2024-09-10T21:03:57.029723+0000","last_peered":"2024-09-10T21:03:57.029723+0000","last_clean":"2024-09-10T21:03:57.029723+0000","last_became_active":"2024-09-10T21:01:58.788317+0000","last_became_peered":"2024-09-10T21:01:58.788317+0000","last_unstale":"2024-09-10T21:03:57.029723+0000","last_undegraded":"2024-09-10T21:03:57.029723+0000","last_fullsized":"2024-09-10T21:03:57.029723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:52.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504421+0000","last_change":"2024-09-10T21:02:00.807104+0000","last_active":"2024-09-10T21:03:40.504421+0000","last_peered":"2024-09-10T21:03:40.504421+0000","last_clean":"2024-09-10T21:03:40.504421+0000","last_became_active":"2024-09-10T21:02:00.806815+0000","last_became_peered":"2024-09-10T21:02:00.806815+0000","last_unstale":"2024-09-10T21:03:40.504421+0000","last_undegraded":"2024-09-10T21:03:40.504421+0000","last_fullsized":"2024-09-10T21:03:40.504421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:24.747662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.890722+0000","last_change":"2024-09-10T21:02:41.890657+0000","last_active":"2024-09-10T21:02:41.890722+0000","last_peered":"2024-09-10T21:02:41.890722+0000","last_clean":"2024-09-10T21:02:41.890722+0000","last_became_active":"2024-09-10T21:02:41.873778+0000","last_became_peered":"2024-09-10T21:02:41.873778+0000","last_unstale":"2024-09-10T21:02:41.890722+0000","last_undegraded":"2024-09-10T21:02:41.890722+0000","last_fullsized":"2024-09-10T21:02:41.890722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:58.678679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490766+0000","last_change":"2024-09-10T21:02:00.806796+0000","last_active":"2024-09-10T21:03:40.490766+0000","last_peered":"2024-09-10T21:03:40.490766+0000","last_clean":"2024-09-10T21:03:40.490766+0000","last_became_active":"2024-09-10T21:02:00.805802+0000","last_became_peered":"2024-09-10T21:02:00.805802+0000","last_unstale":"2024-09-10T21:03:40.490766+0000","last_undegraded":"2024-09-10T21:03:40.490766+0000","last_fullsized":"2024-09-10T21:03:40.490766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.962870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871699+0000","last_change":"2024-09-10T21:01:58.781088+0000","last_active":"2024-09-10T21:02:41.871699+0000","last_peered":"2024-09-10T21:02:41.871699+0000","last_clean":"2024-09-10T21:02:41.871699+0000","last_became_active":"2024-09-10T21:01:58.780731+0000","last_became_peered":"2024-09-10T21:01:58.780731+0000","last_unstale":"2024-09-10T21:02:41.871699+0000","last_undegraded":"2024-09-10T21:02:41.871699+0000","last_fullsized":"2024-09-10T21:02:41.871699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:50.636077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491805+0000","last_change":"2024-09-10T21:02:00.806551+0000","last_active":"2024-09-10T21:03:40.491805+0000","last_peered":"2024-09-10T21:03:40.491805+0000","last_clean":"2024-09-10T21:03:40.491805+0000","last_became_active":"2024-09-10T21:02:00.806134+0000","last_became_peered":"2024-09-10T21:02:00.806134+0000","last_unstale":"2024-09-10T21:03:40.491805+0000","last_undegraded":"2024-09-10T21:03:40.491805+0000","last_fullsized":"2024-09-10T21:03:40.491805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.574372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916954+0000","last_change":"2024-09-10T21:01:58.780294+0000","last_active":"2024-09-10T21:02:41.916954+0000","last_peered":"2024-09-10T21:02:41.916954+0000","last_clean":"2024-09-10T21:02:41.916954+0000","last_became_active":"2024-09-10T21:01:58.779767+0000","last_became_peered":"2024-09-10T21:01:58.779767+0000","last_unstale":"2024-09-10T21:02:41.916954+0000","last_undegraded":"2024-09-10T21:02:41.916954+0000","last_fullsized":"2024-09-10T21:02:41.916954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:21.734159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503245+0000","last_change":"2024-09-10T21:02:00.809393+0000","last_active":"2024-09-10T21:03:40.503245+0000","last_peered":"2024-09-10T21:03:40.503245+0000","last_clean":"2024-09-10T21:03:40.503245+0000","last_became_active":"2024-09-10T21:02:00.809012+0000","last_became_peered":"2024-09-10T21:02:00.809012+0000","last_unstale":"2024-09-10T21:03:40.503245+0000","last_undegraded":"2024-09-10T21:03:40.503245+0000","last_fullsized":"2024-09-10T21:03:40.503245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:03.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868189+0000","last_change":"2024-09-10T21:01:58.786507+0000","last_active":"2024-09-10T21:02:41.868189+0000","last_peered":"2024-09-10T21:02:41.868189+0000","last_clean":"2024-09-10T21:02:41.868189+0000","last_became_active":"2024-09-10T21:01:58.786115+0000","last_became_peered":"2024-09-10T21:01:58.786115+0000","last_unstale":"2024-09-10T21:02:41.868189+0000","last_undegraded":"2024-09-10T21:02:41.868189+0000","last_fullsized":"2024-09-10T21:02:41.868189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.985353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504534+0000","last_change":"2024-09-10T21:02:00.805050+0000","last_active":"2024-09-10T21:03:40.504534+0000","last_peered":"2024-09-10T21:03:40.504534+0000","last_clean":"2024-09-10T21:03:40.504534+0000","last_became_active":"2024-09-10T21:02:00.804706+0000","last_became_peered":"2024-09-10T21:02:00.804706+0000","last_unstale":"2024-09-10T21:03:40.504534+0000","last_undegraded":"2024-09-10T21:03:40.504534+0000","last_fullsized":"2024-09-10T21:03:40.504534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:15.463986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":141,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:58.030099+0000","last_change":"2024-09-10T21:01:59.114525+0000","last_active":"2024-09-10T21:03:58.030099+0000","last_peered":"2024-09-10T21:03:58.030099+0000","last_clean":"2024-09-10T21:03:58.030099+0000","last_became_active":"2024-09-10T21:01:59.114076+0000","last_became_peered":"2024-09-10T21:01:59.114076+0000","last_unstale":"2024-09-10T21:03:58.030099+0000","last_undegraded":"2024-09-10T21:03:58.030099+0000","last_fullsized":"2024-09-10T21:03:58.030099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:53.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509798+0000","last_change":"2024-09-10T21:02:00.799803+0000","last_active":"2024-09-10T21:03:40.509798+0000","last_peered":"2024-09-10T21:03:40.509798+0000","last_clean":"2024-09-10T21:03:40.509798+0000","last_became_active":"2024-09-10T21:02:00.799459+0000","last_became_peered":"2024-09-10T21:02:00.799459+0000","last_unstale":"2024-09-10T21:03:40.509798+0000","last_undegraded":"2024-09-10T21:03:40.509798+0000","last_fullsized":"2024-09-10T21:03:40.509798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:57.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870115+0000","last_change":"2024-09-10T21:00:48.381903+0000","last_active":"2024-09-10T21:02:41.870115+0000","last_peered":"2024-09-10T21:02:41.870115+0000","last_clean":"2024-09-10T21:02:41.870115+0000","last_became_active":"2024-09-10T21:00:48.376874+0000","last_became_peered":"2024-09-10T21:00:48.376874+0000","last_unstale":"2024-09-10T21:02:41.870115+0000","last_undegraded":"2024-09-10T21:02:41.870115+0000","last_fullsized":"2024-09-10T21:02:41.870115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_clean_scrub_stamp":"2024-09-10T20:57:24.516525+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:35.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866334+0000","last_change":"2024-09-10T21:01:58.788005+0000","last_active":"2024-09-10T21:02:41.866334+0000","last_peered":"2024-09-10T21:02:41.866334+0000","last_clean":"2024-09-10T21:02:41.866334+0000","last_became_active":"2024-09-10T21:01:58.786471+0000","last_became_peered":"2024-09-10T21:01:58.786471+0000","last_unstale":"2024-09-10T21:02:41.866334+0000","last_undegraded":"2024-09-10T21:02:41.866334+0000","last_fullsized":"2024-09-10T21:02:41.866334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:15.622814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.505266+0000","last_change":"2024-09-10T21:02:00.802573+0000","last_active":"2024-09-10T21:03:40.505266+0000","last_peered":"2024-09-10T21:03:40.505266+0000","last_clean":"2024-09-10T21:03:40.505266+0000","last_became_active":"2024-09-10T21:02:00.802263+0000","last_became_peered":"2024-09-10T21:02:00.802263+0000","last_unstale":"2024-09-10T21:03:40.505266+0000","last_undegraded":"2024-09-10T21:03:40.505266+0000","last_fullsized":"2024-09-10T21:03:40.505266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:54.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870004+0000","last_change":"2024-09-10T21:01:58.785700+0000","last_active":"2024-09-10T21:02:41.870004+0000","last_peered":"2024-09-10T21:02:41.870004+0000","last_clean":"2024-09-10T21:02:41.870004+0000","last_became_active":"2024-09-10T21:01:58.785346+0000","last_became_peered":"2024-09-10T21:01:58.785346+0000","last_unstale":"2024-09-10T21:02:41.870004+0000","last_undegraded":"2024-09-10T21:02:41.870004+0000","last_fullsized":"2024-09-10T21:02:41.870004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:39.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.495838+0000","last_change":"2024-09-10T21:02:00.808045+0000","last_active":"2024-09-10T21:03:40.495838+0000","last_peered":"2024-09-10T21:03:40.495838+0000","last_clean":"2024-09-10T21:03:40.495838+0000","last_became_active":"2024-09-10T21:02:00.807757+0000","last_became_peered":"2024-09-10T21:02:00.807757+0000","last_unstale":"2024-09-10T21:03:40.495838+0000","last_undegraded":"2024-09-10T21:03:40.495838+0000","last_fullsized":"2024-09-10T21:03:40.495838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:16.198290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'7","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:58.030473+0000","last_change":"2024-09-10T21:01:58.884939+0000","last_active":"2024-09-10T21:03:58.030473+0000","last_peered":"2024-09-10T21:03:58.030473+0000","last_clean":"2024-09-10T21:03:58.030473+0000","last_became_active":"2024-09-10T21:01:58.884517+0000","last_became_peered":"2024-09-10T21:01:58.884517+0000","last_unstale":"2024-09-10T21:03:58.030473+0000","last_undegraded":"2024-09-10T21:03:58.030473+0000","last_fullsized":"2024-09-10T21:03:58.030473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.579662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490615+0000","last_change":"2024-09-10T21:02:00.813021+0000","last_active":"2024-09-10T21:03:40.490615+0000","last_peered":"2024-09-10T21:03:40.490615+0000","last_clean":"2024-09-10T21:03:40.490615+0000","last_became_active":"2024-09-10T21:02:00.812622+0000","last_became_peered":"2024-09-10T21:02:00.812622+0000","last_unstale":"2024-09-10T21:03:40.490615+0000","last_undegraded":"2024-09-10T21:03:40.490615+0000","last_fullsized":"2024-09-10T21:03:40.490615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:48.348100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:54.009554+0000","last_change":"2024-09-10T21:01:58.792187+0000","last_active":"2024-09-10T21:03:54.009554+0000","last_peered":"2024-09-10T21:03:54.009554+0000","last_clean":"2024-09-10T21:03:54.009554+0000","last_became_active":"2024-09-10T21:01:58.791806+0000","last_became_peered":"2024-09-10T21:01:58.791806+0000","last_unstale":"2024-09-10T21:03:54.009554+0000","last_undegraded":"2024-09-10T21:03:54.009554+0000","last_fullsized":"2024-09-10T21:03:54.009554+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:48.493396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502741+0000","last_change":"2024-09-10T21:02:00.804073+0000","last_active":"2024-09-10T21:03:40.502741+0000","last_peered":"2024-09-10T21:03:40.502741+0000","last_clean":"2024-09-10T21:03:40.502741+0000","last_became_active":"2024-09-10T21:02:00.803760+0000","last_became_peered":"2024-09-10T21:02:00.803760+0000","last_unstale":"2024-09-10T21:03:40.502741+0000","last_undegraded":"2024-09-10T21:03:40.502741+0000","last_fullsized":"2024-09-10T21:03:40.502741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:01.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865323+0000","last_change":"2024-09-10T21:01:58.885088+0000","last_active":"2024-09-10T21:02:41.865323+0000","last_peered":"2024-09-10T21:02:41.865323+0000","last_clean":"2024-09-10T21:02:41.865323+0000","last_became_active":"2024-09-10T21:01:58.884729+0000","last_became_peered":"2024-09-10T21:01:58.884729+0000","last_unstale":"2024-09-10T21:02:41.865323+0000","last_undegraded":"2024-09-10T21:02:41.865323+0000","last_fullsized":"2024-09-10T21:02:41.865323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:37.451629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490451+0000","last_change":"2024-09-10T21:02:00.810329+0000","last_active":"2024-09-10T21:03:40.490451+0000","last_peered":"2024-09-10T21:03:40.490451+0000","last_clean":"2024-09-10T21:03:40.490451+0000","last_became_active":"2024-09-10T21:02:00.809985+0000","last_became_peered":"2024-09-10T21:02:00.809985+0000","last_unstale":"2024-09-10T21:03:40.490451+0000","last_undegraded":"2024-09-10T21:03:40.490451+0000","last_fullsized":"2024-09-10T21:03:40.490451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:39.631969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.880605+0000","last_change":"2024-09-10T21:02:41.880117+0000","last_active":"2024-09-10T21:02:41.880605+0000","last_peered":"2024-09-10T21:02:41.880605+0000","last_clean":"2024-09-10T21:02:41.880605+0000","last_became_active":"2024-09-10T21:02:41.872494+0000","last_became_peered":"2024-09-10T21:02:41.872494+0000","last_unstale":"2024-09-10T21:02:41.880605+0000","last_undegraded":"2024-09-10T21:02:41.880605+0000","last_fullsized":"2024-09-10T21:02:41.880605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:06.965479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491585+0000","last_change":"2024-09-10T21:02:00.804961+0000","last_active":"2024-09-10T21:03:40.491585+0000","last_peered":"2024-09-10T21:03:40.491585+0000","last_clean":"2024-09-10T21:03:40.491585+0000","last_became_active":"2024-09-10T21:02:00.804147+0000","last_became_peered":"2024-09-10T21:02:00.804147+0000","last_unstale":"2024-09-10T21:03:40.491585+0000","last_undegraded":"2024-09-10T21:03:40.491585+0000","last_fullsized":"2024-09-10T21:03:40.491585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:15.801815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.806420+0000","last_change":"2024-09-10T21:01:58.781292+0000","last_active":"2024-09-10T21:03:40.806420+0000","last_peered":"2024-09-10T21:03:40.806420+0000","last_clean":"2024-09-10T21:03:40.806420+0000","last_became_active":"2024-09-10T21:01:58.780995+0000","last_became_peered":"2024-09-10T21:01:58.780995+0000","last_unstale":"2024-09-10T21:03:40.806420+0000","last_undegraded":"2024-09-10T21:03:40.806420+0000","last_fullsized":"2024-09-10T21:03:40.806420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:56.914723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.493246+0000","last_change":"2024-09-10T21:02:00.805878+0000","last_active":"2024-09-10T21:03:40.493246+0000","last_peered":"2024-09-10T21:03:40.493246+0000","last_clean":"2024-09-10T21:03:40.493246+0000","last_became_active":"2024-09-10T21:02:00.804565+0000","last_became_peered":"2024-09-10T21:02:00.804565+0000","last_unstale":"2024-09-10T21:03:40.493246+0000","last_undegraded":"2024-09-10T21:03:40.493246+0000","last_fullsized":"2024-09-10T21:03:40.493246+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:24.359643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866263+0000","last_change":"2024-09-10T21:01:59.114862+0000","last_active":"2024-09-10T21:02:41.866263+0000","last_peered":"2024-09-10T21:02:41.866263+0000","last_clean":"2024-09-10T21:02:41.866263+0000","last_became_active":"2024-09-10T21:01:59.114411+0000","last_became_peered":"2024-09-10T21:01:59.114411+0000","last_unstale":"2024-09-10T21:02:41.866263+0000","last_undegraded":"2024-09-10T21:02:41.866263+0000","last_fullsized":"2024-09-10T21:02:41.866263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.645829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498781+0000","last_change":"2024-09-10T21:02:00.799230+0000","last_active":"2024-09-10T21:03:40.498781+0000","last_peered":"2024-09-10T21:03:40.498781+0000","last_clean":"2024-09-10T21:03:40.498781+0000","last_became_active":"2024-09-10T21:02:00.798992+0000","last_became_peered":"2024-09-10T21:02:00.798992+0000","last_unstale":"2024-09-10T21:03:40.498781+0000","last_undegraded":"2024-09-10T21:03:40.498781+0000","last_fullsized":"2024-09-10T21:03:40.498781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:40.322066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867967+0000","last_change":"2024-09-10T21:01:58.887042+0000","last_active":"2024-09-10T21:02:41.867967+0000","last_peered":"2024-09-10T21:02:41.867967+0000","last_clean":"2024-09-10T21:02:41.867967+0000","last_became_active":"2024-09-10T21:01:58.886710+0000","last_became_peered":"2024-09-10T21:01:58.886710+0000","last_unstale":"2024-09-10T21:02:41.867967+0000","last_undegraded":"2024-09-10T21:02:41.867967+0000","last_fullsized":"2024-09-10T21:02:41.867967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:42.672899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.480934+0000","last_change":"2024-09-10T21:02:00.809763+0000","last_active":"2024-09-10T21:03:40.480934+0000","last_peered":"2024-09-10T21:03:40.480934+0000","last_clean":"2024-09-10T21:03:40.480934+0000","last_became_active":"2024-09-10T21:02:00.808955+0000","last_became_peered":"2024-09-10T21:02:00.808955+0000","last_unstale":"2024-09-10T21:03:40.480934+0000","last_undegraded":"2024-09-10T21:03:40.480934+0000","last_fullsized":"2024-09-10T21:03:40.480934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:22.622852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977719+0000","last_change":"2024-09-10T21:01:58.786312+0000","last_active":"2024-09-10T21:02:41.977719+0000","last_peered":"2024-09-10T21:02:41.977719+0000","last_clean":"2024-09-10T21:02:41.977719+0000","last_became_active":"2024-09-10T21:01:58.785894+0000","last_became_peered":"2024-09-10T21:01:58.785894+0000","last_unstale":"2024-09-10T21:02:41.977719+0000","last_undegraded":"2024-09-10T21:02:41.977719+0000","last_fullsized":"2024-09-10T21:02:41.977719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:11.682904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500168+0000","last_change":"2024-09-10T21:02:00.808374+0000","last_active":"2024-09-10T21:03:40.500168+0000","last_peered":"2024-09-10T21:03:40.500168+0000","last_clean":"2024-09-10T21:03:40.500168+0000","last_became_active":"2024-09-10T21:02:00.807565+0000","last_became_peered":"2024-09-10T21:02:00.807565+0000","last_unstale":"2024-09-10T21:03:40.500168+0000","last_undegraded":"2024-09-10T21:03:40.500168+0000","last_fullsized":"2024-09-10T21:03:40.500168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:24.227480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870458+0000","last_change":"2024-09-10T21:01:58.782375+0000","last_active":"2024-09-10T21:02:41.870458+0000","last_peered":"2024-09-10T21:02:41.870458+0000","last_clean":"2024-09-10T21:02:41.870458+0000","last_became_active":"2024-09-10T21:01:58.781964+0000","last_became_peered":"2024-09-10T21:01:58.781964+0000","last_unstale":"2024-09-10T21:02:41.870458+0000","last_undegraded":"2024-09-10T21:02:41.870458+0000","last_fullsized":"2024-09-10T21:02:41.870458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.460126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.485903+0000","last_change":"2024-09-10T21:02:00.806612+0000","last_active":"2024-09-10T21:03:40.485903+0000","last_peered":"2024-09-10T21:03:40.485903+0000","last_clean":"2024-09-10T21:03:40.485903+0000","last_became_active":"2024-09-10T21:02:00.803470+0000","last_became_peered":"2024-09-10T21:02:00.803470+0000","last_unstale":"2024-09-10T21:03:40.485903+0000","last_undegraded":"2024-09-10T21:03:40.485903+0000","last_fullsized":"2024-09-10T21:03:40.485903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:08.350931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871476+0000","last_change":"2024-09-10T21:01:58.788987+0000","last_active":"2024-09-10T21:02:41.871476+0000","last_peered":"2024-09-10T21:02:41.871476+0000","last_clean":"2024-09-10T21:02:41.871476+0000","last_became_active":"2024-09-10T21:01:58.788605+0000","last_became_peered":"2024-09-10T21:01:58.788605+0000","last_unstale":"2024-09-10T21:02:41.871476+0000","last_undegraded":"2024-09-10T21:02:41.871476+0000","last_fullsized":"2024-09-10T21:02:41.871476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:45.798350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509907+0000","last_change":"2024-09-10T21:02:00.804200+0000","last_active":"2024-09-10T21:03:40.509907+0000","last_peered":"2024-09-10T21:03:40.509907+0000","last_clean":"2024-09-10T21:03:40.509907+0000","last_became_active":"2024-09-10T21:02:00.803875+0000","last_became_peered":"2024-09-10T21:02:00.803875+0000","last_unstale":"2024-09-10T21:03:40.509907+0000","last_undegraded":"2024-09-10T21:03:40.509907+0000","last_fullsized":"2024-09-10T21:03:40.509907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:15.930163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.507079+0000","last_change":"2024-09-10T21:02:41.874580+0000","last_active":"2024-09-10T21:03:40.507079+0000","last_peered":"2024-09-10T21:03:40.507079+0000","last_clean":"2024-09-10T21:03:40.507079+0000","last_became_active":"2024-09-10T21:02:41.873545+0000","last_became_peered":"2024-09-10T21:02:41.873545+0000","last_unstale":"2024-09-10T21:03:40.507079+0000","last_undegraded":"2024-09-10T21:03:40.507079+0000","last_fullsized":"2024-09-10T21:03:40.507079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:05.278855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870983+0000","last_change":"2024-09-10T21:01:58.784834+0000","last_active":"2024-09-10T21:02:41.870983+0000","last_peered":"2024-09-10T21:02:41.870983+0000","last_clean":"2024-09-10T21:02:41.870983+0000","last_became_active":"2024-09-10T21:01:58.784464+0000","last_became_peered":"2024-09-10T21:01:58.784464+0000","last_unstale":"2024-09-10T21:02:41.870983+0000","last_undegraded":"2024-09-10T21:02:41.870983+0000","last_fullsized":"2024-09-10T21:02:41.870983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:09.864824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510409+0000","last_change":"2024-09-10T21:02:00.805677+0000","last_active":"2024-09-10T21:03:40.510409+0000","last_peered":"2024-09-10T21:03:40.510409+0000","last_clean":"2024-09-10T21:03:40.510409+0000","last_became_active":"2024-09-10T21:02:00.805341+0000","last_became_peered":"2024-09-10T21:02:00.805341+0000","last_unstale":"2024-09-10T21:03:40.510409+0000","last_undegraded":"2024-09-10T21:03:40.510409+0000","last_fullsized":"2024-09-10T21:03:40.510409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.236954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.889813+0000","last_change":"2024-09-10T21:02:41.889336+0000","last_active":"2024-09-10T21:02:41.889813+0000","last_peered":"2024-09-10T21:02:41.889813+0000","last_clean":"2024-09-10T21:02:41.889813+0000","last_became_active":"2024-09-10T21:02:41.874164+0000","last_became_peered":"2024-09-10T21:02:41.874164+0000","last_unstale":"2024-09-10T21:02:41.889813+0000","last_undegraded":"2024-09-10T21:02:41.889813+0000","last_fullsized":"2024-09-10T21:02:41.889813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.799546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504133+0000","last_change":"2024-09-10T21:02:00.801514+0000","last_active":"2024-09-10T21:03:40.504133+0000","last_peered":"2024-09-10T21:03:40.504133+0000","last_clean":"2024-09-10T21:03:40.504133+0000","last_became_active":"2024-09-10T21:02:00.801221+0000","last_became_peered":"2024-09-10T21:02:00.801221+0000","last_unstale":"2024-09-10T21:03:40.504133+0000","last_undegraded":"2024-09-10T21:03:40.504133+0000","last_fullsized":"2024-09-10T21:03:40.504133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:35.564890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978209+0000","last_change":"2024-09-10T21:01:58.884876+0000","last_active":"2024-09-10T21:02:41.978209+0000","last_peered":"2024-09-10T21:02:41.978209+0000","last_clean":"2024-09-10T21:02:41.978209+0000","last_became_active":"2024-09-10T21:01:58.884530+0000","last_became_peered":"2024-09-10T21:01:58.884530+0000","last_unstale":"2024-09-10T21:02:41.978209+0000","last_undegraded":"2024-09-10T21:02:41.978209+0000","last_fullsized":"2024-09-10T21:02:41.978209+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.851017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.479680+0000","last_change":"2024-09-10T21:02:00.811148+0000","last_active":"2024-09-10T21:03:40.479680+0000","last_peered":"2024-09-10T21:03:40.479680+0000","last_clean":"2024-09-10T21:03:40.479680+0000","last_became_active":"2024-09-10T21:02:00.809265+0000","last_became_peered":"2024-09-10T21:02:00.809265+0000","last_unstale":"2024-09-10T21:03:40.479680+0000","last_undegraded":"2024-09-10T21:03:40.479680+0000","last_fullsized":"2024-09-10T21:03:40.479680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:25.269876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":143,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.011599+0000","last_change":"2024-09-10T21:01:58.786917+0000","last_active":"2024-09-10T21:03:56.011599+0000","last_peered":"2024-09-10T21:03:56.011599+0000","last_clean":"2024-09-10T21:03:56.011599+0000","last_became_active":"2024-09-10T21:01:58.786595+0000","last_became_peered":"2024-09-10T21:01:58.786595+0000","last_unstale":"2024-09-10T21:03:56.011599+0000","last_undegraded":"2024-09-10T21:03:56.011599+0000","last_fullsized":"2024-09-10T21:03:56.011599+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:30.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74899,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1095,"num_read_kb":631,"num_write":267,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1237,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":224697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":407671,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1064937,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":406957,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":407223,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495671,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":1001366,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":847872,"data_stored":407206,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855427,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":999672,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":408529,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":406499,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":473646,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":474272,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":408545,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:04:00.986 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-10T21:04:00.986 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:04:01.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.122+0000 7f27370d6640 1 -- 172.21.15.33:0/1625366080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 msgr2=0x7f2730109d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.122+0000 7f27370d6640 1 --2- 172.21.15.33:0/1625366080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f2730109d40 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2728009cd0 tx=0x7f272802a8e0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.122+0000 7f27370d6640 1 -- 172.21.15.33:0/1625366080 shutdown_connections 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.122+0000 7f27370d6640 1 --2- 172.21.15.33:0/1625366080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f2730109d40 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.122+0000 7f27370d6640 1 --2- 172.21.15.33:0/1625366080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2730102a00 0x7f2730102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 --2- 172.21.15.33:0/1625366080 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27301018b0 0x7f2730101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 -- 172.21.15.33:0/1625366080 >> 172.21.15.33:0/1625366080 conn(0x7f27300fd190 msgr2=0x7f27300ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 -- 172.21.15.33:0/1625366080 shutdown_connections 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 -- 172.21.15.33:0/1625366080 wait complete. 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 Processor -- start 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 -- start start 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27301018b0 0x7f273006bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2730102a00 0x7f273019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:01.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f27301a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d520 con 0x7f27301033c0 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d3a0 con 0x7f2730102a00 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f27370d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f273010d6a0 con 0x7f27301018b0 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f2735673640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f27301a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.123+0000 7f2735673640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f27301a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52766/0 (socket says 172.21.15.33:52766) 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2735673640 1 -- 172.21.15.33:0/2489741129 learned_addr learned my addr 172.21.15.33:0/2489741129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2734e72640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27301018b0 0x7f273006bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f272ffff640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2730102a00 0x7f273019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2735673640 1 -- 172.21.15.33:0/2489741129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27301018b0 msgr2=0x7f273006bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2735673640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27301018b0 0x7f273006bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2735673640 1 -- 172.21.15.33:0/2489741129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2730102a00 msgr2=0x7f273019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:01.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2735673640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2730102a00 0x7f273019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2735673640 1 -- 172.21.15.33:0/2489741129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27301a0930 con 0x7f27301033c0 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f272ffff640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2730102a00 0x7f273019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f2735673640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f27301a0070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2728033040 tx=0x7f2728003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f272dffb640 1 -- 172.21.15.33:0/2489741129 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728004450 con 0x7f27301033c0 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f272dffb640 1 -- 172.21.15.33:0/2489741129 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2728003e10 con 0x7f27301033c0 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27301a0bc0 con 0x7f27301033c0 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f272dffb640 1 -- 172.21.15.33:0/2489741129 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272803b400 con 0x7f27301033c0 2024-09-10T21:04:01.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.124+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27301a1160 con 0x7f27301033c0 2024-09-10T21:04:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.125+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2730108c20 con 0x7f27301033c0 2024-09-10T21:04:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.127+0000 7f272dffb640 1 -- 172.21.15.33:0/2489741129 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f272804d020 con 0x7f27301033c0 2024-09-10T21:04:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.127+0000 7f272dffb640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f27200b5360 0x7f27200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.128+0000 7f2734e72640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f27200b5360 0x7f27200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.128+0000 7f272dffb640 1 -- 172.21.15.33:0/2489741129 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f272802e030 con 0x7f27301033c0 2024-09-10T21:04:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.128+0000 7f2734e72640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f27200b5360 0x7f27200b7820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f27240058c0 tx=0x7f272400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.131+0000 7f272dffb640 1 -- 172.21.15.33:0/2489741129 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f27280cd8a0 con 0x7f27301033c0 2024-09-10T21:04:01.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.316+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 --> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2730063590 con 0x7f27200b5360 2024-09-10T21:04:01.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.320+0000 7f272dffb640 1 -- 172.21.15.33:0/2489741129 <== mgr.14148 v2:172.21.15.33:6800/3102318098 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259138 (secure 0 0 0) 0x7f2730063590 con 0x7f27200b5360 2024-09-10T21:04:01.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:04:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-09-10T21:04:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f27200b5360 msgr2=0x7f27200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f27200b5360 0x7f27200b7820 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f27240058c0 tx=0x7f272400a2d0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 msgr2=0x7f27301a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f27301a0070 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2728033040 tx=0x7f2728003c20 comp rx=0 tx=0).stop 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 shutdown_connections 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f27200b5360 0x7f27200b7820 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f27301033c0 0x7f27301a0070 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2730102a00 0x7f273019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 --2- 172.21.15.33:0/2489741129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27301018b0 0x7f273006bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 >> 172.21.15.33:0/2489741129 conn(0x7f27300fd190 msgr2=0x7f2730105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 shutdown_connections 2024-09-10T21:04:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:01.324+0000 7f27370d6640 1 -- 172.21.15.33:0/2489741129 wait complete. 2024-09-10T21:04:01.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:01 smithi033 ceph-mon[30860]: pgmap v347: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:04:01.338 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:01 smithi033 ceph-mon[30860]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:01.339 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-10T21:04:00.113197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":665501,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1171,"num_read_kb":695,"num_write":392,"num_write_kb":2311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30,"num_bytes_recovered":2953077,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":824,"ondisk_log_size":824,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":355516,"kb_used_data":10428,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124586308,"statfs":{"total":1151940427776,"available":1151576379392,"internally_reserved":0,"allocated":10678272,"data_stored":6866523,"data_compressed":176985,"data_compressed_allocated":233472,"data_compressed_original":1966080,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4930,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":118,"num_read_kb":62,"num_write":9,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003095"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498746+0000","last_change":"2024-09-10T21:02:00.805421+0000","last_active":"2024-09-10T21:03:40.498746+0000","last_peered":"2024-09-10T21:03:40.498746+0000","last_clean":"2024-09-10T21:03:40.498746+0000","last_became_active":"2024-09-10T21:02:00.804274+0000","last_became_peered":"2024-09-10T21:02:00.804274+0000","last_unstale":"2024-09-10T21:03:40.498746+0000","last_undegraded":"2024-09-10T21:03:40.498746+0000","last_fullsized":"2024-09-10T21:03:40.498746+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:12.019998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.497068+0000","last_change":"2024-09-10T21:02:00.806104+0000","last_active":"2024-09-10T21:03:40.497068+0000","last_peered":"2024-09-10T21:03:40.497068+0000","last_clean":"2024-09-10T21:03:40.497068+0000","last_became_active":"2024-09-10T21:02:00.805510+0000","last_became_peered":"2024-09-10T21:02:00.805510+0000","last_unstale":"2024-09-10T21:03:40.497068+0000","last_undegraded":"2024-09-10T21:03:40.497068+0000","last_fullsized":"2024-09-10T21:03:40.497068+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:37.596398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503426+0000","last_change":"2024-09-10T21:02:00.807691+0000","last_active":"2024-09-10T21:03:40.503426+0000","last_peered":"2024-09-10T21:03:40.503426+0000","last_clean":"2024-09-10T21:03:40.503426+0000","last_became_active":"2024-09-10T21:02:00.807364+0000","last_became_peered":"2024-09-10T21:02:00.807364+0000","last_unstale":"2024-09-10T21:03:40.503426+0000","last_undegraded":"2024-09-10T21:03:40.503426+0000","last_fullsized":"2024-09-10T21:03:40.503426+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:27.323317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499586+0000","last_change":"2024-09-10T21:02:00.809542+0000","last_active":"2024-09-10T21:03:40.499586+0000","last_peered":"2024-09-10T21:03:40.499586+0000","last_clean":"2024-09-10T21:03:40.499586+0000","last_became_active":"2024-09-10T21:02:00.809034+0000","last_became_peered":"2024-09-10T21:02:00.809034+0000","last_unstale":"2024-09-10T21:03:40.499586+0000","last_undegraded":"2024-09-10T21:03:40.499586+0000","last_fullsized":"2024-09-10T21:03:40.499586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:29.923532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491227+0000","last_change":"2024-09-10T21:02:00.809355+0000","last_active":"2024-09-10T21:03:40.491227+0000","last_peered":"2024-09-10T21:03:40.491227+0000","last_clean":"2024-09-10T21:03:40.491227+0000","last_became_active":"2024-09-10T21:02:00.808863+0000","last_became_peered":"2024-09-10T21:02:00.808863+0000","last_unstale":"2024-09-10T21:03:40.491227+0000","last_undegraded":"2024-09-10T21:03:40.491227+0000","last_fullsized":"2024-09-10T21:03:40.491227+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:29.619004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502481+0000","last_change":"2024-09-10T21:02:00.803756+0000","last_active":"2024-09-10T21:03:40.502481+0000","last_peered":"2024-09-10T21:03:40.502481+0000","last_clean":"2024-09-10T21:03:40.502481+0000","last_became_active":"2024-09-10T21:02:00.803406+0000","last_became_peered":"2024-09-10T21:02:00.803406+0000","last_unstale":"2024-09-10T21:03:40.502481+0000","last_undegraded":"2024-09-10T21:03:40.502481+0000","last_fullsized":"2024-09-10T21:03:40.502481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:18:53.437629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504748+0000","last_change":"2024-09-10T21:02:00.808845+0000","last_active":"2024-09-10T21:03:40.504748+0000","last_peered":"2024-09-10T21:03:40.504748+0000","last_clean":"2024-09-10T21:03:40.504748+0000","last_became_active":"2024-09-10T21:02:00.807759+0000","last_became_peered":"2024-09-10T21:02:00.807759+0000","last_unstale":"2024-09-10T21:03:40.504748+0000","last_undegraded":"2024-09-10T21:03:40.504748+0000","last_fullsized":"2024-09-10T21:03:40.504748+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.809389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.487348+0000","last_change":"2024-09-10T21:02:00.811199+0000","last_active":"2024-09-10T21:03:40.487348+0000","last_peered":"2024-09-10T21:03:40.487348+0000","last_clean":"2024-09-10T21:03:40.487348+0000","last_became_active":"2024-09-10T21:02:00.810692+0000","last_became_peered":"2024-09-10T21:02:00.810692+0000","last_unstale":"2024-09-10T21:03:40.487348+0000","last_undegraded":"2024-09-10T21:03:40.487348+0000","last_fullsized":"2024-09-10T21:03:40.487348+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:16:59.557973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509702+0000","last_change":"2024-09-10T21:02:00.812686+0000","last_active":"2024-09-10T21:03:40.509702+0000","last_peered":"2024-09-10T21:03:40.509702+0000","last_clean":"2024-09-10T21:03:40.509702+0000","last_became_active":"2024-09-10T21:02:00.812363+0000","last_became_peered":"2024-09-10T21:02:00.812363+0000","last_unstale":"2024-09-10T21:03:40.509702+0000","last_undegraded":"2024-09-10T21:03:40.509702+0000","last_fullsized":"2024-09-10T21:03:40.509702+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:37.090301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503735+0000","last_change":"2024-09-10T21:02:00.812058+0000","last_active":"2024-09-10T21:03:40.503735+0000","last_peered":"2024-09-10T21:03:40.503735+0000","last_clean":"2024-09-10T21:03:40.503735+0000","last_became_active":"2024-09-10T21:02:00.811490+0000","last_became_peered":"2024-09-10T21:02:00.811490+0000","last_unstale":"2024-09-10T21:03:40.503735+0000","last_undegraded":"2024-09-10T21:03:40.503735+0000","last_fullsized":"2024-09-10T21:03:40.503735+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:31.557485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.507421+0000","last_change":"2024-09-10T21:02:00.809991+0000","last_active":"2024-09-10T21:03:40.507421+0000","last_peered":"2024-09-10T21:03:40.507421+0000","last_clean":"2024-09-10T21:03:40.507421+0000","last_became_active":"2024-09-10T21:02:00.809197+0000","last_became_peered":"2024-09-10T21:02:00.809197+0000","last_unstale":"2024-09-10T21:03:40.507421+0000","last_undegraded":"2024-09-10T21:03:40.507421+0000","last_fullsized":"2024-09-10T21:03:40.507421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.416692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490787+0000","last_change":"2024-09-10T21:02:41.918484+0000","last_active":"2024-09-10T21:03:40.490787+0000","last_peered":"2024-09-10T21:03:40.490787+0000","last_clean":"2024-09-10T21:03:40.490787+0000","last_became_active":"2024-09-10T21:02:41.918068+0000","last_became_peered":"2024-09-10T21:02:41.918068+0000","last_unstale":"2024-09-10T21:03:40.490787+0000","last_undegraded":"2024-09-10T21:03:40.490787+0000","last_fullsized":"2024-09-10T21:03:40.490787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:08.954072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.487079+0000","last_change":"2024-09-10T21:02:00.807280+0000","last_active":"2024-09-10T21:03:40.487079+0000","last_peered":"2024-09-10T21:03:40.487079+0000","last_clean":"2024-09-10T21:03:40.487079+0000","last_became_active":"2024-09-10T21:02:00.804247+0000","last_became_peered":"2024-09-10T21:02:00.804247+0000","last_unstale":"2024-09-10T21:03:40.487079+0000","last_undegraded":"2024-09-10T21:03:40.487079+0000","last_fullsized":"2024-09-10T21:03:40.487079+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:28.163960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498116+0000","last_change":"2024-09-10T21:02:00.805723+0000","last_active":"2024-09-10T21:03:40.498116+0000","last_peered":"2024-09-10T21:03:40.498116+0000","last_clean":"2024-09-10T21:03:40.498116+0000","last_became_active":"2024-09-10T21:02:00.805104+0000","last_became_peered":"2024-09-10T21:02:00.805104+0000","last_unstale":"2024-09-10T21:03:40.498116+0000","last_undegraded":"2024-09-10T21:03:40.498116+0000","last_fullsized":"2024-09-10T21:03:40.498116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:19:04.056884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500888+0000","last_change":"2024-09-10T21:02:00.810497+0000","last_active":"2024-09-10T21:03:40.500888+0000","last_peered":"2024-09-10T21:03:40.500888+0000","last_clean":"2024-09-10T21:03:40.500888+0000","last_became_active":"2024-09-10T21:02:00.809392+0000","last_became_peered":"2024-09-10T21:02:00.809392+0000","last_unstale":"2024-09-10T21:03:40.500888+0000","last_undegraded":"2024-09-10T21:03:40.500888+0000","last_fullsized":"2024-09-10T21:03:40.500888+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:41:51.214771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509646+0000","last_change":"2024-09-10T21:02:00.813973+0000","last_active":"2024-09-10T21:03:40.509646+0000","last_peered":"2024-09-10T21:03:40.509646+0000","last_clean":"2024-09-10T21:03:40.509646+0000","last_became_active":"2024-09-10T21:02:00.813580+0000","last_became_peered":"2024-09-10T21:02:00.813580+0000","last_unstale":"2024-09-10T21:03:40.509646+0000","last_undegraded":"2024-09-10T21:03:40.509646+0000","last_fullsized":"2024-09-10T21:03:40.509646+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:48.263556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498592+0000","last_change":"2024-09-10T21:02:00.804844+0000","last_active":"2024-09-10T21:03:40.498592+0000","last_peered":"2024-09-10T21:03:40.498592+0000","last_clean":"2024-09-10T21:03:40.498592+0000","last_became_active":"2024-09-10T21:02:00.803866+0000","last_became_peered":"2024-09-10T21:02:00.803866+0000","last_unstale":"2024-09-10T21:03:40.498592+0000","last_undegraded":"2024-09-10T21:03:40.498592+0000","last_fullsized":"2024-09-10T21:03:40.498592+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:47:05.231877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.506529+0000","last_change":"2024-09-10T21:02:00.801343+0000","last_active":"2024-09-10T21:03:40.506529+0000","last_peered":"2024-09-10T21:03:40.506529+0000","last_clean":"2024-09-10T21:03:40.506529+0000","last_became_active":"2024-09-10T21:02:00.801008+0000","last_became_peered":"2024-09-10T21:02:00.801008+0000","last_unstale":"2024-09-10T21:03:40.506529+0000","last_undegraded":"2024-09-10T21:03:40.506529+0000","last_fullsized":"2024-09-10T21:03:40.506529+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:36:31.167925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498608+0000","last_change":"2024-09-10T21:02:41.917997+0000","last_active":"2024-09-10T21:03:40.498608+0000","last_peered":"2024-09-10T21:03:40.498608+0000","last_clean":"2024-09-10T21:03:40.498608+0000","last_became_active":"2024-09-10T21:02:41.917621+0000","last_became_peered":"2024-09-10T21:02:41.917621+0000","last_unstale":"2024-09-10T21:03:40.498608+0000","last_undegraded":"2024-09-10T21:03:40.498608+0000","last_fullsized":"2024-09-10T21:03:40.498608+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:12.742475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,6],"acting":[2,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491709+0000","last_change":"2024-09-10T21:02:00.810502+0000","last_active":"2024-09-10T21:03:40.491709+0000","last_peered":"2024-09-10T21:03:40.491709+0000","last_clean":"2024-09-10T21:03:40.491709+0000","last_became_active":"2024-09-10T21:02:00.810209+0000","last_became_peered":"2024-09-10T21:02:00.810209+0000","last_unstale":"2024-09-10T21:03:40.491709+0000","last_undegraded":"2024-09-10T21:03:40.491709+0000","last_fullsized":"2024-09-10T21:03:40.491709+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:51.311272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502162+0000","last_change":"2024-09-10T21:02:00.797119+0000","last_active":"2024-09-10T21:03:40.502162+0000","last_peered":"2024-09-10T21:03:40.502162+0000","last_clean":"2024-09-10T21:03:40.502162+0000","last_became_active":"2024-09-10T21:02:00.796788+0000","last_became_peered":"2024-09-10T21:02:00.796788+0000","last_unstale":"2024-09-10T21:03:40.502162+0000","last_undegraded":"2024-09-10T21:03:40.502162+0000","last_fullsized":"2024-09-10T21:03:40.502162+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:07.915878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510423+0000","last_change":"2024-09-10T21:02:00.802284+0000","last_active":"2024-09-10T21:03:40.510423+0000","last_peered":"2024-09-10T21:03:40.510423+0000","last_clean":"2024-09-10T21:03:40.510423+0000","last_became_active":"2024-09-10T21:02:00.801964+0000","last_became_peered":"2024-09-10T21:02:00.801964+0000","last_unstale":"2024-09-10T21:03:40.510423+0000","last_undegraded":"2024-09-10T21:03:40.510423+0000","last_fullsized":"2024-09-10T21:03:40.510423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:45.064158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490488+0000","last_change":"2024-09-10T21:02:00.809196+0000","last_active":"2024-09-10T21:03:40.490488+0000","last_peered":"2024-09-10T21:03:40.490488+0000","last_clean":"2024-09-10T21:03:40.490488+0000","last_became_active":"2024-09-10T21:02:00.808903+0000","last_became_peered":"2024-09-10T21:02:00.808903+0000","last_unstale":"2024-09-10T21:03:40.490488+0000","last_undegraded":"2024-09-10T21:03:40.490488+0000","last_fullsized":"2024-09-10T21:03:40.490488+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:01.601660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.488110+0000","last_change":"2024-09-10T21:02:00.808195+0000","last_active":"2024-09-10T21:03:40.488110+0000","last_peered":"2024-09-10T21:03:40.488110+0000","last_clean":"2024-09-10T21:03:40.488110+0000","last_became_active":"2024-09-10T21:02:00.807670+0000","last_became_peered":"2024-09-10T21:02:00.807670+0000","last_unstale":"2024-09-10T21:03:40.488110+0000","last_undegraded":"2024-09-10T21:03:40.488110+0000","last_fullsized":"2024-09-10T21:03:40.488110+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:12.549344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.479659+0000","last_change":"2024-09-10T21:02:00.811401+0000","last_active":"2024-09-10T21:03:40.479659+0000","last_peered":"2024-09-10T21:03:40.479659+0000","last_clean":"2024-09-10T21:03:40.479659+0000","last_became_active":"2024-09-10T21:02:00.811055+0000","last_became_peered":"2024-09-10T21:02:00.811055+0000","last_unstale":"2024-09-10T21:03:40.479659+0000","last_undegraded":"2024-09-10T21:03:40.479659+0000","last_fullsized":"2024-09-10T21:03:40.479659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:58.641873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510083+0000","last_change":"2024-09-10T21:02:00.813796+0000","last_active":"2024-09-10T21:03:40.510083+0000","last_peered":"2024-09-10T21:03:40.510083+0000","last_clean":"2024-09-10T21:03:40.510083+0000","last_became_active":"2024-09-10T21:02:00.813468+0000","last_became_peered":"2024-09-10T21:02:00.813468+0000","last_unstale":"2024-09-10T21:03:40.510083+0000","last_undegraded":"2024-09-10T21:03:40.510083+0000","last_fullsized":"2024-09-10T21:03:40.510083+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:25.964890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.488390+0000","last_change":"2024-09-10T21:02:00.811436+0000","last_active":"2024-09-10T21:03:40.488390+0000","last_peered":"2024-09-10T21:03:40.488390+0000","last_clean":"2024-09-10T21:03:40.488390+0000","last_became_active":"2024-09-10T21:02:00.810956+0000","last_became_peered":"2024-09-10T21:02:00.810956+0000","last_unstale":"2024-09-10T21:03:40.488390+0000","last_undegraded":"2024-09-10T21:03:40.488390+0000","last_fullsized":"2024-09-10T21:03:40.488390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:47.306832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":22,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.486792+0000","last_change":"2024-09-10T21:02:00.799370+0000","last_active":"2024-09-10T21:03:40.486792+0000","last_peered":"2024-09-10T21:03:40.486792+0000","last_clean":"2024-09-10T21:03:40.486792+0000","last_became_active":"2024-09-10T21:02:00.799044+0000","last_became_peered":"2024-09-10T21:02:00.799044+0000","last_unstale":"2024-09-10T21:03:40.486792+0000","last_undegraded":"2024-09-10T21:03:40.486792+0000","last_fullsized":"2024-09-10T21:03:40.486792+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:40.332088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498868+0000","last_change":"2024-09-10T21:02:00.799830+0000","last_active":"2024-09-10T21:03:40.498868+0000","last_peered":"2024-09-10T21:03:40.498868+0000","last_clean":"2024-09-10T21:03:40.498868+0000","last_became_active":"2024-09-10T21:02:00.799533+0000","last_became_peered":"2024-09-10T21:02:00.799533+0000","last_unstale":"2024-09-10T21:03:40.498868+0000","last_undegraded":"2024-09-10T21:03:40.498868+0000","last_fullsized":"2024-09-10T21:03:40.498868+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:23.253299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499851+0000","last_change":"2024-09-10T21:02:00.812739+0000","last_active":"2024-09-10T21:03:40.499851+0000","last_peered":"2024-09-10T21:03:40.499851+0000","last_clean":"2024-09-10T21:03:40.499851+0000","last_became_active":"2024-09-10T21:02:00.812395+0000","last_became_peered":"2024-09-10T21:02:00.812395+0000","last_unstale":"2024-09-10T21:03:40.499851+0000","last_undegraded":"2024-09-10T21:03:40.499851+0000","last_fullsized":"2024-09-10T21:03:40.499851+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:55.727155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503421+0000","last_change":"2024-09-10T21:02:00.806291+0000","last_active":"2024-09-10T21:03:40.503421+0000","last_peered":"2024-09-10T21:03:40.503421+0000","last_clean":"2024-09-10T21:03:40.503421+0000","last_became_active":"2024-09-10T21:02:00.805943+0000","last_became_peered":"2024-09-10T21:02:00.805943+0000","last_unstale":"2024-09-10T21:03:40.503421+0000","last_undegraded":"2024-09-10T21:03:40.503421+0000","last_fullsized":"2024-09-10T21:03:40.503421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:21.354105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500561+0000","last_change":"2024-09-10T21:02:00.808774+0000","last_active":"2024-09-10T21:03:40.500561+0000","last_peered":"2024-09-10T21:03:40.500561+0000","last_clean":"2024-09-10T21:03:40.500561+0000","last_became_active":"2024-09-10T21:02:00.808027+0000","last_became_peered":"2024-09-10T21:02:00.808027+0000","last_unstale":"2024-09-10T21:03:40.500561+0000","last_undegraded":"2024-09-10T21:03:40.500561+0000","last_fullsized":"2024-09-10T21:03:40.500561+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:19.403684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.486316+0000","last_change":"2024-09-10T21:02:00.808349+0000","last_active":"2024-09-10T21:03:40.486316+0000","last_peered":"2024-09-10T21:03:40.486316+0000","last_clean":"2024-09-10T21:03:40.486316+0000","last_became_active":"2024-09-10T21:02:00.807839+0000","last_became_peered":"2024-09-10T21:02:00.807839+0000","last_unstale":"2024-09-10T21:03:40.486316+0000","last_undegraded":"2024-09-10T21:03:40.486316+0000","last_fullsized":"2024-09-10T21:03:40.486316+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:53.894070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"77'7","reported_seq":151,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:55.977242+0000","last_change":"2024-09-10T21:02:41.877376+0000","last_active":"2024-09-10T21:03:55.977242+0000","last_peered":"2024-09-10T21:03:55.977242+0000","last_clean":"2024-09-10T21:03:55.977242+0000","last_became_active":"2024-09-10T21:02:41.871397+0000","last_became_peered":"2024-09-10T21:02:41.871397+0000","last_unstale":"2024-09-10T21:03:55.977242+0000","last_undegraded":"2024-09-10T21:03:55.977242+0000","last_fullsized":"2024-09-10T21:03:55.977242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:59.961597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1013,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.489060+0000","last_change":"2024-09-10T21:02:41.979196+0000","last_active":"2024-09-10T21:03:40.489060+0000","last_peered":"2024-09-10T21:03:40.489060+0000","last_clean":"2024-09-10T21:03:40.489060+0000","last_became_active":"2024-09-10T21:02:41.978781+0000","last_became_peered":"2024-09-10T21:02:41.978781+0000","last_unstale":"2024-09-10T21:03:40.489060+0000","last_undegraded":"2024-09-10T21:03:40.489060+0000","last_fullsized":"2024-09-10T21:03:40.489060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:15.100055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.917027+0000","last_change":"2024-09-10T21:01:58.787619+0000","last_active":"2024-09-10T21:02:41.917027+0000","last_peered":"2024-09-10T21:02:41.917027+0000","last_clean":"2024-09-10T21:02:41.917027+0000","last_became_active":"2024-09-10T21:01:58.787138+0000","last_became_peered":"2024-09-10T21:01:58.787138+0000","last_unstale":"2024-09-10T21:02:41.917027+0000","last_undegraded":"2024-09-10T21:02:41.917027+0000","last_fullsized":"2024-09-10T21:02:41.917027+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:33.592458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916753+0000","last_change":"2024-09-10T21:01:58.780115+0000","last_active":"2024-09-10T21:02:41.916753+0000","last_peered":"2024-09-10T21:02:41.916753+0000","last_clean":"2024-09-10T21:02:41.916753+0000","last_became_active":"2024-09-10T21:01:58.779757+0000","last_became_peered":"2024-09-10T21:01:58.779757+0000","last_unstale":"2024-09-10T21:02:41.916753+0000","last_undegraded":"2024-09-10T21:02:41.916753+0000","last_fullsized":"2024-09-10T21:02:41.916753+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:29.337978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491520+0000","last_change":"2024-09-10T21:02:00.800925+0000","last_active":"2024-09-10T21:03:40.491520+0000","last_peered":"2024-09-10T21:03:40.491520+0000","last_clean":"2024-09-10T21:03:40.491520+0000","last_became_active":"2024-09-10T21:02:00.798567+0000","last_became_peered":"2024-09-10T21:02:00.798567+0000","last_unstale":"2024-09-10T21:03:40.491520+0000","last_undegraded":"2024-09-10T21:03:40.491520+0000","last_fullsized":"2024-09-10T21:03:40.491520+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:30:01.873716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867127+0000","last_change":"2024-09-10T21:01:58.786040+0000","last_active":"2024-09-10T21:02:41.867127+0000","last_peered":"2024-09-10T21:02:41.867127+0000","last_clean":"2024-09-10T21:02:41.867127+0000","last_became_active":"2024-09-10T21:01:58.785703+0000","last_became_peered":"2024-09-10T21:01:58.785703+0000","last_unstale":"2024-09-10T21:02:41.867127+0000","last_undegraded":"2024-09-10T21:02:41.867127+0000","last_fullsized":"2024-09-10T21:02:41.867127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:39.741614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.497664+0000","last_change":"2024-09-10T21:02:00.798368+0000","last_active":"2024-09-10T21:03:40.497664+0000","last_peered":"2024-09-10T21:03:40.497664+0000","last_clean":"2024-09-10T21:03:40.497664+0000","last_became_active":"2024-09-10T21:02:00.798080+0000","last_became_peered":"2024-09-10T21:02:00.798080+0000","last_unstale":"2024-09-10T21:03:40.497664+0000","last_undegraded":"2024-09-10T21:03:40.497664+0000","last_fullsized":"2024-09-10T21:03:40.497664+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:31.585395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"77'6","reported_seq":238,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.978268+0000","last_change":"2024-09-10T21:01:58.788119+0000","last_active":"2024-09-10T21:03:56.978268+0000","last_peered":"2024-09-10T21:03:56.978268+0000","last_clean":"2024-09-10T21:03:56.978268+0000","last_became_active":"2024-09-10T21:01:58.787724+0000","last_became_peered":"2024-09-10T21:01:58.787724+0000","last_unstale":"2024-09-10T21:03:56.978268+0000","last_undegraded":"2024-09-10T21:03:56.978268+0000","last_fullsized":"2024-09-10T21:03:56.978268+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:42.872858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":209,"num_read_kb":209,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510209+0000","last_change":"2024-09-10T21:02:00.804425+0000","last_active":"2024-09-10T21:03:40.510209+0000","last_peered":"2024-09-10T21:03:40.510209+0000","last_clean":"2024-09-10T21:03:40.510209+0000","last_became_active":"2024-09-10T21:02:00.804093+0000","last_became_peered":"2024-09-10T21:02:00.804093+0000","last_unstale":"2024-09-10T21:03:40.510209+0000","last_undegraded":"2024-09-10T21:03:40.510209+0000","last_fullsized":"2024-09-10T21:03:40.510209+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:46.183326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870381+0000","last_change":"2024-09-10T21:01:58.791323+0000","last_active":"2024-09-10T21:02:41.870381+0000","last_peered":"2024-09-10T21:02:41.870381+0000","last_clean":"2024-09-10T21:02:41.870381+0000","last_became_active":"2024-09-10T21:01:58.790551+0000","last_became_peered":"2024-09-10T21:01:58.790551+0000","last_unstale":"2024-09-10T21:02:41.870381+0000","last_undegraded":"2024-09-10T21:02:41.870381+0000","last_fullsized":"2024-09-10T21:02:41.870381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:18.013259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.499172+0000","last_change":"2024-09-10T21:02:00.804025+0000","last_active":"2024-09-10T21:03:40.499172+0000","last_peered":"2024-09-10T21:03:40.499172+0000","last_clean":"2024-09-10T21:03:40.499172+0000","last_became_active":"2024-09-10T21:02:00.803663+0000","last_became_peered":"2024-09-10T21:02:00.803663+0000","last_unstale":"2024-09-10T21:03:40.499172+0000","last_undegraded":"2024-09-10T21:03:40.499172+0000","last_fullsized":"2024-09-10T21:03:40.499172+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:18:30.609020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868035+0000","last_change":"2024-09-10T21:01:58.887067+0000","last_active":"2024-09-10T21:02:41.868035+0000","last_peered":"2024-09-10T21:02:41.868035+0000","last_clean":"2024-09-10T21:02:41.868035+0000","last_became_active":"2024-09-10T21:01:58.886611+0000","last_became_peered":"2024-09-10T21:01:58.886611+0000","last_unstale":"2024-09-10T21:02:41.868035+0000","last_undegraded":"2024-09-10T21:02:41.868035+0000","last_fullsized":"2024-09-10T21:02:41.868035+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:06.154001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.508877+0000","last_change":"2024-09-10T21:02:41.917998+0000","last_active":"2024-09-10T21:03:40.508877+0000","last_peered":"2024-09-10T21:03:40.508877+0000","last_clean":"2024-09-10T21:03:40.508877+0000","last_became_active":"2024-09-10T21:02:41.917665+0000","last_became_peered":"2024-09-10T21:02:41.917665+0000","last_unstale":"2024-09-10T21:03:40.508877+0000","last_undegraded":"2024-09-10T21:03:40.508877+0000","last_fullsized":"2024-09-10T21:03:40.508877+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:15.147379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'37","reported_seq":280,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.978810+0000","last_change":"2024-09-10T21:01:58.790966+0000","last_active":"2024-09-10T21:03:56.978810+0000","last_peered":"2024-09-10T21:03:56.978810+0000","last_clean":"2024-09-10T21:03:56.978810+0000","last_became_active":"2024-09-10T21:01:58.790636+0000","last_became_peered":"2024-09-10T21:01:58.790636+0000","last_unstale":"2024-09-10T21:03:56.978810+0000","last_undegraded":"2024-09-10T21:03:56.978810+0000","last_fullsized":"2024-09-10T21:03:56.978810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:06.209074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66041,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":220,"num_read_kb":174,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.501096+0000","last_change":"2024-09-10T21:02:00.810610+0000","last_active":"2024-09-10T21:03:40.501096+0000","last_peered":"2024-09-10T21:03:40.501096+0000","last_clean":"2024-09-10T21:03:40.501096+0000","last_became_active":"2024-09-10T21:02:00.810224+0000","last_became_peered":"2024-09-10T21:02:00.810224+0000","last_unstale":"2024-09-10T21:03:40.501096+0000","last_undegraded":"2024-09-10T21:03:40.501096+0000","last_fullsized":"2024-09-10T21:03:40.501096+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:32.785312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'9","reported_seq":34,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870285+0000","last_change":"2024-09-10T21:01:58.786476+0000","last_active":"2024-09-10T21:02:41.870285+0000","last_peered":"2024-09-10T21:02:41.870285+0000","last_clean":"2024-09-10T21:02:41.870285+0000","last_became_active":"2024-09-10T21:01:58.786153+0000","last_became_peered":"2024-09-10T21:01:58.786153+0000","last_unstale":"2024-09-10T21:02:41.870285+0000","last_undegraded":"2024-09-10T21:02:41.870285+0000","last_fullsized":"2024-09-10T21:02:41.870285+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:06:00.118591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.501464+0000","last_change":"2024-09-10T21:02:00.796138+0000","last_active":"2024-09-10T21:03:40.501464+0000","last_peered":"2024-09-10T21:03:40.501464+0000","last_clean":"2024-09-10T21:03:40.501464+0000","last_became_active":"2024-09-10T21:02:00.795847+0000","last_became_peered":"2024-09-10T21:02:00.795847+0000","last_unstale":"2024-09-10T21:03:40.501464+0000","last_undegraded":"2024-09-10T21:03:40.501464+0000","last_fullsized":"2024-09-10T21:03:40.501464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:29.462134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867576+0000","last_change":"2024-09-10T21:01:58.789314+0000","last_active":"2024-09-10T21:02:41.867576+0000","last_peered":"2024-09-10T21:02:41.867576+0000","last_clean":"2024-09-10T21:02:41.867576+0000","last_became_active":"2024-09-10T21:01:58.788594+0000","last_became_peered":"2024-09-10T21:01:58.788594+0000","last_unstale":"2024-09-10T21:02:41.867576+0000","last_undegraded":"2024-09-10T21:02:41.867576+0000","last_fullsized":"2024-09-10T21:02:41.867576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:44.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":33,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.506541+0000","last_change":"2024-09-10T21:02:00.799828+0000","last_active":"2024-09-10T21:03:40.506541+0000","last_peered":"2024-09-10T21:03:40.506541+0000","last_clean":"2024-09-10T21:03:40.506541+0000","last_became_active":"2024-09-10T21:02:00.799465+0000","last_became_peered":"2024-09-10T21:02:00.799465+0000","last_unstale":"2024-09-10T21:03:40.506541+0000","last_undegraded":"2024-09-10T21:03:40.506541+0000","last_fullsized":"2024-09-10T21:03:40.506541+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:56.794326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"77'8","reported_seq":142,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:57.029723+0000","last_change":"2024-09-10T21:01:58.788730+0000","last_active":"2024-09-10T21:03:57.029723+0000","last_peered":"2024-09-10T21:03:57.029723+0000","last_clean":"2024-09-10T21:03:57.029723+0000","last_became_active":"2024-09-10T21:01:58.788317+0000","last_became_peered":"2024-09-10T21:01:58.788317+0000","last_unstale":"2024-09-10T21:03:57.029723+0000","last_undegraded":"2024-09-10T21:03:57.029723+0000","last_fullsized":"2024-09-10T21:03:57.029723+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:52.588745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1428,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504421+0000","last_change":"2024-09-10T21:02:00.807104+0000","last_active":"2024-09-10T21:03:40.504421+0000","last_peered":"2024-09-10T21:03:40.504421+0000","last_clean":"2024-09-10T21:03:40.504421+0000","last_became_active":"2024-09-10T21:02:00.806815+0000","last_became_peered":"2024-09-10T21:02:00.806815+0000","last_unstale":"2024-09-10T21:03:40.504421+0000","last_undegraded":"2024-09-10T21:03:40.504421+0000","last_fullsized":"2024-09-10T21:03:40.504421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:24.747662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'17","reported_seq":66,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.890722+0000","last_change":"2024-09-10T21:02:41.890657+0000","last_active":"2024-09-10T21:02:41.890722+0000","last_peered":"2024-09-10T21:02:41.890722+0000","last_clean":"2024-09-10T21:02:41.890722+0000","last_became_active":"2024-09-10T21:02:41.873778+0000","last_became_peered":"2024-09-10T21:02:41.873778+0000","last_unstale":"2024-09-10T21:02:41.890722+0000","last_undegraded":"2024-09-10T21:02:41.890722+0000","last_fullsized":"2024-09-10T21:02:41.890722+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:38:58.678679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490766+0000","last_change":"2024-09-10T21:02:00.806796+0000","last_active":"2024-09-10T21:03:40.490766+0000","last_peered":"2024-09-10T21:03:40.490766+0000","last_clean":"2024-09-10T21:03:40.490766+0000","last_became_active":"2024-09-10T21:02:00.805802+0000","last_became_peered":"2024-09-10T21:02:00.805802+0000","last_unstale":"2024-09-10T21:03:40.490766+0000","last_undegraded":"2024-09-10T21:03:40.490766+0000","last_fullsized":"2024-09-10T21:03:40.490766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:05.962870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"77'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871699+0000","last_change":"2024-09-10T21:01:58.781088+0000","last_active":"2024-09-10T21:02:41.871699+0000","last_peered":"2024-09-10T21:02:41.871699+0000","last_clean":"2024-09-10T21:02:41.871699+0000","last_became_active":"2024-09-10T21:01:58.780731+0000","last_became_peered":"2024-09-10T21:01:58.780731+0000","last_unstale":"2024-09-10T21:02:41.871699+0000","last_undegraded":"2024-09-10T21:02:41.871699+0000","last_fullsized":"2024-09-10T21:02:41.871699+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:50.636077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491805+0000","last_change":"2024-09-10T21:02:00.806551+0000","last_active":"2024-09-10T21:03:40.491805+0000","last_peered":"2024-09-10T21:03:40.491805+0000","last_clean":"2024-09-10T21:03:40.491805+0000","last_became_active":"2024-09-10T21:02:00.806134+0000","last_became_peered":"2024-09-10T21:02:00.806134+0000","last_unstale":"2024-09-10T21:03:40.491805+0000","last_undegraded":"2024-09-10T21:03:40.491805+0000","last_fullsized":"2024-09-10T21:03:40.491805+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.574372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'5","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.916954+0000","last_change":"2024-09-10T21:01:58.780294+0000","last_active":"2024-09-10T21:02:41.916954+0000","last_peered":"2024-09-10T21:02:41.916954+0000","last_clean":"2024-09-10T21:02:41.916954+0000","last_became_active":"2024-09-10T21:01:58.779767+0000","last_became_peered":"2024-09-10T21:01:58.779767+0000","last_unstale":"2024-09-10T21:02:41.916954+0000","last_undegraded":"2024-09-10T21:02:41.916954+0000","last_fullsized":"2024-09-10T21:02:41.916954+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:21.734159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.503245+0000","last_change":"2024-09-10T21:02:00.809393+0000","last_active":"2024-09-10T21:03:40.503245+0000","last_peered":"2024-09-10T21:03:40.503245+0000","last_clean":"2024-09-10T21:03:40.503245+0000","last_became_active":"2024-09-10T21:02:00.809012+0000","last_became_peered":"2024-09-10T21:02:00.809012+0000","last_unstale":"2024-09-10T21:03:40.503245+0000","last_undegraded":"2024-09-10T21:03:40.503245+0000","last_fullsized":"2024-09-10T21:03:40.503245+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:03.902869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'3","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.868189+0000","last_change":"2024-09-10T21:01:58.786507+0000","last_active":"2024-09-10T21:02:41.868189+0000","last_peered":"2024-09-10T21:02:41.868189+0000","last_clean":"2024-09-10T21:02:41.868189+0000","last_became_active":"2024-09-10T21:01:58.786115+0000","last_became_peered":"2024-09-10T21:01:58.786115+0000","last_unstale":"2024-09-10T21:02:41.868189+0000","last_undegraded":"2024-09-10T21:02:41.868189+0000","last_fullsized":"2024-09-10T21:02:41.868189+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:36.985353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504534+0000","last_change":"2024-09-10T21:02:00.805050+0000","last_active":"2024-09-10T21:03:40.504534+0000","last_peered":"2024-09-10T21:03:40.504534+0000","last_clean":"2024-09-10T21:03:40.504534+0000","last_became_active":"2024-09-10T21:02:00.804706+0000","last_became_peered":"2024-09-10T21:02:00.804706+0000","last_unstale":"2024-09-10T21:03:40.504534+0000","last_undegraded":"2024-09-10T21:03:40.504534+0000","last_fullsized":"2024-09-10T21:03:40.504534+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:15.463986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"77'7","reported_seq":141,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:58.030099+0000","last_change":"2024-09-10T21:01:59.114525+0000","last_active":"2024-09-10T21:03:58.030099+0000","last_peered":"2024-09-10T21:03:58.030099+0000","last_clean":"2024-09-10T21:03:58.030099+0000","last_became_active":"2024-09-10T21:01:59.114076+0000","last_became_peered":"2024-09-10T21:01:59.114076+0000","last_unstale":"2024-09-10T21:03:58.030099+0000","last_undegraded":"2024-09-10T21:03:58.030099+0000","last_fullsized":"2024-09-10T21:03:58.030099+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:23:53.663095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":111,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509798+0000","last_change":"2024-09-10T21:02:00.799803+0000","last_active":"2024-09-10T21:03:40.509798+0000","last_peered":"2024-09-10T21:03:40.509798+0000","last_clean":"2024-09-10T21:03:40.509798+0000","last_became_active":"2024-09-10T21:02:00.799459+0000","last_became_peered":"2024-09-10T21:02:00.799459+0000","last_unstale":"2024-09-10T21:03:40.509798+0000","last_undegraded":"2024-09-10T21:03:40.509798+0000","last_fullsized":"2024-09-10T21:03:40.509798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:57.278606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"20'92","reported_seq":99,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870115+0000","last_change":"2024-09-10T21:00:48.381903+0000","last_active":"2024-09-10T21:02:41.870115+0000","last_peered":"2024-09-10T21:02:41.870115+0000","last_clean":"2024-09-10T21:02:41.870115+0000","last_became_active":"2024-09-10T21:00:48.376874+0000","last_became_peered":"2024-09-10T21:00:48.376874+0000","last_unstale":"2024-09-10T21:02:41.870115+0000","last_undegraded":"2024-09-10T21:02:41.870115+0000","last_fullsized":"2024-09-10T21:02:41.870115+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T20:57:24.516525+0000","last_clean_scrub_stamp":"2024-09-10T20:57:24.516525+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:35.984253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"75'6","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866334+0000","last_change":"2024-09-10T21:01:58.788005+0000","last_active":"2024-09-10T21:02:41.866334+0000","last_peered":"2024-09-10T21:02:41.866334+0000","last_clean":"2024-09-10T21:02:41.866334+0000","last_became_active":"2024-09-10T21:01:58.786471+0000","last_became_peered":"2024-09-10T21:01:58.786471+0000","last_unstale":"2024-09-10T21:02:41.866334+0000","last_undegraded":"2024-09-10T21:02:41.866334+0000","last_fullsized":"2024-09-10T21:02:41.866334+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:15.622814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.505266+0000","last_change":"2024-09-10T21:02:00.802573+0000","last_active":"2024-09-10T21:03:40.505266+0000","last_peered":"2024-09-10T21:03:40.505266+0000","last_clean":"2024-09-10T21:03:40.505266+0000","last_became_active":"2024-09-10T21:02:00.802263+0000","last_became_peered":"2024-09-10T21:02:00.802263+0000","last_unstale":"2024-09-10T21:03:40.505266+0000","last_undegraded":"2024-09-10T21:03:40.505266+0000","last_fullsized":"2024-09-10T21:03:40.505266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:54.897975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"75'4","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870004+0000","last_change":"2024-09-10T21:01:58.785700+0000","last_active":"2024-09-10T21:02:41.870004+0000","last_peered":"2024-09-10T21:02:41.870004+0000","last_clean":"2024-09-10T21:02:41.870004+0000","last_became_active":"2024-09-10T21:01:58.785346+0000","last_became_peered":"2024-09-10T21:01:58.785346+0000","last_unstale":"2024-09-10T21:02:41.870004+0000","last_undegraded":"2024-09-10T21:02:41.870004+0000","last_fullsized":"2024-09-10T21:02:41.870004+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:39.498222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.495838+0000","last_change":"2024-09-10T21:02:00.808045+0000","last_active":"2024-09-10T21:03:40.495838+0000","last_peered":"2024-09-10T21:03:40.495838+0000","last_clean":"2024-09-10T21:03:40.495838+0000","last_became_active":"2024-09-10T21:02:00.807757+0000","last_became_peered":"2024-09-10T21:02:00.807757+0000","last_unstale":"2024-09-10T21:03:40.495838+0000","last_undegraded":"2024-09-10T21:03:40.495838+0000","last_fullsized":"2024-09-10T21:03:40.495838+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:16.198290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"77'7","reported_seq":138,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:58.030473+0000","last_change":"2024-09-10T21:01:58.884939+0000","last_active":"2024-09-10T21:03:58.030473+0000","last_peered":"2024-09-10T21:03:58.030473+0000","last_clean":"2024-09-10T21:03:58.030473+0000","last_became_active":"2024-09-10T21:01:58.884517+0000","last_became_peered":"2024-09-10T21:01:58.884517+0000","last_unstale":"2024-09-10T21:03:58.030473+0000","last_undegraded":"2024-09-10T21:03:58.030473+0000","last_fullsized":"2024-09-10T21:03:58.030473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.579662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":108,"num_read_kb":108,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490615+0000","last_change":"2024-09-10T21:02:00.813021+0000","last_active":"2024-09-10T21:03:40.490615+0000","last_peered":"2024-09-10T21:03:40.490615+0000","last_clean":"2024-09-10T21:03:40.490615+0000","last_became_active":"2024-09-10T21:02:00.812622+0000","last_became_peered":"2024-09-10T21:02:00.812622+0000","last_unstale":"2024-09-10T21:03:40.490615+0000","last_undegraded":"2024-09-10T21:03:40.490615+0000","last_fullsized":"2024-09-10T21:03:40.490615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:48.348100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"77'7","reported_seq":136,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:54.009554+0000","last_change":"2024-09-10T21:01:58.792187+0000","last_active":"2024-09-10T21:03:54.009554+0000","last_peered":"2024-09-10T21:03:54.009554+0000","last_clean":"2024-09-10T21:03:54.009554+0000","last_became_active":"2024-09-10T21:01:58.791806+0000","last_became_peered":"2024-09-10T21:01:58.791806+0000","last_unstale":"2024-09-10T21:03:54.009554+0000","last_undegraded":"2024-09-10T21:03:54.009554+0000","last_fullsized":"2024-09-10T21:03:54.009554+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:48.493396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.502741+0000","last_change":"2024-09-10T21:02:00.804073+0000","last_active":"2024-09-10T21:03:40.502741+0000","last_peered":"2024-09-10T21:03:40.502741+0000","last_clean":"2024-09-10T21:03:40.502741+0000","last_became_active":"2024-09-10T21:02:00.803760+0000","last_became_peered":"2024-09-10T21:02:00.803760+0000","last_unstale":"2024-09-10T21:03:40.502741+0000","last_undegraded":"2024-09-10T21:03:40.502741+0000","last_fullsized":"2024-09-10T21:03:40.502741+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:01.629861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.865323+0000","last_change":"2024-09-10T21:01:58.885088+0000","last_active":"2024-09-10T21:02:41.865323+0000","last_peered":"2024-09-10T21:02:41.865323+0000","last_clean":"2024-09-10T21:02:41.865323+0000","last_became_active":"2024-09-10T21:01:58.884729+0000","last_became_peered":"2024-09-10T21:01:58.884729+0000","last_unstale":"2024-09-10T21:02:41.865323+0000","last_undegraded":"2024-09-10T21:02:41.865323+0000","last_fullsized":"2024-09-10T21:02:41.865323+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:37.451629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.490451+0000","last_change":"2024-09-10T21:02:00.810329+0000","last_active":"2024-09-10T21:03:40.490451+0000","last_peered":"2024-09-10T21:03:40.490451+0000","last_clean":"2024-09-10T21:03:40.490451+0000","last_became_active":"2024-09-10T21:02:00.809985+0000","last_became_peered":"2024-09-10T21:02:00.809985+0000","last_unstale":"2024-09-10T21:03:40.490451+0000","last_undegraded":"2024-09-10T21:03:40.490451+0000","last_fullsized":"2024-09-10T21:03:40.490451+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:39.631969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"75'4","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.880605+0000","last_change":"2024-09-10T21:02:41.880117+0000","last_active":"2024-09-10T21:02:41.880605+0000","last_peered":"2024-09-10T21:02:41.880605+0000","last_clean":"2024-09-10T21:02:41.880605+0000","last_became_active":"2024-09-10T21:02:41.872494+0000","last_became_peered":"2024-09-10T21:02:41.872494+0000","last_unstale":"2024-09-10T21:02:41.880605+0000","last_undegraded":"2024-09-10T21:02:41.880605+0000","last_fullsized":"2024-09-10T21:02:41.880605+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:06.965479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.491585+0000","last_change":"2024-09-10T21:02:00.804961+0000","last_active":"2024-09-10T21:03:40.491585+0000","last_peered":"2024-09-10T21:03:40.491585+0000","last_clean":"2024-09-10T21:03:40.491585+0000","last_became_active":"2024-09-10T21:02:00.804147+0000","last_became_peered":"2024-09-10T21:02:00.804147+0000","last_unstale":"2024-09-10T21:03:40.491585+0000","last_undegraded":"2024-09-10T21:03:40.491585+0000","last_fullsized":"2024-09-10T21:03:40.491585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:15.801815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"77'16","reported_seq":39,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.806420+0000","last_change":"2024-09-10T21:01:58.781292+0000","last_active":"2024-09-10T21:03:40.806420+0000","last_peered":"2024-09-10T21:03:40.806420+0000","last_clean":"2024-09-10T21:03:40.806420+0000","last_became_active":"2024-09-10T21:01:58.780995+0000","last_became_peered":"2024-09-10T21:01:58.780995+0000","last_unstale":"2024-09-10T21:03:40.806420+0000","last_undegraded":"2024-09-10T21:03:40.806420+0000","last_fullsized":"2024-09-10T21:03:40.806420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:56.914723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.493246+0000","last_change":"2024-09-10T21:02:00.805878+0000","last_active":"2024-09-10T21:03:40.493246+0000","last_peered":"2024-09-10T21:03:40.493246+0000","last_clean":"2024-09-10T21:03:40.493246+0000","last_became_active":"2024-09-10T21:02:00.804565+0000","last_became_peered":"2024-09-10T21:02:00.804565+0000","last_unstale":"2024-09-10T21:03:40.493246+0000","last_undegraded":"2024-09-10T21:03:40.493246+0000","last_fullsized":"2024-09-10T21:03:40.493246+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:24.359643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'7","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.866263+0000","last_change":"2024-09-10T21:01:59.114862+0000","last_active":"2024-09-10T21:02:41.866263+0000","last_peered":"2024-09-10T21:02:41.866263+0000","last_clean":"2024-09-10T21:02:41.866263+0000","last_became_active":"2024-09-10T21:01:59.114411+0000","last_became_peered":"2024-09-10T21:01:59.114411+0000","last_unstale":"2024-09-10T21:02:41.866263+0000","last_undegraded":"2024-09-10T21:02:41.866263+0000","last_fullsized":"2024-09-10T21:02:41.866263+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:19:31.645829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.498781+0000","last_change":"2024-09-10T21:02:00.799230+0000","last_active":"2024-09-10T21:03:40.498781+0000","last_peered":"2024-09-10T21:03:40.498781+0000","last_clean":"2024-09-10T21:03:40.498781+0000","last_became_active":"2024-09-10T21:02:00.798992+0000","last_became_peered":"2024-09-10T21:02:00.798992+0000","last_unstale":"2024-09-10T21:03:40.498781+0000","last_undegraded":"2024-09-10T21:03:40.498781+0000","last_fullsized":"2024-09-10T21:03:40.498781+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:28:40.322066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.867967+0000","last_change":"2024-09-10T21:01:58.887042+0000","last_active":"2024-09-10T21:02:41.867967+0000","last_peered":"2024-09-10T21:02:41.867967+0000","last_clean":"2024-09-10T21:02:41.867967+0000","last_became_active":"2024-09-10T21:01:58.886710+0000","last_became_peered":"2024-09-10T21:01:58.886710+0000","last_unstale":"2024-09-10T21:02:41.867967+0000","last_undegraded":"2024-09-10T21:02:41.867967+0000","last_fullsized":"2024-09-10T21:02:41.867967+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:59:42.672899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":20,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.480934+0000","last_change":"2024-09-10T21:02:00.809763+0000","last_active":"2024-09-10T21:03:40.480934+0000","last_peered":"2024-09-10T21:03:40.480934+0000","last_clean":"2024-09-10T21:03:40.480934+0000","last_became_active":"2024-09-10T21:02:00.808955+0000","last_became_peered":"2024-09-10T21:02:00.808955+0000","last_unstale":"2024-09-10T21:03:40.480934+0000","last_undegraded":"2024-09-10T21:03:40.480934+0000","last_fullsized":"2024-09-10T21:03:40.480934+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:22.622852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"75'7","reported_seq":30,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.977719+0000","last_change":"2024-09-10T21:01:58.786312+0000","last_active":"2024-09-10T21:02:41.977719+0000","last_peered":"2024-09-10T21:02:41.977719+0000","last_clean":"2024-09-10T21:02:41.977719+0000","last_became_active":"2024-09-10T21:01:58.785894+0000","last_became_peered":"2024-09-10T21:01:58.785894+0000","last_unstale":"2024-09-10T21:02:41.977719+0000","last_undegraded":"2024-09-10T21:02:41.977719+0000","last_fullsized":"2024-09-10T21:02:41.977719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:11.682904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.500168+0000","last_change":"2024-09-10T21:02:00.808374+0000","last_active":"2024-09-10T21:03:40.500168+0000","last_peered":"2024-09-10T21:03:40.500168+0000","last_clean":"2024-09-10T21:03:40.500168+0000","last_became_active":"2024-09-10T21:02:00.807565+0000","last_became_peered":"2024-09-10T21:02:00.807565+0000","last_unstale":"2024-09-10T21:03:40.500168+0000","last_undegraded":"2024-09-10T21:03:40.500168+0000","last_fullsized":"2024-09-10T21:03:40.500168+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:24.227480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870458+0000","last_change":"2024-09-10T21:01:58.782375+0000","last_active":"2024-09-10T21:02:41.870458+0000","last_peered":"2024-09-10T21:02:41.870458+0000","last_clean":"2024-09-10T21:02:41.870458+0000","last_became_active":"2024-09-10T21:01:58.781964+0000","last_became_peered":"2024-09-10T21:01:58.781964+0000","last_unstale":"2024-09-10T21:02:41.870458+0000","last_undegraded":"2024-09-10T21:02:41.870458+0000","last_fullsized":"2024-09-10T21:02:41.870458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:02.460126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.485903+0000","last_change":"2024-09-10T21:02:00.806612+0000","last_active":"2024-09-10T21:03:40.485903+0000","last_peered":"2024-09-10T21:03:40.485903+0000","last_clean":"2024-09-10T21:03:40.485903+0000","last_became_active":"2024-09-10T21:02:00.803470+0000","last_became_peered":"2024-09-10T21:02:00.803470+0000","last_unstale":"2024-09-10T21:03:40.485903+0000","last_undegraded":"2024-09-10T21:03:40.485903+0000","last_fullsized":"2024-09-10T21:03:40.485903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:08.350931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.871476+0000","last_change":"2024-09-10T21:01:58.788987+0000","last_active":"2024-09-10T21:02:41.871476+0000","last_peered":"2024-09-10T21:02:41.871476+0000","last_clean":"2024-09-10T21:02:41.871476+0000","last_became_active":"2024-09-10T21:01:58.788605+0000","last_became_peered":"2024-09-10T21:01:58.788605+0000","last_unstale":"2024-09-10T21:02:41.871476+0000","last_undegraded":"2024-09-10T21:02:41.871476+0000","last_fullsized":"2024-09-10T21:02:41.871476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:45.798350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.509907+0000","last_change":"2024-09-10T21:02:00.804200+0000","last_active":"2024-09-10T21:03:40.509907+0000","last_peered":"2024-09-10T21:03:40.509907+0000","last_clean":"2024-09-10T21:03:40.509907+0000","last_became_active":"2024-09-10T21:02:00.803875+0000","last_became_peered":"2024-09-10T21:02:00.803875+0000","last_unstale":"2024-09-10T21:03:40.509907+0000","last_undegraded":"2024-09-10T21:03:40.509907+0000","last_fullsized":"2024-09-10T21:03:40.509907+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:45:15.930163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":40,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.507079+0000","last_change":"2024-09-10T21:02:41.874580+0000","last_active":"2024-09-10T21:03:40.507079+0000","last_peered":"2024-09-10T21:03:40.507079+0000","last_clean":"2024-09-10T21:03:40.507079+0000","last_became_active":"2024-09-10T21:02:41.873545+0000","last_became_peered":"2024-09-10T21:02:41.873545+0000","last_unstale":"2024-09-10T21:03:40.507079+0000","last_undegraded":"2024-09-10T21:03:40.507079+0000","last_fullsized":"2024-09-10T21:03:40.507079+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:05.278855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'5","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.870983+0000","last_change":"2024-09-10T21:01:58.784834+0000","last_active":"2024-09-10T21:02:41.870983+0000","last_peered":"2024-09-10T21:02:41.870983+0000","last_clean":"2024-09-10T21:02:41.870983+0000","last_became_active":"2024-09-10T21:01:58.784464+0000","last_became_peered":"2024-09-10T21:01:58.784464+0000","last_unstale":"2024-09-10T21:02:41.870983+0000","last_undegraded":"2024-09-10T21:02:41.870983+0000","last_fullsized":"2024-09-10T21:02:41.870983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:09.864824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":27,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.510409+0000","last_change":"2024-09-10T21:02:00.805677+0000","last_active":"2024-09-10T21:03:40.510409+0000","last_peered":"2024-09-10T21:03:40.510409+0000","last_clean":"2024-09-10T21:03:40.510409+0000","last_became_active":"2024-09-10T21:02:00.805341+0000","last_became_peered":"2024-09-10T21:02:00.805341+0000","last_unstale":"2024-09-10T21:03:40.510409+0000","last_undegraded":"2024-09-10T21:03:40.510409+0000","last_fullsized":"2024-09-10T21:03:40.510409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.236954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'12","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.889813+0000","last_change":"2024-09-10T21:02:41.889336+0000","last_active":"2024-09-10T21:02:41.889813+0000","last_peered":"2024-09-10T21:02:41.889813+0000","last_clean":"2024-09-10T21:02:41.889813+0000","last_became_active":"2024-09-10T21:02:41.874164+0000","last_became_peered":"2024-09-10T21:02:41.874164+0000","last_unstale":"2024-09-10T21:02:41.889813+0000","last_undegraded":"2024-09-10T21:02:41.889813+0000","last_fullsized":"2024-09-10T21:02:41.889813+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.799546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.504133+0000","last_change":"2024-09-10T21:02:00.801514+0000","last_active":"2024-09-10T21:03:40.504133+0000","last_peered":"2024-09-10T21:03:40.504133+0000","last_clean":"2024-09-10T21:03:40.504133+0000","last_became_active":"2024-09-10T21:02:00.801221+0000","last_became_peered":"2024-09-10T21:02:00.801221+0000","last_unstale":"2024-09-10T21:03:40.504133+0000","last_undegraded":"2024-09-10T21:03:40.504133+0000","last_fullsized":"2024-09-10T21:03:40.504133+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:35.564890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:02:41.978209+0000","last_change":"2024-09-10T21:01:58.884876+0000","last_active":"2024-09-10T21:02:41.978209+0000","last_peered":"2024-09-10T21:02:41.978209+0000","last_clean":"2024-09-10T21:02:41.978209+0000","last_became_active":"2024-09-10T21:01:58.884530+0000","last_became_peered":"2024-09-10T21:01:58.884530+0000","last_unstale":"2024-09-10T21:02:41.978209+0000","last_undegraded":"2024-09-10T21:02:41.978209+0000","last_fullsized":"2024-09-10T21:02:41.978209+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:34.851017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:40.479680+0000","last_change":"2024-09-10T21:02:00.811148+0000","last_active":"2024-09-10T21:03:40.479680+0000","last_peered":"2024-09-10T21:03:40.479680+0000","last_clean":"2024-09-10T21:03:40.479680+0000","last_became_active":"2024-09-10T21:02:00.809265+0000","last_became_peered":"2024-09-10T21:02:00.809265+0000","last_unstale":"2024-09-10T21:03:40.479680+0000","last_undegraded":"2024-09-10T21:03:40.479680+0000","last_fullsized":"2024-09-10T21:03:40.479680+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:59.771290+0000","last_clean_scrub_stamp":"2024-09-10T21:01:59.771290+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:25.269876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"77'14","reported_seq":143,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-10T21:03:56.011599+0000","last_change":"2024-09-10T21:01:58.786917+0000","last_active":"2024-09-10T21:03:56.011599+0000","last_peered":"2024-09-10T21:03:56.011599+0000","last_clean":"2024-09-10T21:03:56.011599+0000","last_became_active":"2024-09-10T21:01:58.786595+0000","last_became_peered":"2024-09-10T21:01:58.786595+0000","last_unstale":"2024-09-10T21:03:56.011599+0000","last_undegraded":"2024-09-10T21:03:56.011599+0000","last_fullsized":"2024-09-10T21:03:56.011599+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:01:57.758859+0000","last_clean_scrub_stamp":"2024-09-10T21:01:57.758859+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:33:30.807900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74899,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1095,"num_read_kb":631,"num_write":267,"num_write_kb":159,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1237,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":224697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":64,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281596,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743460,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":847872,"data_stored":407671,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":63,"seq":270582939690,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1064937,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":58,"seq":249108103215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":406957,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":52,"seq":223338299443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":851968,"data_stored":407223,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495671,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713076,"statfs":{"total":95995035648,"available":95962189824,"internally_reserved":0,"allocated":913408,"data_stored":1001366,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":41,"seq":176093659198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32012,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713140,"statfs":{"total":95995035648,"available":95962255360,"internally_reserved":0,"allocated":847872,"data_stored":407206,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":35,"seq":150323855427,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":999672,"data_compressed":41604,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":30,"seq":128849018953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":836,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":856064,"data_stored":408529,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":24,"seq":103079215181,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717284,"statfs":{"total":95995035648,"available":95966498816,"internally_reserved":0,"allocated":864256,"data_stored":406499,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":17,"seq":73014444115,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":473646,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":921600,"data_stored":474272,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":408545,"data_compressed":5797,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34187,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35807,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":68437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:04:01.343 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:04:01.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:01 smithi104 ceph-mon[35325]: pgmap v347: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:04:01.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:01 smithi104 ceph-mon[35325]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:01.637 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:01 smithi174 ceph-mon[35635]: pgmap v347: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:04:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:01 smithi174 ceph-mon[35635]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:02.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 -- 172.21.15.33:0/1637701991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 msgr2=0x7f61481031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 --2- 172.21.15.33:0/1637701991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481031a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6144009900 tx=0x7f614402a410 comp rx=0 tx=0).stop 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 -- 172.21.15.33:0/1637701991 shutdown_connections 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 --2- 172.21.15.33:0/1637701991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148109150 0x7f614810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 --2- 172.21.15.33:0/1637701991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61481087d0 0x7f6148108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 --2- 172.21.15.33:0/1637701991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481031a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 -- 172.21.15.33:0/1637701991 >> 172.21.15.33:0/1637701991 conn(0x7f61480fe830 msgr2=0x7f6148100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 -- 172.21.15.33:0/1637701991 shutdown_connections 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.112+0000 7f615020a640 1 -- 172.21.15.33:0/1637701991 wait complete. 2024-09-10T21:04:02.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.113+0000 7f615020a640 1 Processor -- start 2024-09-10T21:04:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.113+0000 7f615020a640 1 -- start start 2024-09-10T21:04:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f615020a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f615020a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61481087d0 0x7f6148199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f615020a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148109150 0x7f614819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:02.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f615020a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6148112dd0 con 0x7f6148102dc0 2024-09-10T21:04:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f615020a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6148112c50 con 0x7f61481087d0 2024-09-10T21:04:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f615020a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6148112f50 con 0x7f6148109150 2024-09-10T21:04:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52798/0 (socket says 172.21.15.33:52798) 2024-09-10T21:04:02.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 -- 172.21.15.33:0/2175291816 learned_addr learned my addr 172.21.15.33:0/2175291816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614d7a5640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61481087d0 0x7f6148199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614e7a7640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148109150 0x7f614819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 -- 172.21.15.33:0/2175291816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148109150 msgr2=0x7f614819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148109150 0x7f614819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 -- 172.21.15.33:0/2175291816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61481087d0 msgr2=0x7f6148199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61481087d0 0x7f6148199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 -- 172.21.15.33:0/2175291816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614819f370 con 0x7f6148102dc0 2024-09-10T21:04:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614e7a7640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148109150 0x7f614819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.114+0000 7f614dfa6640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481992c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6144005940 tx=0x7f61440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.115+0000 7f613effd640 1 -- 172.21.15.33:0/2175291816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614403f030 con 0x7f6148102dc0 2024-09-10T21:04:02.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.115+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614819f600 con 0x7f6148102dc0 2024-09-10T21:04:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.115+0000 7f613effd640 1 -- 172.21.15.33:0/2175291816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6144003cb0 con 0x7f6148102dc0 2024-09-10T21:04:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.115+0000 7f613effd640 1 -- 172.21.15.33:0/2175291816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144035070 con 0x7f6148102dc0 2024-09-10T21:04:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.115+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61481a5d20 con 0x7f6148102dc0 2024-09-10T21:04:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.116+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61481031a0 con 0x7f6148102dc0 2024-09-10T21:04:02.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.117+0000 7f613effd640 1 -- 172.21.15.33:0/2175291816 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6144003800 con 0x7f6148102dc0 2024-09-10T21:04:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.118+0000 7f613effd640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61340b52e0 0x7f61340b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.118+0000 7f613effd640 1 -- 172.21.15.33:0/2175291816 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6144106a00 con 0x7f6148102dc0 2024-09-10T21:04:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.118+0000 7f614d7a5640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61340b52e0 0x7f61340b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:02.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.118+0000 7f614d7a5640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61340b52e0 0x7f61340b77a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6148108270 tx=0x7f613800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:02.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.121+0000 7f613effd640 1 -- 172.21.15.33:0/2175291816 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f61440cd8c0 con 0x7f6148102dc0 2024-09-10T21:04:02.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.345+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f614802e5c0 con 0x7f6148102dc0 2024-09-10T21:04:02.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:02 smithi033 ceph-mon[30860]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:02.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:02 smithi033 ceph-mon[30860]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:02.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.346+0000 7f613effd640 1 -- 172.21.15.33:0/2175291816 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11973 (secure 0 0 0) 0x7f61440d6070 con 0x7f6148102dc0 2024-09-10T21:04:02.351 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:04:02.351 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"btime":"2024-09-10T21:03:57:162551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:03:56.306223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:04:02.351 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61340b52e0 msgr2=0x7f61340b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61340b52e0 0x7f61340b77a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6148108270 tx=0x7f613800b390 comp rx=0 tx=0).stop 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 msgr2=0x7f61481992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481992c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6144005940 tx=0x7f61440042f0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 shutdown_connections 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f61340b52e0 0x7f61340b77a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148109150 0x7f614819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61481087d0 0x7f6148199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 --2- 172.21.15.33:0/2175291816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6148102dc0 0x7f61481992c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 >> 172.21.15.33:0/2175291816 conn(0x7f61480fe830 msgr2=0x7f6148100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 shutdown_connections 2024-09-10T21:04:02.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:02.349+0000 7f615020a640 1 -- 172.21.15.33:0/2175291816 wait complete. 2024-09-10T21:04:02.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:02 smithi104 ceph-mon[35325]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:02.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:02 smithi104 ceph-mon[35325]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:02.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:02 smithi174 ceph-mon[35635]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:02.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:02 smithi174 ceph-mon[35635]: from='client.15357 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T21:04:03.159 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:04:03.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:03 smithi033 ceph-mon[30860]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T21:04:03.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:03 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2175291816' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:03.428 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:03.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:03 smithi104 ceph-mon[35325]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T21:04:03.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:03 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2175291816' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:03 smithi174 ceph-mon[35635]: pgmap v348: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 9 op/s 2024-09-10T21:04:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:03 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2175291816' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 -- 172.21.15.33:0/3418383906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 msgr2=0x7f81a41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 --2- 172.21.15.33:0/3418383906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a41031a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f81a0009900 tx=0x7f81a002a410 comp rx=0 tx=0).stop 2024-09-10T21:04:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 -- 172.21.15.33:0/3418383906 shutdown_connections 2024-09-10T21:04:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 --2- 172.21.15.33:0/3418383906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a4109150 0x7f81a410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 --2- 172.21.15.33:0/3418383906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81a41087d0 0x7f81a4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:03.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 --2- 172.21.15.33:0/3418383906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a41031a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 -- 172.21.15.33:0/3418383906 >> 172.21.15.33:0/3418383906 conn(0x7f81a40fe830 msgr2=0x7f81a4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 -- 172.21.15.33:0/3418383906 shutdown_connections 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.964+0000 7f81ac419640 1 -- 172.21.15.33:0/3418383906 wait complete. 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 Processor -- start 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 -- start start 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a41087d0 0x7f81a41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81a4109150 0x7f81a419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4112e10 con 0x7f81a4102dc0 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4112c90 con 0x7f81a41087d0 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81ac419640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4112f90 con 0x7f81a4109150 2024-09-10T21:04:03.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a4199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52826/0 (socket says 172.21.15.33:52826) 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 -- 172.21.15.33:0/3033600287 learned_addr learned my addr 172.21.15.33:0/3033600287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81a99b4640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a41087d0 0x7f81a41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa9b6640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81a4109150 0x7f81a419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 -- 172.21.15.33:0/3033600287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81a4109150 msgr2=0x7f81a419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81a4109150 0x7f81a419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 -- 172.21.15.33:0/3033600287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a41087d0 msgr2=0x7f81a41998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a41087d0 0x7f81a41998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.965+0000 7f81aa1b5640 1 -- 172.21.15.33:0/3033600287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a419f410 con 0x7f81a4102dc0 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.966+0000 7f81aa1b5640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a4199360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005940 tx=0x7f81a0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:03.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.966+0000 7f819b7fe640 1 -- 172.21.15.33:0/3033600287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a003f030 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.966+0000 7f819b7fe640 1 -- 172.21.15.33:0/3033600287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a0003aa0 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.966+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a419f6a0 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.966+0000 7f819b7fe640 1 -- 172.21.15.33:0/3033600287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0035070 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.966+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41a5dc0 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.967+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a41031a0 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.968+0000 7f819b7fe640 1 -- 172.21.15.33:0/3033600287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f81a003bcc0 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.969+0000 7f819b7fe640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81900b52e0 0x7f81900b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.969+0000 7f81a99b4640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81900b52e0 0x7f81900b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.969+0000 7f819b7fe640 1 -- 172.21.15.33:0/3033600287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f81a01067b0 con 0x7f81a4102dc0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.970+0000 7f81a99b4640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81900b52e0 0x7f81900b77a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81a40ffae0 tx=0x7f819400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:03.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:03.972+0000 7f819b7fe640 1 -- 172.21.15.33:0/3033600287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f81a00cd020 con 0x7f81a4102dc0 2024-09-10T21:04:04.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.159+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81a419a490 con 0x7f81a4102dc0 2024-09-10T21:04:04.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.160+0000 7f819b7fe640 1 -- 172.21.15.33:0/3033600287 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22763 (secure 0 0 0) 0x7f81a00d1a50 con 0x7f81a4102dc0 2024-09-10T21:04:04.165 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:04:04.165 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:41.855317+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:04:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81900b52e0 msgr2=0x7f81900b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81900b52e0 0x7f81900b77a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f81a40ffae0 tx=0x7f819400b390 comp rx=0 tx=0).stop 2024-09-10T21:04:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 msgr2=0x7f81a4199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a4199360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005940 tx=0x7f81a0004270 comp rx=0 tx=0).stop 2024-09-10T21:04:04.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 shutdown_connections 2024-09-10T21:04:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81900b52e0 0x7f81900b77a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81a4109150 0x7f81a419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a41087d0 0x7f81a41998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 --2- 172.21.15.33:0/3033600287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81a4102dc0 0x7f81a4199360 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 >> 172.21.15.33:0/3033600287 conn(0x7f81a40fe830 msgr2=0x7f81a4100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 shutdown_connections 2024-09-10T21:04:04.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:04.163+0000 7f81ac419640 1 -- 172.21.15.33:0/3033600287 wait complete. 2024-09-10T21:04:04.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:04 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3033600287' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:04:04.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:04 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3033600287' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:04:04.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:04 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3033600287' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:04:04.867 DEBUG:tasks.fwd_scrub:Yielding 2024-09-10T21:04:04.867 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T21:04:04.882 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:04:04.882 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T21:04:04.882 INFO:tasks.workunit:timeout=3h 2024-09-10T21:04:04.882 INFO:tasks.workunit:cleanup=True 2024-09-10T21:04:04.882 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T21:04:04.883 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-10T21:04:04.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:04:04.883 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:04:04.903 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T21:04:04.903 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-10T21:04:04.903 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-10T21:04:04.903 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:04:04.903 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-10T21:04:04.903 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-09-10 21:02:19.574107671 +0000 2024-09-10T21:04:04.903 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-09-10 21:02:19.574107671 +0000 2024-09-10T21:04:04.904 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-09-10 21:03:55.604297082 +0000 2024-09-10T21:04:04.904 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-09-10 21:02:19.574107671 +0000 2024-09-10T21:04:04.904 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T21:04:04.904 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T21:04:04.978 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:04:05.035 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T21:04:05.152 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:05.555 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:05 smithi033 ceph-mon[30860]: pgmap v349: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T21:04:05.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:05 smithi104 ceph-mon[35325]: pgmap v349: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T21:04:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:05 smithi174 ceph-mon[35635]: pgmap v349: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 -- 172.21.15.33:0/2410885074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c10d140 msgr2=0x7f498c10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 --2- 172.21.15.33:0/2410885074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c10d140 0x7f498c10f5a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4984009ca0 tx=0x7f498402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 -- 172.21.15.33:0/2410885074 shutdown_connections 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 --2- 172.21.15.33:0/2410885074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c10d140 0x7f498c10f5a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 --2- 172.21.15.33:0/2410885074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f498c104600 0x7f498c10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 --2- 172.21.15.33:0/2410885074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c103ce0 0x7f498c1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 -- 172.21.15.33:0/2410885074 >> 172.21.15.33:0/2410885074 conn(0x7f498c0fd700 msgr2=0x7f498c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:05.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 -- 172.21.15.33:0/2410885074 shutdown_connections 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.638+0000 7f4990dd7640 1 -- 172.21.15.33:0/2410885074 wait complete. 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 Processor -- start 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 -- start start 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c103ce0 0x7f498c19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f498c104600 0x7f498c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10d140 0x7f498c1a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f498c112e40 con 0x7f498c103ce0 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f498c112cc0 con 0x7f498c104600 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f4990dd7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f498c112fc0 con 0x7f498c10d140 2024-09-10T21:04:05.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.639+0000 7f498a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c103ce0 0x7f498c19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c103ce0 0x7f498c19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52846/0 (socket says 172.21.15.33:52846) 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 -- 172.21.15.33:0/3347740835 learned_addr learned my addr 172.21.15.33:0/3347740835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498ad9d640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10d140 0x7f498c1a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f4983fff640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f498c104600 0x7f498c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 -- 172.21.15.33:0/3347740835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10d140 msgr2=0x7f498c1a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10d140 0x7f498c1a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 -- 172.21.15.33:0/3347740835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f498c104600 msgr2=0x7f498c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f498c104600 0x7f498c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 -- 172.21.15.33:0/3347740835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498c1a1500 con 0x7f498c103ce0 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498ad9d640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10d140 0x7f498c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f498a59c640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c103ce0 0x7f498c19b450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b4a0 tx=0x7f497c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:05.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f49837fe640 1 -- 172.21.15.33:0/3347740835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c014070 con 0x7f498c103ce0 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f49837fe640 1 -- 172.21.15.33:0/3347740835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f497c010ac0 con 0x7f498c103ce0 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498c1a17f0 con 0x7f498c103ce0 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f49837fe640 1 -- 172.21.15.33:0/3347740835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c017c30 con 0x7f498c103ce0 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.640+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f498c1a7f30 con 0x7f498c103ce0 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49837fe640 1 -- 172.21.15.33:0/3347740835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f497c010610 con 0x7f498c103ce0 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 Processor -- start 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 -- start start 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x5614669fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669fb800 0x561466a07250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a09b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561466971e60 con 0x5614669faee0 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561466961290 con 0x561466a07790 2024-09-10T21:04:05.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.643+0000 7f49931ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5614669650a0 con 0x5614669fb800 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x5614669fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x5614669fb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52850/0 (socket says 172.21.15.33:52850) 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 -- 172.21.15.33:0/1567143899 learned_addr learned my addr 172.21.15.33:0/1567143899 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f4983fff640 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a09b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498a59c640 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669fb800 0x561466a07250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 -- 172.21.15.33:0/1567143899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669fb800 msgr2=0x561466a07250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669fb800 0x561466a07250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 -- 172.21.15.33:0/1567143899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 msgr2=0x561466a09b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a09b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 -- 172.21.15.33:0/1567143899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614666bf930 con 0x5614669faee0 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498a59c640 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669fb800 0x561466a07250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f49837fe640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49780b52e0 0x7f49780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498ad9d640 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x5614669fb2c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4984009a50 tx=0x7f4984031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29ccb0cdd3e23c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:05.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f498a59c640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49780b52e0 0x7f49780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f4980ff9640 1 -- 172.21.15.33:0/1567143899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984031650 con 0x5614669faee0 2024-09-10T21:04:05.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.644+0000 7f49837fe640 1 -- 172.21.15.33:0/3347740835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f497c0e5570 con 0x7f498c103ce0 2024-09-10T21:04:05.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f4980ff9640 1 -- 172.21.15.33:0/1567143899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4984040030 con 0x5614669faee0 2024-09-10T21:04:05.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f4980ff9640 1 -- 172.21.15.33:0/1567143899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49840046a0 con 0x5614669faee0 2024-09-10T21:04:05.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f49931ac740 1 -- 172.21.15.33:0/1567143899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 msgr2=0x5614669fb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f498a59c640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49780b52e0 0x7f49780b77a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f497c000c00 tx=0x7f497c010f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:05.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f49931ac740 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x5614669fb2c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4984009a50 tx=0x7f4984031af0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f49931ac740 1 -- 172.21.15.33:0/1567143899 shutdown_connections 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f49931ac740 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a09b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f49931ac740 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669fb800 0x561466a07250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f49931ac740 1 --2- 172.21.15.33:0/1567143899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x5614669fb2c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.645+0000 7f49931ac740 1 -- 172.21.15.33:0/1567143899 >> 172.21.15.33:0/1567143899 conn(0x5614669f8d10 msgr2=0x5614669f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 -- 172.21.15.33:0/1567143899 shutdown_connections 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 -- 172.21.15.33:0/1567143899 wait complete. 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 -- start start 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 Processor -- start 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x561466a046e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a04c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561466a07790 0x561466a05160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5614667c2890 con 0x5614669faee0 2024-09-10T21:04:05.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5614668fbd40 con 0x5614669fb800 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561466a0a3e0 con 0x561466a07790 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f49931ac740 10 client.0 _ll_register_callbacks cb 0x56146695fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x561466a046e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x561466a046e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52858/0 (socket says 172.21.15.33:52858) 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.646+0000 7f498ad9d640 1 -- 172.21.15.33:0/4095265116 learned_addr learned my addr 172.21.15.33:0/4095265116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f4983fff640 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a04c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f498ad9d640 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561466a07790 0x561466a05160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f498ad9d640 1 -- 172.21.15.33:0/4095265116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561466a07790 msgr2=0x561466a05160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f498ad9d640 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561466a07790 0x561466a05160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f498ad9d640 1 -- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 msgr2=0x561466a04c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f498ad9d640 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a04c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f498ad9d640 1 -- 172.21.15.33:0/4095265116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561466a13770 con 0x5614669faee0 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f498ad9d640 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x561466a046e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f498402a900 tx=0x7f49840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf51abad58a3b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f4971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:05.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f4971ffb640 1 -- 172.21.15.33:0/4095265116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498404e040 con 0x5614669faee0 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f4971ffb640 1 -- 172.21.15.33:0/4095265116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4984003970 con 0x5614669faee0 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4984040030 con 0x5614669faee0 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f4971ffb640 1 -- 172.21.15.33:0/4095265116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4984003b70 con 0x5614669faee0 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f49931ac740 10 client.15381 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561466a16120 con 0x5614669faee0 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.647+0000 7f4971ffb640 1 -- 172.21.15.33:0/4095265116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4984040030 con 0x5614669faee0 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f4971ffb640 1 -- 172.21.15.33:0/4095265116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f498402fb00 con 0x5614669faee0 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f49931ac740 10 client.15381 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f49931ac740 10 client.15381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f49931ac740 20 client.15381 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f49931ac740 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a17470 0x561466a19860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f49931ac740 4 client.15381 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561466973db0 con 0x561466a17470 2024-09-10T21:04:05.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f4983fff640 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a17470 0x561466a19860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f4983fff640 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a17470 0x561466a19860 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c24454ab1f2cdcb server_cookie=1264a6185216e11a in_seq=0 out_seq=0 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.648+0000 7f4971ffb640 10 client.15381 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.649+0000 7f4971ffb640 1 -- 172.21.15.33:0/4095265116 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561466973db0 con 0x561466a17470 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.649+0000 7f4971ffb640 10 client.15381 handle_command_reply: tid=0 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.649+0000 7f49931ac740 1 client.15381 shutdown 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 msgr2=0x561466a046e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x561466a046e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f498402a900 tx=0x7f49840095c0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 shutdown_connections 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a17470 0x561466a19860 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561466a07790 0x561466a05160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a04c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 --2- 172.21.15.33:0/4095265116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669faee0 0x561466a046e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f4971ffb640 0 client.15381 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 >> 172.21.15.33:0/4095265116 conn(0x5614669f8d10 msgr2=0x5614669f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:05.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 shutdown_connections 2024-09-10T21:04:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 1 -- 172.21.15.33:0/4095265116 wait complete. 2024-09-10T21:04:05.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.650+0000 7f49931ac740 20 client.15381 trim_cache size 0 max 16384 2024-09-10T21:04:05.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.667+0000 7f49931ac740 1 Processor -- start 2024-09-10T21:04:05.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f49931ac740 1 -- start start 2024-09-10T21:04:05.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f49931ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x5614669fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f49931ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669fb800 0x561466a018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f49931ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a01e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f49931ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1b06e0 con 0x5614669fb800 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f49931ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1b2220 con 0x561466a07790 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f49931ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1b0db0 con 0x5614669faee0 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669fb800 0x561466a018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669fb800 0x561466a018f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52874/0 (socket says 172.21.15.33:52874) 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 -- 172.21.15.33:0/346393909 learned_addr learned my addr 172.21.15.33:0/346393909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f498ad9d640 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x5614669fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f498ad9d640 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a01e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 -- 172.21.15.33:0/346393909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 msgr2=0x5614669fa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x5614669fa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 -- 172.21.15.33:0/346393909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 msgr2=0x561466a01e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a01e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 -- 172.21.15.33:0/346393909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614669f9790 con 0x5614669fb800 2024-09-10T21:04:05.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f498ad9d640 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x5614669fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:05.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.668+0000 7f4983fff640 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669fb800 0x561466a018f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4974008aa0 tx=0x7f4974004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf54947c167172f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:05.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f4973fff640 1 -- 172.21.15.33:0/346393909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974003d20 con 0x5614669fb800 2024-09-10T21:04:05.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f4973fff640 1 -- 172.21.15.33:0/346393909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f497401c8d0 con 0x5614669fb800 2024-09-10T21:04:05.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f4973fff640 1 -- 172.21.15.33:0/346393909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974010760 con 0x5614669fb800 2024-09-10T21:04:05.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 -- 172.21.15.33:0/346393909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669fb800 msgr2=0x561466a018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669fb800 0x561466a018f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f4974008aa0 tx=0x7f4974004410 comp rx=0 tx=0).stop 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 -- 172.21.15.33:0/346393909 shutdown_connections 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561466a07790 0x561466a01e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5614669fb800 0x561466a018f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 --2- 172.21.15.33:0/346393909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x5614669fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 -- 172.21.15.33:0/346393909 >> 172.21.15.33:0/346393909 conn(0x5614669f8d10 msgr2=0x561466a06530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 -- 172.21.15.33:0/346393909 shutdown_connections 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.669+0000 7f49931ac740 1 -- 172.21.15.33:0/346393909 wait complete. 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 -- start start 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 Processor -- start 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x561466a1af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a1b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561466a07790 0x561466a82fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1b1b50 con 0x561466a07790 2024-09-10T21:04:05.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1b1480 con 0x5614669fb800 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f498c1ae6a0 con 0x5614669faee0 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f49931ac740 10 client.0 _ll_register_callbacks cb 0x7f498c06c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561466a07790 0x561466a82fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561466a07790 0x561466a82fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52888/0 (socket says 172.21.15.33:52888) 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 -- 172.21.15.33:0/1369810051 learned_addr learned my addr 172.21.15.33:0/1369810051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f4983fff640 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a1b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x561466a1af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 -- 172.21.15.33:0/1369810051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 msgr2=0x561466a1af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x561466a1af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 -- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 msgr2=0x561466a1b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a1b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.670+0000 7f498ad9d640 1 -- 172.21.15.33:0/1369810051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561466a83550 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f498ad9d640 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561466a07790 0x561466a82fe0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4984005cc0 tx=0x7f49840095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce555536cf492b09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f4971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f4971ffb640 1 -- 172.21.15.33:0/1369810051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4984003970 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f4971ffb640 1 -- 172.21.15.33:0/1369810051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4984030610 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f497401c8d0 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f4971ffb640 1 -- 172.21.15.33:0/1369810051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4984030810 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 10 client.15387 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x561466a84890 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f4971ffb640 1 -- 172.21.15.33:0/1369810051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f497401c8d0 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f4971ffb640 1 -- 172.21.15.33:0/1369810051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f4984030ad0 con 0x561466a07790 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 10 client.15387 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 10 client.15387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:05.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 20 client.15387 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a84f80 0x561466a873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 4 client.15387 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.671+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56146696f860 con 0x561466a84f80 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.672+0000 7f4983fff640 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a84f80 0x561466a873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.672+0000 7f4971ffb640 10 client.15387 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.672+0000 7f4983fff640 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a84f80 0x561466a873a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=737130dacab9eef7 server_cookie=20ae787eba1834e3 in_seq=0 out_seq=0 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f4971ffb640 1 -- 172.21.15.33:0/1369810051 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56146696f860 con 0x561466a84f80 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f4971ffb640 10 client.15387 handle_command_reply: tid=0 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 client.15387 shutdown 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561466a07790 msgr2=0x561466a82fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561466a07790 0x561466a82fe0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4984005cc0 tx=0x7f49840095c0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 shutdown_connections 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561466a84f80 0x561466a873a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561466a07790 0x561466a82fe0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5614669fb800 0x561466a1b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 --2- 172.21.15.33:0/1369810051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5614669faee0 0x561466a1af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f4971ffb640 0 client.15387 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 >> 172.21.15.33:0/1369810051 conn(0x5614669f8d10 msgr2=0x561466a05d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 shutdown_connections 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.674+0000 7f49931ac740 1 -- 172.21.15.33:0/1369810051 wait complete. 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f49931ac740 20 client.15387 trim_cache size 0 max 16384 2024-09-10T21:04:05.685 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:04:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:04:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3d0a5da9-0921-4663-ac39-3add4adc1499", 2024-09-10T21:04:05.686 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:04:05.686 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:04:05.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49780b52e0 msgr2=0x7f49780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49780b52e0 0x7f49780b77a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f497c000c00 tx=0x7f497c010f90 comp rx=0 tx=0).stop 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c103ce0 msgr2=0x7f498c19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c103ce0 0x7f498c19b450 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f497c00b4a0 tx=0x7f497c00b960 comp rx=0 tx=0).stop 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 shutdown_connections 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49780b52e0 0x7f49780b77a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f498c10d140 0x7f498c1a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f498c104600 0x7f498c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 --2- 172.21.15.33:0/3347740835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f498c103ce0 0x7f498c19b450 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 >> 172.21.15.33:0/3347740835 conn(0x7f498c0fd700 msgr2=0x7f498c0fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 shutdown_connections 2024-09-10T21:04:05.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:05.675+0000 7f4990dd7640 1 -- 172.21.15.33:0/3347740835 wait complete. 2024-09-10T21:04:06.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:06 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:04:06.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:06 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:06.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:06 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:04:06.404 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:06 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:06.434 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:04:06.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:06 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:04:06.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:06 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:06.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:06 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:04:06.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:06 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:06.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:06 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:04:06.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:06 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:06.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:06 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:04:06.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:06 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:06.738 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/2524537145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101c90 msgr2=0x7ff6c4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/2524537145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101c90 0x7ff6c4102070 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009900 tx=0x7ff6c002a410 comp rx=0 tx=0).stop 2024-09-10T21:04:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/2524537145 shutdown_connections 2024-09-10T21:04:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/2524537145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c4108020 0x7ff6c410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/2524537145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6c41076a0 0x7ff6c4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/2524537145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101c90 0x7ff6c4102070 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/2524537145 >> 172.21.15.33:0/2524537145 conn(0x7ff6c40fd700 msgr2=0x7ff6c40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/2524537145 shutdown_connections 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.238+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/2524537145 wait complete. 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 Processor -- start 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 -- start start 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c4101c90 0x7ff6c410f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6c41076a0 0x7ff6c410f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4108020 0x7ff6c410fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4112dc0 con 0x7ff6c4108020 2024-09-10T21:04:07.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4112c40 con 0x7ff6c4101c90 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6cb7d0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c4112f40 con 0x7ff6c41076a0 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4108020 0x7ff6c410fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4108020 0x7ff6c410fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52900/0 (socket says 172.21.15.33:52900) 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/1136868779 learned_addr learned my addr 172.21.15.33:0/1136868779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c956c640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c4101c90 0x7ff6c410f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6c41076a0 0x7ff6c410f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/1136868779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6c41076a0 msgr2=0x7ff6c410f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6c41076a0 0x7ff6c410f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/1136868779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c4101c90 msgr2=0x7ff6c410f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c4101c90 0x7ff6c410f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/1136868779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6c41a76d0 con 0x7ff6c4108020 2024-09-10T21:04:07.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.239+0000 7ff6c956c640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c4101c90 0x7ff6c410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.240+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4108020 0x7ff6c410fdb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00eaa0 tx=0x7ff6bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.240+0000 7ff6ba7fc640 1 -- 172.21.15.33:0/1136868779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00cdb0 con 0x7ff6c4108020 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.240+0000 7ff6ba7fc640 1 -- 172.21.15.33:0/1136868779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6bc005020 con 0x7ff6c4108020 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.240+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6c41a79c0 con 0x7ff6c4108020 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.240+0000 7ff6ba7fc640 1 -- 172.21.15.33:0/1136868779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc0116d0 con 0x7ff6c4108020 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.240+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6c41a7f00 con 0x7ff6c4108020 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 Processor -- start 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6ba7fc640 1 -- 172.21.15.33:0/1136868779 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff6bc00f040 con 0x7ff6c4108020 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 -- start start 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df8f4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f49c0 0x5591df900410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df902d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591df86ae50 con 0x5591df8f49c0 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591df85a280 con 0x5591df8f40a0 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5591df85e090 con 0x5591df900950 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6c956c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f49c0 0x5591df900410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6c956c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f49c0 0x5591df900410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52910/0 (socket says 172.21.15.33:52910) 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6c956c640 1 -- 172.21.15.33:0/2550203732 learned_addr learned my addr 172.21.15.33:0/2550203732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.243+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df8f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df902d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c956c640 1 -- 172.21.15.33:0/2550203732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 msgr2=0x5591df902d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c956c640 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df902d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c956c640 1 -- 172.21.15.33:0/2550203732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 msgr2=0x5591df8f4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c956c640 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df8f4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c956c640 1 -- 172.21.15.33:0/2550203732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591df5b8650 con 0x5591df8f49c0 2024-09-10T21:04:07.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df902d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6ba7fc640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff6b00b5360 0x7ff6b00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c956c640 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f49c0 0x5591df900410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009900 tx=0x7ff6c0032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36eb62ad538a0a49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff6b00b5360 0x7ff6b00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6ba7fc640 1 -- 172.21.15.33:0/1136868779 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff6bc0e5a70 con 0x7ff6c4108020 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6af7fe640 1 -- 172.21.15.33:0/2550203732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c003f070 con 0x5591df8f49c0 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6af7fe640 1 -- 172.21.15.33:0/2550203732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c00316a0 con 0x5591df8f49c0 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.244+0000 7ff6af7fe640 1 -- 172.21.15.33:0/2550203732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0035070 con 0x5591df8f49c0 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2550203732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f49c0 msgr2=0x5591df900410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff6b00b5360 0x7ff6b00b7820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4110e20 tx=0x7ff6b4004730 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f49c0 0x5591df900410 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009900 tx=0x7ff6c0032cd0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2550203732 shutdown_connections 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df902d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f49c0 0x5591df900410 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2550203732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df8f4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2550203732 >> 172.21.15.33:0/2550203732 conn(0x5591df8f1ed0 msgr2=0x5591df8f3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2550203732 shutdown_connections 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2550203732 wait complete. 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 -- start start 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.245+0000 7ff6cdba5740 1 Processor -- start 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df900280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df8fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df900950 0x5591df8fdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591df6bb700 con 0x5591df900950 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591df7f5110 con 0x5591df8f40a0 2024-09-10T21:04:07.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5591df903570 con 0x5591df8f49c0 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6cdba5740 10 client.0 _ll_register_callbacks cb 0x5591df80e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df900950 0x5591df8fdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df900950 0x5591df8fdd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52916/0 (socket says 172.21.15.33:52916) 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/2298472145 learned_addr learned my addr 172.21.15.33:0/2298472145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c956c640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df900280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df8fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/2298472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 msgr2=0x5591df8fd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df8fd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 msgr2=0x5591df900280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df900280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c9d6d640 1 -- 172.21.15.33:0/2298472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591df822c40 con 0x5591df900950 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df8fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.246+0000 7ff6c956c640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df900280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:07.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df900950 0x5591df8fdd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0abf30 tx=0x7ff6bc0abad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9f6248f560825c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6acff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6acff9640 1 -- 172.21.15.33:0/2298472145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc01f030 con 0x5591df900950 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6acff9640 1 -- 172.21.15.33:0/2298472145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff6bc0054f0 con 0x5591df900950 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c00316a0 con 0x5591df900950 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6acff9640 1 -- 172.21.15.33:0/2298472145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6bc0ac230 con 0x5591df900950 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 10 client.15399 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591df90eee0 con 0x5591df900950 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6acff9640 1 -- 172.21.15.33:0/2298472145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c00316a0 con 0x5591df900950 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6acff9640 1 -- 172.21.15.33:0/2298472145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6bc0a7b80 con 0x5591df900950 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 10 client.15399 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 20 client.15399 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df9103f0 0x5591df912860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 4 client.15399 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.247+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591df86cda0 con 0x5591df9103f0 2024-09-10T21:04:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.248+0000 7ff6c956c640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df9103f0 0x5591df912860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.248+0000 7ff6c956c640 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df9103f0 0x5591df912860 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1bbbb5b91275229 server_cookie=38b00e03fa10d69a in_seq=0 out_seq=0 2024-09-10T21:04:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.248+0000 7ff6acff9640 10 client.15399 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6acff9640 1 -- 172.21.15.33:0/2298472145 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591df86cda0 con 0x5591df9103f0 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6acff9640 10 client.15399 handle_command_reply: tid=0 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 client.15399 shutdown 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df900950 msgr2=0x5591df8fdd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df900950 0x5591df8fdd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0abf30 tx=0x7ff6bc0abad0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 shutdown_connections 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df9103f0 0x5591df912860 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df900950 0x5591df8fdd50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df8fd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6acff9640 0 client.15399 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/2298472145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f40a0 0x5591df900280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 >> 172.21.15.33:0/2298472145 conn(0x5591df8f1ed0 msgr2=0x5591df8f7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 shutdown_connections 2024-09-10T21:04:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 1 -- 172.21.15.33:0/2298472145 wait complete. 2024-09-10T21:04:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.249+0000 7ff6cdba5740 20 client.15399 trim_cache size 0 max 16384 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 Processor -- start 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 -- start start 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df8fcd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f49c0 0x5591df90c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df8ff370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b0550 con 0x5591df8f40a0 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b2090 con 0x5591df8f49c0 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b0c20 con 0x5591df900950 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6c956c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df8fcd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6c956c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df8fcd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52922/0 (socket says 172.21.15.33:52922) 2024-09-10T21:04:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.267+0000 7ff6c956c640 1 -- 172.21.15.33:0/3158444965 learned_addr learned my addr 172.21.15.33:0/3158444965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df8ff370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c956c640 1 -- 172.21.15.33:0/3158444965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 msgr2=0x5591df8ff370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c956c640 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df8ff370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c956c640 1 -- 172.21.15.33:0/3158444965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f49c0 msgr2=0x5591df90c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c956c640 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f49c0 0x5591df90c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c956c640 1 -- 172.21.15.33:0/3158444965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591df8f2bb0 con 0x5591df8f40a0 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df8ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6c956c640 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df8fcd50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009900 tx=0x7ff6c00052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db1991ef6b286aa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6aeffd640 1 -- 172.21.15.33:0/3158444965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0050020 con 0x5591df8f40a0 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6aeffd640 1 -- 172.21.15.33:0/3158444965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c0005540 con 0x5591df8f40a0 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.268+0000 7ff6aeffd640 1 -- 172.21.15.33:0/3158444965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c003f040 con 0x5591df8f40a0 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 -- 172.21.15.33:0/3158444965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 msgr2=0x5591df8fcd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df8fcd50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0009900 tx=0x7ff6c00052d0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 -- 172.21.15.33:0/3158444965 shutdown_connections 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df900950 0x5591df8ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df8f49c0 0x5591df90c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/3158444965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df8fcd50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 -- 172.21.15.33:0/3158444965 >> 172.21.15.33:0/3158444965 conn(0x5591df8f1ed0 msgr2=0x5591df8fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 -- 172.21.15.33:0/3158444965 shutdown_connections 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 -- 172.21.15.33:0/3158444965 wait complete. 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 -- start start 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.269+0000 7ff6cdba5740 1 Processor -- start 2024-09-10T21:04:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df903c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df904160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6cdba5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df900950 0x5591df97d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b19c0 con 0x5591df8f40a0 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41b12f0 con 0x5591df900950 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6cdba5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6c41ae510 con 0x5591df8f49c0 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6cdba5740 10 client.0 _ll_register_callbacks cb 0x7ff6bc00cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df903c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df903c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52928/0 (socket says 172.21.15.33:52928) 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 -- 172.21.15.33:0/220003057 learned_addr learned my addr 172.21.15.33:0/220003057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c9d6d640 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df900950 0x5591df97d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df904160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 -- 172.21.15.33:0/220003057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 msgr2=0x5591df904160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df904160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 -- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df900950 msgr2=0x5591df97d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df900950 0x5591df97d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 -- 172.21.15.33:0/220003057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591df904a50 con 0x5591df8f40a0 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6c956c640 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df903c20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0031820 tx=0x7ff6c00058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766f99bbd7ef6613 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.270+0000 7ff6acff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6acff9640 1 -- 172.21.15.33:0/220003057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6c0059060 con 0x5591df8f40a0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6c0005540 con 0x5591df8f40a0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6acff9640 1 -- 172.21.15.33:0/220003057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff6c00048d0 con 0x5591df8f40a0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6acff9640 1 -- 172.21.15.33:0/220003057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff6c003f070 con 0x5591df8f40a0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 10 client.15405 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5591df97e920 con 0x5591df8f40a0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6acff9640 1 -- 172.21.15.33:0/220003057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6c0005540 con 0x5591df8f40a0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6acff9640 1 -- 172.21.15.33:0/220003057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7ff6c003b800 con 0x5591df8f40a0 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 10 client.15405 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 10 client.15405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 20 client.15405 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df97ec00 0x5591df980ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 4 client.15405 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.271+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5591df868850 con 0x5591df97ec00 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df97ec00 0x5591df980ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6acff9640 10 client.15405 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6c8d6b640 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df97ec00 0x5591df980ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f083a592e627c9 server_cookie=670fa0fec9e87706 in_seq=0 out_seq=0 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6acff9640 1 -- 172.21.15.33:0/220003057 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x5591df868850 con 0x5591df97ec00 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6acff9640 10 client.15405 handle_command_reply: tid=0 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 client.15405 shutdown 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 msgr2=0x5591df903c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df903c20 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff6c0031820 tx=0x7ff6c00058b0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 shutdown_connections 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5591df97ec00 0x5591df980ff0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6acff9640 0 client.15405 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591df900950 0x5591df97d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591df8f49c0 0x5591df904160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 --2- 172.21.15.33:0/220003057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591df8f40a0 0x5591df903c20 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.272+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 >> 172.21.15.33:0/220003057 conn(0x5591df8f1ed0 msgr2=0x5591df90b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 shutdown_connections 2024-09-10T21:04:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cdba5740 1 -- 172.21.15.33:0/220003057 wait complete. 2024-09-10T21:04:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cdba5740 20 client.15405 trim_cache size 0 max 16384 2024-09-10T21:04:07.289 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "3d0a5da9-0921-4663-ac39-3add4adc1499", 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:04:07.290 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:04:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:04:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:04:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:07.291 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:07.291 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:04:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff6b00b5360 msgr2=0x7ff6b00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff6b00b5360 0x7ff6b00b7820 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4110e20 tx=0x7ff6b4004730 comp rx=0 tx=0).stop 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4108020 msgr2=0x7ff6c410fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4108020 0x7ff6c410fdb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00eaa0 tx=0x7ff6bc00ef60 comp rx=0 tx=0).stop 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 shutdown_connections 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff6b00b5360 0x7ff6b00b7820 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4108020 0x7ff6c410fdb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6c41076a0 0x7ff6c410f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 --2- 172.21.15.33:0/1136868779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff6c4101c90 0x7ff6c410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 >> 172.21.15.33:0/1136868779 conn(0x7ff6c40fd700 msgr2=0x7ff6c40ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 shutdown_connections 2024-09-10T21:04:07.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:07.273+0000 7ff6cb7d0640 1 -- 172.21.15.33:0/1136868779 wait complete. 2024-09-10T21:04:07.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:07 smithi033 ceph-mon[30860]: pgmap v350: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:04:07.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:07 smithi104 ceph-mon[35325]: pgmap v350: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:04:07.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:07 smithi174 ceph-mon[35635]: pgmap v350: 97 pgs: 97 active+clean; 650 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-10T21:04:08.012 INFO:tasks.cephfs.filesystem:scrub status for tag:3d0a5da9-0921-4663-ac39-3add4adc1499 - {'path': '/', 'tag': '3d0a5da9-0921-4663-ac39-3add4adc1499', 'options': 'recursive,force'} 2024-09-10T21:04:09.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:09 smithi104 ceph-mon[35325]: pgmap v351: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-10T21:04:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:09 smithi174 ceph-mon[35635]: pgmap v351: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-10T21:04:09.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:09 smithi033 ceph-mon[30860]: pgmap v351: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-09-10T21:04:10.280 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:10 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:04:10.285 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:10 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:04:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:10 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: pgmap v352: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:04:12.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:04:12.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:11 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: pgmap v352: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:04:12.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:04:12.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:11 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: pgmap v352: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:04:12.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:11 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:04:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:13 smithi174 ceph-mon[35635]: pgmap v353: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T21:04:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:13 smithi033 ceph-mon[30860]: pgmap v353: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T21:04:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:13 smithi104 ceph-mon[35325]: pgmap v353: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 853 B/s wr, 9 op/s 2024-09-10T21:04:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:15 smithi174 ceph-mon[35635]: pgmap v354: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T21:04:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:15 smithi033 ceph-mon[30860]: pgmap v354: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T21:04:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:15 smithi104 ceph-mon[35325]: pgmap v354: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-10T21:04:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:16 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:04:17.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:16 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:04:17.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:16 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:04:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:17 smithi174 ceph-mon[35635]: pgmap v355: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T21:04:18.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:17 smithi033 ceph-mon[30860]: pgmap v355: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T21:04:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:17 smithi104 ceph-mon[35325]: pgmap v355: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 853 B/s wr, 10 op/s 2024-09-10T21:04:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:19 smithi174 ceph-mon[35635]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T21:04:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:19 smithi033 ceph-mon[30860]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T21:04:20.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:19 smithi104 ceph-mon[35325]: pgmap v356: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-10T21:04:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:21 smithi174 ceph-mon[35635]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:21 smithi033 ceph-mon[30860]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:22.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:21 smithi104 ceph-mon[35325]: pgmap v357: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:23 smithi174 ceph-mon[35635]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:23 smithi033 ceph-mon[30860]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:24.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:23 smithi104 ceph-mon[35325]: pgmap v358: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:25 smithi174 ceph-mon[35635]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:04:26.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:25 smithi033 ceph-mon[30860]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:04:26.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:25 smithi104 ceph-mon[35325]: pgmap v359: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:04:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:27 smithi174 ceph-mon[35635]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:28.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:27 smithi033 ceph-mon[30860]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:27 smithi104 ceph-mon[35325]: pgmap v360: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:30.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:29 smithi174 ceph-mon[35635]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:04:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:29 smithi033 ceph-mon[30860]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:04:30.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:29 smithi104 ceph-mon[35325]: pgmap v361: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:04:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:31 smithi033 ceph-mon[30860]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:32.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:31 smithi104 ceph-mon[35325]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:32.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:31 smithi174 ceph-mon[35635]: pgmap v362: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:32 smithi033 ceph-mon[30860]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:33.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:32 smithi104 ceph-mon[35325]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:33.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:32 smithi174 ceph-mon[35635]: pgmap v363: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:35.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:35 smithi104 ceph-mon[35325]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:04:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:35 smithi174 ceph-mon[35635]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:04:35.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:35 smithi033 ceph-mon[30860]: pgmap v364: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-10T21:04:37.479 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:37 smithi033 ceph-mon[30860]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:37.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:37 smithi104 ceph-mon[35325]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:37 smithi174 ceph-mon[35635]: pgmap v365: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:38.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:04:38.358 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:38.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 -- 172.21.15.33:0/513791823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 msgr2=0x7f7bf410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 --2- 172.21.15.33:0/513791823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf410e820 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7bec009ca0 tx=0x7f7bec02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 -- 172.21.15.33:0/513791823 shutdown_connections 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 --2- 172.21.15.33:0/513791823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf410e820 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 --2- 172.21.15.33:0/513791823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf41076a0 0x7f7bf4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 --2- 172.21.15.33:0/513791823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bf4101c90 0x7f7bf4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 -- 172.21.15.33:0/513791823 >> 172.21.15.33:0/513791823 conn(0x7f7bf40fd700 msgr2=0x7f7bf40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 -- 172.21.15.33:0/513791823 shutdown_connections 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.955+0000 7f7bf9208640 1 -- 172.21.15.33:0/513791823 wait complete. 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.956+0000 7f7bf9208640 1 Processor -- start 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.956+0000 7f7bf9208640 1 -- start start 2024-09-10T21:04:38.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.957+0000 7f7bf9208640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4101c90 0x7f7bf4199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.957+0000 7f7bf9208640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bf41076a0 0x7f7bf4199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.957+0000 7f7bf9208640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf419e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.957+0000 7f7bf9208640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf4111ce0 con 0x7f7bf4108020 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.957+0000 7f7bf9208640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf4111b60 con 0x7f7bf41076a0 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.957+0000 7f7bf9208640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf4111e60 con 0x7f7bf4101c90 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.957+0000 7f7bf2d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4101c90 0x7f7bf4199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf419e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf419e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49430/0 (socket says 172.21.15.33:49430) 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 -- 172.21.15.33:0/1486643916 learned_addr learned my addr 172.21.15.33:0/1486643916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf259c640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bf41076a0 0x7f7bf4199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 -- 172.21.15.33:0/1486643916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4101c90 msgr2=0x7f7bf4199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4101c90 0x7f7bf4199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 -- 172.21.15.33:0/1486643916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bf41076a0 msgr2=0x7f7bf4199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bf41076a0 0x7f7bf4199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.958+0000 7f7bf359e640 1 -- 172.21.15.33:0/1486643916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf419f0d0 con 0x7f7bf4108020 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.959+0000 7f7bf359e640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf419e900 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7bec004af0 tx=0x7f7bec031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.959+0000 7f7bdbfff640 1 -- 172.21.15.33:0/1486643916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec003c20 con 0x7f7bf4108020 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.959+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf419f360 con 0x7f7bf4108020 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.959+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf41a5a80 con 0x7f7bf4108020 2024-09-10T21:04:38.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.960+0000 7f7bdbfff640 1 -- 172.21.15.33:0/1486643916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bec003dc0 con 0x7f7bf4108020 2024-09-10T21:04:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.962+0000 7f7bdbfff640 1 -- 172.21.15.33:0/1486643916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec02fba0 con 0x7f7bf4108020 2024-09-10T21:04:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.963+0000 7f7bfb5dd740 1 Processor -- start 2024-09-10T21:04:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.965+0000 7f7bdbfff640 1 -- 172.21.15.33:0/1486643916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7bec004040 con 0x7f7bf4108020 2024-09-10T21:04:38.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.965+0000 7f7bfb5dd740 1 -- start start 2024-09-10T21:04:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.965+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4777480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.965+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47779c0 0x55b6c4787820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c478a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6c46ede50 con 0x55b6c47770a0 2024-09-10T21:04:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b6c46dd280 con 0x55b6c47779c0 2024-09-10T21:04:38.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6c46e1090 con 0x55b6c4787d60 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c478a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c478a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36014/0 (socket says 172.21.15.33:36014) 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bf259c640 1 -- 172.21.15.33:0/929236281 learned_addr learned my addr 172.21.15.33:0/929236281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bf259c640 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4777480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.966+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47779c0 0x55b6c4787820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bf259c640 1 -- 172.21.15.33:0/929236281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47779c0 msgr2=0x55b6c4787820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bf259c640 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47779c0 0x55b6c4787820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bf259c640 1 -- 172.21.15.33:0/929236281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 msgr2=0x55b6c4777480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bf259c640 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4777480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bf259c640 1 -- 172.21.15.33:0/929236281 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c443b650 con 0x55b6c4787d60 2024-09-10T21:04:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bf259c640 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c478a150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7be8011810 tx=0x7f7be80097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dee5faf21e0e4c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bd97fa640 1 -- 172.21.15.33:0/929236281 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004030 con 0x55b6c4787d60 2024-09-10T21:04:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bdbfff640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bdc0b5200 0x7f7bdc0b76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bdc0b5200 0x7f7bdc0b76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.967+0000 7f7bdbfff640 1 -- 172.21.15.33:0/1486643916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7bec106e50 con 0x7f7bf4108020 2024-09-10T21:04:38.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.968+0000 7f7bd97fa640 1 -- 172.21.15.33:0/929236281 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8005020 con 0x55b6c4787d60 2024-09-10T21:04:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.968+0000 7f7bd97fa640 1 -- 172.21.15.33:0/929236281 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8004c60 con 0x55b6c4787d60 2024-09-10T21:04:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.968+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/929236281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 msgr2=0x55b6c478a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.968+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c478a150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7be8011810 tx=0x7f7be80097a0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/929236281 shutdown_connections 2024-09-10T21:04:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c478a150 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47779c0 0x55b6c4787820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/929236281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4777480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bdc0b5200 0x7f7bdc0b76c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55b6c477a220 tx=0x7f7be4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/929236281 >> 172.21.15.33:0/929236281 conn(0x55b6c4774ed0 msgr2=0x55b6c4776070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/929236281 shutdown_connections 2024-09-10T21:04:38.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.969+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/929236281 wait complete. 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.970+0000 7f7bfb5dd740 1 -- start start 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.970+0000 7f7bfb5dd740 1 Processor -- start 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.971+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4787260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.971+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4787800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.971+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c4787d60 0x55b6c47808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.971+0000 7f7bf2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4787260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.971+0000 7f7bf2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4787260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49452/0 (socket says 172.21.15.33:49452) 2024-09-10T21:04:38.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.971+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4787800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.971+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4787800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36018/0 (socket says 172.21.15.33:36018) 2024-09-10T21:04:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf2d9d640 1 -- 172.21.15.33:0/4223429247 learned_addr learned my addr 172.21.15.33:0/4223429247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b6c453e700 con 0x55b6c47770a0 2024-09-10T21:04:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b6c4678110 con 0x55b6c4787d60 2024-09-10T21:04:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6c478aa10 con 0x55b6c47779c0 2024-09-10T21:04:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bfb5dd740 10 client.0 _ll_register_callbacks cb 0x55b6c4691990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf359e640 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c4787d60 0x55b6c47808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf2d9d640 1 -- 172.21.15.33:0/4223429247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 msgr2=0x55b6c4787800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4787800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf2d9d640 1 -- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c4787d60 msgr2=0x55b6c47808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c4787d60 0x55b6c47808c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf2d9d640 1 -- 172.21.15.33:0/4223429247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c46a5c40 con 0x55b6c47770a0 2024-09-10T21:04:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4787260 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7be40097f0 tx=0x7f7be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd022ba8befd797 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:38.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.972+0000 7f7bd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:38.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/4223429247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be401ee50 con 0x55b6c47770a0 2024-09-10T21:04:38.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8005020 con 0x55b6c47770a0 2024-09-10T21:04:38.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/4223429247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7be4004020 con 0x55b6c47770a0 2024-09-10T21:04:38.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/4223429247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7be4004220 con 0x55b6c47770a0 2024-09-10T21:04:38.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 10 client.15417 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:38.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6c479a8d0 con 0x55b6c47770a0 2024-09-10T21:04:38.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/4223429247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8005020 con 0x55b6c47770a0 2024-09-10T21:04:38.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/4223429247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7be401a040 con 0x55b6c47770a0 2024-09-10T21:04:38.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 10 client.15417 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:38.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 10 client.15417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 20 client.15417 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c479e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 4 client.15417 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.973+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6c46efda0 con 0x55b6c479bd80 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.974+0000 7f7bf259c640 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c479e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.974+0000 7f7bf259c640 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c479e200 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4adc8866fd8d7629 server_cookie=2202c21d3e219991 in_seq=0 out_seq=0 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.974+0000 7f7bd6ff5640 10 client.15417 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/4223429247 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6c46efda0 con 0x55b6c479bd80 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bd6ff5640 10 client.15417 handle_command_reply: tid=0 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 client.15417 shutdown 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 msgr2=0x55b6c4787260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4787260 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7be40097f0 tx=0x7f7be40046c0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 shutdown_connections 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c479e200 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bd6ff5640 0 client.15417 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c4787d60 0x55b6c47808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4787800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/4223429247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47770a0 0x55b6c4787260 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 >> 172.21.15.33:0/4223429247 conn(0x55b6c4774ed0 msgr2=0x55b6c477aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 shutdown_connections 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.975+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/4223429247 wait complete. 2024-09-10T21:04:38.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.976+0000 7f7bfb5dd740 20 client.15417 trim_cache size 0 max 16384 2024-09-10T21:04:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 Processor -- start 2024-09-10T21:04:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 -- start start 2024-09-10T21:04:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4776460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47779c0 0x55b6c4770060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c477d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7be000e4c0 con 0x55b6c47779c0 2024-09-10T21:04:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7be0010000 con 0x55b6c47770a0 2024-09-10T21:04:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7be000eb90 con 0x55b6c4787d60 2024-09-10T21:04:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bf2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4776460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47779c0 0x55b6c4770060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47779c0 0x55b6c4770060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49468/0 (socket says 172.21.15.33:49468) 2024-09-10T21:04:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.994+0000 7f7bf259c640 1 -- 172.21.15.33:0/1461460568 learned_addr learned my addr 172.21.15.33:0/1461460568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:39.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bf359e640 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c477d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bf259c640 1 -- 172.21.15.33:0/1461460568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 msgr2=0x55b6c477d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bf259c640 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c477d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bf259c640 1 -- 172.21.15.33:0/1461460568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 msgr2=0x55b6c4776460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bf259c640 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4776460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bf259c640 1 -- 172.21.15.33:0/1461460568 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6c477f610 con 0x55b6c47779c0 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bf259c640 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47779c0 0x55b6c4770060 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7be800a010 tx=0x7f7be8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474e3c27a566e9a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bd8ff9640 1 -- 172.21.15.33:0/1461460568 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be802e040 con 0x55b6c47779c0 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bd8ff9640 1 -- 172.21.15.33:0/1461460568 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8011250 con 0x55b6c47779c0 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.995+0000 7f7bd8ff9640 1 -- 172.21.15.33:0/1461460568 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be802c970 con 0x55b6c47779c0 2024-09-10T21:04:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/1461460568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47779c0 msgr2=0x55b6c4770060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:39.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47779c0 0x55b6c4770060 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7be800a010 tx=0x7f7be8005850 comp rx=0 tx=0).stop 2024-09-10T21:04:39.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/1461460568 shutdown_connections 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c4787d60 0x55b6c477d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c47779c0 0x55b6c4770060 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/1461460568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4776460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/1461460568 >> 172.21.15.33:0/1461460568 conn(0x55b6c4774ed0 msgr2=0x55b6c4775d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/1461460568 shutdown_connections 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.996+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/1461460568 wait complete. 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 -- start start 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 Processor -- start 2024-09-10T21:04:39.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4798eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:39.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4795590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c4787d60 0x55b6c4792dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7be000f930 con 0x55b6c4787d60 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7be000f260 con 0x55b6c47770a0 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7be000c480 con 0x55b6c47779c0 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bfb5dd740 10 client.0 _ll_register_callbacks cb 0x7f7be4009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bf2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4798eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bf359e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c4787d60 0x55b6c4792dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4795590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bf259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4795590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36030/0 (socket says 172.21.15.33:36030) 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.997+0000 7f7bf259c640 1 -- 172.21.15.33:0/3823598408 learned_addr learned my addr 172.21.15.33:0/3823598408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bf359e640 1 -- 172.21.15.33:0/3823598408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 msgr2=0x55b6c4795590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:39.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bf359e640 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4795590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bf359e640 1 -- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 msgr2=0x55b6c4798eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bf359e640 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4798eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bf359e640 1 -- 172.21.15.33:0/3823598408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6c4795c20 con 0x55b6c4787d60 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:04:39.007 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:39.008 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:04:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4798eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bf359e640 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c4787d60 0x55b6c4792dc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7bec1048e0 tx=0x7f7bec0cd430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0c8af666bfb08c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bd6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/3823598408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec0cd170 con 0x55b6c4787d60 2024-09-10T21:04:39.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/3823598408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7bec0d66d0 con 0x55b6c4787d60 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.998+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8011250 con 0x55b6c4787d60 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/3823598408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7bec0cdd60 con 0x55b6c4787d60 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 10 client.15423 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6c4801680 con 0x55b6c4787d60 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/3823598408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bec036030 con 0x55b6c4787d60 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/3823598408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f7bec0cc8c0 con 0x55b6c4787d60 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 10 client.15423 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 20 client.15423 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c4803dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 4 client.15423 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:04:39.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7be8011250 con 0x55b6c479bd80 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:38.999+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c4803dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.000+0000 7f7bf2d9d640 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c4803dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a3f1aa8e0e4999d server_cookie=4be792e0f11aec7f in_seq=0 out_seq=0 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.000+0000 7f7bd6ff5640 10 client.15423 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.000+0000 7f7bd6ff5640 1 -- 172.21.15.33:0/3823598408 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x7f7be8011250 con 0x55b6c479bd80 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.000+0000 7f7bd6ff5640 10 client.15423 handle_command_reply: tid=0 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.000+0000 7f7bfb5dd740 1 client.15423 shutdown 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.000+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c4787d60 msgr2=0x55b6c4792dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:39.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.000+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c4787d60 0x55b6c4792dc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7bec1048e0 tx=0x7f7bec0cd430 comp rx=0 tx=0).stop 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 shutdown_connections 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b6c479bd80 0x55b6c4803dd0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bd6ff5640 0 client.15423 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b6c4787d60 0x55b6c4792dc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6c47779c0 0x55b6c4795590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 --2- 172.21.15.33:0/3823598408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b6c47770a0 0x55b6c4798eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 >> 172.21.15.33:0/3823598408 conn(0x55b6c4774ed0 msgr2=0x55b6c4773700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 shutdown_connections 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 1 -- 172.21.15.33:0/3823598408 wait complete. 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.001+0000 7f7bfb5dd740 20 client.15423 trim_cache size 0 max 16384 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bdc0b5200 msgr2=0x7f7bdc0b76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:39.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bdc0b5200 0x7f7bdc0b76c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x55b6c477a220 tx=0x7f7be4005f90 comp rx=0 tx=0).stop 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 msgr2=0x7f7bf419e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf419e900 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7bec004af0 tx=0x7f7bec031a10 comp rx=0 tx=0).stop 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 shutdown_connections 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bdc0b5200 0x7f7bdc0b76c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7bf4108020 0x7f7bf419e900 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bf41076a0 0x7f7bf4199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 --2- 172.21.15.33:0/1486643916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4101c90 0x7f7bf4199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.002+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 >> 172.21.15.33:0/1486643916 conn(0x7f7bf40fd700 msgr2=0x7f7bf406aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.003+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 shutdown_connections 2024-09-10T21:04:39.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:39.003+0000 7f7bf9208640 1 -- 172.21.15.33:0/1486643916 wait complete. 2024-09-10T21:04:39.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:39 smithi033 ceph-mon[30860]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:04:39.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:39 smithi104 ceph-mon[35325]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:04:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:39 smithi174 ceph-mon[35635]: pgmap v366: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-10T21:04:39.742 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:04:39.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:04:40.022 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:40.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.543+0000 7f0745ed0640 1 -- 172.21.15.33:0/2682506997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0740102dc0 msgr2=0x7f07401031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:40.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.543+0000 7f0745ed0640 1 --2- 172.21.15.33:0/2682506997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0740102dc0 0x7f07401031a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f0730009900 tx=0x7f073002a410 comp rx=0 tx=0).stop 2024-09-10T21:04:40.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 -- 172.21.15.33:0/2682506997 shutdown_connections 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 --2- 172.21.15.33:0/2682506997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0740109150 0x7f074010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 --2- 172.21.15.33:0/2682506997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07401087d0 0x7f0740108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 --2- 172.21.15.33:0/2682506997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0740102dc0 0x7f07401031a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 -- 172.21.15.33:0/2682506997 >> 172.21.15.33:0/2682506997 conn(0x7f07400fe870 msgr2=0x7f0740100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 -- 172.21.15.33:0/2682506997 shutdown_connections 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 -- 172.21.15.33:0/2682506997 wait complete. 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.544+0000 7f0745ed0640 1 Processor -- start 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f0745ed0640 1 -- start start 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f0745ed0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0740102dc0 0x7f07401992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f0745ed0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07401087d0 0x7f0740199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:40.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f073effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07401087d0 0x7f0740199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f073effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07401087d0 0x7f0740199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43518/0 (socket says 172.21.15.33:43518) 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f0745ed0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0740109150 0x7f074019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f073effd640 1 -- 172.21.15.33:0/3584001771 learned_addr learned my addr 172.21.15.33:0/3584001771 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0740112e60 con 0x7f07401087d0 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0740112ce0 con 0x7f0740109150 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.545+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0740112fe0 con 0x7f0740102dc0 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073f7fe640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0740102dc0 0x7f07401992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073ffff640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0740109150 0x7f074019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073effd640 1 -- 172.21.15.33:0/3584001771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0740102dc0 msgr2=0x7f07401992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073effd640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0740102dc0 0x7f07401992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073effd640 1 -- 172.21.15.33:0/3584001771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0740109150 msgr2=0x7f074019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073effd640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0740109150 0x7f074019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073effd640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f074019f380 con 0x7f07401087d0 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073ffff640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0740109150 0x7f074019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073effd640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07401087d0 0x7f0740199810 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9f0 tx=0x7f073400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.546+0000 7f073cff9640 1 -- 172.21.15.33:0/3584001771 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073400cd80 con 0x7f07401087d0 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.547+0000 7f073cff9640 1 -- 172.21.15.33:0/3584001771 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07340108e0 con 0x7f07401087d0 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.547+0000 7f073cff9640 1 -- 172.21.15.33:0/3584001771 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734018980 con 0x7f07401087d0 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.547+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f074019f670 con 0x7f07401087d0 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.548+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0740075380 con 0x7f07401087d0 2024-09-10T21:04:40.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.549+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0740104070 con 0x7f07401087d0 2024-09-10T21:04:40.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.551+0000 7f073cff9640 1 -- 172.21.15.33:0/3584001771 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0734010430 con 0x7f07401087d0 2024-09-10T21:04:40.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.553+0000 7f073cff9640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07280b5230 0x7f07280b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:40.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.553+0000 7f073f7fe640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07280b5230 0x7f07280b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:40.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.553+0000 7f073cff9640 1 -- 172.21.15.33:0/3584001771 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f07340e5f30 con 0x7f07401087d0 2024-09-10T21:04:40.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.553+0000 7f073f7fe640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07280b5230 0x7f07280b76f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0730005ab0 tx=0x7f0730005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:40.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.557+0000 7f073cff9640 1 -- 172.21.15.33:0/3584001771 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f07340acdb0 con 0x7f07401087d0 2024-09-10T21:04:40.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.780+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f074019a400 con 0x7f07401087d0 2024-09-10T21:04:40.785 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:04:40.786 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"btime":"2024-09-10T21:03:57:162551+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:03:56.306223+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:04:40.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.781+0000 7f073cff9640 1 -- 172.21.15.33:0/3584001771 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11973 (secure 0 0 0) 0x7f07340ac6b0 con 0x7f07401087d0 2024-09-10T21:04:40.786 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-09-10T21:04:40.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07280b5230 msgr2=0x7f07280b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:40.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07280b5230 0x7f07280b76f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f0730005ab0 tx=0x7f0730005a20 comp rx=0 tx=0).stop 2024-09-10T21:04:40.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07401087d0 msgr2=0x7f0740199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:40.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07401087d0 0x7f0740199810 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f073400e9f0 tx=0x7f073400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 shutdown_connections 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07280b5230 0x7f07280b76f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0740109150 0x7f074019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07401087d0 0x7f0740199810 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 --2- 172.21.15.33:0/3584001771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0740102dc0 0x7f07401992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.784+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 >> 172.21.15.33:0/3584001771 conn(0x7f07400fe870 msgr2=0x7f0740100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.785+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 shutdown_connections 2024-09-10T21:04:40.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:40.785+0000 7f0745ed0640 1 -- 172.21.15.33:0/3584001771 wait complete. 2024-09-10T21:04:41.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:41 smithi033 ceph-mon[30860]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:41.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:41 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3584001771' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:41.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:04:41.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:41 smithi104 ceph-mon[35325]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:41.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:41 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3584001771' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:41 smithi174 ceph-mon[35635]: pgmap v367: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-10T21:04:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:41 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3584001771' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:04:41.825 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:42.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.356+0000 7feb93d15640 1 -- 172.21.15.33:0/1234967226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c1087f0 msgr2=0x7feb8c108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.356+0000 7feb93d15640 1 --2- 172.21.15.33:0/1234967226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c1087f0 0x7feb8c108c30 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7feb7c009900 tx=0x7feb7c02a410 comp rx=0 tx=0).stop 2024-09-10T21:04:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.356+0000 7feb93d15640 1 -- 172.21.15.33:0/1234967226 shutdown_connections 2024-09-10T21:04:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.356+0000 7feb93d15640 1 --2- 172.21.15.33:0/1234967226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb8c109170 0x7feb8c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.356+0000 7feb93d15640 1 --2- 172.21.15.33:0/1234967226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c1087f0 0x7feb8c108c30 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.356+0000 7feb93d15640 1 --2- 172.21.15.33:0/1234967226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c102de0 0x7feb8c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.356+0000 7feb93d15640 1 -- 172.21.15.33:0/1234967226 >> 172.21.15.33:0/1234967226 conn(0x7feb8c0fe830 msgr2=0x7feb8c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.357+0000 7feb93d15640 1 -- 172.21.15.33:0/1234967226 shutdown_connections 2024-09-10T21:04:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.357+0000 7feb93d15640 1 -- 172.21.15.33:0/1234967226 wait complete. 2024-09-10T21:04:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.357+0000 7feb93d15640 1 Processor -- start 2024-09-10T21:04:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.357+0000 7feb93d15640 1 -- start start 2024-09-10T21:04:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb93d15640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c102de0 0x7feb8c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb93d15640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb8c1087f0 0x7feb8c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb93d15640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c109170 0x7feb8c19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb93d15640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c112df0 con 0x7feb8c102de0 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb93d15640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c112c70 con 0x7feb8c109170 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb93d15640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c112f70 con 0x7feb8c1087f0 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb922b2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c109170 0x7feb8c19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb922b2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c109170 0x7feb8c19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57192/0 (socket says 172.21.15.33:57192) 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb922b2640 1 -- 172.21.15.33:0/483303150 learned_addr learned my addr 172.21.15.33:0/483303150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb91ab1640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c102de0 0x7feb8c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.358+0000 7feb912b0640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb8c1087f0 0x7feb8c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb922b2640 1 -- 172.21.15.33:0/483303150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb8c1087f0 msgr2=0x7feb8c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb922b2640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb8c1087f0 0x7feb8c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb922b2640 1 -- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c102de0 msgr2=0x7feb8c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb922b2640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c102de0 0x7feb8c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb922b2640 1 -- 172.21.15.33:0/483303150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb8c19f310 con 0x7feb8c109170 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb91ab1640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c102de0 0x7feb8c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb922b2640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c109170 0x7feb8c19eb40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb8400eb10 tx=0x7feb8400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.359+0000 7feb82ffd640 1 -- 172.21.15.33:0/483303150 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84004b20 con 0x7feb8c109170 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.360+0000 7feb82ffd640 1 -- 172.21.15.33:0/483303150 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb84004cc0 con 0x7feb8c109170 2024-09-10T21:04:42.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.360+0000 7feb82ffd640 1 -- 172.21.15.33:0/483303150 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84010690 con 0x7feb8c109170 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.360+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb8c19f600 con 0x7feb8c109170 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.360+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb8c1a5cc0 con 0x7feb8c109170 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.363+0000 7feb960ea740 1 Processor -- start 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb82ffd640 1 -- 172.21.15.33:0/483303150 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7feb84005020 con 0x7feb8c109170 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb960ea740 1 -- start start 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb960ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d994d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb960ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3da5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb960ea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3da59a0 0x560ba3da7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb960ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ba3d0fea0 con 0x560ba3d99a10 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb960ea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560ba3cff2d0 con 0x560ba3da59a0 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.364+0000 7feb960ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ba3d030e0 con 0x560ba3d990f0 2024-09-10T21:04:42.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.365+0000 7feb82ffd640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feb780b52e0 0x7feb780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.365+0000 7feb922b2640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feb780b52e0 0x7feb780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.365+0000 7feb82ffd640 1 -- 172.21.15.33:0/483303150 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feb840e6150 con 0x7feb8c109170 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.365+0000 7feb912b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.365+0000 7feb912b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:60896/0 (socket says 172.21.15.33:60896) 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.365+0000 7feb912b0640 1 -- 172.21.15.33:0/2307501738 learned_addr learned my addr 172.21.15.33:0/2307501738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.365+0000 7feb912b0640 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3da59a0 0x560ba3da7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb91ab1640 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3da5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb912b0640 1 -- 172.21.15.33:0/2307501738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3da59a0 msgr2=0x560ba3da7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb912b0640 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3da59a0 0x560ba3da7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb912b0640 1 -- 172.21.15.33:0/2307501738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 msgr2=0x560ba3da5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb912b0640 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3da5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb912b0640 1 -- 172.21.15.33:0/2307501738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba3a5d650 con 0x560ba3d990f0 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb91ab1640 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3da5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb922b2640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feb780b52e0 0x7feb780b77a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feb840ac8a0 tx=0x7feb84071080 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.366+0000 7feb912b0640 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d994d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02a930 tx=0x7feb7c005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7756abcdd92ff21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.367+0000 7feb77fff640 1 -- 172.21.15.33:0/2307501738 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c035030 con 0x560ba3d990f0 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.367+0000 7feb77fff640 1 -- 172.21.15.33:0/2307501738 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7c0035f0 con 0x560ba3d990f0 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.367+0000 7feb77fff640 1 -- 172.21.15.33:0/2307501738 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c035030 con 0x560ba3d990f0 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.367+0000 7feb960ea740 1 -- 172.21.15.33:0/2307501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 msgr2=0x560ba3d994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.367+0000 7feb960ea740 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d994d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feb7c02a930 tx=0x7feb7c005b50 comp rx=0 tx=0).stop 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 -- 172.21.15.33:0/2307501738 shutdown_connections 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3da59a0 0x560ba3da7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3da5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 --2- 172.21.15.33:0/2307501738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d994d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 -- 172.21.15.33:0/2307501738 >> 172.21.15.33:0/2307501738 conn(0x560ba3d96f20 msgr2=0x560ba3d980c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 -- 172.21.15.33:0/2307501738 shutdown_connections 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 -- 172.21.15.33:0/2307501738 wait complete. 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 -- start start 2024-09-10T21:04:42.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.368+0000 7feb960ea740 1 Processor -- start 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb960ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3da5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb960ea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3da26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb960ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb960ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560ba3b60700 con 0x560ba3da59a0 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb960ea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560ba3c9a160 con 0x560ba3d99a10 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb960ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ba3da8610 con 0x560ba3d990f0 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb960ea740 10 client.0 _ll_register_callbacks cb 0x560ba3cb39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43550/0 (socket says 172.21.15.33:43550) 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 -- 172.21.15.33:0/4022256887 learned_addr learned my addr 172.21.15.33:0/4022256887 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:42.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb91ab1640 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3da26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3da5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 -- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 msgr2=0x560ba3da5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3da5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 -- 172.21.15.33:0/4022256887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 msgr2=0x560ba3da26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3da26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.369+0000 7feb912b0640 1 -- 172.21.15.33:0/4022256887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba3cc7c90 con 0x560ba3da59a0 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb912b0640 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da2d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feb7c031440 tx=0x7feb7c003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385dd4aecef5c4b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:42.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb757fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb757fa640 1 -- 172.21.15.33:0/4022256887 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7c035030 con 0x560ba3da59a0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb757fa640 1 -- 172.21.15.33:0/4022256887 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feb7c0039b0 con 0x560ba3da59a0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb7c0035f0 con 0x560ba3da59a0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb757fa640 1 -- 172.21.15.33:0/4022256887 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb7c003bb0 con 0x560ba3da59a0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb960ea740 10 client.15438 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ba3db3ea0 con 0x560ba3da59a0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.370+0000 7feb757fa640 1 -- 172.21.15.33:0/4022256887 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb7c0035f0 con 0x560ba3da59a0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb757fa640 1 -- 172.21.15.33:0/4022256887 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb7c04b990 con 0x560ba3da59a0 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb960ea740 10 client.15438 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb960ea740 10 client.15438 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb960ea740 20 client.15438 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:42.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb960ea740 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3db75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb960ea740 4 client.15438 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ba3d11df0 con 0x560ba3db51f0 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.371+0000 7feb91ab1640 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3db75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.372+0000 7feb757fa640 10 client.15438 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.372+0000 7feb91ab1640 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3db75e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=986f9b602cc7b34d server_cookie=f5a055b38c376e79 in_seq=0 out_seq=0 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb757fa640 1 -- 172.21.15.33:0/4022256887 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560ba3d11df0 con 0x560ba3db51f0 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb757fa640 10 client.15438 handle_command_reply: tid=0 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 client.15438 shutdown 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 msgr2=0x560ba3da2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da2d00 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7feb7c031440 tx=0x7feb7c003f00 comp rx=0 tx=0).stop 2024-09-10T21:04:42.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 shutdown_connections 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3db75e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb757fa640 0 client.15438 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da2d00 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3da26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 --2- 172.21.15.33:0/4022256887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3da5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 >> 172.21.15.33:0/4022256887 conn(0x560ba3d96f20 msgr2=0x560ba3d9cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 shutdown_connections 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 1 -- 172.21.15.33:0/4022256887 wait complete. 2024-09-10T21:04:42.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.373+0000 7feb960ea740 20 client.15438 trim_cache size 0 max 16384 2024-09-10T21:04:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 Processor -- start 2024-09-10T21:04:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 -- start start 2024-09-10T21:04:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d96760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3d984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1ae3e0 con 0x560ba3da59a0 2024-09-10T21:04:42.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1aff20 con 0x560ba3d99a10 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb960ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1aeab0 con 0x560ba3d990f0 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb912b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb912b0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43556/0 (socket says 172.21.15.33:43556) 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb912b0640 1 -- 172.21.15.33:0/1543496405 learned_addr learned my addr 172.21.15.33:0/1543496405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.387+0000 7feb91ab1640 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3d984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb91ab1640 1 -- 172.21.15.33:0/1543496405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 msgr2=0x560ba3d96760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb91ab1640 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d96760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb91ab1640 1 -- 172.21.15.33:0/1543496405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 msgr2=0x560ba3da6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb91ab1640 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb91ab1640 1 -- 172.21.15.33:0/1543496405 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ba3da0bd0 con 0x560ba3d99a10 2024-09-10T21:04:42.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb912b0640 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb91ab1640 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3d984a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x560ba3c9a1c0 tx=0x7feb88005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=573d21e3000454b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb777fe640 1 -- 172.21.15.33:0/1543496405 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88004050 con 0x560ba3d99a10 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb777fe640 1 -- 172.21.15.33:0/1543496405 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb880041f0 con 0x560ba3d99a10 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.388+0000 7feb777fe640 1 -- 172.21.15.33:0/1543496405 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8801e6d0 con 0x560ba3d99a10 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.389+0000 7feb960ea740 1 -- 172.21.15.33:0/1543496405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 msgr2=0x560ba3d984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.389+0000 7feb960ea740 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3d984a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x560ba3c9a1c0 tx=0x7feb88005d30 comp rx=0 tx=0).stop 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.389+0000 7feb960ea740 1 -- 172.21.15.33:0/1543496405 shutdown_connections 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.389+0000 7feb960ea740 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3da59a0 0x560ba3da6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.389+0000 7feb960ea740 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d99a10 0x560ba3d984a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.389+0000 7feb960ea740 1 --2- 172.21.15.33:0/1543496405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3d990f0 0x560ba3d96760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.389+0000 7feb960ea740 1 -- 172.21.15.33:0/1543496405 >> 172.21.15.33:0/1543496405 conn(0x560ba3d96f20 msgr2=0x560ba3da4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.390+0000 7feb960ea740 1 -- 172.21.15.33:0/1543496405 shutdown_connections 2024-09-10T21:04:42.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.390+0000 7feb960ea740 1 -- 172.21.15.33:0/1543496405 wait complete. 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.390+0000 7feb960ea740 1 -- start start 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.390+0000 7feb960ea740 1 Processor -- start 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb960ea740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d990f0 0x560ba3e1b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb960ea740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3e1b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb960ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3da59a0 0x560ba3e1f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb960ea740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1af850 con 0x560ba3d99a10 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb960ea740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1af180 con 0x560ba3d990f0 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb960ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb8c1ac3a0 con 0x560ba3da59a0 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb960ea740 10 client.0 _ll_register_callbacks cb 0x7feb7c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb912b0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d990f0 0x560ba3e1b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb91ab1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3e1b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb91ab1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3e1b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43570/0 (socket says 172.21.15.33:43570) 2024-09-10T21:04:42.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.391+0000 7feb91ab1640 1 -- 172.21.15.33:0/3397169286 learned_addr learned my addr 172.21.15.33:0/3397169286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:42.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb91ab1640 1 -- 172.21.15.33:0/3397169286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3da59a0 msgr2=0x560ba3e1f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb91ab1640 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3da59a0 0x560ba3e1f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb91ab1640 1 -- 172.21.15.33:0/3397169286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d990f0 msgr2=0x560ba3e1b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb91ab1640 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d990f0 0x560ba3e1b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb91ab1640 1 -- 172.21.15.33:0/3397169286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ba3e1f890 con 0x560ba3d99a10 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb91ab1640 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3e1b660 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feb88022040 tx=0x7feb88004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6feadb27080a0951 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb757fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb757fa640 1 -- 172.21.15.33:0/3397169286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb88003a80 con 0x560ba3d99a10 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb757fa640 1 -- 172.21.15.33:0/3397169286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feb88003c20 con 0x560ba3d99a10 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.392+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb880041f0 con 0x560ba3d99a10 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.393+0000 7feb757fa640 1 -- 172.21.15.33:0/3397169286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb8801e470 con 0x560ba3d99a10 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.393+0000 7feb960ea740 10 client.15444 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:42.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.393+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560ba3e26d10 con 0x560ba3d99a10 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.393+0000 7feb757fa640 1 -- 172.21.15.33:0/3397169286 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb880041f0 con 0x560ba3d99a10 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb757fa640 1 -- 172.21.15.33:0/3397169286 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7feb8802d020 con 0x560ba3d99a10 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb960ea740 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb960ea740 10 client.15444 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb960ea740 20 client.15444 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb960ea740 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3e293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb960ea740 4 client.15444 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ba3d0d8a0 con 0x560ba3db51f0 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb912b0640 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3e293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb912b0640 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3e293a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=38e394c2e6e98705 server_cookie=cf58ad00d02cea77 in_seq=0 out_seq=0 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.394+0000 7feb757fa640 10 client.15444 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb757fa640 1 -- 172.21.15.33:0/3397169286 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ba3d0d8a0 con 0x560ba3db51f0 2024-09-10T21:04:42.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb757fa640 10 client.15444 handle_command_reply: tid=0 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 client.15444 shutdown 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 msgr2=0x560ba3e1b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3e1b660 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feb88022040 tx=0x7feb88004500 comp rx=0 tx=0).stop 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 shutdown_connections 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ba3db51f0 0x560ba3e293a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb757fa640 0 client.15444 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ba3da59a0 0x560ba3e1f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ba3d99a10 0x560ba3e1b660 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 --2- 172.21.15.33:0/3397169286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ba3d990f0 0x560ba3e1b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 >> 172.21.15.33:0/3397169286 conn(0x560ba3d96f20 msgr2=0x560ba3da4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 shutdown_connections 2024-09-10T21:04:42.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.395+0000 7feb960ea740 1 -- 172.21.15.33:0/3397169286 wait complete. 2024-09-10T21:04:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.396+0000 7feb960ea740 20 client.15444 trim_cache size 0 max 16384 2024-09-10T21:04:42.404 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:04:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.396+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feb780b52e0 msgr2=0x7feb780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.396+0000 7feb93d15640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feb780b52e0 0x7feb780b77a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7feb840ac8a0 tx=0x7feb84071080 comp rx=0 tx=0).stop 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.396+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c109170 msgr2=0x7feb8c19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.396+0000 7feb93d15640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c109170 0x7feb8c19eb40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feb8400eb10 tx=0x7feb8400c2d0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 shutdown_connections 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feb780b52e0 0x7feb780b77a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb8c109170 0x7feb8c19eb40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb8c1087f0 0x7feb8c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 --2- 172.21.15.33:0/483303150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb8c102de0 0x7feb8c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 >> 172.21.15.33:0/483303150 conn(0x7feb8c0fe830 msgr2=0x7feb8c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 shutdown_connections 2024-09-10T21:04:42.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:42.397+0000 7feb93d15640 1 -- 172.21.15.33:0/483303150 wait complete. 2024-09-10T21:04:43.177 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:04:43.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:43 smithi033 ceph-mon[30860]: pgmap v368: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:43.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:43 smithi104 ceph-mon[35325]: pgmap v368: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:43 smithi174 ceph-mon[35635]: pgmap v368: 97 pgs: 97 active+clean; 656 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-09-10T21:04:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.013+0000 7f64959c5640 1 -- 172.21.15.33:0/2424664724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 msgr2=0x7f6490101ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.013+0000 7f64959c5640 1 --2- 172.21.15.33:0/2424664724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f6490101ea0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6480009900 tx=0x7f648002a410 comp rx=0 tx=0).stop 2024-09-10T21:04:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 -- 172.21.15.33:0/2424664724 shutdown_connections 2024-09-10T21:04:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 --2- 172.21.15.33:0/2424664724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6490106f00 0x7f6490107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 --2- 172.21.15.33:0/2424664724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64901023e0 0x7f64901068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 --2- 172.21.15.33:0/2424664724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f6490101ea0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 -- 172.21.15.33:0/2424664724 >> 172.21.15.33:0/2424664724 conn(0x7f64900fd770 msgr2=0x7f64900ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:44.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 -- 172.21.15.33:0/2424664724 shutdown_connections 2024-09-10T21:04:44.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 -- 172.21.15.33:0/2424664724 wait complete. 2024-09-10T21:04:44.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.014+0000 7f64959c5640 1 Processor -- start 2024-09-10T21:04:44.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f64959c5640 1 -- start start 2024-09-10T21:04:44.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f64959c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f649019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f64959c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64901023e0 0x7f649019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f64959c5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6490106f00 0x7f64901a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f64959c5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6490119390 con 0x7f6490101ac0 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f64959c5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6490119210 con 0x7f6490106f00 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f64959c5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6490119510 con 0x7f64901023e0 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f648effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f649019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f648effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f649019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43594/0 (socket says 172.21.15.33:43594) 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f648effd640 1 -- 172.21.15.33:0/1559363791 learned_addr learned my addr 172.21.15.33:0/1559363791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f648e7fc640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64901023e0 0x7f649019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f648effd640 1 -- 172.21.15.33:0/1559363791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64901023e0 msgr2=0x7f649019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.015+0000 7f648effd640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64901023e0 0x7f649019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f648effd640 1 -- 172.21.15.33:0/1559363791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6490106f00 msgr2=0x7f64901a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f648effd640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6490106f00 0x7f64901a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f648effd640 1 -- 172.21.15.33:0/1559363791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64901a5950 con 0x7f6490101ac0 2024-09-10T21:04:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f648effd640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f649019f8a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6480005940 tx=0x7f64800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f64949c3640 1 -- 172.21.15.33:0/1559363791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648003f030 con 0x7f6490101ac0 2024-09-10T21:04:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f64949c3640 1 -- 172.21.15.33:0/1559363791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6480003cb0 con 0x7f6490101ac0 2024-09-10T21:04:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64901a5be0 con 0x7f6490101ac0 2024-09-10T21:04:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f64949c3640 1 -- 172.21.15.33:0/1559363791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480035070 con 0x7f6490101ac0 2024-09-10T21:04:44.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.016+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64901ac300 con 0x7f6490101ac0 2024-09-10T21:04:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.019+0000 7f6497d9a740 1 Processor -- start 2024-09-10T21:04:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f6497d9a740 1 -- start start 2024-09-10T21:04:44.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a44b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44bdf0 0x56372a457840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a457d80 0x56372a45a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f6497d9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56372a3c1ea0 con 0x56372a457d80 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f6497d9a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56372a3b12d0 con 0x56372a44b4d0 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f6497d9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56372a3b50e0 con 0x56372a44bdf0 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f648e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a457d80 0x56372a45a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a44b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a44b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57254/0 (socket says 172.21.15.33:57254) 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.020+0000 7f648f7fe640 1 -- 172.21.15.33:0/3146439021 learned_addr learned my addr 172.21.15.33:0/3146439021 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.021+0000 7f64949c3640 1 -- 172.21.15.33:0/1559363791 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f648003bbf0 con 0x7f6490101ac0 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.021+0000 7f648e7fc640 1 -- 172.21.15.33:0/3146439021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44bdf0 msgr2=0x56372a457840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.021+0000 7f648e7fc640 1 --2- 172.21.15.33:0/3146439021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44bdf0 0x56372a457840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.021+0000 7f648e7fc640 1 -- 172.21.15.33:0/3146439021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 msgr2=0x56372a44b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.021+0000 7f648e7fc640 1 --2- 172.21.15.33:0/3146439021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a44b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.021+0000 7f648e7fc640 1 -- 172.21.15.33:0/3146439021 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56372a10f650 con 0x56372a457d80 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.021+0000 7f648e7fc640 1 --2- 172.21.15.33:0/3146439021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a457d80 0x56372a45a170 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f649006c170 tx=0x7f648400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11a5570ab5806df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.022+0000 7f64949c3640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64780b5360 0x7f64780b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.022+0000 7f6475ffb640 1 -- 172.21.15.33:0/3146439021 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484004220 con 0x56372a457d80 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.022+0000 7f6475ffb640 1 -- 172.21.15.33:0/3146439021 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64840043c0 con 0x56372a457d80 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.022+0000 7f6475ffb640 1 -- 172.21.15.33:0/3146439021 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484018860 con 0x56372a457d80 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.022+0000 7f64949c3640 1 -- 172.21.15.33:0/1559363791 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6480106740 con 0x7f6490101ac0 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.022+0000 7f648f7fe640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64780b5360 0x7f64780b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 -- 172.21.15.33:0/3146439021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a457d80 msgr2=0x56372a45a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3146439021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a457d80 0x56372a45a170 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f649006c170 tx=0x7f648400f790 comp rx=0 tx=0).stop 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f648f7fe640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64780b5360 0x7f64780b7820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64880092f0 tx=0x7f6488009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 -- 172.21.15.33:0/3146439021 shutdown_connections 2024-09-10T21:04:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3146439021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a457d80 0x56372a45a170 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3146439021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44bdf0 0x56372a457840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3146439021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a44b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 -- 172.21.15.33:0/3146439021 >> 172.21.15.33:0/3146439021 conn(0x56372a449300 msgr2=0x56372a44a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:44.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 -- 172.21.15.33:0/3146439021 shutdown_connections 2024-09-10T21:04:44.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.023+0000 7f6497d9a740 1 -- 172.21.15.33:0/3146439021 wait complete. 2024-09-10T21:04:44.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 -- start start 2024-09-10T21:04:44.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 Processor -- start 2024-09-10T21:04:44.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a457730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a454b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a457d80 0x56372a455250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56372a212700 con 0x56372a44bdf0 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56372a34c160 con 0x56372a457d80 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56372a45aa00 con 0x56372a44b4d0 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.024+0000 7f6497d9a740 10 client.0 _ll_register_callbacks cb 0x56372a3659e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a457730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a454b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a454b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43604/0 (socket says 172.21.15.33:43604) 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 -- 172.21.15.33:0/3690492049 learned_addr learned my addr 172.21.15.33:0/3690492049 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:44.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648f7fe640 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a457d80 0x56372a455250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 -- 172.21.15.33:0/3690492049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 msgr2=0x56372a457730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a457730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 -- 172.21.15.33:0/3690492049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a457d80 msgr2=0x56372a455250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:44.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a457d80 0x56372a455250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.025+0000 7f648e7fc640 1 -- 172.21.15.33:0/3690492049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56372a379c90 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.026+0000 7f648e7fc640 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a454b80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6484018520 tx=0x7f64840055c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ecf1277de8be6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.026+0000 7f646effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.026+0000 7f646effd640 1 -- 172.21.15.33:0/3690492049 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484026020 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.026+0000 7f646effd640 1 -- 172.21.15.33:0/3690492049 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f64840048c0 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.026+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64840043c0 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f646effd640 1 -- 172.21.15.33:0/3690492049 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f648401c070 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56372a466440 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f646effd640 1 -- 172.21.15.33:0/3690492049 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64840043c0 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f646effd640 1 -- 172.21.15.33:0/3690492049 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6484003ae0 con 0x56372a44bdf0 2024-09-10T21:04:44.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:44.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 10 client.15456 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:04:44.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 20 client.15456 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:44.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a4677f0 0x56372a469be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 4 client.15456 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:44.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.027+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56372a3c3df0 con 0x56372a4677f0 2024-09-10T21:04:44.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.028+0000 7f648effd640 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a4677f0 0x56372a469be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.028+0000 7f646effd640 10 client.15456 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.028+0000 7f648effd640 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a4677f0 0x56372a469be0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38c4b76f12bb909d server_cookie=9b8b66884bc5e80d in_seq=0 out_seq=0 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.029+0000 7f646effd640 1 -- 172.21.15.33:0/3690492049 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56372a3c3df0 con 0x56372a4677f0 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.029+0000 7f646effd640 10 client.15456 handle_command_reply: tid=0 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.029+0000 7f6497d9a740 1 client.15456 shutdown 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 msgr2=0x56372a454b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a454b80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6484018520 tx=0x7f64840055c0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 shutdown_connections 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a457d80 0x56372a455250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a454b80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a4677f0 0x56372a469be0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3690492049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a457730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f646effd640 0 client.15456 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:04:44.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 >> 172.21.15.33:0/3690492049 conn(0x56372a449300 msgr2=0x56372a44ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:44.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 shutdown_connections 2024-09-10T21:04:44.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 1 -- 172.21.15.33:0/3690492049 wait complete. 2024-09-10T21:04:44.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.030+0000 7f6497d9a740 20 client.15456 trim_cache size 0 max 16384 2024-09-10T21:04:44.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f6497d9a740 1 Processor -- start 2024-09-10T21:04:44.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f6497d9a740 1 -- start start 2024-09-10T21:04:44.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a449300 0x56372a463c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a44a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a4568c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f648effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a449300 0x56372a463c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f648effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a449300 0x56372a463c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57262/0 (socket says 172.21.15.33:57262) 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f648e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a44a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a4568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.057+0000 7f648e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a44a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:60968/0 (socket says 172.21.15.33:60968) 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a4568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43612/0 (socket says 172.21.15.33:43612) 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f6497d9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64901b49f0 con 0x56372a44bdf0 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f6497d9a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64901b6530 con 0x56372a449300 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f6497d9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64901b50c0 con 0x56372a44b4d0 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f648effd640 1 -- 172.21.15.33:0/925983809 learned_addr learned my addr 172.21.15.33:0/925983809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f648f7fe640 1 -- 172.21.15.33:0/925983809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 msgr2=0x56372a44a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f648f7fe640 1 --2- 172.21.15.33:0/925983809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a44a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f648f7fe640 1 -- 172.21.15.33:0/925983809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a449300 msgr2=0x56372a463c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:44.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f648f7fe640 1 --2- 172.21.15.33:0/925983809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a449300 0x56372a463c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.058+0000 7f648f7fe640 1 -- 172.21.15.33:0/925983809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56372a449fe0 con 0x56372a44bdf0 2024-09-10T21:04:44.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.059+0000 7f648f7fe640 1 --2- 172.21.15.33:0/925983809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a4568c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6488005fd0 tx=0x7f6488014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e4212478ea0deb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:44.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.059+0000 7f64757fa640 1 -- 172.21.15.33:0/925983809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488003b70 con 0x56372a44bdf0 2024-09-10T21:04:44.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.059+0000 7f64757fa640 1 -- 172.21.15.33:0/925983809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6488003d10 con 0x56372a44bdf0 2024-09-10T21:04:44.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.059+0000 7f64757fa640 1 -- 172.21.15.33:0/925983809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64880045d0 con 0x56372a44bdf0 2024-09-10T21:04:44.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 -- 172.21.15.33:0/925983809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 msgr2=0x56372a4568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 --2- 172.21.15.33:0/925983809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a4568c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6488005fd0 tx=0x7f6488014000 comp rx=0 tx=0).stop 2024-09-10T21:04:44.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 -- 172.21.15.33:0/925983809 shutdown_connections 2024-09-10T21:04:44.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 --2- 172.21.15.33:0/925983809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a4568c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 --2- 172.21.15.33:0/925983809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a44b4d0 0x56372a44a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 --2- 172.21.15.33:0/925983809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a449300 0x56372a463c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 -- 172.21.15.33:0/925983809 >> 172.21.15.33:0/925983809 conn(0x56372a469ee0 msgr2=0x56372a4561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 -- 172.21.15.33:0/925983809 shutdown_connections 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.060+0000 7f6497d9a740 1 -- 172.21.15.33:0/925983809 wait complete. 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 -- start start 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 Processor -- start 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a449300 0x56372a45fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a460470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a45ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f64901b5e60 con 0x56372a44bdf0 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f64901b5790 con 0x56372a44b4d0 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64901b29b0 con 0x56372a449300 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f6497d9a740 10 client.0 _ll_register_callbacks cb 0x7f6488009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a45ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f648f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a45ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43614/0 (socket says 172.21.15.33:43614) 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f648f7fe640 1 -- 172.21.15.33:0/3181830723 learned_addr learned my addr 172.21.15.33:0/3181830723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.061+0000 7f648e7fc640 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a460470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f648f7fe640 1 -- 172.21.15.33:0/3181830723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a449300 msgr2=0x56372a45fed0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f648f7fe640 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a449300 0x56372a45fed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f648f7fe640 1 -- 172.21.15.33:0/3181830723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 msgr2=0x56372a460470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f648f7fe640 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a460470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f648f7fe640 1 -- 172.21.15.33:0/3181830723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56372a45b4b0 con 0x56372a44bdf0 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f648f7fe640 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a45ac80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64880041b0 tx=0x7f64880125d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37efb99bd74737c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f646effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f646effd640 1 -- 172.21.15.33:0/3181830723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488004b20 con 0x56372a44bdf0 2024-09-10T21:04:44.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6488003d10 con 0x56372a44bdf0 2024-09-10T21:04:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f646effd640 1 -- 172.21.15.33:0/3181830723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6488004cc0 con 0x56372a44bdf0 2024-09-10T21:04:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f646effd640 1 -- 172.21.15.33:0/3181830723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6488004310 con 0x56372a44bdf0 2024-09-10T21:04:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f6497d9a740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.062+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56372a4d5000 con 0x56372a44bdf0 2024-09-10T21:04:44.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f646effd640 1 -- 172.21.15.33:0/3181830723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6488003d10 con 0x56372a44bdf0 2024-09-10T21:04:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f646effd640 1 -- 172.21.15.33:0/3181830723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f648800f040 con 0x56372a44bdf0 2024-09-10T21:04:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f6497d9a740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f6497d9a740 10 client.15462 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:04:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f6497d9a740 20 client.15462 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:44.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a457d80 0x56372a4d76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:44.072 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:04:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f6497d9a740 4 client.15462 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.063+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56372a44ab70 con 0x56372a457d80 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.064+0000 7f648effd640 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a457d80 0x56372a4d76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.066+0000 7f646effd640 10 client.15462 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.066+0000 7f648effd640 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a457d80 0x56372a4d76e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f6f2c1d99b6224d0 server_cookie=45abbceb05522afe in_seq=0 out_seq=0 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.066+0000 7f646effd640 1 -- 172.21.15.33:0/3181830723 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56372a44ab70 con 0x56372a457d80 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.066+0000 7f646effd640 10 client.15462 handle_command_reply: tid=0 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.066+0000 7f6497d9a740 1 client.15462 shutdown 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 msgr2=0x56372a45ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a45ac80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f64880041b0 tx=0x7f64880125d0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 shutdown_connections 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56372a44bdf0 0x56372a45ac80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56372a44b4d0 0x56372a460470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56372a457d80 0x56372a4d76e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 --2- 172.21.15.33:0/3181830723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56372a449300 0x56372a45fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f646effd640 0 client.15462 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 >> 172.21.15.33:0/3181830723 conn(0x56372a469ee0 msgr2=0x56372a4569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 shutdown_connections 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 1 -- 172.21.15.33:0/3181830723 wait complete. 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f6497d9a740 20 client.15462 trim_cache size 0 max 16384 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64780b5360 msgr2=0x7f64780b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.067+0000 7f64959c5640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64780b5360 0x7f64780b7820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64880092f0 tx=0x7f6488009600 comp rx=0 tx=0).stop 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 msgr2=0x7f649019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f649019f8a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6480005940 tx=0x7f64800042f0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 shutdown_connections 2024-09-10T21:04:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64780b5360 0x7f64780b7820 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6490106f00 0x7f64901a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64901023e0 0x7f649019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 --2- 172.21.15.33:0/1559363791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6490101ac0 0x7f649019f8a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:44.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 >> 172.21.15.33:0/1559363791 conn(0x7f64900fd770 msgr2=0x7f6490104830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:44.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 shutdown_connections 2024-09-10T21:04:44.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:44.068+0000 7f64959c5640 1 -- 172.21.15.33:0/1559363791 wait complete. 2024-09-10T21:04:44.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:04:45.226 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:45.494 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:45 smithi033 ceph-mon[30860]: pgmap v369: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:45.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:45 smithi104 ceph-mon[35325]: pgmap v369: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:45 smithi174 ceph-mon[35635]: pgmap v369: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 -- 172.21.15.33:0/1784351175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0101ae0 msgr2=0x7fa3a0101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/1784351175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0101ae0 0x7fa3a0101ec0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa390009900 tx=0x7fa39002a410 comp rx=0 tx=0).stop 2024-09-10T21:04:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 -- 172.21.15.33:0/1784351175 shutdown_connections 2024-09-10T21:04:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/1784351175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a0107e70 0x7fa3a010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/1784351175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3a01074f0 0x7fa3a0107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/1784351175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0101ae0 0x7fa3a0101ec0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 -- 172.21.15.33:0/1784351175 >> 172.21.15.33:0/1784351175 conn(0x7fa3a00fd5b0 msgr2=0x7fa3a00ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 -- 172.21.15.33:0/1784351175 shutdown_connections 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 -- 172.21.15.33:0/1784351175 wait complete. 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.713+0000 7fa3a51a2640 1 Processor -- start 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa3a51a2640 1 -- start start 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa3a51a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3a01074f0 0x7fa3a010ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa3a51a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0107e70 0x7fa3a010f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa3a51a2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a010f8d0 0x7fa3a01a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0107e70 0x7fa3a010f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa3a51a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0112c20 con 0x7fa3a0107e70 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa3a51a2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0112aa0 con 0x7fa3a010f8d0 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa3a51a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a0112da0 con 0x7fa3a01074f0 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0107e70 0x7fa3a010f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43626/0 (socket says 172.21.15.33:43626) 2024-09-10T21:04:45.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3477134197 learned_addr learned my addr 172.21.15.33:0/3477134197 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.714+0000 7fa39ffff640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3a01074f0 0x7fa3a010ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a010f8d0 0x7fa3a01a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3477134197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3a01074f0 msgr2=0x7fa3a010ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3a01074f0 0x7fa3a010ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3477134197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a010f8d0 msgr2=0x7fa3a01a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a010f8d0 0x7fa3a01a73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3477134197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a01a7a50 con 0x7fa3a0107e70 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0107e70 0x7fa3a010f390 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d770 tx=0x7fa39400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.715+0000 7fa39d7fa640 1 -- 172.21.15.33:0/3477134197 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394017070 con 0x7fa3a0107e70 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.716+0000 7fa39d7fa640 1 -- 172.21.15.33:0/3477134197 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3940054d0 con 0x7fa3a0107e70 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.716+0000 7fa39d7fa640 1 -- 172.21.15.33:0/3477134197 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394004c30 con 0x7fa3a0107e70 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.716+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a01a7d40 con 0x7fa3a0107e70 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.716+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a01a81d0 con 0x7fa3a0107e70 2024-09-10T21:04:45.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.719+0000 7fa3a87d9740 1 Processor -- start 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.719+0000 7fa3a87d9740 1 -- start start 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d5ae0 0x5650181d5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d6400 0x5650181e1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181e2390 0x5650181e4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565018143ea0 con 0x5650181e2390 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5650181332d0 con 0x5650181d6400 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650181370e0 con 0x5650181d5ae0 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa39d7fa640 1 -- 172.21.15.33:0/3477134197 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa394023060 con 0x7fa3a0107e70 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d5ae0 0x5650181d5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d5ae0 0x5650181d5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:32768/0 (socket says 172.21.15.33:32768) 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 -- 172.21.15.33:0/961763780 learned_addr learned my addr 172.21.15.33:0/961763780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa39ffff640 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d6400 0x5650181e1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181e2390 0x5650181e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 -- 172.21.15.33:0/961763780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d6400 msgr2=0x5650181e1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d6400 0x5650181e1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 -- 172.21.15.33:0/961763780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181e2390 msgr2=0x5650181e4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181e2390 0x5650181e4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa3a49a1640 1 -- 172.21.15.33:0/961763780 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565017e91650 con 0x5650181d5ae0 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.720+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181e2390 0x5650181e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.721+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d5ae0 0x5650181d5ec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa398009fd0 tx=0x7fa3980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9747db2b802c0c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.721+0000 7fa3867fc640 1 -- 172.21.15.33:0/961763780 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398004d20 con 0x5650181d5ae0 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.721+0000 7fa3867fc640 1 -- 172.21.15.33:0/961763780 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa398004050 con 0x5650181d5ae0 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.721+0000 7fa3867fc640 1 -- 172.21.15.33:0/961763780 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398005280 con 0x5650181d5ae0 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.721+0000 7fa39d7fa640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3880b5430 0x7fa3880b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.721+0000 7fa39d7fa640 1 -- 172.21.15.33:0/3477134197 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa3940e4fd0 con 0x7fa3a0107e70 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa39ffff640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3880b5430 0x7fa3880b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 -- 172.21.15.33:0/961763780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d5ae0 msgr2=0x5650181d5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d5ae0 0x5650181d5ec0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa398009fd0 tx=0x7fa3980118d0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa39ffff640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3880b5430 0x7fa3880b78f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa390009230 tx=0x7fa3900058b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 -- 172.21.15.33:0/961763780 shutdown_connections 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181e2390 0x5650181e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d6400 0x5650181e1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/961763780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d5ae0 0x5650181d5ec0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 -- 172.21.15.33:0/961763780 >> 172.21.15.33:0/961763780 conn(0x5650181d3910 msgr2=0x5650181d4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 -- 172.21.15.33:0/961763780 shutdown_connections 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.722+0000 7fa3a87d9740 1 -- 172.21.15.33:0/961763780 wait complete. 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.723+0000 7fa3a87d9740 1 -- start start 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.723+0000 7fa3a87d9740 1 Processor -- start 2024-09-10T21:04:45.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa39ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa39ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181e1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43658/0 (socket says 172.21.15.33:43658) 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa39ffff640 1 -- 172.21.15.33:0/2328702164 learned_addr learned my addr 172.21.15.33:0/2328702164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181df010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181df550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565017f94700 con 0x5650181d5ae0 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5650180ce160 con 0x5650181e2390 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650181e5010 con 0x5650181d6400 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181df010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.724+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181df550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa3a87d9740 10 client.0 _ll_register_callbacks cb 0x5650180e79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:45.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa39f7fe640 1 -- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 msgr2=0x5650181df550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181df550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa39f7fe640 1 -- 172.21.15.33:0/2328702164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 msgr2=0x5650181e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa39f7fe640 1 -- 172.21.15.33:0/2328702164 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650180fbc90 con 0x5650181d6400 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa398004050 con 0x5650181d6400 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181df010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ab720 tx=0x7fa3940ab1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a709f844e2ef89d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa37f7fe640 10 client.25072 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa37f7fe640 1 -- 172.21.15.33:0/2328702164 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394020040 con 0x5650181d6400 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa37f7fe640 1 -- 172.21.15.33:0/2328702164 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa3940aa9f0 con 0x5650181d6400 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa37f7fe640 1 -- 172.21.15.33:0/2328702164 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3940ce040 con 0x5650181d6400 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa3a87d9740 10 client.25072 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:45.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.725+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=28}) -- 0x5650181f0720 con 0x5650181d6400 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa37f7fe640 1 -- 172.21.15.33:0/2328702164 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa3940e8020 con 0x5650181d6400 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa3a87d9740 10 client.25072 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa3a87d9740 10 client.25072 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa3a87d9740 20 client.25072 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5650181f1ac0 0x5650181f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa3a87d9740 4 client.25072 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565018145df0 con 0x5650181f1ac0 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.726+0000 7fa39ffff640 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5650181f1ac0 0x5650181f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.727+0000 7fa37f7fe640 10 client.25072 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.727+0000 7fa39ffff640 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5650181f1ac0 0x5650181f3ef0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e9f33f6119c2b726 server_cookie=57573586df9aee36 in_seq=0 out_seq=0 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa37f7fe640 1 -- 172.21.15.33:0/2328702164 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565018145df0 con 0x5650181f1ac0 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa37f7fe640 10 client.25072 handle_command_reply: tid=0 2024-09-10T21:04:45.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 client.25072 shutdown 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 msgr2=0x5650181df010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181df010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ab720 tx=0x7fa3940ab1a0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 shutdown_connections 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181df550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181df010 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5650181f1ac0 0x5650181f3ef0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/2328702164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa37f7fe640 0 client.25072 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 >> 172.21.15.33:0/2328702164 conn(0x5650181d3910 msgr2=0x5650181ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 shutdown_connections 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.728+0000 7fa3a87d9740 1 -- 172.21.15.33:0/2328702164 wait complete. 2024-09-10T21:04:45.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.729+0000 7fa3a87d9740 20 client.25072 trim_cache size 0 max 16384 2024-09-10T21:04:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.755+0000 7fa3a87d9740 1 Processor -- start 2024-09-10T21:04:45.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.755+0000 7fa3a87d9740 1 -- start start 2024-09-10T21:04:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01ac550 con 0x5650181d5ae0 2024-09-10T21:04:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01ae090 con 0x5650181e2390 2024-09-10T21:04:45.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01acc20 con 0x5650181d6400 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:32780/0 (socket says 172.21.15.33:32780) 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa39f7fe640 1 -- 172.21.15.33:0/1318195399 learned_addr learned my addr 172.21.15.33:0/1318195399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.756+0000 7fa39ffff640 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.757+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.757+0000 7fa39f7fe640 1 -- 172.21.15.33:0/1318195399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 msgr2=0x5650181e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.757+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181e3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.757+0000 7fa39f7fe640 1 -- 172.21.15.33:0/1318195399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 msgr2=0x5650181ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.757+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.757+0000 7fa39f7fe640 1 -- 172.21.15.33:0/1318195399 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650181d5440 con 0x5650181d6400 2024-09-10T21:04:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.757+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:45.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.758+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181e3780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3940b01c0 tx=0x7fa39401c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=617dd5ea92d88188 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.758+0000 7fa385ffb640 1 -- 172.21.15.33:0/1318195399 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394020050 con 0x5650181d6400 2024-09-10T21:04:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.758+0000 7fa385ffb640 1 -- 172.21.15.33:0/1318195399 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa398004050 con 0x5650181d6400 2024-09-10T21:04:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.758+0000 7fa385ffb640 1 -- 172.21.15.33:0/1318195399 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3940ced40 con 0x5650181d6400 2024-09-10T21:04:45.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 -- 172.21.15.33:0/1318195399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 msgr2=0x5650181e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181e3780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa3940b01c0 tx=0x7fa39401c000 comp rx=0 tx=0).stop 2024-09-10T21:04:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 -- 172.21.15.33:0/1318195399 shutdown_connections 2024-09-10T21:04:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181e2390 0x5650181e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181d6400 0x5650181e3780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/1318195399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181d5ae0 0x5650181ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 -- 172.21.15.33:0/1318195399 >> 172.21.15.33:0/1318195399 conn(0x5650181d3910 msgr2=0x5650181c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:45.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 -- 172.21.15.33:0/1318195399 shutdown_connections 2024-09-10T21:04:45.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.759+0000 7fa3a87d9740 1 -- 172.21.15.33:0/1318195399 wait complete. 2024-09-10T21:04:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.760+0000 7fa3a87d9740 1 -- start start 2024-09-10T21:04:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.760+0000 7fa3a87d9740 1 Processor -- start 2024-09-10T21:04:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d5ae0 0x5650181ea2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181e2390 0x5650181ea810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a87d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181f1ac0 0x5650181db2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01ad9c0 con 0x5650181f1ac0 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01ad2f0 con 0x5650181d5ae0 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a87d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a01aa510 con 0x5650181e2390 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a87d9740 10 client.0 _ll_register_callbacks cb 0x7fa39400d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa3a49a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181f1ac0 0x5650181db2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181e2390 0x5650181ea810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa39f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181e2390 0x5650181ea810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:32796/0 (socket says 172.21.15.33:32796) 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.761+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3481827258 learned_addr learned my addr 172.21.15.33:0/3481827258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3481827258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d5ae0 msgr2=0x5650181ea2d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d5ae0 0x5650181ea2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3481827258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181f1ac0 msgr2=0x5650181db2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181f1ac0 0x5650181db2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa39f7fe640 1 -- 172.21.15.33:0/3481827258 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650181eaf00 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa3a49a1640 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181f1ac0 0x5650181db2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa39f7fe640 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181e2390 0x5650181ea810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ab490 tx=0x7fa3940a7de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=648f949b8a8e36d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa37f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa37f7fe640 1 -- 172.21.15.33:0/3481827258 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa394020050 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa37f7fe640 1 -- 172.21.15.33:0/3481827258 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa39405b210 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.762+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa398004050 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa37f7fe640 1 -- 172.21.15.33:0/3481827258 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa3940a7630 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 10 client.25078 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56501825fe40 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa37f7fe640 1 -- 172.21.15.33:0/3481827258 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa398004050 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa37f7fe640 1 -- 172.21.15.33:0/3481827258 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa3940acac0 con 0x5650181e2390 2024-09-10T21:04:45.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 10 client.25078 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 10 client.25078 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:04:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 20 client.25078 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x565018260370 0x565018262760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 4 client.25078 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:04:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.763+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5650181418a0 con 0x565018260370 2024-09-10T21:04:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.764+0000 7fa39ffff640 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x565018260370 0x565018262760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.764+0000 7fa37f7fe640 10 client.25078 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:04:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.764+0000 7fa39ffff640 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x565018260370 0x565018262760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa78e38c4369930 server_cookie=47d229e5ce31dd1b in_seq=0 out_seq=0 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.764+0000 7fa37f7fe640 1 -- 172.21.15.33:0/3481827258 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5650181418a0 con 0x565018260370 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.764+0000 7fa37f7fe640 10 client.25078 handle_command_reply: tid=0 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 client.25078 shutdown 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181e2390 msgr2=0x5650181ea810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181e2390 0x5650181ea810 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3940ab490 tx=0x7fa3940a7de0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 shutdown_connections 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x565018260370 0x565018262760 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5650181f1ac0 0x5650181db2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa37f7fe640 0 client.25078 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650181e2390 0x5650181ea810 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 --2- 172.21.15.33:0/3481827258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5650181d5ae0 0x5650181ea2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.765+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 >> 172.21.15.33:0/3481827258 conn(0x5650181d3910 msgr2=0x5650181e0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.766+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 shutdown_connections 2024-09-10T21:04:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.766+0000 7fa3a87d9740 1 -- 172.21.15.33:0/3481827258 wait complete. 2024-09-10T21:04:45.773 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.766+0000 7fa3a87d9740 20 client.25078 trim_cache size 0 max 16384 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.766+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3880b5430 msgr2=0x7fa3880b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.766+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3880b5430 0x7fa3880b78f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa390009230 tx=0x7fa3900058b0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0107e70 msgr2=0x7fa3a010f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0107e70 0x7fa3a010f390 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fa39400d770 tx=0x7fa39400dc30 comp rx=0 tx=0).stop 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 shutdown_connections 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3880b5430 0x7fa3880b78f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a010f8d0 0x7fa3a01a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3a0107e70 0x7fa3a010f390 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 --2- 172.21.15.33:0/3477134197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3a01074f0 0x7fa3a010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 >> 172.21.15.33:0/3477134197 conn(0x7fa3a00fd5b0 msgr2=0x7fa3a00ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 shutdown_connections 2024-09-10T21:04:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:45.767+0000 7fa3a51a2640 1 -- 172.21.15.33:0/3477134197 wait complete. 2024-09-10T21:04:46.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:04:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:47.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 -- 172.21.15.33:0/257846209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 msgr2=0x7f762410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 --2- 172.21.15.33:0/257846209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f762410cab0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7618009900 tx=0x7f761802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 -- 172.21.15.33:0/257846209 shutdown_connections 2024-09-10T21:04:47.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 --2- 172.21.15.33:0/257846209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f762410d100 0x7f762410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 --2- 172.21.15.33:0/257846209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f762410cab0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 --2- 172.21.15.33:0/257846209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7624103ca0 0x7f7624104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 -- 172.21.15.33:0/257846209 >> 172.21.15.33:0/257846209 conn(0x7f76240fd700 msgr2=0x7f76240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 -- 172.21.15.33:0/257846209 shutdown_connections 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.285+0000 7f7629b9b640 1 -- 172.21.15.33:0/257846209 wait complete. 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.286+0000 7f7629b9b640 1 Processor -- start 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.286+0000 7f7629b9b640 1 -- start start 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7629b9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7624103ca0 0x7f76241991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7629b9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f7624199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7629b9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762410d100 0x7f762419eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7629b9b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7624112d70 con 0x7f76241045c0 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7629b9b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7624112bf0 con 0x7f762410d100 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7629b9b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7624112ef0 con 0x7f7624103ca0 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7622ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f7624199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7622ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f7624199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43690/0 (socket says 172.21.15.33:43690) 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7622ffd640 1 -- 172.21.15.33:0/584458420 learned_addr learned my addr 172.21.15.33:0/584458420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f76237fe640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7624103ca0 0x7f76241991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7623fff640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762410d100 0x7f762419eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f76237fe640 1 -- 172.21.15.33:0/584458420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762410d100 msgr2=0x7f762419eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f76237fe640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762410d100 0x7f762419eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f76237fe640 1 -- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 msgr2=0x7f7624199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f76237fe640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f7624199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f76237fe640 1 -- 172.21.15.33:0/584458420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762419f290 con 0x7f7624103ca0 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.287+0000 7f7622ffd640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f7624199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.288+0000 7f76237fe640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7624103ca0 0x7f76241991e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f761400c970 tx=0x7f761400ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.288+0000 7f7620ff9640 1 -- 172.21.15.33:0/584458420 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614007bb0 con 0x7f7624103ca0 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.288+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762419f580 con 0x7f7624103ca0 2024-09-10T21:04:47.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.288+0000 7f7620ff9640 1 -- 172.21.15.33:0/584458420 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7614007d50 con 0x7f7624103ca0 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.288+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76241a5e10 con 0x7f7624103ca0 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.288+0000 7f7620ff9640 1 -- 172.21.15.33:0/584458420 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7614018780 con 0x7f7624103ca0 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.290+0000 7f7620ff9640 1 -- 172.21.15.33:0/584458420 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7614018920 con 0x7f7624103ca0 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.291+0000 7f7620ff9640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f760c0b5360 0x7f760c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.291+0000 7f762bf70740 1 Processor -- start 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.291+0000 7f762bf70740 1 -- start start 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7622ffd640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f760c0b5360 0x7f760c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7620ff9640 1 -- 172.21.15.33:0/584458420 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f76140e5920 con 0x7f7624103ca0 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f762bf70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 0x56373fd688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f762bf70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd74840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f762bf70740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd77170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f762bf70740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56373fcdeea0 con 0x56373fd74d80 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f762bf70740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56373fcce2d0 con 0x56373fd68df0 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f762bf70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56373fcd20e0 con 0x56373fd684d0 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd77170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd77170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43700/0 (socket says 172.21.15.33:43700) 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 -- 172.21.15.33:0/2512220023 learned_addr learned my addr 172.21.15.33:0/2512220023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7622ffd640 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd74840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 -- 172.21.15.33:0/2512220023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 msgr2=0x56373fd688b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 0x56373fd688b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 -- 172.21.15.33:0/2512220023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 msgr2=0x56373fd74840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:47.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd74840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7623fff640 1 -- 172.21.15.33:0/2512220023 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373fa2c650 con 0x56373fd74d80 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.292+0000 7f7622ffd640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f760c0b5360 0x7f760c0b7820 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f76180096c0 tx=0x7f7618032f70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.293+0000 7f7623fff640 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd77170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f761c00f5c0 tx=0x7f761c00fa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9bbe71ed7b29fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.293+0000 7f7609ffb640 1 -- 172.21.15.33:0/2512220023 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c00d7e0 con 0x56373fd74d80 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.293+0000 7f7609ffb640 1 -- 172.21.15.33:0/2512220023 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761c00de30 con 0x56373fd74d80 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.293+0000 7f7609ffb640 1 -- 172.21.15.33:0/2512220023 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f761c0183f0 con 0x56373fd74d80 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 -- 172.21.15.33:0/2512220023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 msgr2=0x56373fd77170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd77170 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f761c00f5c0 tx=0x7f761c00fa80 comp rx=0 tx=0).stop 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 -- 172.21.15.33:0/2512220023 shutdown_connections 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd77170 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd74840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 --2- 172.21.15.33:0/2512220023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 0x56373fd688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 -- 172.21.15.33:0/2512220023 >> 172.21.15.33:0/2512220023 conn(0x56373fd66300 msgr2=0x56373fd674a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 -- 172.21.15.33:0/2512220023 shutdown_connections 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.294+0000 7f762bf70740 1 -- 172.21.15.33:0/2512220023 wait complete. 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 -- start start 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 Processor -- start 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd684d0 0x56373fd74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd68df0 0x56373fd71aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56373fb2f700 con 0x56373fd74d80 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56373fc69160 con 0x56373fd684d0 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.295+0000 7f762bf70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56373fd7be10 con 0x56373fd68df0 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f762bf70740 10 client.0 _ll_register_callbacks cb 0x56373fc829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f7623fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd684d0 0x56373fd74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f7622ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd68df0 0x56373fd71aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd684d0 0x56373fd74650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57354/0 (socket says 172.21.15.33:57354) 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 -- 172.21.15.33:0/2894111243 learned_addr learned my addr 172.21.15.33:0/2894111243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 -- 172.21.15.33:0/2894111243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd68df0 msgr2=0x56373fd71aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd68df0 0x56373fd71aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 -- 172.21.15.33:0/2894111243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 msgr2=0x56373fd72080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd72080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 -- 172.21.15.33:0/2894111243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373fc96c90 con 0x56373fd684d0 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f7623fff640 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f76237fe640 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd684d0 0x56373fd74650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76140a8670 tx=0x7f76140abb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=572e2f44558b1753 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f7602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f7602ffd640 1 -- 172.21.15.33:0/2894111243 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76140b53a0 con 0x56373fd684d0 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.296+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f761c00de30 con 0x56373fd684d0 2024-09-10T21:04:47.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f7602ffd640 1 -- 172.21.15.33:0/2894111243 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f76140ab300 con 0x56373fd684d0 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f7602ffd640 1 -- 172.21.15.33:0/2894111243 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76140ac0a0 con 0x56373fd684d0 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 10 client.25100 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56373fd77810 con 0x56373fd684d0 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f7602ffd640 1 -- 172.21.15.33:0/2894111243 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f761c00de30 con 0x56373fd684d0 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f7602ffd640 1 -- 172.21.15.33:0/2894111243 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76140aaf70 con 0x56373fd684d0 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 10 client.25100 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 10 client.25100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 20 client.25100 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fd79140 0x56373fd7b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 4 client.25100 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.297+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56373fce0df0 con 0x56373fd79140 2024-09-10T21:04:47.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.298+0000 7f7622ffd640 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fd79140 0x56373fd7b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.298+0000 7f7622ffd640 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fd79140 0x56373fd7b560 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf4544963c21a864 server_cookie=852e4b26afd8c0c8 in_seq=0 out_seq=0 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.298+0000 7f7602ffd640 10 client.25100 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f7602ffd640 1 -- 172.21.15.33:0/2894111243 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56373fce0df0 con 0x56373fd79140 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f7602ffd640 10 client.25100 handle_command_reply: tid=0 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 client.25100 shutdown 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd684d0 msgr2=0x56373fd74650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd684d0 0x56373fd74650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f76140a8670 tx=0x7f76140abb20 comp rx=0 tx=0).stop 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 shutdown_connections 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fd79140 0x56373fd7b560 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd74d80 0x56373fd72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd68df0 0x56373fd71aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 --2- 172.21.15.33:0/2894111243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd684d0 0x56373fd74650 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 >> 172.21.15.33:0/2894111243 conn(0x56373fd66300 msgr2=0x56373fd6be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:47.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.299+0000 7f7602ffd640 0 client.25100 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.300+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 shutdown_connections 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.300+0000 7f762bf70740 1 -- 172.21.15.33:0/2894111243 wait complete. 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.300+0000 7f762bf70740 20 client.25100 trim_cache size 0 max 16384 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 Processor -- start 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 -- start start 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 0x56373fd65c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd68df0 0x56373fd673a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd74d80 0x56373fd7b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f761000e480 con 0x56373fd68df0 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f761000ffc0 con 0x56373fd74d80 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f762bf70740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f761000eb50 con 0x56373fd684d0 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.313+0000 7f7622ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd68df0 0x56373fd673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd68df0 0x56373fd673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43716/0 (socket says 172.21.15.33:43716) 2024-09-10T21:04:47.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 -- 172.21.15.33:0/1795660460 learned_addr learned my addr 172.21.15.33:0/1795660460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f76237fe640 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 0x56373fd65c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 -- 172.21.15.33:0/1795660460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 msgr2=0x56373fd65c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7623fff640 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd74d80 0x56373fd7b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 0x56373fd65c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 -- 172.21.15.33:0/1795660460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd74d80 msgr2=0x56373fd7b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd74d80 0x56373fd7b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 -- 172.21.15.33:0/1795660460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56373fd723f0 con 0x56373fd68df0 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7623fff640 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd74d80 0x56373fd7b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f7622ffd640 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd68df0 0x56373fd673a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7618000c00 tx=0x7f7618047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d30cabb27da555c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f76097fa640 1 -- 172.21.15.33:0/1795660460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7618035040 con 0x56373fd68df0 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f76097fa640 1 -- 172.21.15.33:0/1795660460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76180055b0 con 0x56373fd68df0 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.314+0000 7f76097fa640 1 -- 172.21.15.33:0/1795660460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180035f0 con 0x56373fd68df0 2024-09-10T21:04:47.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.315+0000 7f762bf70740 1 -- 172.21.15.33:0/1795660460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd68df0 msgr2=0x56373fd673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.315+0000 7f762bf70740 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd68df0 0x56373fd673a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7618000c00 tx=0x7f7618047000 comp rx=0 tx=0).stop 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.315+0000 7f762bf70740 1 -- 172.21.15.33:0/1795660460 shutdown_connections 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.315+0000 7f762bf70740 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd74d80 0x56373fd7b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.315+0000 7f762bf70740 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd68df0 0x56373fd673a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.315+0000 7f762bf70740 1 --2- 172.21.15.33:0/1795660460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd684d0 0x56373fd65c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.315+0000 7f762bf70740 1 -- 172.21.15.33:0/1795660460 >> 172.21.15.33:0/1795660460 conn(0x56373fd66300 msgr2=0x56373fd71850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.316+0000 7f762bf70740 1 -- 172.21.15.33:0/1795660460 shutdown_connections 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.316+0000 7f762bf70740 1 -- 172.21.15.33:0/1795660460 wait complete. 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.316+0000 7f762bf70740 1 -- start start 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.316+0000 7f762bf70740 1 Processor -- start 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd684d0 0x56373fd8c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd8c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd74d80 0x56373fd80180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd684d0 0x56373fd8c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd684d0 0x56373fd8c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43730/0 (socket says 172.21.15.33:43730) 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 -- 172.21.15.33:0/2356008051 learned_addr learned my addr 172.21.15.33:0/2356008051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f761000f8f0 con 0x56373fd684d0 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f761000f220 con 0x56373fd68df0 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f7622ffd640 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd8c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f761000c440 con 0x56373fd74d80 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f7623fff640 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd74d80 0x56373fd80180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 10 client.0 _ll_register_callbacks cb 0x7f7614007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 -- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd74d80 msgr2=0x56373fd80180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd74d80 0x56373fd80180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 -- 172.21.15.33:0/2356008051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 msgr2=0x56373fd8c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:47.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd8c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f76237fe640 1 -- 172.21.15.33:0/2356008051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56373fd808c0 con 0x56373fd684d0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.317+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76180055b0 con 0x56373fd684d0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f76237fe640 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd684d0 0x56373fd8c290 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f76140abda0 tx=0x7f76140b53b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bbe9a4dce14e97 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f7602ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f7602ffd640 1 -- 172.21.15.33:0/2356008051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76140b0b80 con 0x56373fd684d0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f7602ffd640 1 -- 172.21.15.33:0/2356008051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f76140cddd0 con 0x56373fd684d0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f7602ffd640 1 -- 172.21.15.33:0/2356008051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f76140b5630 con 0x56373fd684d0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=28}) -- 0x56373fd81a30 con 0x56373fd684d0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f7602ffd640 1 -- 172.21.15.33:0/2356008051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f76140971c0 con 0x56373fd684d0 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 10 client.15492 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:04:47.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 20 client.15492 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fdf1c80 0x56373fdf4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 4 client.15492 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.318+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56373fcdc8a0 con 0x56373fdf1c80 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.319+0000 7f7622ffd640 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fdf1c80 0x56373fdf4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.319+0000 7f7622ffd640 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fdf1c80 0x56373fdf4070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b27ca1c75ed29e71 server_cookie=9e65c2fcdc344227 in_seq=0 out_seq=0 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.319+0000 7f7602ffd640 10 client.15492 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.319+0000 7f7602ffd640 1 -- 172.21.15.33:0/2356008051 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56373fcdc8a0 con 0x56373fdf1c80 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.319+0000 7f7602ffd640 10 client.15492 handle_command_reply: tid=0 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.319+0000 7f762bf70740 1 client.15492 shutdown 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.319+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd684d0 msgr2=0x56373fd8c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd684d0 0x56373fd8c290 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f76140abda0 tx=0x7f76140b53b0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 shutdown_connections 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56373fd74d80 0x56373fd80180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56373fd68df0 0x56373fd8c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56373fdf1c80 0x56373fdf4070 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 --2- 172.21.15.33:0/2356008051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56373fd684d0 0x56373fd8c290 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f7602ffd640 0 client.15492 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 >> 172.21.15.33:0/2356008051 conn(0x56373fd66300 msgr2=0x56373fd673a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 shutdown_connections 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 1 -- 172.21.15.33:0/2356008051 wait complete. 2024-09-10T21:04:47.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.320+0000 7f762bf70740 20 client.15492 trim_cache size 0 max 16384 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f760c0b5360 msgr2=0x7f760c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f760c0b5360 0x7f760c0b7820 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f76180096c0 tx=0x7f7618032f70 comp rx=0 tx=0).stop 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7624103ca0 msgr2=0x7f76241991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7624103ca0 0x7f76241991e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f761400c970 tx=0x7f761400ce30 comp rx=0 tx=0).stop 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 shutdown_connections 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f760c0b5360 0x7f760c0b7820 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f762410d100 0x7f762419eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76241045c0 0x7f7624199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 --2- 172.21.15.33:0/584458420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7624103ca0 0x7f76241991e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 >> 172.21.15.33:0/584458420 conn(0x7f76240fd700 msgr2=0x7f762410f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 shutdown_connections 2024-09-10T21:04:47.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:47.321+0000 7f7629b9b640 1 -- 172.21.15.33:0/584458420 wait complete. 2024-09-10T21:04:47.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:47 smithi104 ceph-mon[35325]: pgmap v370: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:47.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:47 smithi174 ceph-mon[35635]: pgmap v370: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:47 smithi033 ceph-mon[30860]: pgmap v370: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:48.154 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:04:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.072+0000 7f6728bcc640 1 -- 172.21.15.33:0/3828316790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67241021b0 msgr2=0x7f672410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.072+0000 7f6728bcc640 1 --2- 172.21.15.33:0/3828316790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67241021b0 0x7f672410ef00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6718009900 tx=0x7f671802a480 comp rx=0 tx=0).stop 2024-09-10T21:04:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.073+0000 7f6728bcc640 1 -- 172.21.15.33:0/3828316790 shutdown_connections 2024-09-10T21:04:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.073+0000 7f6728bcc640 1 --2- 172.21.15.33:0/3828316790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f672410f440 0x7f6724111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.073+0000 7f6728bcc640 1 --2- 172.21.15.33:0/3828316790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67241021b0 0x7f672410ef00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.073+0000 7f6728bcc640 1 --2- 172.21.15.33:0/3828316790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6724101890 0x7f6724101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.073+0000 7f6728bcc640 1 -- 172.21.15.33:0/3828316790 >> 172.21.15.33:0/3828316790 conn(0x7f67240fd700 msgr2=0x7f67240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.073+0000 7f6728bcc640 1 -- 172.21.15.33:0/3828316790 shutdown_connections 2024-09-10T21:04:49.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.073+0000 7f6728bcc640 1 -- 172.21.15.33:0/3828316790 wait complete. 2024-09-10T21:04:49.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 Processor -- start 2024-09-10T21:04:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 -- start start 2024-09-10T21:04:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6724101890 0x7f672419b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67241021b0 0x7f672419ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f672410f440 0x7f67241a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6724114f40 con 0x7f672410f440 2024-09-10T21:04:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6724114dc0 con 0x7f67241021b0 2024-09-10T21:04:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6728bcc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67241150c0 con 0x7f6724101890 2024-09-10T21:04:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67241021b0 0x7f672419ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67241021b0 0x7f672419ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57384/0 (socket says 172.21.15.33:57384) 2024-09-10T21:04:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6721d9b640 1 -- 172.21.15.33:0/1133361947 learned_addr learned my addr 172.21.15.33:0/1133361947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:49.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f672259c640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6724101890 0x7f672419b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.074+0000 7f6722d9d640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f672410f440 0x7f67241a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.075+0000 7f6721d9b640 1 -- 172.21.15.33:0/1133361947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6724101890 msgr2=0x7f672419b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.075+0000 7f6721d9b640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6724101890 0x7f672419b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.075+0000 7f6721d9b640 1 -- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f672410f440 msgr2=0x7f67241a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.075+0000 7f6721d9b640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f672410f440 0x7f67241a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.075+0000 7f6721d9b640 1 -- 172.21.15.33:0/1133361947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67241a1580 con 0x7f67241021b0 2024-09-10T21:04:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.075+0000 7f6722d9d640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f672410f440 0x7f67241a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.075+0000 7f6721d9b640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67241021b0 0x7f672419ba10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6718005940 tx=0x7f67180042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.076+0000 7f67137fe640 1 -- 172.21.15.33:0/1133361947 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671803f030 con 0x7f67241021b0 2024-09-10T21:04:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.076+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67241a1810 con 0x7f67241021b0 2024-09-10T21:04:49.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.076+0000 7f67137fe640 1 -- 172.21.15.33:0/1133361947 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6718003cb0 con 0x7f67241021b0 2024-09-10T21:04:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.077+0000 7f67137fe640 1 -- 172.21.15.33:0/1133361947 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718035070 con 0x7f67241021b0 2024-09-10T21:04:49.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.077+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6724103aa0 con 0x7f67241021b0 2024-09-10T21:04:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.079+0000 7f67137fe640 1 -- 172.21.15.33:0/1133361947 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f671804c060 con 0x7f67241021b0 2024-09-10T21:04:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.080+0000 7f67137fe640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f670c0b5230 0x7f670c0b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.080+0000 7f672afa1740 1 Processor -- start 2024-09-10T21:04:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.080+0000 7f672259c640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f670c0b5230 0x7f670c0b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.080+0000 7f672afa1740 1 -- start start 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f672afa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead800f0 0x55d5ead804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f672afa1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead80a10 0x55d5ead8c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f672afa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead80a10 0x55d5ead8c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f67137fe640 1 -- 172.21.15.33:0/1133361947 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6718105f70 con 0x7f67241021b0 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead80a10 0x55d5ead8c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57396/0 (socket says 172.21.15.33:57396) 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6722d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6722d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:32846/0 (socket says 172.21.15.33:32846) 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6721d9b640 1 -- 172.21.15.33:0/532442828 learned_addr learned my addr 172.21.15.33:0/532442828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d5eacf6ea0 con 0x55d5ead800f0 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d5eace62d0 con 0x55d5ead80a10 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5eacea0e0 con 0x55d5ead8c9a0 2024-09-10T21:04:49.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6721d9b640 1 -- 172.21.15.33:0/532442828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 msgr2=0x55d5ead8ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6721d9b640 1 --2- 172.21.15.33:0/532442828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6721d9b640 1 -- 172.21.15.33:0/532442828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead800f0 msgr2=0x55d5ead804d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.081+0000 7f6721d9b640 1 --2- 172.21.15.33:0/532442828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead800f0 0x55d5ead804d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.082+0000 7f6721d9b640 1 -- 172.21.15.33:0/532442828 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5eaa44650 con 0x55d5ead80a10 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.082+0000 7f6721d9b640 1 --2- 172.21.15.33:0/532442828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead80a10 0x55d5ead8c460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67180cc680 tx=0x7f6718005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99304326fa52c6f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.082+0000 7f672259c640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f670c0b5230 0x7f670c0b76f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6714005d00 tx=0x7f6714017040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.082+0000 7f6710ff9640 1 -- 172.21.15.33:0/532442828 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6718049070 con 0x55d5ead80a10 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.082+0000 7f6710ff9640 1 -- 172.21.15.33:0/532442828 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67180cc200 con 0x55d5ead80a10 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.082+0000 7f6710ff9640 1 -- 172.21.15.33:0/532442828 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67180d5c80 con 0x55d5ead80a10 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead80a10 msgr2=0x55d5ead8c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 --2- 172.21.15.33:0/532442828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead80a10 0x55d5ead8c460 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f67180cc680 tx=0x7f6718005fd0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 shutdown_connections 2024-09-10T21:04:49.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 --2- 172.21.15.33:0/532442828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 --2- 172.21.15.33:0/532442828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead80a10 0x55d5ead8c460 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 --2- 172.21.15.33:0/532442828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead800f0 0x55d5ead804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 >> 172.21.15.33:0/532442828 conn(0x55d5ead7df20 msgr2=0x55d5ead7f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 shutdown_connections 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.083+0000 7f672afa1740 1 -- 172.21.15.33:0/532442828 wait complete. 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.084+0000 7f672afa1740 1 -- start start 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.084+0000 7f672afa1740 1 Processor -- start 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.084+0000 7f672afa1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead8c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.084+0000 7f672afa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead80a10 0x55d5ead89820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672afa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead8c9a0 0x55d5ead89f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead8c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672afa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d5eab47700 con 0x55d5ead8c9a0 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672afa1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d5eac81160 con 0x55d5ead800f0 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead8c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57402/0 (socket says 172.21.15.33:57402) 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672afa1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5eace62d0 con 0x55d5ead80a10 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 -- 172.21.15.33:0/2913876244 learned_addr learned my addr 172.21.15.33:0/2913876244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f6721d9b640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead80a10 0x55d5ead89820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672afa1740 10 client.0 _ll_register_callbacks cb 0x55d5eac9a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f6722d9d640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead8c9a0 0x55d5ead89f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 -- 172.21.15.33:0/2913876244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead80a10 msgr2=0x55d5ead89820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead80a10 0x55d5ead89820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 -- 172.21.15.33:0/2913876244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead8c9a0 msgr2=0x55d5ead89f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead8c9a0 0x55d5ead89f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f672259c640 1 -- 172.21.15.33:0/2913876244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5eacaec90 con 0x55d5ead800f0 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f6721d9b640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead80a10 0x55d5ead89820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.085+0000 7f6722d9d640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead8c9a0 0x55d5ead89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f672259c640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead8c3d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6714009fb0 tx=0x7f67140046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e17f12a7354d64e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:49.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f6709ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f6709ffb640 1 -- 172.21.15.33:0/2913876244 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714014070 con 0x55d5ead800f0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f6709ffb640 1 -- 172.21.15.33:0/2913876244 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6714015260 con 0x55d5ead800f0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67180cc200 con 0x55d5ead800f0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f6709ffb640 1 -- 172.21.15.33:0/2913876244 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6714015460 con 0x55d5ead800f0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f672afa1740 10 client.25115 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.086+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d5eadabd50 con 0x55d5ead800f0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f6709ffb640 1 -- 172.21.15.33:0/2913876244 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67180cc200 con 0x55d5ead800f0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f6709ffb640 1 -- 172.21.15.33:0/2913876244 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f671403c020 con 0x55d5ead800f0 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f672afa1740 10 client.25115 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f672afa1740 10 client.25115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f672afa1740 20 client.25115 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:49.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f672afa1740 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eadaf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f672afa1740 4 client.25115 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5eacf8df0 con 0x55d5eadad300 2024-09-10T21:04:49.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.087+0000 7f6721d9b640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eadaf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.088+0000 7f6721d9b640 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eadaf6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21706cfd32459dbb server_cookie=c79ba02355f1905a in_seq=0 out_seq=0 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.088+0000 7f6709ffb640 10 client.25115 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f6709ffb640 1 -- 172.21.15.33:0/2913876244 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d5eacf8df0 con 0x55d5eadad300 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f6709ffb640 10 client.25115 handle_command_reply: tid=0 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 client.25115 shutdown 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 msgr2=0x55d5ead8c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead8c3d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6714009fb0 tx=0x7f67140046c0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 shutdown_connections 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eadaf6f0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f6709ffb640 0 client.25115 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead8c9a0 0x55d5ead89f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead80a10 0x55d5ead89820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 --2- 172.21.15.33:0/2913876244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead8c3d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 >> 172.21.15.33:0/2913876244 conn(0x55d5ead7df20 msgr2=0x55d5ead83a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:49.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 shutdown_connections 2024-09-10T21:04:49.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 1 -- 172.21.15.33:0/2913876244 wait complete. 2024-09-10T21:04:49.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.089+0000 7f672afa1740 20 client.25115 trim_cache size 0 max 16384 2024-09-10T21:04:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 Processor -- start 2024-09-10T21:04:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 -- start start 2024-09-10T21:04:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead841d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5ead7d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6724070bb0 con 0x55d5ead80a10 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f67240726f0 con 0x55d5ead800f0 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672afa1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6724071280 con 0x55d5ead8c9a0 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5ead7d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5ead7d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43774/0 (socket says 172.21.15.33:43774) 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f6721d9b640 1 -- 172.21.15.33:0/2615821074 learned_addr learned my addr 172.21.15.33:0/2615821074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f6722d9d640 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.116+0000 7f672259c640 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead841d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f6722d9d640 1 -- 172.21.15.33:0/2615821074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 msgr2=0x55d5ead841d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f6722d9d640 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead841d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f6722d9d640 1 -- 172.21.15.33:0/2615821074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 msgr2=0x55d5ead7d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f6722d9d640 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5ead7d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f6722d9d640 1 -- 172.21.15.33:0/2615821074 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5ead895d0 con 0x55d5ead8c9a0 2024-09-10T21:04:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f6721d9b640 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5ead7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f6722d9d640 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8b560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6724102870 tx=0x7f671c013c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1a2dec091139822 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f670bfff640 1 -- 172.21.15.33:0/2615821074 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c013e60 con 0x55d5ead8c9a0 2024-09-10T21:04:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f670bfff640 1 -- 172.21.15.33:0/2615821074 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f671c0118d0 con 0x55d5ead8c9a0 2024-09-10T21:04:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.117+0000 7f670bfff640 1 -- 172.21.15.33:0/2615821074 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c0055e0 con 0x55d5ead8c9a0 2024-09-10T21:04:49.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 -- 172.21.15.33:0/2615821074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 msgr2=0x55d5ead8b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8b560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6724102870 tx=0x7f671c013c10 comp rx=0 tx=0).stop 2024-09-10T21:04:49.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 -- 172.21.15.33:0/2615821074 shutdown_connections 2024-09-10T21:04:49.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead8c9a0 0x55d5ead8b560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5ead7d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 --2- 172.21.15.33:0/2615821074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead800f0 0x55d5ead841d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 -- 172.21.15.33:0/2615821074 >> 172.21.15.33:0/2615821074 conn(0x55d5ead7df20 msgr2=0x55d5ead8ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 -- 172.21.15.33:0/2615821074 shutdown_connections 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.118+0000 7f672afa1740 1 -- 172.21.15.33:0/2615821074 wait complete. 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.119+0000 7f672afa1740 1 -- start start 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.119+0000 7f672afa1740 1 Processor -- start 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.119+0000 7f672afa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead800f0 0x55d5eada55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.119+0000 7f672afa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5eada5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.119+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5eada5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.119+0000 7f672259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead800f0 0x55d5eada55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead800f0 0x55d5eada55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:32860/0 (socket says 172.21.15.33:32860) 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f6721d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5eada5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43776/0 (socket says 172.21.15.33:43776) 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 -- 172.21.15.33:0/1864605587 learned_addr learned my addr 172.21.15.33:0/1864605587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672afa1740 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead8c9a0 0x55d5eada0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6724072020 con 0x55d5ead80a10 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6724071950 con 0x55d5ead8c9a0 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f672406eb70 con 0x55d5ead800f0 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f6722d9d640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead8c9a0 0x55d5eada0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 -- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead8c9a0 msgr2=0x55d5eada0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead8c9a0 0x55d5eada0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 -- 172.21.15.33:0/1864605587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 msgr2=0x55d5eada5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5eada5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 -- 172.21.15.33:0/1864605587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5eada0ab0 con 0x55d5ead800f0 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672afa1740 10 client.0 _ll_register_callbacks cb 0x7f671400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f6722d9d640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead8c9a0 0x55d5eada0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f6721d9b640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5eada5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f671c0118d0 con 0x55d5ead800f0 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f672259c640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead800f0 0x55d5eada55c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6714009fb0 tx=0x7f6714015890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7abe0e82bc60495 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f6709ffb640 10 client.25105 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:04:49.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.120+0000 7f6709ffb640 1 -- 172.21.15.33:0/1864605587 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6714014040 con 0x55d5ead800f0 2024-09-10T21:04:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f6709ffb640 1 -- 172.21.15.33:0/1864605587 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6714015260 con 0x55d5ead800f0 2024-09-10T21:04:49.130 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:04:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f6709ffb640 1 -- 172.21.15.33:0/1864605587 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f67140057d0 con 0x55d5ead800f0 2024-09-10T21:04:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 10 client.25105 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55d5eae1a8b0 con 0x55d5ead800f0 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f6709ffb640 1 -- 172.21.15.33:0/1864605587 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6714005250 con 0x55d5ead800f0 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 10 client.25105 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 10 client.25105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 20 client.25105 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eae1cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 4 client.25105 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5eacf48a0 con 0x55d5eadad300 2024-09-10T21:04:49.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.121+0000 7f6721d9b640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eae1cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f6709ffb640 10 client.25105 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f6721d9b640 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eae1cef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=641e77b608126b8d server_cookie=d5f0c80b0dfae01e in_seq=0 out_seq=0 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f6709ffb640 1 -- 172.21.15.33:0/1864605587 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5eacf48a0 con 0x55d5eadad300 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f6709ffb640 10 client.25105 handle_command_reply: tid=0 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 client.25105 shutdown 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead800f0 msgr2=0x55d5eada55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead800f0 0x55d5eada55c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6714009fb0 tx=0x7f6714015890 comp rx=0 tx=0).stop 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 shutdown_connections 2024-09-10T21:04:49.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d5eadad300 0x55d5eae1cef0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d5ead8c9a0 0x55d5eada0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d5ead80a10 0x55d5eada5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 --2- 172.21.15.33:0/1864605587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5ead800f0 0x55d5eada55c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 >> 172.21.15.33:0/1864605587 conn(0x55d5ead7df20 msgr2=0x55d5ead8b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f6709ffb640 0 client.25105 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 shutdown_connections 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.122+0000 7f672afa1740 1 -- 172.21.15.33:0/1864605587 wait complete. 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f672afa1740 20 client.25105 trim_cache size 0 max 16384 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f670c0b5230 msgr2=0x7f670c0b76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f670c0b5230 0x7f670c0b76f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6714005d00 tx=0x7f6714017040 comp rx=0 tx=0).stop 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67241021b0 msgr2=0x7f672419ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67241021b0 0x7f672419ba10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6718005940 tx=0x7f67180042f0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 shutdown_connections 2024-09-10T21:04:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f670c0b5230 0x7f670c0b76f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f672410f440 0x7f67241a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f67241021b0 0x7f672419ba10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 --2- 172.21.15.33:0/1133361947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6724101890 0x7f672419b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 >> 172.21.15.33:0/1133361947 conn(0x7f67240fd700 msgr2=0x7f67240ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 shutdown_connections 2024-09-10T21:04:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:49.123+0000 7f6728bcc640 1 -- 172.21.15.33:0/1133361947 wait complete. 2024-09-10T21:04:49.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:49 smithi104 ceph-mon[35325]: pgmap v371: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:49 smithi174 ceph-mon[35635]: pgmap v371: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:49.645 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:49 smithi033 ceph-mon[30860]: pgmap v371: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:50.987 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:04:50.988 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:04:51.336 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:51.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:51 smithi033 ceph-mon[30860]: pgmap v372: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:51 smithi104 ceph-mon[35325]: pgmap v372: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:51 smithi174 ceph-mon[35635]: pgmap v372: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 -- 172.21.15.33:0/749804160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d78101c90 msgr2=0x7f6d78102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/749804160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d78101c90 0x7f6d78102070 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009900 tx=0x7f6d6c02a410 comp rx=0 tx=0).stop 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 -- 172.21.15.33:0/749804160 shutdown_connections 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/749804160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d78108020 0x7f6d7810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/749804160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d781076a0 0x7f6d78107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/749804160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d78101c90 0x7f6d78102070 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 -- 172.21.15.33:0/749804160 >> 172.21.15.33:0/749804160 conn(0x7f6d780fd700 msgr2=0x7f6d780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 -- 172.21.15.33:0/749804160 shutdown_connections 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 -- 172.21.15.33:0/749804160 wait complete. 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.880+0000 7f6d7ed80640 1 Processor -- start 2024-09-10T21:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7ed80640 1 -- start start 2024-09-10T21:04:51.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7ed80640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d78101c90 0x7f6d781992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7ed80640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d781076a0 0x7f6d78199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7ed80640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d78108020 0x7f6d7819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7ed80640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78111ca0 con 0x7f6d781076a0 2024-09-10T21:04:51.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7ed80640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78111b20 con 0x7f6d78108020 2024-09-10T21:04:51.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7ed80640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d78111e20 con 0x7f6d78101c90 2024-09-10T21:04:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d78101c90 0x7f6d781992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d781076a0 0x7f6d78199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d78101c90 0x7f6d781992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59794/0 (socket says 172.21.15.33:59794) 2024-09-10T21:04:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/834867772 learned_addr learned my addr 172.21.15.33:0/834867772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d7d31d640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d78108020 0x7f6d7819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d77fff640 1 -- 172.21.15.33:0/834867772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d78101c90 msgr2=0x7f6d781992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d77fff640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d78101c90 0x7f6d781992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d77fff640 1 -- 172.21.15.33:0/834867772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d78108020 msgr2=0x7f6d7819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d77fff640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d78108020 0x7f6d7819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.881+0000 7f6d77fff640 1 -- 172.21.15.33:0/834867772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d7819f390 con 0x7f6d781076a0 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.882+0000 7f6d77fff640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d781076a0 0x7f6d78199820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005fd0 tx=0x7f6d6400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.882+0000 7f6d75ffb640 1 -- 172.21.15.33:0/834867772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64009c30 con 0x7f6d781076a0 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.882+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d7819f680 con 0x7f6d781076a0 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.882+0000 7f6d75ffb640 1 -- 172.21.15.33:0/834867772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d64009dd0 con 0x7f6d781076a0 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.882+0000 7f6d75ffb640 1 -- 172.21.15.33:0/834867772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64003e20 con 0x7f6d781076a0 2024-09-10T21:04:51.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.882+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d780fdae0 con 0x7f6d781076a0 2024-09-10T21:04:51.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.885+0000 7f6d81155740 1 Processor -- start 2024-09-10T21:04:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d81155740 1 -- start start 2024-09-10T21:04:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d81155740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d81155740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d81155740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d81155740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55893074ee60 con 0x5589307d8800 2024-09-10T21:04:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d81155740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55893073e290 con 0x5589307e4790 2024-09-10T21:04:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d81155740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589307420a0 con 0x5589307d7ee0 2024-09-10T21:04:51.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50784/0 (socket says 172.21.15.33:50784) 2024-09-10T21:04:51.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/4103609284 learned_addr learned my addr 172.21.15.33:0/4103609284 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:51.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7d31d640 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/4103609284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 msgr2=0x5589307d82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307d82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/4103609284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 msgr2=0x5589307e6b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307e6b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.886+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/4103609284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55893049c930 con 0x5589307d8800 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.887+0000 7f6d7d31d640 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.887+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e4250 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009a20 tx=0x7f6d6c031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdeb60b0f894390 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.887+0000 7f6d75ffb640 1 -- 172.21.15.33:0/834867772 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6d64003fe0 con 0x7f6d781076a0 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.887+0000 7f6d62ffd640 1 -- 172.21.15.33:0/4103609284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c031e60 con 0x5589307d8800 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.887+0000 7f6d62ffd640 1 -- 172.21.15.33:0/4103609284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6c031760 con 0x5589307d8800 2024-09-10T21:04:51.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.887+0000 7f6d62ffd640 1 -- 172.21.15.33:0/4103609284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c035040 con 0x5589307d8800 2024-09-10T21:04:51.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 -- 172.21.15.33:0/4103609284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 msgr2=0x5589307e4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d75ffb640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d680b52e0 0x7f6d680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e4250 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009a20 tx=0x7f6d6c031c40 comp rx=0 tx=0).stop 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 -- 172.21.15.33:0/4103609284 shutdown_connections 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e4250 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 --2- 172.21.15.33:0/4103609284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 -- 172.21.15.33:0/4103609284 >> 172.21.15.33:0/4103609284 conn(0x5589307d5d10 msgr2=0x5589307d6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d7d31d640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d680b52e0 0x7f6d680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d75ffb640 1 -- 172.21.15.33:0/834867772 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6d640e5730 con 0x7f6d781076a0 2024-09-10T21:04:51.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.888+0000 7f6d81155740 1 -- 172.21.15.33:0/4103609284 shutdown_connections 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.889+0000 7f6d81155740 1 -- 172.21.15.33:0/4103609284 wait complete. 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.889+0000 7f6d7d31d640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d680b52e0 0x7f6d680b77a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d70005fd0 tx=0x7f6d70009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.889+0000 7f6d81155740 1 -- start start 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.889+0000 7f6d81155740 1 Processor -- start 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d81155740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d81155740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d81155740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d81155740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55893059f890 con 0x5589307d8800 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d81155740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5589306d8d40 con 0x5589307e4790 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d81155740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589307e73e0 con 0x5589307d7ee0 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d81155740 10 client.0 _ll_register_callbacks cb 0x55893073cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307e17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59808/0 (socket says 172.21.15.33:59808) 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/2431135628 learned_addr learned my addr 172.21.15.33:0/2431135628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d77fff640 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/2431135628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 msgr2=0x5589307e2230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307e2230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/2431135628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 msgr2=0x5589307e1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/2431135628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589307f9050 con 0x5589307d7ee0 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.890+0000 7f6d77fff640 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307e17b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031440 tx=0x7f6d6c031f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4a3d90a6f70d933 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d57fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d57fff640 1 -- 172.21.15.33:0/2431135628 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c052060 con 0x5589307d7ee0 2024-09-10T21:04:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d6c031760 con 0x5589307d7ee0 2024-09-10T21:04:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d57fff640 1 -- 172.21.15.33:0/2431135628 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6d6c031560 con 0x5589307d7ee0 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d57fff640 1 -- 172.21.15.33:0/2431135628 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c035070 con 0x5589307d7ee0 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d81155740 10 client.25117 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.891+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5589307fba00 con 0x5589307d7ee0 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d57fff640 1 -- 172.21.15.33:0/2431135628 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d6c031760 con 0x5589307d7ee0 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d57fff640 1 -- 172.21.15.33:0/2431135628 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6d6c044740 con 0x5589307d7ee0 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d81155740 10 client.25117 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d81155740 10 client.25117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d81155740 20 client.25117 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d81155740 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589307fcd50 0x5589307ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d81155740 4 client.25117 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558930750db0 con 0x5589307fcd50 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.892+0000 7f6d77fff640 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589307fcd50 0x5589307ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.893+0000 7f6d57fff640 10 client.25117 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.893+0000 7f6d77fff640 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589307fcd50 0x5589307ff140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea5fc4b8c18a2b8 server_cookie=449beabf1e32e52f in_seq=0 out_seq=0 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.893+0000 7f6d57fff640 1 -- 172.21.15.33:0/2431135628 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558930750db0 con 0x5589307fcd50 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.893+0000 7f6d57fff640 10 client.25117 handle_command_reply: tid=0 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 client.25117 shutdown 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 msgr2=0x5589307e17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307e17b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031440 tx=0x7f6d6c031f30 comp rx=0 tx=0).stop 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 shutdown_connections 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589307fcd50 0x5589307ff140 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307e2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 --2- 172.21.15.33:0/2431135628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307e17b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 >> 172.21.15.33:0/2431135628 conn(0x5589307d5d10 msgr2=0x5589307d4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d57fff640 0 client.25117 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 shutdown_connections 2024-09-10T21:04:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 1 -- 172.21.15.33:0/2431135628 wait complete. 2024-09-10T21:04:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.894+0000 7f6d81155740 20 client.25117 trim_cache size 0 max 16384 2024-09-10T21:04:51.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 Processor -- start 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 -- start start 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307de9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ac290 con 0x5589307d8800 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781addd0 con 0x5589307e4790 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d81155740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ac960 con 0x5589307d7ee0 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50800/0 (socket says 172.21.15.33:50800) 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d77fff640 1 -- 172.21.15.33:0/441366289 learned_addr learned my addr 172.21.15.33:0/441366289 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.929+0000 7f6d7d31d640 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307de9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d77fff640 1 -- 172.21.15.33:0/441366289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 msgr2=0x5589307dbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:51.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d77fff640 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307dbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d77fff640 1 -- 172.21.15.33:0/441366289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 msgr2=0x5589307de9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:51.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d77fff640 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307de9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d77fff640 1 -- 172.21.15.33:0/441366289 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589307e0710 con 0x5589307d8800 2024-09-10T21:04:51.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:04:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d77fff640 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d640abf30 tx=0x7f6d640e3100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de27c34f107397d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d627fc640 1 -- 172.21.15.33:0/441366289 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d64020040 con 0x5589307d8800 2024-09-10T21:04:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d627fc640 1 -- 172.21.15.33:0/441366289 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d640b09d0 con 0x5589307d8800 2024-09-10T21:04:51.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.930+0000 7f6d627fc640 1 -- 172.21.15.33:0/441366289 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d640971c0 con 0x5589307d8800 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.931+0000 7f6d81155740 1 -- 172.21.15.33:0/441366289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 msgr2=0x5589307e1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.931+0000 7f6d81155740 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1e90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6d640abf30 tx=0x7f6d640e3100 comp rx=0 tx=0).stop 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 -- 172.21.15.33:0/441366289 shutdown_connections 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307e4790 0x5589307de9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x5589307e1e90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 --2- 172.21.15.33:0/441366289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307d7ee0 0x5589307dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 -- 172.21.15.33:0/441366289 >> 172.21.15.33:0/441366289 conn(0x5589307d5d10 msgr2=0x5589307e1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 -- 172.21.15.33:0/441366289 shutdown_connections 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 -- 172.21.15.33:0/441366289 wait complete. 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 -- start start 2024-09-10T21:04:51.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.932+0000 7f6d81155740 1 Processor -- start 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d81155740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307d7ee0 0x558930800130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d81155740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x558930800670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d81155740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307e4790 0x5589308688c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d81155740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ad700 con 0x5589307d8800 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d81155740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781ad030 con 0x5589307d7ee0 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307d7ee0 0x558930800130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d7cb1c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307d7ee0 0x558930800130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38276/0 (socket says 172.21.15.33:38276) 2024-09-10T21:04:51.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/40050082 learned_addr learned my addr 172.21.15.33:0/40050082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d77fff640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x558930800670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d781aa250 con 0x5589307e4790 2024-09-10T21:04:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.933+0000 7f6d7d31d640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307e4790 0x5589308688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/40050082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307e4790 msgr2=0x5589308688c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:51.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307e4790 0x5589308688c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/40050082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 msgr2=0x558930800670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x558930800670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d7cb1c640 1 -- 172.21.15.33:0/40050082 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558930868e00 con 0x5589307d7ee0 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d77fff640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x558930800670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d81155740 10 client.0 _ll_register_callbacks cb 0x7f6d70009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d7cb1c640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307d7ee0 0x558930800130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031440 tx=0x7f6d6c031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f494a752c4a189f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d57fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d640b09d0 con 0x5589307d7ee0 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d57fff640 1 -- 172.21.15.33:0/40050082 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c05c030 con 0x5589307d7ee0 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d57fff640 1 -- 172.21.15.33:0/40050082 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6d6c04a040 con 0x5589307d7ee0 2024-09-10T21:04:51.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d57fff640 1 -- 172.21.15.33:0/40050082 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d6c052070 con 0x5589307d7ee0 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d81155740 10 client.25130 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.934+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=28}) -- 0x558930869e90 con 0x5589307d7ee0 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d57fff640 1 -- 172.21.15.33:0/40050082 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f6d6c004030 con 0x5589307d7ee0 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d81155740 10 client.25130 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d81155740 10 client.25130 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d81155740 20 client.25130 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d81155740 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55893086a4b0 0x55893086c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d81155740 4 client.25130 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f6d640b09d0 con 0x55893086a4b0 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d77fff640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55893086a4b0 0x55893086c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d57fff640 10 client.25130 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:51.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.935+0000 7f6d77fff640 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55893086a4b0 0x55893086c8d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e06275173e6c491 server_cookie=db20d39123c4d2d in_seq=0 out_seq=0 2024-09-10T21:04:51.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d57fff640 1 -- 172.21.15.33:0/40050082 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f6d640b09d0 con 0x55893086a4b0 2024-09-10T21:04:51.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d57fff640 10 client.25130 handle_command_reply: tid=0 2024-09-10T21:04:51.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 client.25130 shutdown 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307d7ee0 msgr2=0x558930800130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307d7ee0 0x558930800130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c031440 tx=0x7f6d6c031810 comp rx=0 tx=0).stop 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 shutdown_connections 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55893086a4b0 0x55893086c8d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589307e4790 0x5589308688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d57fff640 0 client.25130 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589307d8800 0x558930800670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 --2- 172.21.15.33:0/40050082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589307d7ee0 0x558930800130 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.937+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 >> 172.21.15.33:0/40050082 conn(0x5589307d5d10 msgr2=0x5589307f8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.938+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 shutdown_connections 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.938+0000 7f6d81155740 1 -- 172.21.15.33:0/40050082 wait complete. 2024-09-10T21:04:51.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.938+0000 7f6d81155740 20 client.25130 trim_cache size 0 max 16384 2024-09-10T21:04:51.945 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:04:51.948 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:04:51.948 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "f9774723-3a34-4b63-a0e3-38b64011cc24", 2024-09-10T21:04:51.948 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:04:51.948 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:04:51.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d680b52e0 msgr2=0x7f6d680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d680b52e0 0x7f6d680b77a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6d70005fd0 tx=0x7f6d70009600 comp rx=0 tx=0).stop 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d781076a0 msgr2=0x7f6d78199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d781076a0 0x7f6d78199820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6d64005fd0 tx=0x7f6d6400dde0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 shutdown_connections 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d680b52e0 0x7f6d680b77a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d78108020 0x7f6d7819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d781076a0 0x7f6d78199820 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 --2- 172.21.15.33:0/834867772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d78101c90 0x7f6d781992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 >> 172.21.15.33:0/834867772 conn(0x7f6d780fd700 msgr2=0x7f6d7806ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 shutdown_connections 2024-09-10T21:04:51.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:51.939+0000 7f6d7ed80640 1 -- 172.21.15.33:0/834867772 wait complete. 2024-09-10T21:04:52.490 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:52 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:04:52.490 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:52 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:52.490 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:52 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:04:52.491 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:52 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:52 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:04:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:52 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:52 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:04:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:52 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:52 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:04:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:52 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:52 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:04:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:52 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:04:52.977 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:04:53.264 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:04:53.543 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:53 smithi033 ceph-mon[30860]: pgmap v373: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:53.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:53 smithi104 ceph-mon[35325]: pgmap v373: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:53.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:53 smithi174 ceph-mon[35635]: pgmap v373: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 -- 172.21.15.33:0/1406900118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5038103ca0 msgr2=0x7f5038104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 --2- 172.21.15.33:0/1406900118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5038103ca0 0x7f5038104080 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5028009900 tx=0x7f502802a410 comp rx=0 tx=0).stop 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 -- 172.21.15.33:0/1406900118 shutdown_connections 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 --2- 172.21.15.33:0/1406900118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f503810d100 0x7f503810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 --2- 172.21.15.33:0/1406900118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50381045c0 0x7f503810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 --2- 172.21.15.33:0/1406900118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5038103ca0 0x7f5038104080 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 -- 172.21.15.33:0/1406900118 >> 172.21.15.33:0/1406900118 conn(0x7f50380fd700 msgr2=0x7f50380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 -- 172.21.15.33:0/1406900118 shutdown_connections 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.779+0000 7f503d88c640 1 -- 172.21.15.33:0/1406900118 wait complete. 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.780+0000 7f503d88c640 1 Processor -- start 2024-09-10T21:04:53.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.780+0000 7f503d88c640 1 -- start start 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f503d88c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5038103ca0 0x7f50381992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f503d88c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50381045c0 0x7f5038199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f503d88c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f503810d100 0x7f503819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f503d88c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5038112d70 con 0x7f503810d100 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f50377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f503810d100 0x7f503819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f50367fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50381045c0 0x7f5038199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f50367fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50381045c0 0x7f5038199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38296/0 (socket says 172.21.15.33:38296) 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f50367fc640 1 -- 172.21.15.33:0/3534675885 learned_addr learned my addr 172.21.15.33:0/3534675885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f50377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f503810d100 0x7f503819eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50822/0 (socket says 172.21.15.33:50822) 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5038112bf0 con 0x7f50381045c0 2024-09-10T21:04:53.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.781+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5038112ef0 con 0x7f5038103ca0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f5036ffd640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5038103ca0 0x7f50381992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f5036ffd640 1 -- 172.21.15.33:0/3534675885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50381045c0 msgr2=0x7f5038199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f5036ffd640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50381045c0 0x7f5038199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f5036ffd640 1 -- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f503810d100 msgr2=0x7f503819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f5036ffd640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f503810d100 0x7f503819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f5036ffd640 1 -- 172.21.15.33:0/3534675885 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503819f370 con 0x7f5038103ca0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f5036ffd640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5038103ca0 0x7f50381992c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5028006010 tx=0x7f5028004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.782+0000 7f503c88a640 1 -- 172.21.15.33:0/3534675885 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502803f070 con 0x7f5038103ca0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.783+0000 7f503c88a640 1 -- 172.21.15.33:0/3534675885 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5028003ca0 con 0x7f5038103ca0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.783+0000 7f503c88a640 1 -- 172.21.15.33:0/3534675885 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028035070 con 0x7f5038103ca0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.783+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503819f600 con 0x7f5038103ca0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.783+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503819f9d0 con 0x7f5038103ca0 2024-09-10T21:04:53.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.785+0000 7f503fc61740 1 Processor -- start 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.785+0000 7f503fc61740 1 -- start start 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f503fc61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4eb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f503fc61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4fb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f503fc61740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4fe150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f503fc61740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae4f461e50 con 0x55ae4f4fbd60 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f503fc61740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ae4f451280 con 0x55ae4f4eb9c0 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f503fc61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae4f455090 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f50367fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4fe150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f50367fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4fe150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50836/0 (socket says 172.21.15.33:50836) 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f50367fc640 1 -- 172.21.15.33:0/1442991970 learned_addr learned my addr 172.21.15.33:0/1442991970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f50377fe640 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4eb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f50367fc640 1 -- 172.21.15.33:0/1442991970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 msgr2=0x55ae4f4eb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f503c88a640 1 -- 172.21.15.33:0/3534675885 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f502804b060 con 0x7f5038103ca0 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f50367fc640 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4eb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.786+0000 7f50367fc640 1 -- 172.21.15.33:0/1442991970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb9c0 msgr2=0x55ae4f4fb820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.787+0000 7f50367fc640 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4fb820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.787+0000 7f50367fc640 1 -- 172.21.15.33:0/1442991970 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4f1af650 con 0x55ae4f4fbd60 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.787+0000 7f50377fe640 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.787+0000 7f50367fc640 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4fe150 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d40 tx=0x7f502c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617acbd883d06251 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.787+0000 7f503c88a640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50200b5360 0x7f50200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f501dffb640 1 -- 172.21.15.33:0/1442991970 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c004220 con 0x55ae4f4fbd60 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f501dffb640 1 -- 172.21.15.33:0/1442991970 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0043c0 con 0x55ae4f4fbd60 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f501dffb640 1 -- 172.21.15.33:0/1442991970 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c0197d0 con 0x55ae4f4fbd60 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f503c88a640 1 -- 172.21.15.33:0/3534675885 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5028105c40 con 0x7f5038103ca0 2024-09-10T21:04:53.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f50377fe640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50200b5360 0x7f50200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f503fc61740 1 -- 172.21.15.33:0/1442991970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 msgr2=0x55ae4f4fe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f503fc61740 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4fe150 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f502c009d40 tx=0x7f502c011810 comp rx=0 tx=0).stop 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.788+0000 7f50377fe640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50200b5360 0x7f50200b7820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5030005fd0 tx=0x7f5030009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 -- 172.21.15.33:0/1442991970 shutdown_connections 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4fe150 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4fb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 --2- 172.21.15.33:0/1442991970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4eb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 -- 172.21.15.33:0/1442991970 >> 172.21.15.33:0/1442991970 conn(0x55ae4f4e8ed0 msgr2=0x55ae4f4ea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 -- 172.21.15.33:0/1442991970 shutdown_connections 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 -- 172.21.15.33:0/1442991970 wait complete. 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.789+0000 7f503fc61740 1 -- start start 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 1 Processor -- start 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4fb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ae4f2b2700 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ae4f3ec110 con 0x55ae4f4fbd60 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae4f4fea10 con 0x55ae4f4eb9c0 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f503fc61740 10 client.0 _ll_register_callbacks cb 0x55ae4f405990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.790+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4fb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4fb660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50848/0 (socket says 172.21.15.33:50848) 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 -- 172.21.15.33:0/1896365050 learned_addr learned my addr 172.21.15.33:0/1896365050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f50367fc640 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f50377fe640 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 -- 172.21.15.33:0/1896365050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 msgr2=0x55ae4f4f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 -- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 msgr2=0x55ae4f4f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4f90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 -- 172.21.15.33:0/1896365050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4f419c40 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5036ffd640 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4fb660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f50280c9720 tx=0x7f50280cc280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df92cdc53fbf0169 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5016ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5016ffd640 1 -- 172.21.15.33:0/1896365050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028048040 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5016ffd640 1 -- 172.21.15.33:0/1896365050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50280cc910 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f502c0043c0 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f5016ffd640 1 -- 172.21.15.33:0/1896365050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50280d52d0 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f503fc61740 10 client.15537 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.791+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ae4f50a380 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f5016ffd640 1 -- 172.21.15.33:0/1896365050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502c0043c0 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f5016ffd640 1 -- 172.21.15.33:0/1896365050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f50280d5500 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f503fc61740 10 client.15537 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f503fc61740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f503fc61740 20 client.15537 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f503fc61740 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f50dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f503fc61740 4 client.15537 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae4f463da0 con 0x55ae4f50b7e0 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.792+0000 7f50367fc640 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f50dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.793+0000 7f50367fc640 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f50dbd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd87cefb482f56cd server_cookie=8b4388b1d07ab233 in_seq=0 out_seq=0 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.793+0000 7f5016ffd640 10 client.15537 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.793+0000 7f5016ffd640 1 -- 172.21.15.33:0/1896365050 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ae4f463da0 con 0x55ae4f50b7e0 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.793+0000 7f5016ffd640 10 client.15537 handle_command_reply: tid=0 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.793+0000 7f503fc61740 1 client.15537 shutdown 2024-09-10T21:04:53.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb0a0 msgr2=0x55ae4f4fb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4fb660 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f50280c9720 tx=0x7f50280cc280 comp rx=0 tx=0).stop 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 shutdown_connections 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f50dbd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f4f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f5016ffd640 0 client.15537 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 --2- 172.21.15.33:0/1896365050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4fb660 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 >> 172.21.15.33:0/1896365050 conn(0x55ae4f4e8ed0 msgr2=0x55ae4f4eea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 shutdown_connections 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 1 -- 172.21.15.33:0/1896365050 wait complete. 2024-09-10T21:04:53.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.794+0000 7f503fc61740 20 client.15537 trim_cache size 0 max 16384 2024-09-10T21:04:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.816+0000 7f503fc61740 1 Processor -- start 2024-09-10T21:04:53.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.816+0000 7f503fc61740 1 -- start start 2024-09-10T21:04:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f503fc61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f507cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f503fc61740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f513470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f503fc61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f5139b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f503fc61740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50381ac0c0 con 0x55ae4f4eb9c0 2024-09-10T21:04:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f503fc61740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50381adc00 con 0x55ae4f4fbd60 2024-09-10T21:04:53.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f503fc61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50381ac790 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f507cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f507cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59866/0 (socket says 172.21.15.33:59866) 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 -- 172.21.15.33:0/2824828001 learned_addr learned my addr 172.21.15.33:0/2824828001 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f50377fe640 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f5139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 -- 172.21.15.33:0/2824828001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 msgr2=0x55ae4f5139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f5139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 -- 172.21.15.33:0/2824828001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb9c0 msgr2=0x55ae4f513470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f513470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.817+0000 7f5036ffd640 1 -- 172.21.15.33:0/2824828001 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae4f4e98a0 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.818+0000 7f5036ffd640 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f507cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5028005c40 tx=0x7f50280cc3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97a24d38e270e48f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:53.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.818+0000 7f501d7fa640 1 -- 172.21.15.33:0/2824828001 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502807c020 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.818+0000 7f501d7fa640 1 -- 172.21.15.33:0/2824828001 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50280cbfc0 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.818+0000 7f501d7fa640 1 -- 172.21.15.33:0/2824828001 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5028048030 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 -- 172.21.15.33:0/2824828001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 msgr2=0x55ae4f507cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f507cf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5028005c40 tx=0x7f50280cc3b0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 -- 172.21.15.33:0/2824828001 shutdown_connections 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4fbd60 0x55ae4f5139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f513470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 --2- 172.21.15.33:0/2824828001 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f507cf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 -- 172.21.15.33:0/2824828001 >> 172.21.15.33:0/2824828001 conn(0x55ae4f4e8ed0 msgr2=0x55ae4f4e9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 -- 172.21.15.33:0/2824828001 shutdown_connections 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 -- 172.21.15.33:0/2824828001 wait complete. 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.819+0000 7f503fc61740 1 -- start start 2024-09-10T21:04:53.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f503fc61740 1 Processor -- start 2024-09-10T21:04:53.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f503fc61740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f503fc61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f503fc61740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f501d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f50377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f501d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f50377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f501d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50868/0 (socket says 172.21.15.33:50868) 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f50377fe640 1 -- 172.21.15.33:0/1710071441 learned_addr learned my addr 172.21.15.33:0/1710071441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f503fc61740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50381ad530 con 0x55ae4f4fbd60 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f5036ffd640 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f50367fc640 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50381ace60 con 0x55ae4f4eb0a0 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.820+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50381aa080 con 0x55ae4f4eb9c0 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f503fc61740 10 client.0 _ll_register_callbacks cb 0x7f5028005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f50377fe640 1 -- 172.21.15.33:0/1710071441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 msgr2=0x55ae4f4f3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:53.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f50377fe640 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f50377fe640 1 -- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb0a0 msgr2=0x55ae4f4f35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:04:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f50377fe640 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4f35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f50377fe640 1 -- 172.21.15.33:0/1710071441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae4f502430 con 0x55ae4f4fbd60 2024-09-10T21:04:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50280cbfc0 con 0x55ae4f4fbd60 2024-09-10T21:04:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f50377fe640 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f501d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5030012ca0 tx=0x7f5030004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb8a0c3dbb59c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:04:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f5016ffd640 10 client.15540 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:04:53.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.821+0000 7f5016ffd640 1 -- 172.21.15.33:0/1710071441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f503002e060 con 0x55ae4f4fbd60 2024-09-10T21:04:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f5016ffd640 1 -- 172.21.15.33:0/1710071441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5030029410 con 0x55ae4f4fbd60 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f5016ffd640 1 -- 172.21.15.33:0/1710071441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5030029650 con 0x55ae4f4fbd60 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 10 client.15540 fetch_fsmap learned FSMap version 28 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55ae4f503740 con 0x55ae4f4fbd60 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f5016ffd640 1 -- 172.21.15.33:0/1710071441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f503000f040 con 0x55ae4f4fbd60 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 10 client.15540 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 10 client.15540 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 20 client.15540 populate_metadata read hostname 'smithi033' 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f503d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 4 client.15540 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae4f45f850 con 0x55ae4f50b7e0 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.822+0000 7f5036ffd640 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f503d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:04:53.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.823+0000 7f5016ffd640 10 client.15540 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.823+0000 7f5036ffd640 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f503d50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb307beaeb70e6b9 server_cookie=6eb5fa8e9fd3127f in_seq=0 out_seq=0 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.823+0000 7f5016ffd640 1 -- 172.21.15.33:0/1710071441 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x55ae4f45f850 con 0x55ae4f50b7e0 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.823+0000 7f5016ffd640 10 client.15540 handle_command_reply: tid=0 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.823+0000 7f503fc61740 1 client.15540 shutdown 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.823+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 msgr2=0x55ae4f501d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.823+0000 7f503fc61740 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f501d40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5030012ca0 tx=0x7f5030004a40 comp rx=0 tx=0).stop 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 shutdown_connections 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55ae4f50b7e0 0x55ae4f503d50 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ae4f4fbd60 0x55ae4f501d40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae4f4eb9c0 0x55ae4f4f3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "f9774723-3a34-4b63-a0e3-38b64011cc24", 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:04:53.831 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:04:53.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:53.833 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:04:53.833 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:04:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 --2- 172.21.15.33:0/1710071441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ae4f4eb0a0 0x55ae4f4f35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 >> 172.21.15.33:0/1710071441 conn(0x55ae4f4e8ed0 msgr2=0x55ae4f4e7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f5016ffd640 0 client.15540 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:04:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 shutdown_connections 2024-09-10T21:04:53.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 1 -- 172.21.15.33:0/1710071441 wait complete. 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.824+0000 7f503fc61740 20 client.15540 trim_cache size 0 max 16384 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50200b5360 msgr2=0x7f50200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50200b5360 0x7f50200b7820 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5030005fd0 tx=0x7f5030009600 comp rx=0 tx=0).stop 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5038103ca0 msgr2=0x7f50381992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5038103ca0 0x7f50381992c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5028006010 tx=0x7f5028004060 comp rx=0 tx=0).stop 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 shutdown_connections 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50200b5360 0x7f50200b7820 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f503810d100 0x7f503819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50381045c0 0x7f5038199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 --2- 172.21.15.33:0/3534675885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5038103ca0 0x7f50381992c0 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5028006010 tx=0x7f5028004060 comp rx=0 tx=0).stop 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.825+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 >> 172.21.15.33:0/3534675885 conn(0x7f50380fd700 msgr2=0x7f50380fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:04:53.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.826+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 shutdown_connections 2024-09-10T21:04:53.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:04:53.826+0000 7f503d88c640 1 -- 172.21.15.33:0/3534675885 wait complete. 2024-09-10T21:04:54.692 INFO:tasks.cephfs.filesystem:scrub status for tag:f9774723-3a34-4b63-a0e3-38b64011cc24 - {'path': '/', 'tag': 'f9774723-3a34-4b63-a0e3-38b64011cc24', 'options': 'recursive,force'} 2024-09-10T21:04:55.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:55 smithi104 ceph-mon[35325]: pgmap v374: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:55 smithi174 ceph-mon[35635]: pgmap v374: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:55 smithi033 ceph-mon[30860]: pgmap v374: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-10T21:04:56.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:04:56.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:04:56.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:04:57.516 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 96% (11705/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-10T21:04:57.545 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:57 smithi174 ceph-mon[35635]: pgmap v375: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T21:04:57.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:57 smithi104 ceph-mon[35325]: pgmap v375: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T21:04:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:57 smithi033 ceph-mon[30860]: pgmap v375: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T21:04:58.450 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T21:04:58.451 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-09-10T21:04:58.452 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:04:58.455 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T21:04:58.601 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T21:04:58.603 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T21:04:58.603 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T21:04:59.335 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T21:04:59.415 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T21:04:59.456 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T21:04:59.457 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T21:04:59.457 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T21:04:59.498 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T21:04:59.502 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T21:04:59.502 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T21:04:59.518 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-10T21:04:59.520 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-10T21:04:59.520 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-10T21:04:59.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:04:59 smithi104 ceph-mon[35325]: pgmap v376: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-10T21:04:59.581 INFO:tasks.workunit.client.0.smithi033.stderr:+ dbench 1 2024-09-10T21:04:59.589 INFO:tasks.workunit.client.0.smithi033.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T21:04:59.589 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-09-10T21:04:59.589 INFO:tasks.workunit.client.0.smithi033.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T21:04:59.595 INFO:tasks.workunit.client.0.smithi033.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-10T21:04:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:04:59 smithi174 ceph-mon[35635]: pgmap v376: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-10T21:04:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:04:59 smithi033 ceph-mon[30860]: pgmap v376: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-10T21:04:59.695 INFO:tasks.workunit.client.0.smithi033.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-10T21:04:59.696 INFO:tasks.workunit.client.0.smithi033.stdout:releasing clients 2024-09-10T21:05:00.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 3927 40.06 MB/sec warmup 1 sec latency 6.001 ms 2024-09-10T21:05:01.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:01 smithi104 ceph-mon[35325]: pgmap v377: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T21:05:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:01 smithi174 ceph-mon[35635]: pgmap v377: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T21:05:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:01 smithi033 ceph-mon[30860]: pgmap v377: 97 pgs: 97 active+clean; 662 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-10T21:05:01.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 7464 29.59 MB/sec warmup 2 sec latency 20.590 ms 2024-09-10T21:05:02.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 10987 26.10 MB/sec warmup 3 sec latency 13.363 ms 2024-09-10T21:05:03.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:03 smithi104 ceph-mon[35325]: pgmap v378: 97 pgs: 97 active+clean; 1.2 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-09-10T21:05:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:03 smithi174 ceph-mon[35635]: pgmap v378: 97 pgs: 97 active+clean; 1.2 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-09-10T21:05:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:03 smithi033 ceph-mon[30860]: pgmap v378: 97 pgs: 97 active+clean; 1.2 MiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 51 KiB/s wr, 10 op/s 2024-09-10T21:05:03.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 14475 24.28 MB/sec warmup 4 sec latency 12.782 ms 2024-09-10T21:05:04.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 17872 23.05 MB/sec warmup 5 sec latency 36.246 ms 2024-09-10T21:05:05.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:05 smithi104 ceph-mon[35325]: pgmap v379: 97 pgs: 97 active+clean; 5.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 435 KiB/s wr, 64 op/s 2024-09-10T21:05:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:05 smithi174 ceph-mon[35635]: pgmap v379: 97 pgs: 97 active+clean; 5.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 435 KiB/s wr, 64 op/s 2024-09-10T21:05:05.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:05 smithi033 ceph-mon[30860]: pgmap v379: 97 pgs: 97 active+clean; 5.0 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 435 KiB/s wr, 64 op/s 2024-09-10T21:05:05.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 21465 22.40 MB/sec warmup 6 sec latency 12.147 ms 2024-09-10T21:05:06.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 24879 21.89 MB/sec warmup 7 sec latency 20.959 ms 2024-09-10T21:05:07.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:07 smithi104 ceph-mon[35325]: pgmap v380: 97 pgs: 97 active+clean; 5.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 531 KiB/s wr, 69 op/s 2024-09-10T21:05:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:07 smithi174 ceph-mon[35635]: pgmap v380: 97 pgs: 97 active+clean; 5.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 531 KiB/s wr, 69 op/s 2024-09-10T21:05:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:07 smithi033 ceph-mon[30860]: pgmap v380: 97 pgs: 97 active+clean; 5.8 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 531 KiB/s wr, 69 op/s 2024-09-10T21:05:07.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 28068 21.02 MB/sec warmup 8 sec latency 13.283 ms 2024-09-10T21:05:08.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 31592 20.79 MB/sec warmup 9 sec latency 12.583 ms 2024-09-10T21:05:09.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:09 smithi104 ceph-mon[35325]: pgmap v381: 97 pgs: 97 active+clean; 12 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-09-10T21:05:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:09 smithi174 ceph-mon[35635]: pgmap v381: 97 pgs: 97 active+clean; 12 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-09-10T21:05:09.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:09 smithi033 ceph-mon[30860]: pgmap v381: 97 pgs: 97 active+clean; 12 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-09-10T21:05:09.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 35287 20.75 MB/sec warmup 10 sec latency 12.621 ms 2024-09-10T21:05:10.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 38528 20.44 MB/sec warmup 11 sec latency 15.015 ms 2024-09-10T21:05:11.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:11 smithi104 ceph-mon[35325]: pgmap v382: 97 pgs: 97 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-09-10T21:05:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:11 smithi174 ceph-mon[35635]: pgmap v382: 97 pgs: 97 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-09-10T21:05:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:11 smithi033 ceph-mon[30860]: pgmap v382: 97 pgs: 97 active+clean; 14 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 170 op/s 2024-09-10T21:05:11.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 42050 20.32 MB/sec warmup 12 sec latency 11.862 ms 2024-09-10T21:05:12.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:05:12.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:05:12.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:05:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:05:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:05:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:12.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:05:12.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:05:12.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 44611 19.69 MB/sec warmup 13 sec latency 19.895 ms 2024-09-10T21:05:13.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:13 smithi104 ceph-mon[35325]: pgmap v383: 97 pgs: 97 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-10T21:05:13.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:13 smithi174 ceph-mon[35635]: pgmap v383: 97 pgs: 97 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-10T21:05:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:13.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:13 smithi033 ceph-mon[30860]: pgmap v383: 97 pgs: 97 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-09-10T21:05:13.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:05:13.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 47853 19.63 MB/sec warmup 14 sec latency 12.506 ms 2024-09-10T21:05:14.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 50964 19.39 MB/sec warmup 15 sec latency 12.539 ms 2024-09-10T21:05:15.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:15 smithi104 ceph-mon[35325]: pgmap v384: 97 pgs: 97 active+clean; 19 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-09-10T21:05:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:15 smithi174 ceph-mon[35635]: pgmap v384: 97 pgs: 97 active+clean; 19 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-09-10T21:05:15.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:15 smithi033 ceph-mon[30860]: pgmap v384: 97 pgs: 97 active+clean; 19 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.9 MiB/s wr, 246 op/s 2024-09-10T21:05:15.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 54226 19.30 MB/sec warmup 16 sec latency 14.617 ms 2024-09-10T21:05:16.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 57400 19.23 MB/sec warmup 17 sec latency 14.708 ms 2024-09-10T21:05:17.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:17 smithi104 ceph-mon[35325]: pgmap v385: 97 pgs: 97 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-09-10T21:05:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:17 smithi174 ceph-mon[35635]: pgmap v385: 97 pgs: 97 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-09-10T21:05:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:17 smithi033 ceph-mon[30860]: pgmap v385: 97 pgs: 97 active+clean; 20 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-09-10T21:05:17.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 60782 19.22 MB/sec warmup 18 sec latency 13.187 ms 2024-09-10T21:05:18.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 64174 19.20 MB/sec warmup 19 sec latency 14.333 ms 2024-09-10T21:05:19.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:19 smithi104 ceph-mon[35325]: pgmap v386: 97 pgs: 97 active+clean; 27 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-09-10T21:05:19.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:19 smithi174 ceph-mon[35635]: pgmap v386: 97 pgs: 97 active+clean; 27 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-09-10T21:05:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:19 smithi033 ceph-mon[30860]: pgmap v386: 97 pgs: 97 active+clean; 27 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 292 op/s 2024-09-10T21:05:19.696 INFO:tasks.workunit.client.0.smithi033.stdout: 1 67581 19.15 MB/sec warmup 20 sec latency 17.607 ms 2024-09-10T21:05:20.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 70606 18.93 MB/sec warmup 21 sec latency 15.335 ms 2024-09-10T21:05:21.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:21 smithi104 ceph-mon[35325]: pgmap v387: 97 pgs: 97 active+clean; 27 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-09-10T21:05:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:21 smithi174 ceph-mon[35635]: pgmap v387: 97 pgs: 97 active+clean; 27 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-09-10T21:05:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:21 smithi033 ceph-mon[30860]: pgmap v387: 97 pgs: 97 active+clean; 27 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 739 KiB/s rd, 1.7 MiB/s wr, 203 op/s 2024-09-10T21:05:21.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 73487 18.69 MB/sec warmup 22 sec latency 14.725 ms 2024-09-10T21:05:22.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 76522 18.61 MB/sec warmup 23 sec latency 12.282 ms 2024-09-10T21:05:23.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:23 smithi104 ceph-mon[35325]: pgmap v388: 97 pgs: 97 active+clean; 28 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-09-10T21:05:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:23 smithi174 ceph-mon[35635]: pgmap v388: 97 pgs: 97 active+clean; 28 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-09-10T21:05:23.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:23 smithi033 ceph-mon[30860]: pgmap v388: 97 pgs: 97 active+clean; 28 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-09-10T21:05:23.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 79370 18.50 MB/sec warmup 24 sec latency 15.210 ms 2024-09-10T21:05:24.694 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:05:24.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 82375 18.45 MB/sec warmup 25 sec latency 16.798 ms 2024-09-10T21:05:24.975 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:25.388 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:25 smithi033 ceph-mon[30860]: pgmap v389: 97 pgs: 97 active+clean; 34 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-09-10T21:05:25.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:25 smithi104 ceph-mon[35325]: pgmap v389: 97 pgs: 97 active+clean; 34 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-09-10T21:05:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:25 smithi174 ceph-mon[35635]: pgmap v389: 97 pgs: 97 active+clean; 34 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-09-10T21:05:25.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 85587 18.46 MB/sec warmup 26 sec latency 36.041 ms 2024-09-10T21:05:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.862+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/96203458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a68101c90 msgr2=0x7f2a68102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.862+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/96203458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a68101c90 0x7f2a68102070 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009900 tx=0x7f2a6402a410 comp rx=0 tx=0).stop 2024-09-10T21:05:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.863+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/96203458 shutdown_connections 2024-09-10T21:05:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.863+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/96203458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68108020 0x7f2a6810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.863+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/96203458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a681076a0 0x7f2a68107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.863+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/96203458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a68101c90 0x7f2a68102070 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.863+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/96203458 >> 172.21.15.33:0/96203458 conn(0x7f2a680fd700 msgr2=0x7f2a680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.863+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/96203458 shutdown_connections 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.863+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/96203458 wait complete. 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 Processor -- start 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 -- start start 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68101c90 0x7f2a6819b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a681076a0 0x7f2a6819ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a68108020 0x7f2a681a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68111d30 con 0x7f2a681076a0 2024-09-10T21:05:25.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68111bb0 con 0x7f2a68101c90 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6f8fd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a68111eb0 con 0x7f2a68108020 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6ce98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a681076a0 0x7f2a6819ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6de9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a68108020 0x7f2a681a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6de9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a68108020 0x7f2a681a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59780/0 (socket says 172.21.15.33:59780) 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6de9a640 1 -- 172.21.15.33:0/2791496472 learned_addr learned my addr 172.21.15.33:0/2791496472 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.864+0000 7f2a6ce98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a681076a0 0x7f2a6819ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39224/0 (socket says 172.21.15.33:39224) 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6de9a640 1 -- 172.21.15.33:0/2791496472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68101c90 msgr2=0x7f2a6819b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68101c90 0x7f2a6819b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6de9a640 1 -- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a681076a0 msgr2=0x7f2a6819ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a681076a0 0x7f2a6819ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6de9a640 1 -- 172.21.15.33:0/2791496472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a681a15d0 con 0x7f2a68108020 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a68108020 0x7f2a681a0e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000eb10 tx=0x7f2a6000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a5e7fc640 1 -- 172.21.15.33:0/2791496472 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60004b20 con 0x7f2a68108020 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a681a18c0 con 0x7f2a68108020 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a5e7fc640 1 -- 172.21.15.33:0/2791496472 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a60004cc0 con 0x7f2a68108020 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a5e7fc640 1 -- 172.21.15.33:0/2791496472 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a60010670 con 0x7f2a68108020 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.865+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a681a7f80 con 0x7f2a68108020 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.868+0000 7f2a5e7fc640 1 -- 172.21.15.33:0/2791496472 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2a60005020 con 0x7f2a68108020 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 Processor -- start 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 -- start start 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee95ed480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a5e7fc640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a540b52e0 0x7f2a540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee95f9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95fbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ee9563e50 con 0x559ee95ed0a0 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ee9553280 con 0x559ee95f9950 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.869+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ee9557090 con 0x559ee95ed9c0 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95fbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95fbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:34004/0 (socket says 172.21.15.33:34004) 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 -- 172.21.15.33:0/4179767844 learned_addr learned my addr 172.21.15.33:0/4179767844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee95f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 -- 172.21.15.33:0/4179767844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 msgr2=0x559ee95f9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee95f9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 -- 172.21.15.33:0/4179767844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 msgr2=0x559ee95ed480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:25.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee95ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6de9a640 1 -- 172.21.15.33:0/4179767844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee92b1650 con 0x559ee95f9950 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a6d699640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a540b52e0 0x7f2a540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.870+0000 7f2a5e7fc640 1 -- 172.21.15.33:0/2791496472 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2a600e61b0 con 0x7f2a68108020 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.871+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95fbd40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a600c6a60 tx=0x7f2a600ac3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7653e479a22bf93d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.871+0000 7f2a6d699640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a540b52e0 0x7f2a540b77a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402a930 tx=0x7f2a64031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.872+0000 7f2a537fe640 1 -- 172.21.15.33:0/4179767844 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600ac480 con 0x559ee95f9950 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.872+0000 7f2a537fe640 1 -- 172.21.15.33:0/4179767844 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a600ac970 con 0x559ee95f9950 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.872+0000 7f2a537fe640 1 -- 172.21.15.33:0/4179767844 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a600b1450 con 0x559ee95f9950 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4179767844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 msgr2=0x559ee95fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95fbd40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2a600c6a60 tx=0x7f2a600ac3f0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4179767844 shutdown_connections 2024-09-10T21:05:25.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95fbd40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee95f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4179767844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee95ed480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4179767844 >> 172.21.15.33:0/4179767844 conn(0x559ee95eaed0 msgr2=0x559ee95ec070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4179767844 shutdown_connections 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4179767844 wait complete. 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 -- start start 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.873+0000 7f2a71cd2740 1 Processor -- start 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95f9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ee93b4700 con 0x559ee95f9950 2024-09-10T21:05:25.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ee94ee110 con 0x559ee95ed9c0 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ee960d5e0 con 0x559ee95ed0a0 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a71cd2740 10 client.0 _ll_register_callbacks cb 0x559ee9507990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6ce98640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6de9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95f9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59798/0 (socket says 172.21.15.33:59798) 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 -- 172.21.15.33:0/4053557644 learned_addr learned my addr 172.21.15.33:0/4053557644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6de9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39242/0 (socket says 172.21.15.33:39242) 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 -- 172.21.15.33:0/4053557644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 msgr2=0x559ee95f6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 -- 172.21.15.33:0/4053557644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 msgr2=0x559ee95f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:25.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.874+0000 7f2a6d699640 1 -- 172.21.15.33:0/4053557644 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee951bc40 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a6d699640 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95f9240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a64004570 tx=0x7f2a64009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72374d1cd073dec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a50ff9640 1 -- 172.21.15.33:0/4053557644 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6404b020 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a600ac970 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a50ff9640 1 -- 172.21.15.33:0/4053557644 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2a640039e0 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a50ff9640 1 -- 172.21.15.33:0/4053557644 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a64035040 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a71cd2740 10 client.25147 fetch_fsmap learned FSMap version 28 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.875+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ee9618f90 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a50ff9640 1 -- 172.21.15.33:0/4053557644 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a600ac970 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a50ff9640 1 -- 172.21.15.33:0/4053557644 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a64031ce0 con 0x559ee95ed0a0 2024-09-10T21:05:25.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a71cd2740 10 client.25147 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a71cd2740 10 client.25147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a71cd2740 20 client.25147 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee961a860 0x559ee961cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a71cd2740 4 client.25147 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ee9565da0 con 0x559ee961a860 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.876+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee961a860 0x559ee961cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.877+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee961a860 0x559ee961cca0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb85b8e78273383 server_cookie=2de92a8196274053 in_seq=0 out_seq=0 2024-09-10T21:05:25.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.877+0000 7f2a50ff9640 10 client.25147 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.878+0000 7f2a50ff9640 1 -- 172.21.15.33:0/4053557644 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ee9565da0 con 0x559ee961a860 2024-09-10T21:05:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.878+0000 7f2a50ff9640 10 client.25147 handle_command_reply: tid=0 2024-09-10T21:05:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.878+0000 7f2a71cd2740 1 client.25147 shutdown 2024-09-10T21:05:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 msgr2=0x559ee95f9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95f9240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2a64004570 tx=0x7f2a64009260 comp rx=0 tx=0).stop 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 shutdown_connections 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee961a860 0x559ee961cca0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a50ff9640 0 client.25147 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/4053557644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95f9240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 >> 172.21.15.33:0/4053557644 conn(0x559ee95eaed0 msgr2=0x559ee95f0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 shutdown_connections 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 1 -- 172.21.15.33:0/4053557644 wait complete. 2024-09-10T21:05:25.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.879+0000 7f2a71cd2740 20 client.25147 trim_cache size 0 max 16384 2024-09-10T21:05:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 Processor -- start 2024-09-10T21:05:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 -- start start 2024-09-10T21:05:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95e5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a680750b0 con 0x559ee95f9950 2024-09-10T21:05:25.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681a9a90 con 0x559ee95ed9c0 2024-09-10T21:05:25.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.916+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681a8620 con 0x559ee95ed0a0 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6d699640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95e5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:34016/0 (socket says 172.21.15.33:34016) 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6d699640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95e5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59800/0 (socket says 172.21.15.33:59800) 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 -- 172.21.15.33:0/208580509 learned_addr learned my addr 172.21.15.33:0/208580509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 -- 172.21.15.33:0/208580509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 msgr2=0x559ee95e5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:25.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95e5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 -- 172.21.15.33:0/208580509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 msgr2=0x559ee95f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 -- 172.21.15.33:0/208580509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ee95eb950 con 0x559ee95ed9c0 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6de9a640 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.917+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f4f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a580058c0 tx=0x7f2a58005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a8a412ecb89538 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.918+0000 7f2a52ffd640 1 -- 172.21.15.33:0/208580509 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58003d20 con 0x559ee95ed9c0 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.918+0000 7f2a52ffd640 1 -- 172.21.15.33:0/208580509 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a580108e0 con 0x559ee95ed9c0 2024-09-10T21:05:25.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.918+0000 7f2a52ffd640 1 -- 172.21.15.33:0/208580509 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5801e760 con 0x559ee95ed9c0 2024-09-10T21:05:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 -- 172.21.15.33:0/208580509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 msgr2=0x559ee95f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f4f10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2a580058c0 tx=0x7f2a58005850 comp rx=0 tx=0).stop 2024-09-10T21:05:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 -- 172.21.15.33:0/208580509 shutdown_connections 2024-09-10T21:05:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95f9950 0x559ee95f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95ed9c0 0x559ee95f4f10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/208580509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed0a0 0x559ee95e5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 -- 172.21.15.33:0/208580509 >> 172.21.15.33:0/208580509 conn(0x559ee95eaed0 msgr2=0x559ee95ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 -- 172.21.15.33:0/208580509 shutdown_connections 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.919+0000 7f2a71cd2740 1 -- 172.21.15.33:0/208580509 wait complete. 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.920+0000 7f2a71cd2740 1 -- start start 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.920+0000 7f2a71cd2740 1 Processor -- start 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.920+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee96128a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.920+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee9612de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.920+0000 7f2a71cd2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.920+0000 7f2a6d699640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee96128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6d699640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee96128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39262/0 (socket says 172.21.15.33:39262) 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6d699640 1 -- 172.21.15.33:0/2470670943 learned_addr learned my addr 172.21.15.33:0/2470670943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a71cd2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681a93c0 con 0x559ee95ed0a0 2024-09-10T21:05:25.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2a681a8cf0 con 0x559ee95f9950 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6819c650 con 0x559ee95ed9c0 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a71cd2740 10 client.0 _ll_register_callbacks cb 0x7f2a600093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee9612de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6d699640 1 -- 172.21.15.33:0/2470670943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 msgr2=0x559ee9612de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6d699640 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee9612de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6d699640 1 -- 172.21.15.33:0/2470670943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 msgr2=0x559ee95f28b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6d699640 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95f28b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6d699640 1 -- 172.21.15.33:0/2470670943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ee9613630 con 0x559ee95ed0a0 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.921+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee9612de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.922+0000 7f2a6d699640 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee96128a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2a64047590 tx=0x7f2a64047460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31e9eb3d2126ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.922+0000 7f2a50ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.922+0000 7f2a50ff9640 1 -- 172.21.15.33:0/2470670943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6404b040 con 0x559ee95ed0a0 2024-09-10T21:05:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.922+0000 7f2a50ff9640 1 -- 172.21.15.33:0/2470670943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2a64003bc0 con 0x559ee95ed0a0 2024-09-10T21:05:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.922+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a580108e0 con 0x559ee95ed0a0 2024-09-10T21:05:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.922+0000 7f2a50ff9640 1 -- 172.21.15.33:0/2470670943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a64003dc0 con 0x559ee95ed0a0 2024-09-10T21:05:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 10 client.15555 fetch_fsmap learned FSMap version 28 2024-09-10T21:05:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559ee95f3ef0 con 0x559ee95ed0a0 2024-09-10T21:05:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a50ff9640 1 -- 172.21.15.33:0/2470670943 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a580108e0 con 0x559ee95ed0a0 2024-09-10T21:05:25.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a50ff9640 1 -- 172.21.15.33:0/2470670943 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a64064450 con 0x559ee95ed0a0 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 10 client.15555 fetch_fsmap finished waiting for FSMap version 28 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 20 client.15555 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee9688780 0x559ee968ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 4 client.15555 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.923+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559ee9561850 con 0x559ee9688780 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.924+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee9688780 0x559ee968ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.924+0000 7f2a6ce98640 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee9688780 0x559ee968ab70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef63a88d321579a server_cookie=92ef7da30687e16 in_seq=0 out_seq=0 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.924+0000 7f2a50ff9640 10 client.15555 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.924+0000 7f2a50ff9640 1 -- 172.21.15.33:0/2470670943 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x559ee9561850 con 0x559ee9688780 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.924+0000 7f2a50ff9640 10 client.15555 handle_command_reply: tid=0 2024-09-10T21:05:25.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.924+0000 7f2a71cd2740 1 client.15555 shutdown 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 msgr2=0x559ee96128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee96128a0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2a64047590 tx=0x7f2a64047460 comp rx=0 tx=0).stop 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 shutdown_connections 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559ee9688780 0x559ee968ab70 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ee95f9950 0x559ee95f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ee95ed9c0 0x559ee9612de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a50ff9640 0 client.15555 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 --2- 172.21.15.33:0/2470670943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ee95ed0a0 0x559ee96128a0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 >> 172.21.15.33:0/2470670943 conn(0x559ee95eaed0 msgr2=0x559ee95f4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.925+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 shutdown_connections 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.926+0000 7f2a71cd2740 1 -- 172.21.15.33:0/2470670943 wait complete. 2024-09-10T21:05:25.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.926+0000 7f2a71cd2740 20 client.15555 trim_cache size 0 max 16384 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:05:25.935 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:05:25.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:05:25.937 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:05:25.937 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.926+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a540b52e0 msgr2=0x7f2a540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.926+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a540b52e0 0x7f2a540b77a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2a6402a930 tx=0x7f2a64031400 comp rx=0 tx=0).stop 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.926+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a68108020 msgr2=0x7f2a681a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.926+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a68108020 0x7f2a681a0e00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000eb10 tx=0x7f2a6000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 shutdown_connections 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2a540b52e0 0x7f2a540b77a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a68108020 0x7f2a681a0e00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a681076a0 0x7f2a6819ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 --2- 172.21.15.33:0/2791496472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a68101c90 0x7f2a6819b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 >> 172.21.15.33:0/2791496472 conn(0x7f2a680fd700 msgr2=0x7f2a6806ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:25.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 shutdown_connections 2024-09-10T21:05:25.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:25.927+0000 7f2a6f8fd640 1 -- 172.21.15.33:0/2791496472 wait complete. 2024-09-10T21:05:26.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88705 18.42 MB/sec warmup 27 sec latency 88.800 ms 2024-09-10T21:05:26.793 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:05:26.794 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:05:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:27.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:27 smithi033 ceph-mon[30860]: pgmap v390: 97 pgs: 97 active+clean; 35 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-09-10T21:05:27.355 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:27 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:05:27.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:27 smithi104 ceph-mon[35325]: pgmap v390: 97 pgs: 97 active+clean; 35 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-09-10T21:05:27.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:27 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3698881558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 msgr2=0x7f1ccc108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3698881558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc108c10 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0009900 tx=0x7f1cc002a410 comp rx=0 tx=0).stop 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3698881558 shutdown_connections 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3698881558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ccc109150 0x7f1ccc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3698881558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc108c10 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3698881558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc102dc0 0x7f1ccc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3698881558 >> 172.21.15.33:0/3698881558 conn(0x7f1ccc0fe830 msgr2=0x7f1ccc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:27.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3698881558 shutdown_connections 2024-09-10T21:05:27.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.567+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3698881558 wait complete. 2024-09-10T21:05:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 Processor -- start 2024-09-10T21:05:27.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 -- start start 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc102dc0 0x7f1ccc19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ccc109150 0x7f1ccc1a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc112e20 con 0x7f1ccc1087d0 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc112ca0 con 0x7f1ccc102dc0 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd28b8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ccc112fa0 con 0x7f1ccc109150 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc19ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39294/0 (socket says 172.21.15.33:39294) 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1ccb7fe640 1 -- 172.21.15.33:0/3535197417 learned_addr learned my addr 172.21.15.33:0/3535197417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1cd0e55640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ccc109150 0x7f1ccc1a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.568+0000 7f1ccbfff640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc102dc0 0x7f1ccc19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1ccb7fe640 1 -- 172.21.15.33:0/3535197417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ccc109150 msgr2=0x7f1ccc1a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:27.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1ccb7fe640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ccc109150 0x7f1ccc1a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1ccb7fe640 1 -- 172.21.15.33:0/3535197417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc102dc0 msgr2=0x7f1ccc19b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1ccb7fe640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc102dc0 0x7f1ccc19b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1ccb7fe640 1 -- 172.21.15.33:0/3535197417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ccc1a1590 con 0x7f1ccc1087d0 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1ccb7fe640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc19ba20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0005ab0 tx=0x7f1cc0003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1cc97fa640 1 -- 172.21.15.33:0/3535197417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc003f070 con 0x7f1ccc1087d0 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ccc1a1820 con 0x7f1ccc1087d0 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1cc97fa640 1 -- 172.21.15.33:0/3535197417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc0031d60 con 0x7f1ccc1087d0 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1cc97fa640 1 -- 172.21.15.33:0/3535197417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0035030 con 0x7f1ccc1087d0 2024-09-10T21:05:27.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.569+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ccc1a7f40 con 0x7f1ccc1087d0 2024-09-10T21:05:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.570+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ccc1031a0 con 0x7f1ccc1087d0 2024-09-10T21:05:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.572+0000 7f1cc97fa640 1 -- 172.21.15.33:0/3535197417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1cc003b580 con 0x7f1ccc1087d0 2024-09-10T21:05:27.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.572+0000 7f1cc97fa640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1cb40b52e0 0x7f1cb40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.572+0000 7f1cc97fa640 1 -- 172.21.15.33:0/3535197417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1cc0106c70 con 0x7f1ccc1087d0 2024-09-10T21:05:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.572+0000 7f1ccbfff640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1cb40b52e0 0x7f1cb40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:27.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.573+0000 7f1ccbfff640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1cb40b52e0 0x7f1cb40b77a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005d00 tx=0x7f1cbc00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:27.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.575+0000 7f1cc97fa640 1 -- 172.21.15.33:0/3535197417 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1cc00cd4e0 con 0x7f1ccc1087d0 2024-09-10T21:05:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:27 smithi174 ceph-mon[35635]: pgmap v390: 97 pgs: 97 active+clean; 35 MiB data, 483 MiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-09-10T21:05:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:27 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:05:27.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88795 17.77 MB/sec warmup 28 sec latency 573.575 ms 2024-09-10T21:05:27.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.796+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ccc02e5c0 con 0x7f1ccc1087d0 2024-09-10T21:05:27.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.797+0000 7f1cc97fa640 1 -- 172.21.15.33:0/3535197417 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11974 (secure 0 0 0) 0x7f1cc00d1f10 con 0x7f1ccc1087d0 2024-09-10T21:05:27.801 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:05:27.801 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"btime":"2024-09-10T21:05:26:245759+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:05:25.981807+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:05:27.802 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-09-10T21:05:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1cb40b52e0 msgr2=0x7f1cb40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1cb40b52e0 0x7f1cb40b77a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc005d00 tx=0x7f1cbc00b410 comp rx=0 tx=0).stop 2024-09-10T21:05:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 msgr2=0x7f1ccc19ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc19ba20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0005ab0 tx=0x7f1cc0003cd0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 shutdown_connections 2024-09-10T21:05:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1cb40b52e0 0x7f1cb40b77a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ccc109150 0x7f1ccc1a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ccc1087d0 0x7f1ccc19ba20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 --2- 172.21.15.33:0/3535197417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1ccc102dc0 0x7f1ccc19b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 >> 172.21.15.33:0/3535197417 conn(0x7f1ccc0fe830 msgr2=0x7f1ccc105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 shutdown_connections 2024-09-10T21:05:27.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:27.799+0000 7f1cd28b8640 1 -- 172.21.15.33:0/3535197417 wait complete. 2024-09-10T21:05:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:28 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3535197417' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:05:28.513 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:05:28.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:28 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3535197417' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:05:28.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:28 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3535197417' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:05:28.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88795 17.16 MB/sec warmup 29 sec latency 1573.622 ms 2024-09-10T21:05:28.775 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:29.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:29 smithi104 ceph-mon[35325]: pgmap v391: 97 pgs: 97 active+clean; 39 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2024-09-10T21:05:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:29 smithi174 ceph-mon[35635]: pgmap v391: 97 pgs: 97 active+clean; 39 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2024-09-10T21:05:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:29 smithi033 ceph-mon[30860]: pgmap v391: 97 pgs: 97 active+clean; 39 MiB data, 503 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.1 MiB/s wr, 280 op/s 2024-09-10T21:05:29.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88795 16.59 MB/sec warmup 30 sec latency 2573.733 ms 2024-09-10T21:05:30.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88795 16.05 MB/sec warmup 31 sec latency 3573.799 ms 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 -- 172.21.15.33:0/1928951722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0095220 msgr2=0x7f60f00a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 --2- 172.21.15.33:0/1928951722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0095220 0x7f60f00a1260 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f60f8068c00 tx=0x7f60f8097c90 comp rx=0 tx=0).stop 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 -- 172.21.15.33:0/1928951722 shutdown_connections 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 --2- 172.21.15.33:0/1928951722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0095220 0x7f60f00a1260 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 --2- 172.21.15.33:0/1928951722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f00948a0 0x7f60f0094ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 --2- 172.21.15.33:0/1928951722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60f0097270 0x7f60f0097650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 -- 172.21.15.33:0/1928951722 >> 172.21.15.33:0/1928951722 conn(0x7f60f000a150 msgr2=0x7f60f000c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:30.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 -- 172.21.15.33:0/1928951722 shutdown_connections 2024-09-10T21:05:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.958+0000 7f60fdc51640 1 -- 172.21.15.33:0/1928951722 wait complete. 2024-09-10T21:05:30.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.959+0000 7f60fdc51640 1 Processor -- start 2024-09-10T21:05:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.959+0000 7f60fdc51640 1 -- start start 2024-09-10T21:05:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60fdc51640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f00948a0 0x7f60f009fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60fdc51640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60f0095220 0x7f60f00a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60fdc51640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0097270 0x7f60f00a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60fdc51640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60f00a30b0 con 0x7f60f00948a0 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60fdc51640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60f00a2f30 con 0x7f60f0095220 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60fdc51640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60f00a3230 con 0x7f60f0097270 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60f6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60f0095220 0x7f60f00a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.960+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0097270 0x7f60f00a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.961+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0097270 0x7f60f00a07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:44222/0 (socket says 172.21.15.33:44222) 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.961+0000 7f60f7fff640 1 -- 172.21.15.33:0/238879455 learned_addr learned my addr 172.21.15.33:0/238879455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.961+0000 7f60f77fe640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f00948a0 0x7f60f009fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.962+0000 7f60f7fff640 1 -- 172.21.15.33:0/238879455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60f0095220 msgr2=0x7f60f00a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.962+0000 7f60f7fff640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60f0095220 0x7f60f00a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.962+0000 7f60f7fff640 1 -- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f00948a0 msgr2=0x7f60f009fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:30.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.962+0000 7f60f7fff640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f00948a0 0x7f60f009fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.962+0000 7f60f7fff640 1 -- 172.21.15.33:0/238879455 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f0146e70 con 0x7f60f0097270 2024-09-10T21:05:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.962+0000 7f60f7fff640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0097270 0x7f60f00a07d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60f80669a0 tx=0x7f60f8078b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.963+0000 7f60f4ff9640 1 -- 172.21.15.33:0/238879455 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80ae070 con 0x7f60f0097270 2024-09-10T21:05:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.963+0000 7f60f4ff9640 1 -- 172.21.15.33:0/238879455 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60f8098a40 con 0x7f60f0097270 2024-09-10T21:05:30.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.963+0000 7f60f4ff9640 1 -- 172.21.15.33:0/238879455 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60f80a99d0 con 0x7f60f0097270 2024-09-10T21:05:30.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.963+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f0147100 con 0x7f60f0097270 2024-09-10T21:05:30.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.963+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f01475c0 con 0x7f60f0097270 2024-09-10T21:05:30.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.967+0000 7f6100026740 1 Processor -- start 2024-09-10T21:05:30.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.968+0000 7f6100026740 1 -- start start 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.969+0000 7f6100026740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.969+0000 7f6100026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.969+0000 7f6100026740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 0x55b25f8ea360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.969+0000 7f6100026740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b25f851ea0 con 0x55b25f8db6c0 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.969+0000 7f6100026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b25f8412d0 con 0x55b25f8dbfe0 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.969+0000 7f6100026740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b25f8450e0 con 0x55b25f8e7f70 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.969+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.970+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43238/0 (socket says 172.21.15.33:43238) 2024-09-10T21:05:30.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.970+0000 7f60f77fe640 1 -- 172.21.15.33:0/2310711138 learned_addr learned my addr 172.21.15.33:0/2310711138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.971+0000 7f60f77fe640 1 -- 172.21.15.33:0/2310711138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 msgr2=0x55b25f8ea360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:05:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.971+0000 7f60f77fe640 1 --2- 172.21.15.33:0/2310711138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 0x55b25f8ea360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.971+0000 7f60f77fe640 1 -- 172.21.15.33:0/2310711138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 msgr2=0x55b25f8dbaa0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:05:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.971+0000 7f60f77fe640 1 --2- 172.21.15.33:0/2310711138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8dbaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.971+0000 7f60f77fe640 1 -- 172.21.15.33:0/2310711138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25f59f650 con 0x55b25f8dbfe0 2024-09-10T21:05:30.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.972+0000 7f60f77fe640 1 --2- 172.21.15.33:0/2310711138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e7a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009cf0 tx=0x7f60ec011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=898bf9db6353e495 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.972+0000 7f60e1ffb640 1 -- 172.21.15.33:0/2310711138 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec004230 con 0x55b25f8dbfe0 2024-09-10T21:05:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.973+0000 7f60e1ffb640 1 -- 172.21.15.33:0/2310711138 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec0043d0 con 0x55b25f8dbfe0 2024-09-10T21:05:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.973+0000 7f60e1ffb640 1 -- 172.21.15.33:0/2310711138 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec019880 con 0x55b25f8dbfe0 2024-09-10T21:05:30.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.973+0000 7f60f4ff9640 1 -- 172.21.15.33:0/238879455 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f60f8098bf0 con 0x7f60f0097270 2024-09-10T21:05:30.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.974+0000 7f6100026740 1 -- 172.21.15.33:0/2310711138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 msgr2=0x55b25f8e7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:30.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.974+0000 7f6100026740 1 --2- 172.21.15.33:0/2310711138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e7a30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009cf0 tx=0x7f60ec011810 comp rx=0 tx=0).stop 2024-09-10T21:05:30.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.975+0000 7f60f4ff9640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f60e80b5480 0x7f60e80b7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.975+0000 7f60f4ff9640 1 -- 172.21.15.33:0/238879455 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f60f8051730 con 0x7f60f0097270 2024-09-10T21:05:30.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.975+0000 7f6100026740 1 -- 172.21.15.33:0/2310711138 shutdown_connections 2024-09-10T21:05:30.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.975+0000 7f6100026740 1 --2- 172.21.15.33:0/2310711138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 0x55b25f8ea360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.975+0000 7f6100026740 1 --2- 172.21.15.33:0/2310711138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e7a30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.975+0000 7f6100026740 1 --2- 172.21.15.33:0/2310711138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.975+0000 7f6100026740 1 -- 172.21.15.33:0/2310711138 >> 172.21.15.33:0/2310711138 conn(0x55b25f8d94f0 msgr2=0x55b25f8da690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:30.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.979+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f60e80b5480 0x7f60e80b7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.979+0000 7f6100026740 1 -- 172.21.15.33:0/2310711138 shutdown_connections 2024-09-10T21:05:30.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.980+0000 7f6100026740 1 -- 172.21.15.33:0/2310711138 wait complete. 2024-09-10T21:05:30.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.980+0000 7f6100026740 1 -- start start 2024-09-10T21:05:30.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f6100026740 1 Processor -- start 2024-09-10T21:05:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f60e80b5480 0x7f60e80b7940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f60f0006410 tx=0x7f60e4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f6100026740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8e77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f6100026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f6100026740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 0x55b25f8e5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f6100026740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b25f6a2700 con 0x55b25f8db6c0 2024-09-10T21:05:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f6100026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b25f7dc160 con 0x55b25f8dbfe0 2024-09-10T21:05:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.981+0000 7f6100026740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b25f8eac40 con 0x55b25f8e7f70 2024-09-10T21:05:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.982+0000 7f6100026740 10 client.0 _ll_register_callbacks cb 0x55b25f7f59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.982+0000 7f60f6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.982+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8e77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.982+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8e77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45398/0 (socket says 172.21.15.33:45398) 2024-09-10T21:05:30.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.982+0000 7f60f77fe640 1 -- 172.21.15.33:0/750904980 learned_addr learned my addr 172.21.15.33:0/750904980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.982+0000 7f60f7fff640 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 0x55b25f8e5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:30.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.983+0000 7f60f77fe640 1 -- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 msgr2=0x55b25f8e5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.983+0000 7f60f77fe640 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 0x55b25f8e5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.983+0000 7f60f77fe640 1 -- 172.21.15.33:0/750904980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 msgr2=0x55b25f8e4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.983+0000 7f60f77fe640 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.983+0000 7f60f77fe640 1 -- 172.21.15.33:0/750904980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25f809c90 con 0x55b25f8db6c0 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.983+0000 7f60f77fe640 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8e77b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f60ec005d00 tx=0x7f60ec003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b18899323bb85 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.984+0000 7f60d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.984+0000 7f60d6ffd640 1 -- 172.21.15.33:0/750904980 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec027070 con 0x55b25f8db6c0 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.985+0000 7f60d6ffd640 1 -- 172.21.15.33:0/750904980 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f60ec005040 con 0x55b25f8db6c0 2024-09-10T21:05:30.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.985+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60ec0043d0 con 0x55b25f8db6c0 2024-09-10T21:05:30.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.986+0000 7f60d6ffd640 1 -- 172.21.15.33:0/750904980 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60ec033050 con 0x55b25f8db6c0 2024-09-10T21:05:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.988+0000 7f6100026740 10 client.15567 fetch_fsmap learned FSMap version 29 2024-09-10T21:05:30.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.988+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b25f907570 con 0x55b25f8db6c0 2024-09-10T21:05:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.991+0000 7f60d6ffd640 1 -- 172.21.15.33:0/750904980 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec0043d0 con 0x55b25f8db6c0 2024-09-10T21:05:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.991+0000 7f60d6ffd640 1 -- 172.21.15.33:0/750904980 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f60ec004be0 con 0x55b25f8db6c0 2024-09-10T21:05:30.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.991+0000 7f6100026740 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T21:05:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.991+0000 7f6100026740 10 client.15567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:05:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.991+0000 7f6100026740 20 client.15567 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.992+0000 7f6100026740 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f908e90 0x55b25f90b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:30.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.992+0000 7f6100026740 4 client.15567 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:30.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.992+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b25f853df0 con 0x55b25f908e90 2024-09-10T21:05:30.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.995+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f908e90 0x55b25f90b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:30.999+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f908e90 0x55b25f90b310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7fe96ec6ed2ba5f9 server_cookie=e260bc9af07d4e80 in_seq=0 out_seq=0 2024-09-10T21:05:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.000+0000 7f60d6ffd640 10 client.15567 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:05:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.000+0000 7f60d6ffd640 1 -- 172.21.15.33:0/750904980 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b25f853df0 con 0x55b25f908e90 2024-09-10T21:05:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.000+0000 7f60d6ffd640 10 client.15567 handle_command_reply: tid=0 2024-09-10T21:05:31.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.000+0000 7f6100026740 1 client.15567 shutdown 2024-09-10T21:05:31.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.001+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 msgr2=0x55b25f8e77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:31.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.001+0000 7f6100026740 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8e77b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f60ec005d00 tx=0x7f60ec003820 comp rx=0 tx=0).stop 2024-09-10T21:05:31.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.003+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 shutdown_connections 2024-09-10T21:05:31.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.003+0000 7f6100026740 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f908e90 0x55b25f90b310 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.003+0000 7f6100026740 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8e7f70 0x55b25f8e5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.003+0000 7f6100026740 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.003+0000 7f6100026740 1 --2- 172.21.15.33:0/750904980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8db6c0 0x55b25f8e77b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.003+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 >> 172.21.15.33:0/750904980 conn(0x55b25f8d94f0 msgr2=0x55b25f8df060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:31.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.003+0000 7f60d6ffd640 0 client.15567 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:05:31.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.004+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 shutdown_connections 2024-09-10T21:05:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.006+0000 7f6100026740 1 -- 172.21.15.33:0/750904980 wait complete. 2024-09-10T21:05:31.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.006+0000 7f6100026740 20 client.15567 trim_cache size 0 max 16384 2024-09-10T21:05:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.031+0000 7f6100026740 1 Processor -- start 2024-09-10T21:05:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f6100026740 1 -- start start 2024-09-10T21:05:31.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f6100026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8db6c0 0x55b25f8d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f6100026740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f6100026740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8e7f70 0x55b25f8e39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f6100026740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60f014fd50 con 0x55b25f8e7f70 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f6100026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60f01518c0 con 0x55b25f8db6c0 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f6100026740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0150420 con 0x55b25f8dbfe0 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8db6c0 0x55b25f8d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8db6c0 0x55b25f8d4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43256/0 (socket says 172.21.15.33:43256) 2024-09-10T21:05:31.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 -- 172.21.15.33:0/1181775468 learned_addr learned my addr 172.21.15.33:0/1181775468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f7fff640 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8e7f70 0x55b25f8e39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 -- 172.21.15.33:0/1181775468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8dbfe0 msgr2=0x55b25f8e3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 -- 172.21.15.33:0/1181775468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8e7f70 msgr2=0x55b25f8e39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8e7f70 0x55b25f8e39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f77fe640 1 -- 172.21.15.33:0/1181775468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b25f8df970 con 0x55b25f8db6c0 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.032+0000 7f60f7fff640 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8e7f70 0x55b25f8e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.033+0000 7f60f77fe640 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8db6c0 0x55b25f8d4570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0199a0 tx=0x7f60ec0199d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=877fe84d1edae3ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:31.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.033+0000 7f60e17fa640 1 -- 172.21.15.33:0/1181775468 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec030020 con 0x55b25f8db6c0 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.033+0000 7f60e17fa640 1 -- 172.21.15.33:0/1181775468 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec009910 con 0x55b25f8db6c0 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.033+0000 7f60e17fa640 1 -- 172.21.15.33:0/1181775468 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec027070 con 0x55b25f8db6c0 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 -- 172.21.15.33:0/1181775468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8db6c0 msgr2=0x55b25f8d4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8db6c0 0x55b25f8d4570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0199a0 tx=0x7f60ec0199d0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 -- 172.21.15.33:0/1181775468 shutdown_connections 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8e7f70 0x55b25f8e39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8dbfe0 0x55b25f8e3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 --2- 172.21.15.33:0/1181775468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8db6c0 0x55b25f8d4570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 -- 172.21.15.33:0/1181775468 >> 172.21.15.33:0/1181775468 conn(0x55b25f8d94f0 msgr2=0x55b25f8e49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 -- 172.21.15.33:0/1181775468 shutdown_connections 2024-09-10T21:05:31.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 -- 172.21.15.33:0/1181775468 wait complete. 2024-09-10T21:05:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.034+0000 7f6100026740 1 -- start start 2024-09-10T21:05:31.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f6100026740 1 Processor -- start 2024-09-10T21:05:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f6100026740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8db6c0 0x55b25f8eb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f6100026740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8dbfe0 0x55b25f8ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f6100026740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8e7f70 0x55b25f973dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f6100026740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f60f01511c0 con 0x55b25f8dbfe0 2024-09-10T21:05:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f6100026740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f60f0150af0 con 0x55b25f8e7f70 2024-09-10T21:05:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8e7f70 0x55b25f973dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f60f77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8db6c0 0x55b25f8eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f60f7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8e7f70 0x55b25f973dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43266/0 (socket says 172.21.15.33:43266) 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f60f7fff640 1 -- 172.21.15.33:0/3291938370 learned_addr learned my addr 172.21.15.33:0/3291938370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8dbfe0 0x55b25f8ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.035+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f60f014dd10 con 0x55b25f8db6c0 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f6100026740 10 client.0 _ll_register_callbacks cb 0x7f60ec009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60f77fe640 1 -- 172.21.15.33:0/3291938370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8e7f70 msgr2=0x55b25f973dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60f77fe640 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8e7f70 0x55b25f973dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60f77fe640 1 -- 172.21.15.33:0/3291938370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8dbfe0 msgr2=0x55b25f8ebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60f77fe640 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8dbfe0 0x55b25f8ebc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60f77fe640 1 -- 172.21.15.33:0/3291938370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b25f9744c0 con 0x55b25f8db6c0 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60f7fff640 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8e7f70 0x55b25f973dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60f77fe640 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8db6c0 0x55b25f8eb710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000c00 tx=0x7f60ec0198e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a55048495e34f6dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:31.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.036+0000 7f60d6ffd640 1 -- 172.21.15.33:0/3291938370 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec044060 con 0x55b25f8db6c0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60ec009910 con 0x55b25f8db6c0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f60d6ffd640 1 -- 172.21.15.33:0/3291938370 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f60ec0051b0 con 0x55b25f8db6c0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f60d6ffd640 1 -- 172.21.15.33:0/3291938370 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60ec027040 con 0x55b25f8db6c0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 10 client.25174 fetch_fsmap learned FSMap version 29 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b25f9757a0 con 0x55b25f8db6c0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f60d6ffd640 1 -- 172.21.15.33:0/3291938370 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec009910 con 0x55b25f8db6c0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f60d6ffd640 1 -- 172.21.15.33:0/3291938370 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8366+0+0 (secure 0 0 0) 0x7f60ec049070 con 0x55b25f8db6c0 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 10 client.25174 fetch_fsmap finished waiting for FSMap version 29 2024-09-10T21:05:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 10 client.25174 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 20 client.25174 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f975db0 0x55b25f9781a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 4 client.25174 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b25f84f8a0 con 0x55b25f975db0 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.037+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f975db0 0x55b25f9781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f60d6ffd640 10 client.25174 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f60f6ffd640 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f975db0 0x55b25f9781a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75710e374cf64341 server_cookie=6428c915d2b1cfb2 in_seq=0 out_seq=0 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f60d6ffd640 1 -- 172.21.15.33:0/3291938370 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b25f84f8a0 con 0x55b25f975db0 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f60d6ffd640 10 client.25174 handle_command_reply: tid=0 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 client.25174 shutdown 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8db6c0 msgr2=0x55b25f8eb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8db6c0 0x55b25f8eb710 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000c00 tx=0x7f60ec0198e0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 shutdown_connections 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b25f975db0 0x55b25f9781a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b25f8e7f70 0x55b25f973dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b25f8dbfe0 0x55b25f8ebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 --2- 172.21.15.33:0/3291938370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b25f8db6c0 0x55b25f8eb710 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 >> 172.21.15.33:0/3291938370 conn(0x55b25f8d94f0 msgr2=0x55b25f8d4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f60d6ffd640 0 client.25174 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 shutdown_connections 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.038+0000 7f6100026740 1 -- 172.21.15.33:0/3291938370 wait complete. 2024-09-10T21:05:31.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f6100026740 20 client.25174 trim_cache size 0 max 16384 2024-09-10T21:05:31.051 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f60e80b5480 msgr2=0x7f60e80b7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f60e80b5480 0x7f60e80b7940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f60f0006410 tx=0x7f60e4008040 comp rx=0 tx=0).stop 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0097270 msgr2=0x7f60f00a07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0097270 0x7f60f00a07d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f60f80669a0 tx=0x7f60f8078b80 comp rx=0 tx=0).stop 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 shutdown_connections 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f60e80b5480 0x7f60e80b7940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f60f0097270 0x7f60f00a07d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f60f0095220 0x7f60f00a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 --2- 172.21.15.33:0/238879455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60f00948a0 0x7f60f009fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 >> 172.21.15.33:0/238879455 conn(0x7f60f000a150 msgr2=0x7f60f00a5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 shutdown_connections 2024-09-10T21:05:31.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:31.039+0000 7f60fdc51640 1 -- 172.21.15.33:0/238879455 wait complete. 2024-09-10T21:05:31.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:31 smithi104 ceph-mon[35325]: pgmap v392: 97 pgs: 97 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-10T21:05:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:31 smithi174 ceph-mon[35635]: pgmap v392: 97 pgs: 97 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-10T21:05:31.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:31 smithi033 ceph-mon[30860]: pgmap v392: 97 pgs: 97 active+clean; 40 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-09-10T21:05:31.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88840 15.55 MB/sec warmup 32 sec latency 4442.481 ms 2024-09-10T21:05:32.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:05:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:32.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:32 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:05:32.622 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:32 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:05:32.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:32 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:05:32.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88934 15.11 MB/sec warmup 33 sec latency 593.924 ms 2024-09-10T21:05:33.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 -- 172.21.15.33:0/917240324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4101c90 msgr2=0x7fbcf4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/917240324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4101c90 0x7fbcf4102070 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009ca0 tx=0x7fbce402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 -- 172.21.15.33:0/917240324 shutdown_connections 2024-09-10T21:05:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/917240324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf4108020 0x7fbcf410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/917240324 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf41076a0 0x7fbcf4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/917240324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4101c90 0x7fbcf4102070 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 -- 172.21.15.33:0/917240324 >> 172.21.15.33:0/917240324 conn(0x7fbcf40fd700 msgr2=0x7fbcf40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:33.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 -- 172.21.15.33:0/917240324 shutdown_connections 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.847+0000 7fbcf8a50640 1 -- 172.21.15.33:0/917240324 wait complete. 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 Processor -- start 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 -- start start 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf4101c90 0x7fbcf4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf41076a0 0x7fbcf4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4108020 0x7fbcf419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4112e30 con 0x7fbcf4108020 2024-09-10T21:05:33.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4112cb0 con 0x7fbcf41076a0 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf8a50640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf4112fb0 con 0x7fbcf4101c90 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4108020 0x7fbcf419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4108020 0x7fbcf419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45428/0 (socket says 172.21.15.33:45428) 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf2d9d640 1 -- 172.21.15.33:0/1649847391 learned_addr learned my addr 172.21.15.33:0/1649847391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf259c640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf4101c90 0x7fbcf4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.848+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf41076a0 0x7fbcf4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf2d9d640 1 -- 172.21.15.33:0/1649847391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf4101c90 msgr2=0x7fbcf4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf2d9d640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf4101c90 0x7fbcf4199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf2d9d640 1 -- 172.21.15.33:0/1649847391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf41076a0 msgr2=0x7fbcf4199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf2d9d640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf41076a0 0x7fbcf4199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf2d9d640 1 -- 172.21.15.33:0/1649847391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf419f3b0 con 0x7fbcf4108020 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf41076a0 0x7fbcf4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf2d9d640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4108020 0x7fbcf419ebe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00d950 tx=0x7fbcec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbceb7fe640 1 -- 172.21.15.33:0/1649847391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec017030 con 0x7fbcf4108020 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf419f6a0 con 0x7fbcf4108020 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbceb7fe640 1 -- 172.21.15.33:0/1649847391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcec004020 con 0x7fbcf4108020 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbceb7fe640 1 -- 172.21.15.33:0/1649847391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec003b50 con 0x7fbcf4108020 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.849+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf41a5d60 con 0x7fbcf4108020 2024-09-10T21:05:33.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.851+0000 7fbcfae25740 1 Processor -- start 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbceb7fe640 1 -- 172.21.15.33:0/1649847391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbcec003cf0 con 0x7fbcf4108020 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcfae25740 1 -- start start 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e78f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78fdf0 0x55e51e79b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcfae25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e51e705ea0 con 0x55e51e79bd80 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcfae25740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e51e6f52d0 con 0x55e51e78fdf0 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcfae25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e51e6f90e0 con 0x55e51e78f4d0 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcf1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcf1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45442/0 (socket says 172.21.15.33:45442) 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/2810249915 learned_addr learned my addr 172.21.15.33:0/2810249915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:33.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e78f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.852+0000 7fbcf259c640 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78fdf0 0x55e51e79b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbceb7fe640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbce00b52e0 0x7fbce00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/2810249915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78fdf0 msgr2=0x55e51e79b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78fdf0 0x55e51e79b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/2810249915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 msgr2=0x55e51e79e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/2810249915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e51e453650 con 0x55e51e78f4d0 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbcf2d9d640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbce00b52e0 0x7fbce00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbceb7fe640 1 -- 172.21.15.33:0/1649847391 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbcec0e56b0 con 0x7fbcf4108020 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e78f8b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc014a60 tx=0x7fbcdc005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4e1a1843f0e601 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:33.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbce8ff9640 1 -- 172.21.15.33:0/2810249915 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc004030 con 0x55e51e78f4d0 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbce8ff9640 1 -- 172.21.15.33:0/2810249915 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc011d50 con 0x55e51e78f4d0 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.853+0000 7fbce8ff9640 1 -- 172.21.15.33:0/2810249915 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc0038c0 con 0x55e51e78f4d0 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 -- 172.21.15.33:0/2810249915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 msgr2=0x55e51e78f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcf2d9d640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbce00b52e0 0x7fbce00b77a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0a8440 tx=0x7fbcec093040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e78f8b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc014a60 tx=0x7fbcdc005850 comp rx=0 tx=0).stop 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 -- 172.21.15.33:0/2810249915 shutdown_connections 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78fdf0 0x55e51e79b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 --2- 172.21.15.33:0/2810249915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e78f8b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 -- 172.21.15.33:0/2810249915 >> 172.21.15.33:0/2810249915 conn(0x55e51e78d300 msgr2=0x55e51e78e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 -- 172.21.15.33:0/2810249915 shutdown_connections 2024-09-10T21:05:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 -- 172.21.15.33:0/2810249915 wait complete. 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 -- start start 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.854+0000 7fbcfae25740 1 Processor -- start 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e79b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78fdf0 0x55e51e798ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e799110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcfae25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e51e556700 con 0x55e51e78fdf0 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcfae25740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e51e690160 con 0x55e51e79bd80 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcfae25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e51e79ea00 con 0x55e51e78f4d0 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcfae25740 10 client.0 _ll_register_callbacks cb 0x55e51e6a99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78fdf0 0x55e51e798ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78fdf0 0x55e51e798ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45458/0 (socket says 172.21.15.33:45458) 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 -- 172.21.15.33:0/1700578915 learned_addr learned my addr 172.21.15.33:0/1700578915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e799110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e79b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 -- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 msgr2=0x55e51e79b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e79b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 -- 172.21.15.33:0/1700578915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 msgr2=0x55e51e799110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e799110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 -- 172.21.15.33:0/1700578915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e51e6bdc90 con 0x55e51e78fdf0 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e799110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcf259c640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78fdf0 0x55e51e798ae0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbce402ade0 tx=0x7fbce4031680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7160d8c755cc0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.855+0000 7fbcd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/1700578915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4031900 con 0x55e51e78fdf0 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/1700578915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbce4003970 con 0x55e51e78fdf0 2024-09-10T21:05:33.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcdc011d50 con 0x55e51e78fdf0 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/1700578915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbce4003b70 con 0x55e51e78fdf0 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 10 client.15585 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e51e7aa360 con 0x55e51e78fdf0 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/1700578915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc011d50 con 0x55e51e78fdf0 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/1700578915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fbce4004610 con 0x55e51e78fdf0 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 10 client.15585 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 10 client.15585 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 20 client.15585 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e7ab830 0x55e51e7adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 4 client.15585 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e51e707df0 con 0x55e51e7ab830 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.856+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e7ab830 0x55e51e7adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.857+0000 7fbcd9ffb640 10 client.15585 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.857+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e7ab830 0x55e51e7adc20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=71b3edff70ce473b server_cookie=9375e29ff31013f in_seq=0 out_seq=0 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/1700578915 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e51e707df0 con 0x55e51e7ab830 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcd9ffb640 10 client.15585 handle_command_reply: tid=0 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 client.15585 shutdown 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78fdf0 msgr2=0x55e51e798ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78fdf0 0x55e51e798ae0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fbce402ade0 tx=0x7fbce4031680 comp rx=0 tx=0).stop 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 shutdown_connections 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e799110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78fdf0 0x55e51e798ae0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e7ab830 0x55e51e7adc20 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 --2- 172.21.15.33:0/1700578915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78f4d0 0x55e51e79b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 >> 172.21.15.33:0/1700578915 conn(0x55e51e78d300 msgr2=0x55e51e792ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcd9ffb640 0 client.15585 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 shutdown_connections 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 1 -- 172.21.15.33:0/1700578915 wait complete. 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.858+0000 7fbcfae25740 20 client.15585 trim_cache size 0 max 16384 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.871+0000 7fbcfae25740 1 Processor -- start 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.871+0000 7fbcfae25740 1 -- start start 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78f4d0 0x55e51e78cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e79a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e79d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcfae25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41ae410 con 0x55e51e78f4d0 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcfae25740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41b0010 con 0x55e51e79bd80 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcfae25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41aeae0 con 0x55e51e78fdf0 2024-09-10T21:05:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78f4d0 0x55e51e78cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78f4d0 0x55e51e78cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45474/0 (socket says 172.21.15.33:45474) 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/3024044632 learned_addr learned my addr 172.21.15.33:0/3024044632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf259c640 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e79a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e79d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf259c640 1 -- 172.21.15.33:0/3024044632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 msgr2=0x55e51e79d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf259c640 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e79d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf259c640 1 -- 172.21.15.33:0/3024044632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78f4d0 msgr2=0x55e51e78cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf259c640 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78f4d0 0x55e51e78cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf259c640 1 -- 172.21.15.33:0/3024044632 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e51e788bb0 con 0x55e51e78fdf0 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78f4d0 0x55e51e78cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcf259c640 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e79a820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbce4004af0 tx=0x7fbce402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=779d65d21e9ba7a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.872+0000 7fbcdbfff640 1 -- 172.21.15.33:0/3024044632 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4003970 con 0x55e51e78fdf0 2024-09-10T21:05:33.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcdbfff640 1 -- 172.21.15.33:0/3024044632 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce4036050 con 0x55e51e78fdf0 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcdbfff640 1 -- 172.21.15.33:0/3024044632 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce4030820 con 0x55e51e78fdf0 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 -- 172.21.15.33:0/3024044632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 msgr2=0x55e51e79a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e79a820 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbce4004af0 tx=0x7fbce402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 -- 172.21.15.33:0/3024044632 shutdown_connections 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e79bd80 0x55e51e79d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e79a820 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3024044632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e78f4d0 0x55e51e78cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 -- 172.21.15.33:0/3024044632 >> 172.21.15.33:0/3024044632 conn(0x55e51e78d300 msgr2=0x55e51e79a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 -- 172.21.15.33:0/3024044632 shutdown_connections 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 -- 172.21.15.33:0/3024044632 wait complete. 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.873+0000 7fbcfae25740 1 -- start start 2024-09-10T21:05:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 1 Processor -- start 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78f4d0 0x55e51e7a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e7a4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41af880 con 0x55e51e79bd80 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41af1b0 con 0x55e51e78f4d0 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf41ac3d0 con 0x55e51e78fdf0 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcfae25740 10 client.0 _ll_register_callbacks cb 0x7fbce40099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45490/0 (socket says 172.21.15.33:45490) 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/3113160045 learned_addr learned my addr 172.21.15.33:0/3113160045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf259c640 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e7a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78f4d0 0x55e51e7a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 msgr2=0x55e51e7a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e7a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/3113160045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78f4d0 msgr2=0x55e51e7a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78f4d0 0x55e51e7a4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.874+0000 7fbcf1d9b640 1 -- 172.21.15.33:0/3113160045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e51e79f680 con 0x55e51e79bd80 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcf1d9b640 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79ee50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006010 tx=0x7fbcdc00c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48da144aa7971b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcd9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/3113160045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc029040 con 0x55e51e79bd80 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/3113160045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbcdc0118a0 con 0x55e51e79bd80 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce4036050 con 0x55e51e79bd80 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/3113160045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fbcdc0142d0 con 0x55e51e79bd80 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 10 client.15591 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e51e819420 con 0x55e51e79bd80 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/3113160045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbce4036050 con 0x55e51e79bd80 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/3113160045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fbcdc004680 con 0x55e51e79bd80 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 10 client.15591 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 10 client.15591 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 20 client.15591 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e819980 0x55e51e81bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 4 client.15591 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.875+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e51e7038a0 con 0x55e51e819980 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcf259c640 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e819980 0x55e51e81bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcf259c640 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e819980 0x55e51e81bda0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=635e8d431b9fcc8a server_cookie=4a8ba03e7f5da0a1 in_seq=0 out_seq=0 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcd9ffb640 10 client.15591 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:05:33.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcd9ffb640 1 -- 172.21.15.33:0/3113160045 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e51e7038a0 con 0x55e51e819980 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcd9ffb640 10 client.15591 handle_command_reply: tid=0 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 client.15591 shutdown 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 msgr2=0x55e51e79ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79ee50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc006010 tx=0x7fbcdc00c570 comp rx=0 tx=0).stop 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 shutdown_connections 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e51e79bd80 0x55e51e79ee50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e51e819980 0x55e51e81bda0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e51e78fdf0 0x55e51e7a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcd9ffb640 0 client.15591 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 --2- 172.21.15.33:0/3113160045 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e51e78f4d0 0x55e51e7a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.876+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 >> 172.21.15.33:0/3113160045 conn(0x55e51e78d300 msgr2=0x55e51e79a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 shutdown_connections 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcfae25740 1 -- 172.21.15.33:0/3113160045 wait complete. 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcfae25740 20 client.15591 trim_cache size 0 max 16384 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbce00b52e0 msgr2=0x7fbce00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbce00b52e0 0x7fbce00b77a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fbcec0a8440 tx=0x7fbcec093040 comp rx=0 tx=0).stop 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4108020 msgr2=0x7fbcf419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4108020 0x7fbcf419ebe0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00d950 tx=0x7fbcec00de10 comp rx=0 tx=0).stop 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 shutdown_connections 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbce00b52e0 0x7fbce00b77a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcf4108020 0x7fbcf419ebe0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf41076a0 0x7fbcf4199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 --2- 172.21.15.33:0/1649847391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf4101c90 0x7fbcf4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.877+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 >> 172.21.15.33:0/1649847391 conn(0x7fbcf40fd700 msgr2=0x7fbcf40ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.878+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 shutdown_connections 2024-09-10T21:05:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:32.878+0000 7fbcf8a50640 1 -- 172.21.15.33:0/1649847391 wait complete. 2024-09-10T21:05:33.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:33 smithi104 ceph-mon[35325]: pgmap v393: 97 pgs: 97 active+clean; 48 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-10T21:05:33.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:33 smithi174 ceph-mon[35635]: pgmap v393: 97 pgs: 97 active+clean; 48 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-10T21:05:33.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:33 smithi033 ceph-mon[30860]: pgmap v393: 97 pgs: 97 active+clean; 48 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 2.1 MiB/s wr, 189 op/s 2024-09-10T21:05:33.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88934 14.67 MB/sec warmup 34 sec latency 1594.067 ms 2024-09-10T21:05:33.798 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:05:34.063 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.555+0000 7fe32a6a9640 1 -- 172.21.15.33:0/802443257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32410d140 msgr2=0x7fe32410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.555+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/802443257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32410d140 0x7fe32410f5a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009ca0 tx=0x7fe31c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.555+0000 7fe32a6a9640 1 -- 172.21.15.33:0/802443257 shutdown_connections 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.555+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/802443257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe32410d140 0x7fe32410f5a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.556+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/802443257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe324104600 0x7fe32410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.556+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/802443257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe324103ce0 0x7fe3241040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.556+0000 7fe32a6a9640 1 -- 172.21.15.33:0/802443257 >> 172.21.15.33:0/802443257 conn(0x7fe3240fd700 msgr2=0x7fe3240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.556+0000 7fe32a6a9640 1 -- 172.21.15.33:0/802443257 shutdown_connections 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.556+0000 7fe32a6a9640 1 -- 172.21.15.33:0/802443257 wait complete. 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.556+0000 7fe32a6a9640 1 Processor -- start 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.556+0000 7fe32a6a9640 1 -- start start 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe32a6a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324103ce0 0x7fe324199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe32a6a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe324104600 0x7fe324199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe32a6a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32410d140 0x7fe32419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe32a6a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe324112e10 con 0x7fe324103ce0 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe32a6a9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe324112c90 con 0x7fe32410d140 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe32a6a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe324112f90 con 0x7fe324104600 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324103ce0 0x7fe324199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324103ce0 0x7fe324199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45508/0 (socket says 172.21.15.33:45508) 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 -- 172.21.15.33:0/2513076306 learned_addr learned my addr 172.21.15.33:0/2513076306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe328c46640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32410d140 0x7fe32419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe3237fe640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe324104600 0x7fe324199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 -- 172.21.15.33:0/2513076306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe324104600 msgr2=0x7fe324199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe324104600 0x7fe324199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 -- 172.21.15.33:0/2513076306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32410d140 msgr2=0x7fe32419ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32410d140 0x7fe32419ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 -- 172.21.15.33:0/2513076306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32419f3f0 con 0x7fe324103ce0 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe328c46640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32410d140 0x7fe32419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe3237fe640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe324104600 0x7fe324199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe323fff640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324103ce0 0x7fe324199340 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c890 tx=0x7fe31400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe3217fa640 1 -- 172.21.15.33:0/2513076306 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314004220 con 0x7fe324103ce0 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32419f6e0 con 0x7fe324103ce0 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe3217fa640 1 -- 172.21.15.33:0/2513076306 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3140043c0 con 0x7fe324103ce0 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.557+0000 7fe3217fa640 1 -- 172.21.15.33:0/2513076306 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314004a00 con 0x7fe324103ce0 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.558+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3241a5f70 con 0x7fe324103ce0 2024-09-10T21:05:34.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 Processor -- start 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe3217fa640 1 -- 172.21.15.33:0/2513076306 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe3140079b0 con 0x7fe324103ce0 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 -- start start 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563632225ea0 con 0x5636322af0f0 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636322152d0 con 0x5636322afa10 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5636322190e0 con 0x5636322bfdb0 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45510/0 (socket says 172.21.15.33:45510) 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.560+0000 7fe328c46640 1 -- 172.21.15.33:0/4116613994 learned_addr learned my addr 172.21.15.33:0/4116613994 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:34.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe323fff640 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe3237fe640 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe328c46640 1 -- 172.21.15.33:0/4116613994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 msgr2=0x5636322c21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe328c46640 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322c21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe328c46640 1 -- 172.21.15.33:0/4116613994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 msgr2=0x5636322bf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe328c46640 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe328c46640 1 -- 172.21.15.33:0/4116613994 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563631f73650 con 0x5636322af0f0 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe3217fa640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe30c0b5360 0x7fe30c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe323fff640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe30c0b5360 0x7fe30c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe328c46640 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322af4d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009a20 tx=0x7fe31c031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6160f2b8423bfd51 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe3217fa640 1 -- 172.21.15.33:0/2513076306 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe3140e59a0 con 0x7fe324103ce0 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe30a7fc640 1 -- 172.21.15.33:0/4116613994 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c031710 con 0x5636322af0f0 2024-09-10T21:05:34.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe30a7fc640 1 -- 172.21.15.33:0/4116613994 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c040070 con 0x5636322af0f0 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.561+0000 7fe30a7fc640 1 -- 172.21.15.33:0/4116613994 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c004830 con 0x5636322af0f0 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 -- 172.21.15.33:0/4116613994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 msgr2=0x5636322af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe323fff640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe30c0b5360 0x7fe30c0b7820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe314005fd0 tx=0x7fe3140cf040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322af4d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe31c009a20 tx=0x7fe31c031af0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 -- 172.21.15.33:0/4116613994 shutdown_connections 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322c21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/4116613994 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322af4d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 -- 172.21.15.33:0/4116613994 >> 172.21.15.33:0/4116613994 conn(0x5636322acf20 msgr2=0x5636322ae0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 -- 172.21.15.33:0/4116613994 shutdown_connections 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.562+0000 7fe32ca7e740 1 -- 172.21.15.33:0/4116613994 wait complete. 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 -- start start 2024-09-10T21:05:34.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 Processor -- start 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322bf6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322bd1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563632076700 con 0x5636322af0f0 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636321b0160 con 0x5636322afa10 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5636322c2a00 con 0x5636322bfdb0 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe32ca7e740 10 client.0 _ll_register_callbacks cb 0x5636321c99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322bf6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322bf6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45512/0 (socket says 172.21.15.33:45512) 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.563+0000 7fe328c46640 1 -- 172.21.15.33:0/180705593 learned_addr learned my addr 172.21.15.33:0/180705593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe3237fe640 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe328c46640 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322bd1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe328c46640 1 -- 172.21.15.33:0/180705593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 msgr2=0x5636322bd1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe328c46640 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322bd1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe328c46640 1 -- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 msgr2=0x5636322bcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe328c46640 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bcb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe328c46640 1 -- 172.21.15.33:0/180705593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636321ddc90 con 0x5636322af0f0 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe328c46640 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322bf6e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe31c004510 tx=0x7fe31c004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a945aa6f95bc1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe3037fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe3037fe640 1 -- 172.21.15.33:0/180705593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c050020 con 0x5636322af0f0 2024-09-10T21:05:34.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31c040070 con 0x5636322af0f0 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe3037fe640 1 -- 172.21.15.33:0/180705593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe31c02f870 con 0x5636322af0f0 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe3037fe640 1 -- 172.21.15.33:0/180705593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe31c02fa70 con 0x5636322af0f0 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe32ca7e740 10 client.15603 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5636322ce3d0 con 0x5636322af0f0 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.564+0000 7fe3037fe640 1 -- 172.21.15.33:0/180705593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c040070 con 0x5636322af0f0 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe3037fe640 1 -- 172.21.15.33:0/180705593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe31c0305c0 con 0x5636322af0f0 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe32ca7e740 10 client.15603 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe32ca7e740 10 client.15603 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe32ca7e740 20 client.15603 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636322cf720 0x5636322d1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe32ca7e740 4 client.15603 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:34.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563632227df0 con 0x5636322cf720 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe3237fe640 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636322cf720 0x5636322d1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe3237fe640 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636322cf720 0x5636322d1b10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5b2cc3f44338264c server_cookie=100e3e1cede71d81 in_seq=0 out_seq=0 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.565+0000 7fe3037fe640 10 client.15603 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.566+0000 7fe3037fe640 1 -- 172.21.15.33:0/180705593 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563632227df0 con 0x5636322cf720 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.566+0000 7fe3037fe640 10 client.15603 handle_command_reply: tid=0 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.566+0000 7fe32ca7e740 1 client.15603 shutdown 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.566+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 msgr2=0x5636322bf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.566+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322bf6e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe31c004510 tx=0x7fe31c004410 comp rx=0 tx=0).stop 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 shutdown_connections 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322bd1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322bcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636322cf720 0x5636322d1b10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/180705593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322bf6e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 >> 172.21.15.33:0/180705593 conn(0x5636322acf20 msgr2=0x5636322b2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe3037fe640 0 client.15603 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 shutdown_connections 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 1 -- 172.21.15.33:0/180705593 wait complete. 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.567+0000 7fe32ca7e740 20 client.15603 trim_cache size 0 max 16384 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 Processor -- start 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 -- start start 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322ac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322d76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241ac2e0 con 0x5636322af0f0 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241adee0 con 0x5636322afa10 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.580+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241ac9b0 con 0x5636322bfdb0 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45520/0 (socket says 172.21.15.33:45520) 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 -- 172.21.15.33:0/665717562 learned_addr learned my addr 172.21.15.33:0/665717562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322d76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe3237fe640 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322ac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 -- 172.21.15.33:0/665717562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 msgr2=0x5636322d76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322d76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 -- 172.21.15.33:0/665717562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 msgr2=0x5636322ac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322ac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 -- 172.21.15.33:0/665717562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636322bb030 con 0x5636322af0f0 2024-09-10T21:05:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe3237fe640 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe328c46640 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322cbd00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe31c02f590 tx=0x7fe31c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621d4589570f306 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe309ffb640 1 -- 172.21.15.33:0/665717562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c050040 con 0x5636322af0f0 2024-09-10T21:05:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe309ffb640 1 -- 172.21.15.33:0/665717562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c040070 con 0x5636322af0f0 2024-09-10T21:05:34.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.581+0000 7fe309ffb640 1 -- 172.21.15.33:0/665717562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c030e10 con 0x5636322af0f0 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 -- 172.21.15.33:0/665717562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 msgr2=0x5636322cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322cbd00 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe31c02f590 tx=0x7fe31c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 -- 172.21.15.33:0/665717562 shutdown_connections 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322bfdb0 0x5636322d76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322ac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/665717562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322af0f0 0x5636322cbd00 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 -- 172.21.15.33:0/665717562 >> 172.21.15.33:0/665717562 conn(0x5636322acf20 msgr2=0x5636322adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 -- 172.21.15.33:0/665717562 shutdown_connections 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.582+0000 7fe32ca7e740 1 -- 172.21.15.33:0/665717562 wait complete. 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 -- start start 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 Processor -- start 2024-09-10T21:05:34.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322af0f0 0x5636322d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322b8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322bfdb0 0x5636322b8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241ad750 con 0x5636322bfdb0 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241ad080 con 0x5636322afa10 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3241aa2c0 con 0x5636322af0f0 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe32ca7e740 10 client.0 _ll_register_callbacks cb 0x7fe3140e8f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322bfdb0 0x5636322b8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe328c46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322bfdb0 0x5636322b8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45536/0 (socket says 172.21.15.33:45536) 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe328c46640 1 -- 172.21.15.33:0/365709351 learned_addr learned my addr 172.21.15.33:0/365709351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.583+0000 7fe328c46640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322af0f0 0x5636322d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe3237fe640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322b8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe328c46640 1 -- 172.21.15.33:0/365709351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322af0f0 msgr2=0x5636322d5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe328c46640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322af0f0 0x5636322d5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe328c46640 1 -- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 msgr2=0x5636322b8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe328c46640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322b8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe328c46640 1 -- 172.21.15.33:0/365709351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636322d0ad0 con 0x5636322bfdb0 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe3237fe640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe328c46640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322bfdb0 0x5636322b8cf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0309c0 tx=0x7fe31c02f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7c4b6eec89f5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe3037fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe3037fe640 1 -- 172.21.15.33:0/365709351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe31c02fed0 con 0x5636322bfdb0 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe3037fe640 1 -- 172.21.15.33:0/365709351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe31c003970 con 0x5636322bfdb0 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe31c040070 con 0x5636322bfdb0 2024-09-10T21:05:34.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe3037fe640 1 -- 172.21.15.33:0/365709351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe31c003b70 con 0x5636322bfdb0 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe32ca7e740 10 client.15609 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.584+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563632338400 con 0x5636322bfdb0 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe3037fe640 1 -- 172.21.15.33:0/365709351 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe31c040070 con 0x5636322bfdb0 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe3037fe640 1 -- 172.21.15.33:0/365709351 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fe31c045710 con 0x5636322bfdb0 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe32ca7e740 10 client.15609 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe32ca7e740 10 client.15609 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe32ca7e740 20 client.15609 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636323388e0 0x563632338cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe32ca7e740 4 client.15609 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636322238a0 con 0x5636323388e0 2024-09-10T21:05:34.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.585+0000 7fe3237fe640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636323388e0 0x563632338cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe3037fe640 10 client.15609 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe3237fe640 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636323388e0 0x563632338cf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9926a9c90e3b247 server_cookie=b106b19d98d5b2cd in_seq=0 out_seq=0 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe3037fe640 1 -- 172.21.15.33:0/365709351 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636322238a0 con 0x5636323388e0 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe3037fe640 10 client.15609 handle_command_reply: tid=0 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 client.15609 shutdown 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322bfdb0 msgr2=0x5636322b8cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322bfdb0 0x5636322b8cf0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe31c0309c0 tx=0x7fe31c02f4e0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 shutdown_connections 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5636323388e0 0x563632338cf0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636322bfdb0 0x5636322b8cf0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636322afa10 0x5636322b8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 --2- 172.21.15.33:0/365709351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636322af0f0 0x5636322d5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 >> 172.21.15.33:0/365709351 conn(0x5636322acf20 msgr2=0x5636322ae420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe3037fe640 0 client.15609 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:05:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 shutdown_connections 2024-09-10T21:05:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 1 -- 172.21.15.33:0/365709351 wait complete. 2024-09-10T21:05:34.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.586+0000 7fe32ca7e740 20 client.15609 trim_cache size 0 max 16384 2024-09-10T21:05:34.607 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe30c0b5360 msgr2=0x7fe30c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe30c0b5360 0x7fe30c0b7820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe314005fd0 tx=0x7fe3140cf040 comp rx=0 tx=0).stop 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324103ce0 msgr2=0x7fe324199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324103ce0 0x7fe324199340 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c890 tx=0x7fe31400cd50 comp rx=0 tx=0).stop 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 shutdown_connections 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe30c0b5360 0x7fe30c0b7820 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe32410d140 0x7fe32419ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe324104600 0x7fe324199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 --2- 172.21.15.33:0/2513076306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324103ce0 0x7fe324199340 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 >> 172.21.15.33:0/2513076306 conn(0x7fe3240fd700 msgr2=0x7fe3240fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:34.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 shutdown_connections 2024-09-10T21:05:34.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:34.587+0000 7fe32a6a9640 1 -- 172.21.15.33:0/2513076306 wait complete. 2024-09-10T21:05:34.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88934 14.25 MB/sec warmup 35 sec latency 2594.183 ms 2024-09-10T21:05:35.398 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:05:35.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:35 smithi104 ceph-mon[35325]: pgmap v394: 97 pgs: 97 active+clean; 56 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 7.1 MiB/s wr, 330 op/s 2024-09-10T21:05:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:35 smithi174 ceph-mon[35635]: pgmap v394: 97 pgs: 97 active+clean; 56 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 7.1 MiB/s wr, 330 op/s 2024-09-10T21:05:35.659 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:35 smithi033 ceph-mon[30860]: pgmap v394: 97 pgs: 97 active+clean; 56 MiB data, 498 MiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 7.1 MiB/s wr, 330 op/s 2024-09-10T21:05:35.697 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88934 13.85 MB/sec warmup 36 sec latency 3594.225 ms 2024-09-10T21:05:36.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 -- 172.21.15.33:0/1343730780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a8103ca0 msgr2=0x7fa1a8104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/1343730780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a8103ca0 0x7fa1a8104080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009900 tx=0x7fa19c02a410 comp rx=0 tx=0).stop 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 -- 172.21.15.33:0/1343730780 shutdown_connections 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/1343730780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1a810d100 0x7fa1a810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/1343730780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a81045c0 0x7fa1a810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/1343730780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a8103ca0 0x7fa1a8104080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 -- 172.21.15.33:0/1343730780 >> 172.21.15.33:0/1343730780 conn(0x7fa1a80fd700 msgr2=0x7fa1a80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 -- 172.21.15.33:0/1343730780 shutdown_connections 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.233+0000 7fa1aeb01640 1 -- 172.21.15.33:0/1343730780 wait complete. 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.234+0000 7fa1aeb01640 1 Processor -- start 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.234+0000 7fa1aeb01640 1 -- start start 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.234+0000 7fa1aeb01640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a8103ca0 0x7fa1a8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1aeb01640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1a81045c0 0x7fa1a8199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1aeb01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a810d100 0x7fa1a819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1aeb01640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8112df0 con 0x7fa1a810d100 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1aeb01640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8112c70 con 0x7fa1a8103ca0 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1aeb01640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8112f70 con 0x7fa1a81045c0 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a810d100 0x7fa1a819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a810d100 0x7fa1a819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45560/0 (socket says 172.21.15.33:45560) 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 -- 172.21.15.33:0/450280058 learned_addr learned my addr 172.21.15.33:0/450280058 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a8103ca0 0x7fa1a8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1a81045c0 0x7fa1a8199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 -- 172.21.15.33:0/450280058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1a81045c0 msgr2=0x7fa1a8199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1a81045c0 0x7fa1a8199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 -- 172.21.15.33:0/450280058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a8103ca0 msgr2=0x7fa1a8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a8103ca0 0x7fa1a8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 -- 172.21.15.33:0/450280058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a819f3d0 con 0x7fa1a810d100 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1ad09e640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a810d100 0x7fa1a819ec00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000eaa0 tx=0x7fa1a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1a5ffb640 1 -- 172.21.15.33:0/450280058 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000cdb0 con 0x7fa1a810d100 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a819f6c0 con 0x7fa1a810d100 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1a5ffb640 1 -- 172.21.15.33:0/450280058 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a0005020 con 0x7fa1a810d100 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.235+0000 7fa1a5ffb640 1 -- 172.21.15.33:0/450280058 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a00116d0 con 0x7fa1a810d100 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.236+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a5f50 con 0x7fa1a810d100 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 Processor -- start 2024-09-10T21:05:36.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1a5ffb640 1 -- 172.21.15.33:0/450280058 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa1a000f040 con 0x7fa1a810d100 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 -- start start 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfa48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d4abf1aea0 con 0x55d4abfa4df0 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d4abf0a2d0 con 0x55d4abfa44d0 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.238+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4abf0e0e0 con 0x55d4abfb0d80 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45570/0 (socket says 172.21.15.33:45570) 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 -- 172.21.15.33:0/3550125081 learned_addr learned my addr 172.21.15.33:0/3550125081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfa48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 -- 172.21.15.33:0/3550125081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 msgr2=0x55d4abfb3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 -- 172.21.15.33:0/3550125081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 msgr2=0x55d4abfa48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfa48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 -- 172.21.15.33:0/3550125081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4abc68650 con 0x55d4abfa4df0 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1a5ffb640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1980b52e0 0x7fa1980b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb0840 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009900 tx=0x7fa19c032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93b16e887d10805 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1980b52e0 0x7fa1980b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa1a5ffb640 1 -- 172.21.15.33:0/450280058 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa1a00e5980 con 0x7fa1a810d100 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa192ffd640 1 -- 172.21.15.33:0/3550125081 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c040070 con 0x55d4abfa4df0 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa192ffd640 1 -- 172.21.15.33:0/3550125081 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0316a0 con 0x55d4abfa4df0 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.239+0000 7fa192ffd640 1 -- 172.21.15.33:0/3550125081 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c035040 con 0x55d4abfa4df0 2024-09-10T21:05:36.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3550125081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 msgr2=0x55d4abfb0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1980b52e0 0x7fa1980b77a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa1a810f7b0 tx=0x7fa194009510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb0840 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009900 tx=0x7fa19c032e60 comp rx=0 tx=0).stop 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3550125081 shutdown_connections 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb0840 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3550125081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfa48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3550125081 >> 172.21.15.33:0/3550125081 conn(0x55d4abfa2300 msgr2=0x55d4abfa34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3550125081 shutdown_connections 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.240+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3550125081 wait complete. 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 -- start start 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 Processor -- start 2024-09-10T21:05:36.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa44d0 0x55d4abfb0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa4df0 0x55d4abfada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d4abd6b700 con 0x55d4abfa44d0 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d4abea5160 con 0x55d4abfa4df0 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4abfb39a0 con 0x55d4abfb0d80 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1b0ed6740 10 client.0 _ll_register_callbacks cb 0x55d4abebe9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1ac89d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa44d0 0x55d4abfb0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1ac89d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa44d0 0x55d4abfb0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45584/0 (socket says 172.21.15.33:45584) 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa4df0 0x55d4abfada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1ac89d640 1 -- 172.21.15.33:0/1028250902 learned_addr learned my addr 172.21.15.33:0/1028250902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:36.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.241+0000 7fa1ad09e640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1a7fff640 1 -- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 msgr2=0x55d4abfae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1a7fff640 1 -- 172.21.15.33:0/1028250902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa44d0 msgr2=0x55d4abfb0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa44d0 0x55d4abfb0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1a7fff640 1 -- 172.21.15.33:0/1028250902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4abed2c90 con 0x55d4abfa4df0 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa44d0 0x55d4abfb0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa4df0 0x55d4abfada80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1a806b950 tx=0x7fa194009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d11c3dafe5e1af6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa187fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa187fff640 1 -- 172.21.15.33:0/1028250902 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194027070 con 0x55d4abfa4df0 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa187fff640 1 -- 172.21.15.33:0/1028250902 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa194015e30 con 0x55d4abfa4df0 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19c0316a0 con 0x55d4abfa4df0 2024-09-10T21:05:36.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa187fff640 1 -- 172.21.15.33:0/1028250902 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa194033050 con 0x55d4abfa4df0 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1b0ed6740 10 client.25217 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.242+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d4abfbf350 con 0x55d4abfa4df0 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa187fff640 1 -- 172.21.15.33:0/1028250902 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c0316a0 con 0x55d4abfa4df0 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa187fff640 1 -- 172.21.15.33:0/1028250902 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa194022070 con 0x55d4abfa4df0 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa1b0ed6740 10 client.25217 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa1b0ed6740 10 client.25217 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa1b0ed6740 20 client.25217 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:36.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4abfc0800 0x55d4abfc2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa1b0ed6740 4 client.25217 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4abf1cdf0 con 0x55d4abfc0800 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.243+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4abfc0800 0x55d4abfc2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.244+0000 7fa187fff640 10 client.25217 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.244+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4abfc0800 0x55d4abfc2ca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb5b013439ef040c server_cookie=daa151523f422a19 in_seq=0 out_seq=0 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.244+0000 7fa187fff640 1 -- 172.21.15.33:0/1028250902 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4abf1cdf0 con 0x55d4abfc0800 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.244+0000 7fa187fff640 10 client.25217 handle_command_reply: tid=0 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.244+0000 7fa1b0ed6740 1 client.25217 shutdown 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa4df0 msgr2=0x55d4abfada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa4df0 0x55d4abfada80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1a806b950 tx=0x7fa194009060 comp rx=0 tx=0).stop 2024-09-10T21:05:36.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 shutdown_connections 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa4df0 0x55d4abfada80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4abfc0800 0x55d4abfc2ca0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/1028250902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa44d0 0x55d4abfb0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 >> 172.21.15.33:0/1028250902 conn(0x55d4abfa2300 msgr2=0x55d4abfa7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa187fff640 0 client.25217 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 shutdown_connections 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/1028250902 wait complete. 2024-09-10T21:05:36.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.245+0000 7fa1b0ed6740 20 client.25217 trim_cache size 0 max 16384 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 Processor -- start 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 -- start start 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfbcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfacf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfac300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81ae490 con 0x55d4abfa4df0 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81b0090 con 0x55d4abfa44d0 2024-09-10T21:05:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.258+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81aeb60 con 0x55d4abfb0d80 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfacf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfacf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45594/0 (socket says 172.21.15.33:45594) 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 -- 172.21.15.33:0/3895119342 learned_addr learned my addr 172.21.15.33:0/3895119342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfbcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1ad09e640 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 -- 172.21.15.33:0/3895119342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 msgr2=0x55d4abfac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 -- 172.21.15.33:0/3895119342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 msgr2=0x55d4abfbcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfbcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 -- 172.21.15.33:0/3895119342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4abfa2d80 con 0x55d4abfa4df0 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1ad09e640 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfacf70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa194023e60 tx=0x7fa194013f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a647caf4cd9e7fed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1927fc640 1 -- 172.21.15.33:0/3895119342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194027040 con 0x55d4abfa4df0 2024-09-10T21:05:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1927fc640 1 -- 172.21.15.33:0/3895119342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19403c070 con 0x55d4abfa4df0 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.259+0000 7fa1927fc640 1 -- 172.21.15.33:0/3895119342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194016a90 con 0x55d4abfa4df0 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3895119342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 msgr2=0x55d4abfacf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfacf70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa194023e60 tx=0x7fa194013f50 comp rx=0 tx=0).stop 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3895119342 shutdown_connections 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfb0d80 0x55d4abfac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfacf70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/3895119342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfa44d0 0x55d4abfbcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3895119342 >> 172.21.15.33:0/3895119342 conn(0x55d4abfa2300 msgr2=0x55d4abfa2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3895119342 shutdown_connections 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/3895119342 wait complete. 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.260+0000 7fa1b0ed6740 1 -- start start 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 1 Processor -- start 2024-09-10T21:05:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfa44d0 0x55d4abfb8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81af900 con 0x55d4abfa4df0 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81af230 con 0x55d4abfb0d80 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81ac450 con 0x55d4abfa44d0 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1b0ed6740 10 client.0 _ll_register_callbacks cb 0x7fa194009940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45598/0 (socket says 172.21.15.33:45598) 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 -- 172.21.15.33:0/2813913678 learned_addr learned my addr 172.21.15.33:0/2813913678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1ad09e640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfa44d0 0x55d4abfb8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 -- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfa44d0 msgr2=0x55d4abfb8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfa44d0 0x55d4abfb8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 -- 172.21.15.33:0/2813913678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfb0d80 msgr2=0x55d4abfb3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1a7fff640 1 -- 172.21.15.33:0/2813913678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4abfb9920 con 0x55d4abfa4df0 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.261+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfa44d0 0x55d4abfb8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1a7fff640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb91e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa194013bf0 tx=0x7fa194006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8a64d31b9d35d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa187fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa187fff640 1 -- 172.21.15.33:0/2813913678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194005bc0 con 0x55d4abfa4df0 2024-09-10T21:05:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa187fff640 1 -- 172.21.15.33:0/2813913678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa194005d60 con 0x55d4abfa4df0 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19403c070 con 0x55d4abfa4df0 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa187fff640 1 -- 172.21.15.33:0/2813913678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa194015e90 con 0x55d4abfa4df0 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 10 client.15627 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d4ac02dc20 con 0x55d4abfa4df0 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa187fff640 1 -- 172.21.15.33:0/2813913678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19403c070 con 0x55d4abfa4df0 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa187fff640 1 -- 172.21.15.33:0/2813913678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fa194022040 con 0x55d4abfa4df0 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 10 client.15627 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 10 client.15627 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 20 client.15627 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4ac02e2c0 0x55d4ac0306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 4 client.15627 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.262+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4abf188a0 con 0x55d4ac02e2c0 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.263+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4ac02e2c0 0x55d4ac0306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.263+0000 7fa1ac89d640 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4ac02e2c0 0x55d4ac0306b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c7c01e4f0d28321 server_cookie=d023f4059046c014 in_seq=0 out_seq=0 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.263+0000 7fa187fff640 10 client.15627 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa187fff640 1 -- 172.21.15.33:0/2813913678 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4abf188a0 con 0x55d4ac02e2c0 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa187fff640 10 client.15627 handle_command_reply: tid=0 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 client.15627 shutdown 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 msgr2=0x55d4abfb91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb91e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa194013bf0 tx=0x7fa194006d60 comp rx=0 tx=0).stop 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 shutdown_connections 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4abfb0d80 0x55d4abfb3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55d4ac02e2c0 0x55d4ac0306b0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa187fff640 0 client.15627 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4abfa4df0 0x55d4abfb91e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 --2- 172.21.15.33:0/2813913678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4abfa44d0 0x55d4abfb8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 >> 172.21.15.33:0/2813913678 conn(0x55d4abfa2300 msgr2=0x55d4abfa2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 shutdown_connections 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 1 -- 172.21.15.33:0/2813913678 wait complete. 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.264+0000 7fa1b0ed6740 20 client.15627 trim_cache size 0 max 16384 2024-09-10T21:05:36.277 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:05:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1980b52e0 msgr2=0x7fa1980b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1980b52e0 0x7fa1980b77a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa1a810f7b0 tx=0x7fa194009510 comp rx=0 tx=0).stop 2024-09-10T21:05:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a810d100 msgr2=0x7fa1a819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a810d100 0x7fa1a819ec00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000eaa0 tx=0x7fa1a000ef60 comp rx=0 tx=0).stop 2024-09-10T21:05:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 shutdown_connections 2024-09-10T21:05:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1980b52e0 0x7fa1980b77a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa1a810d100 0x7fa1a819ec00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1a81045c0 0x7fa1a8199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 --2- 172.21.15.33:0/450280058 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1a8103ca0 0x7fa1a8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 >> 172.21.15.33:0/450280058 conn(0x7fa1a80fd700 msgr2=0x7fa1a80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 shutdown_connections 2024-09-10T21:05:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:36.265+0000 7fa1aeb01640 1 -- 172.21.15.33:0/450280058 wait complete. 2024-09-10T21:05:36.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88934 13.48 MB/sec warmup 37 sec latency 4594.320 ms 2024-09-10T21:05:37.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:05:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:37.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:37 smithi033 ceph-mon[30860]: pgmap v395: 97 pgs: 97 active+clean; 56 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 8.3 MiB/s wr, 303 op/s 2024-09-10T21:05:37.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:37 smithi104 ceph-mon[35325]: pgmap v395: 97 pgs: 97 active+clean; 56 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 8.3 MiB/s wr, 303 op/s 2024-09-10T21:05:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:37 smithi174 ceph-mon[35635]: pgmap v395: 97 pgs: 97 active+clean; 56 MiB data, 507 MiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 8.3 MiB/s wr, 303 op/s 2024-09-10T21:05:37.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88963 13.13 MB/sec warmup 38 sec latency 4993.557 ms 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 -- 172.21.15.33:0/3409591680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984104600 msgr2=0x7fc98410caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/3409591680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984104600 0x7fc98410caf0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc978009900 tx=0x7fc97802a410 comp rx=0 tx=0).stop 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 -- 172.21.15.33:0/3409591680 shutdown_connections 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/3409591680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc98410d140 0x7fc98410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/3409591680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984104600 0x7fc98410caf0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/3409591680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984103ce0 0x7fc9841040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 -- 172.21.15.33:0/3409591680 >> 172.21.15.33:0/3409591680 conn(0x7fc9840fd700 msgr2=0x7fc9840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 -- 172.21.15.33:0/3409591680 shutdown_connections 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.794+0000 7fc98a7f9640 1 -- 172.21.15.33:0/3409591680 wait complete. 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 Processor -- start 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 -- start start 2024-09-10T21:05:37.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984103ce0 0x7fc984101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984104600 0x7fc984101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc98410d140 0x7fc984102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc984112e10 con 0x7fc984103ce0 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc984112c90 con 0x7fc984104600 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc98a7f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc984112f90 con 0x7fc98410d140 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984103ce0 0x7fc984101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984103ce0 0x7fc984101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45614/0 (socket says 172.21.15.33:45614) 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 -- 172.21.15.33:0/1402497237 learned_addr learned my addr 172.21.15.33:0/1402497237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc9837fe640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984104600 0x7fc984101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 -- 172.21.15.33:0/1402497237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc98410d140 msgr2=0x7fc984102410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc98410d140 0x7fc984102410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 -- 172.21.15.33:0/1402497237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984104600 msgr2=0x7fc984101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984104600 0x7fc984101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.795+0000 7fc983fff640 1 -- 172.21.15.33:0/1402497237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a7780 con 0x7fc984103ce0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.796+0000 7fc9837fe640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984104600 0x7fc984101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.796+0000 7fc983fff640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984103ce0 0x7fc984101990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc97400d950 tx=0x7fc97400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.796+0000 7fc9817fa640 1 -- 172.21.15.33:0/1402497237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc974017070 con 0x7fc984103ce0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.796+0000 7fc9817fa640 1 -- 172.21.15.33:0/1402497237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc974009c30 con 0x7fc984103ce0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.796+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a7a70 con 0x7fc984103ce0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.796+0000 7fc9817fa640 1 -- 172.21.15.33:0/1402497237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97400f620 con 0x7fc984103ce0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.796+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a8030 con 0x7fc984103ce0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 Processor -- start 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 -- start start 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc9817fa640 1 -- 172.21.15.33:0/1402497237 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc974010460 con 0x7fc984103ce0 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5646ca84aea0 con 0x5646ca8d4df0 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5646ca83a2d0 con 0x5646ca8d44d0 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc98cbce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646ca83e0e0 con 0x5646ca8e0d80 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.799+0000 7fc983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45628/0 (socket says 172.21.15.33:45628) 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 -- 172.21.15.33:0/1472037204 learned_addr learned my addr 172.21.15.33:0/1472037204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc988d96640 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc9837fe640 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 -- 172.21.15.33:0/1472037204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 msgr2=0x5646ca8e3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 -- 172.21.15.33:0/1472037204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 msgr2=0x5646ca8d48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8d48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 -- 172.21.15.33:0/1472037204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ca598650 con 0x5646ca8d4df0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc988d96640 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc983fff640 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e0840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc9740ebd00 tx=0x7fc9740ed750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40754d99289f01f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc96a7fc640 1 -- 172.21.15.33:0/1472037204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740f7670 con 0x5646ca8d4df0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc96a7fc640 1 -- 172.21.15.33:0/1472037204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9740f2070 con 0x5646ca8d4df0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc96a7fc640 1 -- 172.21.15.33:0/1472037204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9740eeb30 con 0x5646ca8d4df0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.800+0000 7fc9817fa640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc96c0b52e0 0x7fc96c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc988d96640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc96c0b52e0 0x7fc96c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc9817fa640 1 -- 172.21.15.33:0/1402497237 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc9740e58d0 con 0x7fc984103ce0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 -- 172.21.15.33:0/1472037204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 msgr2=0x5646ca8e0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e0840 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fc9740ebd00 tx=0x7fc9740ed750 comp rx=0 tx=0).stop 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc988d96640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc96c0b52e0 0x7fc96c0b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc984103a50 tx=0x7fc97c009470 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:37.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 -- 172.21.15.33:0/1472037204 shutdown_connections 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e0840 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 --2- 172.21.15.33:0/1472037204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8d48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 -- 172.21.15.33:0/1472037204 >> 172.21.15.33:0/1472037204 conn(0x5646ca8d2300 msgr2=0x5646ca8d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 -- 172.21.15.33:0/1472037204 shutdown_connections 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 -- 172.21.15.33:0/1472037204 wait complete. 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.801+0000 7fc98cbce740 1 -- start start 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 1 Processor -- start 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8e0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5646ca69b700 con 0x5646ca8d4df0 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5646ca7d5160 con 0x5646ca8e0d80 2024-09-10T21:05:37.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646ca8e3a00 con 0x5646ca8d44d0 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc98cbce740 10 client.0 _ll_register_callbacks cb 0x5646ca7ee9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc9837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc9837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8dda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45638/0 (socket says 172.21.15.33:45638) 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc9837fe640 1 -- 172.21.15.33:0/2647893174 learned_addr learned my addr 172.21.15.33:0/2647893174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc983fff640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.802+0000 7fc988d96640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9837fe640 1 -- 172.21.15.33:0/2647893174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 msgr2=0x5646ca8e0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9837fe640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8e0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9837fe640 1 -- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 msgr2=0x5646ca8de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9837fe640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9837fe640 1 -- 172.21.15.33:0/2647893174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ca802c90 con 0x5646ca8d4df0 2024-09-10T21:05:37.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc988d96640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9837fe640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8dda90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc978005ab0 tx=0x7fc978032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fe92aed4a9720b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9637fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9637fe640 1 -- 172.21.15.33:0/2647893174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97803f070 con 0x5646ca8d4df0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9637fe640 1 -- 172.21.15.33:0/2647893174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc978004040 con 0x5646ca8d4df0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9740f2070 con 0x5646ca8d4df0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9637fe640 1 -- 172.21.15.33:0/2647893174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc97804b050 con 0x5646ca8d4df0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc98cbce740 10 client.15639 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646ca8ef360 con 0x5646ca8d4df0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.803+0000 7fc9637fe640 1 -- 172.21.15.33:0/2647893174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9740f2070 con 0x5646ca8d4df0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc9637fe640 1 -- 172.21.15.33:0/2647893174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc9780046e0 con 0x5646ca8d4df0 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc98cbce740 10 client.15639 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc98cbce740 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc98cbce740 20 client.15639 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca8f08e0 0x5646ca8f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc98cbce740 4 client.15639 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646ca84cdf0 con 0x5646ca8f08e0 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc983fff640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca8f08e0 0x5646ca8f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc9637fe640 10 client.15639 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.804+0000 7fc983fff640 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca8f08e0 0x5646ca8f2d20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be146551e1c10fde server_cookie=bb78cda202282451 in_seq=0 out_seq=0 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc9637fe640 1 -- 172.21.15.33:0/2647893174 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646ca84cdf0 con 0x5646ca8f08e0 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc9637fe640 10 client.15639 handle_command_reply: tid=0 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 client.15639 shutdown 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 msgr2=0x5646ca8dda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8dda90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc978005ab0 tx=0x7fc978032e90 comp rx=0 tx=0).stop 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 shutdown_connections 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca8f08e0 0x5646ca8f2d20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8dda90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc9637fe640 0 client.15639 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2647893174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8e0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 >> 172.21.15.33:0/2647893174 conn(0x5646ca8d2300 msgr2=0x5646ca8d7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 shutdown_connections 2024-09-10T21:05:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.805+0000 7fc98cbce740 1 -- 172.21.15.33:0/2647893174 wait complete. 2024-09-10T21:05:37.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.806+0000 7fc98cbce740 20 client.15639 trim_cache size 0 max 16384 2024-09-10T21:05:37.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 Processor -- start 2024-09-10T21:05:37.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 -- start start 2024-09-10T21:05:37.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8ecc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8e2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b0720 con 0x5646ca8d4df0 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b2320 con 0x5646ca8e0d80 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc98cbce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b0df0 con 0x5646ca8d44d0 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc9837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc9837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8d1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45646/0 (socket says 172.21.15.33:45646) 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc9837fe640 1 -- 172.21.15.33:0/2935800286 learned_addr learned my addr 172.21.15.33:0/2935800286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.819+0000 7fc983fff640 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8ecc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc988d96640 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8e2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc9837fe640 1 -- 172.21.15.33:0/2935800286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 msgr2=0x5646ca8ecc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc9837fe640 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8ecc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc9837fe640 1 -- 172.21.15.33:0/2935800286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 msgr2=0x5646ca8e2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc9837fe640 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8e2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc9837fe640 1 -- 172.21.15.33:0/2935800286 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ca8d2d80 con 0x5646ca8d4df0 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc988d96640 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc983fff640 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc9837fe640 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8d1310 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc978003610 tx=0x7fc97802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad40127bde33e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc969ffb640 1 -- 172.21.15.33:0/2935800286 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97803f070 con 0x5646ca8d4df0 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc969ffb640 1 -- 172.21.15.33:0/2935800286 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc978004ca0 con 0x5646ca8d4df0 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.820+0000 7fc969ffb640 1 -- 172.21.15.33:0/2935800286 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978035070 con 0x5646ca8d4df0 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 -- 172.21.15.33:0/2935800286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 msgr2=0x5646ca8d1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8d1310 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc978003610 tx=0x7fc97802a410 comp rx=0 tx=0).stop 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 -- 172.21.15.33:0/2935800286 shutdown_connections 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8e0d80 0x5646ca8e2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8d1310 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 --2- 172.21.15.33:0/2935800286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8d44d0 0x5646ca8ecc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 -- 172.21.15.33:0/2935800286 >> 172.21.15.33:0/2935800286 conn(0x5646ca8d2300 msgr2=0x5646ca8d0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 -- 172.21.15.33:0/2935800286 shutdown_connections 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 -- 172.21.15.33:0/2935800286 wait complete. 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 -- start start 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.821+0000 7fc98cbce740 1 Processor -- start 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b1b90 con 0x5646ca8d4df0 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841b14c0 con 0x5646ca8d44d0 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841ae6e0 con 0x5646ca8e0d80 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 10 client.0 _ll_register_callbacks cb 0x7fc98406c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45660/0 (socket says 172.21.15.33:45660) 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 -- 172.21.15.33:0/3059790944 learned_addr learned my addr 172.21.15.33:0/3059790944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc983fff640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc988d96640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 -- 172.21.15.33:0/3059790944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 msgr2=0x5646ca8e3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 -- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 msgr2=0x5646ca8e8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8e8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 -- 172.21.15.33:0/3059790944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ca8e44e0 con 0x5646ca8d4df0 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc988d96640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9837fe640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e94d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc978004ba0 tx=0x7fc978004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6199555e2d037387 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9637fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9637fe640 1 -- 172.21.15.33:0/3059790944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc978035040 con 0x5646ca8d4df0 2024-09-10T21:05:37.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc9637fe640 1 -- 172.21.15.33:0/3059790944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc978004e50 con 0x5646ca8d4df0 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.822+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc978004ca0 con 0x5646ca8d4df0 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc9637fe640 1 -- 172.21.15.33:0/3059790944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc97803f040 con 0x5646ca8d4df0 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 10 client.15645 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646ca95e280 con 0x5646ca8d4df0 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc9637fe640 1 -- 172.21.15.33:0/3059790944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc978004ca0 con 0x5646ca8d4df0 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc9637fe640 1 -- 172.21.15.33:0/3059790944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc97803ba30 con 0x5646ca8d4df0 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 10 client.15645 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 10 client.15645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 20 client.15645 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca95ea50 0x5646ca960e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 4 client.15645 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646ca8488a0 con 0x5646ca95ea50 2024-09-10T21:05:37.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.823+0000 7fc983fff640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca95ea50 0x5646ca960e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc983fff640 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca95ea50 0x5646ca960e70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=547a59978737c98b server_cookie=2701168fd4dc9f49 in_seq=0 out_seq=0 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc9637fe640 10 client.15645 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc9637fe640 1 -- 172.21.15.33:0/3059790944 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646ca8488a0 con 0x5646ca95ea50 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc9637fe640 10 client.15645 handle_command_reply: tid=0 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 client.15645 shutdown 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 msgr2=0x5646ca8e94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e94d0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc978004ba0 tx=0x7fc978004970 comp rx=0 tx=0).stop 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 shutdown_connections 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5646ca95ea50 0x5646ca960e70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ca8e0d80 0x5646ca8e3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5646ca8d4df0 0x5646ca8e94d0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc9637fe640 0 client.15645 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 --2- 172.21.15.33:0/3059790944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5646ca8d44d0 0x5646ca8e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 >> 172.21.15.33:0/3059790944 conn(0x5646ca8d2300 msgr2=0x5646ca8d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 shutdown_connections 2024-09-10T21:05:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 1 -- 172.21.15.33:0/3059790944 wait complete. 2024-09-10T21:05:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.824+0000 7fc98cbce740 20 client.15645 trim_cache size 0 max 16384 2024-09-10T21:05:37.842 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:05:37.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc96c0b52e0 msgr2=0x7fc96c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc96c0b52e0 0x7fc96c0b77a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc984103a50 tx=0x7fc97c009470 comp rx=0 tx=0).stop 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984103ce0 msgr2=0x7fc984101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984103ce0 0x7fc984101990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc97400d950 tx=0x7fc97400de10 comp rx=0 tx=0).stop 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 shutdown_connections 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc96c0b52e0 0x7fc96c0b77a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc98410d140 0x7fc984102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc984104600 0x7fc984101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 --2- 172.21.15.33:0/1402497237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc984103ce0 0x7fc984101990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 >> 172.21.15.33:0/1402497237 conn(0x7fc9840fd700 msgr2=0x7fc9840fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 shutdown_connections 2024-09-10T21:05:37.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:37.825+0000 7fc98a7f9640 1 -- 172.21.15.33:0/1402497237 wait complete. 2024-09-10T21:05:38.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88963 12.79 MB/sec warmup 39 sec latency 1556.429 ms 2024-09-10T21:05:39.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:39 smithi104 ceph-mon[35325]: pgmap v396: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 8.2 MiB/s wr, 305 op/s 2024-09-10T21:05:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:39 smithi174 ceph-mon[35635]: pgmap v396: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 8.2 MiB/s wr, 305 op/s 2024-09-10T21:05:39.671 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:05:39.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:05:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:39 smithi033 ceph-mon[30860]: pgmap v396: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 8.2 MiB/s wr, 305 op/s 2024-09-10T21:05:39.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 88963 12.47 MB/sec warmup 40 sec latency 2556.475 ms 2024-09-10T21:05:39.934 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 -- 172.21.15.33:0/2327128722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c101c90 msgr2=0x7fc80c102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 --2- 172.21.15.33:0/2327128722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c101c90 0x7fc80c102070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009900 tx=0x7fc7fc02a410 comp rx=0 tx=0).stop 2024-09-10T21:05:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 -- 172.21.15.33:0/2327128722 shutdown_connections 2024-09-10T21:05:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 --2- 172.21.15.33:0/2327128722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc80c108020 0x7fc80c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 --2- 172.21.15.33:0/2327128722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc80c1076a0 0x7fc80c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 --2- 172.21.15.33:0/2327128722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c101c90 0x7fc80c102070 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 -- 172.21.15.33:0/2327128722 >> 172.21.15.33:0/2327128722 conn(0x7fc80c0fd700 msgr2=0x7fc80c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 -- 172.21.15.33:0/2327128722 shutdown_connections 2024-09-10T21:05:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 -- 172.21.15.33:0/2327128722 wait complete. 2024-09-10T21:05:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 Processor -- start 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.458+0000 7fc8131f0640 1 -- start start 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc8131f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc80c101c90 0x7fc80c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc8131f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c1076a0 0x7fc80c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc8131f0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc80c108020 0x7fc80c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc8131f0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111ce0 con 0x7fc80c1076a0 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc8131f0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111b60 con 0x7fc80c108020 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc8131f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c111e60 con 0x7fc80c101c90 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c1076a0 0x7fc80c19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c1076a0 0x7fc80c19ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60380/0 (socket says 172.21.15.33:60380) 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 -- 172.21.15.33:0/1273632364 learned_addr learned my addr 172.21.15.33:0/1273632364 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc810f8c640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc80c101c90 0x7fc80c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc81178d640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc80c108020 0x7fc80c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 -- 172.21.15.33:0/1273632364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc80c101c90 msgr2=0x7fc80c19b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc80c101c90 0x7fc80c19b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 -- 172.21.15.33:0/1273632364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc80c108020 msgr2=0x7fc80c1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc80c108020 0x7fc80c1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 -- 172.21.15.33:0/1273632364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c1a15c0 con 0x7fc80c1076a0 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc810f8c640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc80c101c90 0x7fc80c19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc81178d640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc80c108020 0x7fc80c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c1076a0 0x7fc80c19ba50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc80000b4a0 tx=0x7fc80000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.459+0000 7fc809ffb640 1 -- 172.21.15.33:0/1273632364 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800014070 con 0x7fc80c1076a0 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.460+0000 7fc809ffb640 1 -- 172.21.15.33:0/1273632364 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc800010ac0 con 0x7fc80c1076a0 2024-09-10T21:05:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.460+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1a18b0 con 0x7fc80c1076a0 2024-09-10T21:05:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.460+0000 7fc809ffb640 1 -- 172.21.15.33:0/1273632364 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800017c30 con 0x7fc80c1076a0 2024-09-10T21:05:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.460+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1a7ff0 con 0x7fc80c1076a0 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.462+0000 7fc809ffb640 1 -- 172.21.15.33:0/1273632364 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc800010610 con 0x7fc80c1076a0 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.462+0000 7fc8155c5740 1 Processor -- start 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.462+0000 7fc8155c5740 1 -- start start 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f94ff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f950b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f950db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc8155c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a6f9475e60 con 0x55a6f94feee0 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc8155c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a6f9465290 con 0x55a6f950b790 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc8155c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a6f94690a0 con 0x55a6f94ff800 2024-09-10T21:05:40.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f94ff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f94ff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60396/0 (socket says 172.21.15.33:60396) 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 -- 172.21.15.33:0/1276507796 learned_addr learned my addr 172.21.15.33:0/1276507796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc809ffb640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7f40b52e0 0x7fc7f40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f950db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc810f8c640 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f950b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 -- 172.21.15.33:0/1276507796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 msgr2=0x55a6f950b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f950b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 -- 172.21.15.33:0/1276507796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 msgr2=0x55a6f950db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f950db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc81178d640 1 -- 172.21.15.33:0/1276507796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f91c3930 con 0x55a6f94feee0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc809ffb640 1 -- 172.21.15.33:0/1273632364 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc8000e55b0 con 0x7fc80c1076a0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.463+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f950db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.464+0000 7fc81178d640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7f40b52e0 0x7fc7f40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.464+0000 7fc81178d640 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f94ff2c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc80400e480 tx=0x7fc80400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfd98d97c873263 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.464+0000 7fc7f2ffd640 1 -- 172.21.15.33:0/1276507796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804012e60 con 0x55a6f94feee0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.464+0000 7fc7f2ffd640 1 -- 172.21.15.33:0/1276507796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80401ce30 con 0x55a6f94feee0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.464+0000 7fc7f2ffd640 1 -- 172.21.15.33:0/1276507796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80401b610 con 0x55a6f94feee0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.464+0000 7fc81178d640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7f40b52e0 0x7fc7f40b77a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc80401f000 tx=0x7fc804020000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 -- 172.21.15.33:0/1276507796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 msgr2=0x55a6f94ff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f94ff2c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc80400e480 tx=0x7fc80400e940 comp rx=0 tx=0).stop 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 -- 172.21.15.33:0/1276507796 shutdown_connections 2024-09-10T21:05:40.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f950db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f950b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1276507796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f94ff2c0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 -- 172.21.15.33:0/1276507796 >> 172.21.15.33:0/1276507796 conn(0x55a6f94fcd10 msgr2=0x55a6f94fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 -- 172.21.15.33:0/1276507796 shutdown_connections 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 -- 172.21.15.33:0/1276507796 wait complete. 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 -- start start 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.465+0000 7fc8155c5740 1 Processor -- start 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9508790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f9508cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9509210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc8155c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a6f92c6890 con 0x55a6f94feee0 2024-09-10T21:05:40.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc8155c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a6f93ffd40 con 0x55a6f950b790 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc8155c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a6f950e440 con 0x55a6f94ff800 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc8155c5740 10 client.0 _ll_register_callbacks cb 0x55a6f9463c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9508790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9508790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60398/0 (socket says 172.21.15.33:60398) 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 -- 172.21.15.33:0/1557884341 learned_addr learned my addr 172.21.15.33:0/1557884341 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc81178d640 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9509210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f9508cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 -- 172.21.15.33:0/1557884341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 msgr2=0x55a6f9508cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f9508cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 -- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 msgr2=0x55a6f9509210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9509210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 -- 172.21.15.33:0/1557884341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f9517870 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc810f8c640 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9508790 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02a930 tx=0x7fc7fc0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3935b871071f389 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.466+0000 7fc7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc7ebfff640 1 -- 172.21.15.33:0/1557884341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc035070 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc7ebfff640 1 -- 172.21.15.33:0/1557884341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc7fc004240 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc80401ce30 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc7ebfff640 1 -- 172.21.15.33:0/1557884341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc004440 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 10 client.15657 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6f951a200 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc7ebfff640 1 -- 172.21.15.33:0/1557884341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80401ce30 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc7ebfff640 1 -- 172.21.15.33:0/1557884341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc7fc044680 con 0x55a6f94feee0 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 10 client.15657 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:40.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 20 client.15657 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f951b720 0x55a6f951db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 4 client.15657 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.467+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6f9477db0 con 0x55a6f951b720 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.468+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f951b720 0x55a6f951db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.468+0000 7fc80bfff640 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f951b720 0x55a6f951db80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31ff4316931e9e96 server_cookie=b9d97f9d852f47da in_seq=0 out_seq=0 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.468+0000 7fc7ebfff640 10 client.15657 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc7ebfff640 1 -- 172.21.15.33:0/1557884341 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a6f9477db0 con 0x55a6f951b720 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc7ebfff640 10 client.15657 handle_command_reply: tid=0 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 client.15657 shutdown 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 msgr2=0x55a6f9508790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9508790 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc02a930 tx=0x7fc7fc0317e0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 shutdown_connections 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f951b720 0x55a6f951db80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9509210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f9508cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 --2- 172.21.15.33:0/1557884341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9508790 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 >> 172.21.15.33:0/1557884341 conn(0x55a6f94fcd10 msgr2=0x55a6f94fbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc7ebfff640 0 client.15657 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 shutdown_connections 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 1 -- 172.21.15.33:0/1557884341 wait complete. 2024-09-10T21:05:40.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.469+0000 7fc8155c5740 20 client.15657 trim_cache size 0 max 16384 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 Processor -- start 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 -- start start 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94feee0 0x55a6f9506220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f94ff800 0x55a6f9523450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f950b790 0x55a6f9523990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b0620 con 0x55a6f950b790 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b2220 con 0x55a6f94ff800 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc8155c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b0cf0 con 0x55a6f94feee0 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc81178d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f950b790 0x55a6f9523990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc81178d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f950b790 0x55a6f9523990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60412/0 (socket says 172.21.15.33:60412) 2024-09-10T21:05:40.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.487+0000 7fc81178d640 1 -- 172.21.15.33:0/289190377 learned_addr learned my addr 172.21.15.33:0/289190377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc80bfff640 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f94ff800 0x55a6f9523450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc810f8c640 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94feee0 0x55a6f9506220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc81178d640 1 -- 172.21.15.33:0/289190377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94feee0 msgr2=0x55a6f9506220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc81178d640 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94feee0 0x55a6f9506220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc81178d640 1 -- 172.21.15.33:0/289190377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f94ff800 msgr2=0x55a6f9523450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc81178d640 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f94ff800 0x55a6f9523450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc81178d640 1 -- 172.21.15.33:0/289190377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6f94fcb40 con 0x55a6f950b790 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc810f8c640 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94feee0 0x55a6f9506220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc80bfff640 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f94ff800 0x55a6f9523450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:40.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc81178d640 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f950b790 0x55a6f9523990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc804012580 tx=0x7fc804011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1798ccfc0371c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc7f27fc640 1 -- 172.21.15.33:0/289190377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804034030 con 0x55a6f950b790 2024-09-10T21:05:40.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc7f27fc640 1 -- 172.21.15.33:0/289190377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80400abc0 con 0x55a6f950b790 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.488+0000 7fc7f27fc640 1 -- 172.21.15.33:0/289190377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc804004270 con 0x55a6f950b790 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 -- 172.21.15.33:0/289190377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f950b790 msgr2=0x55a6f9523990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f950b790 0x55a6f9523990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc804012580 tx=0x7fc804011000 comp rx=0 tx=0).stop 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 -- 172.21.15.33:0/289190377 shutdown_connections 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f950b790 0x55a6f9523990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f94ff800 0x55a6f9523450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 --2- 172.21.15.33:0/289190377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94feee0 0x55a6f9506220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 -- 172.21.15.33:0/289190377 >> 172.21.15.33:0/289190377 conn(0x55a6f94fcd10 msgr2=0x55a6f950a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 -- 172.21.15.33:0/289190377 shutdown_connections 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 -- 172.21.15.33:0/289190377 wait complete. 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.489+0000 7fc8155c5740 1 -- start start 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 1 Processor -- start 2024-09-10T21:05:40.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9513c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f95101f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9510730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b1a90 con 0x55a6f94feee0 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1b13c0 con 0x55a6f950b790 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1ae5e0 con 0x55a6f94ff800 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc8155c5740 10 client.0 _ll_register_callbacks cb 0x7fc7fc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc810f8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9513c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc810f8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9513c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60424/0 (socket says 172.21.15.33:60424) 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc810f8c640 1 -- 172.21.15.33:0/2503791867 learned_addr learned my addr 172.21.15.33:0/2503791867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc81178d640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9510730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.490+0000 7fc80bfff640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f95101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc810f8c640 1 -- 172.21.15.33:0/2503791867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 msgr2=0x55a6f95101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc810f8c640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f95101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc810f8c640 1 -- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 msgr2=0x55a6f9510730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc810f8c640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9510730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc810f8c640 1 -- 172.21.15.33:0/2503791867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6f94fdc00 con 0x55a6f94feee0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc81178d640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9510730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc810f8c640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9513c70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc044a50 tx=0x7fc7fc031a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964a588c7ccfe3d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc7ebfff640 1 -- 172.21.15.33:0/2503791867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc035070 con 0x55a6f94feee0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc7ebfff640 1 -- 172.21.15.33:0/2503791867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc7fc044630 con 0x55a6f94feee0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc80400abc0 con 0x55a6f94feee0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc7ebfff640 1 -- 172.21.15.33:0/2503791867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc000e60 con 0x55a6f94feee0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc8155c5740 10 client.15663 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a6f9589290 con 0x55a6f94feee0 2024-09-10T21:05:40.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.491+0000 7fc7ebfff640 1 -- 172.21.15.33:0/2503791867 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc80400abc0 con 0x55a6f94feee0 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc7ebfff640 1 -- 172.21.15.33:0/2503791867 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7fc7fc03b5a0 con 0x55a6f94feee0 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc8155c5740 10 client.15663 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc8155c5740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc8155c5740 20 client.15663 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc8155c5740 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f9589a70 0x55a6f958be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc8155c5740 4 client.15663 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a6f9473860 con 0x55a6f9589a70 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc80bfff640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f9589a70 0x55a6f958be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc7ebfff640 10 client.15663 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:40.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.492+0000 7fc80bfff640 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f9589a70 0x55a6f958be90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb86578893a4d08 server_cookie=83fe01872236d8f5 in_seq=0 out_seq=0 2024-09-10T21:05:40.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc7ebfff640 1 -- 172.21.15.33:0/2503791867 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a6f9473860 con 0x55a6f9589a70 2024-09-10T21:05:40.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc7ebfff640 10 client.15663 handle_command_reply: tid=0 2024-09-10T21:05:40.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 client.15663 shutdown 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 msgr2=0x55a6f9513c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9513c70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc044a50 tx=0x7fc7fc031a80 comp rx=0 tx=0).stop 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 shutdown_connections 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55a6f9589a70 0x55a6f958be90 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a6f950b790 0x55a6f9510730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a6f94ff800 0x55a6f95101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 --2- 172.21.15.33:0/2503791867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a6f94feee0 0x55a6f9513c70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc7ebfff640 0 client.15663 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.494+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 >> 172.21.15.33:0/2503791867 conn(0x55a6f94fcd10 msgr2=0x55a6f94fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 shutdown_connections 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8155c5740 1 -- 172.21.15.33:0/2503791867 wait complete. 2024-09-10T21:05:40.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8155c5740 20 client.15663 trim_cache size 0 max 16384 2024-09-10T21:05:40.503 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:05:40.503 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:05:40.503 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "238d99b0-b1dc-4b1f-9885-f601eee0e81e", 2024-09-10T21:05:40.503 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:05:40.503 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7f40b52e0 msgr2=0x7fc7f40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7f40b52e0 0x7fc7f40b77a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc80401f000 tx=0x7fc804020000 comp rx=0 tx=0).stop 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c1076a0 msgr2=0x7fc80c19ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c1076a0 0x7fc80c19ba50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc80000b4a0 tx=0x7fc80000b960 comp rx=0 tx=0).stop 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 shutdown_connections 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7f40b52e0 0x7fc7f40b77a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc80c108020 0x7fc80c1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc80c1076a0 0x7fc80c19ba50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 --2- 172.21.15.33:0/1273632364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc80c101c90 0x7fc80c19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.495+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 >> 172.21.15.33:0/1273632364 conn(0x7fc80c0fd700 msgr2=0x7fc80c06abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:40.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.496+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 shutdown_connections 2024-09-10T21:05:40.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:40.496+0000 7fc8131f0640 1 -- 172.21.15.33:0/1273632364 wait complete. 2024-09-10T21:05:40.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89050 12.16 MB/sec warmup 41 sec latency 3442.555 ms 2024-09-10T21:05:41.233 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:05:41.536 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:05:41.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: pgmap v397: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 7.7 MiB/s wr, 238 op/s 2024-09-10T21:05:41.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:41 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: pgmap v397: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 7.7 MiB/s wr, 238 op/s 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:05:41.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:41 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.657 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: pgmap v397: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 7.7 MiB/s wr, 238 op/s 2024-09-10T21:05:41.657 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:05:41.657 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.657 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:05:41.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:05:41.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:05:41.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:05:41.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:41 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:41.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89050 11.87 MB/sec warmup 42 sec latency 1007.695 ms 2024-09-10T21:05:42.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.022+0000 7f35c0f59640 1 -- 172.21.15.33:0/3450746115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 msgr2=0x7f35bc1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.022+0000 7f35c0f59640 1 --2- 172.21.15.33:0/3450746115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc1040c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009ca0 tx=0x7f35ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 -- 172.21.15.33:0/3450746115 shutdown_connections 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 --2- 172.21.15.33:0/3450746115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35bc10d140 0x7f35bc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 --2- 172.21.15.33:0/3450746115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35bc104600 0x7f35bc10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 --2- 172.21.15.33:0/3450746115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc1040c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 -- 172.21.15.33:0/3450746115 >> 172.21.15.33:0/3450746115 conn(0x7f35bc0fd700 msgr2=0x7f35bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 -- 172.21.15.33:0/3450746115 shutdown_connections 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 -- 172.21.15.33:0/3450746115 wait complete. 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 Processor -- start 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.023+0000 7f35c0f59640 1 -- start start 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35c0f59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35c0f59640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35bc104600 0x7f35bc1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35c0f59640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35bc10d140 0x7f35bc19ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35c0f59640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc112e10 con 0x7f35bc103ce0 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35c0f59640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc112c90 con 0x7f35bc10d140 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35c0f59640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc112f90 con 0x7f35bc104600 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60444/0 (socket says 172.21.15.33:60444) 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 -- 172.21.15.33:0/1082766095 learned_addr learned my addr 172.21.15.33:0/1082766095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35bc10d140 0x7f35bc19ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35bc104600 0x7f35bc1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 -- 172.21.15.33:0/1082766095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35bc104600 msgr2=0x7f35bc1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35bc104600 0x7f35bc1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 -- 172.21.15.33:0/1082766095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35bc10d140 msgr2=0x7f35bc19ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35bc10d140 0x7f35bc19ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.024+0000 7f35ba59c640 1 -- 172.21.15.33:0/1082766095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35bc19f410 con 0x7f35bc103ce0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.025+0000 7f35ba59c640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc199360 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f35ac02ade0 tx=0x7f35ac003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.025+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1082766095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac003d60 con 0x7f35bc103ce0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.025+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35bc19f6a0 con 0x7f35bc103ce0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.025+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1082766095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35ac0044f0 con 0x7f35bc103ce0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.025+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1082766095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35ac02fd10 con 0x7f35bc103ce0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.025+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35bc1a5e40 con 0x7f35bc103ce0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.027+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1082766095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f35ac004040 con 0x7f35bc103ce0 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.027+0000 7f35c332e740 1 Processor -- start 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35c332e740 1 -- start start 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35c332e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47480 0x564948c47860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35c332e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c57c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35c332e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35c332e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564948bbde50 con 0x564948c47480 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35c332e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564948bad280 con 0x564948c47da0 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35c332e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564948bb1090 con 0x564948c58140 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47480 0x564948c47860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47480 0x564948c47860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60448/0 (socket says 172.21.15.33:60448) 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 -- 172.21.15.33:0/4044253172 learned_addr learned my addr 172.21.15.33:0/4044253172 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35ba59c640 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c57c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 -- 172.21.15.33:0/4044253172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 msgr2=0x564948c5a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:42.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 -- 172.21.15.33:0/4044253172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 msgr2=0x564948c57c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c57c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35bad9d640 1 -- 172.21.15.33:0/4044253172 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56494890b650 con 0x564948c47480 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.028+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.029+0000 7f35ab7fe640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35a40b52e0 0x7f35a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.029+0000 7f35bad9d640 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47480 0x564948c47860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35b400abb0 tx=0x7f35b400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6500835a758c9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.029+0000 7f35ba59c640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35a40b52e0 0x7f35a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.029+0000 7f35a8ff9640 1 -- 172.21.15.33:0/4044253172 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b401d070 con 0x564948c47480 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.029+0000 7f35a8ff9640 1 -- 172.21.15.33:0/4044253172 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b4018af0 con 0x564948c47480 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.029+0000 7f35a8ff9640 1 -- 172.21.15.33:0/4044253172 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b4010a40 con 0x564948c47480 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.029+0000 7f35ab7fe640 1 -- 172.21.15.33:0/1082766095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f35ac107010 con 0x7f35bc103ce0 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 -- 172.21.15.33:0/4044253172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47480 msgr2=0x564948c47860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35ba59c640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35a40b52e0 0x7f35a40b77a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009430 tx=0x7f35ac10aa90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47480 0x564948c47860 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f35b400abb0 tx=0x7f35b400f790 comp rx=0 tx=0).stop 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 -- 172.21.15.33:0/4044253172 shutdown_connections 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c57c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 --2- 172.21.15.33:0/4044253172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47480 0x564948c47860 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 -- 172.21.15.33:0/4044253172 >> 172.21.15.33:0/4044253172 conn(0x564948c452b0 msgr2=0x564948c46450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 -- 172.21.15.33:0/4044253172 shutdown_connections 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 -- 172.21.15.33:0/4044253172 wait complete. 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 -- start start 2024-09-10T21:05:42.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.030+0000 7f35c332e740 1 Processor -- start 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47480 0x564948c57a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c55540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564948a0e700 con 0x564948c58140 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564948b48110 con 0x564948c47da0 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564948c5adf0 con 0x564948c47480 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 10 client.0 _ll_register_callbacks cb 0x564948b61990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c55540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c55540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60460/0 (socket says 172.21.15.33:60460) 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 -- 172.21.15.33:0/681969183 learned_addr learned my addr 172.21.15.33:0/681969183 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47480 0x564948c57a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 -- 172.21.15.33:0/681969183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47480 msgr2=0x564948c57a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47480 0x564948c57a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 -- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 msgr2=0x564948c54ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c54ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 -- 172.21.15.33:0/681969183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564948b75c40 con 0x564948c58140 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35bad9d640 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c55540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f35b400afd0 tx=0x7f35b4012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5af1f53a1b03b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35a1ffb640 1 -- 172.21.15.33:0/681969183 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b401d070 con 0x564948c58140 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b4018af0 con 0x564948c58140 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.031+0000 7f35a1ffb640 1 -- 172.21.15.33:0/681969183 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f35b4010a40 con 0x564948c58140 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35a1ffb640 1 -- 172.21.15.33:0/681969183 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35b4005c40 con 0x564948c58140 2024-09-10T21:05:42.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 10 client.15675 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564948c66730 con 0x564948c58140 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35a1ffb640 1 -- 172.21.15.33:0/681969183 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b4018af0 con 0x564948c58140 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35a1ffb640 1 -- 172.21.15.33:0/681969183 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f35b4028cd0 con 0x564948c58140 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 10 client.15675 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 20 client.15675 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948c67b90 0x564948c69f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 4 client.15675 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564948bbfda0 con 0x564948c67b90 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.032+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948c67b90 0x564948c69f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.033+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948c67b90 0x564948c69f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ce9fe9918ee915f server_cookie=7915c4eea412fe89 in_seq=0 out_seq=0 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.033+0000 7f35a1ffb640 10 client.15675 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.033+0000 7f35a1ffb640 1 -- 172.21.15.33:0/681969183 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564948bbfda0 con 0x564948c67b90 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.033+0000 7f35a1ffb640 10 client.15675 handle_command_reply: tid=0 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 client.15675 shutdown 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 msgr2=0x564948c55540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c55540 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f35b400afd0 tx=0x7f35b4012040 comp rx=0 tx=0).stop 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 shutdown_connections 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948c67b90 0x564948c69f80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c55540 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47da0 0x564948c54ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 --2- 172.21.15.33:0/681969183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47480 0x564948c57a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35a1ffb640 0 client.15675 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 >> 172.21.15.33:0/681969183 conn(0x564948c452b0 msgr2=0x564948c4ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:42.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 shutdown_connections 2024-09-10T21:05:42.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 1 -- 172.21.15.33:0/681969183 wait complete. 2024-09-10T21:05:42.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.034+0000 7f35c332e740 20 client.15675 trim_cache size 0 max 16384 2024-09-10T21:05:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 Processor -- start 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 -- start start 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47da0 0x564948c64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c6f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1ac2f0 con 0x564948c58140 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1adef0 con 0x564948c47480 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35c332e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1ac9c0 con 0x564948c47da0 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c6f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c6f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60476/0 (socket says 172.21.15.33:60476) 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 -- 172.21.15.33:0/1485511843 learned_addr learned my addr 172.21.15.33:0/1485511843 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 -- 172.21.15.33:0/1485511843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47da0 msgr2=0x564948c64080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47da0 0x564948c64080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 -- 172.21.15.33:0/1485511843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 msgr2=0x564948c44bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c44bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.052+0000 7f35bad9d640 1 -- 172.21.15.33:0/1485511843 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564948c47bd0 con 0x564948c58140 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c6f6a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f35b401b040 tx=0x7f35b400a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f3af24f910a1159 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35a3fff640 1 -- 172.21.15.33:0/1485511843 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b401d040 con 0x564948c58140 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35a3fff640 1 -- 172.21.15.33:0/1485511843 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b402d070 con 0x564948c58140 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35a3fff640 1 -- 172.21.15.33:0/1485511843 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b40106e0 con 0x564948c58140 2024-09-10T21:05:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35c332e740 1 -- 172.21.15.33:0/1485511843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 msgr2=0x564948c6f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35c332e740 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c6f6a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f35b401b040 tx=0x7f35b400a5a0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35c332e740 1 -- 172.21.15.33:0/1485511843 shutdown_connections 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35c332e740 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c58140 0x564948c6f6a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35c332e740 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c47da0 0x564948c64080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35c332e740 1 --2- 172.21.15.33:0/1485511843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c44bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.053+0000 7f35c332e740 1 -- 172.21.15.33:0/1485511843 >> 172.21.15.33:0/1485511843 conn(0x564948c452b0 msgr2=0x564948c56d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 -- 172.21.15.33:0/1485511843 shutdown_connections 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 -- 172.21.15.33:0/1485511843 wait complete. 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 -- start start 2024-09-10T21:05:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 Processor -- start 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c600e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47da0 0x564948c60620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1ad760 con 0x564948c47da0 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1ad090 con 0x564948c47480 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35bc1aa2b0 con 0x564948c58140 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35c332e740 10 client.0 _ll_register_callbacks cb 0x7f35b401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35b9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47da0 0x564948c60620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35b9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47da0 0x564948c60620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60484/0 (socket says 172.21.15.33:60484) 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35b9d9b640 1 -- 172.21.15.33:0/1342773161 learned_addr learned my addr 172.21.15.33:0/1342773161 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:05:42.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.054+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35b9d9b640 1 -- 172.21.15.33:0/1342773161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 msgr2=0x564948c5d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35b9d9b640 1 -- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 msgr2=0x564948c600e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c600e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35b9d9b640 1 -- 172.21.15.33:0/1342773161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564948c60e50 con 0x564948c47da0 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35b9d9b640 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47da0 0x564948c60620 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x564948c6a640 tx=0x7f35b0017810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951522b4e8e10bc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35a1ffb640 1 -- 172.21.15.33:0/1342773161 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b000f040 con 0x564948c47da0 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35a1ffb640 1 -- 172.21.15.33:0/1342773161 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f35b0026420 con 0x564948c47da0 2024-09-10T21:05:42.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35b402d070 con 0x564948c47da0 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35a1ffb640 1 -- 172.21.15.33:0/1342773161 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f35b0026620 con 0x564948c47da0 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35c332e740 10 client.15681 fetch_fsmap learned FSMap version 30 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.055+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564948cd4e00 con 0x564948c47da0 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35a1ffb640 1 -- 172.21.15.33:0/1342773161 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35b402d070 con 0x564948c47da0 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35a1ffb640 1 -- 172.21.15.33:0/1342773161 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 30) ==== 8378+0+0 (secure 0 0 0) 0x7f35b0025670 con 0x564948c47da0 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35c332e740 10 client.15681 fetch_fsmap finished waiting for FSMap version 30 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35c332e740 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:05:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35c332e740 20 client.15681 populate_metadata read hostname 'smithi033' 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35c332e740 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948cd56c0 0x564948cd7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35c332e740 4 client.15681 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564948bbb850 con 0x564948cd56c0 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948cd56c0 0x564948cd7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35a1ffb640 10 client.15681 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.056+0000 7f35bad9d640 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948cd56c0 0x564948cd7ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=291903cbae2ce8c7 server_cookie=6592bdadf37c8ebb in_seq=0 out_seq=0 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35a1ffb640 1 -- 172.21.15.33:0/1342773161 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x564948bbb850 con 0x564948cd56c0 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35a1ffb640 10 client.15681 handle_command_reply: tid=0 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 client.15681 shutdown 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47da0 msgr2=0x564948c60620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47da0 0x564948c60620 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x564948c6a640 tx=0x7f35b0017810 comp rx=0 tx=0).stop 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 shutdown_connections 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564948cd56c0 0x564948cd7ae0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35a1ffb640 0 client.15681 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564948c58140 0x564948c5d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564948c47da0 0x564948c60620 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 --2- 172.21.15.33:0/1342773161 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564948c47480 0x564948c600e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 >> 172.21.15.33:0/1342773161 conn(0x564948c452b0 msgr2=0x564948c45f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 shutdown_connections 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 1 -- 172.21.15.33:0/1342773161 wait complete. 2024-09-10T21:05:42.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c332e740 20 client.15681 trim_cache size 0 max 16384 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "238d99b0-b1dc-4b1f-9885-f601eee0e81e", 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:05:42.069 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:05:42.070 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:05:42.071 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:05:42.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35a40b52e0 msgr2=0x7f35a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c0f59640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35a40b52e0 0x7f35a40b77a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009430 tx=0x7f35ac10aa90 comp rx=0 tx=0).stop 2024-09-10T21:05:42.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 msgr2=0x7f35bc199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:05:42.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.057+0000 7f35c0f59640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc199360 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f35ac02ade0 tx=0x7f35ac003970 comp rx=0 tx=0).stop 2024-09-10T21:05:42.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 shutdown_connections 2024-09-10T21:05:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f35a40b52e0 0x7f35a40b77a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f35bc10d140 0x7f35bc19ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35bc104600 0x7f35bc1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 --2- 172.21.15.33:0/1082766095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35bc103ce0 0x7f35bc199360 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:05:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 >> 172.21.15.33:0/1082766095 conn(0x7f35bc0fd700 msgr2=0x7f35bc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:05:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 shutdown_connections 2024-09-10T21:05:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:05:42.058+0000 7f35c0f59640 1 -- 172.21.15.33:0/1082766095 wait complete. 2024-09-10T21:05:42.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:42 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:05:42.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:42 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:42.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:42 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:05:42.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:42 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:42.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:42 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:05:42.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:42 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:42.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:42 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:05:42.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:42 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:42.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89110 11.61 MB/sec warmup 43 sec latency 1401.885 ms 2024-09-10T21:05:42.810 INFO:tasks.cephfs.filesystem:scrub status for tag:238d99b0-b1dc-4b1f-9885-f601eee0e81e - {'path': '/', 'tag': '238d99b0-b1dc-4b1f-9885-f601eee0e81e', 'options': 'recursive,force'} 2024-09-10T21:05:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:42 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:05:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:42 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:42 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:05:42.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:42 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:05:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:43 smithi174 ceph-mon[35635]: pgmap v398: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 7.7 MiB/s wr, 232 op/s 2024-09-10T21:05:43.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:43 smithi033 ceph-mon[30860]: pgmap v398: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 7.7 MiB/s wr, 232 op/s 2024-09-10T21:05:43.698 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89110 11.34 MB/sec warmup 44 sec latency 1538.195 ms 2024-09-10T21:05:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:43 smithi104 ceph-mon[35325]: pgmap v398: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 7.7 MiB/s wr, 232 op/s 2024-09-10T21:05:44.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89110 11.09 MB/sec warmup 45 sec latency 2538.293 ms 2024-09-10T21:05:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:45 smithi174 ceph-mon[35635]: pgmap v399: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 7.0 MiB/s wr, 234 op/s 2024-09-10T21:05:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:45 smithi033 ceph-mon[30860]: pgmap v399: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 7.0 MiB/s wr, 234 op/s 2024-09-10T21:05:45.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89110 10.85 MB/sec warmup 46 sec latency 3538.407 ms 2024-09-10T21:05:45.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:45 smithi104 ceph-mon[35325]: pgmap v399: 97 pgs: 97 active+clean; 51 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 7.0 MiB/s wr, 234 op/s 2024-09-10T21:05:46.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:05:46.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:05:46.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:05:46.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:05:46.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 89110 10.62 MB/sec warmup 47 sec latency 4538.516 ms 2024-09-10T21:05:46.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:05:46.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:05:47.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:47 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:05:47.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:47 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:05:47.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:47 smithi174 ceph-mon[35635]: pgmap v400: 97 pgs: 97 active+clean; 51 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-09-10T21:05:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:47 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:05:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:47 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:05:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:47 smithi033 ceph-mon[30860]: pgmap v400: 97 pgs: 97 active+clean; 51 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-09-10T21:05:47.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 90285 10.50 MB/sec warmup 48 sec latency 4937.227 ms 2024-09-10T21:05:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:47 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:05:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:47 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:05:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:47 smithi104 ceph-mon[35325]: pgmap v400: 97 pgs: 97 active+clean; 51 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 2.0 MiB/s wr, 93 op/s 2024-09-10T21:05:48.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 92690 10.57 MB/sec warmup 49 sec latency 9.771 ms 2024-09-10T21:05:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:49 smithi174 ceph-mon[35635]: pgmap v401: 97 pgs: 97 active+clean; 51 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 134 KiB/s wr, 37 op/s 2024-09-10T21:05:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:49 smithi033 ceph-mon[30860]: pgmap v401: 97 pgs: 97 active+clean; 51 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 134 KiB/s wr, 37 op/s 2024-09-10T21:05:49.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 95236 10.61 MB/sec warmup 50 sec latency 10.060 ms 2024-09-10T21:05:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:49 smithi104 ceph-mon[35325]: pgmap v401: 97 pgs: 97 active+clean; 51 MiB data, 501 MiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 134 KiB/s wr, 37 op/s 2024-09-10T21:05:50.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 96952 10.58 MB/sec warmup 51 sec latency 10.052 ms 2024-09-10T21:05:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:51 smithi174 ceph-mon[35635]: pgmap v402: 97 pgs: 97 active+clean; 52 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 206 KiB/s wr, 41 op/s 2024-09-10T21:05:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:51 smithi033 ceph-mon[30860]: pgmap v402: 97 pgs: 97 active+clean; 52 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 206 KiB/s wr, 41 op/s 2024-09-10T21:05:51.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 99597 10.69 MB/sec warmup 52 sec latency 67.617 ms 2024-09-10T21:05:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:51 smithi104 ceph-mon[35325]: pgmap v402: 97 pgs: 97 active+clean; 52 MiB data, 502 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 206 KiB/s wr, 41 op/s 2024-09-10T21:05:52.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 101750 10.63 MB/sec warmup 53 sec latency 10.568 ms 2024-09-10T21:05:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:53 smithi174 ceph-mon[35635]: pgmap v403: 97 pgs: 97 active+clean; 53 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 265 KiB/s wr, 46 op/s 2024-09-10T21:05:53.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:53 smithi033 ceph-mon[30860]: pgmap v403: 97 pgs: 97 active+clean; 53 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 265 KiB/s wr, 46 op/s 2024-09-10T21:05:53.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 103502 10.66 MB/sec warmup 54 sec latency 10.063 ms 2024-09-10T21:05:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:53 smithi104 ceph-mon[35325]: pgmap v403: 97 pgs: 97 active+clean; 53 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 265 KiB/s wr, 46 op/s 2024-09-10T21:05:54.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 106417 10.78 MB/sec warmup 55 sec latency 14.389 ms 2024-09-10T21:05:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:55 smithi174 ceph-mon[35635]: pgmap v404: 97 pgs: 97 active+clean; 56 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 730 KiB/s wr, 153 op/s 2024-09-10T21:05:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:55 smithi033 ceph-mon[30860]: pgmap v404: 97 pgs: 97 active+clean; 56 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 730 KiB/s wr, 153 op/s 2024-09-10T21:05:55.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 108118 10.71 MB/sec warmup 56 sec latency 9.649 ms 2024-09-10T21:05:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:55 smithi104 ceph-mon[35325]: pgmap v404: 97 pgs: 97 active+clean; 56 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 730 KiB/s wr, 153 op/s 2024-09-10T21:05:56.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 110435 10.77 MB/sec warmup 57 sec latency 10.870 ms 2024-09-10T21:05:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:57 smithi174 ceph-mon[35635]: pgmap v405: 97 pgs: 97 active+clean; 56 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 727 KiB/s wr, 148 op/s 2024-09-10T21:05:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:57 smithi033 ceph-mon[30860]: pgmap v405: 97 pgs: 97 active+clean; 56 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 727 KiB/s wr, 148 op/s 2024-09-10T21:05:57.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 112872 10.78 MB/sec warmup 58 sec latency 73.466 ms 2024-09-10T21:05:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:57 smithi104 ceph-mon[35325]: pgmap v405: 97 pgs: 97 active+clean; 56 MiB data, 535 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 727 KiB/s wr, 148 op/s 2024-09-10T21:05:58.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 114593 10.76 MB/sec warmup 59 sec latency 9.553 ms 2024-09-10T21:05:59.613 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:05:59 smithi104 ceph-mon[35325]: pgmap v406: 97 pgs: 97 active+clean; 59 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-09-10T21:05:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:05:59 smithi174 ceph-mon[35635]: pgmap v406: 97 pgs: 97 active+clean; 59 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-09-10T21:05:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:05:59 smithi033 ceph-mon[30860]: pgmap v406: 97 pgs: 97 active+clean; 59 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 MiB/s wr, 220 op/s 2024-09-10T21:05:59.699 INFO:tasks.workunit.client.0.smithi033.stdout: 1 117367 10.86 MB/sec warmup 60 sec latency 9.876 ms 2024-09-10T21:06:00.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 119532 10.81 MB/sec warmup 61 sec latency 11.309 ms 2024-09-10T21:06:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:01 smithi174 ceph-mon[35635]: pgmap v407: 97 pgs: 97 active+clean; 61 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 252 op/s 2024-09-10T21:06:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:01 smithi033 ceph-mon[30860]: pgmap v407: 97 pgs: 97 active+clean; 61 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 252 op/s 2024-09-10T21:06:01.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 121300 10.83 MB/sec warmup 62 sec latency 8.984 ms 2024-09-10T21:06:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:01 smithi104 ceph-mon[35325]: pgmap v407: 97 pgs: 97 active+clean; 61 MiB data, 572 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 252 op/s 2024-09-10T21:06:02.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 124175 10.93 MB/sec warmup 63 sec latency 11.439 ms 2024-09-10T21:06:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:03 smithi174 ceph-mon[35635]: pgmap v408: 97 pgs: 97 active+clean; 61 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-10T21:06:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:03 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:03 smithi033 ceph-mon[30860]: pgmap v408: 97 pgs: 97 active+clean; 61 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-10T21:06:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:03 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:03.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 125890 10.87 MB/sec warmup 64 sec latency 11.835 ms 2024-09-10T21:06:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:03 smithi104 ceph-mon[35325]: pgmap v408: 97 pgs: 97 active+clean; 61 MiB data, 578 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 242 op/s 2024-09-10T21:06:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:03 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:04.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 128193 10.91 MB/sec warmup 65 sec latency 9.401 ms 2024-09-10T21:06:05.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:05 smithi174 ceph-mon[35635]: pgmap v409: 97 pgs: 97 active+clean; 66 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 343 op/s 2024-09-10T21:06:05.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:05 smithi033 ceph-mon[30860]: pgmap v409: 97 pgs: 97 active+clean; 66 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 343 op/s 2024-09-10T21:06:05.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 130915 10.95 MB/sec warmup 66 sec latency 9.282 ms 2024-09-10T21:06:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:05 smithi104 ceph-mon[35325]: pgmap v409: 97 pgs: 97 active+clean; 66 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 343 op/s 2024-09-10T21:06:06.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 132509 10.91 MB/sec warmup 67 sec latency 12.257 ms 2024-09-10T21:06:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:07 smithi174 ceph-mon[35635]: pgmap v410: 97 pgs: 97 active+clean; 67 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-09-10T21:06:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:07 smithi033 ceph-mon[30860]: pgmap v410: 97 pgs: 97 active+clean; 67 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-09-10T21:06:07.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 135189 10.99 MB/sec warmup 68 sec latency 11.194 ms 2024-09-10T21:06:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:07 smithi104 ceph-mon[35325]: pgmap v410: 97 pgs: 97 active+clean; 67 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 270 op/s 2024-09-10T21:06:08.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:08 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:08.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:08 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:08.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 137428 10.95 MB/sec warmup 69 sec latency 10.919 ms 2024-09-10T21:06:08.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:08 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:09 smithi174 ceph-mon[35635]: pgmap v411: 97 pgs: 97 active+clean; 69 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 303 op/s 2024-09-10T21:06:09.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:09 smithi033 ceph-mon[30860]: pgmap v411: 97 pgs: 97 active+clean; 69 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 303 op/s 2024-09-10T21:06:09.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 139235 10.96 MB/sec warmup 70 sec latency 10.002 ms 2024-09-10T21:06:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:09 smithi104 ceph-mon[35325]: pgmap v411: 97 pgs: 97 active+clean; 69 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.5 MiB/s wr, 303 op/s 2024-09-10T21:06:10.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 142139 11.06 MB/sec warmup 71 sec latency 10.703 ms 2024-09-10T21:06:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:11 smithi174 ceph-mon[35635]: pgmap v412: 97 pgs: 97 active+clean; 71 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-09-10T21:06:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:11 smithi033 ceph-mon[30860]: pgmap v412: 97 pgs: 97 active+clean; 71 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-09-10T21:06:11.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 144249 11.01 MB/sec warmup 72 sec latency 11.023 ms 2024-09-10T21:06:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:11 smithi104 ceph-mon[35325]: pgmap v412: 97 pgs: 97 active+clean; 71 MiB data, 613 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-09-10T21:06:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:12 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:06:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:12 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:06:12.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 145988 11.03 MB/sec warmup 73 sec latency 9.019 ms 2024-09-10T21:06:12.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:06:12.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:12 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:06:13.102 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:13.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:13 smithi033 ceph-mon[30860]: pgmap v413: 97 pgs: 97 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-10T21:06:13.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:06:13.552 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:06:13.553 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:06:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:13 smithi174 ceph-mon[35635]: pgmap v413: 97 pgs: 97 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-10T21:06:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:06:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:06:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:06:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 -- 172.21.15.33:0/2532789689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94901087d0 msgr2=0x7f9490108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 --2- 172.21.15.33:0/2532789689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94901087d0 0x7f9490108c10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9484009900 tx=0x7f948402a410 comp rx=0 tx=0).stop 2024-09-10T21:06:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 -- 172.21.15.33:0/2532789689 shutdown_connections 2024-09-10T21:06:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 --2- 172.21.15.33:0/2532789689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9490109150 0x7f949010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 --2- 172.21.15.33:0/2532789689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94901087d0 0x7f9490108c10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 --2- 172.21.15.33:0/2532789689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9490102dc0 0x7f94901031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 -- 172.21.15.33:0/2532789689 >> 172.21.15.33:0/2532789689 conn(0x7f94900fe830 msgr2=0x7f9490100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 -- 172.21.15.33:0/2532789689 shutdown_connections 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 -- 172.21.15.33:0/2532789689 wait complete. 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.636+0000 7f94953dd640 1 Processor -- start 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f94953dd640 1 -- start start 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f94953dd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9490102dc0 0x7f94900754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f94953dd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94901087d0 0x7f94900759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f94953dd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490109150 0x7f9490075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f94953dd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9490112dd0 con 0x7f9490109150 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f94953dd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9490112c50 con 0x7f9490102dc0 2024-09-10T21:06:13.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f94953dd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9490112f50 con 0x7f94901087d0 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490109150 0x7f9490075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9490102dc0 0x7f94900754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94901087d0 0x7f94900759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94901087d0 0x7f94900759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:50828/0 (socket says 172.21.15.33:50828) 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9490102dc0 0x7f94900754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59054/0 (socket says 172.21.15.33:59054) 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490109150 0x7f9490075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41636/0 (socket says 172.21.15.33:41636) 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948e7fc640 1 -- 172.21.15.33:0/3597084882 learned_addr learned my addr 172.21.15.33:0/3597084882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 -- 172.21.15.33:0/3597084882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94901087d0 msgr2=0x7f94900759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94901087d0 0x7f94900759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 -- 172.21.15.33:0/3597084882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9490102dc0 msgr2=0x7f94900754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9490102dc0 0x7f94900754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 -- 172.21.15.33:0/3597084882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94901a7740 con 0x7f9490109150 2024-09-10T21:06:13.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.637+0000 7f948f7fe640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490109150 0x7f9490075f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9488009fd0 tx=0x7f948800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.638+0000 7f9477fff640 1 -- 172.21.15.33:0/3597084882 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948800ce20 con 0x7f9490109150 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.638+0000 7f9477fff640 1 -- 172.21.15.33:0/3597084882 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9488005020 con 0x7f9490109150 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.638+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94901a7a30 con 0x7f9490109150 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.638+0000 7f9477fff640 1 -- 172.21.15.33:0/3597084882 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9488010690 con 0x7f9490109150 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.638+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94901a7f40 con 0x7f9490109150 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 Processor -- start 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f9477fff640 1 -- 172.21.15.33:0/3597084882 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f94880054d0 con 0x7f9490109150 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 -- start start 2024-09-10T21:06:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d12480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9c88e50 con 0x55d3e9d129c0 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9c78280 con 0x55d3e9d120a0 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f94977b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9c7c090 con 0x55d3e9d1e950 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f948effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d12480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d12480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59066/0 (socket says 172.21.15.33:59066) 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f948e7fc640 1 -- 172.21.15.33:0/3251896327 learned_addr learned my addr 172.21.15.33:0/3251896327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.641+0000 7f948e7fc640 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f948effd640 1 -- 172.21.15.33:0/3251896327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 msgr2=0x55d3e9d20d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f948effd640 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f948effd640 1 -- 172.21.15.33:0/3251896327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 msgr2=0x55d3e9d12480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f948effd640 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d12480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f948effd640 1 -- 172.21.15.33:0/3251896327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e99d6650 con 0x55d3e9d129c0 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f9477fff640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94780b52e0 0x7f94780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f948effd640 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1e410 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9480009c20 tx=0x7f9480011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2064f0118406fe16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f948e7fc640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94780b52e0 0x7f94780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f9477fff640 1 -- 172.21.15.33:0/3597084882 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f94880e61f0 con 0x7f9490109150 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f94757fa640 1 -- 172.21.15.33:0/3251896327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480004050 con 0x55d3e9d129c0 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f94757fa640 1 -- 172.21.15.33:0/3251896327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94800054d0 con 0x55d3e9d129c0 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.642+0000 7f94757fa640 1 -- 172.21.15.33:0/3251896327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480004d90 con 0x55d3e9d129c0 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f948e7fc640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94780b52e0 0x7f94780b77a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9484005ec0 tx=0x7f9484005e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 -- 172.21.15.33:0/3251896327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 msgr2=0x55d3e9d1e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1e410 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9480009c20 tx=0x7f9480011810 comp rx=0 tx=0).stop 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 -- 172.21.15.33:0/3251896327 shutdown_connections 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1e410 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 --2- 172.21.15.33:0/3251896327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d12480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 -- 172.21.15.33:0/3251896327 >> 172.21.15.33:0/3251896327 conn(0x55d3e9d0fed0 msgr2=0x55d3e9d11070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 -- 172.21.15.33:0/3251896327 shutdown_connections 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 -- 172.21.15.33:0/3251896327 wait complete. 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.643+0000 7f94977b2740 1 -- start start 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 1 Processor -- start 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d1e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d1bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9ad9700 con 0x55d3e9d129c0 2024-09-10T21:06:13.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9c13110 con 0x55d3e9d120a0 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d3e9d215d0 con 0x55d3e9d1e950 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f94977b2740 10 client.0 _ll_register_callbacks cb 0x55d3e9c2c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41646/0 (socket says 172.21.15.33:41646) 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 -- 172.21.15.33:0/1376245312 learned_addr learned my addr 172.21.15.33:0/1376245312 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d1e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948f7fe640 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d1bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 -- 172.21.15.33:0/1376245312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 msgr2=0x55d3e9d1bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d1bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 -- 172.21.15.33:0/1376245312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 msgr2=0x55d3e9d1e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d1e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 -- 172.21.15.33:0/1376245312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e9c40c40 con 0x55d3e9d129c0 2024-09-10T21:06:13.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f948e7fc640 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1b6c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9484009230 tx=0x7f948402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756db23013e5d292 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.644+0000 7f9472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f9472ff5640 1 -- 172.21.15.33:0/1376245312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9484035040 con 0x55d3e9d129c0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f9472ff5640 1 -- 172.21.15.33:0/1376245312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9484004050 con 0x55d3e9d129c0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94800054d0 con 0x55d3e9d129c0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f9472ff5640 1 -- 172.21.15.33:0/1376245312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9484004250 con 0x55d3e9d129c0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 10 client.15693 fetch_fsmap learned FSMap version 32 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3e9d2cf30 con 0x55d3e9d129c0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f9472ff5640 1 -- 172.21.15.33:0/1376245312 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94800054d0 con 0x55d3e9d129c0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f9472ff5640 1 -- 172.21.15.33:0/1376245312 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9484003710 con 0x55d3e9d129c0 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 10 client.15693 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 20 client.15693 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:13.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d2e490 0x55d3e9d30920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 4 client.15693 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3e9c8ada0 con 0x55d3e9d2e490 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.645+0000 7f948effd640 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d2e490 0x55d3e9d30920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.646+0000 7f9472ff5640 10 client.15693 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.646+0000 7f948effd640 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d2e490 0x55d3e9d30920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5bdd9256f026113 server_cookie=6c0afd58dad5ccb in_seq=0 out_seq=0 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f9472ff5640 1 -- 172.21.15.33:0/1376245312 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3e9c8ada0 con 0x55d3e9d2e490 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f9472ff5640 10 client.15693 handle_command_reply: tid=0 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 client.15693 shutdown 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 msgr2=0x55d3e9d1b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1b6c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9484009230 tx=0x7f948402a410 comp rx=0 tx=0).stop 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 shutdown_connections 2024-09-10T21:06:13.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d2e490 0x55d3e9d30920 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d1e950 0x55d3e9d1bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1b6c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 --2- 172.21.15.33:0/1376245312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d120a0 0x55d3e9d1e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f9472ff5640 0 client.15693 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 >> 172.21.15.33:0/1376245312 conn(0x55d3e9d0fed0 msgr2=0x55d3e9d15a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 shutdown_connections 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 1 -- 172.21.15.33:0/1376245312 wait complete. 2024-09-10T21:06:13.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.647+0000 7f94977b2740 20 client.15693 trim_cache size 0 max 16384 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 Processor -- start 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 -- start start 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d2a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94901b0670 con 0x55d3e9d1e950 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94901b22d0 con 0x55d3e9d129c0 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f94977b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94901b0d40 con 0x55d3e9d120a0 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f948effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d2a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f948effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d2a800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:50846/0 (socket says 172.21.15.33:50846) 2024-09-10T21:06:13.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f948f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41662/0 (socket says 172.21.15.33:41662) 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f948effd640 1 -- 172.21.15.33:0/595684455 learned_addr learned my addr 172.21.15.33:0/595684455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.666+0000 7f948e7fc640 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f948effd640 1 -- 172.21.15.33:0/595684455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 msgr2=0x55d3e9d1fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f948effd640 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f948effd640 1 -- 172.21.15.33:0/595684455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 msgr2=0x55d3e9d20280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f948effd640 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f948effd640 1 -- 172.21.15.33:0/595684455 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3e9d30fa0 con 0x55d3e9d120a0 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f948f7fe640 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f948effd640 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d2a800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9480009c20 tx=0x7f9480005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=530bd3ee8edfaa39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f9474ff9640 1 -- 172.21.15.33:0/595684455 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94800097a0 con 0x55d3e9d120a0 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f9474ff9640 1 -- 172.21.15.33:0/595684455 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9480029070 con 0x55d3e9d120a0 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.667+0000 7f9474ff9640 1 -- 172.21.15.33:0/595684455 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480003ca0 con 0x55d3e9d120a0 2024-09-10T21:06:13.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 -- 172.21.15.33:0/595684455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 msgr2=0x55d3e9d2a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d2a800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9480009c20 tx=0x7f9480005f90 comp rx=0 tx=0).stop 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 -- 172.21.15.33:0/595684455 shutdown_connections 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d20280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d1fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 --2- 172.21.15.33:0/595684455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d2a800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 -- 172.21.15.33:0/595684455 >> 172.21.15.33:0/595684455 conn(0x55d3e9d0fed0 msgr2=0x55d3e9d19f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 -- 172.21.15.33:0/595684455 shutdown_connections 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 -- 172.21.15.33:0/595684455 wait complete. 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 -- start start 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.668+0000 7f94977b2740 1 Processor -- start 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f94977b2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d26d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f94977b2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d18930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f94977b2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d18e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f94977b2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94901b1b40 con 0x55d3e9d1e950 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f94977b2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94901b1410 con 0x55d3e9d129c0 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f94977b2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94901ae630 con 0x55d3e9d120a0 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f94977b2740 10 client.0 _ll_register_callbacks cb 0x7f9484032e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d26d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d18930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d26d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:50862/0 (socket says 172.21.15.33:50862) 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 -- 172.21.15.33:0/2224013810 learned_addr learned my addr 172.21.15.33:0/2224013810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948f7fe640 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d18e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 -- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 msgr2=0x55d3e9d18930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d18930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 -- 172.21.15.33:0/2224013810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 msgr2=0x55d3e9d18e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d18e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 -- 172.21.15.33:0/2224013810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3e9d19400 con 0x55d3e9d120a0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948e7fc640 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d18930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f948effd640 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d26d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9480009730 tx=0x7f94800097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b2c9228a9e8d53 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.669+0000 7f9472ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f9472ff5640 1 -- 172.21.15.33:0/2224013810 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480039060 con 0x55d3e9d120a0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f9472ff5640 1 -- 172.21.15.33:0/2224013810 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9480003a20 con 0x55d3e9d120a0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9480029070 con 0x55d3e9d120a0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f9472ff5640 1 -- 172.21.15.33:0/2224013810 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9480003800 con 0x55d3e9d120a0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 10 client.25264 fetch_fsmap learned FSMap version 32 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d3e9d18710 con 0x55d3e9d120a0 2024-09-10T21:06:13.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f9472ff5640 1 -- 172.21.15.33:0/2224013810 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9480029070 con 0x55d3e9d120a0 2024-09-10T21:06:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f9472ff5640 1 -- 172.21.15.33:0/2224013810 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f9480018070 con 0x55d3e9d120a0 2024-09-10T21:06:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 10 client.25264 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T21:06:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 10 client.25264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 20 client.25264 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d9c7a0 0x55d3e9d9ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 4 client.25264 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:06:13.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.670+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3e9c86850 con 0x55d3e9d9c7a0 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f948e7fc640 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d9c7a0 0x55d3e9d9ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f9472ff5640 10 client.25264 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f948e7fc640 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d9c7a0 0x55d3e9d9ebc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c21360d8042d67f server_cookie=a68d08db504f5855 in_seq=0 out_seq=0 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f9472ff5640 1 -- 172.21.15.33:0/2224013810 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x55d3e9c86850 con 0x55d3e9d9c7a0 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f9472ff5640 10 client.25264 handle_command_reply: tid=0 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 client.25264 shutdown 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 msgr2=0x55d3e9d26d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d26d70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9480009730 tx=0x7f94800097a0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 shutdown_connections 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d3e9d9c7a0 0x55d3e9d9ebc0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3e9d1e950 0x55d3e9d18e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3e9d129c0 0x55d3e9d18930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f9472ff5640 0 client.25264 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 --2- 172.21.15.33:0/2224013810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3e9d120a0 0x55d3e9d26d70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 >> 172.21.15.33:0/2224013810 conn(0x55d3e9d0fed0 msgr2=0x55d3e9d0e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.671+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 shutdown_connections 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94977b2740 1 -- 172.21.15.33:0/2224013810 wait complete. 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94977b2740 20 client.25264 trim_cache size 0 max 16384 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:06:13.682 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:06:13.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:06:13.684 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:06:13.685 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:06:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94780b52e0 msgr2=0x7f94780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94780b52e0 0x7f94780b77a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9484005ec0 tx=0x7f9484005e50 comp rx=0 tx=0).stop 2024-09-10T21:06:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490109150 msgr2=0x7f9490075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490109150 0x7f9490075f20 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9488009fd0 tx=0x7f948800ef10 comp rx=0 tx=0).stop 2024-09-10T21:06:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 shutdown_connections 2024-09-10T21:06:13.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94780b52e0 0x7f94780b77a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490109150 0x7f9490075f20 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94901087d0 0x7f94900759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 --2- 172.21.15.33:0/3597084882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9490102dc0 0x7f94900754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 >> 172.21.15.33:0/3597084882 conn(0x7f94900fe830 msgr2=0x7f9490100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 shutdown_connections 2024-09-10T21:06:13.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:13.672+0000 7f94953dd640 1 -- 172.21.15.33:0/3597084882 wait complete. 2024-09-10T21:06:13.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 148633 11.06 MB/sec warmup 74 sec latency 13.148 ms 2024-09-10T21:06:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:13 smithi104 ceph-mon[35325]: pgmap v413: 97 pgs: 97 active+clean; 73 MiB data, 643 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.3 MiB/s wr, 278 op/s 2024-09-10T21:06:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:06:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:06:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:06:14.695 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:06:14.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:06:14.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 150251 11.02 MB/sec warmup 75 sec latency 9.769 ms 2024-09-10T21:06:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.436+0000 7f267b89d640 1 -- 172.21.15.33:0/3305336787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26741076a0 msgr2=0x7f2674107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:15.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.436+0000 7f267b89d640 1 --2- 172.21.15.33:0/3305336787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26741076a0 0x7f2674107ae0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2668009900 tx=0x7f266802a410 comp rx=0 tx=0).stop 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 -- 172.21.15.33:0/3305336787 shutdown_connections 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 --2- 172.21.15.33:0/3305336787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2674108020 0x7f267410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 --2- 172.21.15.33:0/3305336787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26741076a0 0x7f2674107ae0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 --2- 172.21.15.33:0/3305336787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 0x7f2674102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 -- 172.21.15.33:0/3305336787 >> 172.21.15.33:0/3305336787 conn(0x7f26740fd700 msgr2=0x7f26740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 -- 172.21.15.33:0/3305336787 shutdown_connections 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 -- 172.21.15.33:0/3305336787 wait complete. 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 Processor -- start 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.437+0000 7f267b89d640 1 -- start start 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f267b89d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 0x7f26741992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f267b89d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26741076a0 0x7f2674199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:15.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f267b89d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674108020 0x7f267419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f267b89d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2674112dc0 con 0x7f2674108020 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f267b89d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2674112c40 con 0x7f26741076a0 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f267b89d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2674112f40 con 0x7f2674101c90 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 0x7f26741992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679e3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674108020 0x7f267419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2678e38640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26741076a0 0x7f2674199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 0x7f26741992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:50868/0 (socket says 172.21.15.33:50868) 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2678e38640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26741076a0 0x7f2674199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59120/0 (socket says 172.21.15.33:59120) 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679e3a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674108020 0x7f267419ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41692/0 (socket says 172.21.15.33:41692) 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 -- 172.21.15.33:0/1860506821 learned_addr learned my addr 172.21.15.33:0/1860506821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 -- 172.21.15.33:0/1860506821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26741076a0 msgr2=0x7f2674199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26741076a0 0x7f2674199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 -- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674108020 msgr2=0x7f267419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674108020 0x7f267419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 -- 172.21.15.33:0/1860506821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267419f3a0 con 0x7f2674101c90 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679e3a640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674108020 0x7f267419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2678e38640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26741076a0 0x7f2674199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.438+0000 7f2679639640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 0x7f26741992f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f266400eb90 tx=0x7f266400c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.439+0000 7f26727fc640 1 -- 172.21.15.33:0/1860506821 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664004d20 con 0x7f2674101c90 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.439+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f267419f690 con 0x7f2674101c90 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.439+0000 7f26727fc640 1 -- 172.21.15.33:0/1860506821 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266400ee90 con 0x7f2674101c90 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.439+0000 7f26727fc640 1 -- 172.21.15.33:0/1860506821 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664010670 con 0x7f2674101c90 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.439+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267410f210 con 0x7f2674101c90 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.440+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2674102070 con 0x7f2674101c90 2024-09-10T21:06:15.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.441+0000 7f26727fc640 1 -- 172.21.15.33:0/1860506821 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2664010830 con 0x7f2674101c90 2024-09-10T21:06:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.442+0000 7f26727fc640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f265c0b52e0 0x7f265c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.442+0000 7f2678e38640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f265c0b52e0 0x7f265c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.442+0000 7f26727fc640 1 -- 172.21.15.33:0/1860506821 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f26640e6420 con 0x7f2674101c90 2024-09-10T21:06:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.443+0000 7f2678e38640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f265c0b52e0 0x7f265c0b77a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2668031440 tx=0x7f2668005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:15.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.445+0000 7f26727fc640 1 -- 172.21.15.33:0/1860506821 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f2664056220 con 0x7f2674101c90 2024-09-10T21:06:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:15 smithi174 ceph-mon[35635]: pgmap v414: 97 pgs: 97 active+clean; 76 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 335 op/s 2024-09-10T21:06:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.672+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f267419a420 con 0x7f2674101c90 2024-09-10T21:06:15.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:15 smithi033 ceph-mon[30860]: pgmap v414: 97 pgs: 97 active+clean; 76 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 335 op/s 2024-09-10T21:06:15.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.672+0000 7f26727fc640 1 -- 172.21.15.33:0/1860506821 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11983 (secure 0 0 0) 0x7f26640ad2e0 con 0x7f2674101c90 2024-09-10T21:06:15.678 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:06:15.678 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"btime":"2024-09-10T21:06:07:350276+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:06:07.081199+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:06:15.678 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 32 2024-09-10T21:06:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f265c0b52e0 msgr2=0x7f265c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f265c0b52e0 0x7f265c0b77a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2668031440 tx=0x7f2668005b50 comp rx=0 tx=0).stop 2024-09-10T21:06:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 msgr2=0x7f26741992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 0x7f26741992f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f266400eb90 tx=0x7f266400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 shutdown_connections 2024-09-10T21:06:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f265c0b52e0 0x7f265c0b77a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2674108020 0x7f267419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26741076a0 0x7f2674199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 --2- 172.21.15.33:0/1860506821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2674101c90 0x7f26741992f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:15.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 >> 172.21.15.33:0/1860506821 conn(0x7f26740fd700 msgr2=0x7f26740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:15.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 shutdown_connections 2024-09-10T21:06:15.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:15.675+0000 7f267b89d640 1 -- 172.21.15.33:0/1860506821 wait complete. 2024-09-10T21:06:15.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 152862 11.09 MB/sec warmup 76 sec latency 63.297 ms 2024-09-10T21:06:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:15 smithi104 ceph-mon[35325]: pgmap v414: 97 pgs: 97 active+clean; 76 MiB data, 671 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 335 op/s 2024-09-10T21:06:16.381 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:06:16.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:16 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1860506821' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:06:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:16 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1860506821' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:06:16.689 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:16.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 155004 11.04 MB/sec warmup 77 sec latency 12.388 ms 2024-09-10T21:06:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:16 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1860506821' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:06:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 -- 172.21.15.33:0/4235360912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 msgr2=0x7f116410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 --2- 172.21.15.33:0/4235360912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116410e820 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f115c009ca0 tx=0x7f115c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 -- 172.21.15.33:0/4235360912 shutdown_connections 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 --2- 172.21.15.33:0/4235360912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116410e820 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 --2- 172.21.15.33:0/4235360912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11641076a0 0x7f1164107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 --2- 172.21.15.33:0/4235360912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1164101c90 0x7f1164102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 -- 172.21.15.33:0/4235360912 >> 172.21.15.33:0/4235360912 conn(0x7f11640fd700 msgr2=0x7f11640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 -- 172.21.15.33:0/4235360912 shutdown_connections 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.239+0000 7f1169b30640 1 -- 172.21.15.33:0/4235360912 wait complete. 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 Processor -- start 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 -- start start 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1164101c90 0x7f1164199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11641076a0 0x7f1164199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1164111ce0 con 0x7f1164108020 2024-09-10T21:06:17.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1164111b60 con 0x7f1164101c90 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1169b30640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1164111e60 con 0x7f11641076a0 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1163fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1163fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116419eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41696/0 (socket says 172.21.15.33:41696) 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f1163fff640 1 -- 172.21.15.33:0/293171692 learned_addr learned my addr 172.21.15.33:0/293171692 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f11637fe640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1164101c90 0x7f1164199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.240+0000 7f115bfff640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11641076a0 0x7f1164199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1163fff640 1 -- 172.21.15.33:0/293171692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11641076a0 msgr2=0x7f1164199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1163fff640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11641076a0 0x7f1164199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1163fff640 1 -- 172.21.15.33:0/293171692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1164101c90 msgr2=0x7f1164199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1163fff640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1164101c90 0x7f1164199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1163fff640 1 -- 172.21.15.33:0/293171692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f116419f2f0 con 0x7f1164108020 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f11637fe640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1164101c90 0x7f1164199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:17.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1163fff640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116419eb20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f115c005cc0 tx=0x7f115c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f11617fa640 1 -- 172.21.15.33:0/293171692 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c003b70 con 0x7f1164108020 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f116419f580 con 0x7f1164108020 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f11617fa640 1 -- 172.21.15.33:0/293171692 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115c0318f0 con 0x7f1164108020 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f11617fa640 1 -- 172.21.15.33:0/293171692 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c02f950 con 0x7f1164108020 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.241+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11641a5ca0 con 0x7f1164108020 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 Processor -- start 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f11617fa640 1 -- 172.21.15.33:0/293171692 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f115c02fb50 con 0x7f1164108020 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 -- start start 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b14055cea0 con 0x55b1405e60f0 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b14054c2d0 con 0x55b1405e6a10 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f116bf05740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b1405500e0 con 0x55b1405f6db0 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f115bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f11637fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f115bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41702/0 (socket says 172.21.15.33:41702) 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f115bfff640 1 -- 172.21.15.33:0/3119051899 learned_addr learned my addr 172.21.15.33:0/3119051899 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.244+0000 7f115bfff640 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11637fe640 1 -- 172.21.15.33:0/3119051899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 msgr2=0x55b1405f91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11637fe640 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11617fa640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11500b52e0 0x7f11500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11637fe640 1 -- 172.21.15.33:0/3119051899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 msgr2=0x55b1405e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11637fe640 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11637fe640 1 -- 172.21.15.33:0/3119051899 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1402aa650 con 0x55b1405e6a10 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f115bfff640 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f1163fff640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11500b52e0 0x7f11500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11637fe640 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f6870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1154009fd0 tx=0x7f11540118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b503ac633455105 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f11617fa640 1 -- 172.21.15.33:0/293171692 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f115c106cf0 con 0x7f1164108020 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f1159ffb640 1 -- 172.21.15.33:0/3119051899 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154004310 con 0x55b1405e6a10 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f1159ffb640 1 -- 172.21.15.33:0/3119051899 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11540044b0 con 0x55b1405e6a10 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.245+0000 7f1159ffb640 1 -- 172.21.15.33:0/3119051899 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11540035f0 con 0x55b1405e6a10 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 -- 172.21.15.33:0/3119051899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 msgr2=0x55b1405f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f6870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1154009fd0 tx=0x7f11540118d0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f1163fff640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11500b52e0 0x7f11500b77a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f115c0cd440 tx=0x7f115c09aa90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 -- 172.21.15.33:0/3119051899 shutdown_connections 2024-09-10T21:06:17.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f6870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 --2- 172.21.15.33:0/3119051899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 -- 172.21.15.33:0/3119051899 >> 172.21.15.33:0/3119051899 conn(0x55b1405e3f20 msgr2=0x55b1405e50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 -- 172.21.15.33:0/3119051899 shutdown_connections 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 -- 172.21.15.33:0/3119051899 wait complete. 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.246+0000 7f116bf05740 1 -- start start 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 1 Processor -- start 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405f6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b1403ad700 con 0x55b1405e60f0 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b1404e7160 con 0x55b1405e6a10 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b1405f9a60 con 0x55b1405f6db0 2024-09-10T21:06:17.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f116bf05740 10 client.0 _ll_register_callbacks cb 0x55b1405009e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f115bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f11637fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f115bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41712/0 (socket says 172.21.15.33:41712) 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f115bfff640 1 -- 172.21.15.33:0/2257294055 learned_addr learned my addr 172.21.15.33:0/2257294055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f115bfff640 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f11637fe640 1 -- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 msgr2=0x55b1405f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f11637fe640 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f11637fe640 1 -- 172.21.15.33:0/2257294055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 msgr2=0x55b1405f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f11637fe640 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405f6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f11637fe640 1 -- 172.21.15.33:0/2257294055 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b140514c90 con 0x55b1405e6a10 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.247+0000 7f115bfff640 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f11637fe640 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f3ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1154005890 tx=0x7f11540048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=686588d8a2620b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:17.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f1142ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f1142ffd640 1 -- 172.21.15.33:0/2257294055 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154029070 con 0x55b1405e6a10 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f1142ffd640 1 -- 172.21.15.33:0/2257294055 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1154003a00 con 0x55b1405e6a10 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11540044b0 con 0x55b1405e6a10 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f1142ffd640 1 -- 172.21.15.33:0/2257294055 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1154035050 con 0x55b1405e6a10 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 10 client.25298 fetch_fsmap learned FSMap version 32 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b1406053a0 con 0x55b1405e6a10 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f1142ffd640 1 -- 172.21.15.33:0/2257294055 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11540044b0 con 0x55b1405e6a10 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f1142ffd640 1 -- 172.21.15.33:0/2257294055 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f115400fcc0 con 0x55b1405e6a10 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 10 client.25298 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 10 client.25298 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 20 client.25298 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b140606750 0x55b140608b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 4 client.25298 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.248+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b14055edf0 con 0x55b140606750 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.249+0000 7f115bfff640 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b140606750 0x55b140608b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.249+0000 7f1142ffd640 10 client.25298 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.249+0000 7f115bfff640 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b140606750 0x55b140608b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9e0882e209afb64 server_cookie=6799d3d5a9df2eba in_seq=0 out_seq=0 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f1142ffd640 1 -- 172.21.15.33:0/2257294055 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b14055edf0 con 0x55b140606750 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f1142ffd640 10 client.25298 handle_command_reply: tid=0 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 client.25298 shutdown 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 msgr2=0x55b1405f3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f3ad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1154005890 tx=0x7f11540048f0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 shutdown_connections 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b140606750 0x55b140608b40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405f6db0 0x55b1405f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405e6a10 0x55b1405f3ad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f1142ffd640 0 client.25298 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:06:17.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 --2- 172.21.15.33:0/2257294055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b1405f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 >> 172.21.15.33:0/2257294055 conn(0x55b1405e3f20 msgr2=0x55b1405e9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:17.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 shutdown_connections 2024-09-10T21:06:17.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 1 -- 172.21.15.33:0/2257294055 wait complete. 2024-09-10T21:06:17.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.250+0000 7f116bf05740 20 client.25298 trim_cache size 0 max 16384 2024-09-10T21:06:17.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 Processor -- start 2024-09-10T21:06:17.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 -- start start 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b140602cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e6a10 0x55b1405e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11641ac0a0 con 0x55b1405e60f0 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11641add00 con 0x55b1405f6db0 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f116bf05740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11641ac770 con 0x55b1405e6a10 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f115bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b140602cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f11637fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e6a10 0x55b1405e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f11637fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e6a10 0x55b1405e3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:50920/0 (socket says 172.21.15.33:50920) 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f115bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b140602cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41722/0 (socket says 172.21.15.33:41722) 2024-09-10T21:06:17.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f11637fe640 1 -- 172.21.15.33:0/967209644 learned_addr learned my addr 172.21.15.33:0/967209644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.264+0000 7f115bfff640 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11637fe640 1 -- 172.21.15.33:0/967209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 msgr2=0x55b1405e54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11637fe640 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405e54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11637fe640 1 -- 172.21.15.33:0/967209644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 msgr2=0x55b140602cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11637fe640 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b140602cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11637fe640 1 -- 172.21.15.33:0/967209644 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1405f1fd0 con 0x55b1405e6a10 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11637fe640 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e6a10 0x55b1405e3760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1154004020 tx=0x7f1154009e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5db36345e1da278 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11597fa640 1 -- 172.21.15.33:0/967209644 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154032030 con 0x55b1405e6a10 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11597fa640 1 -- 172.21.15.33:0/967209644 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1154009b30 con 0x55b1405e6a10 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.265+0000 7f11597fa640 1 -- 172.21.15.33:0/967209644 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154029050 con 0x55b1405e6a10 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 -- 172.21.15.33:0/967209644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e6a10 msgr2=0x55b1405e3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e6a10 0x55b1405e3760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1154004020 tx=0x7f1154009e20 comp rx=0 tx=0).stop 2024-09-10T21:06:17.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 -- 172.21.15.33:0/967209644 shutdown_connections 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405e54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e6a10 0x55b1405e3760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 --2- 172.21.15.33:0/967209644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e60f0 0x55b140602cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 -- 172.21.15.33:0/967209644 >> 172.21.15.33:0/967209644 conn(0x55b1405e3f20 msgr2=0x55b1405e4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 -- 172.21.15.33:0/967209644 shutdown_connections 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 -- 172.21.15.33:0/967209644 wait complete. 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.266+0000 7f116bf05740 1 -- start start 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 1 Processor -- start 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e60f0 0x55b1405fed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e6a10 0x55b1405ff260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f11641ad570 con 0x55b1405e6a10 2024-09-10T21:06:17.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f11641ace40 con 0x55b1405f6db0 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11641aa060 con 0x55b1405e60f0 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f116bf05740 10 client.0 _ll_register_callbacks cb 0x7f114c009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f115bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e60f0 0x55b1405fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f11637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e6a10 0x55b1405ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f115bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e60f0 0x55b1405fed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:50932/0 (socket says 172.21.15.33:50932) 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f115bfff640 1 -- 172.21.15.33:0/1995351763 learned_addr learned my addr 172.21.15.33:0/1995351763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f115bfff640 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f11637fe640 1 -- 172.21.15.33:0/1995351763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e60f0 msgr2=0x55b1405fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f11637fe640 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e60f0 0x55b1405fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f11637fe640 1 -- 172.21.15.33:0/1995351763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 msgr2=0x55b1405fbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f11637fe640 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405fbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f11637fe640 1 -- 172.21.15.33:0/1995351763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1405ffa90 con 0x55b1405e6a10 2024-09-10T21:06:17.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.267+0000 7f115bfff640 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f11637fe640 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e6a10 0x55b1405ff260 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55b1405f4780 tx=0x7f11540236a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac1772d4be8caca3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f1142ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f1142ffd640 1 -- 172.21.15.33:0/1995351763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115403d070 con 0x55b1405e6a10 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f1142ffd640 1 -- 172.21.15.33:0/1995351763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1154003800 con 0x55b1405e6a10 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1154009b30 con 0x55b1405e6a10 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f1142ffd640 1 -- 172.21.15.33:0/1995351763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1154049050 con 0x55b1405e6a10 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 10 client.15723 fetch_fsmap learned FSMap version 32 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b140673ac0 con 0x55b1405e6a10 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f1142ffd640 1 -- 172.21.15.33:0/1995351763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1154009b30 con 0x55b1405e6a10 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f1142ffd640 1 -- 172.21.15.33:0/1995351763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 32) ==== 8386+0+0 (secure 0 0 0) 0x7f115400f9e0 con 0x55b1405e6a10 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 10 client.15723 fetch_fsmap finished waiting for FSMap version 32 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 10 client.15723 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:06:17.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 20 client.15723 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b1406741e0 0x55b140676600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 4 client.15723 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.268+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b14055a8a0 con 0x55b1406741e0 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f115bfff640 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b1406741e0 0x55b140676600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f1142ffd640 10 client.15723 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f115bfff640 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b1406741e0 0x55b140676600 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3b5040da36634ecd server_cookie=2969bacc93275264 in_seq=0 out_seq=0 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f1142ffd640 1 -- 172.21.15.33:0/1995351763 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b14055a8a0 con 0x55b1406741e0 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f1142ffd640 10 client.15723 handle_command_reply: tid=0 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f116bf05740 1 client.15723 shutdown 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e6a10 msgr2=0x55b1405ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f116bf05740 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e6a10 0x55b1405ff260 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x55b1405f4780 tx=0x7f11540236a0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 shutdown_connections 2024-09-10T21:06:17.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f116bf05740 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55b1406741e0 0x55b140676600 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.269+0000 7f116bf05740 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b1405f6db0 0x55b1405fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f116bf05740 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b1405e6a10 0x55b1405ff260 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f116bf05740 1 --2- 172.21.15.33:0/1995351763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b1405e60f0 0x55b1405fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 >> 172.21.15.33:0/1995351763 conn(0x55b1405e3f20 msgr2=0x55b1405e5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1142ffd640 0 client.15723 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 shutdown_connections 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f116bf05740 1 -- 172.21.15.33:0/1995351763 wait complete. 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f116bf05740 20 client.15723 trim_cache size 0 max 16384 2024-09-10T21:06:17.334 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11500b52e0 msgr2=0x7f11500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11500b52e0 0x7f11500b77a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f115c0cd440 tx=0x7f115c09aa90 comp rx=0 tx=0).stop 2024-09-10T21:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 msgr2=0x7f116419eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116419eb20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f115c005cc0 tx=0x7f115c031700 comp rx=0 tx=0).stop 2024-09-10T21:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 shutdown_connections 2024-09-10T21:06:17.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f11500b52e0 0x7f11500b77a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1164108020 0x7f116419eb20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f11641076a0 0x7f1164199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 --2- 172.21.15.33:0/293171692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1164101c90 0x7f1164199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.270+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 >> 172.21.15.33:0/293171692 conn(0x7f11640fd700 msgr2=0x7f116406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.271+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 shutdown_connections 2024-09-10T21:06:17.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:17.271+0000 7f1169b30640 1 -- 172.21.15.33:0/293171692 wait complete. 2024-09-10T21:06:17.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:17 smithi033 ceph-mon[30860]: pgmap v415: 97 pgs: 97 active+clean; 78 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-09-10T21:06:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:17 smithi174 ceph-mon[35635]: pgmap v415: 97 pgs: 97 active+clean; 78 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-09-10T21:06:17.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 156668 11.06 MB/sec warmup 78 sec latency 39.407 ms 2024-09-10T21:06:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:17 smithi104 ceph-mon[35325]: pgmap v415: 97 pgs: 97 active+clean; 78 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 284 op/s 2024-09-10T21:06:17.998 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:06:18.270 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:18.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:18 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:18.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 159266 11.08 MB/sec warmup 79 sec latency 43.530 ms 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 -- 172.21.15.33:0/500096061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7458101cb0 msgr2=0x7f7458102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 --2- 172.21.15.33:0/500096061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7458101cb0 0x7f7458102090 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c02a410 comp rx=0 tx=0).stop 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 -- 172.21.15.33:0/500096061 shutdown_connections 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 --2- 172.21.15.33:0/500096061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7458108040 0x7f745810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 --2- 172.21.15.33:0/500096061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74581076c0 0x7f7458107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 --2- 172.21.15.33:0/500096061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7458101cb0 0x7f7458102090 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 -- 172.21.15.33:0/500096061 >> 172.21.15.33:0/500096061 conn(0x7f74580fd700 msgr2=0x7f74580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 -- 172.21.15.33:0/500096061 shutdown_connections 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.785+0000 7f745ef4c640 1 -- 172.21.15.33:0/500096061 wait complete. 2024-09-10T21:06:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 Processor -- start 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 -- start start 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7458101cb0 0x7f745819fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74581076c0 0x7f74581a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7458108040 0x7f74581a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7458111cc0 con 0x7f74581076c0 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7458111b40 con 0x7f7458101cb0 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745ef4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7458111e40 con 0x7f7458108040 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f7457fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74581076c0 0x7f74581a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f7457fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74581076c0 0x7f74581a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41742/0 (socket says 172.21.15.33:41742) 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f7457fff640 1 -- 172.21.15.33:0/2286581585 learned_addr learned my addr 172.21.15.33:0/2286581585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745cce8640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7458101cb0 0x7f745819fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.786+0000 7f745d4e9640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7458108040 0x7f74581a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7457fff640 1 -- 172.21.15.33:0/2286581585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7458108040 msgr2=0x7f74581a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7457fff640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7458108040 0x7f74581a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7457fff640 1 -- 172.21.15.33:0/2286581585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7458101cb0 msgr2=0x7f745819fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7457fff640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7458101cb0 0x7f745819fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7457fff640 1 -- 172.21.15.33:0/2286581585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74581a26e0 con 0x7f74581076c0 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f745cce8640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7458101cb0 0x7f745819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7457fff640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74581076c0 0x7f74581a0050 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f744400e9f0 tx=0x7f744400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7455ffb640 1 -- 172.21.15.33:0/2286581585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744400cd80 con 0x7f74581076c0 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7455ffb640 1 -- 172.21.15.33:0/2286581585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74440108e0 con 0x7f74581076c0 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74581a29d0 con 0x7f74581076c0 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f7455ffb640 1 -- 172.21.15.33:0/2286581585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444018980 con 0x7f74581076c0 2024-09-10T21:06:18.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.787+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581ac2d0 con 0x7f74581076c0 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 Processor -- start 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 -- start start 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7455ffb640 1 -- 172.21.15.33:0/2286581585 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7444018b20 con 0x7f74581076c0 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accc5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5579acc2fea0 con 0x5579accb9df0 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5579acc1f2d0 con 0x5579accb94d0 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.790+0000 7f7461321740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579acc230e0 con 0x5579accc5d80 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745d4e9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accc5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accc5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41752/0 (socket says 172.21.15.33:41752) 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 -- 172.21.15.33:0/4028900148 learned_addr learned my addr 172.21.15.33:0/4028900148 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f7457fff640 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 -- 172.21.15.33:0/4028900148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 msgr2=0x5579accc8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 -- 172.21.15.33:0/4028900148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 msgr2=0x5579accb98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 -- 172.21.15.33:0/4028900148 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579ac97d650 con 0x5579accb9df0 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745d4e9640 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745cce8640 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accc5840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5f2fca54fad9ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f7455ffb640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74480b52e0 0x7f74480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f7442ffd640 1 -- 172.21.15.33:0/4028900148 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c040070 con 0x5579accb9df0 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f745d4e9640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74480b52e0 0x7f74480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f7455ffb640 1 -- 172.21.15.33:0/2286581585 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f74440e5f80 con 0x7f74581076c0 2024-09-10T21:06:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f7442ffd640 1 -- 172.21.15.33:0/4028900148 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f744c031860 con 0x5579accb9df0 2024-09-10T21:06:18.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.791+0000 7f7442ffd640 1 -- 172.21.15.33:0/4028900148 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c035040 con 0x5579accb9df0 2024-09-10T21:06:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 -- 172.21.15.33:0/4028900148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 msgr2=0x5579accc5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f745d4e9640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74480b52e0 0x7f74480b77a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7450009fd0 tx=0x7f7450009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accc5840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f744c009900 tx=0x7f744c005d90 comp rx=0 tx=0).stop 2024-09-10T21:06:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 -- 172.21.15.33:0/4028900148 shutdown_connections 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accc5840 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 --2- 172.21.15.33:0/4028900148 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 -- 172.21.15.33:0/4028900148 >> 172.21.15.33:0/4028900148 conn(0x5579accb7300 msgr2=0x5579accb84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 -- 172.21.15.33:0/4028900148 shutdown_connections 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.792+0000 7f7461321740 1 -- 172.21.15.33:0/4028900148 wait complete. 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 -- start start 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 Processor -- start 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accc5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accc2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accc5d80 0x5579accc3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5579aca80700 con 0x5579accc5d80 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5579acbba160 con 0x5579accb94d0 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5579accc89a0 con 0x5579accb9df0 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7461321740 10 client.0 _ll_register_callbacks cb 0x5579acbd39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f745cce8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accc5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f745d4e9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accc5d80 0x5579accc3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f745cce8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accc5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59220/0 (socket says 172.21.15.33:59220) 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f745cce8640 1 -- 172.21.15.33:0/2437745950 learned_addr learned my addr 172.21.15.33:0/2437745950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.793+0000 7f7457fff640 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accc2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f745cce8640 1 -- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 msgr2=0x5579accc2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f745cce8640 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accc2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f745cce8640 1 -- 172.21.15.33:0/2437745950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accc5d80 msgr2=0x5579accc3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f745cce8640 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accc5d80 0x5579accc3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f745cce8640 1 -- 172.21.15.33:0/2437745950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579acbe7c90 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f745d4e9640 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accc5d80 0x5579accc3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f745cce8640 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accc5650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f744c02a930 tx=0x7f744c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b3dbb1676c1947 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f7437fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f7437fff640 1 -- 172.21.15.33:0/2437745950 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c04a070 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f7437fff640 1 -- 172.21.15.33:0/2437745950 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f744c0035f0 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f744c031860 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f7437fff640 1 -- 172.21.15.33:0/2437745950 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f744c035040 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f7461321740 10 client.25316 fetch_fsmap learned FSMap version 33 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.794+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5579accd42b0 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7437fff640 1 -- 172.21.15.33:0/2437745950 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f744c031860 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7437fff640 1 -- 172.21.15.33:0/2437745950 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f744c03c6e0 con 0x5579accb94d0 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7461321740 10 client.25316 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:06:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7461321740 10 client.25316 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7461321740 20 client.25316 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7461321740 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579accd5600 0x5579accd79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7461321740 4 client.25316 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579acc31df0 con 0x5579accd5600 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7457fff640 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579accd5600 0x5579accd79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7457fff640 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579accd5600 0x5579accd79f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c055bc127808e8fa server_cookie=17cce4f73a65301c in_seq=0 out_seq=0 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.795+0000 7f7437fff640 10 client.25316 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.796+0000 7f7437fff640 1 -- 172.21.15.33:0/2437745950 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5579acc31df0 con 0x5579accd5600 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.796+0000 7f7437fff640 10 client.25316 handle_command_reply: tid=0 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.796+0000 7f7461321740 1 client.25316 shutdown 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 msgr2=0x5579accc5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accc5650 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f744c02a930 tx=0x7f744c005e80 comp rx=0 tx=0).stop 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 shutdown_connections 2024-09-10T21:06:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accc5d80 0x5579accc3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579accd5600 0x5579accd79f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accc2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 --2- 172.21.15.33:0/2437745950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accc5650 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7437fff640 0 client.25316 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 >> 172.21.15.33:0/2437745950 conn(0x5579accb7300 msgr2=0x5579accbcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 shutdown_connections 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 1 -- 172.21.15.33:0/2437745950 wait complete. 2024-09-10T21:06:18.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.797+0000 7f7461321740 20 client.25316 trim_cache size 0 max 16384 2024-09-10T21:06:18.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:18 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 Processor -- start 2024-09-10T21:06:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 -- start start 2024-09-10T21:06:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accb6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b4a40 con 0x5579accb9df0 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b66d0 con 0x5579accb94d0 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.811+0000 7f7461321740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b5110 con 0x5579accc5d80 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accb6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f745cce8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accb6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41780/0 (socket says 172.21.15.33:41780) 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 -- 172.21.15.33:0/860953792 learned_addr learned my addr 172.21.15.33:0/860953792 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:18.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f745d4e9640 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 -- 172.21.15.33:0/860953792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 msgr2=0x5579accc47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 -- 172.21.15.33:0/860953792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 msgr2=0x5579accb88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 -- 172.21.15.33:0/860953792 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579accb2bb0 con 0x5579accb9df0 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f745cce8640 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f7457fff640 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accb6c10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f74440ac4b0 tx=0x7f7444018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d23095dda02bfd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f74427fc640 1 -- 172.21.15.33:0/860953792 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74440abf20 con 0x5579accb9df0 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f74427fc640 1 -- 172.21.15.33:0/860953792 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74440ac2c0 con 0x5579accb9df0 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.812+0000 7f74427fc640 1 -- 172.21.15.33:0/860953792 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7444010b60 con 0x5579accb9df0 2024-09-10T21:06:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 -- 172.21.15.33:0/860953792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 msgr2=0x5579accb6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accb6c10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f74440ac4b0 tx=0x7f7444018420 comp rx=0 tx=0).stop 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 -- 172.21.15.33:0/860953792 shutdown_connections 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accc5d80 0x5579accc47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb9df0 0x5579accb6c10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 --2- 172.21.15.33:0/860953792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accb94d0 0x5579accb88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 -- 172.21.15.33:0/860953792 >> 172.21.15.33:0/860953792 conn(0x5579accb7300 msgr2=0x5579accc40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 -- 172.21.15.33:0/860953792 shutdown_connections 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 -- 172.21.15.33:0/860953792 wait complete. 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.813+0000 7f7461321740 1 -- start start 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 1 Processor -- start 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb94d0 0x5579accd6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accd7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accc5d80 0x5579accd75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b5f40 con 0x5579accb94d0 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b57e0 con 0x5579accc5d80 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74581b2a00 con 0x5579accb9df0 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7461321740 10 client.0 _ll_register_callbacks cb 0x7f7450009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f745cce8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb94d0 0x5579accd6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f745cce8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb94d0 0x5579accd6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41786/0 (socket says 172.21.15.33:41786) 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f7457fff640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accd7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f745cce8640 1 -- 172.21.15.33:0/3481433949 learned_addr learned my addr 172.21.15.33:0/3481433949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.814+0000 7f745d4e9640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accc5d80 0x5579accd75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f745cce8640 1 -- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 msgr2=0x5579accd7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f745cce8640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accd7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f745cce8640 1 -- 172.21.15.33:0/3481433949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accc5d80 msgr2=0x5579accd75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:18.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f745cce8640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accc5d80 0x5579accd75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f745cce8640 1 -- 172.21.15.33:0/3481433949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579acd41360 con 0x5579accb94d0 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7457fff640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accd7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f745d4e9640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accc5d80 0x5579accd75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f745cce8640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb94d0 0x5579accd6b30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f744c03cab0 tx=0x7f744c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8667c7aaf52e8dd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7437fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7437fff640 1 -- 172.21.15.33:0/3481433949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744c040040 con 0x5579accb94d0 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7437fff640 1 -- 172.21.15.33:0/3481433949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f744c005240 con 0x5579accb94d0 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74440ac2c0 con 0x5579accb94d0 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7437fff640 1 -- 172.21.15.33:0/3481433949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f744c004250 con 0x5579accb94d0 2024-09-10T21:06:18.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7461321740 10 client.15741 fetch_fsmap learned FSMap version 33 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5579acd42660 con 0x5579accb94d0 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.815+0000 7f7437fff640 1 -- 172.21.15.33:0/3481433949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74440ac2c0 con 0x5579accb94d0 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7437fff640 1 -- 172.21.15.33:0/3481433949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f744c0035f0 con 0x5579accb94d0 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7461321740 10 client.15741 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7461321740 10 client.15741 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7461321740 20 client.15741 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7461321740 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579acd42dd0 0x5579acd451f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7461321740 4 client.15741 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579acc2d8a0 con 0x5579acd42dd0 2024-09-10T21:06:18.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7457fff640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579acd42dd0 0x5579acd451f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:18.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7437fff640 10 client.15741 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:06:18.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7457fff640 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579acd42dd0 0x5579acd451f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dff1f9e48ed5e166 server_cookie=8ddfa61df4a9bbbe in_seq=0 out_seq=0 2024-09-10T21:06:18.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7437fff640 1 -- 172.21.15.33:0/3481433949 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579acc2d8a0 con 0x5579acd42dd0 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.816+0000 7f7437fff640 10 client.15741 handle_command_reply: tid=0 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 client.15741 shutdown 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb94d0 msgr2=0x5579accd6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb94d0 0x5579accd6b30 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f744c03cab0 tx=0x7f744c005b50 comp rx=0 tx=0).stop 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 shutdown_connections 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5579accc5d80 0x5579accd75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5579acd42dd0 0x5579acd451f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5579accb9df0 0x5579accd7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 --2- 172.21.15.33:0/3481433949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5579accb94d0 0x5579accd6b30 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7437fff640 0 client.15741 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 >> 172.21.15.33:0/3481433949 conn(0x5579accb7300 msgr2=0x5579accb8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:18.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 shutdown_connections 2024-09-10T21:06:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 1 -- 172.21.15.33:0/3481433949 wait complete. 2024-09-10T21:06:18.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.817+0000 7f7461321740 20 client.15741 trim_cache size 0 max 16384 2024-09-10T21:06:18.828 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:06:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74480b52e0 msgr2=0x7f74480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74480b52e0 0x7f74480b77a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7450009fd0 tx=0x7f7450009500 comp rx=0 tx=0).stop 2024-09-10T21:06:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74581076c0 msgr2=0x7f74581a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74581076c0 0x7f74581a0050 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f744400e9f0 tx=0x7f744400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 shutdown_connections 2024-09-10T21:06:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74480b52e0 0x7f74480b77a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7458108040 0x7f74581a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74581076c0 0x7f74581a0050 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 --2- 172.21.15.33:0/2286581585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7458101cb0 0x7f745819fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 >> 172.21.15.33:0/2286581585 conn(0x7f74580fd700 msgr2=0x7f74580ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 shutdown_connections 2024-09-10T21:06:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:18.818+0000 7f745ef4c640 1 -- 172.21.15.33:0/2286581585 wait complete. 2024-09-10T21:06:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:18 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:19.658 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:19 smithi033 ceph-mon[30860]: pgmap v416: 97 pgs: 97 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-09-10T21:06:19.687 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:06:19.700 INFO:tasks.workunit.client.0.smithi033.stdout: 1 159645 11.00 MB/sec warmup 80 sec latency 852.402 ms 2024-09-10T21:06:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:19 smithi104 ceph-mon[35325]: pgmap v416: 97 pgs: 97 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-09-10T21:06:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:19 smithi174 ceph-mon[35635]: pgmap v416: 97 pgs: 97 active+clean; 79 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.4 MiB/s wr, 277 op/s 2024-09-10T21:06:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 -- 172.21.15.33:0/1328676941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc102dc0 msgr2=0x7f08bc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 --2- 172.21.15.33:0/1328676941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc102dc0 0x7f08bc1031a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009900 tx=0x7f08b402a410 comp rx=0 tx=0).stop 2024-09-10T21:06:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 -- 172.21.15.33:0/1328676941 shutdown_connections 2024-09-10T21:06:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 --2- 172.21.15.33:0/1328676941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08bc109150 0x7f08bc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 --2- 172.21.15.33:0/1328676941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08bc1087d0 0x7f08bc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 --2- 172.21.15.33:0/1328676941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc102dc0 0x7f08bc1031a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 -- 172.21.15.33:0/1328676941 >> 172.21.15.33:0/1328676941 conn(0x7f08bc0fe830 msgr2=0x7f08bc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 -- 172.21.15.33:0/1328676941 shutdown_connections 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.415+0000 7f08c1637640 1 -- 172.21.15.33:0/1328676941 wait complete. 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.416+0000 7f08c1637640 1 Processor -- start 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.416+0000 7f08c1637640 1 -- start start 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.416+0000 7f08c1637640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08bc102dc0 0x7f08bc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.416+0000 7f08c1637640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08bc1087d0 0x7f08bc1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.416+0000 7f08c1637640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc109150 0x7f08bc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.416+0000 7f08c1637640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc112e10 con 0x7f08bc109150 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.416+0000 7f08c1637640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc112c90 con 0x7f08bc1087d0 2024-09-10T21:06:20.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08c1637640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc112f90 con 0x7f08bc102dc0 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc109150 0x7f08bc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc109150 0x7f08bc19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50930/0 (socket says 172.21.15.33:50930) 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 -- 172.21.15.33:0/3834885268 learned_addr learned my addr 172.21.15.33:0/3834885268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08baffd640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08bc102dc0 0x7f08bc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08bc1087d0 0x7f08bc1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 -- 172.21.15.33:0/3834885268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08bc102dc0 msgr2=0x7f08bc199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08bc102dc0 0x7f08bc199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 -- 172.21.15.33:0/3834885268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08bc1087d0 msgr2=0x7f08bc1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08bc1087d0 0x7f08bc1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 -- 172.21.15.33:0/3834885268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08bc19f330 con 0x7f08bc109150 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08bc1087d0 0x7f08bc1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc109150 0x7f08bc19eb60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00eaa0 tx=0x7f08ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08b3fff640 1 -- 172.21.15.33:0/3834885268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac00cdb0 con 0x7f08bc109150 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08bc19f620 con 0x7f08bc109150 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08b3fff640 1 -- 172.21.15.33:0/3834885268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08ac005020 con 0x7f08bc109150 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08b3fff640 1 -- 172.21.15.33:0/3834885268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ac0116d0 con 0x7f08bc109150 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.417+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08bc1a5ce0 con 0x7f08bc109150 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.420+0000 7f08c3a0c740 1 Processor -- start 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08b3fff640 1 -- 172.21.15.33:0/3834885268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f08ac00f040 con 0x7f08bc109150 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08c3a0c740 1 -- start start 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd86a10 0x55575bd96870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd96db0 0x55575bd991a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55575bcfcea0 con 0x55575bd860f0 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55575bcec2d0 con 0x55575bd96db0 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55575bcf00e0 con 0x55575bd86a10 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd864d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50934/0 (socket says 172.21.15.33:50934) 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08ba7fc640 1 -- 172.21.15.33:0/2387950108 learned_addr learned my addr 172.21.15.33:0/2387950108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.421+0000 7f08baffd640 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd86a10 0x55575bd96870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08ba7fc640 1 -- 172.21.15.33:0/2387950108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd86a10 msgr2=0x55575bd96870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd86a10 0x55575bd96870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08ba7fc640 1 -- 172.21.15.33:0/2387950108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd96db0 msgr2=0x55575bd991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd96db0 0x55575bd991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08ba7fc640 1 -- 172.21.15.33:0/2387950108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575ba4a650 con 0x55575bd860f0 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08b3fff640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08a80b52e0 0x7f08a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08baffd640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08a80b52e0 0x7f08a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd864d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f08a400bde0 tx=0x7f08a4011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab418365072f670d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08b3fff640 1 -- 172.21.15.33:0/3834885268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f08ac0e5a40 con 0x7f08bc109150 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08b17fa640 1 -- 172.21.15.33:0/2387950108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a401a600 con 0x55575bd860f0 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08b17fa640 1 -- 172.21.15.33:0/2387950108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08a40128a0 con 0x55575bd860f0 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.422+0000 7f08b17fa640 1 -- 172.21.15.33:0/2387950108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a400f7b0 con 0x55575bd860f0 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2387950108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 msgr2=0x55575bd864d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08baffd640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08a80b52e0 0x7f08a80b77a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f08b4031440 tx=0x7f08b4005c00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd864d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f08a400bde0 tx=0x7f08a4011750 comp rx=0 tx=0).stop 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2387950108 shutdown_connections 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd96db0 0x55575bd991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd86a10 0x55575bd96870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2387950108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd864d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2387950108 >> 172.21.15.33:0/2387950108 conn(0x55575bd83f20 msgr2=0x55575bd850c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2387950108 shutdown_connections 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2387950108 wait complete. 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 -- start start 2024-09-10T21:06:20.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.423+0000 7f08c3a0c740 1 Processor -- start 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.424+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.424+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd93b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.424+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd941b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.424+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55575bb4d700 con 0x55575bd860f0 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.424+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55575bc87160 con 0x55575bd86a10 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.424+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55575bd99a60 con 0x55575bd96db0 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.424+0000 7f08c3a0c740 10 client.0 _ll_register_callbacks cb 0x55575bca09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd96700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50942/0 (socket says 172.21.15.33:50942) 2024-09-10T21:06:20.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 -- 172.21.15.33:0/3117646201 learned_addr learned my addr 172.21.15.33:0/3117646201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd93b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 -- 172.21.15.33:0/3117646201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 msgr2=0x55575bd941b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd941b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 -- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 msgr2=0x55575bd93b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd93b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 -- 172.21.15.33:0/3117646201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575bcb4c90 con 0x55575bd860f0 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:20.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08baffd640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd96700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08b4032fd0 tx=0x7f08b4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55ab0ae14a387a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08a27fc640 1 -- 172.21.15.33:0/3117646201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b404d070 con 0x55575bd860f0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08a27fc640 1 -- 172.21.15.33:0/3117646201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f08b4005030 con 0x55575bd860f0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08a40128a0 con 0x55575bd860f0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.425+0000 7f08a27fc640 1 -- 172.21.15.33:0/3117646201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08b4059050 con 0x55575bd860f0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 10 client.15753 fetch_fsmap learned FSMap version 33 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55575bd8de50 con 0x55575bd860f0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08a27fc640 1 -- 172.21.15.33:0/3117646201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08a40128a0 con 0x55575bd860f0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08a27fc640 1 -- 172.21.15.33:0/3117646201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f08b40035f0 con 0x55575bd860f0 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 10 client.15753 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:06:20.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 10 client.15753 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:06:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 20 client.15753 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575bd8f320 0x55575bd8f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 4 client.15753 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55575bcfedf0 con 0x55575bd8f320 2024-09-10T21:06:20.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.426+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575bd8f320 0x55575bd8f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.427+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575bd8f320 0x55575bd8f740 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb0b948a1e5eeea7 server_cookie=5e3a9fbbbcb0c799 in_seq=0 out_seq=0 2024-09-10T21:06:20.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.427+0000 7f08a27fc640 10 client.15753 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:06:20.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08a27fc640 1 -- 172.21.15.33:0/3117646201 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55575bcfedf0 con 0x55575bd8f320 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08a27fc640 10 client.15753 handle_command_reply: tid=0 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 client.15753 shutdown 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 msgr2=0x55575bd96700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd96700 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f08b4032fd0 tx=0x7f08b4009260 comp rx=0 tx=0).stop 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 shutdown_connections 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd941b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd93b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575bd8f320 0x55575bd8f740 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3117646201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd96700 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 >> 172.21.15.33:0/3117646201 conn(0x55575bd83f20 msgr2=0x55575bd89a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08a27fc640 0 client.15753 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:06:20.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 shutdown_connections 2024-09-10T21:06:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3117646201 wait complete. 2024-09-10T21:06:20.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.428+0000 7f08c3a0c740 20 client.15753 trim_cache size 0 max 16384 2024-09-10T21:06:20.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.441+0000 7f08c3a0c740 1 Processor -- start 2024-09-10T21:06:20.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.441+0000 7f08c3a0c740 1 -- start start 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd85790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd95890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ac170 con 0x55575bd860f0 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ade00 con 0x55575bd86a10 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ac840 con 0x55575bd96db0 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd85790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd85790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50948/0 (socket says 172.21.15.33:50948) 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 -- 172.21.15.33:0/2377352804 learned_addr learned my addr 172.21.15.33:0/2377352804 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd95890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 -- 172.21.15.33:0/2377352804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 msgr2=0x55575bd95890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd95890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 -- 172.21.15.33:0/2377352804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 msgr2=0x55575bd83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 -- 172.21.15.33:0/2377352804 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55575bd8c250 con 0x55575bd860f0 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd95890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.442+0000 7f08baffd640 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd85790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006010 tx=0x7f08b40098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2327503a476b861f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08b0ff9640 1 -- 172.21.15.33:0/2377352804 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b404d070 con 0x55575bd860f0 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08b0ff9640 1 -- 172.21.15.33:0/2377352804 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b40054f0 con 0x55575bd860f0 2024-09-10T21:06:20.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08b0ff9640 1 -- 172.21.15.33:0/2377352804 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4035040 con 0x55575bd860f0 2024-09-10T21:06:20.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2377352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 msgr2=0x55575bd85790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd85790 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f08b4006010 tx=0x7f08b40098c0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2377352804 shutdown_connections 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bd95890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd86a10 0x55575bd83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/2377352804 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd860f0 0x55575bd85790 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2377352804 >> 172.21.15.33:0/2377352804 conn(0x55575bd83f20 msgr2=0x55575bd95180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2377352804 shutdown_connections 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.443+0000 7f08c3a0c740 1 -- 172.21.15.33:0/2377352804 wait complete. 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 -- start start 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 Processor -- start 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd860f0 0x55575bdb3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd86a10 0x55575bdaed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bdaf280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1ad670 con 0x55575bd86a10 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1acf10 con 0x55575bd860f0 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08bc1aa130 con 0x55575bd96db0 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08c3a0c740 10 client.0 _ll_register_callbacks cb 0x7f08b4005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd86a10 0x55575bdaed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd86a10 0x55575bdaed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50964/0 (socket says 172.21.15.33:50964) 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08ba7fc640 1 -- 172.21.15.33:0/3598365974 learned_addr learned my addr 172.21.15.33:0/3598365974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08bb7fe640 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bdaf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.444+0000 7f08baffd640 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd860f0 0x55575bdb3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08ba7fc640 1 -- 172.21.15.33:0/3598365974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 msgr2=0x55575bdaf280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bdaf280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08ba7fc640 1 -- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd860f0 msgr2=0x55575bdb3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd860f0 0x55575bdb3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08ba7fc640 1 -- 172.21.15.33:0/3598365974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55575bdaf9c0 con 0x55575bd86a10 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08ba7fc640 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd86a10 0x55575bdaed40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f08a400bd80 tx=0x7f08a400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15788743e20f3d55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08a27fc640 1 -- 172.21.15.33:0/3598365974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4004450 con 0x55575bd86a10 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08a27fc640 1 -- 172.21.15.33:0/3598365974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f08a40035f0 con 0x55575bd86a10 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08b40054f0 con 0x55575bd86a10 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08a27fc640 1 -- 172.21.15.33:0/3598365974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f08a400b4a0 con 0x55575bd86a10 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08c3a0c740 10 client.15759 fetch_fsmap learned FSMap version 33 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55575be18340 con 0x55575bd86a10 2024-09-10T21:06:20.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08a27fc640 1 -- 172.21.15.33:0/3598365974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08b40054f0 con 0x55575bd86a10 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.445+0000 7f08a27fc640 1 -- 172.21.15.33:0/3598365974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f08a40129a0 con 0x55575bd86a10 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08c3a0c740 10 client.15759 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08c3a0c740 10 client.15759 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08c3a0c740 20 client.15759 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575be18930 0x55575be1ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08c3a0c740 4 client.15759 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55575bcfa8a0 con 0x55575be18930 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08baffd640 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575be18930 0x55575be1ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08baffd640 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575be18930 0x55575be1ad50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e1d6790207ff659 server_cookie=fb0a3ee44f28ef31 in_seq=0 out_seq=0 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08a27fc640 10 client.15759 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08a27fc640 1 -- 172.21.15.33:0/3598365974 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55575bcfa8a0 con 0x55575be18930 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08a27fc640 10 client.15759 handle_command_reply: tid=0 2024-09-10T21:06:20.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.446+0000 7f08c3a0c740 1 client.15759 shutdown 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd86a10 msgr2=0x55575bdaed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd86a10 0x55575bdaed40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f08a400bd80 tx=0x7f08a400b7e0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 shutdown_connections 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55575bd96db0 0x55575bdaf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55575be18930 0x55575be1ad50 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55575bd86a10 0x55575bdaed40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 --2- 172.21.15.33:0/3598365974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55575bd860f0 0x55575bdb3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08a27fc640 0 client.15759 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 >> 172.21.15.33:0/3598365974 conn(0x55575bd83f20 msgr2=0x55575bd959a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 shutdown_connections 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 1 -- 172.21.15.33:0/3598365974 wait complete. 2024-09-10T21:06:20.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c3a0c740 20 client.15759 trim_cache size 0 max 16384 2024-09-10T21:06:20.458 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:06:20.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08a80b52e0 msgr2=0x7f08a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08a80b52e0 0x7f08a80b77a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f08b4031440 tx=0x7f08b4005c00 comp rx=0 tx=0).stop 2024-09-10T21:06:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc109150 msgr2=0x7f08bc19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc109150 0x7f08bc19eb60 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00eaa0 tx=0x7f08ac00ef60 comp rx=0 tx=0).stop 2024-09-10T21:06:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 shutdown_connections 2024-09-10T21:06:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08a80b52e0 0x7f08a80b77a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08bc109150 0x7f08bc19eb60 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08bc1087d0 0x7f08bc1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.447+0000 7f08c1637640 1 --2- 172.21.15.33:0/3834885268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08bc102dc0 0x7f08bc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:20.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.448+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 >> 172.21.15.33:0/3834885268 conn(0x7f08bc0fe830 msgr2=0x7f08bc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:20.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.448+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 shutdown_connections 2024-09-10T21:06:20.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:20.448+0000 7f08c1637640 1 -- 172.21.15.33:0/3834885268 wait complete. 2024-09-10T21:06:20.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 159645 10.86 MB/sec warmup 81 sec latency 1852.481 ms 2024-09-10T21:06:21.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:06:21.525 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:21.670 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:21 smithi033 ceph-mon[30860]: pgmap v417: 97 pgs: 97 active+clean; 81 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-09-10T21:06:21.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 159645 10.73 MB/sec warmup 82 sec latency 2852.566 ms 2024-09-10T21:06:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:21 smithi104 ceph-mon[35325]: pgmap v417: 97 pgs: 97 active+clean; 81 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-09-10T21:06:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:21 smithi174 ceph-mon[35635]: pgmap v417: 97 pgs: 97 active+clean; 81 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.3 MiB/s wr, 290 op/s 2024-09-10T21:06:22.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 -- 172.21.15.33:0/3977407991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 msgr2=0x7f2d1c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 --2- 172.21.15.33:0/3977407991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c108c10 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c009900 tx=0x7f2d0c02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 -- 172.21.15.33:0/3977407991 shutdown_connections 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 --2- 172.21.15.33:0/3977407991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d1c109150 0x7f2d1c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 --2- 172.21.15.33:0/3977407991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c108c10 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 --2- 172.21.15.33:0/3977407991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d1c102dc0 0x7f2d1c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 -- 172.21.15.33:0/3977407991 >> 172.21.15.33:0/3977407991 conn(0x7f2d1c0fe830 msgr2=0x7f2d1c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 -- 172.21.15.33:0/3977407991 shutdown_connections 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.027+0000 7f2d2357c640 1 -- 172.21.15.33:0/3977407991 wait complete. 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 Processor -- start 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 -- start start 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d1c102dc0 0x7f2d1c199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d1c109150 0x7f2d1c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c112df0 con 0x7f2d1c1087d0 2024-09-10T21:06:22.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c112c70 con 0x7f2d1c109150 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d2357c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c112f70 con 0x7f2d1c102dc0 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d20b17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d21318640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d1c102dc0 0x7f2d1c199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d20b17640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50976/0 (socket says 172.21.15.33:50976) 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d20b17640 1 -- 172.21.15.33:0/1762519463 learned_addr learned my addr 172.21.15.33:0/1762519463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.028+0000 7f2d21b19640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d1c109150 0x7f2d1c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d20b17640 1 -- 172.21.15.33:0/1762519463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d1c102dc0 msgr2=0x7f2d1c199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d20b17640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d1c102dc0 0x7f2d1c199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d20b17640 1 -- 172.21.15.33:0/1762519463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d1c109150 msgr2=0x7f2d1c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d20b17640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d1c109150 0x7f2d1c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d20b17640 1 -- 172.21.15.33:0/1762519463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c19f2f0 con 0x7f2d1c1087d0 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d20b17640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c199780 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005ab0 tx=0x7f2d0c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:22.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d127fc640 1 -- 172.21.15.33:0/1762519463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c03f070 con 0x7f2d1c1087d0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d127fc640 1 -- 172.21.15.33:0/1762519463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c031d60 con 0x7f2d1c1087d0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d127fc640 1 -- 172.21.15.33:0/1762519463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c035030 con 0x7f2d1c1087d0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c19f580 con 0x7f2d1c1087d0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.029+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c1a5ca0 con 0x7f2d1c1087d0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 Processor -- start 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 -- start start 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b1f040ea0 con 0x556b1f0ca0f0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556b1f0302d0 con 0x556b1f0caa10 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d25951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b1f0340e0 con 0x556b1f0d69a0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d127fc640 1 -- 172.21.15.33:0/1762519463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2d0c03b4f0 con 0x7f2d1c1087d0 2024-09-10T21:06:22.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d21b19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d21318640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d21318640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56888/0 (socket says 172.21.15.33:56888) 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d21b19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50988/0 (socket says 172.21.15.33:50988) 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.032+0000 7f2d21318640 1 -- 172.21.15.33:0/1833915703 learned_addr learned my addr 172.21.15.33:0/1833915703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d20b17640 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d21318640 1 -- 172.21.15.33:0/1833915703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 msgr2=0x556b1f0d8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d21318640 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d21318640 1 -- 172.21.15.33:0/1833915703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 msgr2=0x556b1f0ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d21318640 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d21318640 1 -- 172.21.15.33:0/1833915703 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1ed8e650 con 0x556b1f0caa10 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d21318640 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d6460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009cf0 tx=0x7f2d18011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c8bd6d5cf5bfacb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d077fe640 1 -- 172.21.15.33:0/1833915703 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18004230 con 0x556b1f0caa10 2024-09-10T21:06:22.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d077fe640 1 -- 172.21.15.33:0/1833915703 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d180043d0 con 0x556b1f0caa10 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d077fe640 1 -- 172.21.15.33:0/1833915703 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18019880 con 0x556b1f0caa10 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.033+0000 7f2d127fc640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d080b5360 0x7f2d080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d21b19640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d080b5360 0x7f2d080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d127fc640 1 -- 172.21.15.33:0/1762519463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2d0c1065c0 con 0x7f2d1c1087d0 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 -- 172.21.15.33:0/1833915703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 msgr2=0x556b1f0d6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d6460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009cf0 tx=0x7f2d18011810 comp rx=0 tx=0).stop 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d21b19640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d080b5360 0x7f2d080b7820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d14005fd0 tx=0x7f2d14009600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 -- 172.21.15.33:0/1833915703 shutdown_connections 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d6460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 --2- 172.21.15.33:0/1833915703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 -- 172.21.15.33:0/1833915703 >> 172.21.15.33:0/1833915703 conn(0x556b1f0c7f20 msgr2=0x556b1f0c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 -- 172.21.15.33:0/1833915703 shutdown_connections 2024-09-10T21:06:22.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 -- 172.21.15.33:0/1833915703 wait complete. 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 -- start start 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.034+0000 7f2d25951740 1 Processor -- start 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d25951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0ca0f0 0x556b1f0d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d25951740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d25951740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0d69a0 0x556b1f0d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d25951740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556b1ee91700 con 0x556b1f0d69a0 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d25951740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556b1efcb160 con 0x556b1f0caa10 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d25951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b1f0d9620 con 0x556b1f0ca0f0 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d25951740 10 client.0 _ll_register_callbacks cb 0x556b1efe49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21b19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0d69a0 0x556b1f0d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d20b17640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21318640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0ca0f0 0x556b1f0d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d20b17640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56904/0 (socket says 172.21.15.33:56904) 2024-09-10T21:06:22.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21b19640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0d69a0 0x556b1f0d3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50990/0 (socket says 172.21.15.33:50990) 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d20b17640 1 -- 172.21.15.33:0/3742199977 learned_addr learned my addr 172.21.15.33:0/3742199977 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21318640 1 -- 172.21.15.33:0/3742199977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 msgr2=0x556b1f0d3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21318640 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21318640 1 -- 172.21.15.33:0/3742199977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0d69a0 msgr2=0x556b1f0d3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21318640 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0d69a0 0x556b1f0d3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21318640 1 -- 172.21.15.33:0/3742199977 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1eff8c90 con 0x556b1f0ca0f0 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d20b17640 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.035+0000 7f2d21b19640 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0d69a0 0x556b1f0d3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d21318640 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0ca0f0 0x556b1f0d6240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d18003f50 tx=0x7f2d18003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7799ee6a57a9b1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d04ff9640 1 -- 172.21.15.33:0/3742199977 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18027070 con 0x556b1f0ca0f0 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d04ff9640 1 -- 172.21.15.33:0/3742199977 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2d18005240 con 0x556b1f0ca0f0 2024-09-10T21:06:22.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d180043d0 con 0x556b1f0ca0f0 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d04ff9640 1 -- 172.21.15.33:0/3742199977 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2d18033050 con 0x556b1f0ca0f0 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d25951740 10 client.25333 fetch_fsmap learned FSMap version 33 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b1f0e4f40 con 0x556b1f0ca0f0 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.036+0000 7f2d04ff9640 1 -- 172.21.15.33:0/3742199977 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d180043d0 con 0x556b1f0ca0f0 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d04ff9640 1 -- 172.21.15.33:0/3742199977 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f2d18004b70 con 0x556b1f0ca0f0 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d25951740 10 client.25333 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d25951740 10 client.25333 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d25951740 20 client.25333 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d25951740 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f0e62f0 0x556b1f0e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d25951740 4 client.25333 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b1f042df0 con 0x556b1f0e62f0 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d20b17640 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f0e62f0 0x556b1f0e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d04ff9640 10 client.25333 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.037+0000 7f2d20b17640 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f0e62f0 0x556b1f0e86e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=66970b2772602165 server_cookie=eb047adf3a2f64aa in_seq=0 out_seq=0 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d04ff9640 1 -- 172.21.15.33:0/3742199977 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b1f042df0 con 0x556b1f0e62f0 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d04ff9640 10 client.25333 handle_command_reply: tid=0 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 client.25333 shutdown 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0ca0f0 msgr2=0x556b1f0d6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0ca0f0 0x556b1f0d6240 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2d18003f50 tx=0x7f2d18003820 comp rx=0 tx=0).stop 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 shutdown_connections 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f0e62f0 0x556b1f0e86e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0d69a0 0x556b1f0d3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d04ff9640 0 client.25333 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 --2- 172.21.15.33:0/3742199977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0ca0f0 0x556b1f0d6240 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 >> 172.21.15.33:0/3742199977 conn(0x556b1f0c7f20 msgr2=0x556b1f0cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 shutdown_connections 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 1 -- 172.21.15.33:0/3742199977 wait complete. 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.038+0000 7f2d25951740 20 client.25333 trim_cache size 0 max 16384 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.053+0000 7f2d25951740 1 Processor -- start 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.053+0000 7f2d25951740 1 -- start start 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d25951740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d25951740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d25951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d25951740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1ae3e0 con 0x556b1f0ca0f0 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d25951740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1b0070 con 0x556b1f0caa10 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d25951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1aeab0 con 0x556b1f0d69a0 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d21318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d21318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51006/0 (socket says 172.21.15.33:51006) 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d21318640 1 -- 172.21.15.33:0/2779892575 learned_addr learned my addr 172.21.15.33:0/2779892575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d20b17640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d21b19640 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d20b17640 1 -- 172.21.15.33:0/2779892575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 msgr2=0x556b1f0d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d20b17640 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d20b17640 1 -- 172.21.15.33:0/2779892575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 msgr2=0x556b1f0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d20b17640 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d20b17640 1 -- 172.21.15.33:0/2779892575 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b1f0e8ca0 con 0x556b1f0caa10 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d21318640 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.054+0000 7f2d20b17640 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d53d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0ccdc0 tx=0x7f2d0c03b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f505abc4bc69fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d06ffd640 1 -- 172.21.15.33:0/2779892575 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c049040 con 0x556b1f0caa10 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d06ffd640 1 -- 172.21.15.33:0/2779892575 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c0cc940 con 0x556b1f0caa10 2024-09-10T21:06:22.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d06ffd640 1 -- 172.21.15.33:0/2779892575 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c0d6310 con 0x556b1f0caa10 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 -- 172.21.15.33:0/2779892575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 msgr2=0x556b1f0d53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d53d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0ccdc0 tx=0x7f2d0c03b460 comp rx=0 tx=0).stop 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 -- 172.21.15.33:0/2779892575 shutdown_connections 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0d69a0 0x556b1f0d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0caa10 0x556b1f0d53d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 --2- 172.21.15.33:0/2779892575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 -- 172.21.15.33:0/2779892575 >> 172.21.15.33:0/2779892575 conn(0x556b1f0c7f20 msgr2=0x556b1f0d4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 -- 172.21.15.33:0/2779892575 shutdown_connections 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.055+0000 7f2d25951740 1 -- 172.21.15.33:0/2779892575 wait complete. 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 -- start start 2024-09-10T21:06:22.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 Processor -- start 2024-09-10T21:06:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0daa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0caa10 0x556b1f0daf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0d69a0 0x556b1f0db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1af8e0 con 0x556b1f0ca0f0 2024-09-10T21:06:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1af180 con 0x556b1f0d69a0 2024-09-10T21:06:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.056+0000 7f2d25951740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c1ac3a0 con 0x556b1f0caa10 2024-09-10T21:06:22.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d25951740 10 client.0 _ll_register_callbacks cb 0x7f2d14009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0daa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0daa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51010/0 (socket says 172.21.15.33:51010) 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 -- 172.21.15.33:0/2768632398 learned_addr learned my addr 172.21.15.33:0/2768632398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21b19640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0d69a0 0x556b1f0db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d20b17640 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0caa10 0x556b1f0daf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 -- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0caa10 msgr2=0x556b1f0daf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0caa10 0x556b1f0daf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 -- 172.21.15.33:0/2768632398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0d69a0 msgr2=0x556b1f0db4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0d69a0 0x556b1f0db4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 -- 172.21.15.33:0/2768632398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b1f0c7ca0 con 0x556b1f0ca0f0 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21b19640 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0d69a0 0x556b1f0db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d21318640 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0daa40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2d18011d10 tx=0x7f2d18009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5d0a5c7b3419ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:22.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d04ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d04ff9640 1 -- 172.21.15.33:0/2768632398 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1803b070 con 0x556b1f0ca0f0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d04ff9640 1 -- 172.21.15.33:0/2768632398 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2d18004fc0 con 0x556b1f0ca0f0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.057+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c0cc940 con 0x556b1f0ca0f0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d04ff9640 1 -- 172.21.15.33:0/2768632398 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f2d18047050 con 0x556b1f0ca0f0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 10 client.15777 fetch_fsmap learned FSMap version 33 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x556b1f153450 con 0x556b1f0ca0f0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d04ff9640 1 -- 172.21.15.33:0/2768632398 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c0cc940 con 0x556b1f0ca0f0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d04ff9640 1 -- 172.21.15.33:0/2768632398 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 33) ==== 8390+0+0 (secure 0 0 0) 0x7f2d18005250 con 0x556b1f0ca0f0 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 10 client.15777 fetch_fsmap finished waiting for FSMap version 33 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 10 client.15777 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:06:22.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 20 client.15777 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f153b70 0x556b1f155f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 4 client.15777 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:06:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b1f03e8a0 con 0x556b1f153b70 2024-09-10T21:06:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.058+0000 7f2d20b17640 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f153b70 0x556b1f155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d20b17640 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f153b70 0x556b1f155f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2928c119409849be server_cookie=b449cb9be0887537 in_seq=0 out_seq=0 2024-09-10T21:06:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d04ff9640 10 client.15777 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:06:22.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d04ff9640 1 -- 172.21.15.33:0/2768632398 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b1f03e8a0 con 0x556b1f153b70 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d04ff9640 10 client.15777 handle_command_reply: tid=0 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 client.15777 shutdown 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 msgr2=0x556b1f0daa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0daa40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2d18011d10 tx=0x7f2d18009ea0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 shutdown_connections 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556b1f0d69a0 0x556b1f0db4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b1f0caa10 0x556b1f0daf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x556b1f153b70 0x556b1f155f90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 --2- 172.21.15.33:0/2768632398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556b1f0ca0f0 0x556b1f0daa40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d04ff9640 0 client.15777 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 >> 172.21.15.33:0/2768632398 conn(0x556b1f0c7f20 msgr2=0x556b1f0d54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 shutdown_connections 2024-09-10T21:06:22.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 1 -- 172.21.15.33:0/2768632398 wait complete. 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.059+0000 7f2d25951740 20 client.15777 trim_cache size 0 max 16384 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d080b5360 msgr2=0x7f2d080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d080b5360 0x7f2d080b7820 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d14005fd0 tx=0x7f2d14009600 comp rx=0 tx=0).stop 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 msgr2=0x7f2d1c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c199780 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005ab0 tx=0x7f2d0c003c80 comp rx=0 tx=0).stop 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 shutdown_connections 2024-09-10T21:06:22.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d080b5360 0x7f2d080b7820 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d1c109150 0x7f2d1c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d1c1087d0 0x7f2d1c199780 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 --2- 172.21.15.33:0/1762519463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d1c102dc0 0x7f2d1c199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 >> 172.21.15.33:0/1762519463 conn(0x7f2d1c0fe830 msgr2=0x7f2d1c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 shutdown_connections 2024-09-10T21:06:22.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:22.060+0000 7f2d2357c640 1 -- 172.21.15.33:0/1762519463 wait complete. 2024-09-10T21:06:22.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 161092 10.68 MB/sec warmup 83 sec latency 2916.039 ms 2024-09-10T21:06:22.961 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:06:23.254 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:23.651 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:23 smithi033 ceph-mon[30860]: pgmap v418: 97 pgs: 97 active+clean; 82 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 278 op/s 2024-09-10T21:06:23.651 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:23 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:23.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 163555 10.72 MB/sec warmup 84 sec latency 37.402 ms 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 -- 172.21.15.33:0/2548057559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a0109150 msgr2=0x7f15a010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 --2- 172.21.15.33:0/2548057559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a0109150 0x7f15a010f950 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1598009ca0 tx=0x7f159802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 -- 172.21.15.33:0/2548057559 shutdown_connections 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 --2- 172.21.15.33:0/2548057559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a0109150 0x7f15a010f950 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 --2- 172.21.15.33:0/2548057559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15a01087d0 0x7f15a0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 --2- 172.21.15.33:0/2548057559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0102dc0 0x7f15a01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 -- 172.21.15.33:0/2548057559 >> 172.21.15.33:0/2548057559 conn(0x7f15a00fe830 msgr2=0x7f15a0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:23.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 -- 172.21.15.33:0/2548057559 shutdown_connections 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.733+0000 7f15a5f24640 1 -- 172.21.15.33:0/2548057559 wait complete. 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 Processor -- start 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 -- start start 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0102dc0 0x7f15a0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a01087d0 0x7f15a0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15a0109150 0x7f15a019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0112dd0 con 0x7f15a01087d0 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0112c50 con 0x7f15a0102dc0 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.734+0000 7f15a5f24640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15a0112f50 con 0x7f15a0109150 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a01087d0 0x7f15a0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0102dc0 0x7f15a0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a01087d0 0x7f15a0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51032/0 (socket says 172.21.15.33:51032) 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159effd640 1 -- 172.21.15.33:0/599195090 learned_addr learned my addr 172.21.15.33:0/599195090 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159ffff640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15a0109150 0x7f15a019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159f7fe640 1 -- 172.21.15.33:0/599195090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15a0109150 msgr2=0x7f15a019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159f7fe640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15a0109150 0x7f15a019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159f7fe640 1 -- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a01087d0 msgr2=0x7f15a0199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159f7fe640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a01087d0 0x7f15a0199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159f7fe640 1 -- 172.21.15.33:0/599195090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a019f2e0 con 0x7f15a0102dc0 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159effd640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a01087d0 0x7f15a0199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159f7fe640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0102dc0 0x7f15a0199230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f159000e9f0 tx=0x7f159000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159cff9640 1 -- 172.21.15.33:0/599195090 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159000ccc0 con 0x7f15a0102dc0 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159cff9640 1 -- 172.21.15.33:0/599195090 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15900108e0 con 0x7f15a0102dc0 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a019f5d0 con 0x7f15a0102dc0 2024-09-10T21:06:23.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.735+0000 7f159cff9640 1 -- 172.21.15.33:0/599195090 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1590018880 con 0x7f15a0102dc0 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.736+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a01a5c90 con 0x7f15a0102dc0 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 Processor -- start 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f159cff9640 1 -- 172.21.15.33:0/599195090 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1590010430 con 0x7f15a0102dc0 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 -- start start 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6f0f0 0x56156dd6f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd7b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd7dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56156dce5ea0 con 0x56156dd7b9a0 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56156dcd52d0 con 0x56156dd6f0f0 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f15a82f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56156dcd90e0 con 0x56156dd6fa10 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f159effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd7dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f159ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6f0f0 0x56156dd6f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd7b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f159effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd7dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51044/0 (socket says 172.21.15.33:51044) 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.738+0000 7f159ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6f0f0 0x56156dd6f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56962/0 (socket says 172.21.15.33:56962) 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159effd640 1 -- 172.21.15.33:0/4048122950 learned_addr learned my addr 172.21.15.33:0/4048122950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159effd640 1 -- 172.21.15.33:0/4048122950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 msgr2=0x56156dd7b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159effd640 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd7b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159effd640 1 -- 172.21.15.33:0/4048122950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6f0f0 msgr2=0x56156dd6f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159effd640 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6f0f0 0x56156dd6f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159effd640 1 -- 172.21.15.33:0/4048122950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56156da33650 con 0x56156dd7b9a0 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159ffff640 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6f0f0 0x56156dd6f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159effd640 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd7dd90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f15940115f0 tx=0x7f1594011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5fa24197afcf9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159cff9640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f15880b52e0 0x7f15880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f1585ffb640 1 -- 172.21.15.33:0/4048122950 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159401e070 con 0x56156dd7b9a0 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f1585ffb640 1 -- 172.21.15.33:0/4048122950 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400f660 con 0x56156dd7b9a0 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f1585ffb640 1 -- 172.21.15.33:0/4048122950 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594003850 con 0x56156dd7b9a0 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159ffff640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f15880b52e0 0x7f15880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.739+0000 7f159cff9640 1 -- 172.21.15.33:0/599195090 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f15900e5f30 con 0x7f15a0102dc0 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 -- 172.21.15.33:0/4048122950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 msgr2=0x56156dd7dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd7dd90 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f15940115f0 tx=0x7f1594011ab0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f159ffff640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f15880b52e0 0x7f15880b77a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f159802f500 tx=0x7f1598005f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 -- 172.21.15.33:0/4048122950 shutdown_connections 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd7dd90 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd7b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 --2- 172.21.15.33:0/4048122950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6f0f0 0x56156dd6f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 -- 172.21.15.33:0/4048122950 >> 172.21.15.33:0/4048122950 conn(0x56156dd6cf20 msgr2=0x56156dd6e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:23.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 -- 172.21.15.33:0/4048122950 shutdown_connections 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 -- 172.21.15.33:0/4048122950 wait complete. 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.740+0000 7f15a82f9740 1 -- start start 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 1 Processor -- start 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6f0f0 0x56156dd7b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd78710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd78d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56156db36700 con 0x56156dd6f0f0 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56156dc70160 con 0x56156dd7b9a0 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56156dd7e620 con 0x56156dd6fa10 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f15a82f9740 10 client.0 _ll_register_callbacks cb 0x56156dc899e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6f0f0 0x56156dd7b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd78d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6f0f0 0x56156dd7b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51060/0 (socket says 172.21.15.33:51060) 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159f7fe640 1 -- 172.21.15.33:0/3405433506 learned_addr learned my addr 172.21.15.33:0/3405433506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159effd640 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd78710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159f7fe640 1 -- 172.21.15.33:0/3405433506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 msgr2=0x56156dd78710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159f7fe640 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd78710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.741+0000 7f159f7fe640 1 -- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 msgr2=0x56156dd78d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f159f7fe640 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd78d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f159f7fe640 1 -- 172.21.15.33:0/3405433506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56156dc9dc90 con 0x56156dd6f0f0 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f159ffff640 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd78d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f159f7fe640 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6f0f0 0x56156dd7b2c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f15900a8d30 tx=0x7f159000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd6a4b3cc573b75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f157effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:23.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f157effd640 1 -- 172.21.15.33:0/3405433506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15900a6e60 con 0x56156dd6f0f0 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f157effd640 1 -- 172.21.15.33:0/3405433506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f15900e9230 con 0x56156dd6f0f0 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159400f660 con 0x56156dd6f0f0 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f157effd640 1 -- 172.21.15.33:0/3405433506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f15900ac530 con 0x56156dd6f0f0 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f15a82f9740 10 client.15789 fetch_fsmap learned FSMap version 34 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56156dd89f80 con 0x56156dd6f0f0 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f157effd640 1 -- 172.21.15.33:0/3405433506 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159400f660 con 0x56156dd6f0f0 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.742+0000 7f157effd640 1 -- 172.21.15.33:0/3405433506 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f15900a80b0 con 0x56156dd6f0f0 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f15a82f9740 10 client.15789 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f15a82f9740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f15a82f9740 20 client.15789 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:23.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f15a82f9740 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156dd8b480 0x56156dd8d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f15a82f9740 4 client.15789 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56156dce7df0 con 0x56156dd8b480 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f159effd640 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156dd8b480 0x56156dd8d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f159effd640 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156dd8b480 0x56156dd8d940 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bbdc8c7a29ccd1c server_cookie=e4c41e327c284ee4 in_seq=0 out_seq=0 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.743+0000 7f157effd640 10 client.15789 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f157effd640 1 -- 172.21.15.33:0/3405433506 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56156dce7df0 con 0x56156dd8b480 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f157effd640 10 client.15789 handle_command_reply: tid=0 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 client.15789 shutdown 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6f0f0 msgr2=0x56156dd7b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6f0f0 0x56156dd7b2c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f15900a8d30 tx=0x7f159000ce60 comp rx=0 tx=0).stop 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 shutdown_connections 2024-09-10T21:06:23.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156dd8b480 0x56156dd8d940 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd78d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6fa10 0x56156dd78710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f157effd640 0 client.15789 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 --2- 172.21.15.33:0/3405433506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6f0f0 0x56156dd7b2c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 >> 172.21.15.33:0/3405433506 conn(0x56156dd6cf20 msgr2=0x56156dd72a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 shutdown_connections 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 1 -- 172.21.15.33:0/3405433506 wait complete. 2024-09-10T21:06:23.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.744+0000 7f15a82f9740 20 client.15789 trim_cache size 0 max 16384 2024-09-10T21:06:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.759+0000 7f15a82f9740 1 Processor -- start 2024-09-10T21:06:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.759+0000 7f15a82f9740 1 -- start start 2024-09-10T21:06:23.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd87850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6fa10 0x56156dd77c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd76f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f15a82f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01ae3f0 con 0x56156dd6fa10 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f15a82f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01b00b0 con 0x56156dd7b9a0 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f15a82f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01aeac0 con 0x56156dd6f0f0 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6fa10 0x56156dd77c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6fa10 0x56156dd77c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51064/0 (socket says 172.21.15.33:51064) 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 -- 172.21.15.33:0/2155388493 learned_addr learned my addr 172.21.15.33:0/2155388493 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd76f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159f7fe640 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd87850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 -- 172.21.15.33:0/2155388493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 msgr2=0x56156dd87850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd87850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 -- 172.21.15.33:0/2155388493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 msgr2=0x56156dd76f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd76f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 -- 172.21.15.33:0/2155388493 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56156dd8dfc0 con 0x56156dd6fa10 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159ffff640 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd76f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f159effd640 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6fa10 0x56156dd77c00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622104fdc31ad6ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.760+0000 7f15857fa640 1 -- 172.21.15.33:0/2155388493 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594031070 con 0x56156dd6fa10 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15857fa640 1 -- 172.21.15.33:0/2155388493 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159401f040 con 0x56156dd6fa10 2024-09-10T21:06:23.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15857fa640 1 -- 172.21.15.33:0/2155388493 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159401e040 con 0x56156dd6fa10 2024-09-10T21:06:23.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 -- 172.21.15.33:0/2155388493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6fa10 msgr2=0x56156dd77c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6fa10 0x56156dd77c00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1594000c00 tx=0x7f1594009c50 comp rx=0 tx=0).stop 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 -- 172.21.15.33:0/2155388493 shutdown_connections 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd7b9a0 0x56156dd76f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd6fa10 0x56156dd77c00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 --2- 172.21.15.33:0/2155388493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd87850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 -- 172.21.15.33:0/2155388493 >> 172.21.15.33:0/2155388493 conn(0x56156dd6cf20 msgr2=0x56156dd6dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 -- 172.21.15.33:0/2155388493 shutdown_connections 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.761+0000 7f15a82f9740 1 -- 172.21.15.33:0/2155388493 wait complete. 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 -- start start 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 Processor -- start 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd84170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6fa10 0x56156dd75750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd75dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01af920 con 0x56156dd7b9a0 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01af190 con 0x56156dd6fa10 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15a01ac3b0 con 0x56156dd6f0f0 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f15a82f9740 10 client.0 _ll_register_callbacks cb 0x7f15900e9660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f159ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd75dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f159ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd75dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51072/0 (socket says 172.21.15.33:51072) 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f159f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd84170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f159ffff640 1 -- 172.21.15.33:0/279726094 learned_addr learned my addr 172.21.15.33:0/279726094 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.762+0000 7f159effd640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6fa10 0x56156dd75750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159effd640 1 -- 172.21.15.33:0/279726094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 msgr2=0x56156dd84170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159effd640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd84170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159effd640 1 -- 172.21.15.33:0/279726094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 msgr2=0x56156dd75dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159effd640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd75dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159effd640 1 -- 172.21.15.33:0/279726094 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56156dd82dd0 con 0x56156dd6fa10 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159ffff640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd75dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159f7fe640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd84170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f159effd640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6fa10 0x56156dd75750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1594011510 tx=0x7f159400f4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b8a625a865684a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f157effd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f157effd640 1 -- 172.21.15.33:0/279726094 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594031070 con 0x56156dd6fa10 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f159401f040 con 0x56156dd6fa10 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f157effd640 1 -- 172.21.15.33:0/279726094 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f159400fb20 con 0x56156dd6fa10 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f157effd640 1 -- 172.21.15.33:0/279726094 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f159401e040 con 0x56156dd6fa10 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f15a82f9740 10 client.25370 fetch_fsmap learned FSMap version 34 2024-09-10T21:06:23.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.763+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56156ddfca00 con 0x56156dd6fa10 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f157effd640 1 -- 172.21.15.33:0/279726094 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f159401f040 con 0x56156dd6fa10 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f157effd640 1 -- 172.21.15.33:0/279726094 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f15940394f0 con 0x56156dd6fa10 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f15a82f9740 10 client.25370 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f15a82f9740 10 client.25370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f15a82f9740 20 client.25370 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f15a82f9740 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156ddfcf60 0x56156ddff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f15a82f9740 4 client.25370 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56156dce38a0 con 0x56156ddfcf60 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f159f7fe640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156ddfcf60 0x56156ddff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f159f7fe640 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156ddfcf60 0x56156ddff380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2fa0dd4f15f6778 server_cookie=f56e996a8a7da62c in_seq=0 out_seq=0 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.764+0000 7f157effd640 10 client.25370 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:23.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f157effd640 1 -- 172.21.15.33:0/279726094 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56156dce38a0 con 0x56156ddfcf60 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f157effd640 10 client.25370 handle_command_reply: tid=0 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 client.25370 shutdown 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6fa10 msgr2=0x56156dd75750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6fa10 0x56156dd75750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1594011510 tx=0x7f159400f4d0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 shutdown_connections 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56156ddfcf60 0x56156ddff380 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56156dd7b9a0 0x56156dd75dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56156dd6fa10 0x56156dd75750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 --2- 172.21.15.33:0/279726094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56156dd6f0f0 0x56156dd84170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f157effd640 0 client.25370 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 >> 172.21.15.33:0/279726094 conn(0x56156dd6cf20 msgr2=0x56156dd76ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 shutdown_connections 2024-09-10T21:06:23.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 1 -- 172.21.15.33:0/279726094 wait complete. 2024-09-10T21:06:23.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a82f9740 20 client.25370 trim_cache size 0 max 16384 2024-09-10T21:06:23.776 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:06:23.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.765+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f15880b52e0 msgr2=0x7f15880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f15880b52e0 0x7f15880b77a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f159802f500 tx=0x7f1598005f50 comp rx=0 tx=0).stop 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0102dc0 msgr2=0x7f15a0199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0102dc0 0x7f15a0199230 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f159000e9f0 tx=0x7f159000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 shutdown_connections 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f15880b52e0 0x7f15880b77a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15a0109150 0x7f15a019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f15a01087d0 0x7f15a0199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 --2- 172.21.15.33:0/599195090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f15a0102dc0 0x7f15a0199230 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 >> 172.21.15.33:0/599195090 conn(0x7f15a00fe830 msgr2=0x7f15a00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 shutdown_connections 2024-09-10T21:06:23.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:23.766+0000 7f15a5f24640 1 -- 172.21.15.33:0/599195090 wait complete. 2024-09-10T21:06:23.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:23 smithi104 ceph-mon[35325]: pgmap v418: 97 pgs: 97 active+clean; 82 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 278 op/s 2024-09-10T21:06:23.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:23 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:23 smithi174 ceph-mon[35635]: pgmap v418: 97 pgs: 97 active+clean; 82 MiB data, 748 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.2 MiB/s wr, 278 op/s 2024-09-10T21:06:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:23 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:06:24.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 165900 10.72 MB/sec warmup 85 sec latency 29.725 ms 2024-09-10T21:06:25.537 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:06:25.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:06:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:25 smithi033 ceph-mon[30860]: pgmap v419: 97 pgs: 97 active+clean; 88 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-09-10T21:06:25.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 167676 10.70 MB/sec warmup 86 sec latency 10.997 ms 2024-09-10T21:06:25.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:25 smithi104 ceph-mon[35325]: pgmap v419: 97 pgs: 97 active+clean; 88 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-09-10T21:06:25.817 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:25.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:25 smithi174 ceph-mon[35635]: pgmap v419: 97 pgs: 97 active+clean; 88 MiB data, 752 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.7 MiB/s wr, 279 op/s 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 -- 172.21.15.33:0/1774712000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c101870 msgr2=0x7ff06c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 --2- 172.21.15.33:0/1774712000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c101870 0x7ff06c101c50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009900 tx=0x7ff05c02a410 comp rx=0 tx=0).stop 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 -- 172.21.15.33:0/1774712000 shutdown_connections 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 --2- 172.21.15.33:0/1774712000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c10f420 0x7ff06c111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 --2- 172.21.15.33:0/1774712000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff06c102190 0x7ff06c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 --2- 172.21.15.33:0/1774712000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c101870 0x7ff06c101c50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 -- 172.21.15.33:0/1774712000 >> 172.21.15.33:0/1774712000 conn(0x7ff06c0fd700 msgr2=0x7ff06c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 -- 172.21.15.33:0/1774712000 shutdown_connections 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.289+0000 7ff07279f640 1 -- 172.21.15.33:0/1774712000 wait complete. 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.290+0000 7ff07279f640 1 Processor -- start 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.290+0000 7ff07279f640 1 -- start start 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff07279f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c101870 0x7ff06c19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff07279f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c102190 0x7ff06c19ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff07279f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff06c10f420 0x7ff06c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff07279f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c114f90 con 0x7ff06c102190 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff07279f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c114e10 con 0x7ff06c10f420 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff07279f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c115110 con 0x7ff06c101870 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c102190 0x7ff06c19ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c102190 0x7ff06c19ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51090/0 (socket says 172.21.15.33:51090) 2024-09-10T21:06:26.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff070d3c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff06c10f420 0x7ff06c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 -- 172.21.15.33:0/2112016548 learned_addr learned my addr 172.21.15.33:0/2112016548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:26.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06bfff640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c101870 0x7ff06c19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 -- 172.21.15.33:0/2112016548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c101870 msgr2=0x7ff06c19b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c101870 0x7ff06c19b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 -- 172.21.15.33:0/2112016548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff06c10f420 msgr2=0x7ff06c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff06c10f420 0x7ff06c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff06b7fe640 1 -- 172.21.15.33:0/2112016548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff06c1a1570 con 0x7ff06c102190 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.291+0000 7ff070d3c640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff06c10f420 0x7ff06c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.292+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c102190 0x7ff06c19ba00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e9f0 tx=0x7ff06000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.292+0000 7ff0697fa640 1 -- 172.21.15.33:0/2112016548 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06000cd80 con 0x7ff06c102190 2024-09-10T21:06:26.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.292+0000 7ff0697fa640 1 -- 172.21.15.33:0/2112016548 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0600108e0 con 0x7ff06c102190 2024-09-10T21:06:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.292+0000 7ff0697fa640 1 -- 172.21.15.33:0/2112016548 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff060018980 con 0x7ff06c102190 2024-09-10T21:06:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.292+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff06c1a1860 con 0x7ff06c102190 2024-09-10T21:06:26.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.292+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff06c1a7f20 con 0x7ff06c102190 2024-09-10T21:06:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff0697fa640 1 -- 172.21.15.33:0/2112016548 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff060010430 con 0x7ff06c102190 2024-09-10T21:06:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 Processor -- start 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 -- start start 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f39ee0 0x564513f3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f46250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f46790 0x564513f48b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564513eb0e60 con 0x564513f3a800 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564513ea0290 con 0x564513f46790 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.295+0000 7ff074b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564513ea40a0 con 0x564513f39ee0 2024-09-10T21:06:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f46250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f46790 0x564513f48b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f46250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51098/0 (socket says 172.21.15.33:51098) 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 -- 172.21.15.33:0/4195262253 learned_addr learned my addr 172.21.15.33:0/4195262253 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff070d3c640 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f39ee0 0x564513f3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff0697fa640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0540b5360 0x7ff0540b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff070d3c640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0540b5360 0x7ff0540b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 -- 172.21.15.33:0/4195262253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f39ee0 msgr2=0x564513f3a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f39ee0 0x564513f3a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 -- 172.21.15.33:0/4195262253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f46790 msgr2=0x564513f48b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f46790 0x564513f48b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 -- 172.21.15.33:0/4195262253 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564513bfe930 con 0x564513f3a800 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff070d3c640 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f39ee0 0x564513f3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:06:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff0697fa640 1 -- 172.21.15.33:0/2112016548 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff0600e5e30 con 0x7ff06c102190 2024-09-10T21:06:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff06bfff640 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f46250 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff05c033000 tx=0x7ff05c032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415483d821e6ee0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.296+0000 7ff0527fc640 1 -- 172.21.15.33:0/4195262253 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035040 con 0x564513f3a800 2024-09-10T21:06:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff070d3c640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0540b5360 0x7ff0540b7820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff06c102850 tx=0x7ff064004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff0527fc640 1 -- 172.21.15.33:0/4195262253 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c004030 con 0x564513f3a800 2024-09-10T21:06:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff0527fc640 1 -- 172.21.15.33:0/4195262253 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0054f0 con 0x564513f3a800 2024-09-10T21:06:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 -- 172.21.15.33:0/4195262253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 msgr2=0x564513f46250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f46250 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff05c033000 tx=0x7ff05c032b20 comp rx=0 tx=0).stop 2024-09-10T21:06:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 -- 172.21.15.33:0/4195262253 shutdown_connections 2024-09-10T21:06:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f46790 0x564513f48b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f46250 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 --2- 172.21.15.33:0/4195262253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f39ee0 0x564513f3a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 -- 172.21.15.33:0/4195262253 >> 172.21.15.33:0/4195262253 conn(0x564513f37d10 msgr2=0x564513f38eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 -- 172.21.15.33:0/4195262253 shutdown_connections 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.297+0000 7ff074b74740 1 -- 172.21.15.33:0/4195262253 wait complete. 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 -- start start 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 Processor -- start 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f437a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f43ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f44220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564513d01890 con 0x564513f3a800 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564513e3ad40 con 0x564513f39ee0 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564513f493e0 con 0x564513f46790 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.298+0000 7ff074b74740 10 client.0 _ll_register_callbacks cb 0x564513e9ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f43ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f437a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f43ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51100/0 (socket says 172.21.15.33:51100) 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 -- 172.21.15.33:0/1911095797 learned_addr learned my addr 172.21.15.33:0/1911095797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff070d3c640 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f44220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 -- 172.21.15.33:0/1911095797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 msgr2=0x564513f44220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f44220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 -- 172.21.15.33:0/1911095797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 msgr2=0x564513f437a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f437a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 -- 172.21.15.33:0/1911095797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564513f52810 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f43ce0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff0600a9910 tx=0x7ff060018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a19afe970731b7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff04b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff04b7fe640 1 -- 172.21.15.33:0/1911095797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0600abb00 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c004030 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.299+0000 7ff04b7fe640 1 -- 172.21.15.33:0/1911095797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff060018b20 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff04b7fe640 1 -- 172.21.15.33:0/1911095797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff0600ac1b0 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 10 client.15807 fetch_fsmap learned FSMap version 34 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564513f55210 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff04b7fe640 1 -- 172.21.15.33:0/1911095797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c004030 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff04b7fe640 1 -- 172.21.15.33:0/1911095797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7ff0600ab370 con 0x564513f3a800 2024-09-10T21:06:26.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 10 client.15807 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 10 client.15807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 20 client.15807 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513f56620 0x564513f58a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 4 client.15807 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564513eb2db0 con 0x564513f56620 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.300+0000 7ff06bfff640 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513f56620 0x564513f58a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.301+0000 7ff04b7fe640 10 client.15807 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.301+0000 7ff06bfff640 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513f56620 0x564513f58a50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfb8a2c266ab6592 server_cookie=130b10982ab0e91f in_seq=0 out_seq=0 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.301+0000 7ff04b7fe640 1 -- 172.21.15.33:0/1911095797 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564513eb2db0 con 0x564513f56620 2024-09-10T21:06:26.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.301+0000 7ff04b7fe640 10 client.15807 handle_command_reply: tid=0 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.301+0000 7ff074b74740 1 client.15807 shutdown 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 msgr2=0x564513f43ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f43ce0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff0600a9910 tx=0x7ff060018420 comp rx=0 tx=0).stop 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 shutdown_connections 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513f56620 0x564513f58a50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff04b7fe640 0 client.15807 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f44220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f3a800 0x564513f43ce0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 --2- 172.21.15.33:0/1911095797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f437a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 >> 172.21.15.33:0/1911095797 conn(0x564513f37d10 msgr2=0x564513f36f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 shutdown_connections 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 1 -- 172.21.15.33:0/1911095797 wait complete. 2024-09-10T21:06:26.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.302+0000 7ff074b74740 20 client.15807 trim_cache size 0 max 16384 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 Processor -- start 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 -- start start 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f39ee0 0x564513f412d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f3a800 0x564513f5e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f5e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c1b0640 con 0x564513f39ee0 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c1b2300 con 0x564513f3a800 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.321+0000 7ff074b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c1b0d10 con 0x564513f46790 2024-09-10T21:06:26.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f39ee0 0x564513f412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f39ee0 0x564513f412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51112/0 (socket says 172.21.15.33:51112) 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff070d3c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f5e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 -- 172.21.15.33:0/222648931 learned_addr learned my addr 172.21.15.33:0/222648931 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f3a800 0x564513f5e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 -- 172.21.15.33:0/222648931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 msgr2=0x564513f5e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f5e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 -- 172.21.15.33:0/222648931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f3a800 msgr2=0x564513f5e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f3a800 0x564513f5e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 -- 172.21.15.33:0/222648931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564513f33640 con 0x564513f39ee0 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff070d3c640 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f5e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f3a800 0x564513f5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:06:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff06bfff640 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f39ee0 0x564513f412d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x564513f3d060 tx=0x7ff05c005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37ddc03c9fa6a76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.322+0000 7ff051ffb640 1 -- 172.21.15.33:0/222648931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c052070 con 0x564513f39ee0 2024-09-10T21:06:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff051ffb640 1 -- 172.21.15.33:0/222648931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c005400 con 0x564513f39ee0 2024-09-10T21:06:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff051ffb640 1 -- 172.21.15.33:0/222648931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c035040 con 0x564513f39ee0 2024-09-10T21:06:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff074b74740 1 -- 172.21.15.33:0/222648931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f39ee0 msgr2=0x564513f412d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff074b74740 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f39ee0 0x564513f412d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x564513f3d060 tx=0x7ff05c005050 comp rx=0 tx=0).stop 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff074b74740 1 -- 172.21.15.33:0/222648931 shutdown_connections 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff074b74740 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f46790 0x564513f5e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff074b74740 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f3a800 0x564513f5e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff074b74740 1 --2- 172.21.15.33:0/222648931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f39ee0 0x564513f412d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.323+0000 7ff074b74740 1 -- 172.21.15.33:0/222648931 >> 172.21.15.33:0/222648931 conn(0x564513f37d10 msgr2=0x564513f40bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 -- 172.21.15.33:0/222648931 shutdown_connections 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 -- 172.21.15.33:0/222648931 wait complete. 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 -- start start 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 Processor -- start 2024-09-10T21:06:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f4ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f3a800 0x564513f4c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f46790 0x564513f4cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c1b1b70 con 0x564513f46790 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c1b13e0 con 0x564513f39ee0 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff06c1ae600 con 0x564513f3a800 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.324+0000 7ff074b74740 10 client.0 _ll_register_callbacks cb 0x7ff060009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f4ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff070d3c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f46790 0x564513f4cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f4ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57064/0 (socket says 172.21.15.33:57064) 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 -- 172.21.15.33:0/905734591 learned_addr learned my addr 172.21.15.33:0/905734591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f3a800 0x564513f4c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 -- 172.21.15.33:0/905734591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f3a800 msgr2=0x564513f4c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f3a800 0x564513f4c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 -- 172.21.15.33:0/905734591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f46790 msgr2=0x564513f4cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f46790 0x564513f4cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 -- 172.21.15.33:0/905734591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564513f4d620 con 0x564513f39ee0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff06bfff640 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f4ee20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff05c005e80 tx=0x7ff05c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e54f4b33dee76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff04b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff04b7fe640 1 -- 172.21.15.33:0/905734591 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c052040 con 0x564513f39ee0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff04b7fe640 1 -- 172.21.15.33:0/905734591 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff05c004c60 con 0x564513f39ee0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.325+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff05c005400 con 0x564513f39ee0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff04b7fe640 1 -- 172.21.15.33:0/905734591 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff05c035040 con 0x564513f39ee0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 10 client.25388 fetch_fsmap learned FSMap version 34 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x564513fc4590 con 0x564513f39ee0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff04b7fe640 1 -- 172.21.15.33:0/905734591 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff05c005400 con 0x564513f39ee0 2024-09-10T21:06:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff04b7fe640 1 -- 172.21.15.33:0/905734591 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7ff05c0037f0 con 0x564513f39ee0 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 10 client.25388 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 20 client.25388 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513fc48e0 0x564513fc6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 4 client.25388 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564513eae860 con 0x564513fc48e0 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.326+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513fc48e0 0x564513fc6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.327+0000 7ff06b7fe640 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513fc48e0 0x564513fc6cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ab95d77146de43 server_cookie=6eeea73cb0920705 in_seq=0 out_seq=0 2024-09-10T21:06:26.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.327+0000 7ff04b7fe640 10 client.25388 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:26.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff04b7fe640 1 -- 172.21.15.33:0/905734591 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564513eae860 con 0x564513fc48e0 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff04b7fe640 10 client.25388 handle_command_reply: tid=0 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 client.25388 shutdown 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 msgr2=0x564513f4ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f4ee20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff05c005e80 tx=0x7ff05c005fd0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 shutdown_connections 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564513fc48e0 0x564513fc6cd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564513f46790 0x564513f4cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564513f3a800 0x564513f4c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff04b7fe640 0 client.25388 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 --2- 172.21.15.33:0/905734591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564513f39ee0 0x564513f4ee20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 >> 172.21.15.33:0/905734591 conn(0x564513f37d10 msgr2=0x564513f36750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 shutdown_connections 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.328+0000 7ff074b74740 1 -- 172.21.15.33:0/905734591 wait complete. 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff074b74740 20 client.25388 trim_cache size 0 max 16384 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e", 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:06:26.338 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:06:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0540b5360 msgr2=0x7ff0540b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0540b5360 0x7ff0540b7820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7ff06c102850 tx=0x7ff064004ab0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c102190 msgr2=0x7ff06c19ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c102190 0x7ff06c19ba00 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e9f0 tx=0x7ff06000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 shutdown_connections 2024-09-10T21:06:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0540b5360 0x7ff0540b7820 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff06c10f420 0x7ff06c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff06c102190 0x7ff06c19ba00 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 --2- 172.21.15.33:0/2112016548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff06c101870 0x7ff06c19b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 >> 172.21.15.33:0/2112016548 conn(0x7ff06c0fd700 msgr2=0x7ff06c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 shutdown_connections 2024-09-10T21:06:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:26.329+0000 7ff07279f640 1 -- 172.21.15.33:0/2112016548 wait complete. 2024-09-10T21:06:26.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 170392 10.78 MB/sec warmup 87 sec latency 37.782 ms 2024-09-10T21:06:27.219 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:06:27.500 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: pgmap v420: 97 pgs: 97 active+clean; 88 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 231 op/s 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.623 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:06:27.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.624 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: pgmap v420: 97 pgs: 97 active+clean; 88 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 231 op/s 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 172570 10.75 MB/sec warmup 88 sec latency 12.377 ms 2024-09-10T21:06:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: pgmap v420: 97 pgs: 97 active+clean; 88 MiB data, 757 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 231 op/s 2024-09-10T21:06:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:06:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:27.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:06:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.147+0000 7f75d759e640 1 -- 172.21.15.33:0/1762098929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 msgr2=0x7f75d806eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.147+0000 7f75d759e640 1 --2- 172.21.15.33:0/1762098929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 0x7f75d806eb10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009900 tx=0x7f75c802a440 comp rx=0 tx=0).stop 2024-09-10T21:06:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.148+0000 7f75d759e640 1 -- 172.21.15.33:0/1762098929 shutdown_connections 2024-09-10T21:06:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.148+0000 7f75d759e640 1 --2- 172.21.15.33:0/1762098929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d811c020 0x7f75d811e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.148+0000 7f75d759e640 1 --2- 172.21.15.33:0/1762098929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d806f050 0x7f75d811bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.148+0000 7f75d759e640 1 --2- 172.21.15.33:0/1762098929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 0x7f75d806eb10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.148+0000 7f75d759e640 1 -- 172.21.15.33:0/1762098929 >> 172.21.15.33:0/1762098929 conn(0x7f75d806c950 msgr2=0x7f75d806cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.148+0000 7f75d759e640 1 -- 172.21.15.33:0/1762098929 shutdown_connections 2024-09-10T21:06:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.148+0000 7f75d759e640 1 -- 172.21.15.33:0/1762098929 wait complete. 2024-09-10T21:06:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.149+0000 7f75d759e640 1 Processor -- start 2024-09-10T21:06:28.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.149+0000 7f75d759e640 1 -- start start 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.149+0000 7f75d759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 0x7f75d81a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.149+0000 7f75d759e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d806f050 0x7f75d81a81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.149+0000 7f75d759e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d811c020 0x7f75d81ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d759e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8121a60 con 0x7f75d806e730 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d759e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81218e0 con 0x7f75d806f050 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 0x7f75d81a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 0x7f75d81a7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51144/0 (socket says 172.21.15.33:51144) 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d806f050 0x7f75d81a81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d659c640 1 -- 172.21.15.33:0/1020822104 learned_addr learned my addr 172.21.15.33:0/1020822104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8121be0 con 0x7f75d811c020 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d811c020 0x7f75d81ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d5d9b640 1 -- 172.21.15.33:0/1020822104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d811c020 msgr2=0x7f75d81ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d811c020 0x7f75d81ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d5d9b640 1 -- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 msgr2=0x7f75d81a7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 0x7f75d81a7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d5d9b640 1 -- 172.21.15.33:0/1020822104 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d81add60 con 0x7f75d806f050 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.150+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d806f050 0x7f75d81a81f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009fd0 tx=0x7f75cc00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.151+0000 7f75c77fe640 1 -- 172.21.15.33:0/1020822104 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc015020 con 0x7f75d806f050 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.151+0000 7f75c77fe640 1 -- 172.21.15.33:0/1020822104 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75cc011ce0 con 0x7f75d806f050 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.151+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d81ae050 con 0x7f75d806f050 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.151+0000 7f75c77fe640 1 -- 172.21.15.33:0/1020822104 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc010de0 con 0x7f75d806f050 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.151+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d81b48e0 con 0x7f75d806f050 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.153+0000 7f75deccb740 1 Processor -- start 2024-09-10T21:06:28.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.153+0000 7f75deccb740 1 -- start start 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75deccb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6bca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75c77fe640 1 -- 172.21.15.33:0/1020822104 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f75cc011830 con 0x7f75d806f050 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75deccb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75deccb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6c8f00 0x5613cc6cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75deccb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613cc632e50 con 0x5613cc6bc650 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75deccb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5613cc622280 con 0x5613cc6bcf70 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75deccb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5613cc626090 con 0x5613cc6c8f00 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6bca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6bca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51148/0 (socket says 172.21.15.33:51148) 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 -- 172.21.15.33:0/1669788076 learned_addr learned my addr 172.21.15.33:0/1669788076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:28.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d659c640 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6c8f00 0x5613cc6cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 -- 172.21.15.33:0/1669788076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6c8f00 msgr2=0x5613cc6cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6c8f00 0x5613cc6cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 -- 172.21.15.33:0/1669788076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 msgr2=0x5613cc6c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 -- 172.21.15.33:0/1669788076 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613cc380650 con 0x5613cc6bc650 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6bca30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f75d0010370 tx=0x7f75d0010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb00ce8017ea91b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.154+0000 7f75c4ff9640 1 -- 172.21.15.33:0/1669788076 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0016070 con 0x5613cc6bc650 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75c4ff9640 1 -- 172.21.15.33:0/1669788076 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75d0004d20 con 0x5613cc6bc650 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75c4ff9640 1 -- 172.21.15.33:0/1669788076 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0005350 con 0x5613cc6bc650 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75c77fe640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f75c00b5460 0x7f75c00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75d659c640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f75c00b5460 0x7f75c00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75c77fe640 1 -- 172.21.15.33:0/1020822104 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f75cc0e54e0 con 0x7f75d806f050 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75deccb740 1 -- 172.21.15.33:0/1669788076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 msgr2=0x5613cc6bca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75d659c640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f75c00b5460 0x7f75c00b7920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75c80096c0 tx=0x7f75c800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.155+0000 7f75deccb740 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6bca30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f75d0010370 tx=0x7f75d0010830 comp rx=0 tx=0).stop 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 -- 172.21.15.33:0/1669788076 shutdown_connections 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6c8f00 0x5613cc6cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 --2- 172.21.15.33:0/1669788076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6bca30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 -- 172.21.15.33:0/1669788076 >> 172.21.15.33:0/1669788076 conn(0x5613cc6ba480 msgr2=0x5613cc6bb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 -- 172.21.15.33:0/1669788076 shutdown_connections 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 -- 172.21.15.33:0/1669788076 wait complete. 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 -- start start 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.156+0000 7f75deccb740 1 Processor -- start 2024-09-10T21:06:28.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75deccb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6c8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75deccb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bcf70 0x5613cc6c5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75deccb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75deccb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613cc483700 con 0x5613cc6bcf70 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75deccb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5613cc5bd110 con 0x5613cc6c8f00 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75deccb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5613cc6cbb80 con 0x5613cc6bc650 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75deccb740 10 client.0 _ll_register_callbacks cb 0x5613cc5d6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bcf70 0x5613cc6c5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bcf70 0x5613cc6c5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51164/0 (socket says 172.21.15.33:51164) 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:44960/0 (socket says 172.21.15.33:44960) 2024-09-10T21:06:28.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 -- 172.21.15.33:0/1780961445 learned_addr learned my addr 172.21.15.33:0/1780961445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 -- 172.21.15.33:0/1780961445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 msgr2=0x5613cc6c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 -- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 msgr2=0x5613cc6c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 -- 172.21.15.33:0/1780961445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613cc5eac40 con 0x5613cc6bcf70 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bcf70 0x5613cc6c5c70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0ab9a0 tx=0x7f75cc0a7b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623e443442e8000c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.157+0000 7f75bdffb640 1 -- 172.21.15.33:0/1780961445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0ab410 con 0x5613cc6bcf70 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75bdffb640 1 -- 172.21.15.33:0/1780961445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f75cc010380 con 0x5613cc6bcf70 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d0004d20 con 0x5613cc6bcf70 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75bdffb640 1 -- 172.21.15.33:0/1780961445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f75cc0ab7b0 con 0x5613cc6bcf70 2024-09-10T21:06:28.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 10 client.15825 fetch_fsmap learned FSMap version 34 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5613cc6d7510 con 0x5613cc6bcf70 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75bdffb640 1 -- 172.21.15.33:0/1780961445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75d0004d20 con 0x5613cc6bcf70 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75bdffb640 1 -- 172.21.15.33:0/1780961445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f75cc0a7f10 con 0x5613cc6bcf70 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 10 client.15825 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 10 client.15825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 20 client.15825 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc6d8a10 0x5613cc6dae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 4 client.15825 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613cc634da0 con 0x5613cc6d8a10 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.158+0000 7f75d659c640 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc6d8a10 0x5613cc6dae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.159+0000 7f75bdffb640 10 client.15825 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.159+0000 7f75d659c640 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc6d8a10 0x5613cc6dae00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2f07b3c8896fdd9 server_cookie=628544d3acbfccf0 in_seq=0 out_seq=0 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75bdffb640 1 -- 172.21.15.33:0/1780961445 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613cc634da0 con 0x5613cc6d8a10 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75bdffb640 10 client.15825 handle_command_reply: tid=0 2024-09-10T21:06:28.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 client.15825 shutdown 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bcf70 msgr2=0x5613cc6c5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bcf70 0x5613cc6c5c70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0ab9a0 tx=0x7f75cc0a7b40 comp rx=0 tx=0).stop 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 shutdown_connections 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc6d8a10 0x5613cc6dae00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bcf70 0x5613cc6c5c70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 --2- 172.21.15.33:0/1780961445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 >> 172.21.15.33:0/1780961445 conn(0x5613cc6ba480 msgr2=0x5613cc6bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75bdffb640 0 client.15825 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 shutdown_connections 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 1 -- 172.21.15.33:0/1780961445 wait complete. 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.160+0000 7f75deccb740 20 client.15825 trim_cache size 0 max 16384 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 Processor -- start 2024-09-10T21:06:28.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 -- start start 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6d4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6c8f00 0x5613cc6ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81bd090 con 0x5613cc6c8f00 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81bed50 con 0x5613cc6bcf70 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75deccb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81bd760 con 0x5613cc6bc650 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6c8f00 0x5613cc6ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75d6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6c8f00 0x5613cc6ca2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51176/0 (socket says 172.21.15.33:51176) 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75d6d9d640 1 -- 172.21.15.33:0/379932149 learned_addr learned my addr 172.21.15.33:0/379932149 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.179+0000 7f75d659c640 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6d4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75d6d9d640 1 -- 172.21.15.33:0/379932149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 msgr2=0x5613cc6d4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6d4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75d6d9d640 1 -- 172.21.15.33:0/379932149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 msgr2=0x5613cc6c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75d6d9d640 1 -- 172.21.15.33:0/379932149 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613cc6db480 con 0x5613cc6c8f00 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6c8f00 0x5613cc6ca2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f75d0012f10 tx=0x7f75d0012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c745b107c3bf5f8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75bffff640 1 -- 172.21.15.33:0/379932149 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d002a070 con 0x5613cc6c8f00 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75bffff640 1 -- 172.21.15.33:0/379932149 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75d0004c00 con 0x5613cc6c8f00 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.180+0000 7f75bffff640 1 -- 172.21.15.33:0/379932149 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d0016070 con 0x5613cc6c8f00 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 -- 172.21.15.33:0/379932149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6c8f00 msgr2=0x5613cc6ca2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6c8f00 0x5613cc6ca2f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f75d0012f10 tx=0x7f75d0012ea0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 -- 172.21.15.33:0/379932149 shutdown_connections 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6c8f00 0x5613cc6ca2f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6bcf70 0x5613cc6c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 --2- 172.21.15.33:0/379932149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bc650 0x5613cc6d4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 -- 172.21.15.33:0/379932149 >> 172.21.15.33:0/379932149 conn(0x5613cc6ba480 msgr2=0x5613cc6c72a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 -- 172.21.15.33:0/379932149 shutdown_connections 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 -- 172.21.15.33:0/379932149 wait complete. 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 -- start start 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.181+0000 7f75deccb740 1 Processor -- start 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6d1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bcf70 0x5613cc6cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81be5c0 con 0x5613cc6bc650 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81bde30 con 0x5613cc6c8f00 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f75d81bb050 con 0x5613cc6bcf70 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 10 client.0 _ll_register_callbacks cb 0x7f75cc010fc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:06:28.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6d1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6d1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51180/0 (socket says 172.21.15.33:51180) 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 -- 172.21.15.33:0/743428201 learned_addr learned my addr 172.21.15.33:0/743428201 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d6d9d640 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bcf70 0x5613cc6cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 -- 172.21.15.33:0/743428201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bcf70 msgr2=0x5613cc6cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bcf70 0x5613cc6cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 -- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 msgr2=0x5613cc6cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 -- 172.21.15.33:0/743428201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613cc6ccab0 con 0x5613cc6bc650 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75d659c640 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6d1510 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5613cc6bb360 tx=0x7f75c802a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ba1dfd9327c3c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:06:28.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75bdffb640 1 -- 172.21.15.33:0/743428201 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8035040 con 0x5613cc6bc650 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75d0004c00 con 0x5613cc6bc650 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.182+0000 7f75bdffb640 1 -- 172.21.15.33:0/743428201 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f75c8031b50 con 0x5613cc6bc650 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75bdffb640 1 -- 172.21.15.33:0/743428201 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f75c8060050 con 0x5613cc6bc650 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 10 client.15831 fetch_fsmap learned FSMap version 34 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5613cc746950 con 0x5613cc6bc650 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75bdffb640 1 -- 172.21.15.33:0/743428201 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75d0004c00 con 0x5613cc6bc650 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75bdffb640 1 -- 172.21.15.33:0/743428201 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f75c80035f0 con 0x5613cc6bc650 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 10 client.15831 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 20 client.15831 populate_metadata read hostname 'smithi033' 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc7471c0 0x5613cc7495e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 4 client.15831 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:06:28.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5613cc630850 con 0x5613cc7471c0 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.183+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc7471c0 0x5613cc7495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75d5d9b640 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc7471c0 0x5613cc7495e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab85955170089b30 server_cookie=cca2749348a04de2 in_seq=0 out_seq=0 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75bdffb640 10 client.15831 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75bdffb640 1 -- 172.21.15.33:0/743428201 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x5613cc630850 con 0x5613cc7471c0 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75bdffb640 10 client.15831 handle_command_reply: tid=0 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 client.15831 shutdown 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:06:28.225 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e", 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:06:28.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:06:28.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:06:28.228 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 msgr2=0x5613cc6d1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6d1510 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x5613cc6bb360 tx=0x7f75c802a440 comp rx=0 tx=0).stop 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 shutdown_connections 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613cc7471c0 0x5613cc7495e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5613cc6c8f00 0x5613cc6cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75bdffb640 0 client.15831 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613cc6bcf70 0x5613cc6cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 --2- 172.21.15.33:0/743428201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613cc6bc650 0x5613cc6d1510 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 >> 172.21.15.33:0/743428201 conn(0x5613cc6ba480 msgr2=0x5613cc6c7ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:28.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 shutdown_connections 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.184+0000 7f75deccb740 1 -- 172.21.15.33:0/743428201 wait complete. 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75deccb740 20 client.15831 trim_cache size 0 max 16384 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f75c00b5460 msgr2=0x7f75c00b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f75c00b5460 0x7f75c00b7920 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75c80096c0 tx=0x7f75c800b040 comp rx=0 tx=0).stop 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d806f050 msgr2=0x7f75d81a81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d806f050 0x7f75d81a81f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009fd0 tx=0x7f75cc00b870 comp rx=0 tx=0).stop 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 shutdown_connections 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f75c00b5460 0x7f75c00b7920 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f75d811c020 0x7f75d81ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75d806f050 0x7f75d81a81f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 --2- 172.21.15.33:0/1020822104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f75d806e730 0x7f75d81a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 >> 172.21.15.33:0/1020822104 conn(0x7f75d806c950 msgr2=0x7f75d811c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:06:28.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.185+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 shutdown_connections 2024-09-10T21:06:28.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:06:28.186+0000 7f75d759e640 1 -- 172.21.15.33:0/1020822104 wait complete. 2024-09-10T21:06:28.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 174261 10.76 MB/sec warmup 89 sec latency 60.663 ms 2024-09-10T21:06:29.278 INFO:tasks.cephfs.filesystem:scrub status for tag:7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e - {'path': '/', 'tag': '7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e', 'options': 'recursive,force'} 2024-09-10T21:06:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:29 smithi033 ceph-mon[30860]: pgmap v421: 97 pgs: 97 active+clean; 89 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-09-10T21:06:29.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 176801 10.78 MB/sec warmup 90 sec latency 29.733 ms 2024-09-10T21:06:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:29 smithi104 ceph-mon[35325]: pgmap v421: 97 pgs: 97 active+clean; 89 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-09-10T21:06:29.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:29 smithi174 ceph-mon[35635]: pgmap v421: 97 pgs: 97 active+clean; 89 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-09-10T21:06:30.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 178535 10.76 MB/sec warmup 91 sec latency 12.006 ms 2024-09-10T21:06:31.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:31 smithi033 ceph-mon[30860]: pgmap v422: 97 pgs: 97 active+clean; 90 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-09-10T21:06:31.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 181210 10.82 MB/sec warmup 92 sec latency 14.209 ms 2024-09-10T21:06:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:31 smithi104 ceph-mon[35325]: pgmap v422: 97 pgs: 97 active+clean; 90 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-09-10T21:06:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:31 smithi174 ceph-mon[35635]: pgmap v422: 97 pgs: 97 active+clean; 90 MiB data, 761 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-09-10T21:06:32.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 183403 10.80 MB/sec warmup 93 sec latency 9.943 ms 2024-09-10T21:06:33.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:33 smithi033 ceph-mon[30860]: pgmap v423: 97 pgs: 97 active+clean; 90 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-09-10T21:06:33.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 185144 10.80 MB/sec warmup 94 sec latency 40.008 ms 2024-09-10T21:06:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:33 smithi104 ceph-mon[35325]: pgmap v423: 97 pgs: 97 active+clean; 90 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-09-10T21:06:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:33 smithi174 ceph-mon[35635]: pgmap v423: 97 pgs: 97 active+clean; 90 MiB data, 746 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 251 op/s 2024-09-10T21:06:34.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 187805 10.84 MB/sec warmup 95 sec latency 80.952 ms 2024-09-10T21:06:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:35 smithi033 ceph-mon[30860]: pgmap v424: 97 pgs: 97 active+clean; 91 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2024-09-10T21:06:35.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 189529 10.83 MB/sec warmup 96 sec latency 11.895 ms 2024-09-10T21:06:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:35 smithi104 ceph-mon[35325]: pgmap v424: 97 pgs: 97 active+clean; 91 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2024-09-10T21:06:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:35 smithi174 ceph-mon[35635]: pgmap v424: 97 pgs: 97 active+clean; 91 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2024-09-10T21:06:36.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:36 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:06:36.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:36 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:06:36.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:36 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:06:36.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 191962 10.86 MB/sec warmup 97 sec latency 11.155 ms 2024-09-10T21:06:36.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:36 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:06:36.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:36 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:06:36.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:36 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:06:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:36 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:06:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:36 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:06:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:36 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:06:37.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:37 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:06:37.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:37 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:06:37.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:37 smithi033 ceph-mon[30860]: pgmap v425: 97 pgs: 97 active+clean; 93 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-09-10T21:06:37.701 INFO:tasks.workunit.client.0.smithi033.stdout: 1 194443 10.87 MB/sec warmup 98 sec latency 47.055 ms 2024-09-10T21:06:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:37 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:06:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:37 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:06:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:37 smithi104 ceph-mon[35325]: pgmap v425: 97 pgs: 97 active+clean; 93 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-09-10T21:06:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:37 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:06:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:37 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:06:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:37 smithi174 ceph-mon[35635]: pgmap v425: 97 pgs: 97 active+clean; 93 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 322 op/s 2024-09-10T21:06:38.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 196040 10.84 MB/sec warmup 99 sec latency 37.569 ms 2024-09-10T21:06:39.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 198861 10.91 MB/sec warmup 100 sec latency 20.805 ms 2024-09-10T21:06:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:39 smithi104 ceph-mon[35325]: pgmap v426: 97 pgs: 97 active+clean; 95 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-10T21:06:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:39 smithi174 ceph-mon[35635]: pgmap v426: 97 pgs: 97 active+clean; 95 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-10T21:06:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:39 smithi033 ceph-mon[30860]: pgmap v426: 97 pgs: 97 active+clean; 95 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-10T21:06:40.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 201008 10.88 MB/sec warmup 101 sec latency 11.294 ms 2024-09-10T21:06:41.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 202747 10.89 MB/sec warmup 102 sec latency 9.594 ms 2024-09-10T21:06:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:41 smithi104 ceph-mon[35325]: pgmap v427: 97 pgs: 97 active+clean; 95 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-09-10T21:06:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:41 smithi174 ceph-mon[35635]: pgmap v427: 97 pgs: 97 active+clean; 95 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-09-10T21:06:41.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:41 smithi033 ceph-mon[30860]: pgmap v427: 97 pgs: 97 active+clean; 95 MiB data, 855 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 345 op/s 2024-09-10T21:06:42.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 205675 10.95 MB/sec warmup 103 sec latency 9.815 ms 2024-09-10T21:06:43.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 206950 10.89 MB/sec warmup 104 sec latency 49.498 ms 2024-09-10T21:06:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:43 smithi104 ceph-mon[35325]: pgmap v428: 97 pgs: 97 active+clean; 98 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-09-10T21:06:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:43 smithi174 ceph-mon[35635]: pgmap v428: 97 pgs: 97 active+clean; 98 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-09-10T21:06:43.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:43 smithi033 ceph-mon[30860]: pgmap v428: 97 pgs: 97 active+clean; 98 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 338 op/s 2024-09-10T21:06:44.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 209586 10.94 MB/sec warmup 105 sec latency 29.368 ms 2024-09-10T21:06:45.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 211837 10.92 MB/sec warmup 106 sec latency 11.254 ms 2024-09-10T21:06:45.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:45 smithi104 ceph-mon[35325]: pgmap v429: 97 pgs: 97 active+clean; 101 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 384 op/s 2024-09-10T21:06:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:45 smithi174 ceph-mon[35635]: pgmap v429: 97 pgs: 97 active+clean; 101 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 384 op/s 2024-09-10T21:06:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:45 smithi033 ceph-mon[30860]: pgmap v429: 97 pgs: 97 active+clean; 101 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.5 MiB/s wr, 384 op/s 2024-09-10T21:06:46.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 213553 10.92 MB/sec warmup 107 sec latency 11.376 ms 2024-09-10T21:06:47.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 216364 10.98 MB/sec warmup 108 sec latency 58.821 ms 2024-09-10T21:06:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:47 smithi104 ceph-mon[35325]: pgmap v430: 97 pgs: 97 active+clean; 103 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-09-10T21:06:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:47 smithi174 ceph-mon[35635]: pgmap v430: 97 pgs: 97 active+clean; 103 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-09-10T21:06:47.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:47 smithi033 ceph-mon[30860]: pgmap v430: 97 pgs: 97 active+clean; 103 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 348 op/s 2024-09-10T21:06:48.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 217673 10.91 MB/sec warmup 109 sec latency 29.298 ms 2024-09-10T21:06:49.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 220260 10.97 MB/sec warmup 110 sec latency 25.701 ms 2024-09-10T21:06:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:49 smithi104 ceph-mon[35325]: pgmap v431: 97 pgs: 97 active+clean; 105 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-10T21:06:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:49 smithi174 ceph-mon[35635]: pgmap v431: 97 pgs: 97 active+clean; 105 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-10T21:06:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:49 smithi033 ceph-mon[30860]: pgmap v431: 97 pgs: 97 active+clean; 105 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-10T21:06:50.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 222463 10.94 MB/sec warmup 111 sec latency 11.504 ms 2024-09-10T21:06:51.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 224181 10.94 MB/sec warmup 112 sec latency 9.507 ms 2024-09-10T21:06:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:51 smithi104 ceph-mon[35325]: pgmap v432: 97 pgs: 97 active+clean; 104 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 319 op/s 2024-09-10T21:06:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:51 smithi174 ceph-mon[35635]: pgmap v432: 97 pgs: 97 active+clean; 104 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 319 op/s 2024-09-10T21:06:51.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:51 smithi033 ceph-mon[30860]: pgmap v432: 97 pgs: 97 active+clean; 104 MiB data, 931 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 319 op/s 2024-09-10T21:06:52.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 227054 11.00 MB/sec warmup 113 sec latency 9.921 ms 2024-09-10T21:06:53.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 228724 10.97 MB/sec warmup 114 sec latency 10.925 ms 2024-09-10T21:06:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:53 smithi104 ceph-mon[35325]: pgmap v433: 97 pgs: 97 active+clean; 106 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 372 op/s 2024-09-10T21:06:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:53 smithi174 ceph-mon[35635]: pgmap v433: 97 pgs: 97 active+clean; 106 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 372 op/s 2024-09-10T21:06:53.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:53 smithi033 ceph-mon[30860]: pgmap v433: 97 pgs: 97 active+clean; 106 MiB data, 957 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.2 MiB/s wr, 372 op/s 2024-09-10T21:06:54.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 230992 10.99 MB/sec warmup 115 sec latency 15.952 ms 2024-09-10T21:06:55.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 233379 10.99 MB/sec warmup 116 sec latency 11.141 ms 2024-09-10T21:06:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:55 smithi104 ceph-mon[35325]: pgmap v434: 97 pgs: 97 active+clean; 105 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-10T21:06:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:55 smithi174 ceph-mon[35635]: pgmap v434: 97 pgs: 97 active+clean; 105 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-10T21:06:55.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:55 smithi033 ceph-mon[30860]: pgmap v434: 97 pgs: 97 active+clean; 105 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.4 MiB/s wr, 359 op/s 2024-09-10T21:06:56.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 235015 10.97 MB/sec warmup 117 sec latency 8.802 ms 2024-09-10T21:06:57.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 237831 11.02 MB/sec warmup 118 sec latency 9.195 ms 2024-09-10T21:06:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:57 smithi104 ceph-mon[35325]: pgmap v435: 97 pgs: 97 active+clean; 107 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-09-10T21:06:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:57 smithi174 ceph-mon[35635]: pgmap v435: 97 pgs: 97 active+clean; 107 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-09-10T21:06:57.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:57 smithi033 ceph-mon[30860]: pgmap v435: 97 pgs: 97 active+clean; 107 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 359 op/s 2024-09-10T21:06:58.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 239355 10.99 MB/sec warmup 119 sec latency 13.308 ms 2024-09-10T21:06:59.279 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:06:59.567 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:06:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:06:59 smithi033 ceph-mon[30860]: pgmap v436: 97 pgs: 97 active+clean; 109 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 339 op/s 2024-09-10T21:06:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:06:59 smithi104 ceph-mon[35325]: pgmap v436: 97 pgs: 97 active+clean; 109 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 339 op/s 2024-09-10T21:06:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:06:59 smithi174 ceph-mon[35635]: pgmap v436: 97 pgs: 97 active+clean; 109 MiB data, 1014 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 339 op/s 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 -- 172.21.15.33:0/3145997880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 msgr2=0x7fded810c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 --2- 172.21.15.33:0/3145997880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded810c9e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdecc009900 tx=0x7fdecc02a410 comp rx=0 tx=0).stop 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 -- 172.21.15.33:0/3145997880 shutdown_connections 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 --2- 172.21.15.33:0/3145997880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fded810d0f0 0x7fded810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 --2- 172.21.15.33:0/3145997880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded810c9e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 --2- 172.21.15.33:0/3145997880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 0x7fded8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 -- 172.21.15.33:0/3145997880 >> 172.21.15.33:0/3145997880 conn(0x7fded8076e20 msgr2=0x7fded8077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 -- 172.21.15.33:0/3145997880 shutdown_connections 2024-09-10T21:07:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.076+0000 7fdede828640 1 -- 172.21.15.33:0/3145997880 wait complete. 2024-09-10T21:07:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.077+0000 7fdede828640 1 Processor -- start 2024-09-10T21:07:00.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.077+0000 7fdede828640 1 -- start start 2024-09-10T21:07:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.077+0000 7fdede828640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 0x7fded8199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.077+0000 7fdede828640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded8199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.077+0000 7fdede828640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fded810d0f0 0x7fded819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.077+0000 7fdede828640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fded8112df0 con 0x7fded8102230 2024-09-10T21:07:00.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fdede828640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fded8112c70 con 0x7fded8101830 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fdede828640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fded8112f70 con 0x7fded810d0f0 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded8199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 0x7fded8199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded8199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46330/0 (socket says 172.21.15.33:46330) 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 0x7fded8199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43948/0 (socket says 172.21.15.33:43948) 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded77fe640 1 -- 172.21.15.33:0/3725998224 learned_addr learned my addr 172.21.15.33:0/3725998224 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fded810d0f0 0x7fded819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 -- 172.21.15.33:0/3725998224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fded810d0f0 msgr2=0x7fded819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fded810d0f0 0x7fded819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 -- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 msgr2=0x7fded8199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded8199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 -- 172.21.15.33:0/3725998224 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded819f2f0 con 0x7fded8101830 2024-09-10T21:07:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded77fe640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.078+0000 7fded7fff640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 0x7fded8199240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdec800e9f0 tx=0x7fdec800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.079+0000 7fded57fa640 1 -- 172.21.15.33:0/3725998224 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec800ccc0 con 0x7fded8101830 2024-09-10T21:07:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.079+0000 7fded57fa640 1 -- 172.21.15.33:0/3725998224 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec80108e0 con 0x7fded8101830 2024-09-10T21:07:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.079+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded819f5e0 con 0x7fded8101830 2024-09-10T21:07:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.079+0000 7fded57fa640 1 -- 172.21.15.33:0/3725998224 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec8018880 con 0x7fded8101830 2024-09-10T21:07:00.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.079+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded8102940 con 0x7fded8101830 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 Processor -- start 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 -- start start 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78f480 0x5642fb78f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb79b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb79bd30 0x5642fb79e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5642fb705e50 con 0x5642fb78f480 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642fb6f5280 con 0x5642fb79bd30 2024-09-10T21:07:00.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5642fb6f9090 con 0x5642fb78fda0 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdedcdc5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78f480 0x5642fb78f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdedcdc5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78f480 0x5642fb78f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46338/0 (socket says 172.21.15.33:46338) 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fdedcdc5640 1 -- 172.21.15.33:0/1406066450 learned_addr learned my addr 172.21.15.33:0/1406066450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.082+0000 7fded77fe640 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb79bd30 0x5642fb79e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded57fa640 1 -- 172.21.15.33:0/3725998224 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdec8010430 con 0x7fded8101830 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded77fe640 1 -- 172.21.15.33:0/1406066450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 msgr2=0x5642fb79b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded7fff640 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb79b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded77fe640 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb79b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded77fe640 1 -- 172.21.15.33:0/1406066450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78f480 msgr2=0x5642fb78f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded77fe640 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78f480 0x5642fb78f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded77fe640 1 -- 172.21.15.33:0/1406066450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fb453650 con 0x5642fb79bd30 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78f480 0x5642fb78f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded7fff640 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb79b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fded77fe640 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb79bd30 0x5642fb79e120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdecc004770 tx=0x7fdecc005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c575672027c03bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fdebe7fc640 1 -- 172.21.15.33:0/1406066450 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc035050 con 0x5642fb79bd30 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fdebe7fc640 1 -- 172.21.15.33:0/1406066450 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdecc0054d0 con 0x5642fb79bd30 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.083+0000 7fdebe7fc640 1 -- 172.21.15.33:0/1406066450 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdecc0049e0 con 0x5642fb79bd30 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fded57fa640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdec00b5360 0x7fdec00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdee0bfd740 1 -- 172.21.15.33:0/1406066450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb79bd30 msgr2=0x5642fb79e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb79bd30 0x5642fb79e120 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fdecc004770 tx=0x7fdecc005a20 comp rx=0 tx=0).stop 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdec00b5360 0x7fdec00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdee0bfd740 1 -- 172.21.15.33:0/1406066450 shutdown_connections 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb79bd30 0x5642fb79e120 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb79b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/1406066450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78f480 0x5642fb78f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fdee0bfd740 1 -- 172.21.15.33:0/1406066450 >> 172.21.15.33:0/1406066450 conn(0x5642fb78d2b0 msgr2=0x5642fb78e450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.084+0000 7fded57fa640 1 -- 172.21.15.33:0/3725998224 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdec80e5ec0 con 0x7fded8101830 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.085+0000 7fdee0bfd740 1 -- 172.21.15.33:0/1406066450 shutdown_connections 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.085+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdec00b5360 0x7fdec00b7820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fded0007cd0 tx=0x7fded0007520 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.085+0000 7fdee0bfd740 1 -- 172.21.15.33:0/1406066450 wait complete. 2024-09-10T21:07:00.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.085+0000 7fdee0bfd740 1 -- start start 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.085+0000 7fdee0bfd740 1 Processor -- start 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb79b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb798a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb7990c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5642fb556700 con 0x5642fb79bd30 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642fb690110 con 0x5642fb78f480 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5642fb79e950 con 0x5642fb78fda0 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdee0bfd740 10 client.0 _ll_register_callbacks cb 0x5642fb6a9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:00.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fded77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb798a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fded7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb79b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fded77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb798a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42564/0 (socket says 172.21.15.33:42564) 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fded77fe640 1 -- 172.21.15.33:0/2484654449 learned_addr learned my addr 172.21.15.33:0/2484654449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.086+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb7990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdedcdc5640 1 -- 172.21.15.33:0/2484654449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 msgr2=0x5642fb798a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb798a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdedcdc5640 1 -- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 msgr2=0x5642fb79b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb79b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdedcdc5640 1 -- 172.21.15.33:0/2484654449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fb6bdc40 con 0x5642fb79bd30 2024-09-10T21:07:00.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fded7fff640 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb79b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb7990c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fded000fc90 tx=0x7fded0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0043b018b6d94ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.087+0000 7fdeb77fe640 1 -- 172.21.15.33:0/2484654449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded002b060 con 0x5642fb79bd30 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdeb77fe640 1 -- 172.21.15.33:0/2484654449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fded0016030 con 0x5642fb79bd30 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdecc0054d0 con 0x5642fb79bd30 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdeb77fe640 1 -- 172.21.15.33:0/2484654449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fded0026410 con 0x5642fb79bd30 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 10 client.15843 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5642fb7aa280 con 0x5642fb79bd30 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdeb77fe640 1 -- 172.21.15.33:0/2484654449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fded0026a60 con 0x5642fb79bd30 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdeb77fe640 1 -- 172.21.15.33:0/2484654449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fded0030070 con 0x5642fb79bd30 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 10 client.15843 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:00.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 10 client.15843 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 20 client.15843 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb7ab7e0 0x5642fb7adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 4 client.15843 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.088+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdecc0054d0 con 0x5642fb7ab7e0 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.089+0000 7fded7fff640 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb7ab7e0 0x5642fb7adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.089+0000 7fded7fff640 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb7ab7e0 0x5642fb7adc00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfeb0ea9e276a0e7 server_cookie=f5270e29d98da57 in_seq=0 out_seq=0 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.089+0000 7fdeb77fe640 10 client.15843 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdeb77fe640 1 -- 172.21.15.33:0/2484654449 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fdecc0054d0 con 0x5642fb7ab7e0 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdeb77fe640 10 client.15843 handle_command_reply: tid=0 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 client.15843 shutdown 2024-09-10T21:07:00.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 msgr2=0x5642fb7990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb7990c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fded000fc90 tx=0x7fded0004ab0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 shutdown_connections 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb7ab7e0 0x5642fb7adc00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb7990c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78fda0 0x5642fb798a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2484654449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb79b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 >> 172.21.15.33:0/2484654449 conn(0x5642fb78d2b0 msgr2=0x5642fb792e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.090+0000 7fdeb77fe640 0 client.15843 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.091+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 shutdown_connections 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.091+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2484654449 wait complete. 2024-09-10T21:07:00.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.091+0000 7fdee0bfd740 20 client.15843 trim_cache size 0 max 16384 2024-09-10T21:07:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 Processor -- start 2024-09-10T21:07:00.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 -- start start 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78f480 0x5642fb7a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78fda0 0x5642fb793ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb794220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fded81ac250 con 0x5642fb79bd30 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fded81adf10 con 0x5642fb78fda0 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fded81ac920 con 0x5642fb78f480 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fded7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78f480 0x5642fb7a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fded7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78f480 0x5642fb7a7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42578/0 (socket says 172.21.15.33:42578) 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fded7fff640 1 -- 172.21.15.33:0/2080390053 learned_addr learned my addr 172.21.15.33:0/2080390053 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fded77fe640 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78fda0 0x5642fb793ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.112+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb794220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fded7fff640 1 -- 172.21.15.33:0/2080390053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78fda0 msgr2=0x5642fb793ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fded7fff640 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78fda0 0x5642fb793ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fded7fff640 1 -- 172.21.15.33:0/2080390053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 msgr2=0x5642fb794220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fded7fff640 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb794220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fded7fff640 1 -- 172.21.15.33:0/2080390053 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642fb796f90 con 0x5642fb78f480 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fded7fff640 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78f480 0x5642fb7a7b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdec80a9960 tx=0x7fdec80e3850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43c50c526c44d1d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fdebdffb640 1 -- 172.21.15.33:0/2080390053 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80b5bd0 con 0x5642fb78f480 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fdebdffb640 1 -- 172.21.15.33:0/2080390053 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec80e3a90 con 0x5642fb78f480 2024-09-10T21:07:00.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.113+0000 7fdebdffb640 1 -- 172.21.15.33:0/2080390053 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec80a8330 con 0x5642fb78f480 2024-09-10T21:07:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2080390053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78f480 msgr2=0x5642fb7a7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78f480 0x5642fb7a7b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdec80a9960 tx=0x7fdec80e3850 comp rx=0 tx=0).stop 2024-09-10T21:07:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2080390053 shutdown_connections 2024-09-10T21:07:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb79bd30 0x5642fb794220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78fda0 0x5642fb793ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/2080390053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb78f480 0x5642fb7a7b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2080390053 >> 172.21.15.33:0/2080390053 conn(0x5642fb78d2b0 msgr2=0x5642fb7ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2080390053 shutdown_connections 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.114+0000 7fdee0bfd740 1 -- 172.21.15.33:0/2080390053 wait complete. 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 -- start start 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 Processor -- start 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb7a4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78fda0 0x5642fb7b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb79bd30 0x5642fb7b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fded81ad780 con 0x5642fb78fda0 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fded81acff0 con 0x5642fb78f480 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fded81aa210 con 0x5642fb79bd30 2024-09-10T21:07:00.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdee0bfd740 10 client.0 _ll_register_callbacks cb 0x7fded000f420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fded77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78fda0 0x5642fb7b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fded77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78fda0 0x5642fb7b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46372/0 (socket says 172.21.15.33:46372) 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fded77fe640 1 -- 172.21.15.33:0/3604045777 learned_addr learned my addr 172.21.15.33:0/3604045777 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb79bd30 0x5642fb7b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.115+0000 7fded7fff640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb7a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdedcdc5640 1 -- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 msgr2=0x5642fb7a4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb7a4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdedcdc5640 1 -- 172.21.15.33:0/3604045777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78fda0 msgr2=0x5642fb7b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78fda0 0x5642fb7b2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdedcdc5640 1 -- 172.21.15.33:0/3604045777 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642fb7b3c00 con 0x5642fb79bd30 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fded77fe640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78fda0 0x5642fb7b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:00.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fded7fff640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb7a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdedcdc5640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb79bd30 0x5642fb7b3510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fded0017040 tx=0x7fded0003c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eebb35974e8dcb4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdeb77fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdeb77fe640 1 -- 172.21.15.33:0/3604045777 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fded0036070 con 0x5642fb79bd30 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdec80e3a90 con 0x5642fb79bd30 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdeb77fe640 1 -- 172.21.15.33:0/3604045777 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fded00269c0 con 0x5642fb79bd30 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdeb77fe640 1 -- 172.21.15.33:0/3604045777 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fded000f870 con 0x5642fb79bd30 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdee0bfd740 10 client.25402 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.116+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5642fb818710 con 0x5642fb79bd30 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdeb77fe640 1 -- 172.21.15.33:0/3604045777 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec80e3a90 con 0x5642fb79bd30 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdeb77fe640 1 -- 172.21.15.33:0/3604045777 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fded0030040 con 0x5642fb79bd30 2024-09-10T21:07:00.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdee0bfd740 10 client.25402 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdee0bfd740 10 client.25402 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdee0bfd740 20 client.25402 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb818ec0 0x5642fb81b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdee0bfd740 4 client.25402 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5642fb707da0 con 0x5642fb818ec0 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fded7fff640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb818ec0 0x5642fb81b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fdeb77fe640 10 client.25402 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.117+0000 7fded7fff640 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb818ec0 0x5642fb81b2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473fe5624adac402 server_cookie=fe93059bd2624a5c in_seq=0 out_seq=0 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdeb77fe640 1 -- 172.21.15.33:0/3604045777 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x5642fb707da0 con 0x5642fb818ec0 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdeb77fe640 10 client.25402 handle_command_reply: tid=0 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 client.25402 shutdown 2024-09-10T21:07:00.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb79bd30 msgr2=0x5642fb7b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb79bd30 0x5642fb7b3510 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fded0017040 tx=0x7fded0003c20 comp rx=0 tx=0).stop 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 shutdown_connections 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5642fb818ec0 0x5642fb81b2e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdeb77fe640 0 client.25402 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642fb79bd30 0x5642fb7b3510 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642fb78fda0 0x5642fb7b2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 --2- 172.21.15.33:0/3604045777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642fb78f480 0x5642fb7a4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 >> 172.21.15.33:0/3604045777 conn(0x5642fb78d2b0 msgr2=0x5642fb793ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 shutdown_connections 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 1 -- 172.21.15.33:0/3604045777 wait complete. 2024-09-10T21:07:00.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.118+0000 7fdee0bfd740 20 client.25402 trim_cache size 0 max 16384 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:00.129 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:07:00.130 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:00.131 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:00.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:07:00.132 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:07:00.132 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:07:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdec00b5360 msgr2=0x7fdec00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdec00b5360 0x7fdec00b7820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fded0007cd0 tx=0x7fded0007520 comp rx=0 tx=0).stop 2024-09-10T21:07:00.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 msgr2=0x7fded8199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 0x7fded8199240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdec800e9f0 tx=0x7fdec800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 shutdown_connections 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdec00b5360 0x7fdec00b7820 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fded810d0f0 0x7fded819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fded8102230 0x7fded8199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 --2- 172.21.15.33:0/3725998224 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fded8101830 0x7fded8199240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 >> 172.21.15.33:0/3725998224 conn(0x7fded8076e20 msgr2=0x7fded80ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 shutdown_connections 2024-09-10T21:07:00.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:00.119+0000 7fdede828640 1 -- 172.21.15.33:0/3725998224 wait complete. 2024-09-10T21:07:00.703 INFO:tasks.workunit.client.0.smithi033.stdout: 1 243810 8.00 MB/sec execute 1 sec latency 39.193 ms 2024-09-10T21:07:01.151 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:07:01.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:07:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:01 smithi033 ceph-mon[30860]: pgmap v437: 97 pgs: 97 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-09-10T21:07:01.703 INFO:tasks.workunit.client.0.smithi033.stdout: 1 245541 9.75 MB/sec execute 2 sec latency 9.571 ms 2024-09-10T21:07:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:01 smithi104 ceph-mon[35325]: pgmap v437: 97 pgs: 97 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-09-10T21:07:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:01 smithi174 ceph-mon[35635]: pgmap v437: 97 pgs: 97 active+clean; 110 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 286 op/s 2024-09-10T21:07:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 -- 172.21.15.33:0/1838937394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec103cc0 msgr2=0x7f7aec1040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 --2- 172.21.15.33:0/1838937394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec103cc0 0x7f7aec1040a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009900 tx=0x7f7adc02a410 comp rx=0 tx=0).stop 2024-09-10T21:07:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 -- 172.21.15.33:0/1838937394 shutdown_connections 2024-09-10T21:07:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 --2- 172.21.15.33:0/1838937394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aec10d120 0x7f7aec10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 --2- 172.21.15.33:0/1838937394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7aec1045e0 0x7f7aec10cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 --2- 172.21.15.33:0/1838937394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec103cc0 0x7f7aec1040a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:01.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 -- 172.21.15.33:0/1838937394 >> 172.21.15.33:0/1838937394 conn(0x7f7aec0fd740 msgr2=0x7f7aec0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.909+0000 7f7af227e640 1 -- 172.21.15.33:0/1838937394 shutdown_connections 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.910+0000 7f7af227e640 1 -- 172.21.15.33:0/1838937394 wait complete. 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.910+0000 7f7af227e640 1 Processor -- start 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.910+0000 7f7af227e640 1 -- start start 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af227e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aec103cc0 0x7f7aec199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af227e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7aec1045e0 0x7f7aec1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af227e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec10d120 0x7f7aec19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af227e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec112d90 con 0x7f7aec10d120 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af227e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec112c10 con 0x7f7aec1045e0 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af227e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec112f10 con 0x7f7aec103cc0 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec10d120 0x7f7aec19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:01.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7aebfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aec103cc0 0x7f7aec199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec10d120 0x7f7aec19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46392/0 (socket says 172.21.15.33:46392) 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 -- 172.21.15.33:0/1538827613 learned_addr learned my addr 172.21.15.33:0/1538827613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7aebfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aec103cc0 0x7f7aec199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42610/0 (socket says 172.21.15.33:42610) 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7aeb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7aec1045e0 0x7f7aec1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 -- 172.21.15.33:0/1538827613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aec103cc0 msgr2=0x7f7aec199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aec103cc0 0x7f7aec199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 -- 172.21.15.33:0/1538827613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7aec1045e0 msgr2=0x7f7aec1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7aec1045e0 0x7f7aec1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.911+0000 7f7af081b640 1 -- 172.21.15.33:0/1538827613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec19f330 con 0x7f7aec10d120 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.912+0000 7f7af081b640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec10d120 0x7f7aec19eb60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009fd0 tx=0x7f7ae400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.912+0000 7f7ae97fa640 1 -- 172.21.15.33:0/1538827613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4017070 con 0x7f7aec10d120 2024-09-10T21:07:01.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.912+0000 7f7ae97fa640 1 -- 172.21.15.33:0/1538827613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ae4003e20 con 0x7f7aec10d120 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.912+0000 7f7ae97fa640 1 -- 172.21.15.33:0/1538827613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4010460 con 0x7f7aec10d120 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.912+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec19f620 con 0x7f7aec10d120 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.912+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1a5ce0 con 0x7f7aec10d120 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.913+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aec1040a0 con 0x7f7aec10d120 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.916+0000 7f7ae97fa640 1 -- 172.21.15.33:0/1538827613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7ae4003970 con 0x7f7aec10d120 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.916+0000 7f7ae97fa640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ad40b5360 0x7f7ad40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.916+0000 7f7aebfff640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ad40b5360 0x7f7ad40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.917+0000 7f7ae97fa640 1 -- 172.21.15.33:0/1538827613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7ae40e5830 con 0x7f7aec10d120 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.917+0000 7f7aebfff640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ad40b5360 0x7f7ad40b7820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7adc031440 tx=0x7f7adc005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:01.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:01.919+0000 7f7ae97fa640 1 -- 172.21.15.33:0/1538827613 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f7ae40ac120 con 0x7f7aec10d120 2024-09-10T21:07:02.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.166+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7aec1a60a0 con 0x7f7aec10d120 2024-09-10T21:07:02.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.166+0000 7f7ae97fa640 1 -- 172.21.15.33:0/1538827613 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11987 (secure 0 0 0) 0x7f7ae40b0b50 con 0x7f7aec10d120 2024-09-10T21:07:02.171 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:07:02.171 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"btime":"2024-09-10T21:06:22:397830+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:06:22.081190+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:07:02.172 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-09-10T21:07:02.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ad40b5360 msgr2=0x7f7ad40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:02.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ad40b5360 0x7f7ad40b7820 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7adc031440 tx=0x7f7adc005b50 comp rx=0 tx=0).stop 2024-09-10T21:07:02.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec10d120 msgr2=0x7f7aec19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:02.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec10d120 0x7f7aec19eb60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009fd0 tx=0x7f7ae400b260 comp rx=0 tx=0).stop 2024-09-10T21:07:02.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 shutdown_connections 2024-09-10T21:07:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ad40b5360 0x7f7ad40b7820 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7aec10d120 0x7f7aec19eb60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7aec1045e0 0x7f7aec1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 --2- 172.21.15.33:0/1538827613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7aec103cc0 0x7f7aec199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 >> 172.21.15.33:0/1538827613 conn(0x7f7aec0fd740 msgr2=0x7f7aec0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 shutdown_connections 2024-09-10T21:07:02.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:02.169+0000 7f7af227e640 1 -- 172.21.15.33:0/1538827613 wait complete. 2024-09-10T21:07:02.702 INFO:tasks.workunit.client.0.smithi033.stdout: 1 248149 11.21 MB/sec execute 3 sec latency 42.377 ms 2024-09-10T21:07:02.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:02 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1538827613' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:07:02.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:02 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1538827613' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:07:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:02 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1538827613' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:07:02.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:07:03.226 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:03.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:03 smithi033 ceph-mon[30860]: pgmap v438: 97 pgs: 97 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-09-10T21:07:03.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 249580 10.06 MB/sec execute 4 sec latency 24.323 ms 2024-09-10T21:07:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:03 smithi104 ceph-mon[35325]: pgmap v438: 97 pgs: 97 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-09-10T21:07:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.869+0000 7f5f987ab640 1 -- 172.21.15.33:0/2925373394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 msgr2=0x7f5f90103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.869+0000 7f5f987ab640 1 --2- 172.21.15.33:0/2925373394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90103180 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009900 tx=0x7f5f8c02a410 comp rx=0 tx=0).stop 2024-09-10T21:07:03.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 -- 172.21.15.33:0/2925373394 shutdown_connections 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 --2- 172.21.15.33:0/2925373394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f90109130 0x7f5f9010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 --2- 172.21.15.33:0/2925373394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f901087b0 0x7f5f90108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 --2- 172.21.15.33:0/2925373394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90103180 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 -- 172.21.15.33:0/2925373394 >> 172.21.15.33:0/2925373394 conn(0x7f5f900fe810 msgr2=0x7f5f90100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 -- 172.21.15.33:0/2925373394 shutdown_connections 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 -- 172.21.15.33:0/2925373394 wait complete. 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 Processor -- start 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 -- start start 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f901087b0 0x7f5f90199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.870+0000 7f5f987ab640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f90109130 0x7f5f9019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f987ab640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90112db0 con 0x7f5f90102da0 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f987ab640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90112c30 con 0x7f5f90109130 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f987ab640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f90112f30 con 0x7f5f901087b0 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46410/0 (socket says 172.21.15.33:46410) 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96d48640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f90109130 0x7f5f9019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 -- 172.21.15.33:0/1914968283 learned_addr learned my addr 172.21.15.33:0/1914968283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f95d46640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f901087b0 0x7f5f90199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 -- 172.21.15.33:0/1914968283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f901087b0 msgr2=0x7f5f90199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f901087b0 0x7f5f90199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 -- 172.21.15.33:0/1914968283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f90109130 msgr2=0x7f5f9019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f90109130 0x7f5f9019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.871+0000 7f5f96547640 1 -- 172.21.15.33:0/1914968283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9019f2f0 con 0x7f5f90102da0 2024-09-10T21:07:03.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.872+0000 7f5f96547640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90199240 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c005940 tx=0x7f5f8c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:03.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.872+0000 7f5f877fe640 1 -- 172.21.15.33:0/1914968283 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c03f030 con 0x7f5f90102da0 2024-09-10T21:07:03.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.872+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9019f580 con 0x7f5f90102da0 2024-09-10T21:07:03.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.872+0000 7f5f877fe640 1 -- 172.21.15.33:0/1914968283 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8c0044c0 con 0x7f5f90102da0 2024-09-10T21:07:03.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.872+0000 7f5f877fe640 1 -- 172.21.15.33:0/1914968283 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c035070 con 0x7f5f90102da0 2024-09-10T21:07:03.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.872+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f901a5ca0 con 0x7f5f90102da0 2024-09-10T21:07:03.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.875+0000 7f5f9ab80740 1 Processor -- start 2024-09-10T21:07:03.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.875+0000 7f5f9ab80740 1 -- start start 2024-09-10T21:07:03.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.875+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85ef88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.875+0000 7f5f877fe640 1 -- 172.21.15.33:0/1914968283 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5f8c03ba20 con 0x7f5f90102da0 2024-09-10T21:07:03.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.875+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f04870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.875+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f071a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.875+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ac85e6aea0 con 0x55ac85f04db0 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ac85e5a2d0 con 0x55ac85ef8500 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac85e5e0e0 con 0x55ac85ef8e20 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46416/0 (socket says 172.21.15.33:46416) 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 -- 172.21.15.33:0/954346976 learned_addr learned my addr 172.21.15.33:0/954346976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f96d48640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85ef88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f96547640 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f04870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 -- 172.21.15.33:0/954346976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 msgr2=0x55ac85f04870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f04870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 -- 172.21.15.33:0/954346976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 msgr2=0x55ac85ef88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85ef88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 -- 172.21.15.33:0/954346976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac85bb8650 con 0x55ac85f04db0 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f96d48640 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85ef88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.876+0000 7f5f95d46640 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f071a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009c90 tx=0x7f5f80011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d22235f43fa32c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f84ff9640 1 -- 172.21.15.33:0/954346976 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8001e070 con 0x55ac85f04db0 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f84ff9640 1 -- 172.21.15.33:0/954346976 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f80019ae0 con 0x55ac85f04db0 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f877fe640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f7c0b52e0 0x7f5f7c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f84ff9640 1 -- 172.21.15.33:0/954346976 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000e960 con 0x55ac85f04db0 2024-09-10T21:07:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f96d48640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f7c0b52e0 0x7f5f7c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f877fe640 1 -- 172.21.15.33:0/1914968283 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f8c1061a0 con 0x7f5f90102da0 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f96d48640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f7c0b52e0 0x7f5f7c0b77a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009fd0 tx=0x7f5f88009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f9ab80740 1 -- 172.21.15.33:0/954346976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 msgr2=0x55ac85f071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f071a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009c90 tx=0x7f5f80011810 comp rx=0 tx=0).stop 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f9ab80740 1 -- 172.21.15.33:0/954346976 shutdown_connections 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f071a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f04870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/954346976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85ef88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.877+0000 7f5f9ab80740 1 -- 172.21.15.33:0/954346976 >> 172.21.15.33:0/954346976 conn(0x55ac85ef6330 msgr2=0x55ac85ef74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 -- 172.21.15.33:0/954346976 shutdown_connections 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 -- 172.21.15.33:0/954346976 wait complete. 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 -- start start 2024-09-10T21:07:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 Processor -- start 2024-09-10T21:07:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f02250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ac85cbb700 con 0x55ac85f04db0 2024-09-10T21:07:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ac85df5160 con 0x55ac85ef8500 2024-09-10T21:07:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac85f07a30 con 0x55ac85ef8e20 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.878+0000 7f5f9ab80740 10 client.0 _ll_register_callbacks cb 0x55ac85e0e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f96547640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f96d48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f02250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42660/0 (socket says 172.21.15.33:42660) 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 -- 172.21.15.33:0/1616418910 learned_addr learned my addr 172.21.15.33:0/1616418910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 -- 172.21.15.33:0/1616418910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 msgr2=0x55ac85f04730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f04730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 -- 172.21.15.33:0/1616418910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 msgr2=0x55ac85f02250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f02250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 -- 172.21.15.33:0/1616418910 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac85e22c90 con 0x55ac85ef8e20 2024-09-10T21:07:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f96d48640 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f02250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f95d46640 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000e560 tx=0x7f5f80009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e038c835def3f22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f79ffb640 1 -- 172.21.15.33:0/1616418910 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8001e050 con 0x55ac85ef8e20 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f80019ae0 con 0x55ac85ef8e20 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.879+0000 7f5f79ffb640 1 -- 172.21.15.33:0/1616418910 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f80005450 con 0x55ac85ef8e20 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f79ffb640 1 -- 172.21.15.33:0/1616418910 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5f80005650 con 0x55ac85ef8e20 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 10 client.25420 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac85f13400 con 0x55ac85ef8e20 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f79ffb640 1 -- 172.21.15.33:0/1616418910 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f80019ae0 con 0x55ac85ef8e20 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f79ffb640 1 -- 172.21.15.33:0/1616418910 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f5f8002d030 con 0x55ac85ef8e20 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 10 client.25420 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 10 client.25420 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:07:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 20 client.25420 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f14750 0x55ac85f16b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 4 client.25420 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.880+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac85e6cdf0 con 0x55ac85f14750 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.881+0000 7f5f96547640 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f14750 0x55ac85f16b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.881+0000 7f5f96547640 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f14750 0x55ac85f16b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e2d1f0d6ff09ba9 server_cookie=fc9db7d5baafca0 in_seq=0 out_seq=0 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.881+0000 7f5f79ffb640 10 client.25420 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f79ffb640 1 -- 172.21.15.33:0/1616418910 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac85e6cdf0 con 0x55ac85f14750 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f79ffb640 10 client.25420 handle_command_reply: tid=0 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 client.25420 shutdown 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 msgr2=0x55ac85f01b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000e560 tx=0x7f5f80009f00 comp rx=0 tx=0).stop 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 shutdown_connections 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f14750 0x55ac85f16b40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f02250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/1616418910 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f04730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 >> 172.21.15.33:0/1616418910 conn(0x55ac85ef6330 msgr2=0x55ac85efbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f79ffb640 0 client.25420 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 shutdown_connections 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 1 -- 172.21.15.33:0/1616418910 wait complete. 2024-09-10T21:07:03.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.882+0000 7f5f9ab80740 20 client.25420 trim_cache size 0 max 16384 2024-09-10T21:07:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:03 smithi174 ceph-mon[35635]: pgmap v438: 97 pgs: 97 active+clean; 112 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-09-10T21:07:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.896+0000 7f5f9ab80740 1 Processor -- start 2024-09-10T21:07:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.896+0000 7f5f9ab80740 1 -- start start 2024-09-10T21:07:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f901ac100 con 0x55ac85f04db0 2024-09-10T21:07:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f901addc0 con 0x55ac85ef8500 2024-09-10T21:07:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f901ac7d0 con 0x55ac85ef8e20 2024-09-10T21:07:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f96547640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42672/0 (socket says 172.21.15.33:42672) 2024-09-10T21:07:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 -- 172.21.15.33:0/2203057206 learned_addr learned my addr 172.21.15.33:0/2203057206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:03.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f96d48640 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 -- 172.21.15.33:0/2203057206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 msgr2=0x55ac85f10c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f10c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 -- 172.21.15.33:0/2203057206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 msgr2=0x55ac85f061a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f061a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.897+0000 7f5f95d46640 1 -- 172.21.15.33:0/2203057206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac85eed7e0 con 0x55ac85ef8e20 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.898+0000 7f5f95d46640 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009be0 tx=0x7f5f80009f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eea56b84a1d9a8ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.898+0000 7f5f7bfff640 1 -- 172.21.15.33:0/2203057206 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8001e040 con 0x55ac85ef8e20 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.898+0000 7f5f7bfff640 1 -- 172.21.15.33:0/2203057206 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8001f040 con 0x55ac85ef8e20 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.898+0000 7f5f7bfff640 1 -- 172.21.15.33:0/2203057206 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f800054b0 con 0x55ac85ef8e20 2024-09-10T21:07:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 -- 172.21.15.33:0/2203057206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 msgr2=0x55ac85f01090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009be0 tx=0x7f5f80009f70 comp rx=0 tx=0).stop 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 -- 172.21.15.33:0/2203057206 shutdown_connections 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f061a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8e20 0x55ac85f01090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/2203057206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8500 0x55ac85f10c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 -- 172.21.15.33:0/2203057206 >> 172.21.15.33:0/2203057206 conn(0x55ac85ef6330 msgr2=0x55ac85ef7380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 -- 172.21.15.33:0/2203057206 shutdown_connections 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 -- 172.21.15.33:0/2203057206 wait complete. 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 -- start start 2024-09-10T21:07:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.899+0000 7f5f9ab80740 1 Processor -- start 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8500 0x55ac85f0d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8e20 0x55ac85f1c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f9ab80740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f1c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f901ad630 con 0x55ac85f04db0 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f901acea0 con 0x55ac85ef8e20 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f9ab80740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f901aa0c0 con 0x55ac85ef8500 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f9ab80740 10 client.0 _ll_register_callbacks cb 0x7f5f8001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8e20 0x55ac85f1c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96547640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8500 0x55ac85f0d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f95d46640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8e20 0x55ac85f1c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:44042/0 (socket says 172.21.15.33:44042) 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f95d46640 1 -- 172.21.15.33:0/3524348335 learned_addr learned my addr 172.21.15.33:0/3524348335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96d48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f1c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96547640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8500 0x55ac85f0d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42688/0 (socket says 172.21.15.33:42688) 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96547640 1 -- 172.21.15.33:0/3524348335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8e20 msgr2=0x55ac85f1c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96547640 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8e20 0x55ac85f1c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96547640 1 -- 172.21.15.33:0/3524348335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 msgr2=0x55ac85f1c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96547640 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f1c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96547640 1 -- 172.21.15.33:0/3524348335 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac85f0c170 con 0x55ac85ef8500 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f95d46640 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8e20 0x55ac85f1c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.900+0000 7f5f96d48640 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f1c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f96547640 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8500 0x55ac85f0d580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55ac85ef7210 tx=0x7f5f8c03b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71399736330813fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f79ffb640 1 -- 172.21.15.33:0/3524348335 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c049040 con 0x55ac85ef8500 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f79ffb640 1 -- 172.21.15.33:0/3524348335 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f8c003cf0 con 0x55ac85ef8500 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f8001f040 con 0x55ac85ef8500 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f79ffb640 1 -- 172.21.15.33:0/3524348335 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f5f8c0cc360 con 0x55ac85ef8500 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f9ab80740 10 client.25426 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ac85f81700 con 0x55ac85ef8500 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f79ffb640 1 -- 172.21.15.33:0/3524348335 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8001f040 con 0x55ac85ef8500 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f79ffb640 1 -- 172.21.15.33:0/3524348335 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f5f8c0c8c80 con 0x55ac85ef8500 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f9ab80740 10 client.25426 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f9ab80740 10 client.25426 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:07:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f9ab80740 20 client.25426 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:03.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.901+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f81e30 0x55ac85f84250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:03.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f9ab80740 4 client.25426 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:03.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac85e688a0 con 0x55ac85f81e30 2024-09-10T21:07:03.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f95d46640 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f81e30 0x55ac85f84250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:03.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f79ffb640 10 client.25426 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:03.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f95d46640 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f81e30 0x55ac85f84250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e04c7ef77b18d09b server_cookie=53efe67d064bf3d4 in_seq=0 out_seq=0 2024-09-10T21:07:03.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f79ffb640 1 -- 172.21.15.33:0/3524348335 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac85e688a0 con 0x55ac85f81e30 2024-09-10T21:07:03.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f79ffb640 10 client.25426 handle_command_reply: tid=0 2024-09-10T21:07:03.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.902+0000 7f5f9ab80740 1 client.25426 shutdown 2024-09-10T21:07:03.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8500 msgr2=0x55ac85f0d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8500 0x55ac85f0d580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55ac85ef7210 tx=0x7f5f8c03b460 comp rx=0 tx=0).stop 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 shutdown_connections 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac85f81e30 0x55ac85f84250 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac85f04db0 0x55ac85f1c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac85ef8e20 0x55ac85f1c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 --2- 172.21.15.33:0/3524348335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac85ef8500 0x55ac85f0d580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 >> 172.21.15.33:0/3524348335 conn(0x55ac85ef6330 msgr2=0x55ac85ef0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f79ffb640 0 client.25426 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 shutdown_connections 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 1 -- 172.21.15.33:0/3524348335 wait complete. 2024-09-10T21:07:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f9ab80740 20 client.25426 trim_cache size 0 max 16384 2024-09-10T21:07:03.914 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f7c0b52e0 msgr2=0x7f5f7c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f987ab640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f7c0b52e0 0x7f5f7c0b77a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009fd0 tx=0x7f5f88009500 comp rx=0 tx=0).stop 2024-09-10T21:07:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 msgr2=0x7f5f90199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.903+0000 7f5f987ab640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90199240 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c005940 tx=0x7f5f8c0040c0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 shutdown_connections 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f7c0b52e0 0x7f5f7c0b77a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f90109130 0x7f5f9019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f901087b0 0x7f5f90199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 --2- 172.21.15.33:0/1914968283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f90102da0 0x7f5f90199240 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 >> 172.21.15.33:0/1914968283 conn(0x7f5f900fe810 msgr2=0x7f5f90100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 shutdown_connections 2024-09-10T21:07:03.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:03.904+0000 7f5f987ab640 1 -- 172.21.15.33:0/1914968283 wait complete. 2024-09-10T21:07:04.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 252078 11.38 MB/sec execute 5 sec latency 29.579 ms 2024-09-10T21:07:04.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:07:05.084 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:05.495 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:05 smithi033 ceph-mon[30860]: pgmap v439: 97 pgs: 97 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 354 op/s 2024-09-10T21:07:05.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 -- 172.21.15.33:0/3224470382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4100109150 msgr2=0x7f410010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 --2- 172.21.15.33:0/3224470382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4100109150 0x7f410010f950 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009900 tx=0x7f40f802a410 comp rx=0 tx=0).stop 2024-09-10T21:07:05.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 -- 172.21.15.33:0/3224470382 shutdown_connections 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 --2- 172.21.15.33:0/3224470382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4100109150 0x7f410010f950 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 --2- 172.21.15.33:0/3224470382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f41001087d0 0x7f4100108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 --2- 172.21.15.33:0/3224470382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4100102dc0 0x7f41001031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 -- 172.21.15.33:0/3224470382 >> 172.21.15.33:0/3224470382 conn(0x7f41000fe830 msgr2=0x7f4100100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 -- 172.21.15.33:0/3224470382 shutdown_connections 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.579+0000 7f41070ff640 1 -- 172.21.15.33:0/3224470382 wait complete. 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 Processor -- start 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 -- start start 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4100102dc0 0x7f41001992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41001087d0 0x7f4100199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100109150 0x7f410019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4100112e10 con 0x7f41001087d0 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4100112c90 con 0x7f4100109150 2024-09-10T21:07:05.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f41070ff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4100112f90 con 0x7f4100102dc0 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f40fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41001087d0 0x7f4100199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f40fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41001087d0 0x7f4100199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46440/0 (socket says 172.21.15.33:46440) 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f40fffff640 1 -- 172.21.15.33:0/1262276558 learned_addr learned my addr 172.21.15.33:0/1262276558 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f4104e9b640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4100102dc0 0x7f41001992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.580+0000 7f410569c640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100109150 0x7f410019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f4104e9b640 1 -- 172.21.15.33:0/1262276558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100109150 msgr2=0x7f410019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f4104e9b640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100109150 0x7f410019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f4104e9b640 1 -- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41001087d0 msgr2=0x7f4100199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f4104e9b640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41001087d0 0x7f4100199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f4104e9b640 1 -- 172.21.15.33:0/1262276558 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f410019f370 con 0x7f4100102dc0 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f40fffff640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41001087d0 0x7f4100199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f4104e9b640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4100102dc0 0x7f41001992c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:05.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f40fdffb640 1 -- 172.21.15.33:0/1262276558 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800ccc0 con 0x7f4100102dc0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f410019f660 con 0x7f4100102dc0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f40fdffb640 1 -- 172.21.15.33:0/1262276558 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e80108e0 con 0x7f4100102dc0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f40fdffb640 1 -- 172.21.15.33:0/1262276558 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e8018880 con 0x7f4100102dc0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.581+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001a5d20 con 0x7f4100102dc0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.584+0000 7f40fdffb640 1 -- 172.21.15.33:0/1262276558 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f40e8010430 con 0x7f4100102dc0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.584+0000 7f41094d4740 1 Processor -- start 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.584+0000 7f41094d4740 1 -- start start 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f41094d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631518610f0 0x5631518614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f41094d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563151861a10 0x56315186d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f41094d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f41094d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5631517d7ea0 con 0x5631518610f0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f41094d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5631517c72d0 con 0x56315186d9a0 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f41094d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5631517cb0e0 con 0x563151861a10 2024-09-10T21:07:05.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631518610f0 0x5631518614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f4104e9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563151861a10 0x56315186d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631518610f0 0x5631518614d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46452/0 (socket says 172.21.15.33:46452) 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f40fffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 -- 172.21.15.33:0/415632912 learned_addr learned my addr 172.21.15.33:0/415632912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f40fdffb640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f40ec0b52e0 0x7f40ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 -- 172.21.15.33:0/415632912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563151861a10 msgr2=0x56315186d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563151861a10 0x56315186d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 -- 172.21.15.33:0/415632912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 msgr2=0x56315186fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 -- 172.21.15.33:0/415632912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563151525650 con 0x5631518610f0 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f40fffff640 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f40fdffb640 1 -- 172.21.15.33:0/1262276558 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f40e80e5ef0 con 0x7f4100102dc0 2024-09-10T21:07:05.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631518610f0 0x5631518614d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f40f802a930 tx=0x7f40f8032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123fe9a87b0b1894 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.585+0000 7f410569c640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f40ec0b52e0 0x7f40ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.586+0000 7f40f6ffd640 1 -- 172.21.15.33:0/415632912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f804c070 con 0x5631518610f0 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.586+0000 7f40f6ffd640 1 -- 172.21.15.33:0/415632912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f8005040 con 0x5631518610f0 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.586+0000 7f40f6ffd640 1 -- 172.21.15.33:0/415632912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f8040070 con 0x5631518610f0 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.586+0000 7f41094d4740 1 -- 172.21.15.33:0/415632912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631518610f0 msgr2=0x5631518614d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.586+0000 7f41094d4740 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631518610f0 0x5631518614d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f40f802a930 tx=0x7f40f8032ba0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.586+0000 7f410569c640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f40ec0b52e0 0x7f40ec0b77a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40f8049000 tx=0x7f40f804c000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 -- 172.21.15.33:0/415632912 shutdown_connections 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563151861a10 0x56315186d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 --2- 172.21.15.33:0/415632912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5631518610f0 0x5631518614d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 -- 172.21.15.33:0/415632912 >> 172.21.15.33:0/415632912 conn(0x56315185ef20 msgr2=0x5631518600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 -- 172.21.15.33:0/415632912 shutdown_connections 2024-09-10T21:07:05.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 -- 172.21.15.33:0/415632912 wait complete. 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 -- start start 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.587+0000 7f41094d4740 1 Processor -- start 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f41094d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631518610f0 0x56315186d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f41094d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f41094d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f41094d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563151628700 con 0x563151861a10 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f41094d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563151762160 con 0x56315186d9a0 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f41094d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563151870620 con 0x5631518610f0 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f41094d4740 10 client.0 _ll_register_callbacks cb 0x56315177b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46456/0 (socket says 172.21.15.33:46456) 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 -- 172.21.15.33:0/112828919 learned_addr learned my addr 172.21.15.33:0/112828919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f410569c640 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f4104e9b640 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631518610f0 0x56315186d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 -- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631518610f0 msgr2=0x56315186d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631518610f0 0x56315186d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 -- 172.21.15.33:0/112828919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 msgr2=0x56315186ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 -- 172.21.15.33:0/112828919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315178fc90 con 0x563151861a10 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40fffff640 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186a700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f40f0010be0 tx=0x7f40f0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a397e6c60516399 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40dffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.588+0000 7f40dffff640 1 -- 172.21.15.33:0/112828919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40f0004b20 con 0x563151861a10 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f40dffff640 1 -- 172.21.15.33:0/112828919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f40f0004cc0 con 0x563151861a10 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40f8005040 con 0x563151861a10 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f40dffff640 1 -- 172.21.15.33:0/112828919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f40f0005020 con 0x563151861a10 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 10 client.15885 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56315187c030 con 0x563151861a10 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f40dffff640 1 -- 172.21.15.33:0/112828919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40f8005040 con 0x563151861a10 2024-09-10T21:07:05.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f40dffff640 1 -- 172.21.15.33:0/112828919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f40f001b6e0 con 0x563151861a10 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 10 client.15885 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 10 client.15885 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 20 client.15885 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56315187d590 0x56315187f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 4 client.15885 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631517d9df0 con 0x56315187d590 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.589+0000 7f4104e9b640 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56315187d590 0x56315187f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.590+0000 7f40dffff640 10 client.15885 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.590+0000 7f4104e9b640 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56315187d590 0x56315187f980 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e5760f47be1923 server_cookie=3582653931813b8 in_seq=0 out_seq=0 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.590+0000 7f40dffff640 1 -- 172.21.15.33:0/112828919 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5631517d9df0 con 0x56315187d590 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.590+0000 7f40dffff640 10 client.15885 handle_command_reply: tid=0 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.590+0000 7f41094d4740 1 client.15885 shutdown 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 msgr2=0x56315186a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186a700 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f40f0010be0 tx=0x7f40f0011810 comp rx=0 tx=0).stop 2024-09-10T21:07:05.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 shutdown_connections 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56315186d9a0 0x56315186ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186a700 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56315187d590 0x56315187f980 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 --2- 172.21.15.33:0/112828919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631518610f0 0x56315186d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 >> 172.21.15.33:0/112828919 conn(0x56315185ef20 msgr2=0x563151864a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f40dffff640 0 client.15885 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 shutdown_connections 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 1 -- 172.21.15.33:0/112828919 wait complete. 2024-09-10T21:07:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.591+0000 7f41094d4740 20 client.15885 trim_cache size 0 max 16384 2024-09-10T21:07:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 Processor -- start 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 -- start start 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x5631518798a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x563151867910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41001ae460 con 0x563151861a10 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f41001b0120 con 0x5631518610f0 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.605+0000 7f41094d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41001aeb30 con 0x56315186d9a0 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f40fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x5631518798a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x5631518798a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:44082/0 (socket says 172.21.15.33:44082) 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f40fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46462/0 (socket says 172.21.15.33:46462) 2024-09-10T21:07:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 -- 172.21.15.33:0/4162482559 learned_addr learned my addr 172.21.15.33:0/4162482559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f410569c640 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x563151867910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 -- 172.21.15.33:0/4162482559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 msgr2=0x563151867910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x563151867910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 -- 172.21.15.33:0/4162482559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 msgr2=0x56315186c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 -- 172.21.15.33:0/4162482559 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563151870100 con 0x5631518610f0 2024-09-10T21:07:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f4104e9b640 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x5631518798a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40e80ac6f0 tx=0x7f40e80e38c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a670f7a9eb8305b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f40f67fc640 1 -- 172.21.15.33:0/4162482559 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80a6e60 con 0x5631518610f0 2024-09-10T21:07:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f40f67fc640 1 -- 172.21.15.33:0/4162482559 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e80ac270 con 0x5631518610f0 2024-09-10T21:07:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.606+0000 7f40f67fc640 1 -- 172.21.15.33:0/4162482559 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80b5c40 con 0x5631518610f0 2024-09-10T21:07:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 -- 172.21.15.33:0/4162482559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 msgr2=0x5631518798a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x5631518798a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f40e80ac6f0 tx=0x7f40e80e38c0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 -- 172.21.15.33:0/4162482559 shutdown_connections 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x563151867910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x56315186c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 --2- 172.21.15.33:0/4162482559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x5631518798a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 -- 172.21.15.33:0/4162482559 >> 172.21.15.33:0/4162482559 conn(0x56315185ef20 msgr2=0x56315186bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 -- 172.21.15.33:0/4162482559 shutdown_connections 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 -- 172.21.15.33:0/4162482559 wait complete. 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.607+0000 7f41094d4740 1 -- start start 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 1 Processor -- start 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x563151872060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x5631518725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x5631518e9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41001af990 con 0x563151861a10 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f41001af200 con 0x5631518610f0 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41001ac420 con 0x56315186d9a0 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f41094d4740 10 client.0 _ll_register_callbacks cb 0x7f40f0010510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x563151872060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f40fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x5631518725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x563151872060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:44096/0 (socket says 172.21.15.33:44096) 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 -- 172.21.15.33:0/170851281 learned_addr learned my addr 172.21.15.33:0/170851281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f410569c640 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x5631518e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 -- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 msgr2=0x5631518e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x5631518e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 -- 172.21.15.33:0/170851281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 msgr2=0x5631518725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x5631518725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.608+0000 7f4104e9b640 1 -- 172.21.15.33:0/170851281 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631518e9b10 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f4104e9b640 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x563151872060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40e8000c00 tx=0x7f40e8009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9573127695b85e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f40dffff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f40dffff640 1 -- 172.21.15.33:0/170851281 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80a6e60 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f40dffff640 1 -- 172.21.15.33:0/170851281 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f40e80a89d0 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e80ac270 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f40dffff640 1 -- 172.21.15.33:0/170851281 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f40e80ac0b0 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f41094d4740 10 client.25463 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5631518eadd0 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f40dffff640 1 -- 172.21.15.33:0/170851281 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40e80ac270 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f40dffff640 1 -- 172.21.15.33:0/170851281 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f40e80a80b0 con 0x5631518610f0 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f41094d4740 10 client.25463 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:05.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.609+0000 7f41094d4740 10 client.25463 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f41094d4740 20 client.25463 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f41094d4740 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5631518eb4f0 0x5631518ed910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f41094d4740 4 client.25463 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5631517d58a0 con 0x5631518eb4f0 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f40fffff640 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5631518eb4f0 0x5631518ed910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f40fffff640 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5631518eb4f0 0x5631518ed910 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=95aaca5c28edf7a8 server_cookie=f80bd676422e7253 in_seq=0 out_seq=0 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f40dffff640 10 client.25463 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f40dffff640 1 -- 172.21.15.33:0/170851281 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5631517d58a0 con 0x5631518eb4f0 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f40dffff640 10 client.25463 handle_command_reply: tid=0 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f41094d4740 1 client.25463 shutdown 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 msgr2=0x563151872060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.610+0000 7f41094d4740 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x563151872060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f40e8000c00 tx=0x7f40e8009700 comp rx=0 tx=0).stop 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 shutdown_connections 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5631518eb4f0 0x5631518ed910 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56315186d9a0 0x5631518e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563151861a10 0x5631518725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f40dffff640 0 client.25463 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 --2- 172.21.15.33:0/170851281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5631518610f0 0x563151872060 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 >> 172.21.15.33:0/170851281 conn(0x56315185ef20 msgr2=0x56315186c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 shutdown_connections 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 1 -- 172.21.15.33:0/170851281 wait complete. 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41094d4740 20 client.25463 trim_cache size 0 max 16384 2024-09-10T21:07:05.621 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:05.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f40ec0b52e0 msgr2=0x7f40ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41070ff640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f40ec0b52e0 0x7f40ec0b77a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40f8049000 tx=0x7f40f804c000 comp rx=0 tx=0).stop 2024-09-10T21:07:05.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.611+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4100102dc0 msgr2=0x7f41001992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:05.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4100102dc0 0x7f41001992c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f40e800e9f0 tx=0x7f40e800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 shutdown_connections 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f40ec0b52e0 0x7f40ec0b77a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4100109150 0x7f410019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41001087d0 0x7f4100199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 --2- 172.21.15.33:0/1262276558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4100102dc0 0x7f41001992c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 >> 172.21.15.33:0/1262276558 conn(0x7f41000fe830 msgr2=0x7f4100100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 shutdown_connections 2024-09-10T21:07:05.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:05.612+0000 7f41070ff640 1 -- 172.21.15.33:0/1262276558 wait complete. 2024-09-10T21:07:05.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 253687 10.43 MB/sec execute 6 sec latency 19.393 ms 2024-09-10T21:07:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:05 smithi104 ceph-mon[35325]: pgmap v439: 97 pgs: 97 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 354 op/s 2024-09-10T21:07:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:05 smithi174 ceph-mon[35635]: pgmap v439: 97 pgs: 97 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 354 op/s 2024-09-10T21:07:06.465 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:07:06.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 255816 10.90 MB/sec execute 7 sec latency 12.567 ms 2024-09-10T21:07:06.732 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 -- 172.21.15.33:0/867423217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 msgr2=0x7fbfc8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/867423217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc8102070 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009900 tx=0x7fbfc402a410 comp rx=0 tx=0).stop 2024-09-10T21:07:07.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 -- 172.21.15.33:0/867423217 shutdown_connections 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/867423217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfc8108020 0x7fbfc810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/867423217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc81076a0 0x7fbfc8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/867423217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc8102070 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 -- 172.21.15.33:0/867423217 >> 172.21.15.33:0/867423217 conn(0x7fbfc80fd700 msgr2=0x7fbfc80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 -- 172.21.15.33:0/867423217 shutdown_connections 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 -- 172.21.15.33:0/867423217 wait complete. 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 Processor -- start 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.242+0000 7fbfd01e7640 1 -- start start 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfd01e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfd01e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc81076a0 0x7fbfc8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfd01e7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfc8108020 0x7fbfc819ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfd01e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8112e00 con 0x7fbfc8101c90 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfd01e7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8112c80 con 0x7fbfc8108020 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfd01e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc8112f80 con 0x7fbfc81076a0 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46480/0 (socket says 172.21.15.33:46480) 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfce784640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfc8108020 0x7fbfc819ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 -- 172.21.15.33:0/3302546462 learned_addr learned my addr 172.21.15.33:0/3302546462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcd782640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc81076a0 0x7fbfc8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 -- 172.21.15.33:0/3302546462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc81076a0 msgr2=0x7fbfc8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc81076a0 0x7fbfc8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 -- 172.21.15.33:0/3302546462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfc8108020 msgr2=0x7fbfc819ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfc8108020 0x7fbfc819ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 -- 172.21.15.33:0/3302546462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfc819f3a0 con 0x7fbfc8101c90 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfce784640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfc8108020 0x7fbfc819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.243+0000 7fbfcdf83640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc81992f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005940 tx=0x7fbfc40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.244+0000 7fbfbeffd640 1 -- 172.21.15.33:0/3302546462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc403f030 con 0x7fbfc8101c90 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.244+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfc819f630 con 0x7fbfc8101c90 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.244+0000 7fbfbeffd640 1 -- 172.21.15.33:0/3302546462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc40044c0 con 0x7fbfc8101c90 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.244+0000 7fbfbeffd640 1 -- 172.21.15.33:0/3302546462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4035070 con 0x7fbfc8101c90 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.244+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfc81a5d50 con 0x7fbfc8101c90 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.246+0000 7fbfd25bc740 1 Processor -- start 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.246+0000 7fbfbeffd640 1 -- 172.21.15.33:0/3302546462 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbfc404c060 con 0x7fbfc8101c90 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfd25bc740 1 -- start start 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9bf190 0x56214f9c1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56214f924ea0 con 0x56214f9bf190 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56214f9142d0 con 0x56214f9ae4d0 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56214f9180e0 con 0x56214f9aedf0 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:44134/0 (socket says 172.21.15.33:44134) 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfcdf83640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 -- 172.21.15.33:0/2110552884 learned_addr learned my addr 172.21.15.33:0/2110552884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfcd782640 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9bf190 0x56214f9c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 -- 172.21.15.33:0/2110552884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 msgr2=0x56214f9bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 -- 172.21.15.33:0/2110552884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9bf190 msgr2=0x56214f9c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9bf190 0x56214f9c1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.247+0000 7fbfce784640 1 -- 172.21.15.33:0/2110552884 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214f672650 con 0x56214f9ae4d0 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.248+0000 7fbfbeffd640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbfb40b5360 0x7fbfb40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.248+0000 7fbfce784640 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9ae8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0011810 tx=0x7fbfc0011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9da4c2eee4efb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.248+0000 7fbfcdf83640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbfb40b5360 0x7fbfb40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.248+0000 7fbfbeffd640 1 -- 172.21.15.33:0/3302546462 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbfc4105ae0 con 0x7fbfc8101c90 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.248+0000 7fbfb3fff640 1 -- 172.21.15.33:0/2110552884 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0011310 con 0x56214f9ae4d0 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.248+0000 7fbfb3fff640 1 -- 172.21.15.33:0/2110552884 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc000f9a0 con 0x56214f9ae4d0 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.248+0000 7fbfb3fff640 1 -- 172.21.15.33:0/2110552884 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00056a0 con 0x56214f9ae4d0 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2110552884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 msgr2=0x56214f9ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfcdf83640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbfb40b5360 0x7fbfb40b7820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4031440 tx=0x7fbfc402a410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:07.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9ae8b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0011810 tx=0x7fbfc0011cd0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2110552884 shutdown_connections 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9bf190 0x56214f9c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2110552884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9ae8b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2110552884 >> 172.21.15.33:0/2110552884 conn(0x56214f9ac300 msgr2=0x56214f9ad4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2110552884 shutdown_connections 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2110552884 wait complete. 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 -- start start 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.249+0000 7fbfd25bc740 1 Processor -- start 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9ae4d0 0x56214f9bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9bc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56214f775700 con 0x56214f9ae4d0 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56214f8af160 con 0x56214f9bf190 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56214f9c1e40 con 0x56214f9aedf0 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 10 client.0 _ll_register_callbacks cb 0x56214f8c89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9ae4d0 0x56214f9bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9ae4d0 0x56214f9bea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46492/0 (socket says 172.21.15.33:46492) 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 -- 172.21.15.33:0/3034965515 learned_addr learned my addr 172.21.15.33:0/3034965515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfcd782640 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9bc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 -- 172.21.15.33:0/3034965515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 msgr2=0x56214f9bbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 -- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 msgr2=0x56214f9bc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9bc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 -- 172.21.15.33:0/3034965515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214f8dcc90 con 0x56214f9ae4d0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfce784640 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9ae4d0 0x56214f9bea30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0005160 tx=0x7fbfc0005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282c733d5bd23323 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfb17fa640 1 -- 172.21.15.33:0/3034965515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc002e060 con 0x56214f9ae4d0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfb17fa640 1 -- 172.21.15.33:0/3034965515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbfc0005400 con 0x56214f9ae4d0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.250+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc000f9a0 con 0x56214f9ae4d0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfb17fa640 1 -- 172.21.15.33:0/3034965515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0005600 con 0x56214f9ae4d0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 10 client.15903 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56214f9cd760 con 0x56214f9ae4d0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfb17fa640 1 -- 172.21.15.33:0/3034965515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfc000f9a0 con 0x56214f9ae4d0 2024-09-10T21:07:07.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfb17fa640 1 -- 172.21.15.33:0/3034965515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fbfc0033070 con 0x56214f9ae4d0 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 10 client.15903 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 10 client.15903 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 20 client.15903 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214f9cebc0 0x56214f9d0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 4 client.15903 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56214f926df0 con 0x56214f9cebc0 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.251+0000 7fbfcd782640 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214f9cebc0 0x56214f9d0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.252+0000 7fbfcd782640 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214f9cebc0 0x56214f9d0fb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d1fbc25e8759d93 server_cookie=86b62c10a93783fe in_seq=0 out_seq=0 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.252+0000 7fbfb17fa640 10 client.15903 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.252+0000 7fbfb17fa640 1 -- 172.21.15.33:0/3034965515 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56214f926df0 con 0x56214f9cebc0 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.252+0000 7fbfb17fa640 10 client.15903 handle_command_reply: tid=0 2024-09-10T21:07:07.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.252+0000 7fbfd25bc740 1 client.15903 shutdown 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9ae4d0 msgr2=0x56214f9bea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9ae4d0 0x56214f9bea30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0005160 tx=0x7fbfc0005260 comp rx=0 tx=0).stop 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 shutdown_connections 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9bc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9aedf0 0x56214f9bbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214f9cebc0 0x56214f9d0fb0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/3034965515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9ae4d0 0x56214f9bea30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfb17fa640 0 client.15903 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 >> 172.21.15.33:0/3034965515 conn(0x56214f9ac300 msgr2=0x56214f9ab420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 shutdown_connections 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 1 -- 172.21.15.33:0/3034965515 wait complete. 2024-09-10T21:07:07.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.253+0000 7fbfd25bc740 20 client.15903 trim_cache size 0 max 16384 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.267+0000 7fbfd25bc740 1 Processor -- start 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.267+0000 7fbfd25bc740 1 -- start start 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9bdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9bf190 0x56214f9d66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ac2b0 con 0x56214f9aedf0 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81adf70 con 0x56214f9ae4d0 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ac980 con 0x56214f9bf190 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9bdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9bdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46498/0 (socket says 172.21.15.33:46498) 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 -- 172.21.15.33:0/2819302100 learned_addr learned my addr 172.21.15.33:0/2819302100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:07.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfce784640 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfce784640 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9bf190 0x56214f9d66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 -- 172.21.15.33:0/2819302100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9bf190 msgr2=0x56214f9d66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9bf190 0x56214f9d66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 -- 172.21.15.33:0/2819302100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 msgr2=0x56214f9cb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9cb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 -- 172.21.15.33:0/2819302100 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214f9bbc30 con 0x56214f9aedf0 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfce784640 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9bf190 0x56214f9d66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.268+0000 7fbfcd782640 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9bdbc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x56214f9d1690 tx=0x7fbfb80044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395e2125c02cc521 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfb37fe640 1 -- 172.21.15.33:0/2819302100 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8010430 con 0x56214f9aedf0 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfb37fe640 1 -- 172.21.15.33:0/2819302100 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8010a80 con 0x56214f9aedf0 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfb37fe640 1 -- 172.21.15.33:0/2819302100 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb801ea60 con 0x56214f9aedf0 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2819302100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 msgr2=0x56214f9bdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9bdbc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x56214f9d1690 tx=0x7fbfb80044d0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2819302100 shutdown_connections 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9bf190 0x56214f9d66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9bdbc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2819302100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9ae4d0 0x56214f9cb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2819302100 >> 172.21.15.33:0/2819302100 conn(0x56214f9ac300 msgr2=0x56214f9bd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2819302100 shutdown_connections 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.269+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2819302100 wait complete. 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.270+0000 7fbfd25bc740 1 -- start start 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.270+0000 7fbfd25bc740 1 Processor -- start 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9ae4d0 0x56214f9c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9c7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfd25bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ad7e0 con 0x56214f9aedf0 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81ad050 con 0x56214f9bf190 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfd25bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfc81aa270 con 0x56214f9ae4d0 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfd25bc740 10 client.0 _ll_register_callbacks cb 0x7fbfb80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfce784640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9ae4d0 0x56214f9c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfcd782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9c7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfcd782640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9c7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46508/0 (socket says 172.21.15.33:46508) 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfce784640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9ae4d0 0x56214f9c75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42810/0 (socket says 172.21.15.33:42810) 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfcd782640 1 -- 172.21.15.33:0/2318381619 learned_addr learned my addr 172.21.15.33:0/2318381619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:07.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.271+0000 7fbfce784640 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfcd782640 1 -- 172.21.15.33:0/2318381619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9ae4d0 msgr2=0x56214f9c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfcd782640 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9ae4d0 0x56214f9c75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfcd782640 1 -- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 msgr2=0x56214f9c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfcd782640 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfcd782640 1 -- 172.21.15.33:0/2318381619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214f9c4b20 con 0x56214f9aedf0 2024-09-10T21:07:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfcd782640 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9c7b10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009dd0 tx=0x7fbfb8003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36428d2fbdb8160 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:07.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfb17fa640 1 -- 172.21.15.33:0/2318381619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8014040 con 0x56214f9aedf0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfb17fa640 1 -- 172.21.15.33:0/2318381619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbfb801e4a0 con 0x56214f9aedf0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfb8010a80 con 0x56214f9aedf0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.272+0000 7fbfb17fa640 1 -- 172.21.15.33:0/2318381619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fbfb8010430 con 0x56214f9aedf0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 10 client.15909 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56214fa3c670 con 0x56214f9aedf0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfb17fa640 1 -- 172.21.15.33:0/2318381619 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfb8010a80 con 0x56214f9aedf0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfb17fa640 1 -- 172.21.15.33:0/2318381619 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7fbfb8031420 con 0x56214f9aedf0 2024-09-10T21:07:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 10 client.15909 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 10 client.15909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 20 client.15909 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214fa3cd70 0x56214fa3f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 4 client.15909 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56214f9228a0 con 0x56214fa3cd70 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.273+0000 7fbfce784640 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214fa3cd70 0x56214fa3f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfce784640 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214fa3cd70 0x56214fa3f190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64aab4fdf3f7faed server_cookie=3e1ebc5e2f4791d6 in_seq=0 out_seq=0 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfb17fa640 10 client.15909 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfb17fa640 1 -- 172.21.15.33:0/2318381619 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56214f9228a0 con 0x56214fa3cd70 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfb17fa640 10 client.15909 handle_command_reply: tid=0 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 client.15909 shutdown 2024-09-10T21:07:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 msgr2=0x56214f9c7b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9c7b10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbfb8009dd0 tx=0x7fbfb8003dc0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 shutdown_connections 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56214f9bf190 0x56214f9c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56214fa3cd70 0x56214fa3f190 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56214f9aedf0 0x56214f9c7b10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 --2- 172.21.15.33:0/2318381619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56214f9ae4d0 0x56214f9c75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfb17fa640 0 client.15909 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 >> 172.21.15.33:0/2318381619 conn(0x56214f9ac300 msgr2=0x56214f9bdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 shutdown_connections 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 1 -- 172.21.15.33:0/2318381619 wait complete. 2024-09-10T21:07:07.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.274+0000 7fbfd25bc740 20 client.15909 trim_cache size 0 max 16384 2024-09-10T21:07:07.298 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:07.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbfb40b5360 msgr2=0x7fbfb40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbfb40b5360 0x7fbfb40b7820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4031440 tx=0x7fbfc402a410 comp rx=0 tx=0).stop 2024-09-10T21:07:07.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 msgr2=0x7fbfc81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc81992f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4005940 tx=0x7fbfc40040c0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 shutdown_connections 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbfb40b5360 0x7fbfb40b7820 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfc8108020 0x7fbfc819ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfc81076a0 0x7fbfc8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 --2- 172.21.15.33:0/3302546462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfc8101c90 0x7fbfc81992f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 >> 172.21.15.33:0/3302546462 conn(0x7fbfc80fd700 msgr2=0x7fbfc80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 shutdown_connections 2024-09-10T21:07:07.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:07.275+0000 7fbfd01e7640 1 -- 172.21.15.33:0/3302546462 wait complete. 2024-09-10T21:07:07.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 258183 10.92 MB/sec execute 8 sec latency 12.227 ms 2024-09-10T21:07:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:07 smithi104 ceph-mon[35325]: pgmap v440: 97 pgs: 97 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-09-10T21:07:07.881 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:07 smithi033 ceph-mon[30860]: pgmap v440: 97 pgs: 97 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-09-10T21:07:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:07 smithi174 ceph-mon[35635]: pgmap v440: 97 pgs: 97 active+clean; 116 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 324 op/s 2024-09-10T21:07:07.984 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:07:08.261 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:08.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 259691 10.63 MB/sec execute 9 sec latency 19.383 ms 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.726+0000 7f6cf5008640 1 -- 172.21.15.33:0/1587140729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010a9d0 msgr2=0x7f6cf010ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.726+0000 7f6cf5008640 1 --2- 172.21.15.33:0/1587140729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010a9d0 0x7f6cf010ae10 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009900 tx=0x7f6ce402a410 comp rx=0 tx=0).stop 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.726+0000 7f6cf5008640 1 -- 172.21.15.33:0/1587140729 shutdown_connections 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.726+0000 7f6cf5008640 1 --2- 172.21.15.33:0/1587140729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cf010b350 0x7f6cf0111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.726+0000 7f6cf5008640 1 --2- 172.21.15.33:0/1587140729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010a9d0 0x7f6cf010ae10 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.726+0000 7f6cf5008640 1 --2- 172.21.15.33:0/1587140729 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cf0104fc0 0x7f6cf01053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.726+0000 7f6cf5008640 1 -- 172.21.15.33:0/1587140729 >> 172.21.15.33:0/1587140729 conn(0x7f6cf01009f0 msgr2=0x7f6cf0102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.727+0000 7f6cf5008640 1 -- 172.21.15.33:0/1587140729 shutdown_connections 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.727+0000 7f6cf5008640 1 -- 172.21.15.33:0/1587140729 wait complete. 2024-09-10T21:07:08.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.727+0000 7f6cf5008640 1 Processor -- start 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cf5008640 1 -- start start 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cf5008640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cf0104fc0 0x7f6cf019b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cf5008640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cf010a9d0 0x7f6cf019b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cf5008640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010b350 0x7f6cf01a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cf5008640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0114f50 con 0x7f6cf010b350 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cf5008640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0114dd0 con 0x7f6cf010a9d0 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cf5008640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01150d0 con 0x7f6cf0104fc0 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010b350 0x7f6cf01a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010b350 0x7f6cf01a0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46518/0 (socket says 172.21.15.33:46518) 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cf010a9d0 0x7f6cf019b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 -- 172.21.15.33:0/2848487620 learned_addr learned my addr 172.21.15.33:0/2848487620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:08.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cf0104fc0 0x7f6cf019b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 -- 172.21.15.33:0/2848487620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cf0104fc0 msgr2=0x7f6cf019b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cf0104fc0 0x7f6cf019b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 -- 172.21.15.33:0/2848487620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cf010a9d0 msgr2=0x7f6cf019b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cf010a9d0 0x7f6cf019b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 -- 172.21.15.33:0/2848487620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf01a1510 con 0x7f6cf010b350 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.728+0000 7f6cef59e640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010b350 0x7f6cf01a0d40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800eb10 tx=0x7f6ce800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.729+0000 7f6cd7fff640 1 -- 172.21.15.33:0/2848487620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8004b20 con 0x7f6cf010b350 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.729+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf01a1800 con 0x7f6cf010b350 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.729+0000 7f6cd7fff640 1 -- 172.21.15.33:0/2848487620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce8004cc0 con 0x7f6cf010b350 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.729+0000 7f6cd7fff640 1 -- 172.21.15.33:0/2848487620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8010670 con 0x7f6cf010b350 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.729+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf01a7ec0 con 0x7f6cf010b350 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cd7fff640 1 -- 172.21.15.33:0/2848487620 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6ce8005020 con 0x7f6cf010b350 2024-09-10T21:07:08.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 Processor -- start 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 -- start start 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e270784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e27078a10 0x560e27084460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27086d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e26feeea0 con 0x560e270849a0 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e26fde2d0 con 0x560e270780f0 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e26fe20e0 con 0x560e27078a10 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e270784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e270784d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:44184/0 (socket says 172.21.15.33:44184) 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cee59c640 1 -- 172.21.15.33:0/160997519 learned_addr learned my addr 172.21.15.33:0/160997519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cd7fff640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6cd80b52e0 0x7f6cd80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6cee59c640 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27086d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.732+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e27078a10 0x560e27084460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cef59e640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6cd80b52e0 0x7f6cd80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cee59c640 1 -- 172.21.15.33:0/160997519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e27078a10 msgr2=0x560e27084460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cee59c640 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e27078a10 0x560e27084460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cd7fff640 1 -- 172.21.15.33:0/2848487620 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6ce80e61c0 con 0x7f6cf010b350 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cee59c640 1 -- 172.21.15.33:0/160997519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 msgr2=0x560e27086d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cee59c640 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27086d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cee59c640 1 -- 172.21.15.33:0/160997519 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e26d3c650 con 0x560e270780f0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cee59c640 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e270784d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ce402a930 tx=0x7f6ce4005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668b201fd5a91908 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cef59e640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6cd80b52e0 0x7f6cd80b77a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80ac910 tx=0x7f6ce8075b20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cd57fa640 1 -- 172.21.15.33:0/160997519 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4035030 con 0x560e270780f0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cd57fa640 1 -- 172.21.15.33:0/160997519 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce4004030 con 0x560e270780f0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.733+0000 7f6cd57fa640 1 -- 172.21.15.33:0/160997519 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4003a10 con 0x560e270780f0 2024-09-10T21:07:08.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 -- 172.21.15.33:0/160997519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 msgr2=0x560e270784d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e270784d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6ce402a930 tx=0x7f6ce4005b50 comp rx=0 tx=0).stop 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 -- 172.21.15.33:0/160997519 shutdown_connections 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27086d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e27078a10 0x560e27084460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/160997519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e270784d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 -- 172.21.15.33:0/160997519 >> 172.21.15.33:0/160997519 conn(0x560e27075f20 msgr2=0x560e270770c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 -- 172.21.15.33:0/160997519 shutdown_connections 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.734+0000 7f6cf73dd740 1 -- 172.21.15.33:0/160997519 wait complete. 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 -- start start 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 Processor -- start 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e270842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e27081710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27081d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560e26e3f700 con 0x560e270849a0 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560e26f79160 con 0x560e27078a10 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e270875c0 con 0x560e270780f0 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cf73dd740 10 client.0 _ll_register_callbacks cb 0x560e26f929e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e27081710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27081d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e27081710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:44200/0 (socket says 172.21.15.33:44200) 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6ceed9d640 1 -- 172.21.15.33:0/893671211 learned_addr learned my addr 172.21.15.33:0/893671211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.735+0000 7f6cee59c640 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e270842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6ceed9d640 1 -- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 msgr2=0x560e270842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e270842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6ceed9d640 1 -- 172.21.15.33:0/893671211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 msgr2=0x560e27081d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27081d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6ceed9d640 1 -- 172.21.15.33:0/893671211 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e26fa6c90 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cee59c640 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27081d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e27081710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0012740 tx=0x7f6ce0012c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9065565398a6ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/893671211 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000f920 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ce4004030 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/893671211 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6ce000fac0 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/893671211 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ce000fcc0 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cf73dd740 10 client.25493 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.736+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560e27093000 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/893671211 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce4004030 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/893671211 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f6ce001c420 con 0x560e27078a10 2024-09-10T21:07:08.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cf73dd740 10 client.25493 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cf73dd740 10 client.25493 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cf73dd740 20 client.25493 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27094520 0x560e27096940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cf73dd740 4 client.25493 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e26ff0df0 con 0x560e27094520 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.737+0000 7f6cee59c640 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27094520 0x560e27096940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.738+0000 7f6cd2ff5640 10 client.25493 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.738+0000 7f6cee59c640 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27094520 0x560e27096940 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=533df27642fc15bd server_cookie=3ef7284d2d1212ca in_seq=0 out_seq=0 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.738+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/893671211 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e26ff0df0 con 0x560e27094520 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.738+0000 7f6cd2ff5640 10 client.25493 handle_command_reply: tid=0 2024-09-10T21:07:08.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.738+0000 7f6cf73dd740 1 client.25493 shutdown 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 msgr2=0x560e27081710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e27081710 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0012740 tx=0x7f6ce0012c00 comp rx=0 tx=0).stop 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 shutdown_connections 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27094520 0x560e27096940 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27081d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e27081710 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cd2ff5640 0 client.25493 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:08.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/893671211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e270842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.739+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 >> 172.21.15.33:0/893671211 conn(0x560e27075f20 msgr2=0x560e2707ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.740+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 shutdown_connections 2024-09-10T21:07:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.740+0000 7f6cf73dd740 1 -- 172.21.15.33:0/893671211 wait complete. 2024-09-10T21:07:08.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.740+0000 7f6cf73dd740 20 client.25493 trim_cache size 0 max 16384 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 Processor -- start 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 -- start start 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e27077680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e27078a10 0x560e27075760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270849a0 0x560e27085d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b0590 con 0x560e27078a10 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b2250 con 0x560e270780f0 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b0c60 con 0x560e270849a0 2024-09-10T21:07:08.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e27078a10 0x560e27075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e27078a10 0x560e27075760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46546/0 (socket says 172.21.15.33:46546) 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e27077680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6ceed9d640 1 -- 172.21.15.33:0/3844959070 learned_addr learned my addr 172.21.15.33:0/3844959070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cee59c640 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270849a0 0x560e27085d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cee59c640 1 -- 172.21.15.33:0/3844959070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270849a0 msgr2=0x560e27085d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cee59c640 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270849a0 0x560e27085d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cee59c640 1 -- 172.21.15.33:0/3844959070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e27078a10 msgr2=0x560e27075760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cee59c640 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e27078a10 0x560e27075760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6cee59c640 1 -- 172.21.15.33:0/3844959070 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e27082320 con 0x560e270780f0 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.755+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e27078a10 0x560e27075760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.756+0000 7f6cee59c640 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e27077680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40036f0 tx=0x7f6ce4009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84a7da453d38ba8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.756+0000 7f6cd4ff9640 1 -- 172.21.15.33:0/3844959070 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4035030 con 0x560e270780f0 2024-09-10T21:07:08.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.756+0000 7f6cd4ff9640 1 -- 172.21.15.33:0/3844959070 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce405c070 con 0x560e270780f0 2024-09-10T21:07:08.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.756+0000 7f6cd4ff9640 1 -- 172.21.15.33:0/3844959070 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40055c0 con 0x560e270780f0 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 -- 172.21.15.33:0/3844959070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 msgr2=0x560e27077680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e27077680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6ce40036f0 tx=0x7f6ce4009580 comp rx=0 tx=0).stop 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 -- 172.21.15.33:0/3844959070 shutdown_connections 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270849a0 0x560e27085d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e27078a10 0x560e27075760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/3844959070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e270780f0 0x560e27077680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 -- 172.21.15.33:0/3844959070 >> 172.21.15.33:0/3844959070 conn(0x560e27075f20 msgr2=0x560e27076f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 -- 172.21.15.33:0/3844959070 shutdown_connections 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 -- 172.21.15.33:0/3844959070 wait complete. 2024-09-10T21:07:08.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 -- start start 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.757+0000 7f6cf73dd740 1 Processor -- start 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e2708ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e2708cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cf73dd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27087840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b1ac0 con 0x560e270849a0 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01b1330 con 0x560e27078a10 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cf73dd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf01ae550 con 0x560e270780f0 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cf73dd740 10 client.0 _ll_register_callbacks cb 0x7f6ce0009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e2708cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e2708cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:44216/0 (socket says 172.21.15.33:44216) 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cee59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27087840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 -- 172.21.15.33:0/1914790974 learned_addr learned my addr 172.21.15.33:0/1914790974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:08.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cee59c640 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e2708ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 -- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 msgr2=0x560e2708ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e2708ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 -- 172.21.15.33:0/1914790974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 msgr2=0x560e27087840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27087840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 -- 172.21.15.33:0/1914790974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e270769a0 con 0x560e27078a10 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cee59c640 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27087840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6ceed9d640 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e2708cfb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009e60 tx=0x7f6ce001cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edff70f982aa1ae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cd2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.758+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/1914790974 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0009e90 con 0x560e27078a10 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ce405c070 con 0x560e27078a10 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/1914790974 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6ce0018040 con 0x560e27078a10 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/1914790974 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f6ce0036050 con 0x560e27078a10 2024-09-10T21:07:08.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 10 client.25499 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x560e27101ba0 con 0x560e27078a10 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/1914790974 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ce405c070 con 0x560e27078a10 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/1914790974 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f6ce00035f0 con 0x560e27078a10 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 10 client.25499 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 10 client.25499 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 20 client.25499 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27102460 0x560e27104880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 4 client.25499 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e26fec8a0 con 0x560e27102460 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.759+0000 7f6cee59c640 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27102460 0x560e27104880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cd2ff5640 10 client.25499 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cee59c640 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27102460 0x560e27104880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2df539a1c563d1d server_cookie=de47212126872cad in_seq=0 out_seq=0 2024-09-10T21:07:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cd2ff5640 1 -- 172.21.15.33:0/1914790974 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e26fec8a0 con 0x560e27102460 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cd2ff5640 10 client.25499 handle_command_reply: tid=0 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 client.25499 shutdown 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 msgr2=0x560e2708cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e2708cfb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009e60 tx=0x7f6ce001cd60 comp rx=0 tx=0).stop 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 shutdown_connections 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560e27102460 0x560e27104880 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560e270849a0 0x560e27087840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cd2ff5640 0 client.25499 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560e27078a10 0x560e2708cfb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 --2- 172.21.15.33:0/1914790974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e270780f0 0x560e2708ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 >> 172.21.15.33:0/1914790974 conn(0x560e27075f20 msgr2=0x560e27082d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:08.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 shutdown_connections 2024-09-10T21:07:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 1 -- 172.21.15.33:0/1914790974 wait complete. 2024-09-10T21:07:08.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.760+0000 7f6cf73dd740 20 client.25499 trim_cache size 0 max 16384 2024-09-10T21:07:08.770 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6cd80b52e0 msgr2=0x7f6cd80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6cd80b52e0 0x7f6cd80b77a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80ac910 tx=0x7f6ce8075b20 comp rx=0 tx=0).stop 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010b350 msgr2=0x7f6cf01a0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010b350 0x7f6cf01a0d40 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800eb10 tx=0x7f6ce800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 shutdown_connections 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6cd80b52e0 0x7f6cd80b77a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cf010b350 0x7f6cf01a0d40 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6cf010a9d0 0x7f6cf019b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 --2- 172.21.15.33:0/2848487620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cf0104fc0 0x7f6cf019b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 >> 172.21.15.33:0/2848487620 conn(0x7f6cf01009f0 msgr2=0x7f6cf0102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 shutdown_connections 2024-09-10T21:07:08.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:08.761+0000 7f6cf5008640 1 -- 172.21.15.33:0/2848487620 wait complete. 2024-09-10T21:07:09.634 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:09 smithi033 ceph-mon[30860]: pgmap v441: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-09-10T21:07:09.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:07:09.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 262298 10.99 MB/sec execute 10 sec latency 19.940 ms 2024-09-10T21:07:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:09 smithi104 ceph-mon[35325]: pgmap v441: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-09-10T21:07:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:09 smithi174 ceph-mon[35635]: pgmap v441: 97 pgs: 97 active+clean; 118 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 303 op/s 2024-09-10T21:07:09.941 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:10.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.424+0000 7f28ba170640 1 -- 172.21.15.33:0/1968703697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 msgr2=0x7f28b410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.424+0000 7f28ba170640 1 --2- 172.21.15.33:0/1968703697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 0x7f28b410f550 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009ca0 tx=0x7f28ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- 172.21.15.33:0/1968703697 shutdown_connections 2024-09-10T21:07:10.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 --2- 172.21.15.33:0/1968703697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 0x7f28b410f550 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 --2- 172.21.15.33:0/1968703697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28b4104570 0x7f28b410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 --2- 172.21.15.33:0/1968703697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b4103b70 0x7f28b4103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- 172.21.15.33:0/1968703697 >> 172.21.15.33:0/1968703697 conn(0x7f28b40fd700 msgr2=0x7f28b40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- 172.21.15.33:0/1968703697 shutdown_connections 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- 172.21.15.33:0/1968703697 wait complete. 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 Processor -- start 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- start start 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28b4103b70 0x7f28b419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b4104570 0x7f28b419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 0x7f28b41a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4112d60 con 0x7f28b410d120 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4112be0 con 0x7f28b4103b70 2024-09-10T21:07:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.425+0000 7f28ba170640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28b4112ee0 con 0x7f28b4104570 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 0x7f28b41a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 0x7f28b41a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60352/0 (socket says 172.21.15.33:60352) 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b3fff640 1 -- 172.21.15.33:0/3650214456 learned_addr learned my addr 172.21.15.33:0/3650214456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b37fe640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28b4103b70 0x7f28b419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28abfff640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b4104570 0x7f28b419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b37fe640 1 -- 172.21.15.33:0/3650214456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b4104570 msgr2=0x7f28b419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b37fe640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b4104570 0x7f28b419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b37fe640 1 -- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 msgr2=0x7f28b41a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b37fe640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 0x7f28b41a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b37fe640 1 -- 172.21.15.33:0/3650214456 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b41a14d0 con 0x7f28b4103b70 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28abfff640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b4104570 0x7f28b419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b37fe640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28b4103b70 0x7f28b419b420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f28a400d720 tx=0x7f28a400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:10.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b17fa640 1 -- 172.21.15.33:0/3650214456 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a4017070 con 0x7f28b4103b70 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b17fa640 1 -- 172.21.15.33:0/3650214456 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a400de30 con 0x7f28b4103b70 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b41a17c0 con 0x7f28b4103b70 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28b17fa640 1 -- 172.21.15.33:0/3650214456 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40047a0 con 0x7f28b4103b70 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.426+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b41a7e80 con 0x7f28b4103b70 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 Processor -- start 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 -- start start 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28b17fa640 1 -- 172.21.15.33:0/3650214456 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f28a40099d0 con 0x7f28b4103b70 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c84d0 0x5599bc6c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c8df0 0x5599bc6d4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599bc63eea0 con 0x5599bc6d4d80 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5599bc62e2d0 con 0x5599bc6c8df0 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28bc545740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599bc6320e0 con 0x5599bc6c84d0 2024-09-10T21:07:10.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c8df0 0x5599bc6d4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28abfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60366/0 (socket says 172.21.15.33:60366) 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28abfff640 1 -- 172.21.15.33:0/1337424382 learned_addr learned my addr 172.21.15.33:0/1337424382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.429+0000 7f28b3fff640 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c84d0 0x5599bc6c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28abfff640 1 -- 172.21.15.33:0/1337424382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c84d0 msgr2=0x5599bc6c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28abfff640 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c84d0 0x5599bc6c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28abfff640 1 -- 172.21.15.33:0/1337424382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c8df0 msgr2=0x5599bc6d4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28abfff640 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c8df0 0x5599bc6d4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28abfff640 1 -- 172.21.15.33:0/1337424382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bc38c650 con 0x5599bc6d4d80 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28b17fa640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f28a00b52e0 0x7f28a00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28abfff640 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d7170 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f289c009cf0 tx=0x7f289c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bf2c5decf70baf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28a9ffb640 1 -- 172.21.15.33:0/1337424382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c012450 con 0x5599bc6d4d80 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28b3fff640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f28a00b52e0 0x7f28a00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28b17fa640 1 -- 172.21.15.33:0/3650214456 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f28a40e5890 con 0x7f28b4103b70 2024-09-10T21:07:10.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28a9ffb640 1 -- 172.21.15.33:0/1337424382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289c012aa0 con 0x5599bc6d4d80 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.430+0000 7f28a9ffb640 1 -- 172.21.15.33:0/1337424382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f289c00fb20 con 0x5599bc6d4d80 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28b3fff640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f28a00b52e0 0x7f28a00b77a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005e30 tx=0x7f28ac005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 -- 172.21.15.33:0/1337424382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 msgr2=0x5599bc6d7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d7170 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f289c009cf0 tx=0x7f289c0117b0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 -- 172.21.15.33:0/1337424382 shutdown_connections 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d7170 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c8df0 0x5599bc6d4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 --2- 172.21.15.33:0/1337424382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c84d0 0x5599bc6c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 -- 172.21.15.33:0/1337424382 >> 172.21.15.33:0/1337424382 conn(0x5599bc6c6300 msgr2=0x5599bc6c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 -- 172.21.15.33:0/1337424382 shutdown_connections 2024-09-10T21:07:10.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 -- 172.21.15.33:0/1337424382 wait complete. 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.431+0000 7f28bc545740 1 -- start start 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 1 Processor -- start 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5599bc48f700 con 0x5599bc6d4d80 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5599bc5c9160 con 0x5599bc6c84d0 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599bc6d7a00 con 0x5599bc6c8df0 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28bc545740 10 client.0 _ll_register_callbacks cb 0x5599bc5e29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28abfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60368/0 (socket says 172.21.15.33:60368) 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 -- 172.21.15.33:0/3353064988 learned_addr learned my addr 172.21.15.33:0/3353064988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 -- 172.21.15.33:0/3353064988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 msgr2=0x5599bc6d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 -- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 msgr2=0x5599bc6d4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b3fff640 1 -- 172.21.15.33:0/3353064988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bc5f6c90 con 0x5599bc6d4d80 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.432+0000 7f28b37fe640 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28b3fff640 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d1f10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009dc0 tx=0x7f28ac02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39faeefaaad44454 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f2892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f2892ffd640 1 -- 172.21.15.33:0/3353064988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac003b80 con 0x5599bc6d4d80 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f2892ffd640 1 -- 172.21.15.33:0/3353064988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f28ac003d20 con 0x5599bc6d4d80 2024-09-10T21:07:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f289c012aa0 con 0x5599bc6d4d80 2024-09-10T21:07:10.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f2892ffd640 1 -- 172.21.15.33:0/3353064988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28ac004040 con 0x5599bc6d4d80 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 10 client.15939 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599bc6e3240 con 0x5599bc6d4d80 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f2892ffd640 1 -- 172.21.15.33:0/3353064988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f289c012aa0 con 0x5599bc6d4d80 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f2892ffd640 1 -- 172.21.15.33:0/3353064988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f28ac051070 con 0x5599bc6d4d80 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 10 client.15939 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 10 client.15939 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 20 client.15939 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc6e4780 0x5599bc6e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 4 client.15939 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.433+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599bc640df0 con 0x5599bc6e4780 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.434+0000 7f28b37fe640 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc6e4780 0x5599bc6e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.434+0000 7f2892ffd640 10 client.15939 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.434+0000 7f28b37fe640 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc6e4780 0x5599bc6e6bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990ba0b5309b3a4 server_cookie=a9365a1239cff9f6 in_seq=0 out_seq=0 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f2892ffd640 1 -- 172.21.15.33:0/3353064988 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5599bc640df0 con 0x5599bc6e4780 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f2892ffd640 10 client.15939 handle_command_reply: tid=0 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 client.15939 shutdown 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 msgr2=0x5599bc6d1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d1f10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009dc0 tx=0x7f28ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 shutdown_connections 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc6e4780 0x5599bc6e6bc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d1f10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 --2- 172.21.15.33:0/3353064988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 >> 172.21.15.33:0/3353064988 conn(0x5599bc6c6300 msgr2=0x5599bc6cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f2892ffd640 0 client.15939 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 shutdown_connections 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 1 -- 172.21.15.33:0/3353064988 wait complete. 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.435+0000 7f28bc545740 20 client.15939 trim_cache size 0 max 16384 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.448+0000 7f28bc545740 1 Processor -- start 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.448+0000 7f28bc545740 1 -- start start 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28bc545740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d0250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28bc545740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28bc545740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28bc545740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41b0570 con 0x5599bc6d4d80 2024-09-10T21:07:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28bc545740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41b2230 con 0x5599bc6c84d0 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28bc545740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41b0c40 con 0x5599bc6c8df0 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60382/0 (socket says 172.21.15.33:60382) 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 -- 172.21.15.33:0/2190756984 learned_addr learned my addr 172.21.15.33:0/2190756984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28abfff640 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b37fe640 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 -- 172.21.15.33:0/2190756984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 msgr2=0x5599bc6d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 -- 172.21.15.33:0/2190756984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 msgr2=0x5599bc6d0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 -- 172.21.15.33:0/2190756984 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599bc6c6d80 con 0x5599bc6d4d80 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b37fe640 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:10.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28b3fff640 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6cebe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f28ac043040 tx=0x7f28ac0305c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02852088da5d86f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.449+0000 7f28a97fa640 1 -- 172.21.15.33:0/2190756984 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac02fdb0 con 0x5599bc6d4d80 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28a97fa640 1 -- 172.21.15.33:0/2190756984 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28ac036050 con 0x5599bc6d4d80 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28a97fa640 1 -- 172.21.15.33:0/2190756984 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ac004740 con 0x5599bc6d4d80 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28bc545740 1 -- 172.21.15.33:0/2190756984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 msgr2=0x5599bc6cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28bc545740 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6cebe0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f28ac043040 tx=0x7f28ac0305c0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28bc545740 1 -- 172.21.15.33:0/2190756984 shutdown_connections 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28bc545740 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6cebe0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28bc545740 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28bc545740 1 --2- 172.21.15.33:0/2190756984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6d0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.450+0000 7f28bc545740 1 -- 172.21.15.33:0/2190756984 >> 172.21.15.33:0/2190756984 conn(0x5599bc6c6300 msgr2=0x5599bc6ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.451+0000 7f28bc545740 1 -- 172.21.15.33:0/2190756984 shutdown_connections 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.451+0000 7f28bc545740 1 -- 172.21.15.33:0/2190756984 wait complete. 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.451+0000 7f28bc545740 1 -- start start 2024-09-10T21:07:10.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.451+0000 7f28bc545740 1 Processor -- start 2024-09-10T21:07:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28bc545740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6dd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28bc545740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28bc545740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28bc545740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41b1aa0 con 0x5599bc6d4d80 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28bc545740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41b1310 con 0x5599bc6c84d0 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28bc545740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28b41ae530 con 0x5599bc6c8df0 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28bc545740 10 client.0 _ll_register_callbacks cb 0x7f28ac031f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6dd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6dd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36338/0 (socket says 172.21.15.33:36338) 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 -- 172.21.15.33:0/885320376 learned_addr learned my addr 172.21.15.33:0/885320376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28abfff640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 -- 172.21.15.33:0/885320376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 msgr2=0x5599bc6d89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 -- 172.21.15.33:0/885320376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 msgr2=0x5599bc6d8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 -- 172.21.15.33:0/885320376 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599bc6d97d0 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28abfff640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b3fff640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f28b37fe640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6dd4c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28a40abfe0 tx=0x7f28a40b5040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c95adbdf32ca0a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.452+0000 7f2892ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f2892ffd640 1 -- 172.21.15.33:0/885320376 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a40abe20 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f2892ffd640 1 -- 172.21.15.33:0/885320376 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f28a40a82c0 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28ac036050 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f2892ffd640 1 -- 172.21.15.33:0/885320376 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f28a40ab7c0 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f28bc545740 10 client.25517 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5599bc7517d0 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f2892ffd640 1 -- 172.21.15.33:0/885320376 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28ac036050 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f2892ffd640 1 -- 172.21.15.33:0/885320376 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f28a4097170 con 0x5599bc6c84d0 2024-09-10T21:07:10.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f28bc545740 10 client.25517 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f28bc545740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f28bc545740 20 client.25517 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.453+0000 7f28bc545740 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc751f40 0x5599bc754360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f28bc545740 4 client.25517 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599bc63c8a0 con 0x5599bc751f40 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f28abfff640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc751f40 0x5599bc754360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f2892ffd640 10 client.25517 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f28abfff640 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc751f40 0x5599bc754360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86fbc567db2511 server_cookie=992eb8672fe0ad17 in_seq=0 out_seq=0 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f2892ffd640 1 -- 172.21.15.33:0/885320376 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599bc63c8a0 con 0x5599bc751f40 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f2892ffd640 10 client.25517 handle_command_reply: tid=0 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.454+0000 7f28bc545740 1 client.25517 shutdown 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 msgr2=0x5599bc6dd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6dd4c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f28a40abfe0 tx=0x7f28a40b5040 comp rx=0 tx=0).stop 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 shutdown_connections 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5599bc751f40 0x5599bc754360 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5599bc6d4d80 0x5599bc6d8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599bc6c8df0 0x5599bc6d89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f2892ffd640 0 client.25517 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 --2- 172.21.15.33:0/885320376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5599bc6c84d0 0x5599bc6dd4c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 >> 172.21.15.33:0/885320376 conn(0x5599bc6c6300 msgr2=0x5599bc6d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 shutdown_connections 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 1 -- 172.21.15.33:0/885320376 wait complete. 2024-09-10T21:07:10.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28bc545740 20 client.25517 trim_cache size 0 max 16384 2024-09-10T21:07:10.476 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f28a00b52e0 msgr2=0x7f28a00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f28a00b52e0 0x7f28a00b77a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f28ac005e30 tx=0x7f28ac005c50 comp rx=0 tx=0).stop 2024-09-10T21:07:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28b4103b70 msgr2=0x7f28b419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:10.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28b4103b70 0x7f28b419b420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f28a400d720 tx=0x7f28a400dbe0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 shutdown_connections 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f28a00b52e0 0x7f28a00b77a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28b410d120 0x7f28b41a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28b4104570 0x7f28b419b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 --2- 172.21.15.33:0/3650214456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f28b4103b70 0x7f28b419b420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.455+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 >> 172.21.15.33:0/3650214456 conn(0x7f28b40fd700 msgr2=0x7f28b410f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.456+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 shutdown_connections 2024-09-10T21:07:10.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:10.456+0000 7f28ba170640 1 -- 172.21.15.33:0/3650214456 wait complete. 2024-09-10T21:07:10.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 263620 10.60 MB/sec execute 11 sec latency 37.029 ms 2024-09-10T21:07:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:11 smithi033 ceph-mon[30860]: pgmap v442: 97 pgs: 97 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-10T21:07:11.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 266274 11.11 MB/sec execute 12 sec latency 15.397 ms 2024-09-10T21:07:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:11 smithi104 ceph-mon[35325]: pgmap v442: 97 pgs: 97 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-10T21:07:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:11 smithi174 ceph-mon[35635]: pgmap v442: 97 pgs: 97 active+clean; 119 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 321 op/s 2024-09-10T21:07:12.424 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:07:12.424 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:07:12.690 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:12.717 INFO:tasks.workunit.client.0.smithi033.stdout: 1 268360 10.81 MB/sec execute 13 sec latency 12.396 ms 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 -- 172.21.15.33:0/2634009674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc109130 msgr2=0x7f1acc10f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2634009674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc109130 0x7f1acc10f930 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4009ca0 tx=0x7f1ac402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 -- 172.21.15.33:0/2634009674 shutdown_connections 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2634009674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc109130 0x7f1acc10f930 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2634009674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1acc1087b0 0x7f1acc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2634009674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1acc102da0 0x7f1acc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 -- 172.21.15.33:0/2634009674 >> 172.21.15.33:0/2634009674 conn(0x7f1acc0fe810 msgr2=0x7f1acc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 -- 172.21.15.33:0/2634009674 shutdown_connections 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 -- 172.21.15.33:0/2634009674 wait complete. 2024-09-10T21:07:13.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.200+0000 7f1ad1509640 1 Processor -- start 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1ad1509640 1 -- start start 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1ad1509640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc102da0 0x7f1acc1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1ad1509640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1acc1087b0 0x7f1acc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1ad1509640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1acc109130 0x7f1acc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1ad1509640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112db0 con 0x7f1acc102da0 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1ad1509640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112c30 con 0x7f1acc1087b0 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc102da0 0x7f1acc1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1ad1509640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc112f30 con 0x7f1acc109130 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc102da0 0x7f1acc1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60430/0 (socket says 172.21.15.33:60430) 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 -- 172.21.15.33:0/2310576208 learned_addr learned my addr 172.21.15.33:0/2310576208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:13.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1acc1087b0 0x7f1acc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1acc109130 0x7f1acc19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 -- 172.21.15.33:0/2310576208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1acc109130 msgr2=0x7f1acc19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1acc109130 0x7f1acc19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 -- 172.21.15.33:0/2310576208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1acc1087b0 msgr2=0x7f1acc199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1acc1087b0 0x7f1acc199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acaffd640 1 -- 172.21.15.33:0/2310576208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc19f290 con 0x7f1acc102da0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.201+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1acc109130 0x7f1acc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.202+0000 7f1acaffd640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc102da0 0x7f1acc1991e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00e9f0 tx=0x7f1abc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.202+0000 7f1ab3fff640 1 -- 172.21.15.33:0/2310576208 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00cd80 con 0x7f1acc102da0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.202+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc19f580 con 0x7f1acc102da0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.202+0000 7f1ab3fff640 1 -- 172.21.15.33:0/2310576208 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc0108e0 con 0x7f1acc102da0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.202+0000 7f1ab3fff640 1 -- 172.21.15.33:0/2310576208 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc018980 con 0x7f1acc102da0 2024-09-10T21:07:13.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.202+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a5e10 con 0x7f1acc102da0 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.204+0000 7f1ad38de740 1 Processor -- start 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.204+0000 7f1ab3fff640 1 -- 172.21.15.33:0/2310576208 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1abc010430 con 0x7f1acc102da0 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1ad38de740 1 -- start start 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea96e22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea96ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea96f0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1ad38de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eea9658e60 con 0x55eea96e2800 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1ad38de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eea9648290 con 0x55eea96ee790 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1ad38de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eea964c0a0 con 0x55eea96e1ee0 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea96ee250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea96ee250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60432/0 (socket says 172.21.15.33:60432) 2024-09-10T21:07:13.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1acaffd640 1 -- 172.21.15.33:0/2016448631 learned_addr learned my addr 172.21.15.33:0/2016448631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea96f0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea96e22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1aca7fc640 1 -- 172.21.15.33:0/2016448631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 msgr2=0x55eea96e22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea96e22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1aca7fc640 1 -- 172.21.15.33:0/2016448631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 msgr2=0x55eea96ee250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea96ee250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1aca7fc640 1 -- 172.21.15.33:0/2016448631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea93a6930 con 0x55eea96ee790 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.205+0000 7f1acaffd640 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea96ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.206+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea96f0b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0013ba0 tx=0x7f1ac0009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13956d2aa00ea386 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.206+0000 7f1ab3fff640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ab40b52e0 0x7f1ab40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.206+0000 7f1ab17fa640 1 -- 172.21.15.33:0/2016448631 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0009e20 con 0x55eea96ee790 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.206+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ab40b52e0 0x7f1ab40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.206+0000 7f1ab3fff640 1 -- 172.21.15.33:0/2310576208 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1abc0e5f70 con 0x7f1acc102da0 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.206+0000 7f1ab17fa640 1 -- 172.21.15.33:0/2016448631 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac000e990 con 0x55eea96ee790 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.206+0000 7f1ab17fa640 1 -- 172.21.15.33:0/2016448631 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0003b30 con 0x55eea96ee790 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ab40b52e0 0x7f1ab40b77a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4004af0 tx=0x7f1ac4005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 -- 172.21.15.33:0/2016448631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 msgr2=0x55eea96f0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea96f0b80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0013ba0 tx=0x7f1ac0009b70 comp rx=0 tx=0).stop 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 -- 172.21.15.33:0/2016448631 shutdown_connections 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea96f0b80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea96ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 --2- 172.21.15.33:0/2016448631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea96e22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 -- 172.21.15.33:0/2016448631 >> 172.21.15.33:0/2016448631 conn(0x55eea96dfd10 msgr2=0x55eea96e0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 -- 172.21.15.33:0/2016448631 shutdown_connections 2024-09-10T21:07:13.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.207+0000 7f1ad38de740 1 -- 172.21.15.33:0/2016448631 wait complete. 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 -- start start 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 Processor -- start 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96e1ee0 0x55eea96eb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea96ebc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96ee790 0x55eea96ec160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eea94a9890 con 0x55eea96ee790 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eea95e2d40 con 0x55eea96e1ee0 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eea96f1490 con 0x55eea96e2800 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1ad38de740 10 client.0 _ll_register_callbacks cb 0x55eea9646c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea96ebc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96e1ee0 0x55eea96eb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1acb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96ee790 0x55eea96ec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea96ebc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:45050/0 (socket says 172.21.15.33:45050) 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96e1ee0 0x55eea96eb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36366/0 (socket says 172.21.15.33:36366) 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.208+0000 7f1aca7fc640 1 -- 172.21.15.33:0/11334479 learned_addr learned my addr 172.21.15.33:0/11334479 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aca7fc640 1 -- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96e1ee0 msgr2=0x55eea96eb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96e1ee0 0x55eea96eb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aca7fc640 1 -- 172.21.15.33:0/11334479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96ee790 msgr2=0x55eea96ec160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96ee790 0x55eea96ec160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aca7fc640 1 -- 172.21.15.33:0/11334479 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea96fa820 con 0x55eea96e2800 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1acaffd640 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96e1ee0 0x55eea96eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea96ebc20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0003630 tx=0x7f1ac000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55db43dd051d138c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aaeff5640 1 -- 172.21.15.33:0/11334479 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0004050 con 0x55eea96e2800 2024-09-10T21:07:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ac000e990 con 0x55eea96e2800 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aaeff5640 1 -- 172.21.15.33:0/11334479 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1ac0019070 con 0x55eea96e2800 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1aaeff5640 1 -- 172.21.15.33:0/11334479 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ac00041f0 con 0x55eea96e2800 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1ad38de740 10 client.25507 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.209+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eea96fd1d0 con 0x55eea96e2800 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1aaeff5640 1 -- 172.21.15.33:0/11334479 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ac000e990 con 0x55eea96e2800 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1aaeff5640 1 -- 172.21.15.33:0/11334479 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f1ac001d9f0 con 0x55eea96e2800 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1ad38de740 10 client.25507 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1ad38de740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1ad38de740 20 client.25507 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1ad38de740 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea96fe770 0x55eea9700b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1ad38de740 4 client.25507 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eea965adb0 con 0x55eea96fe770 2024-09-10T21:07:13.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1acaffd640 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea96fe770 0x55eea9700b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1aaeff5640 10 client.25507 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.210+0000 7f1acaffd640 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea96fe770 0x55eea9700b60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d170693d13f425 server_cookie=8d02cbae310b467b in_seq=0 out_seq=0 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1aaeff5640 1 -- 172.21.15.33:0/11334479 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eea965adb0 con 0x55eea96fe770 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1aaeff5640 10 client.25507 handle_command_reply: tid=0 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 client.25507 shutdown 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 msgr2=0x55eea96ebc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea96ebc20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0003630 tx=0x7f1ac000ef90 comp rx=0 tx=0).stop 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 shutdown_connections 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea96fe770 0x55eea9700b60 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96ee790 0x55eea96ec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea96ebc20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 --2- 172.21.15.33:0/11334479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96e1ee0 0x55eea96eb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 >> 172.21.15.33:0/11334479 conn(0x55eea96dfd10 msgr2=0x55eea96def70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1aaeff5640 0 client.25507 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 shutdown_connections 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 1 -- 172.21.15.33:0/11334479 wait complete. 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.212+0000 7f1ad38de740 20 client.25507 trim_cache size 0 max 16384 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 Processor -- start 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 -- start start 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e1ee0 0x55eea96e9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea9703680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea9703bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1ac230 con 0x55eea96e1ee0 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1adef0 con 0x55eea96ee790 2024-09-10T21:07:13.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1ad38de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1ac900 con 0x55eea96e2800 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e1ee0 0x55eea96e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e1ee0 0x55eea96e9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60450/0 (socket says 172.21.15.33:60450) 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1acaffd640 1 -- 172.21.15.33:0/1193832399 learned_addr learned my addr 172.21.15.33:0/1193832399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea9703bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.231+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea9703680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1acaffd640 1 -- 172.21.15.33:0/1193832399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 msgr2=0x55eea9703680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1acaffd640 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea9703680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1acaffd640 1 -- 172.21.15.33:0/1193832399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 msgr2=0x55eea9703bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1acaffd640 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea9703bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1acaffd640 1 -- 172.21.15.33:0/1193832399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eea96db640 con 0x55eea96e1ee0 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea9703680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:07:13.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea9703bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1acaffd640 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e1ee0 0x55eea96e9210 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0108b0 tx=0x7f1abc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535fb25f91f1ca39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1ab0ff9640 1 -- 172.21.15.33:0/1193832399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0abea0 con 0x55eea96e1ee0 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1ab0ff9640 1 -- 172.21.15.33:0/1193832399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc0ac240 con 0x55eea96e1ee0 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.232+0000 7f1ab0ff9640 1 -- 172.21.15.33:0/1193832399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc0b5830 con 0x55eea96e1ee0 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 -- 172.21.15.33:0/1193832399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e1ee0 msgr2=0x55eea96e9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e1ee0 0x55eea96e9210 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0108b0 tx=0x7f1abc018420 comp rx=0 tx=0).stop 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 -- 172.21.15.33:0/1193832399 shutdown_connections 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea9703bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e2800 0x55eea9703680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1193832399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e1ee0 0x55eea96e9210 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 -- 172.21.15.33:0/1193832399 >> 172.21.15.33:0/1193832399 conn(0x55eea96dfd10 msgr2=0x55eea96e8b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 -- 172.21.15.33:0/1193832399 shutdown_connections 2024-09-10T21:07:13.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 -- 172.21.15.33:0/1193832399 wait complete. 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 -- start start 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 Processor -- start 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea97013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea9701920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea976a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.233+0000 7f1ad38de740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1ad760 con 0x55eea96e2800 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1ad38de740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1acfd0 con 0x55eea96ee790 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1ad38de740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1aa1f0 con 0x55eea96e1ee0 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1ad38de740 10 client.0 _ll_register_callbacks cb 0x7f1abc0e98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea9701920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea9701920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60466/0 (socket says 172.21.15.33:60466) 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1acaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea97013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 -- 172.21.15.33:0/1480971744 learned_addr learned my addr 172.21.15.33:0/1480971744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1acb7fe640 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea976a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 -- 172.21.15.33:0/1480971744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 msgr2=0x55eea97013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea97013e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 -- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 msgr2=0x55eea976a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea976a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 -- 172.21.15.33:0/1480971744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eea976a9c0 con 0x55eea96e2800 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aca7fc640 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea9701920 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55eea96edfd0 tx=0x7f1ac000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb420d196dfc489 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aaeff5640 1 -- 172.21.15.33:0/1480971744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0005640 con 0x55eea96e2800 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1aaeff5640 1 -- 172.21.15.33:0/1480971744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1ac0019040 con 0x55eea96e2800 2024-09-10T21:07:13.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.234+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1abc0ac240 con 0x55eea96e2800 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1aaeff5640 1 -- 172.21.15.33:0/1480971744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f1ac0003ac0 con 0x55eea96e2800 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 10 client.15963 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55eea976bce0 con 0x55eea96e2800 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1aaeff5640 1 -- 172.21.15.33:0/1480971744 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc0ac240 con 0x55eea96e2800 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1aaeff5640 1 -- 172.21.15.33:0/1480971744 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f1ac0042030 con 0x55eea96e2800 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 10 client.15963 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 20 client.15963 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea976c480 0x55eea976e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 4 client.15963 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55eea9656860 con 0x55eea976c480 2024-09-10T21:07:13.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.235+0000 7f1acaffd640 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea976c480 0x55eea976e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.236+0000 7f1aaeff5640 10 client.15963 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.236+0000 7f1acaffd640 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea976c480 0x55eea976e8a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f975fa8cfb5465c7 server_cookie=7ea8dca185b876ff in_seq=0 out_seq=0 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1aaeff5640 1 -- 172.21.15.33:0/1480971744 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eea9656860 con 0x55eea976c480 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1aaeff5640 10 client.15963 handle_command_reply: tid=0 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "107d5f4a-5704-4eb9-a25b-e2fdce4273a9", 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:07:13.259 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:07:13.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 client.15963 shutdown 2024-09-10T21:07:13.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 msgr2=0x55eea9701920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea9701920 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x55eea96edfd0 tx=0x7f1ac000eed0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 shutdown_connections 2024-09-10T21:07:13.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55eea976c480 0x55eea976e8a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eea96ee790 0x55eea976a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eea96e2800 0x55eea9701920 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 --2- 172.21.15.33:0/1480971744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eea96e1ee0 0x55eea97013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1aaeff5640 0 client.15963 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 >> 172.21.15.33:0/1480971744 conn(0x55eea96dfd10 msgr2=0x55eea96ecd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 shutdown_connections 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.237+0000 7f1ad38de740 1 -- 172.21.15.33:0/1480971744 wait complete. 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad38de740 20 client.15963 trim_cache size 0 max 16384 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ab40b52e0 msgr2=0x7f1ab40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ab40b52e0 0x7f1ab40b77a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4004af0 tx=0x7f1ac4005d80 comp rx=0 tx=0).stop 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc102da0 msgr2=0x7f1acc1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc102da0 0x7f1acc1991e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00e9f0 tx=0x7f1abc00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 shutdown_connections 2024-09-10T21:07:13.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ab40b52e0 0x7f1ab40b77a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1acc109130 0x7f1acc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1acc1087b0 0x7f1acc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 --2- 172.21.15.33:0/2310576208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1acc102da0 0x7f1acc1991e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.238+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 >> 172.21.15.33:0/2310576208 conn(0x7f1acc0fe810 msgr2=0x7f1acc0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.239+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 shutdown_connections 2024-09-10T21:07:13.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:13.239+0000 7f1ad1509640 1 -- 172.21.15.33:0/2310576208 wait complete. 2024-09-10T21:07:13.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 270033 10.91 MB/sec execute 14 sec latency 40.422 ms 2024-09-10T21:07:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:13 smithi104 ceph-mon[35325]: pgmap v443: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-09-10T21:07:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:07:13.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:07:13.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:07:13.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:07:13.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:13 smithi033 ceph-mon[30860]: pgmap v443: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-09-10T21:07:13.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:07:13.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:07:13.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:07:13.848 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:07:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:13 smithi174 ceph-mon[35635]: pgmap v443: 97 pgs: 97 active+clean; 120 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 338 op/s 2024-09-10T21:07:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:07:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:07:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:07:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:07:14.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:07:14.466 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:14.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 272480 10.94 MB/sec execute 15 sec latency 35.240 ms 2024-09-10T21:07:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:07:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:07:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:07:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:07:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:07:14.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.746 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:07:14.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:07:14.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:07:14.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 -- 172.21.15.33:0/2753067753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c1076a0 msgr2=0x7f820c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:14.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 --2- 172.21.15.33:0/2753067753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c1076a0 0x7f820c107ae0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009900 tx=0x7f81f402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 -- 172.21.15.33:0/2753067753 shutdown_connections 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 --2- 172.21.15.33:0/2753067753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c108020 0x7f820c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 --2- 172.21.15.33:0/2753067753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c1076a0 0x7f820c107ae0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 --2- 172.21.15.33:0/2753067753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f820c101c90 0x7f820c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 -- 172.21.15.33:0/2753067753 >> 172.21.15.33:0/2753067753 conn(0x7f820c0fd700 msgr2=0x7f820c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.976+0000 7f8212d55640 1 -- 172.21.15.33:0/2753067753 shutdown_connections 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 -- 172.21.15.33:0/2753067753 wait complete. 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 Processor -- start 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 -- start start 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c101c90 0x7f820c0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f820c1076a0 0x7f820c0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c108020 0x7f820c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111d30 con 0x7f820c108020 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111bb0 con 0x7f820c101c90 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.977+0000 7f8212d55640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111eb0 con 0x7f820c1076a0 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c101c90 0x7f820c0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f82112f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c108020 0x7f820c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c101c90 0x7f820c0fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36400/0 (socket says 172.21.15.33:36400) 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f820bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f820c1076a0 0x7f820c0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 -- 172.21.15.33:0/2140373814 learned_addr learned my addr 172.21.15.33:0/2140373814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 -- 172.21.15.33:0/2140373814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f820c1076a0 msgr2=0x7f820c0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f820c1076a0 0x7f820c0fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 -- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c108020 msgr2=0x7f820c0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c108020 0x7f820c0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 -- 172.21.15.33:0/2140373814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c1a76d0 con 0x7f820c101c90 2024-09-10T21:07:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f82112f2640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c108020 0x7f820c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8210af1640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c101c90 0x7f820c0fdc00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00e9f0 tx=0x7f81fc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8209ffb640 1 -- 172.21.15.33:0/2140373814 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc00cd80 con 0x7f820c101c90 2024-09-10T21:07:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8209ffb640 1 -- 172.21.15.33:0/2140373814 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81fc0108e0 con 0x7f820c101c90 2024-09-10T21:07:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8209ffb640 1 -- 172.21.15.33:0/2140373814 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc018980 con 0x7f820c101c90 2024-09-10T21:07:14.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.978+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c1a79c0 con 0x7f820c101c90 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.979+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c1a7f20 con 0x7f820c101c90 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 Processor -- start 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 -- start start 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f8209ffb640 1 -- 172.21.15.33:0/2140373814 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f81fc010430 con 0x7f820c101c90 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d392480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c52d308e50 con 0x55c52d39e950 2024-09-10T21:07:14.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c52d2f8280 con 0x55c52d3920a0 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.981+0000 7f821512a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c52d2fc090 con 0x55c52d3929c0 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f820bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f82112f2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d392480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f82112f2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d392480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36406/0 (socket says 172.21.15.33:36406) 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:45092/0 (socket says 172.21.15.33:45092) 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f82112f2640 1 -- 172.21.15.33:0/2184509463 learned_addr learned my addr 172.21.15.33:0/2184509463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 -- 172.21.15.33:0/2184509463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 msgr2=0x55c52d392480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 --2- 172.21.15.33:0/2184509463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d392480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 -- 172.21.15.33:0/2184509463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 msgr2=0x55c52d3a0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 --2- 172.21.15.33:0/2184509463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 -- 172.21.15.33:0/2184509463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c52d056650 con 0x55c52d3929c0 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8210af1640 1 --2- 172.21.15.33:0/2184509463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39e410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0e3940 tx=0x7f81fc0ee570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=64b97e40893b2fb5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.982+0000 7f8209ffb640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81f80b5360 0x7f81f80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f8202ffd640 1 -- 172.21.15.33:0/2184509463 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0f9070 con 0x55c52d3929c0 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f82112f2640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81f80b5360 0x7f81f80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f8202ffd640 1 -- 172.21.15.33:0/2184509463 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81fc0d8990 con 0x55c52d3929c0 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f8202ffd640 1 -- 172.21.15.33:0/2184509463 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0ac060 con 0x55c52d3929c0 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f8209ffb640 1 -- 172.21.15.33:0/2140373814 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f81fc0e5ea0 con 0x7f820c101c90 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f821512a740 1 -- 172.21.15.33:0/2184509463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 msgr2=0x55c52d39e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f82112f2640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81f80b5360 0x7f81f80b7820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8204009fd0 tx=0x7f8204009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.983+0000 7f821512a740 1 --2- 172.21.15.33:0/2184509463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39e410 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0e3940 tx=0x7f81fc0ee570 comp rx=0 tx=0).stop 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 -- 172.21.15.33:0/2184509463 shutdown_connections 2024-09-10T21:07:14.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 --2- 172.21.15.33:0/2184509463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 --2- 172.21.15.33:0/2184509463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39e410 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 --2- 172.21.15.33:0/2184509463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d392480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 -- 172.21.15.33:0/2184509463 >> 172.21.15.33:0/2184509463 conn(0x55c52d38fed0 msgr2=0x55c52d391070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 -- 172.21.15.33:0/2184509463 shutdown_connections 2024-09-10T21:07:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 -- 172.21.15.33:0/2184509463 wait complete. 2024-09-10T21:07:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 -- start start 2024-09-10T21:07:14.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 Processor -- start 2024-09-10T21:07:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3920a0 0x55c52d39e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d39b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.984+0000 7f821512a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d39bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f821512a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c52d159700 con 0x55c52d39e950 2024-09-10T21:07:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f821512a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c52d293110 con 0x55c52d3929c0 2024-09-10T21:07:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f821512a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c52d3a1670 con 0x55c52d3920a0 2024-09-10T21:07:14.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f821512a740 10 client.0 _ll_register_callbacks cb 0x55c52d2ac990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d39bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d39bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60500/0 (socket says 172.21.15.33:60500) 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 -- 172.21.15.33:0/2350597134 learned_addr learned my addr 172.21.15.33:0/2350597134 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f820bfff640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d39b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f8210af1640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3920a0 0x55c52d39e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 -- 172.21.15.33:0/2350597134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3920a0 msgr2=0x55c52d39e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3920a0 0x55c52d39e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 -- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 msgr2=0x55c52d39b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d39b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 -- 172.21.15.33:0/2350597134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c52d2c0c40 con 0x55c52d39e950 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f820bfff640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d39b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f82112f2640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d39bcb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8204011b60 tx=0x7f8204004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ee1bdae317d7c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:14.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f81ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f81ebfff640 1 -- 172.21.15.33:0/2350597134 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204011020 con 0x55c52d39e950 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f81ebfff640 1 -- 172.21.15.33:0/2350597134 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8204017030 con 0x55c52d39e950 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.985+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81fc0d8990 con 0x55c52d39e950 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f81ebfff640 1 -- 172.21.15.33:0/2350597134 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f82040111c0 con 0x55c52d39e950 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 10 client.15975 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c52d3ad020 con 0x55c52d39e950 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f81ebfff640 1 -- 172.21.15.33:0/2350597134 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81fc0d8990 con 0x55c52d39e950 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f81ebfff640 1 -- 172.21.15.33:0/2350597134 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f82040285b0 con 0x55c52d39e950 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 10 client.15975 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 10 client.15975 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 20 client.15975 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d3ae600 0x55c52d3b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:14.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 4 client.15975 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c52d30ada0 con 0x55c52d3ae600 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.986+0000 7f8210af1640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d3ae600 0x55c52d3b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.987+0000 7f8210af1640 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d3ae600 0x55c52d3b0a60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c60faacbc4f0dcf9 server_cookie=1aad7360f82e5a0e in_seq=0 out_seq=0 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.987+0000 7f81ebfff640 10 client.15975 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.987+0000 7f81ebfff640 1 -- 172.21.15.33:0/2350597134 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c52d30ada0 con 0x55c52d3ae600 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.987+0000 7f81ebfff640 10 client.15975 handle_command_reply: tid=0 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.987+0000 7f821512a740 1 client.15975 shutdown 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 msgr2=0x55c52d39bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d39bcb0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8204011b60 tx=0x7f8204004ab0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 shutdown_connections 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d3ae600 0x55c52d3b0a60 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d39bcb0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d39b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 --2- 172.21.15.33:0/2350597134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3920a0 0x55c52d39e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 >> 172.21.15.33:0/2350597134 conn(0x55c52d38fed0 msgr2=0x55c52d395a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f81ebfff640 0 client.15975 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 shutdown_connections 2024-09-10T21:07:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 1 -- 172.21.15.33:0/2350597134 wait complete. 2024-09-10T21:07:14.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:14.988+0000 7f821512a740 20 client.15975 trim_cache size 0 max 16384 2024-09-10T21:07:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 Processor -- start 2024-09-10T21:07:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 -- start start 2024-09-10T21:07:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d3aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b06e0 con 0x55c52d39e950 2024-09-10T21:07:15.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b23a0 con 0x55c52d3920a0 2024-09-10T21:07:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f821512a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b0db0 con 0x55c52d3929c0 2024-09-10T21:07:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.017+0000 7f820bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f820bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:45116/0 (socket says 172.21.15.33:45116) 2024-09-10T21:07:15.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f820bfff640 1 -- 172.21.15.33:0/3378968586 learned_addr learned my addr 172.21.15.33:0/3378968586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f82112f2640 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f820bfff640 1 -- 172.21.15.33:0/3378968586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 msgr2=0x55c52d3aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f820bfff640 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d3aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f820bfff640 1 -- 172.21.15.33:0/3378968586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 msgr2=0x55c52d3a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f820bfff640 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.018+0000 7f820bfff640 1 -- 172.21.15.33:0/3378968586 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c52d39b430 con 0x55c52d3929c0 2024-09-10T21:07:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.019+0000 7f820bfff640 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39fd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009230 tx=0x7f81f4004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdf877b82f1926f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.019+0000 7f82027fc640 1 -- 172.21.15.33:0/3378968586 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4035040 con 0x55c52d3929c0 2024-09-10T21:07:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.019+0000 7f82027fc640 1 -- 172.21.15.33:0/3378968586 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81f40044c0 con 0x55c52d3929c0 2024-09-10T21:07:15.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.019+0000 7f82027fc640 1 -- 172.21.15.33:0/3378968586 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4003850 con 0x55c52d3929c0 2024-09-10T21:07:15.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.020+0000 7f821512a740 1 -- 172.21.15.33:0/3378968586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 msgr2=0x55c52d39fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:15.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.020+0000 7f821512a740 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39fd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009230 tx=0x7f81f4004310 comp rx=0 tx=0).stop 2024-09-10T21:07:15.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 -- 172.21.15.33:0/3378968586 shutdown_connections 2024-09-10T21:07:15.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d39e950 0x55c52d3a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d3929c0 0x55c52d39fd40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 --2- 172.21.15.33:0/3378968586 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3920a0 0x55c52d3aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 -- 172.21.15.33:0/3378968586 >> 172.21.15.33:0/3378968586 conn(0x55c52d38fed0 msgr2=0x55c52d396900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:15.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 -- 172.21.15.33:0/3378968586 shutdown_connections 2024-09-10T21:07:15.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 -- 172.21.15.33:0/3378968586 wait complete. 2024-09-10T21:07:15.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 -- start start 2024-09-10T21:07:15.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.021+0000 7f821512a740 1 Processor -- start 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f821512a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d3920a0 0x55c52d3a6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f821512a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d3a6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f821512a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d39e950 0x55c52d3a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f821512a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b1c10 con 0x55c52d3920a0 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f821512a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1b1480 con 0x55c52d3929c0 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f821512a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1ae6a0 con 0x55c52d39e950 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f821512a740 10 client.0 _ll_register_callbacks cb 0x7f8204009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.022+0000 7f820bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d3a6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f820bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d3a6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36424/0 (socket says 172.21.15.33:36424) 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f820bfff640 1 -- 172.21.15.33:0/46687778 learned_addr learned my addr 172.21.15.33:0/46687778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f8210af1640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d3920a0 0x55c52d3a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:15.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f82112f2640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d39e950 0x55c52d3a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f820bfff640 1 -- 172.21.15.33:0/46687778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d39e950 msgr2=0x55c52d3a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f820bfff640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d39e950 0x55c52d3a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f820bfff640 1 -- 172.21.15.33:0/46687778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d3920a0 msgr2=0x55c52d3a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f820bfff640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d3920a0 0x55c52d3a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f820bfff640 1 -- 172.21.15.33:0/46687778 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c52d3a74c0 con 0x55c52d3929c0 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.023+0000 7f8210af1640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d3920a0 0x55c52d3a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f820bfff640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d3a6e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009850 tx=0x7f81f4004f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fea0f18cf624d114 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f81ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f81ebfff640 1 -- 172.21.15.33:0/46687778 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4035040 con 0x55c52d3929c0 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c52d306850 con 0x55c52d3929c0 2024-09-10T21:07:15.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f81ebfff640 1 -- 172.21.15.33:0/46687778 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f81f4005390 con 0x55c52d3929c0 2024-09-10T21:07:15.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f81ebfff640 1 -- 172.21.15.33:0/46687778 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f81f4005590 con 0x55c52d3929c0 2024-09-10T21:07:15.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f821512a740 10 client.25550 fetch_fsmap learned FSMap version 34 2024-09-10T21:07:15.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c52d424160 con 0x55c52d3929c0 2024-09-10T21:07:15.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.024+0000 7f81ebfff640 1 -- 172.21.15.33:0/46687778 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c52d306850 con 0x55c52d3929c0 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f81ebfff640 1 -- 172.21.15.33:0/46687778 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 34) ==== 8394+0+0 (secure 0 0 0) 0x7f81f404e070 con 0x55c52d3929c0 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f821512a740 10 client.25550 fetch_fsmap finished waiting for FSMap version 34 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f821512a740 10 client.25550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f821512a740 20 client.25550 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f821512a740 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d424890 0x55c52d426cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f821512a740 4 client.25550 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c52d303c80 con 0x55c52d424890 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.025+0000 7f8210af1640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d424890 0x55c52d426cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f8210af1640 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d424890 0x55c52d426cb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df4088568e7170e server_cookie=c382393226ec777c in_seq=0 out_seq=0 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f81ebfff640 10 client.25550 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f81ebfff640 1 -- 172.21.15.33:0/46687778 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x55c52d303c80 con 0x55c52d424890 2024-09-10T21:07:15.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f81ebfff640 10 client.25550 handle_command_reply: tid=0 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 client.25550 shutdown 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 msgr2=0x55c52d3a6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d3a6e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009850 tx=0x7f81f4004f50 comp rx=0 tx=0).stop 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 shutdown_connections 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c52d424890 0x55c52d426cb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52d39e950 0x55c52d3a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52d3929c0 0x55c52d3a6e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 --2- 172.21.15.33:0/46687778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52d3920a0 0x55c52d3a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 >> 172.21.15.33:0/46687778 conn(0x55c52d38fed0 msgr2=0x55c52d38e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:15.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f81ebfff640 0 client.25550 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:15.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.026+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 shutdown_connections 2024-09-10T21:07:15.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.027+0000 7f821512a740 1 -- 172.21.15.33:0/46687778 wait complete. 2024-09-10T21:07:15.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.027+0000 7f821512a740 20 client.25550 trim_cache size 0 max 16384 2024-09-10T21:07:15.034 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:07:15.034 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "107d5f4a-5704-4eb9-a25b-e2fdce4273a9", 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:15.035 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:07:15.036 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:07:15.037 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:07:15.038 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:07:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81f80b5360 msgr2=0x7f81f80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81f80b5360 0x7f81f80b7820 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8204009fd0 tx=0x7f8204009500 comp rx=0 tx=0).stop 2024-09-10T21:07:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c101c90 msgr2=0x7f820c0fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c101c90 0x7f820c0fdc00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f81fc00e9f0 tx=0x7f81fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 shutdown_connections 2024-09-10T21:07:15.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81f80b5360 0x7f81f80b7820 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f820c108020 0x7f820c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f820c1076a0 0x7f820c0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 --2- 172.21.15.33:0/2140373814 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f820c101c90 0x7f820c0fdc00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.028+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 >> 172.21.15.33:0/2140373814 conn(0x7f820c0fd700 msgr2=0x7f820c06ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.029+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 shutdown_connections 2024-09-10T21:07:15.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:15.029+0000 7f8212d55640 1 -- 172.21.15.33:0/2140373814 wait complete. 2024-09-10T21:07:15.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 274025 10.78 MB/sec execute 16 sec latency 43.046 ms 2024-09-10T21:07:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:15 smithi104 ceph-mon[35325]: pgmap v444: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-10T21:07:15.893 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:15 smithi033 ceph-mon[30860]: pgmap v444: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-10T21:07:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:15 smithi174 ceph-mon[35635]: pgmap v444: 97 pgs: 97 active+clean; 122 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.4 MiB/s wr, 369 op/s 2024-09-10T21:07:15.922 INFO:tasks.cephfs.filesystem:scrub status for tag:107d5f4a-5704-4eb9-a25b-e2fdce4273a9 - {'path': '/', 'tag': '107d5f4a-5704-4eb9-a25b-e2fdce4273a9', 'options': 'recursive,force'} 2024-09-10T21:07:16.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 276827 11.16 MB/sec execute 17 sec latency 13.660 ms 2024-09-10T21:07:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:17 smithi033 ceph-mon[30860]: pgmap v445: 97 pgs: 97 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-09-10T21:07:17.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 278469 10.91 MB/sec execute 18 sec latency 12.312 ms 2024-09-10T21:07:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:17 smithi104 ceph-mon[35325]: pgmap v445: 97 pgs: 97 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-09-10T21:07:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:17 smithi174 ceph-mon[35635]: pgmap v445: 97 pgs: 97 active+clean; 121 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 302 op/s 2024-09-10T21:07:18.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 280696 11.06 MB/sec execute 19 sec latency 12.219 ms 2024-09-10T21:07:19.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 283119 11.07 MB/sec execute 20 sec latency 41.972 ms 2024-09-10T21:07:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:19 smithi104 ceph-mon[35325]: pgmap v446: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-10T21:07:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:19 smithi174 ceph-mon[35635]: pgmap v446: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-10T21:07:19.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:19 smithi033 ceph-mon[30860]: pgmap v446: 97 pgs: 97 active+clean; 124 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2024-09-10T21:07:20.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 284697 10.95 MB/sec execute 21 sec latency 23.919 ms 2024-09-10T21:07:21.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 287488 11.23 MB/sec execute 22 sec latency 12.315 ms 2024-09-10T21:07:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:21 smithi104 ceph-mon[35325]: pgmap v447: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-09-10T21:07:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:07:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:07:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:07:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:21 smithi174 ceph-mon[35635]: pgmap v447: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-09-10T21:07:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:07:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:07:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:07:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:21 smithi033 ceph-mon[30860]: pgmap v447: 97 pgs: 97 active+clean; 125 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2024-09-10T21:07:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:07:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:07:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:07:22.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 287778 10.80 MB/sec execute 23 sec latency 632.767 ms 2024-09-10T21:07:22.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:22 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:07:22.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:22 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:07:22.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:22 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:07:22.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:22 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:07:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:22 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:07:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:22 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:07:23.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 287778 10.35 MB/sec execute 24 sec latency 1632.889 ms 2024-09-10T21:07:23.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:23 smithi104 ceph-mon[35325]: pgmap v448: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-09-10T21:07:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:23 smithi174 ceph-mon[35635]: pgmap v448: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-09-10T21:07:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:23 smithi033 ceph-mon[30860]: pgmap v448: 97 pgs: 97 active+clean; 126 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 310 op/s 2024-09-10T21:07:24.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 287778 9.94 MB/sec execute 25 sec latency 2632.988 ms 2024-09-10T21:07:25.718 INFO:tasks.workunit.client.0.smithi033.stdout: 1 287778 9.55 MB/sec execute 26 sec latency 3633.103 ms 2024-09-10T21:07:25.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:25 smithi104 ceph-mon[35325]: pgmap v449: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-10T21:07:25.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:25 smithi174 ceph-mon[35635]: pgmap v449: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-10T21:07:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:25 smithi033 ceph-mon[30860]: pgmap v449: 97 pgs: 97 active+clean; 127 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-10T21:07:26.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 287778 9.20 MB/sec execute 27 sec latency 4633.212 ms 2024-09-10T21:07:27.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 289175 9.07 MB/sec execute 28 sec latency 5005.706 ms 2024-09-10T21:07:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:27 smithi104 ceph-mon[35325]: pgmap v450: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-09-10T21:07:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:27 smithi174 ceph-mon[35635]: pgmap v450: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-09-10T21:07:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:27 smithi033 ceph-mon[30860]: pgmap v450: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 292 op/s 2024-09-10T21:07:28.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 291325 9.23 MB/sec execute 29 sec latency 16.730 ms 2024-09-10T21:07:29.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 293656 9.29 MB/sec execute 30 sec latency 40.326 ms 2024-09-10T21:07:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:29 smithi104 ceph-mon[35325]: pgmap v451: 97 pgs: 97 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 279 op/s 2024-09-10T21:07:29.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:29 smithi174 ceph-mon[35635]: pgmap v451: 97 pgs: 97 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 279 op/s 2024-09-10T21:07:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:29 smithi033 ceph-mon[30860]: pgmap v451: 97 pgs: 97 active+clean; 131 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.0 MiB/s wr, 279 op/s 2024-09-10T21:07:30.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 295369 9.27 MB/sec execute 31 sec latency 14.728 ms 2024-09-10T21:07:31.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 298241 9.55 MB/sec execute 32 sec latency 11.032 ms 2024-09-10T21:07:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:31 smithi104 ceph-mon[35325]: pgmap v452: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-10T21:07:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:31 smithi174 ceph-mon[35635]: pgmap v452: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-10T21:07:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:31 smithi033 ceph-mon[30860]: pgmap v452: 97 pgs: 97 active+clean; 130 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 202 op/s 2024-09-10T21:07:32.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 300472 9.50 MB/sec execute 33 sec latency 10.433 ms 2024-09-10T21:07:33.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 302313 9.57 MB/sec execute 34 sec latency 9.004 ms 2024-09-10T21:07:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:33 smithi104 ceph-mon[35325]: pgmap v453: 97 pgs: 97 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-09-10T21:07:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:33 smithi174 ceph-mon[35635]: pgmap v453: 97 pgs: 97 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-09-10T21:07:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:33 smithi033 ceph-mon[30860]: pgmap v453: 97 pgs: 97 active+clean; 129 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-09-10T21:07:34.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 305205 9.79 MB/sec execute 35 sec latency 41.558 ms 2024-09-10T21:07:35.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 307060 9.71 MB/sec execute 36 sec latency 48.039 ms 2024-09-10T21:07:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:35 smithi104 ceph-mon[35325]: pgmap v454: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 288 op/s 2024-09-10T21:07:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:35 smithi174 ceph-mon[35635]: pgmap v454: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 288 op/s 2024-09-10T21:07:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:35 smithi033 ceph-mon[30860]: pgmap v454: 97 pgs: 97 active+clean; 131 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 288 op/s 2024-09-10T21:07:36.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 309137 9.82 MB/sec execute 37 sec latency 10.718 ms 2024-09-10T21:07:37.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 312101 10.01 MB/sec execute 38 sec latency 11.547 ms 2024-09-10T21:07:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:37 smithi104 ceph-mon[35325]: pgmap v455: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-10T21:07:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:37 smithi174 ceph-mon[35635]: pgmap v455: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-10T21:07:37.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:37 smithi033 ceph-mon[30860]: pgmap v455: 97 pgs: 97 active+clean; 132 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 2.4 MiB/s wr, 231 op/s 2024-09-10T21:07:38.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 313406 9.86 MB/sec execute 39 sec latency 41.347 ms 2024-09-10T21:07:39.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 316071 10.03 MB/sec execute 40 sec latency 14.281 ms 2024-09-10T21:07:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:39 smithi104 ceph-mon[35325]: pgmap v456: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-09-10T21:07:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:39 smithi174 ceph-mon[35635]: pgmap v456: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-09-10T21:07:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:39 smithi033 ceph-mon[30860]: pgmap v456: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.6 MiB/s wr, 266 op/s 2024-09-10T21:07:40.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 318263 9.98 MB/sec execute 41 sec latency 38.550 ms 2024-09-10T21:07:41.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 320114 10.03 MB/sec execute 42 sec latency 9.549 ms 2024-09-10T21:07:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:41 smithi104 ceph-mon[35325]: pgmap v457: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-09-10T21:07:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:41 smithi174 ceph-mon[35635]: pgmap v457: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-09-10T21:07:41.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:41 smithi033 ceph-mon[30860]: pgmap v457: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.0 MiB/s wr, 322 op/s 2024-09-10T21:07:42.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 322950 10.19 MB/sec execute 43 sec latency 11.026 ms 2024-09-10T21:07:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:42 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:42.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:42 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:42.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:42 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:43.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 324936 10.12 MB/sec execute 44 sec latency 9.412 ms 2024-09-10T21:07:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:43 smithi104 ceph-mon[35325]: pgmap v458: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-09-10T21:07:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:43 smithi174 ceph-mon[35635]: pgmap v458: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-09-10T21:07:43.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:43 smithi033 ceph-mon[30860]: pgmap v458: 97 pgs: 97 active+clean; 136 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.0 MiB/s wr, 334 op/s 2024-09-10T21:07:44.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 326844 10.19 MB/sec execute 45 sec latency 11.215 ms 2024-09-10T21:07:45.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 329335 10.25 MB/sec execute 46 sec latency 39.107 ms 2024-09-10T21:07:45.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:45 smithi104 ceph-mon[35325]: pgmap v459: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 393 op/s 2024-09-10T21:07:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:45 smithi174 ceph-mon[35635]: pgmap v459: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 393 op/s 2024-09-10T21:07:45.923 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:07:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:45 smithi033 ceph-mon[30860]: pgmap v459: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 393 op/s 2024-09-10T21:07:46.217 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:46.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 330935 10.21 MB/sec execute 47 sec latency 25.267 ms 2024-09-10T21:07:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.709+0000 7f81706bf640 1 -- 172.21.15.33:0/600055160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81681087d0 msgr2=0x7f8168108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.709+0000 7f81706bf640 1 --2- 172.21.15.33:0/600055160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81681087d0 0x7f8168108c10 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8158009900 tx=0x7f815802a410 comp rx=0 tx=0).stop 2024-09-10T21:07:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.709+0000 7f81706bf640 1 -- 172.21.15.33:0/600055160 shutdown_connections 2024-09-10T21:07:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.709+0000 7f81706bf640 1 --2- 172.21.15.33:0/600055160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168109150 0x7f816810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.709+0000 7f81706bf640 1 --2- 172.21.15.33:0/600055160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81681087d0 0x7f8168108c10 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.709+0000 7f81706bf640 1 --2- 172.21.15.33:0/600055160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8168102dc0 0x7f81681031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.709+0000 7f81706bf640 1 -- 172.21.15.33:0/600055160 >> 172.21.15.33:0/600055160 conn(0x7f81680fe830 msgr2=0x7f8168100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 -- 172.21.15.33:0/600055160 shutdown_connections 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 -- 172.21.15.33:0/600055160 wait complete. 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 Processor -- start 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 -- start start 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168102dc0 0x7f8168199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81681087d0 0x7f81681997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8168109150 0x7f816819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8168112da0 con 0x7f8168109150 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8168112c20 con 0x7f81681087d0 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.710+0000 7f81706bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8168112f20 con 0x7f8168102dc0 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816ec5c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8168109150 0x7f816819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168102dc0 0x7f8168199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816dc5a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81681087d0 0x7f81681997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168102dc0 0x7f8168199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34748/0 (socket says 172.21.15.33:34748) 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 -- 172.21.15.33:0/2629123443 learned_addr learned my addr 172.21.15.33:0/2629123443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 -- 172.21.15.33:0/2629123443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81681087d0 msgr2=0x7f81681997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81681087d0 0x7f81681997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 -- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8168109150 msgr2=0x7f816819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8168109150 0x7f816819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 -- 172.21.15.33:0/2629123443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816819f320 con 0x7f8168102dc0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f816e45b640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168102dc0 0x7f8168199270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f816400ea60 tx=0x7f816400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f815f7fe640 1 -- 172.21.15.33:0/2629123443 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400cde0 con 0x7f8168102dc0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816819f610 con 0x7f8168102dc0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f815f7fe640 1 -- 172.21.15.33:0/2629123443 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81640035f0 con 0x7f8168102dc0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f815f7fe640 1 -- 172.21.15.33:0/2629123443 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164010690 con 0x7f8168102dc0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.711+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81681a5ea0 con 0x7f8168102dc0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.714+0000 7f815f7fe640 1 -- 172.21.15.33:0/2629123443 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8164004000 con 0x7f8168102dc0 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.714+0000 7f8172a94740 1 Processor -- start 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.714+0000 7f8172a94740 1 -- start start 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f8172a94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9ca9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f8172a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f8172a94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cbc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f8172a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fac9c1fe50 con 0x55fac9ca9da0 2024-09-10T21:07:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f8172a94740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fac9c0f280 con 0x55fac9cba140 2024-09-10T21:07:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f8172a94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fac9c13090 con 0x55fac9ca9480 2024-09-10T21:07:46.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816ec5c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9ca9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816ec5c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9ca9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34754/0 (socket says 172.21.15.33:34754) 2024-09-10T21:07:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816ec5c640 1 -- 172.21.15.33:0/2945802753 learned_addr learned my addr 172.21.15.33:0/2945802753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816e45b640 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816dc5a640 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cbc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816ec5c640 1 -- 172.21.15.33:0/2945802753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 msgr2=0x55fac9cbc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816ec5c640 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cbc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816ec5c640 1 -- 172.21.15.33:0/2945802753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 msgr2=0x55fac9cb9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.715+0000 7f816ec5c640 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f816ec5c640 1 -- 172.21.15.33:0/2945802753 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac996d650 con 0x55fac9ca9480 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f815f7fe640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81540b52e0 0x7f81540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f816dc5a640 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cbc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f816e45b640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81540b52e0 0x7f81540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f816ec5c640 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9ca9860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8160009fd0 tx=0x7f81600118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e1bea1ffce06dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f815f7fe640 1 -- 172.21.15.33:0/2629123443 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f81640e5ed0 con 0x7f8168102dc0 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f815cff9640 1 -- 172.21.15.33:0/2945802753 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160004d20 con 0x55fac9ca9480 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f815cff9640 1 -- 172.21.15.33:0/2945802753 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160004050 con 0x55fac9ca9480 2024-09-10T21:07:46.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.716+0000 7f815cff9640 1 -- 172.21.15.33:0/2945802753 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160005280 con 0x55fac9ca9480 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 -- 172.21.15.33:0/2945802753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 msgr2=0x55fac9ca9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9ca9860 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8160009fd0 tx=0x7f81600118d0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f816e45b640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81540b52e0 0x7f81540b77a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f816400d010 tx=0x7f81640e9910 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 -- 172.21.15.33:0/2945802753 shutdown_connections 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cbc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 --2- 172.21.15.33:0/2945802753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9ca9860 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 -- 172.21.15.33:0/2945802753 >> 172.21.15.33:0/2945802753 conn(0x55fac9ca72b0 msgr2=0x55fac9ca8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 -- 172.21.15.33:0/2945802753 shutdown_connections 2024-09-10T21:07:46.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 -- 172.21.15.33:0/2945802753 wait complete. 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 -- start start 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.717+0000 7f8172a94740 1 Processor -- start 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8172a94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cb99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8172a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8172a94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cb73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8172a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fac9a70700 con 0x55fac9ca9da0 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8172a94740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fac9baa110 con 0x55fac9cba140 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8172a94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fac9cbcdf0 con 0x55fac9ca9480 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8172a94740 10 client.0 _ll_register_callbacks cb 0x55fac9bc3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34934/0 (socket says 172.21.15.33:34934) 2024-09-10T21:07:46.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816ec5c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cb99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 -- 172.21.15.33:0/4229471303 learned_addr learned my addr 172.21.15.33:0/4229471303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816ec5c640 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cb73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 -- 172.21.15.33:0/4229471303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 msgr2=0x55fac9cb99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cb99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 -- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 msgr2=0x55fac9cb73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cb73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 -- 172.21.15.33:0/4229471303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac9bd7c40 con 0x55fac9ca9da0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f816dc5a640 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb6e30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f815802a930 tx=0x7f8158032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688faf434039a5dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.718+0000 7f8151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8151ffb640 1 -- 172.21.15.33:0/4229471303 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158035040 con 0x55fac9ca9da0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8151ffb640 1 -- 172.21.15.33:0/4229471303 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8158031720 con 0x55fac9ca9da0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8160004050 con 0x55fac9ca9da0 2024-09-10T21:07:46.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8151ffb640 1 -- 172.21.15.33:0/4229471303 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8158031920 con 0x55fac9ca9da0 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 10 client.15993 fetch_fsmap learned FSMap version 35 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fac9cc86e0 con 0x55fac9ca9da0 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8151ffb640 1 -- 172.21.15.33:0/4229471303 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8160004050 con 0x55fac9ca9da0 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8151ffb640 1 -- 172.21.15.33:0/4229471303 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7f8158004750 con 0x55fac9ca9da0 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 10 client.15993 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 10 client.15993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 20 client.15993 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9cc9c40 0x55fac9ccc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 4 client.15993 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.719+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fac9c21da0 con 0x55fac9cc9c40 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.720+0000 7f816ec5c640 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9cc9c40 0x55fac9ccc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.720+0000 7f8151ffb640 10 client.15993 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:46.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.720+0000 7f816ec5c640 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9cc9c40 0x55fac9ccc060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d97b36a6eaa1ec3 server_cookie=d1deaab31bba5881 in_seq=0 out_seq=0 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.720+0000 7f8151ffb640 1 -- 172.21.15.33:0/4229471303 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fac9c21da0 con 0x55fac9cc9c40 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.720+0000 7f8151ffb640 10 client.15993 handle_command_reply: tid=0 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 client.15993 shutdown 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 msgr2=0x55fac9cb6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb6e30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f815802a930 tx=0x7f8158032f20 comp rx=0 tx=0).stop 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 shutdown_connections 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9cc9c40 0x55fac9ccc060 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cb73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8151ffb640 0 client.15993 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9ca9da0 0x55fac9cb6e30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 --2- 172.21.15.33:0/4229471303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cb99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 >> 172.21.15.33:0/4229471303 conn(0x55fac9ca72b0 msgr2=0x55fac9cacde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 shutdown_connections 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 1 -- 172.21.15.33:0/4229471303 wait complete. 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.721+0000 7f8172a94740 20 client.15993 trim_cache size 0 max 16384 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 Processor -- start 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 -- start start 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9ca9480 0x55fac9cc6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cba140 0x55fac9ca8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9cc9c40 0x55fac9cd1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81681ac250 con 0x55fac9cba140 2024-09-10T21:07:46.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81681adee0 con 0x55fac9ca9480 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.741+0000 7f8172a94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81681ac920 con 0x55fac9cc9c40 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cba140 0x55fac9ca8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816ec5c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9ca9480 0x55fac9cc6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cba140 0x55fac9ca8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34944/0 (socket says 172.21.15.33:34944) 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 -- 172.21.15.33:0/3426397600 learned_addr learned my addr 172.21.15.33:0/3426397600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816ec5c640 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9cc9c40 0x55fac9cd1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 -- 172.21.15.33:0/3426397600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9cc9c40 msgr2=0x55fac9cd1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9cc9c40 0x55fac9cd1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 -- 172.21.15.33:0/3426397600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9ca9480 msgr2=0x55fac9cc6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9ca9480 0x55fac9cc6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 -- 172.21.15.33:0/3426397600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fac9ca9bd0 con 0x55fac9cba140 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816ec5c640 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9ca9480 0x55fac9cc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:46.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f816dc5a640 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cba140 0x55fac9ca8a80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8158004b70 tx=0x7f8158004870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=180825171d55d7f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f8153fff640 1 -- 172.21.15.33:0/3426397600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158035070 con 0x55fac9cba140 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f8153fff640 1 -- 172.21.15.33:0/3426397600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81580316a0 con 0x55fac9cba140 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.742+0000 7f8153fff640 1 -- 172.21.15.33:0/3426397600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8158005600 con 0x55fac9cba140 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 -- 172.21.15.33:0/3426397600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cba140 msgr2=0x55fac9ca8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cba140 0x55fac9ca8a80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8158004b70 tx=0x7f8158004870 comp rx=0 tx=0).stop 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 -- 172.21.15.33:0/3426397600 shutdown_connections 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9cc9c40 0x55fac9cd1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cba140 0x55fac9ca8a80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 --2- 172.21.15.33:0/3426397600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9ca9480 0x55fac9cc6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 -- 172.21.15.33:0/3426397600 >> 172.21.15.33:0/3426397600 conn(0x55fac9ca72b0 msgr2=0x55fac9ca8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 -- 172.21.15.33:0/3426397600 shutdown_connections 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.743+0000 7f8172a94740 1 -- 172.21.15.33:0/3426397600 wait complete. 2024-09-10T21:07:46.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 -- start start 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 Processor -- start 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cc23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cc2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cc9c40 0x55fac9cbf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81681ad780 con 0x55fac9cc9c40 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81681acff0 con 0x55fac9cba140 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81681aa210 con 0x55fac9ca9480 2024-09-10T21:07:46.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f8172a94740 10 client.0 _ll_register_callbacks cb 0x7f8158009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f816ec5c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cc9c40 0x55fac9cbf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f816dc5a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cc2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f816ec5c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cc9c40 0x55fac9cbf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34954/0 (socket says 172.21.15.33:34954) 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f816ec5c640 1 -- 172.21.15.33:0/487589020 learned_addr learned my addr 172.21.15.33:0/487589020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.744+0000 7f816ec5c640 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cc23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f816ec5c640 1 -- 172.21.15.33:0/487589020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 msgr2=0x55fac9cc23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f816ec5c640 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cc23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f816ec5c640 1 -- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 msgr2=0x55fac9cc2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f816ec5c640 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cc2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f816ec5c640 1 -- 172.21.15.33:0/487589020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fac9cbf950 con 0x55fac9cc9c40 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f816dc5a640 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cc2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f816ec5c640 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cc9c40 0x55fac9cbf120 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8160009fd0 tx=0x7f8160030040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab560cf80f8564a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8151ffb640 1 -- 172.21.15.33:0/487589020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160041060 con 0x55fac9cc9c40 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8151ffb640 1 -- 172.21.15.33:0/487589020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8160004260 con 0x55fac9cc9c40 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81580316a0 con 0x55fac9cc9c40 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8151ffb640 1 -- 172.21.15.33:0/487589020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8160004460 con 0x55fac9cc9c40 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8172a94740 10 client.15999 fetch_fsmap learned FSMap version 35 2024-09-10T21:07:46.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55fac9d37780 con 0x55fac9cc9c40 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.745+0000 7f8151ffb640 1 -- 172.21.15.33:0/487589020 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81580316a0 con 0x55fac9cc9c40 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8151ffb640 1 -- 172.21.15.33:0/487589020 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7f8160046070 con 0x55fac9cc9c40 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8172a94740 10 client.15999 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8172a94740 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8172a94740 20 client.15999 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8172a94740 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9d37df0 0x55fac9d3a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8172a94740 4 client.15999 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fac9c1d850 con 0x55fac9d37df0 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f816dc5a640 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9d37df0 0x55fac9d3a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f816dc5a640 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9d37df0 0x55fac9d3a210 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b89a2d5442e518 server_cookie=688e7dbf238ac9b4 in_seq=0 out_seq=0 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8151ffb640 10 client.15999 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:46.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8151ffb640 1 -- 172.21.15.33:0/487589020 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x55fac9c1d850 con 0x55fac9d37df0 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8151ffb640 10 client.15999 handle_command_reply: tid=0 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.746+0000 7f8172a94740 1 client.15999 shutdown 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cc9c40 msgr2=0x55fac9cbf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cc9c40 0x55fac9cbf120 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8160009fd0 tx=0x7f8160030040 comp rx=0 tx=0).stop 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 shutdown_connections 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fac9d37df0 0x55fac9d3a210 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fac9cc9c40 0x55fac9cbf120 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fac9cba140 0x55fac9cc2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8151ffb640 0 client.15999 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 --2- 172.21.15.33:0/487589020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fac9ca9480 0x55fac9cc23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 >> 172.21.15.33:0/487589020 conn(0x55fac9ca72b0 msgr2=0x55fac9cb56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 shutdown_connections 2024-09-10T21:07:46.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 1 -- 172.21.15.33:0/487589020 wait complete. 2024-09-10T21:07:46.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f8172a94740 20 client.15999 trim_cache size 0 max 16384 2024-09-10T21:07:46.773 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:07:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:07:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:07:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:07:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:07:46.773 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:46.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:46.775 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:07:46.776 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:07:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:07:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:07:46.777 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:07:46.777 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:07:46.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81540b52e0 msgr2=0x7f81540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f81706bf640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81540b52e0 0x7f81540b77a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f816400d010 tx=0x7f81640e9910 comp rx=0 tx=0).stop 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168102dc0 msgr2=0x7f8168199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.747+0000 7f81706bf640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168102dc0 0x7f8168199270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f816400ea60 tx=0x7f816400ef20 comp rx=0 tx=0).stop 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 shutdown_connections 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81540b52e0 0x7f81540b77a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8168109150 0x7f816819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81681087d0 0x7f81681997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 --2- 172.21.15.33:0/2629123443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8168102dc0 0x7f8168199270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 >> 172.21.15.33:0/2629123443 conn(0x7f81680fe830 msgr2=0x7f8168100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 shutdown_connections 2024-09-10T21:07:46.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:46.748+0000 7f81706bf640 1 -- 172.21.15.33:0/2629123443 wait complete. 2024-09-10T21:07:47.575 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:07:47.575 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:07:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:47 smithi033 ceph-mon[30860]: pgmap v460: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 314 op/s 2024-09-10T21:07:47.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 333720 10.35 MB/sec execute 48 sec latency 9.243 ms 2024-09-10T21:07:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:47 smithi104 ceph-mon[35325]: pgmap v460: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 314 op/s 2024-09-10T21:07:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:47 smithi174 ceph-mon[35635]: pgmap v460: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 314 op/s 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 -- 172.21.15.33:0/1688652773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 msgr2=0x7f23a4111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 --2- 172.21.15.33:0/1688652773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a4111b50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f239c009ca0 tx=0x7f239c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 -- 172.21.15.33:0/1688652773 shutdown_connections 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 --2- 172.21.15.33:0/1688652773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a4111b50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 --2- 172.21.15.33:0/1688652773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23a410a9d0 0x7f23a410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 --2- 172.21.15.33:0/1688652773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a4104fc0 0x7f23a41053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 -- 172.21.15.33:0/1688652773 >> 172.21.15.33:0/1688652773 conn(0x7f23a41009f0 msgr2=0x7f23a4102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 -- 172.21.15.33:0/1688652773 shutdown_connections 2024-09-10T21:07:48.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.576+0000 7f23ac092640 1 -- 172.21.15.33:0/1688652773 wait complete. 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 Processor -- start 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 -- start start 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a4104fc0 0x7f23a419b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23a410a9d0 0x7f23a419b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a41a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4114f80 con 0x7f23a410b350 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4114e00 con 0x7f23a4104fc0 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23ac092640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23a4115100 con 0x7f23a410a9d0 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23aa62f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a41a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23aa62f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a41a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34974/0 (socket says 172.21.15.33:34974) 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23aa62f640 1 -- 172.21.15.33:0/2200534585 learned_addr learned my addr 172.21.15.33:0/2200534585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.577+0000 7f23a9e2e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a4104fc0 0x7f23a419b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:48.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23a962d640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23a410a9d0 0x7f23a419b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23aa62f640 1 -- 172.21.15.33:0/2200534585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23a410a9d0 msgr2=0x7f23a419b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23aa62f640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23a410a9d0 0x7f23a419b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23aa62f640 1 -- 172.21.15.33:0/2200534585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a4104fc0 msgr2=0x7f23a419b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23aa62f640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a4104fc0 0x7f23a419b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23aa62f640 1 -- 172.21.15.33:0/2200534585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23a41a14a0 con 0x7f23a410b350 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23a9e2e640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a4104fc0 0x7f23a419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23aa62f640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a41a0cd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f239c005cc0 tx=0x7f239c031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f239affd640 1 -- 172.21.15.33:0/2200534585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c003b70 con 0x7f23a410b350 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f239affd640 1 -- 172.21.15.33:0/2200534585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f239c0318f0 con 0x7f23a410b350 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23a41a1730 con 0x7f23a410b350 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.578+0000 7f239affd640 1 -- 172.21.15.33:0/2200534585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f239c02f950 con 0x7f23a410b350 2024-09-10T21:07:48.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.579+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23a41a7e50 con 0x7f23a410b350 2024-09-10T21:07:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.580+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23a41053a0 con 0x7f23a410b350 2024-09-10T21:07:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.581+0000 7f239affd640 1 -- 172.21.15.33:0/2200534585 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f239c02fb90 con 0x7f23a410b350 2024-09-10T21:07:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.582+0000 7f239affd640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23900b5360 0x7f23900b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.582+0000 7f23a9e2e640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23900b5360 0x7f23900b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.582+0000 7f239affd640 1 -- 172.21.15.33:0/2200534585 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f239c106ca0 con 0x7f23a410b350 2024-09-10T21:07:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.582+0000 7f23a9e2e640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23900b5360 0x7f23900b7820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f23a00095f0 tx=0x7f23a0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:48.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.586+0000 7f239affd640 1 -- 172.21.15.33:0/2200534585 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f239c0cdb60 con 0x7f23a410b350 2024-09-10T21:07:48.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 335863 10.29 MB/sec execute 49 sec latency 40.552 ms 2024-09-10T21:07:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.825+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f23a402e5c0 con 0x7f23a410b350 2024-09-10T21:07:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.826+0000 7f239affd640 1 -- 172.21.15.33:0/2200534585 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11985 (secure 0 0 0) 0x7f239c0cd460 con 0x7f23a410b350 2024-09-10T21:07:48.830 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:07:48.830 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"btime":"2024-09-10T21:07:41:501130+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:07:40.983936+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:07:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-09-10T21:07:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23900b5360 msgr2=0x7f23900b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23900b5360 0x7f23900b7820 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f23a00095f0 tx=0x7f23a0005f90 comp rx=0 tx=0).stop 2024-09-10T21:07:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 msgr2=0x7f23a41a0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a41a0cd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f239c005cc0 tx=0x7f239c031700 comp rx=0 tx=0).stop 2024-09-10T21:07:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 shutdown_connections 2024-09-10T21:07:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23900b5360 0x7f23900b7820 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23a410b350 0x7f23a41a0cd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23a410a9d0 0x7f23a419b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 --2- 172.21.15.33:0/2200534585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23a4104fc0 0x7f23a419b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 >> 172.21.15.33:0/2200534585 conn(0x7f23a41009f0 msgr2=0x7f23a4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 shutdown_connections 2024-09-10T21:07:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:48.828+0000 7f23ac092640 1 -- 172.21.15.33:0/2200534585 wait complete. 2024-09-10T21:07:49.685 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:07:49.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 337579 10.32 MB/sec execute 50 sec latency 11.563 ms 2024-09-10T21:07:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:49 smithi104 ceph-mon[35325]: pgmap v461: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 381 op/s 2024-09-10T21:07:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:49 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2200534585' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:07:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:49 smithi174 ceph-mon[35635]: pgmap v461: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 381 op/s 2024-09-10T21:07:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:49 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2200534585' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:07:49.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:49 smithi033 ceph-mon[30860]: pgmap v461: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.2 MiB/s wr, 381 op/s 2024-09-10T21:07:49.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:49 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2200534585' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:07:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 -- 172.21.15.33:0/2358290212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 msgr2=0x7f3a2810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 --2- 172.21.15.33:0/2358290212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2810e820 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009ca0 tx=0x7f3a2002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 -- 172.21.15.33:0/2358290212 shutdown_connections 2024-09-10T21:07:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 --2- 172.21.15.33:0/2358290212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2810e820 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 --2- 172.21.15.33:0/2358290212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a281076a0 0x7f3a28107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 --2- 172.21.15.33:0/2358290212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a28101c90 0x7f3a28102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 -- 172.21.15.33:0/2358290212 >> 172.21.15.33:0/2358290212 conn(0x7f3a280fd700 msgr2=0x7f3a280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 -- 172.21.15.33:0/2358290212 shutdown_connections 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.434+0000 7f3a30612640 1 -- 172.21.15.33:0/2358290212 wait complete. 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 Processor -- start 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 -- start start 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28101c90 0x7f3a28199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a281076a0 0x7f3a28199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28111ca0 con 0x7f3a28108020 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28111b20 con 0x7f3a281076a0 2024-09-10T21:07:50.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a30612640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a28111e20 con 0x7f3a28101c90 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2ebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2e3ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28101c90 0x7f3a28199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2dbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a281076a0 0x7f3a28199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2ebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40582/0 (socket says 172.21.15.33:40582) 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2e3ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28101c90 0x7f3a28199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35780/0 (socket says 172.21.15.33:35780) 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2dbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a281076a0 0x7f3a28199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54994/0 (socket says 172.21.15.33:54994) 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2ebaf640 1 -- 172.21.15.33:0/383145357 learned_addr learned my addr 172.21.15.33:0/383145357 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2dbad640 1 -- 172.21.15.33:0/383145357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28101c90 msgr2=0x7f3a28199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28101c90 0x7f3a28199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2dbad640 1 -- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 msgr2=0x7f3a2819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.435+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a2dbad640 1 -- 172.21.15.33:0/383145357 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2819f300 con 0x7f3a281076a0 2024-09-10T21:07:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a2ebaf640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a281076a0 0x7f3a28199790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e9f0 tx=0x7f3a1800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a1f7fe640 1 -- 172.21.15.33:0/383145357 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800ccc0 con 0x7f3a281076a0 2024-09-10T21:07:50.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2819f5f0 con 0x7f3a281076a0 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a1f7fe640 1 -- 172.21.15.33:0/383145357 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a18005020 con 0x7f3a281076a0 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a1f7fe640 1 -- 172.21.15.33:0/383145357 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18004a00 con 0x7f3a281076a0 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.436+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a281a5cb0 con 0x7f3a281076a0 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.439+0000 7f3a329e7740 1 Processor -- start 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.439+0000 7f3a329e7740 1 -- start start 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.439+0000 7f3a1f7fe640 1 -- 172.21.15.33:0/383145357 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3a18004ba0 con 0x7f3a281076a0 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cca10 0x560f192dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192dcdb0 0x560f192df1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a329e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f19242ea0 con 0x560f192cc0f0 2024-09-10T21:07:50.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a329e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560f192322d0 con 0x560f192dcdb0 2024-09-10T21:07:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a329e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560f192360e0 con 0x560f192cca10 2024-09-10T21:07:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2ebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2ebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40588/0 (socket says 172.21.15.33:40588) 2024-09-10T21:07:50.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2ebaf640 1 -- 172.21.15.33:0/1461454520 learned_addr learned my addr 172.21.15.33:0/1461454520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cca10 0x560f192dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192dcdb0 0x560f192df1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/1461454520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192dcdb0 msgr2=0x560f192df1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192dcdb0 0x560f192df1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/1461454520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 msgr2=0x560f192cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/1461454520 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f18f90650 con 0x560f192cca10 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2ebaf640 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a1f7fe640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3a140b52e0 0x7f3a140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cca10 0x560f192dc870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009c20 tx=0x7f3a24011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1f81d7548cf1cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.440+0000 7f3a2ebaf640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3a140b52e0 0x7f3a140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.441+0000 7f3a1f7fe640 1 -- 172.21.15.33:0/383145357 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3a180e5b20 con 0x7f3a281076a0 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.441+0000 7f3a1cff9640 1 -- 172.21.15.33:0/1461454520 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24004050 con 0x560f192cca10 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.441+0000 7f3a1cff9640 1 -- 172.21.15.33:0/1461454520 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a240054d0 con 0x560f192cca10 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.441+0000 7f3a1cff9640 1 -- 172.21.15.33:0/1461454520 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24004d00 con 0x560f192cca10 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.441+0000 7f3a2ebaf640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3a140b52e0 0x7f3a140b77a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a20033040 tx=0x7f3a20005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.441+0000 7f3a329e7740 1 -- 172.21.15.33:0/1461454520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cca10 msgr2=0x560f192dc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cca10 0x560f192dc870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009c20 tx=0x7f3a24011810 comp rx=0 tx=0).stop 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 -- 172.21.15.33:0/1461454520 shutdown_connections 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192dcdb0 0x560f192df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cca10 0x560f192dc870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 --2- 172.21.15.33:0/1461454520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 -- 172.21.15.33:0/1461454520 >> 172.21.15.33:0/1461454520 conn(0x560f192c9f20 msgr2=0x560f192cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 -- 172.21.15.33:0/1461454520 shutdown_connections 2024-09-10T21:07:50.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 -- 172.21.15.33:0/1461454520 wait complete. 2024-09-10T21:07:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 -- start start 2024-09-10T21:07:50.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.442+0000 7f3a329e7740 1 Processor -- start 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192dc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a329e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560f19093700 con 0x560f192dcdb0 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a329e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560f191cd160 con 0x560f192cca10 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a329e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560f192dfa60 con 0x560f192cc0f0 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a329e7740 10 client.0 _ll_register_callbacks cb 0x560f191e69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192dc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2ebaf640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2dbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192dc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35804/0 (socket says 172.21.15.33:35804) 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/362388674 learned_addr learned my addr 172.21.15.33:0/362388674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/362388674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 msgr2=0x560f192d9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/362388674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 msgr2=0x560f192da110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192da110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/362388674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f191fac90 con 0x560f192cc0f0 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.443+0000 7f3a2ebaf640 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192da110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192dc6b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a24000c00 tx=0x7f3a24005680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b4ab9032046a329 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a11ffb640 1 -- 172.21.15.33:0/362388674 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24029070 con 0x560f192cc0f0 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a240054d0 con 0x560f192cc0f0 2024-09-10T21:07:50.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a11ffb640 1 -- 172.21.15.33:0/362388674 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3a24003a00 con 0x560f192cc0f0 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a11ffb640 1 -- 172.21.15.33:0/362388674 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3a24035050 con 0x560f192cc0f0 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a329e7740 10 client.25567 fetch_fsmap learned FSMap version 35 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560f192eb3b0 con 0x560f192cc0f0 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a11ffb640 1 -- 172.21.15.33:0/362388674 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a240054d0 con 0x560f192cc0f0 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.444+0000 7f3a11ffb640 1 -- 172.21.15.33:0/362388674 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7f3a24004fa0 con 0x560f192cc0f0 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a329e7740 10 client.25567 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a329e7740 10 client.25567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a329e7740 20 client.25567 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a329e7740 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f192ec760 0x560f192eeb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a329e7740 4 client.25567 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f19244df0 con 0x560f192ec760 2024-09-10T21:07:50.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f192ec760 0x560f192eeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f192ec760 0x560f192eeb50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5210337a8307ebaf server_cookie=396d14d96b341193 in_seq=0 out_seq=0 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.445+0000 7f3a11ffb640 10 client.25567 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a11ffb640 1 -- 172.21.15.33:0/362388674 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560f19244df0 con 0x560f192ec760 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a11ffb640 10 client.25567 handle_command_reply: tid=0 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 client.25567 shutdown 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 msgr2=0x560f192dc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192dc6b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a24000c00 tx=0x7f3a24005680 comp rx=0 tx=0).stop 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 shutdown_connections 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f192ec760 0x560f192eeb50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192da110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a11ffb640 0 client.25567 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:50.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 --2- 172.21.15.33:0/362388674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192dc6b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 >> 172.21.15.33:0/362388674 conn(0x560f192c9f20 msgr2=0x560f192cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 shutdown_connections 2024-09-10T21:07:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 1 -- 172.21.15.33:0/362388674 wait complete. 2024-09-10T21:07:50.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.446+0000 7f3a329e7740 20 client.25567 trim_cache size 0 max 16384 2024-09-10T21:07:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 Processor -- start 2024-09-10T21:07:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 -- start start 2024-09-10T21:07:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192e8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281ac240 con 0x560f192dcdb0 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281aded0 con 0x560f192cca10 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a329e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281ac910 con 0x560f192cc0f0 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.460+0000 7f3a2e3ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192e8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2dbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192e8c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35816/0 (socket says 172.21.15.33:35816) 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2dbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55018/0 (socket says 172.21.15.33:55018) 2024-09-10T21:07:50.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/2194718709 learned_addr learned my addr 172.21.15.33:0/2194718709 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2ebaf640 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/2194718709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 msgr2=0x560f192d8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/2194718709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 msgr2=0x560f192cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 -- 172.21.15.33:0/2194718709 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f192d2250 con 0x560f192cc0f0 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192e8c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005370 tx=0x7f3a240056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fffc1bbe012ffe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a13fff640 1 -- 172.21.15.33:0/2194718709 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24029070 con 0x560f192cc0f0 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a13fff640 1 -- 172.21.15.33:0/2194718709 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a24023dd0 con 0x560f192cc0f0 2024-09-10T21:07:50.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.461+0000 7f3a13fff640 1 -- 172.21.15.33:0/2194718709 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a24003800 con 0x560f192cc0f0 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 -- 172.21.15.33:0/2194718709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 msgr2=0x560f192e8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192e8c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a24005370 tx=0x7f3a240056f0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 -- 172.21.15.33:0/2194718709 shutdown_connections 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192dcdb0 0x560f192cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192d8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2194718709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192cc0f0 0x560f192e8c90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 -- 172.21.15.33:0/2194718709 >> 172.21.15.33:0/2194718709 conn(0x560f192c9f20 msgr2=0x560f192ddbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 -- 172.21.15.33:0/2194718709 shutdown_connections 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 -- 172.21.15.33:0/2194718709 wait complete. 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 -- start start 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.462+0000 7f3a329e7740 1 Processor -- start 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192e5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a329e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192dcdb0 0x560f192f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a329e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281ad770 con 0x560f192cc0f0 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a329e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281acfe0 con 0x560f192cca10 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a329e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a281aa200 con 0x560f192dcdb0 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a329e7740 10 client.0 _ll_register_callbacks cb 0x7f3a18005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2e3ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192e5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55024/0 (socket says 172.21.15.33:55024) 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 -- 172.21.15.33:0/2092929842 learned_addr learned my addr 172.21.15.33:0/2092929842 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2ebaf640 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192dcdb0 0x560f192f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 -- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192dcdb0 msgr2=0x560f192f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192dcdb0 0x560f192f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 -- 172.21.15.33:0/2092929842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 msgr2=0x560f192e5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192e5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 -- 172.21.15.33:0/2092929842 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f192f4c20 con 0x560f192cca10 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a2dbad640 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192f3eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3a180a88b0 tx=0x7f3a180e34f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d9da0b938d0c8bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.463+0000 7f3a11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a11ffb640 1 -- 172.21.15.33:0/2092929842 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1801c070 con 0x560f192cca10 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a11ffb640 1 -- 172.21.15.33:0/2092929842 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3a180a8550 con 0x560f192cca10 2024-09-10T21:07:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a24023dd0 con 0x560f192cca10 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a11ffb640 1 -- 172.21.15.33:0/2092929842 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f3a180ac9e0 con 0x560f192cca10 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a329e7740 10 client.25592 fetch_fsmap learned FSMap version 35 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x560f19359540 con 0x560f192cca10 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a11ffb640 1 -- 172.21.15.33:0/2092929842 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a24023dd0 con 0x560f192cca10 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a11ffb640 1 -- 172.21.15.33:0/2092929842 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 35) ==== 8390+0+0 (secure 0 0 0) 0x7f3a180b53e0 con 0x560f192cca10 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a329e7740 10 client.25592 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a329e7740 10 client.25592 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a329e7740 20 client.25592 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.464+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f19359c50 0x560f1935c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a329e7740 4 client.25592 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560f192408a0 con 0x560f19359c50 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f19359c50 0x560f1935c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a2e3ae640 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f19359c50 0x560f1935c070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74962fa28dcd364 server_cookie=bb0f3254d928d8a3 in_seq=0 out_seq=0 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a11ffb640 10 client.25592 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a11ffb640 1 -- 172.21.15.33:0/2092929842 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560f192408a0 con 0x560f19359c50 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a11ffb640 10 client.25592 handle_command_reply: tid=0 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a329e7740 1 client.25592 shutdown 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 msgr2=0x560f192f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.465+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192f3eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3a180a88b0 tx=0x7f3a180e34f0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 shutdown_connections 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560f19359c50 0x560f1935c070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560f192dcdb0 0x560f192f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560f192cca10 0x560f192f3eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a11ffb640 0 client.25592 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 --2- 172.21.15.33:0/2092929842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560f192cc0f0 0x560f192e5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 >> 172.21.15.33:0/2092929842 conn(0x560f192c9f20 msgr2=0x560f192cafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 shutdown_connections 2024-09-10T21:07:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 1 -- 172.21.15.33:0/2092929842 wait complete. 2024-09-10T21:07:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a329e7740 20 client.25592 trim_cache size 0 max 16384 2024-09-10T21:07:50.476 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3a140b52e0 msgr2=0x7f3a140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3a140b52e0 0x7f3a140b77a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a20033040 tx=0x7f3a20005c50 comp rx=0 tx=0).stop 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a281076a0 msgr2=0x7f3a28199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a281076a0 0x7f3a28199790 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e9f0 tx=0x7f3a1800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 shutdown_connections 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3a140b52e0 0x7f3a140b77a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3a28108020 0x7f3a2819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a281076a0 0x7f3a28199790 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 --2- 172.21.15.33:0/383145357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a28101c90 0x7f3a28199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.466+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 >> 172.21.15.33:0/383145357 conn(0x7f3a280fd700 msgr2=0x7f3a2806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.467+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 shutdown_connections 2024-09-10T21:07:50.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:50.467+0000 7f3a30612640 1 -- 172.21.15.33:0/383145357 wait complete. 2024-09-10T21:07:50.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 339989 10.37 MB/sec execute 51 sec latency 52.172 ms 2024-09-10T21:07:51.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:07:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:51 smithi033 ceph-mon[30860]: pgmap v462: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-09-10T21:07:51.719 INFO:tasks.workunit.client.0.smithi033.stdout: 1 341644 10.33 MB/sec execute 52 sec latency 15.318 ms 2024-09-10T21:07:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:51 smithi104 ceph-mon[35325]: pgmap v462: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-09-10T21:07:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:51 smithi174 ceph-mon[35635]: pgmap v462: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.1 MiB/s wr, 359 op/s 2024-09-10T21:07:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 -- 172.21.15.33:0/3121171878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6781076a0 msgr2=0x7fd678107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 --2- 172.21.15.33:0/3121171878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6781076a0 0x7fd678107ae0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009900 tx=0x7fd66c02a410 comp rx=0 tx=0).stop 2024-09-10T21:07:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 -- 172.21.15.33:0/3121171878 shutdown_connections 2024-09-10T21:07:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 --2- 172.21.15.33:0/3121171878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd678108020 0x7fd67810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 --2- 172.21.15.33:0/3121171878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6781076a0 0x7fd678107ae0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 --2- 172.21.15.33:0/3121171878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd678101c90 0x7fd678102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 -- 172.21.15.33:0/3121171878 >> 172.21.15.33:0/3121171878 conn(0x7fd6780fd700 msgr2=0x7fd6780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 -- 172.21.15.33:0/3121171878 shutdown_connections 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.107+0000 7fd67e354640 1 -- 172.21.15.33:0/3121171878 wait complete. 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.108+0000 7fd67e354640 1 Processor -- start 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.108+0000 7fd67e354640 1 -- start start 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd678101c90 0x7fd6781992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6781076a0 0x7fd678199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678108020 0x7fd67819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111ce0 con 0x7fd678108020 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111b60 con 0x7fd6781076a0 2024-09-10T21:07:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd678111e60 con 0x7fd678101c90 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678108020 0x7fd67819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678108020 0x7fd67819ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40630/0 (socket says 172.21.15.33:40630) 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 -- 172.21.15.33:0/409328418 learned_addr learned my addr 172.21.15.33:0/409328418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd6777fe640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6781076a0 0x7fd678199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd677fff640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd678101c90 0x7fd6781992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 -- 172.21.15.33:0/409328418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd678101c90 msgr2=0x7fd6781992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd678101c90 0x7fd6781992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 -- 172.21.15.33:0/409328418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6781076a0 msgr2=0x7fd678199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6781076a0 0x7fd678199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 -- 172.21.15.33:0/409328418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd67819f380 con 0x7fd678108020 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678108020 0x7fd67819ebb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd67000efd0 tx=0x7fd67000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd6757fa640 1 -- 172.21.15.33:0/409328418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd670003b70 con 0x7fd678108020 2024-09-10T21:07:52.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd67819f670 con 0x7fd678108020 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6780fdae0 con 0x7fd678108020 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd6757fa640 1 -- 172.21.15.33:0/409328418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd670003d10 con 0x7fd678108020 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.109+0000 7fd6757fa640 1 -- 172.21.15.33:0/409328418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700108f0 con 0x7fd678108020 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.112+0000 7fd680729740 1 Processor -- start 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.112+0000 7fd6757fa640 1 -- 172.21.15.33:0/409328418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd67001f020 con 0x7fd678108020 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.112+0000 7fd680729740 1 -- start start 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd680729740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706dfa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd680729740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e06460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd680729740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e08d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd680729740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564706d70ea0 con 0x564706dfaa10 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd680729740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564706d602d0 con 0x564706e069a0 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd680729740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564706d640e0 con 0x564706dfa0f0 2024-09-10T21:07:52.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e06460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706dfa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e06460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40642/0 (socket says 172.21.15.33:40642) 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 -- 172.21.15.33:0/2693630531 learned_addr learned my addr 172.21.15.33:0/2693630531 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd6777fe640 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e08d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 -- 172.21.15.33:0/2693630531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 msgr2=0x564706dfa4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706dfa4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 -- 172.21.15.33:0/2693630531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 msgr2=0x564706e08d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e08d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 -- 172.21.15.33:0/2693630531 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564706abe650 con 0x564706dfaa10 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd6757fa640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6600b52e0 0x7fd6600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.113+0000 7fd677fff640 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e06460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd668009cf0 tx=0x7fd668011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2412d67e1161c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.114+0000 7fd6777fe640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6600b52e0 0x7fd6600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.114+0000 7fd6757fa640 1 -- 172.21.15.33:0/409328418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd6700e5b20 con 0x7fd678108020 2024-09-10T21:07:52.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.114+0000 7fd65e7fc640 1 -- 172.21.15.33:0/2693630531 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668004230 con 0x564706dfaa10 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.114+0000 7fd65e7fc640 1 -- 172.21.15.33:0/2693630531 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6680043d0 con 0x564706dfaa10 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.114+0000 7fd65e7fc640 1 -- 172.21.15.33:0/2693630531 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668019880 con 0x564706dfaa10 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 -- 172.21.15.33:0/2693630531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 msgr2=0x564706e06460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd6777fe640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6600b52e0 0x7fd6600b77a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd66c031440 tx=0x7fd66c005780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e06460 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd668009cf0 tx=0x7fd668011810 comp rx=0 tx=0).stop 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 -- 172.21.15.33:0/2693630531 shutdown_connections 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e08d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e06460 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 --2- 172.21.15.33:0/2693630531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706dfa4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 -- 172.21.15.33:0/2693630531 >> 172.21.15.33:0/2693630531 conn(0x564706df7f20 msgr2=0x564706df90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 -- 172.21.15.33:0/2693630531 shutdown_connections 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 -- 172.21.15.33:0/2693630531 wait complete. 2024-09-10T21:07:52.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 -- start start 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.115+0000 7fd680729740 1 Processor -- start 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd680729740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706e06170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd680729740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd680729740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e03b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd680729740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564706bc1700 con 0x564706dfaa10 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd680729740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564706cfb160 con 0x564706e069a0 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd680729740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564706e09670 con 0x564706dfa0f0 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd680729740 10 client.0 _ll_register_callbacks cb 0x564706d149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd677fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706e06170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd67c8f1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e03b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd677fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706e06170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35880/0 (socket says 172.21.15.33:35880) 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e035c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40650/0 (socket says 172.21.15.33:40650) 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd677fff640 1 -- 172.21.15.33:0/1968283879 learned_addr learned my addr 172.21.15.33:0/1968283879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd67c8f1640 1 -- 172.21.15.33:0/1968283879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 msgr2=0x564706e06170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706e06170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd67c8f1640 1 -- 172.21.15.33:0/1968283879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 msgr2=0x564706e035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd67c8f1640 1 -- 172.21.15.33:0/1968283879 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564706d28c90 con 0x564706e069a0 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e03b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6700a9550 tx=0x7fd6700ac8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5187ebac1f0c7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.116+0000 7fd6577fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd6577fe640 1 -- 172.21.15.33:0/1968283879 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700abdf0 con 0x564706e069a0 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd6577fe640 1 -- 172.21.15.33:0/1968283879 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd6700d31b0 con 0x564706e069a0 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6680043d0 con 0x564706e069a0 2024-09-10T21:07:52.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd6577fe640 1 -- 172.21.15.33:0/1968283879 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd6700c1e60 con 0x564706e069a0 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 10 client.25604 fetch_fsmap learned FSMap version 36 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564706e14f10 con 0x564706e069a0 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd6577fe640 1 -- 172.21.15.33:0/1968283879 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6680043d0 con 0x564706e069a0 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd6577fe640 1 -- 172.21.15.33:0/1968283879 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fd6700e3ae0 con 0x564706e069a0 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 10 client.25604 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 10 client.25604 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 20 client.25604 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e16410 0x564706e188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 4 client.25604 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564706d72df0 con 0x564706e16410 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.117+0000 7fd677fff640 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e16410 0x564706e188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.118+0000 7fd677fff640 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e16410 0x564706e188b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b294206d6b90550 server_cookie=8434efd74e0aac4d in_seq=0 out_seq=0 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.118+0000 7fd6577fe640 10 client.25604 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:52.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd6577fe640 1 -- 172.21.15.33:0/1968283879 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564706d72df0 con 0x564706e16410 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd6577fe640 10 client.25604 handle_command_reply: tid=0 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 client.25604 shutdown 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 msgr2=0x564706e03b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e03b00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd6700a9550 tx=0x7fd6700ac8b0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 shutdown_connections 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e03b00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e16410 0x564706e188b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 --2- 172.21.15.33:0/1968283879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706e06170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd6577fe640 0 client.25604 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 >> 172.21.15.33:0/1968283879 conn(0x564706df7f20 msgr2=0x564706dfda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 shutdown_connections 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 1 -- 172.21.15.33:0/1968283879 wait complete. 2024-09-10T21:07:52.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.119+0000 7fd680729740 20 client.25604 trim_cache size 0 max 16384 2024-09-10T21:07:52.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 Processor -- start 2024-09-10T21:07:52.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 -- start start 2024-09-10T21:07:52.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706df9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e007d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e00d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd66400e470 con 0x564706dfaa10 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd6640100d0 con 0x564706e069a0 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd680729740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd66400eb40 con 0x564706dfa0f0 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd677fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706df9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd6777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40652/0 (socket says 172.21.15.33:40652) 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd6777fe640 1 -- 172.21.15.33:0/111398892 learned_addr learned my addr 172.21.15.33:0/111398892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.135+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e00d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd677fff640 1 -- 172.21.15.33:0/111398892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 msgr2=0x564706e00d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd677fff640 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e00d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd677fff640 1 -- 172.21.15.33:0/111398892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 msgr2=0x564706e007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd677fff640 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd677fff640 1 -- 172.21.15.33:0/111398892 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564706e064e0 con 0x564706dfa0f0 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd6777fe640 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd677fff640 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706df9610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd668003f50 tx=0x7fd668011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab30036c0da3fe01 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd65dffb640 1 -- 172.21.15.33:0/111398892 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668031070 con 0x564706dfa0f0 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd65dffb640 1 -- 172.21.15.33:0/111398892 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd668005040 con 0x564706dfa0f0 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.136+0000 7fd65dffb640 1 -- 172.21.15.33:0/111398892 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd668019790 con 0x564706dfa0f0 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 -- 172.21.15.33:0/111398892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 msgr2=0x564706df9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706df9610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd668003f50 tx=0x7fd668011810 comp rx=0 tx=0).stop 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 -- 172.21.15.33:0/111398892 shutdown_connections 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e00d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfaa10 0x564706e007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 --2- 172.21.15.33:0/111398892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfa0f0 0x564706df9610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 -- 172.21.15.33:0/111398892 >> 172.21.15.33:0/111398892 conn(0x564706df7f20 msgr2=0x564706df8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 -- 172.21.15.33:0/111398892 shutdown_connections 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.137+0000 7fd680729740 1 -- 172.21.15.33:0/111398892 wait complete. 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 -- start start 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 Processor -- start 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfa0f0 0x564706e0aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfaa10 0x564706e0b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e0b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd66400f9a0 con 0x564706dfa0f0 2024-09-10T21:07:52.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd66400f210 con 0x564706e069a0 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd66400c430 con 0x564706dfaa10 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.138+0000 7fd680729740 10 client.0 _ll_register_callbacks cb 0x7fd670009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e0b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e0b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55076/0 (socket says 172.21.15.33:55076) 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 -- 172.21.15.33:0/2513885302 learned_addr learned my addr 172.21.15.33:0/2513885302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd677fff640 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfa0f0 0x564706e0aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd6777fe640 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfaa10 0x564706e0b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 -- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfaa10 msgr2=0x564706e0b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfaa10 0x564706e0b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 -- 172.21.15.33:0/2513885302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfa0f0 msgr2=0x564706e0aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfa0f0 0x564706e0aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 -- 172.21.15.33:0/2513885302 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564706e825a0 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd67c8f1640 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e0b550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6700a8860 tx=0x7fd670004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49ae70e110311637 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd6577fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd6577fe640 1 -- 172.21.15.33:0/2513885302 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6700ac0b0 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd6577fe640 1 -- 172.21.15.33:0/2513885302 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd6700ab740 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.139+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd668005040 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd6577fe640 1 -- 172.21.15.33:0/2513885302 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd6700a8930 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 10 client.25607 fetch_fsmap learned FSMap version 36 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564706e83810 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd6577fe640 1 -- 172.21.15.33:0/2513885302 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd668005040 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd6577fe640 1 -- 172.21.15.33:0/2513885302 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7fd670010a90 con 0x564706e069a0 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 10 client.25607 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 10 client.25607 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:07:52.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 20 client.25607 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e83f90 0x564706e863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 4 client.25607 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564706d6e8a0 con 0x564706e83f90 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.140+0000 7fd677fff640 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e83f90 0x564706e863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd677fff640 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e83f90 0x564706e863b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3128ecb226f4372d server_cookie=eb1d084a51a606b1 in_seq=0 out_seq=0 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd6577fe640 10 client.25607 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd6577fe640 1 -- 172.21.15.33:0/2513885302 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564706d6e8a0 con 0x564706e83f90 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd6577fe640 10 client.25607 handle_command_reply: tid=0 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 client.25607 shutdown 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 msgr2=0x564706e0b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e0b550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd6700a8860 tx=0x7fd670004a40 comp rx=0 tx=0).stop 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 shutdown_connections 2024-09-10T21:07:52.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564706e069a0 0x564706e0b550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x564706e83f90 0x564706e863b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564706dfaa10 0x564706e0b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 --2- 172.21.15.33:0/2513885302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564706dfa0f0 0x564706e0aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd6577fe640 0 client.25607 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 >> 172.21.15.33:0/2513885302 conn(0x564706df7f20 msgr2=0x564706e04bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 shutdown_connections 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 1 -- 172.21.15.33:0/2513885302 wait complete. 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.141+0000 7fd680729740 20 client.25607 trim_cache size 0 max 16384 2024-09-10T21:07:52.159 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6600b52e0 msgr2=0x7fd6600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6600b52e0 0x7fd6600b77a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd66c031440 tx=0x7fd66c005780 comp rx=0 tx=0).stop 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678108020 msgr2=0x7fd67819ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678108020 0x7fd67819ebb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd67000efd0 tx=0x7fd67000c6a0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 shutdown_connections 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6600b52e0 0x7fd6600b77a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd678108020 0x7fd67819ebb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6781076a0 0x7fd678199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 --2- 172.21.15.33:0/409328418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd678101c90 0x7fd6781992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 >> 172.21.15.33:0/409328418 conn(0x7fd6780fd700 msgr2=0x7fd67806ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.142+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 shutdown_connections 2024-09-10T21:07:52.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:52.143+0000 7fd67e354640 1 -- 172.21.15.33:0/409328418 wait complete. 2024-09-10T21:07:52.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 344300 10.45 MB/sec execute 53 sec latency 28.646 ms 2024-09-10T21:07:52.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:52 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:52 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:52.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:52 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:52.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:07:53.215 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:53.640 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:53 smithi033 ceph-mon[30860]: pgmap v463: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 307 op/s 2024-09-10T21:07:53.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 346435 10.39 MB/sec execute 54 sec latency 11.626 ms 2024-09-10T21:07:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 -- 172.21.15.33:0/1489449407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 msgr2=0x7f01cc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 --2- 172.21.15.33:0/1489449407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 0x7f01cc10e820 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ca0 tx=0x7f01c402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 -- 172.21.15.33:0/1489449407 shutdown_connections 2024-09-10T21:07:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 --2- 172.21.15.33:0/1489449407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 0x7f01cc10e820 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 --2- 172.21.15.33:0/1489449407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc1076a0 0x7f01cc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 --2- 172.21.15.33:0/1489449407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 0x7f01cc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 -- 172.21.15.33:0/1489449407 >> 172.21.15.33:0/1489449407 conn(0x7f01cc0fd700 msgr2=0x7f01cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 -- 172.21.15.33:0/1489449407 shutdown_connections 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.725+0000 7f01d26b5640 1 -- 172.21.15.33:0/1489449407 wait complete. 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 Processor -- start 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 -- start start 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 0x7f01cc19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc1076a0 0x7f01cc19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 0x7f01cc1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc111d30 con 0x7f01cc108020 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc111bb0 con 0x7f01cc101c90 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.726+0000 7f01d26b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc111eb0 con 0x7f01cc1076a0 2024-09-10T21:07:53.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 0x7f01cc19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 0x7f01cc1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc1076a0 0x7f01cc19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 0x7f01cc19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55096/0 (socket says 172.21.15.33:55096) 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc1076a0 0x7f01cc19b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35936/0 (socket says 172.21.15.33:35936) 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 -- 172.21.15.33:0/3346263859 learned_addr learned my addr 172.21.15.33:0/3346263859 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 -- 172.21.15.33:0/3346263859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc1076a0 msgr2=0x7f01cc19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc1076a0 0x7f01cc19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 -- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 msgr2=0x7f01cc1a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 0x7f01cc1a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 -- 172.21.15.33:0/3346263859 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01cc1a14d0 con 0x7f01cc101c90 2024-09-10T21:07:53.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01cbfff640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 0x7f01cc19b420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9f0 tx=0x7f01bc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01c97fa640 1 -- 172.21.15.33:0/3346263859 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc00cd80 con 0x7f01cc101c90 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01cc1a17c0 con 0x7f01cc101c90 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01c97fa640 1 -- 172.21.15.33:0/3346263859 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01bc0108e0 con 0x7f01cc101c90 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01c97fa640 1 -- 172.21.15.33:0/3346263859 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc018980 con 0x7f01cc101c90 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.727+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01cc1a8050 con 0x7f01cc101c90 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01c97fa640 1 -- 172.21.15.33:0/3346263859 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f01bc010430 con 0x7f01cc101c90 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 Processor -- start 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 -- start start 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb92170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad1baf9ea0 con 0x55ad1bb83df0 2024-09-10T21:07:53.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bae92d0 con 0x55ad1bb834d0 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad1baed0e0 con 0x55ad1bb8fd80 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.731+0000 7f01cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb838b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55102/0 (socket says 172.21.15.33:55102) 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 -- 172.21.15.33:0/739435909 learned_addr learned my addr 172.21.15.33:0/739435909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01cb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb92170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 -- 172.21.15.33:0/739435909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 msgr2=0x55ad1bb92170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb92170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 -- 172.21.15.33:0/739435909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb83df0 msgr2=0x55ad1bb8f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 -- 172.21.15.33:0/739435909 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1b847650 con 0x55ad1bb834d0 2024-09-10T21:07:53.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb92170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01c97fa640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01b40b5360 0x7f01b40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01cbfff640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01b40b5360 0x7f01b40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01d0c52640 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb838b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ca0 tx=0x7f01c402f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f775c3b24b32b911 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01c97fa640 1 -- 172.21.15.33:0/3346263859 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f01bc0e5ea0 con 0x7f01cc101c90 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01b27fc640 1 -- 172.21.15.33:0/739435909 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4031db0 con 0x55ad1bb834d0 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01b27fc640 1 -- 172.21.15.33:0/739435909 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c4036030 con 0x55ad1bb834d0 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.732+0000 7f01b27fc640 1 -- 172.21.15.33:0/739435909 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c40042a0 con 0x55ad1bb834d0 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 -- 172.21.15.33:0/739435909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 msgr2=0x55ad1bb838b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01cbfff640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01b40b5360 0x7f01b40b7820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9c0 tx=0x7f01bc0e9920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb838b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009ca0 tx=0x7f01c402f3c0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 -- 172.21.15.33:0/739435909 shutdown_connections 2024-09-10T21:07:53.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb92170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/739435909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb838b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 -- 172.21.15.33:0/739435909 >> 172.21.15.33:0/739435909 conn(0x55ad1bb81300 msgr2=0x55ad1bb824a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 -- 172.21.15.33:0/739435909 shutdown_connections 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 -- 172.21.15.33:0/739435909 wait complete. 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.733+0000 7f01d4a8a740 1 -- start start 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 1 Processor -- start 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb8f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb8d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ad1b94a700 con 0x55ad1bb8fd80 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ad1ba84160 con 0x55ad1bb834d0 2024-09-10T21:07:53.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad1bb92820 con 0x55ad1bb83df0 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d4a8a740 10 client.0 _ll_register_callbacks cb 0x55ad1ba9d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01cb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb8f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01cb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35950/0 (socket says 172.21.15.33:35950) 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01cb7fe640 1 -- 172.21.15.33:0/2195898315 learned_addr learned my addr 172.21.15.33:0/2195898315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.734+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb8d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d0c52640 1 -- 172.21.15.33:0/2195898315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 msgr2=0x55ad1bb8cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d0c52640 1 -- 172.21.15.33:0/2195898315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb8fd80 msgr2=0x55ad1bb8d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb8d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d0c52640 1 -- 172.21.15.33:0/2195898315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1bab1c90 con 0x55ad1bb834d0 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb8f6d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01c402ade0 tx=0x7f01c4003e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d83969f7e321ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:53.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2195898315 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4059060 con 0x55ad1bb834d0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2195898315 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f01c4004040 con 0x55ad1bb834d0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c4036030 con 0x55ad1bb834d0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2195898315 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f01c4004b40 con 0x55ad1bb834d0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d4a8a740 10 client.25619 fetch_fsmap learned FSMap version 36 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.735+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ad1bb9e1e0 con 0x55ad1bb834d0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2195898315 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c4036030 con 0x55ad1bb834d0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2195898315 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f01c4043070 con 0x55ad1bb834d0 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01d4a8a740 10 client.25619 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01d4a8a740 10 client.25619 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:07:53.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01d4a8a740 20 client.25619 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bb9f530 0x55ad1bba1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01d4a8a740 4 client.25619 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad1bafbdf0 con 0x55ad1bb9f530 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bb9f530 0x55ad1bba1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bb9f530 0x55ad1bba1920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45ec42b51c6f35a4 server_cookie=f5d28807ca7b389a in_seq=0 out_seq=0 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.736+0000 7f01ab7fe640 10 client.25619 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2195898315 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ad1bafbdf0 con 0x55ad1bb9f530 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01ab7fe640 10 client.25619 handle_command_reply: tid=0 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 client.25619 shutdown 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 msgr2=0x55ad1bb8f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb8f6d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01c402ade0 tx=0x7f01c4003e50 comp rx=0 tx=0).stop 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 shutdown_connections 2024-09-10T21:07:53.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bb9f530 0x55ad1bba1920 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb8d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb8cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2195898315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb834d0 0x55ad1bb8f6d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01ab7fe640 0 client.25619 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 >> 172.21.15.33:0/2195898315 conn(0x55ad1bb81300 msgr2=0x55ad1bb86ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.737+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 shutdown_connections 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.738+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2195898315 wait complete. 2024-09-10T21:07:53.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.738+0000 7f01d4a8a740 20 client.25619 trim_cache size 0 max 16384 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 Processor -- start 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 -- start start 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb80c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb83df0 0x55ad1bb86cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb87210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1b05e0 con 0x55ad1bb834d0 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1b2240 con 0x55ad1bb83df0 2024-09-10T21:07:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1b0cb0 con 0x55ad1bb8fd80 2024-09-10T21:07:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb80c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb80c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40718/0 (socket says 172.21.15.33:40718) 2024-09-10T21:07:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d0c52640 1 -- 172.21.15.33:0/1076849283 learned_addr learned my addr 172.21.15.33:0/1076849283 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb83df0 0x55ad1bb86cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01d0c52640 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb87210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01cb7fe640 1 -- 172.21.15.33:0/1076849283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 msgr2=0x55ad1bb87210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb87210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01cb7fe640 1 -- 172.21.15.33:0/1076849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 msgr2=0x55ad1bb80c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb80c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.752+0000 7f01cb7fe640 1 -- 172.21.15.33:0/1076849283 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad1bb877f0 con 0x55ad1bb83df0 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.753+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb83df0 0x55ad1bb86cd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55ad1bb8d980 tx=0x7f01c000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4fd621f7a78490a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.753+0000 7f01b1ffb640 1 -- 172.21.15.33:0/1076849283 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c00035f0 con 0x55ad1bb83df0 2024-09-10T21:07:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.753+0000 7f01b1ffb640 1 -- 172.21.15.33:0/1076849283 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c0003c40 con 0x55ad1bb83df0 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.753+0000 7f01b1ffb640 1 -- 172.21.15.33:0/1076849283 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c001db10 con 0x55ad1bb83df0 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 -- 172.21.15.33:0/1076849283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb83df0 msgr2=0x55ad1bb86cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb83df0 0x55ad1bb86cd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x55ad1bb8d980 tx=0x7f01c000b390 comp rx=0 tx=0).stop 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 -- 172.21.15.33:0/1076849283 shutdown_connections 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb87210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb83df0 0x55ad1bb86cd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/1076849283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb80c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 -- 172.21.15.33:0/1076849283 >> 172.21.15.33:0/1076849283 conn(0x55ad1bb81300 msgr2=0x55ad1bb865c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 -- 172.21.15.33:0/1076849283 shutdown_connections 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 -- 172.21.15.33:0/1076849283 wait complete. 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 -- start start 2024-09-10T21:07:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.754+0000 7f01d4a8a740 1 Processor -- start 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb92c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb93190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d4a8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1b1b10 con 0x55ad1bb834d0 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1b1380 con 0x55ad1bb8fd80 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d4a8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01cc1ae5a0 con 0x55ad1bb83df0 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d4a8a740 10 client.0 _ll_register_callbacks cb 0x7f01bc0e97f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb92c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb92c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40724/0 (socket says 172.21.15.33:40724) 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 -- 172.21.15.33:0/2055811575 learned_addr learned my addr 172.21.15.33:0/2055811575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb93190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 -- 172.21.15.33:0/2055811575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 msgr2=0x55ad1bb93190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb93190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 -- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb8fd80 msgr2=0x55ad1bb936d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:53.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb936d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 -- 172.21.15.33:0/2055811575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad1bc0c3c0 con 0x55ad1bb834d0 2024-09-10T21:07:53.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb93190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01d0c52640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb92c50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009800 tx=0x7f01c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61327a29ed1c3870 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.755+0000 7f01ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2055811575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4064070 con 0x55ad1bb834d0 2024-09-10T21:07:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2055811575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f01c402fb80 con 0x55ad1bb834d0 2024-09-10T21:07:53.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c0003c40 con 0x55ad1bb834d0 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2055811575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f01c4059070 con 0x55ad1bb834d0 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 10 client.16059 fetch_fsmap learned FSMap version 36 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ad1bc0d680 con 0x55ad1bb834d0 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2055811575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c0003c40 con 0x55ad1bb834d0 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2055811575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f01c4043040 con 0x55ad1bb834d0 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 10 client.16059 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 10 client.16059 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 20 client.16059 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bc0dd70 0x55ad1bc10190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 4 client.16059 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.756+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad1baf78a0 con 0x55ad1bc0dd70 2024-09-10T21:07:53.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bc0dd70 0x55ad1bc10190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01ab7fe640 10 client.16059 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01cb7fe640 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bc0dd70 0x55ad1bc10190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fa0abaf7c4cac64e server_cookie=5e438a6c4f5cf027 in_seq=0 out_seq=0 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01ab7fe640 1 -- 172.21.15.33:0/2055811575 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad1baf78a0 con 0x55ad1bc0dd70 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01ab7fe640 10 client.16059 handle_command_reply: tid=0 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 client.16059 shutdown 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 msgr2=0x55ad1bb92c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb92c50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009800 tx=0x7f01c4004a40 comp rx=0 tx=0).stop 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 shutdown_connections 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ad1bb8fd80 0x55ad1bb936d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad1bb83df0 0x55ad1bb93190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55ad1bc0dd70 0x55ad1bc10190 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 --2- 172.21.15.33:0/2055811575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ad1bb834d0 0x55ad1bb92c50 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 >> 172.21.15.33:0/2055811575 conn(0x55ad1bb81300 msgr2=0x55ad1bb86de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.757+0000 7f01ab7fe640 0 client.16059 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:07:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 shutdown_connections 2024-09-10T21:07:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d4a8a740 1 -- 172.21.15.33:0/2055811575 wait complete. 2024-09-10T21:07:53.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d4a8a740 20 client.16059 trim_cache size 0 max 16384 2024-09-10T21:07:53.767 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01b40b5360 msgr2=0x7f01b40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01b40b5360 0x7f01b40b7820 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9c0 tx=0x7f01bc0e9920 comp rx=0 tx=0).stop 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 msgr2=0x7f01cc19b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 0x7f01cc19b420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00e9f0 tx=0x7f01bc00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 shutdown_connections 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f01b40b5360 0x7f01b40b7820 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f01cc108020 0x7f01cc1a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01cc1076a0 0x7f01cc19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 --2- 172.21.15.33:0/3346263859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01cc101c90 0x7f01cc19b420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 >> 172.21.15.33:0/3346263859 conn(0x7f01cc0fd700 msgr2=0x7f01cc06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:53.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.758+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 shutdown_connections 2024-09-10T21:07:53.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:53.759+0000 7f01d26b5640 1 -- 172.21.15.33:0/3346263859 wait complete. 2024-09-10T21:07:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:53 smithi104 ceph-mon[35325]: pgmap v463: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 307 op/s 2024-09-10T21:07:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:53 smithi174 ceph-mon[35635]: pgmap v463: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 307 op/s 2024-09-10T21:07:54.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:07:54.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 348141 10.43 MB/sec execute 55 sec latency 14.210 ms 2024-09-10T21:07:54.810 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.272+0000 7f945a235640 1 -- 172.21.15.33:0/2112548550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 msgr2=0x7f945410f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.272+0000 7f945a235640 1 --2- 172.21.15.33:0/2112548550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 0x7f945410f590 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f944c009ca0 tx=0x7f944c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 -- 172.21.15.33:0/2112548550 shutdown_connections 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 --2- 172.21.15.33:0/2112548550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 0x7f945410f590 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 --2- 172.21.15.33:0/2112548550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94541045f0 0x7f945410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 --2- 172.21.15.33:0/2112548550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9454103cd0 0x7f94541040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 -- 172.21.15.33:0/2112548550 >> 172.21.15.33:0/2112548550 conn(0x7f94540fd770 msgr2=0x7f94540ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 -- 172.21.15.33:0/2112548550 shutdown_connections 2024-09-10T21:07:55.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 -- 172.21.15.33:0/2112548550 wait complete. 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 Processor -- start 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.273+0000 7f945a235640 1 -- start start 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f945a235640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9454103cd0 0x7f9454101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f945a235640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94541045f0 0x7f9454101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f945a235640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 0x7f9454102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f945a235640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112da0 con 0x7f945410d130 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f945a235640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112c20 con 0x7f9454103cd0 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f945a235640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9454112f20 con 0x7f94541045f0 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f9453fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 0x7f9454102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f94537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9454103cd0 0x7f9454101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f9453fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 0x7f9454102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40740/0 (socket says 172.21.15.33:40740) 2024-09-10T21:07:55.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f9453fff640 1 -- 172.21.15.33:0/2570378482 learned_addr learned my addr 172.21.15.33:0/2570378482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f9452ffd640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94541045f0 0x7f9454101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f94537fe640 1 -- 172.21.15.33:0/2570378482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94541045f0 msgr2=0x7f9454101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f94537fe640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94541045f0 0x7f9454101ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f94537fe640 1 -- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 msgr2=0x7f9454102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f94537fe640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 0x7f9454102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f94537fe640 1 -- 172.21.15.33:0/2570378482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94541a7670 con 0x7f9454103cd0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f94537fe640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9454103cd0 0x7f9454101980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f944400e9f0 tx=0x7f944400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.274+0000 7f9450ff9640 1 -- 172.21.15.33:0/2570378482 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944400ccc0 con 0x7f9454103cd0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.275+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94541a7960 con 0x7f9454103cd0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.275+0000 7f9450ff9640 1 -- 172.21.15.33:0/2570378482 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94440108e0 con 0x7f9454103cd0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.275+0000 7f9450ff9640 1 -- 172.21.15.33:0/2570378482 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444018880 con 0x7f9454103cd0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.275+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94541a7ef0 con 0x7f9454103cd0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.277+0000 7f9450ff9640 1 -- 172.21.15.33:0/2570378482 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9444010430 con 0x7f9454103cd0 2024-09-10T21:07:55.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.277+0000 7f945c60a740 1 Processor -- start 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.277+0000 7f945c60a740 1 -- start start 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.277+0000 7f945c60a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.277+0000 7f945c60a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e29b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.277+0000 7f945c60a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e29e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f945c60a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e13e205ea0 con 0x55e13e29bd80 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f945c60a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e13e1f52d0 con 0x55e13e28fdf0 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f945c60a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e13e1f90e0 con 0x55e13e28f4d0 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e29e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f94537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e29b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f94537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e29b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55144/0 (socket says 172.21.15.33:55144) 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e29e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40742/0 (socket says 172.21.15.33:40742) 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f94537fe640 1 -- 172.21.15.33:0/1691323018 learned_addr learned my addr 172.21.15.33:0/1691323018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:55.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9453fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9453fff640 1 -- 172.21.15.33:0/1691323018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 msgr2=0x55e13e29b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9453fff640 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e29b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9453fff640 1 -- 172.21.15.33:0/1691323018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 msgr2=0x55e13e29e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9453fff640 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e29e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9453fff640 1 -- 172.21.15.33:0/1691323018 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13df53650 con 0x55e13e28f4d0 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9452ffd640 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e29e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9450ff9640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94400b52e0 0x7f94400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.278+0000 7f9453fff640 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28f8b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f944c005e30 tx=0x7f944c005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f8883e878f889e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f94537fe640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94400b52e0 0x7f94400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f9449ffb640 1 -- 172.21.15.33:0/1691323018 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c031cf0 con 0x55e13e28f4d0 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f9449ffb640 1 -- 172.21.15.33:0/1691323018 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944c036030 con 0x55e13e28f4d0 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f9449ffb640 1 -- 172.21.15.33:0/1691323018 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c0042a0 con 0x55e13e28f4d0 2024-09-10T21:07:55.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f9450ff9640 1 -- 172.21.15.33:0/2570378482 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f94440e5f30 con 0x7f9454103cd0 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f945c60a740 1 -- 172.21.15.33:0/1691323018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 msgr2=0x55e13e28f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f94537fe640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94400b52e0 0x7f94400b77a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f944401afd0 tx=0x7f94440e9970 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.279+0000 7f945c60a740 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28f8b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f944c005e30 tx=0x7f944c005ef0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- 172.21.15.33:0/1691323018 shutdown_connections 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e29e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e29b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 --2- 172.21.15.33:0/1691323018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28f8b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- 172.21.15.33:0/1691323018 >> 172.21.15.33:0/1691323018 conn(0x55e13e28d300 msgr2=0x55e13e28e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- 172.21.15.33:0/1691323018 shutdown_connections 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- 172.21.15.33:0/1691323018 wait complete. 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- start start 2024-09-10T21:07:55.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 Processor -- start 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e29b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e298a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e299000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e13e056700 con 0x55e13e29bd80 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e13e190160 con 0x55e13e28fdf0 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e13e29e820 con 0x55e13e28f4d0 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.280+0000 7f945c60a740 10 client.0 _ll_register_callbacks cb 0x55e13e1a99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e298a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9453fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e29b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e298a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55150/0 (socket says 172.21.15.33:55150) 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 -- 172.21.15.33:0/2539285980 learned_addr learned my addr 172.21.15.33:0/2539285980 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9453fff640 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e299000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 -- 172.21.15.33:0/2539285980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 msgr2=0x55e13e29b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e29b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 -- 172.21.15.33:0/2539285980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 msgr2=0x55e13e299000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e299000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 -- 172.21.15.33:0/2539285980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13e1bdc90 con 0x55e13e28fdf0 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9452ffd640 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e298a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f943c012700 tx=0x7f943c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b631491862a972a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.281+0000 7f9432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.282+0000 7f9432ffd640 1 -- 172.21.15.33:0/2539285980 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c00f920 con 0x55e13e28fdf0 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.282+0000 7f9432ffd640 1 -- 172.21.15.33:0/2539285980 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f943c00fac0 con 0x55e13e28fdf0 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.282+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944c036030 con 0x55e13e28fdf0 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.282+0000 7f9432ffd640 1 -- 172.21.15.33:0/2539285980 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f943c00fcc0 con 0x55e13e28fdf0 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.282+0000 7f945c60a740 10 client.25637 fetch_fsmap learned FSMap version 36 2024-09-10T21:07:55.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.282+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e13e2aa180 con 0x55e13e28fdf0 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.282+0000 7f9432ffd640 1 -- 172.21.15.33:0/2539285980 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944c036030 con 0x55e13e28fdf0 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f9432ffd640 1 -- 172.21.15.33:0/2539285980 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f943c01c420 con 0x55e13e28fdf0 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f945c60a740 10 client.25637 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f945c60a740 10 client.25637 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f945c60a740 20 client.25637 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f945c60a740 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e2ab700 0x55e13e2adb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f945c60a740 4 client.25637 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e13e207df0 con 0x55e13e2ab700 2024-09-10T21:07:55.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f9453fff640 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e2ab700 0x55e13e2adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.283+0000 7f9432ffd640 10 client.25637 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.284+0000 7f9453fff640 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e2ab700 0x55e13e2adb20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=86c58d360c631ee server_cookie=bf0ac654bc9da4b9 in_seq=0 out_seq=0 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.284+0000 7f9432ffd640 1 -- 172.21.15.33:0/2539285980 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e13e207df0 con 0x55e13e2ab700 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.284+0000 7f9432ffd640 10 client.25637 handle_command_reply: tid=0 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.284+0000 7f945c60a740 1 client.25637 shutdown 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.284+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 msgr2=0x55e13e298a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.284+0000 7f945c60a740 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e298a60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f943c012700 tx=0x7f943c012bc0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 shutdown_connections 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e2ab700 0x55e13e2adb20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e299000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28fdf0 0x55e13e298a60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f9432ffd640 0 client.25637 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 --2- 172.21.15.33:0/2539285980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e29b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 >> 172.21.15.33:0/2539285980 conn(0x55e13e28d300 msgr2=0x55e13e292e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 shutdown_connections 2024-09-10T21:07:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 1 -- 172.21.15.33:0/2539285980 wait complete. 2024-09-10T21:07:55.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.285+0000 7f945c60a740 20 client.25637 trim_cache size 0 max 16384 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 Processor -- start 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 -- start start 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e28fdf0 0x55e13e29a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e29bd80 0x55e13e295c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b0650 con 0x55e13e28fdf0 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b22b0 con 0x55e13e29bd80 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f945c60a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b0d20 con 0x55e13e28f4d0 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e28fdf0 0x55e13e29a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e28fdf0 0x55e13e29a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40760/0 (socket says 172.21.15.33:40760) 2024-09-10T21:07:55.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 -- 172.21.15.33:0/2073934634 learned_addr learned my addr 172.21.15.33:0/2073934634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9453fff640 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e29bd80 0x55e13e295c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9453fff640 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 -- 172.21.15.33:0/2073934634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 msgr2=0x55e13e28cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 -- 172.21.15.33:0/2073934634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e29bd80 msgr2=0x55e13e295c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e29bd80 0x55e13e295c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.300+0000 7f9452ffd640 1 -- 172.21.15.33:0/2073934634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e13e29b240 con 0x55e13e28fdf0 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.301+0000 7f9452ffd640 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e28fdf0 0x55e13e29a7a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb52ecf4353cbb94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.301+0000 7f94497fa640 1 -- 172.21.15.33:0/2073934634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c009dd0 con 0x55e13e28fdf0 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.301+0000 7f94497fa640 1 -- 172.21.15.33:0/2073934634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c0054e0 con 0x55e13e28fdf0 2024-09-10T21:07:55.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.301+0000 7f94497fa640 1 -- 172.21.15.33:0/2073934634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c01ca60 con 0x55e13e28fdf0 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 -- 172.21.15.33:0/2073934634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e28fdf0 msgr2=0x55e13e29a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e28fdf0 0x55e13e29a7a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f943c000c00 tx=0x7f943c0095d0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 -- 172.21.15.33:0/2073934634 shutdown_connections 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e29bd80 0x55e13e295c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e28fdf0 0x55e13e29a7a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 --2- 172.21.15.33:0/2073934634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28f4d0 0x55e13e28cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 -- 172.21.15.33:0/2073934634 >> 172.21.15.33:0/2073934634 conn(0x55e13e28d300 msgr2=0x55e13e29a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 -- 172.21.15.33:0/2073934634 shutdown_connections 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 -- 172.21.15.33:0/2073934634 wait complete. 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 -- start start 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.302+0000 7f945c60a740 1 Processor -- start 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f945c60a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28f4d0 0x55e13e2a0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f945c60a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28fdf0 0x55e13e2a08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f945c60a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e311580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f945c60a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b1b80 con 0x55e13e29bd80 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f945c60a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94541b13f0 con 0x55e13e28f4d0 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f945c60a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94541ae610 con 0x55e13e28fdf0 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f945c60a740 10 client.0 _ll_register_callbacks cb 0x7f943c009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28f4d0 0x55e13e2a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28f4d0 0x55e13e2a0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55156/0 (socket says 172.21.15.33:55156) 2024-09-10T21:07:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 -- 172.21.15.33:0/1769092023 learned_addr learned my addr 172.21.15.33:0/1769092023 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e311580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9452ffd640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28fdf0 0x55e13e2a08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 -- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28fdf0 msgr2=0x55e13e2a08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28fdf0 0x55e13e2a08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 -- 172.21.15.33:0/1769092023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 msgr2=0x55e13e311580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e311580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 -- 172.21.15.33:0/1769092023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e13e311c90 con 0x55e13e28f4d0 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9452ffd640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28fdf0 0x55e13e2a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9453fff640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28f4d0 0x55e13e2a0370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f80 tx=0x7f943c009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f500d3c23f74f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.303+0000 7f9432ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f9432ffd640 1 -- 172.21.15.33:0/1769092023 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944c04a040 con 0x55e13e28f4d0 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f9432ffd640 1 -- 172.21.15.33:0/1769092023 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f944c031cf0 con 0x55e13e28f4d0 2024-09-10T21:07:55.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943c0054e0 con 0x55e13e28f4d0 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f9432ffd640 1 -- 172.21.15.33:0/1769092023 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f944c031510 con 0x55e13e28f4d0 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 10 client.25643 fetch_fsmap learned FSMap version 36 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e13e319050 con 0x55e13e28f4d0 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f9432ffd640 1 -- 172.21.15.33:0/1769092023 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c0054e0 con 0x55e13e28f4d0 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f9432ffd640 1 -- 172.21.15.33:0/1769092023 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 36) ==== 8386+0+0 (secure 0 0 0) 0x7f944c067050 con 0x55e13e28f4d0 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 10 client.25643 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 10 client.25643 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 20 client.25643 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e319540 0x55e13e31b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 4 client.25643 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e13e2038a0 con 0x55e13e319540 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.304+0000 7f9452ffd640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e319540 0x55e13e31b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:55.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f9452ffd640 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e319540 0x55e13e31b960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5160697edbdbd5c8 server_cookie=275f259591bd0966 in_seq=0 out_seq=0 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f9432ffd640 10 client.25643 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f9432ffd640 1 -- 172.21.15.33:0/1769092023 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e13e2038a0 con 0x55e13e319540 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f9432ffd640 10 client.25643 handle_command_reply: tid=0 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 client.25643 shutdown 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28f4d0 msgr2=0x55e13e2a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28f4d0 0x55e13e2a0370 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f80 tx=0x7f943c009d60 comp rx=0 tx=0).stop 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 shutdown_connections 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e13e319540 0x55e13e31b960 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e13e29bd80 0x55e13e311580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e13e28fdf0 0x55e13e2a08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 --2- 172.21.15.33:0/1769092023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e13e28f4d0 0x55e13e2a0370 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f9432ffd640 0 client.25643 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:07:55.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 >> 172.21.15.33:0/1769092023 conn(0x55e13e28d300 msgr2=0x55e13e292d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:55.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 shutdown_connections 2024-09-10T21:07:55.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.305+0000 7f945c60a740 1 -- 172.21.15.33:0/1769092023 wait complete. 2024-09-10T21:07:55.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945c60a740 20 client.25643 trim_cache size 0 max 16384 2024-09-10T21:07:55.315 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94400b52e0 msgr2=0x7f94400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94400b52e0 0x7f94400b77a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f944401afd0 tx=0x7f94440e9970 comp rx=0 tx=0).stop 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9454103cd0 msgr2=0x7f9454101980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9454103cd0 0x7f9454101980 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f944400e9f0 tx=0x7f944400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 shutdown_connections 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94400b52e0 0x7f94400b77a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f945410d130 0x7f9454102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94541045f0 0x7f9454101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 --2- 172.21.15.33:0/2570378482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9454103cd0 0x7f9454101980 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:55.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 >> 172.21.15.33:0/2570378482 conn(0x7f94540fd770 msgr2=0x7f94540fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:55.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.306+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 shutdown_connections 2024-09-10T21:07:55.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:55.307+0000 7f945a235640 1 -- 172.21.15.33:0/2570378482 wait complete. 2024-09-10T21:07:55.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 350619 10.46 MB/sec execute 56 sec latency 17.519 ms 2024-09-10T21:07:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:55 smithi104 ceph-mon[35325]: pgmap v464: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 368 op/s 2024-09-10T21:07:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:55 smithi174 ceph-mon[35635]: pgmap v464: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 368 op/s 2024-09-10T21:07:55.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:55 smithi033 ceph-mon[30860]: pgmap v464: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 368 op/s 2024-09-10T21:07:56.033 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:07:56.320 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:56.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 352137 10.41 MB/sec execute 57 sec latency 15.232 ms 2024-09-10T21:07:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 -- 172.21.15.33:0/1809793299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5081087f0 msgr2=0x7ff508108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/1809793299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5081087f0 0x7ff508108c30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc009900 tx=0x7ff4fc02a410 comp rx=0 tx=0).stop 2024-09-10T21:07:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 -- 172.21.15.33:0/1809793299 shutdown_connections 2024-09-10T21:07:56.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/1809793299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff508109170 0x7ff50810f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/1809793299 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5081087f0 0x7ff508108c30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/1809793299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508102de0 0x7ff5081031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 -- 172.21.15.33:0/1809793299 >> 172.21.15.33:0/1809793299 conn(0x7ff5080fe830 msgr2=0x7ff508100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 -- 172.21.15.33:0/1809793299 shutdown_connections 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.948+0000 7ff50ea5e640 1 -- 172.21.15.33:0/1809793299 wait complete. 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.949+0000 7ff50ea5e640 1 Processor -- start 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.949+0000 7ff50ea5e640 1 -- start start 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50ea5e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff508102de0 0x7ff5081992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50ea5e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5081087f0 0x7ff508199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50ea5e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508109170 0x7ff50819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50ea5e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff508112e30 con 0x7ff508102de0 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50ea5e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff508112cb0 con 0x7ff5081087f0 2024-09-10T21:07:56.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50ea5e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff508112fb0 con 0x7ff508109170 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508109170 0x7ff50819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff5077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5081087f0 0x7ff508199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508109170 0x7ff50819ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36064/0 (socket says 172.21.15.33:36064) 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff5077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5081087f0 0x7ff508199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55162/0 (socket says 172.21.15.33:55162) 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 -- 172.21.15.33:0/2617423077 learned_addr learned my addr 172.21.15.33:0/2617423077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff507fff640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff508102de0 0x7ff5081992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 -- 172.21.15.33:0/2617423077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5081087f0 msgr2=0x7ff508199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5081087f0 0x7ff508199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 -- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff508102de0 msgr2=0x7ff5081992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff508102de0 0x7ff5081992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 -- 172.21.15.33:0/2617423077 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50819f390 con 0x7ff508109170 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.950+0000 7ff50cffb640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508109170 0x7ff50819ebc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff50000eb10 tx=0x7ff50000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.951+0000 7ff5057fa640 1 -- 172.21.15.33:0/2617423077 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500004b20 con 0x7ff508109170 2024-09-10T21:07:56.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.951+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50819f680 con 0x7ff508109170 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.951+0000 7ff5057fa640 1 -- 172.21.15.33:0/2617423077 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff500004cc0 con 0x7ff508109170 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.951+0000 7ff5057fa640 1 -- 172.21.15.33:0/2617423077 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500010670 con 0x7ff508109170 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.951+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff508075380 con 0x7ff508109170 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.953+0000 7ff510e33740 1 Processor -- start 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.953+0000 7ff5057fa640 1 -- 172.21.15.33:0/2617423077 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff500010830 con 0x7ff508109170 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff510e33740 1 -- start start 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff510e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e2fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff510e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e309460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff510e33740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e30bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff510e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56405e273ea0 con 0x56405e2fda10 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff510e33740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56405e2632d0 con 0x56405e3099a0 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff510e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56405e2670e0 con 0x56405e2fd0f0 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e2fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e2fd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36074/0 (socket says 172.21.15.33:36074) 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 -- 172.21.15.33:0/2146263019 learned_addr learned my addr 172.21.15.33:0/2146263019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff507fff640 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e309460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e30bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 -- 172.21.15.33:0/2146263019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 msgr2=0x56405e30bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e30bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 -- 172.21.15.33:0/2146263019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 msgr2=0x56405e309460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e309460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff5077fe640 1 -- 172.21.15.33:0/2146263019 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405dfc1650 con 0x56405e2fd0f0 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.954+0000 7ff507fff640 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e309460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.955+0000 7ff5057fa640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff4f00b5360 0x7ff4f00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.955+0000 7ff5077fe640 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e2fd4d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc02a930 tx=0x7ff4fc005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8934d9aa34ab416c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:56.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.955+0000 7ff507fff640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff4f00b5360 0x7ff4f00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.955+0000 7ff5057fa640 1 -- 172.21.15.33:0/2617423077 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff5000e6150 con 0x7ff508109170 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.955+0000 7ff4ee7fc640 1 -- 172.21.15.33:0/2146263019 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc035030 con 0x56405e2fd0f0 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.955+0000 7ff4ee7fc640 1 -- 172.21.15.33:0/2146263019 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4fc004030 con 0x56405e2fd0f0 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.955+0000 7ff4ee7fc640 1 -- 172.21.15.33:0/2146263019 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0039b0 con 0x56405e2fd0f0 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 -- 172.21.15.33:0/2146263019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 msgr2=0x56405e2fd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff507fff640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff4f00b5360 0x7ff4f00b7820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x56405e2fbe00 tx=0x7ff4f8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e2fd4d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc02a930 tx=0x7ff4fc005b50 comp rx=0 tx=0).stop 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 -- 172.21.15.33:0/2146263019 shutdown_connections 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e30bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e309460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 --2- 172.21.15.33:0/2146263019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e2fd4d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 -- 172.21.15.33:0/2146263019 >> 172.21.15.33:0/2146263019 conn(0x56405e2faf20 msgr2=0x56405e2fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 -- 172.21.15.33:0/2146263019 shutdown_connections 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 -- 172.21.15.33:0/2146263019 wait complete. 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 -- start start 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.956+0000 7ff510e33740 1 Processor -- start 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff510e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fd0f0 0x56405e309310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff510e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fda10 0x56405e306760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff510e33740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e306de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff510e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56405e0c4700 con 0x56405e2fd0f0 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff510e33740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56405e1fe160 con 0x56405e3099a0 2024-09-10T21:07:56.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff510e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56405e30c5c0 con 0x56405e2fda10 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff510e33740 10 client.0 _ll_register_callbacks cb 0x56405e2179e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fd0f0 0x56405e309310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fd0f0 0x56405e309310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40814/0 (socket says 172.21.15.33:40814) 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff507fff640 1 -- 172.21.15.33:0/3120611122 learned_addr learned my addr 172.21.15.33:0/3120611122 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff50cffb640 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e306de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff5077fe640 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fda10 0x56405e306760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff50cffb640 1 -- 172.21.15.33:0/3120611122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fda10 msgr2=0x56405e306760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff50cffb640 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fda10 0x56405e306760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff50cffb640 1 -- 172.21.15.33:0/3120611122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fd0f0 msgr2=0x56405e309310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff50cffb640 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fd0f0 0x56405e309310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff50cffb640 1 -- 172.21.15.33:0/3120611122 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405e22bc90 con 0x56405e3099a0 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff50cffb640 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e306de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff5000a9b80 tx=0x7ff5000ac6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56df8ff983875d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.957+0000 7ff4e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:56.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3120611122 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000a8080 con 0x56405e3099a0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc004030 con 0x56405e3099a0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3120611122 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff5000b5c10 con 0x56405e3099a0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3120611122 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff5000c7050 con 0x56405e3099a0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 10 client.25655 fetch_fsmap learned FSMap version 37 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56405e317fd0 con 0x56405e3099a0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3120611122 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff500097030 con 0x56405e3099a0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3120611122 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7ff5000cf040 con 0x56405e3099a0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 10 client.25655 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 10 client.25655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 20 client.25655 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e319490 0x56405e31b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 4 client.25655 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.958+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56405e275df0 con 0x56405e319490 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.959+0000 7ff507fff640 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e319490 0x56405e31b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.959+0000 7ff507fff640 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e319490 0x56405e31b8a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3356aad9307b25af server_cookie=23fefe0363b19e16 in_seq=0 out_seq=0 2024-09-10T21:07:56.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.959+0000 7ff4e77fe640 10 client.25655 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3120611122 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56405e275df0 con 0x56405e319490 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff4e77fe640 10 client.25655 handle_command_reply: tid=0 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 client.25655 shutdown 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 msgr2=0x56405e306de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e306de0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff5000a9b80 tx=0x7ff5000ac6e0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 shutdown_connections 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e319490 0x56405e31b8a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e3099a0 0x56405e306de0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fda10 0x56405e306760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff4e77fe640 0 client.25655 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 --2- 172.21.15.33:0/3120611122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fd0f0 0x56405e309310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 >> 172.21.15.33:0/3120611122 conn(0x56405e2faf20 msgr2=0x56405e300ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 shutdown_connections 2024-09-10T21:07:56.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 1 -- 172.21.15.33:0/3120611122 wait complete. 2024-09-10T21:07:56.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.960+0000 7ff510e33740 20 client.25655 trim_cache size 0 max 16384 2024-09-10T21:07:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.974+0000 7ff510e33740 1 Processor -- start 2024-09-10T21:07:56.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.974+0000 7ff510e33740 1 -- start start 2024-09-10T21:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff510e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e3157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff510e33740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fda10 0x56405e3039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff510e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e3099a0 0x56405e303ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff510e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081b0650 con 0x56405e3099a0 2024-09-10T21:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff510e33740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081b2280 con 0x56405e2fda10 2024-09-10T21:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff510e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081b0d20 con 0x56405e2fd0f0 2024-09-10T21:07:56.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff50cffb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e3099a0 0x56405e303ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff5077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fda10 0x56405e3039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff5077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fda10 0x56405e3039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55184/0 (socket says 172.21.15.33:55184) 2024-09-10T21:07:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff50cffb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e3099a0 0x56405e303ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40816/0 (socket says 172.21.15.33:40816) 2024-09-10T21:07:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff5077fe640 1 -- 172.21.15.33:0/4236030893 learned_addr learned my addr 172.21.15.33:0/4236030893 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:56.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff507fff640 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e3157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff50cffb640 1 -- 172.21.15.33:0/4236030893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 msgr2=0x56405e3157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff50cffb640 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e3157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff50cffb640 1 -- 172.21.15.33:0/4236030893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fda10 msgr2=0x56405e3039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff50cffb640 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fda10 0x56405e3039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.975+0000 7ff50cffb640 1 -- 172.21.15.33:0/4236030893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56405e304c80 con 0x56405e3099a0 2024-09-10T21:07:56.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.976+0000 7ff50cffb640 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e3099a0 0x56405e303ef0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff5000abbc0 tx=0x7ff50000efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c17abea9bd30c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.976+0000 7ff4edffb640 1 -- 172.21.15.33:0/4236030893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000cb070 con 0x56405e3099a0 2024-09-10T21:07:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.976+0000 7ff4edffb640 1 -- 172.21.15.33:0/4236030893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff500097030 con 0x56405e3099a0 2024-09-10T21:07:56.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.976+0000 7ff4edffb640 1 -- 172.21.15.33:0/4236030893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5000b5c70 con 0x56405e3099a0 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 -- 172.21.15.33:0/4236030893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e3099a0 msgr2=0x56405e303ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e3099a0 0x56405e303ef0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff5000abbc0 tx=0x7ff50000efd0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 -- 172.21.15.33:0/4236030893 shutdown_connections 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e3099a0 0x56405e303ef0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fda10 0x56405e3039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 --2- 172.21.15.33:0/4236030893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e2fd0f0 0x56405e3157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 -- 172.21.15.33:0/4236030893 >> 172.21.15.33:0/4236030893 conn(0x56405e2faf20 msgr2=0x56405e3085b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:56.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 -- 172.21.15.33:0/4236030893 shutdown_connections 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 -- 172.21.15.33:0/4236030893 wait complete. 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.977+0000 7ff510e33740 1 -- start start 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 1 Processor -- start 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fd0f0 0x56405e311850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e311d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e3099a0 0x56405e3029a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081b1b80 con 0x56405e2fda10 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081b13f0 con 0x56405e2fd0f0 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081ae610 con 0x56405e3099a0 2024-09-10T21:07:56.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff507fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fd0f0 0x56405e311850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff510e33740 10 client.0 _ll_register_callbacks cb 0x7ff5000a9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff507fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fd0f0 0x56405e311850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55192/0 (socket says 172.21.15.33:55192) 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff50cffb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e3099a0 0x56405e3029a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff507fff640 1 -- 172.21.15.33:0/3104646061 learned_addr learned my addr 172.21.15.33:0/3104646061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff5077fe640 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e311d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff507fff640 1 -- 172.21.15.33:0/3104646061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e3099a0 msgr2=0x56405e3029a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff507fff640 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e3099a0 0x56405e3029a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.978+0000 7ff507fff640 1 -- 172.21.15.33:0/3104646061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 msgr2=0x56405e311d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff507fff640 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e311d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff507fff640 1 -- 172.21.15.33:0/3104646061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56405e302f10 con 0x56405e2fd0f0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff5077fe640 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e311d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff507fff640 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fd0f0 0x56405e311850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8012cb0 tx=0x7ff4f8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d531eefa2dc3185 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff4e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3104646061 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f801dd00 con 0x56405e2fd0f0 2024-09-10T21:07:56.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3104646061 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff4f8018030 con 0x56405e2fd0f0 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff500097030 con 0x56405e2fd0f0 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3104646061 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff4f8005020 con 0x56405e2fd0f0 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff510e33740 10 client.25661 fetch_fsmap learned FSMap version 37 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56405e3041f0 con 0x56405e2fd0f0 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.979+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3104646061 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff500097030 con 0x56405e2fd0f0 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3104646061 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7ff4f8039020 con 0x56405e2fd0f0 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff510e33740 10 client.25661 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff510e33740 10 client.25661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff510e33740 20 client.25661 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff510e33740 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e387380 0x56405e389770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff510e33740 4 client.25661 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56405e2718a0 con 0x56405e387380 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff5077fe640 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e387380 0x56405e389770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:56.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff5077fe640 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e387380 0x56405e389770 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a7b6f8691fdd80 server_cookie=5dee711c2ad073b in_seq=0 out_seq=0 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.980+0000 7ff4e77fe640 10 client.25661 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff4e77fe640 1 -- 172.21.15.33:0/3104646061 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56405e2718a0 con 0x56405e387380 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff4e77fe640 10 client.25661 handle_command_reply: tid=0 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 client.25661 shutdown 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fd0f0 msgr2=0x56405e311850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fd0f0 0x56405e311850 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8012cb0 tx=0x7ff4f8017000 comp rx=0 tx=0).stop 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 shutdown_connections 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56405e387380 0x56405e389770 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405e3099a0 0x56405e3029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56405e2fda10 0x56405e311d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff4e77fe640 0 client.25661 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 --2- 172.21.15.33:0/3104646061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56405e2fd0f0 0x56405e311850 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.981+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 >> 172.21.15.33:0/3104646061 conn(0x56405e2faf20 msgr2=0x56405e2f9750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:56.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 shutdown_connections 2024-09-10T21:07:56.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff510e33740 1 -- 172.21.15.33:0/3104646061 wait complete. 2024-09-10T21:07:56.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff510e33740 20 client.25661 trim_cache size 0 max 16384 2024-09-10T21:07:56.992 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff4f00b5360 msgr2=0x7ff4f00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff4f00b5360 0x7ff4f00b7820 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x56405e2fbe00 tx=0x7ff4f8005f90 comp rx=0 tx=0).stop 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508109170 msgr2=0x7ff50819ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508109170 0x7ff50819ebc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff50000eb10 tx=0x7ff50000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 shutdown_connections 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff4f00b5360 0x7ff4f00b7820 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff508109170 0x7ff50819ebc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5081087f0 0x7ff508199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 --2- 172.21.15.33:0/2617423077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff508102de0 0x7ff5081992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 >> 172.21.15.33:0/2617423077 conn(0x7ff5080fe830 msgr2=0x7ff508100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.982+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 shutdown_connections 2024-09-10T21:07:56.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:56.983+0000 7ff50ea5e640 1 -- 172.21.15.33:0/2617423077 wait complete. 2024-09-10T21:07:57.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 354912 10.54 MB/sec execute 58 sec latency 9.876 ms 2024-09-10T21:07:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:57 smithi104 ceph-mon[35325]: pgmap v465: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-10T21:07:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:57 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:57 smithi174 ceph-mon[35635]: pgmap v465: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-10T21:07:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:57 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:57.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:57 smithi033 ceph-mon[30860]: pgmap v465: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.7 MiB/s wr, 306 op/s 2024-09-10T21:07:57.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:57 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:07:58.690 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:07:58.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:07:58.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 356615 10.46 MB/sec execute 59 sec latency 13.658 ms 2024-09-10T21:07:58.970 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:07:59.604 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:07:59 smithi033 ceph-mon[30860]: pgmap v466: 97 pgs: 97 active+clean; 149 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-10T21:07:59.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 -- 172.21.15.33:0/1036350952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea8103ca0 msgr2=0x7faea8104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 --2- 172.21.15.33:0/1036350952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea8103ca0 0x7faea8104080 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faea4009900 tx=0x7faea402a410 comp rx=0 tx=0).stop 2024-09-10T21:07:59.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 -- 172.21.15.33:0/1036350952 shutdown_connections 2024-09-10T21:07:59.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 --2- 172.21.15.33:0/1036350952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea810d100 0x7faea810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 --2- 172.21.15.33:0/1036350952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea81045c0 0x7faea810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 --2- 172.21.15.33:0/1036350952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea8103ca0 0x7faea8104080 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 -- 172.21.15.33:0/1036350952 >> 172.21.15.33:0/1036350952 conn(0x7faea80fd700 msgr2=0x7faea80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 -- 172.21.15.33:0/1036350952 shutdown_connections 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.593+0000 7faeaf78a640 1 -- 172.21.15.33:0/1036350952 wait complete. 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 Processor -- start 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 -- start start 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea8103ca0 0x7faea8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea81045c0 0x7faea81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea810d100 0x7faea819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faea8112d70 con 0x7faea81045c0 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faea8112bf0 con 0x7faea810d100 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeaf78a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faea8112ef0 con 0x7faea8103ca0 2024-09-10T21:07:59.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeacd25640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea81045c0 0x7faea81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faead526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea8103ca0 0x7faea8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeacd25640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea81045c0 0x7faea81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48050/0 (socket says 172.21.15.33:48050) 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.594+0000 7faeacd25640 1 -- 172.21.15.33:0/938528082 learned_addr learned my addr 172.21.15.33:0/938528082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeadd27640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea810d100 0x7faea819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeacd25640 1 -- 172.21.15.33:0/938528082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea8103ca0 msgr2=0x7faea8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeacd25640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea8103ca0 0x7faea8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeacd25640 1 -- 172.21.15.33:0/938528082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea810d100 msgr2=0x7faea819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeacd25640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea810d100 0x7faea819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeacd25640 1 -- 172.21.15.33:0/938528082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea819f330 con 0x7faea81045c0 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeadd27640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea810d100 0x7faea819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:07:59.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeacd25640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea81045c0 0x7faea81997c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9c0 tx=0x7fae9800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7fae9e7fc640 1 -- 172.21.15.33:0/938528082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9800ccc0 con 0x7faea81045c0 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea819f620 con 0x7faea81045c0 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7fae9e7fc640 1 -- 172.21.15.33:0/938528082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae980035f0 con 0x7faea81045c0 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7fae9e7fc640 1 -- 172.21.15.33:0/938528082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9800ce60 con 0x7faea81045c0 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.595+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81a5eb0 con 0x7faea81045c0 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 Processor -- start 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7fae9e7fc640 1 -- 172.21.15.33:0/938528082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fae98004000 con 0x7faea81045c0 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 -- start start 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d563250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d565b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55721d4cde60 con 0x55721d556ee0 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55721d4bd290 con 0x55721d563790 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55721d4c10a0 con 0x55721d557800 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeadd27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faead526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d563250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faead526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d563250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34220/0 (socket says 172.21.15.33:34220) 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faead526640 1 -- 172.21.15.33:0/3159619091 learned_addr learned my addr 172.21.15.33:0/3159619091 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.598+0000 7faeacd25640 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d565b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7faeadd27640 1 -- 172.21.15.33:0/3159619091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 msgr2=0x55721d563250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7faeadd27640 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d563250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7faeadd27640 1 -- 172.21.15.33:0/3159619091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 msgr2=0x55721d565b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7faeadd27640 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d565b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7faeadd27640 1 -- 172.21.15.33:0/3159619091 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721d21b930 con 0x55721d556ee0 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7faeadd27640 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5572c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faea0009ef0 tx=0x7faea000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba267c69811d04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7fae9e7fc640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fae940b52e0 0x7fae940b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7fae937fe640 1 -- 172.21.15.33:0/3159619091 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea0003b70 con 0x55721d556ee0 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7fae937fe640 1 -- 172.21.15.33:0/3159619091 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea0003d10 con 0x55721d556ee0 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7fae937fe640 1 -- 172.21.15.33:0/3159619091 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea00118e0 con 0x55721d556ee0 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.599+0000 7fae9e7fc640 1 -- 172.21.15.33:0/938528082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fae980e5f20 con 0x7faea81045c0 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.600+0000 7faead526640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fae940b52e0 0x7fae940b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.600+0000 7faeb1b5f740 1 -- 172.21.15.33:0/3159619091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 msgr2=0x55721d5572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.600+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5572c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7faea0009ef0 tx=0x7faea000f730 comp rx=0 tx=0).stop 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 -- 172.21.15.33:0/3159619091 shutdown_connections 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d565b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d563250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/3159619091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5572c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 -- 172.21.15.33:0/3159619091 >> 172.21.15.33:0/3159619091 conn(0x55721d554d10 msgr2=0x55721d555eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faead526640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fae940b52e0 0x7fae940b77a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faea4005c00 tx=0x7faea4005b90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 -- 172.21.15.33:0/3159619091 shutdown_connections 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 -- 172.21.15.33:0/3159619091 wait complete. 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 -- start start 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.601+0000 7faeb1b5f740 1 Processor -- start 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.602+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.602+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d560ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.602+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d563790 0x55721d561220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.602+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55721d31e890 con 0x55721d556ee0 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.602+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55721d457d40 con 0x55721d557800 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.602+0000 7faeacd25640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d560ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.602+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55721d566440 con 0x55721d563790 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d560ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36820/0 (socket says 172.21.15.33:36820) 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 -- 172.21.15.33:0/2006937680 learned_addr learned my addr 172.21.15.33:0/2006937680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faead526640 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeb1b5f740 10 client.0 _ll_register_callbacks cb 0x55721d4bbc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 -- 172.21.15.33:0/2006937680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d563790 msgr2=0x55721d561220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d563790 0x55721d561220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 -- 172.21.15.33:0/2006937680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 msgr2=0x55721d5607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 -- 172.21.15.33:0/2006937680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721d56f8a0 con 0x55721d557800 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea0003d10 con 0x55721d557800 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faeacd25640 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d560ce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae980ac670 tx=0x7fae980a8df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=214bbf4de2ca9a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7faead526640 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.603+0000 7fae90ff9640 10 client.25670 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7fae90ff9640 1 -- 172.21.15.33:0/2006937680 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980abe50 con 0x55721d557800 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7fae90ff9640 1 -- 172.21.15.33:0/2006937680 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fae98004320 con 0x55721d557800 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7fae90ff9640 1 -- 172.21.15.33:0/2006937680 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fae980ac1f0 con 0x55721d557800 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 10 client.25670 fetch_fsmap learned FSMap version 37 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55721d57aa00 con 0x55721d557800 2024-09-10T21:07:59.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7fae90ff9640 1 -- 172.21.15.33:0/2006937680 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fae980b57e0 con 0x55721d557800 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 10 client.25670 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 10 client.25670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 20 client.25670 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d57be00 0x55721d57e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 4 client.25670 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55721d4cfdb0 con 0x55721d57be00 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.604+0000 7faead526640 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d57be00 0x55721d57e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.605+0000 7fae90ff9640 10 client.25670 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.605+0000 7faead526640 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d57be00 0x55721d57e210 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=690b673db3220445 server_cookie=6f7f902801a5de2d in_seq=0 out_seq=0 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.605+0000 7fae90ff9640 1 -- 172.21.15.33:0/2006937680 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55721d4cfdb0 con 0x55721d57be00 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.605+0000 7fae90ff9640 10 client.25670 handle_command_reply: tid=0 2024-09-10T21:07:59.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 client.25670 shutdown 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 msgr2=0x55721d560ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d560ce0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae980ac670 tx=0x7fae980a8df0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 shutdown_connections 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d57be00 0x55721d57e210 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d563790 0x55721d561220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d560ce0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2006937680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d5607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 >> 172.21.15.33:0/2006937680 conn(0x55721d554d10 msgr2=0x55721d553f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7fae90ff9640 0 client.25670 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 shutdown_connections 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2006937680 wait complete. 2024-09-10T21:07:59.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.606+0000 7faeb1b5f740 20 client.25670 trim_cache size 0 max 16384 2024-09-10T21:07:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.627+0000 7faeb1b5f740 1 Processor -- start 2024-09-10T21:07:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.627+0000 7faeb1b5f740 1 -- start start 2024-09-10T21:07:59.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.627+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d556ee0 0x55721d55a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d55d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d563790 0x55721d55df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ac210 con 0x55721d563790 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ade40 con 0x55721d557800 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ac8e0 con 0x55721d556ee0 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d556ee0 0x55721d55a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d556ee0 0x55721d55a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34246/0 (socket says 172.21.15.33:34246) 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 -- 172.21.15.33:0/2121010896 learned_addr learned my addr 172.21.15.33:0/2121010896 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeadd27640 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d563790 0x55721d55df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeacd25640 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d55d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 -- 172.21.15.33:0/2121010896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 msgr2=0x55721d55d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d55d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 -- 172.21.15.33:0/2121010896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d563790 msgr2=0x55721d55df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d563790 0x55721d55df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 -- 172.21.15.33:0/2121010896 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55721d554b40 con 0x55721d556ee0 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faeadd27640 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d563790 0x55721d55df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.628+0000 7faead526640 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d556ee0 0x55721d55a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faea402a930 tx=0x7faea403f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d3a425da2cf63ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.629+0000 7fae92ffd640 1 -- 172.21.15.33:0/2121010896 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4056070 con 0x55721d556ee0 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.629+0000 7fae92ffd640 1 -- 172.21.15.33:0/2121010896 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea4005040 con 0x55721d556ee0 2024-09-10T21:07:59.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.629+0000 7fae92ffd640 1 -- 172.21.15.33:0/2121010896 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4035040 con 0x55721d556ee0 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.629+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2121010896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d556ee0 msgr2=0x55721d55a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.629+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d556ee0 0x55721d55a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faea402a930 tx=0x7faea403f000 comp rx=0 tx=0).stop 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2121010896 shutdown_connections 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d563790 0x55721d55df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d557800 0x55721d55d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2121010896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d556ee0 0x55721d55a7a0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2121010896 >> 172.21.15.33:0/2121010896 conn(0x55721d554d10 msgr2=0x55721d55a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2121010896 shutdown_connections 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2121010896 wait complete. 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- start start 2024-09-10T21:07:59.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 Processor -- start 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d56bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d5686d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d568c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faea81ad740 con 0x55721d556ee0 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faea81acfb0 con 0x55721d563790 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.630+0000 7faeb1b5f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faea81aa1d0 con 0x55721d557800 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faeb1b5f740 10 client.0 _ll_register_callbacks cb 0x7fae980e9690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d56bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d56bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48084/0 (socket says 172.21.15.33:48084) 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 -- 172.21.15.33:0/2447267144 learned_addr learned my addr 172.21.15.33:0/2447267144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faeadd27640 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d568c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faeacd25640 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d5686d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 -- 172.21.15.33:0/2447267144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 msgr2=0x55721d5686d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d5686d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 -- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 msgr2=0x55721d568c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d568c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 -- 172.21.15.33:0/2447267144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55721d5691a0 con 0x55721d556ee0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faead526640 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d56bc10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faea4054040 tx=0x7faea40054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb3cc466b8144806 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7fae90ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7fae90ff9640 1 -- 172.21.15.33:0/2447267144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea4056040 con 0x55721d556ee0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faea4005040 con 0x55721d556ee0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.631+0000 7fae90ff9640 1 -- 172.21.15.33:0/2447267144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7faea4004750 con 0x55721d556ee0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7fae90ff9640 1 -- 172.21.15.33:0/2447267144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7faea4004990 con 0x55721d556ee0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 10 client.16113 fetch_fsmap learned FSMap version 37 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55721d5e9af0 con 0x55721d556ee0 2024-09-10T21:07:59.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7fae90ff9640 1 -- 172.21.15.33:0/2447267144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faea4005040 con 0x55721d556ee0 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7fae90ff9640 1 -- 172.21.15.33:0/2447267144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7faea4062030 con 0x55721d556ee0 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 10 client.16113 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 20 client.16113 populate_metadata read hostname 'smithi033' 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d5ea1e0 0x55721d5ec600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 4 client.16113 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55721d4cb860 con 0x55721d5ea1e0 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.632+0000 7faeacd25640 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d5ea1e0 0x55721d5ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.633+0000 7fae90ff9640 10 client.16113 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:59.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.633+0000 7faeacd25640 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d5ea1e0 0x55721d5ec600 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d37a8a06d62461c server_cookie=4a9eed14e5439a36 in_seq=0 out_seq=0 2024-09-10T21:07:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7fae90ff9640 1 -- 172.21.15.33:0/2447267144 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55721d4cb860 con 0x55721d5ea1e0 2024-09-10T21:07:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7fae90ff9640 10 client.16113 handle_command_reply: tid=0 2024-09-10T21:07:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 client.16113 shutdown 2024-09-10T21:07:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 msgr2=0x55721d56bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d56bc10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7faea4054040 tx=0x7faea40054a0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 shutdown_connections 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55721d5ea1e0 0x55721d5ec600 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55721d563790 0x55721d568c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7fae90ff9640 0 client.16113 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55721d557800 0x55721d5686d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 --2- 172.21.15.33:0/2447267144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55721d556ee0 0x55721d56bc10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.634+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 >> 172.21.15.33:0/2447267144 conn(0x55721d554d10 msgr2=0x55721d560ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 shutdown_connections 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeb1b5f740 1 -- 172.21.15.33:0/2447267144 wait complete. 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeb1b5f740 20 client.16113 trim_cache size 0 max 16384 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:07:59.642 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:07:59.643 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0512e745-916d-4819-9e47-d7b895d38518", 2024-09-10T21:07:59.643 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:07:59.643 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:07:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fae940b52e0 msgr2=0x7fae940b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fae940b52e0 0x7fae940b77a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faea4005c00 tx=0x7faea4005b90 comp rx=0 tx=0).stop 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea81045c0 msgr2=0x7faea81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea81045c0 0x7faea81997c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fae9800e9c0 tx=0x7fae9800ee80 comp rx=0 tx=0).stop 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 shutdown_connections 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fae940b52e0 0x7fae940b77a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faea810d100 0x7faea819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faea81045c0 0x7faea81997c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 --2- 172.21.15.33:0/938528082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea8103ca0 0x7faea8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 >> 172.21.15.33:0/938528082 conn(0x7faea80fd700 msgr2=0x7faea80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 shutdown_connections 2024-09-10T21:07:59.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:07:59.635+0000 7faeaf78a640 1 -- 172.21.15.33:0/938528082 wait complete. 2024-09-10T21:07:59.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 358553 10.51 MB/sec execute 60 sec latency 33.715 ms 2024-09-10T21:07:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:07:59 smithi104 ceph-mon[35325]: pgmap v466: 97 pgs: 97 active+clean; 149 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-10T21:07:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:07:59 smithi174 ceph-mon[35635]: pgmap v466: 97 pgs: 97 active+clean; 149 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 357 op/s 2024-09-10T21:08:00.543 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:08:00.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 360698 10.46 MB/sec execute 61 sec latency 24.179 ms 2024-09-10T21:08:00.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:08:00.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.864 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:00.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:00.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:00.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:00.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:00.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:00.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 -- 172.21.15.33:0/313391481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c1043b0 msgr2=0x7fb14c10c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 --2- 172.21.15.33:0/313391481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c1043b0 0x7fb14c10c990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb140009900 tx=0x7fb14002a410 comp rx=0 tx=0).stop 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 -- 172.21.15.33:0/313391481 shutdown_connections 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 --2- 172.21.15.33:0/313391481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c10cf60 0x7fb14c10f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 --2- 172.21.15.33:0/313391481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c1043b0 0x7fb14c10c990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 --2- 172.21.15.33:0/313391481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c1039b0 0x7fb14c103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 -- 172.21.15.33:0/313391481 >> 172.21.15.33:0/313391481 conn(0x7fb14c0fd560 msgr2=0x7fb14c0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 -- 172.21.15.33:0/313391481 shutdown_connections 2024-09-10T21:08:01.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.345+0000 7fb150ada640 1 -- 172.21.15.33:0/313391481 wait complete. 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 Processor -- start 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 -- start start 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c1039b0 0x7fb14c19b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c1043b0 0x7fb14c19b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c10cf60 0x7fb14c1a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112bc0 con 0x7fb14c10cf60 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112a40 con 0x7fb14c1043b0 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb150ada640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c112d40 con 0x7fb14c1039b0 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb14b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c1039b0 0x7fb14c19b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c10cf60 0x7fb14c1a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb14affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c1043b0 0x7fb14c19b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb14b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c1039b0 0x7fb14c19b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34290/0 (socket says 172.21.15.33:34290) 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb14b7fe640 1 -- 172.21.15.33:0/2520578302 learned_addr learned my addr 172.21.15.33:0/2520578302 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.346+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c10cf60 0x7fb14c1a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48098/0 (socket says 172.21.15.33:48098) 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb14b7fe640 1 -- 172.21.15.33:0/2520578302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c1043b0 msgr2=0x7fb14c19b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb14b7fe640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c1043b0 0x7fb14c19b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb14b7fe640 1 -- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c10cf60 msgr2=0x7fb14c1a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb14b7fe640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c10cf60 0x7fb14c1a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb14b7fe640 1 -- 172.21.15.33:0/2520578302 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c1a12c0 con 0x7fb14c1039b0 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb14bfff640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c10cf60 0x7fb14c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb14b7fe640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c1039b0 0x7fb14c19b210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00e9f0 tx=0x7fb13c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb148ff9640 1 -- 172.21.15.33:0/2520578302 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00cd80 con 0x7fb14c1039b0 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb148ff9640 1 -- 172.21.15.33:0/2520578302 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13c0108e0 con 0x7fb14c1039b0 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1a15b0 con 0x7fb14c1039b0 2024-09-10T21:08:01.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb148ff9640 1 -- 172.21.15.33:0/2520578302 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c018980 con 0x7fb14c1039b0 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.347+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a7e40 con 0x7fb14c1039b0 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.350+0000 7fb148ff9640 1 -- 172.21.15.33:0/2520578302 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb13c018b20 con 0x7fb14c1039b0 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.350+0000 7fb154111740 1 Processor -- start 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb154111740 1 -- start start 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb154111740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a83660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb154111740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb154111740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a91f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb154111740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d1769f9e50 con 0x55d176a83ba0 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb154111740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d1769e9280 con 0x55d176a83280 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb154111740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d1769ed090 con 0x55d176a8fb30 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a83660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a83660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36864/0 (socket says 172.21.15.33:36864) 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 -- 172.21.15.33:0/1617816315 learned_addr learned my addr 172.21.15.33:0/1617816315 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14b7fe640 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14affd640 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a91f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 -- 172.21.15.33:0/1617816315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 msgr2=0x55d176a91f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a91f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 -- 172.21.15.33:0/1617816315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 msgr2=0x55d176a8f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14bfff640 1 -- 172.21.15.33:0/1617816315 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d176747650 con 0x55d176a83280 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.351+0000 7fb14b7fe640 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.352+0000 7fb14bfff640 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a83660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb1440117e0 tx=0x7fb144011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe85afa2fb890554 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.352+0000 7fb148ff9640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1340b5410 0x7fb1340b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.352+0000 7fb14b7fe640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1340b5410 0x7fb1340b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.352+0000 7fb131ffb640 1 -- 172.21.15.33:0/1617816315 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb144011310 con 0x55d176a83280 2024-09-10T21:08:01.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.352+0000 7fb131ffb640 1 -- 172.21.15.33:0/1617816315 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb144003e20 con 0x55d176a83280 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.352+0000 7fb131ffb640 1 -- 172.21.15.33:0/1617816315 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1440044f0 con 0x55d176a83280 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.352+0000 7fb148ff9640 1 -- 172.21.15.33:0/2520578302 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb13c0e5eb0 con 0x7fb14c1039b0 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb14b7fe640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1340b5410 0x7fb1340b78d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0ac420 tx=0x7fb13c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 -- 172.21.15.33:0/1617816315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 msgr2=0x55d176a83660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a83660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb1440117e0 tx=0x7fb144011ca0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 -- 172.21.15.33:0/1617816315 shutdown_connections 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a91f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 --2- 172.21.15.33:0/1617816315 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a83660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 -- 172.21.15.33:0/1617816315 >> 172.21.15.33:0/1617816315 conn(0x55d176a810b0 msgr2=0x55d176a82250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 -- 172.21.15.33:0/1617816315 shutdown_connections 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.353+0000 7fb154111740 1 -- 172.21.15.33:0/1617816315 wait complete. 2024-09-10T21:08:01.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.354+0000 7fb154111740 1 -- start start 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.354+0000 7fb154111740 1 Processor -- start 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb154111740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83280 0x55d176a8f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb154111740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83ba0 0x55d176a8c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb154111740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a8cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb154111740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d17684a700 con 0x55d176a83280 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb154111740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d176984110 con 0x55d176a83ba0 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb154111740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d176a927b0 con 0x55d176a8fb30 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb154111740 10 client.0 _ll_register_callbacks cb 0x55d17699d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83ba0 0x55d176a8c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83280 0x55d176a8f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83ba0 0x55d176a8c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36878/0 (socket says 172.21.15.33:36878) 2024-09-10T21:08:01.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 -- 172.21.15.33:0/3085387952 learned_addr learned my addr 172.21.15.33:0/3085387952 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14bfff640 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a8cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 -- 172.21.15.33:0/3085387952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 msgr2=0x55d176a8cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a8cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 -- 172.21.15.33:0/3085387952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83280 msgr2=0x55d176a8f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83280 0x55d176a8f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 -- 172.21.15.33:0/3085387952 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1769b1c40 con 0x55d176a83ba0 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14bfff640 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a8cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb14affd640 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83ba0 0x55d176a8c8b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb14002a930 tx=0x7fb140032f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb4e374faaecba72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb12affd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb12affd640 1 -- 172.21.15.33:0/3085387952 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140035040 con 0x55d176a83ba0 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.355+0000 7fb12affd640 1 -- 172.21.15.33:0/3085387952 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb140031720 con 0x55d176a83ba0 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb144003e20 con 0x55d176a83ba0 2024-09-10T21:08:01.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb12affd640 1 -- 172.21.15.33:0/3085387952 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb140031920 con 0x55d176a83ba0 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 10 client.25688 fetch_fsmap learned FSMap version 37 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d176a9e150 con 0x55d176a83ba0 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb12affd640 1 -- 172.21.15.33:0/3085387952 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb144003e20 con 0x55d176a83ba0 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb12affd640 1 -- 172.21.15.33:0/3085387952 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fb140004750 con 0x55d176a83ba0 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 10 client.25688 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 10 client.25688 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 20 client.25688 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176a9f610 0x55d176aa1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 4 client.25688 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1769fbda0 con 0x55d176a9f610 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.356+0000 7fb14bfff640 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176a9f610 0x55d176aa1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.357+0000 7fb14bfff640 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176a9f610 0x55d176aa1a60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6554f6bd44d3b9d1 server_cookie=dbd5b244e60248ee in_seq=0 out_seq=0 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.357+0000 7fb12affd640 10 client.25688 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:01.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.357+0000 7fb12affd640 1 -- 172.21.15.33:0/3085387952 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d1769fbda0 con 0x55d176a9f610 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.357+0000 7fb12affd640 10 client.25688 handle_command_reply: tid=0 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.357+0000 7fb154111740 1 client.25688 shutdown 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83ba0 msgr2=0x55d176a8c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83ba0 0x55d176a8c8b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb14002a930 tx=0x7fb140032f00 comp rx=0 tx=0).stop 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 shutdown_connections 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176a9f610 0x55d176aa1a60 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a8cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb12affd640 0 client.25688 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83ba0 0x55d176a8c8b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 --2- 172.21.15.33:0/3085387952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83280 0x55d176a8f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 >> 172.21.15.33:0/3085387952 conn(0x55d176a810b0 msgr2=0x55d176a86c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 shutdown_connections 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 1 -- 172.21.15.33:0/3085387952 wait complete. 2024-09-10T21:08:01.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.358+0000 7fb154111740 20 client.25688 trim_cache size 0 max 16384 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.378+0000 7fb154111740 1 Processor -- start 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.378+0000 7fb154111740 1 -- start start 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb154111740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a83280 0x55d176a8b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb154111740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb154111740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a8fb30 0x55d176a82170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb154111740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1b0390 con 0x55d176a83ba0 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb154111740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1b1fc0 con 0x55d176a8fb30 2024-09-10T21:08:01.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb154111740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1b0a60 con 0x55d176a83280 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48126/0 (socket says 172.21.15.33:48126) 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a8fb30 0x55d176a82170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 -- 172.21.15.33:0/1122685418 learned_addr learned my addr 172.21.15.33:0/1122685418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14bfff640 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a83280 0x55d176a8b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 -- 172.21.15.33:0/1122685418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a83280 msgr2=0x55d176a8b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a83280 0x55d176a8b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 -- 172.21.15.33:0/1122685418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a8fb30 msgr2=0x55d176a82170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a8fb30 0x55d176a82170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14affd640 1 -- 172.21.15.33:0/1122685418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d176a839d0 con 0x55d176a83ba0 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.379+0000 7fb14bfff640 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a8fb30 0x55d176a82170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.380+0000 7fb14affd640 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8b670 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb140006010 tx=0x7fb140004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5533e5882f7690f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:01.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.380+0000 7fb1317fa640 1 -- 172.21.15.33:0/1122685418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140035030 con 0x55d176a83ba0 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.380+0000 7fb1317fa640 1 -- 172.21.15.33:0/1122685418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1400316a0 con 0x55d176a83ba0 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.380+0000 7fb1317fa640 1 -- 172.21.15.33:0/1122685418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb140046470 con 0x55d176a83ba0 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 -- 172.21.15.33:0/1122685418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 msgr2=0x55d176a8b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8b670 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb140006010 tx=0x7fb140004940 comp rx=0 tx=0).stop 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 -- 172.21.15.33:0/1122685418 shutdown_connections 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a8fb30 0x55d176a82170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a8b670 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 --2- 172.21.15.33:0/1122685418 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a83280 0x55d176a8b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 -- 172.21.15.33:0/1122685418 >> 172.21.15.33:0/1122685418 conn(0x55d176a810b0 msgr2=0x55d176a7c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 -- 172.21.15.33:0/1122685418 shutdown_connections 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 -- 172.21.15.33:0/1122685418 wait complete. 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.381+0000 7fb154111740 1 -- start start 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 1 Processor -- start 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a97c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a981a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a92a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1b18c0 con 0x55d176a83ba0 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1b1130 con 0x55d176a83280 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1ae350 con 0x55d176a8fb30 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb154111740 10 client.0 _ll_register_callbacks cb 0x7fb140009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb14affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a92a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb14bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a92a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34332/0 (socket says 172.21.15.33:34332) 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb14affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48130/0 (socket says 172.21.15.33:48130) 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb14bfff640 1 -- 172.21.15.33:0/69760431 learned_addr learned my addr 172.21.15.33:0/69760431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.382+0000 7fb14bfff640 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a97c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb14bfff640 1 -- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 msgr2=0x55d176a97c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb14bfff640 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a97c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb14bfff640 1 -- 172.21.15.33:0/69760431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 msgr2=0x55d176a981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb14bfff640 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb14bfff640 1 -- 172.21.15.33:0/69760431 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d176a93070 con 0x55d176a8fb30 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb14affd640 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb14bfff640 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a92a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1440117b0 tx=0x7fb140032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ca111bff2a522fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb12affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb12affd640 1 -- 172.21.15.33:0/69760431 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb14400fba0 con 0x55d176a8fb30 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb12affd640 1 -- 172.21.15.33:0/69760431 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb14400fd40 con 0x55d176a8fb30 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1400316a0 con 0x55d176a8fb30 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb12affd640 1 -- 172.21.15.33:0/69760431 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb144003f20 con 0x55d176a8fb30 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb154111740 10 client.25678 fetch_fsmap learned FSMap version 37 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.383+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d176b0cf10 con 0x55d176a8fb30 2024-09-10T21:08:01.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb12affd640 1 -- 172.21.15.33:0/69760431 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1400316a0 con 0x55d176a8fb30 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb12affd640 1 -- 172.21.15.33:0/69760431 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fb144028040 con 0x55d176a8fb30 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb154111740 10 client.25678 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb154111740 10 client.25678 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb154111740 20 client.25678 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb154111740 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176b0d4b0 0x55d176b0f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb154111740 4 client.25678 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d1769f7850 con 0x55d176b0d4b0 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.384+0000 7fb14affd640 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176b0d4b0 0x55d176b0f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb14affd640 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176b0d4b0 0x55d176b0f8d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bea2f79e7fd858d server_cookie=84ef92cf0562b0bd in_seq=0 out_seq=0 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb12affd640 10 client.25678 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb12affd640 1 -- 172.21.15.33:0/69760431 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x55d1769f7850 con 0x55d176b0d4b0 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb12affd640 10 client.25678 handle_command_reply: tid=0 2024-09-10T21:08:01.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 client.25678 shutdown 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 msgr2=0x55d176a92a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a92a30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb1440117b0 tx=0x7fb140032b20 comp rx=0 tx=0).stop 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 shutdown_connections 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d176b0d4b0 0x55d176b0f8d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d176a8fb30 0x55d176a92a30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d176a83ba0 0x55d176a981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb12affd640 0 client.25678 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 --2- 172.21.15.33:0/69760431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d176a83280 0x55d176a97c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 >> 172.21.15.33:0/69760431 conn(0x55d176a810b0 msgr2=0x55d176a82150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 shutdown_connections 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 1 -- 172.21.15.33:0/69760431 wait complete. 2024-09-10T21:08:01.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.385+0000 7fb154111740 20 client.25678 trim_cache size 0 max 16384 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0512e745-916d-4819-9e47-d7b895d38518", 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:08:01.400 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:01.401 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:01.402 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:08:01.403 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:01.404 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1340b5410 msgr2=0x7fb1340b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1340b5410 0x7fb1340b78d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb13c0ac420 tx=0x7fb13c000bc0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c1039b0 msgr2=0x7fb14c19b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c1039b0 0x7fb14c19b210 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00e9f0 tx=0x7fb13c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 shutdown_connections 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb1340b5410 0x7fb1340b78d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb14c10cf60 0x7fb14c1a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb14c1043b0 0x7fb14c19b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 --2- 172.21.15.33:0/2520578302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb14c1039b0 0x7fb14c19b210 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:01.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 >> 172.21.15.33:0/2520578302 conn(0x7fb14c0fd560 msgr2=0x7fb14c10f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:01.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 shutdown_connections 2024-09-10T21:08:01.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:01.386+0000 7fb150ada640 1 -- 172.21.15.33:0/2520578302 wait complete. 2024-09-10T21:08:01.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 362318 10.48 MB/sec execute 62 sec latency 25.267 ms 2024-09-10T21:08:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:01 smithi104 ceph-mon[35325]: pgmap v467: 97 pgs: 97 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-10T21:08:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:01 smithi174 ceph-mon[35635]: pgmap v467: 97 pgs: 97 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-10T21:08:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:01 smithi033 ceph-mon[30860]: pgmap v467: 97 pgs: 97 active+clean; 150 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-10T21:08:02.203 INFO:tasks.cephfs.filesystem:scrub status for tag:0512e745-916d-4819-9e47-d7b895d38518 - {'path': '/', 'tag': '0512e745-916d-4819-9e47-d7b895d38518', 'options': 'recursive,force'} 2024-09-10T21:08:02.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 364782 10.52 MB/sec execute 63 sec latency 14.426 ms 2024-09-10T21:08:02.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:02 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:02 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:02 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:03.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 366318 10.47 MB/sec execute 64 sec latency 10.258 ms 2024-09-10T21:08:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:03 smithi174 ceph-mon[35635]: pgmap v468: 97 pgs: 97 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-10T21:08:03.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:03 smithi033 ceph-mon[30860]: pgmap v468: 97 pgs: 97 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-10T21:08:04.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:03 smithi104 ceph-mon[35325]: pgmap v468: 97 pgs: 97 active+clean; 153 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 326 op/s 2024-09-10T21:08:04.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 368828 10.56 MB/sec execute 65 sec latency 13.393 ms 2024-09-10T21:08:05.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 370104 10.46 MB/sec execute 66 sec latency 36.537 ms 2024-09-10T21:08:05.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:05 smithi174 ceph-mon[35635]: pgmap v469: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-09-10T21:08:05.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:05 smithi033 ceph-mon[30860]: pgmap v469: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-09-10T21:08:06.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:05 smithi104 ceph-mon[35325]: pgmap v469: 97 pgs: 97 active+clean; 155 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-09-10T21:08:06.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 372673 10.55 MB/sec execute 67 sec latency 18.443 ms 2024-09-10T21:08:06.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:06.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:06.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:06.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:06.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:06.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:06.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:06.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:07.720 INFO:tasks.workunit.client.0.smithi033.stdout: 1 374410 10.48 MB/sec execute 68 sec latency 11.639 ms 2024-09-10T21:08:07.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:07 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:07.828 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:07 smithi033 ceph-mon[30860]: pgmap v470: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 315 op/s 2024-09-10T21:08:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:07 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:07 smithi174 ceph-mon[35635]: pgmap v470: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 315 op/s 2024-09-10T21:08:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:07 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:07 smithi104 ceph-mon[35325]: pgmap v470: 97 pgs: 97 active+clean; 156 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 315 op/s 2024-09-10T21:08:08.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 376389 10.53 MB/sec execute 69 sec latency 12.408 ms 2024-09-10T21:08:08.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:08 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:08.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:08 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:09.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:08 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:09.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 378773 10.54 MB/sec execute 70 sec latency 12.813 ms 2024-09-10T21:08:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:09 smithi174 ceph-mon[35635]: pgmap v471: 97 pgs: 97 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-09-10T21:08:09.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:09 smithi033 ceph-mon[30860]: pgmap v471: 97 pgs: 97 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-09-10T21:08:10.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:09 smithi104 ceph-mon[35325]: pgmap v471: 97 pgs: 97 active+clean; 157 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 345 op/s 2024-09-10T21:08:10.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 380244 10.50 MB/sec execute 71 sec latency 30.084 ms 2024-09-10T21:08:11.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 382924 10.56 MB/sec execute 72 sec latency 14.010 ms 2024-09-10T21:08:11.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:11 smithi174 ceph-mon[35635]: pgmap v472: 97 pgs: 97 active+clean; 158 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2024-09-10T21:08:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:11 smithi033 ceph-mon[30860]: pgmap v472: 97 pgs: 97 active+clean; 158 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2024-09-10T21:08:12.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:11 smithi104 ceph-mon[35325]: pgmap v472: 97 pgs: 97 active+clean; 158 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 296 op/s 2024-09-10T21:08:12.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 384557 10.54 MB/sec execute 73 sec latency 8.897 ms 2024-09-10T21:08:13.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 387014 10.59 MB/sec execute 74 sec latency 11.495 ms 2024-09-10T21:08:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:13 smithi174 ceph-mon[35635]: pgmap v473: 97 pgs: 97 active+clean; 159 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-09-10T21:08:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:13 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:13 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:08:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:13 smithi033 ceph-mon[30860]: pgmap v473: 97 pgs: 97 active+clean; 159 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-09-10T21:08:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:13 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:13 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:08:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:13 smithi104 ceph-mon[35325]: pgmap v473: 97 pgs: 97 active+clean; 159 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.5 MiB/s wr, 309 op/s 2024-09-10T21:08:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:13 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:13 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:08:14.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 389166 10.55 MB/sec execute 75 sec latency 14.120 ms 2024-09-10T21:08:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:08:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:08:14.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:08:14.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:08:14.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:08:14.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:08:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:08:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:08:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:08:15.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 390839 10.57 MB/sec execute 76 sec latency 15.410 ms 2024-09-10T21:08:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:15 smithi174 ceph-mon[35635]: pgmap v474: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2024-09-10T21:08:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:15 smithi033 ceph-mon[30860]: pgmap v474: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2024-09-10T21:08:16.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:15 smithi104 ceph-mon[35325]: pgmap v474: 97 pgs: 97 active+clean; 161 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 335 op/s 2024-09-10T21:08:16.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 393426 10.60 MB/sec execute 77 sec latency 20.622 ms 2024-09-10T21:08:17.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 394923 10.57 MB/sec execute 78 sec latency 33.452 ms 2024-09-10T21:08:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:17 smithi174 ceph-mon[35635]: pgmap v475: 97 pgs: 97 active+clean; 162 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-09-10T21:08:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:17 smithi033 ceph-mon[30860]: pgmap v475: 97 pgs: 97 active+clean; 162 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-09-10T21:08:18.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:17 smithi104 ceph-mon[35325]: pgmap v475: 97 pgs: 97 active+clean; 162 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 316 op/s 2024-09-10T21:08:18.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 397572 10.64 MB/sec execute 79 sec latency 12.021 ms 2024-09-10T21:08:19.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 399705 10.60 MB/sec execute 80 sec latency 11.434 ms 2024-09-10T21:08:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:19 smithi174 ceph-mon[35635]: pgmap v476: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-09-10T21:08:19.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:19 smithi033 ceph-mon[30860]: pgmap v476: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-09-10T21:08:20.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:19 smithi104 ceph-mon[35325]: pgmap v476: 97 pgs: 97 active+clean; 166 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.3 MiB/s wr, 357 op/s 2024-09-10T21:08:20.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 401383 10.62 MB/sec execute 81 sec latency 17.037 ms 2024-09-10T21:08:21.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 403839 10.65 MB/sec execute 82 sec latency 20.962 ms 2024-09-10T21:08:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:21 smithi174 ceph-mon[35635]: pgmap v477: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-09-10T21:08:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:21 smithi033 ceph-mon[30860]: pgmap v477: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-09-10T21:08:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:21 smithi104 ceph-mon[35325]: pgmap v477: 97 pgs: 97 active+clean; 167 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 327 op/s 2024-09-10T21:08:22.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 405390 10.61 MB/sec execute 83 sec latency 10.483 ms 2024-09-10T21:08:23.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 408144 10.69 MB/sec execute 84 sec latency 11.062 ms 2024-09-10T21:08:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:23 smithi174 ceph-mon[35635]: pgmap v478: 97 pgs: 97 active+clean; 165 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-10T21:08:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:23 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:23 smithi033 ceph-mon[30860]: pgmap v478: 97 pgs: 97 active+clean; 165 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-10T21:08:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:23 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:23 smithi104 ceph-mon[35325]: pgmap v478: 97 pgs: 97 active+clean; 165 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 309 op/s 2024-09-10T21:08:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:23 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:24.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 409598 10.63 MB/sec execute 85 sec latency 39.853 ms 2024-09-10T21:08:25.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 411893 10.67 MB/sec execute 86 sec latency 10.914 ms 2024-09-10T21:08:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:25 smithi174 ceph-mon[35635]: pgmap v479: 97 pgs: 97 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-10T21:08:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:25 smithi033 ceph-mon[30860]: pgmap v479: 97 pgs: 97 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-10T21:08:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:25 smithi104 ceph-mon[35325]: pgmap v479: 97 pgs: 97 active+clean; 168 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.7 MiB/s wr, 379 op/s 2024-09-10T21:08:26.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 414070 10.64 MB/sec execute 87 sec latency 38.984 ms 2024-09-10T21:08:27.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 415841 10.65 MB/sec execute 88 sec latency 9.856 ms 2024-09-10T21:08:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:27 smithi174 ceph-mon[35635]: pgmap v480: 97 pgs: 97 active+clean; 169 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T21:08:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:27 smithi033 ceph-mon[30860]: pgmap v480: 97 pgs: 97 active+clean; 169 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T21:08:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:27 smithi104 ceph-mon[35325]: pgmap v480: 97 pgs: 97 active+clean; 169 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 307 op/s 2024-09-10T21:08:28.721 INFO:tasks.workunit.client.0.smithi033.stdout: 1 418787 10.74 MB/sec execute 89 sec latency 11.640 ms 2024-09-10T21:08:28.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:28 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:28.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:28 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:29.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:28 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:29.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 420832 10.69 MB/sec execute 90 sec latency 9.582 ms 2024-09-10T21:08:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:29 smithi174 ceph-mon[35635]: pgmap v481: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-09-10T21:08:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:29 smithi033 ceph-mon[30860]: pgmap v481: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-09-10T21:08:29.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:29 smithi104 ceph-mon[35325]: pgmap v481: 97 pgs: 97 active+clean; 170 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.3 MiB/s wr, 355 op/s 2024-09-10T21:08:30.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 422593 10.71 MB/sec execute 91 sec latency 10.431 ms 2024-09-10T21:08:31.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 425031 10.73 MB/sec execute 92 sec latency 32.227 ms 2024-09-10T21:08:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:31 smithi174 ceph-mon[35635]: pgmap v482: 97 pgs: 97 active+clean; 173 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 324 op/s 2024-09-10T21:08:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:31 smithi033 ceph-mon[30860]: pgmap v482: 97 pgs: 97 active+clean; 173 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 324 op/s 2024-09-10T21:08:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:31 smithi104 ceph-mon[35325]: pgmap v482: 97 pgs: 97 active+clean; 173 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 324 op/s 2024-09-10T21:08:32.205 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:08:32.482 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:32.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 426617 10.70 MB/sec execute 93 sec latency 19.591 ms 2024-09-10T21:08:33.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 -- 172.21.15.33:0/3699478444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0108020 msgr2=0x7fbef010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 --2- 172.21.15.33:0/3699478444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0108020 0x7fbef010e820 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fbee8009ca0 tx=0x7fbee802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 -- 172.21.15.33:0/3699478444 shutdown_connections 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 --2- 172.21.15.33:0/3699478444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0108020 0x7fbef010e820 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 --2- 172.21.15.33:0/3699478444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef01076a0 0x7fbef0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 --2- 172.21.15.33:0/3699478444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef0101c90 0x7fbef0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 -- 172.21.15.33:0/3699478444 >> 172.21.15.33:0/3699478444 conn(0x7fbef00fd700 msgr2=0x7fbef00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 -- 172.21.15.33:0/3699478444 shutdown_connections 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.100+0000 7fbef54ef640 1 -- 172.21.15.33:0/3699478444 wait complete. 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 Processor -- start 2024-09-10T21:08:33.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 -- start start 2024-09-10T21:08:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c90 0x7fbef0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef01076a0 0x7fbef0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef0108020 0x7fbef019eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0111ca0 con 0x7fbef0101c90 2024-09-10T21:08:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0111b20 con 0x7fbef01076a0 2024-09-10T21:08:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbef54ef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbef0111e20 con 0x7fbef0108020 2024-09-10T21:08:33.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbeeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c90 0x7fbef0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbeeeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c90 0x7fbef0199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54016/0 (socket says 172.21.15.33:54016) 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbeeeffd640 1 -- 172.21.15.33:0/2359157369 learned_addr learned my addr 172.21.15.33:0/2359157369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbee7fff640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef01076a0 0x7fbef0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.101+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef0108020 0x7fbef019eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeeeffd640 1 -- 172.21.15.33:0/2359157369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef0108020 msgr2=0x7fbef019eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeeeffd640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef0108020 0x7fbef019eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeeeffd640 1 -- 172.21.15.33:0/2359157369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef01076a0 msgr2=0x7fbef0199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeeeffd640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef01076a0 0x7fbef0199760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeeeffd640 1 -- 172.21.15.33:0/2359157369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbef019f2d0 con 0x7fbef0101c90 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeeeffd640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c90 0x7fbef0199220 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbee000d950 tx=0x7fbee000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeecff9640 1 -- 172.21.15.33:0/2359157369 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0017030 con 0x7fbef0101c90 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbef019f5c0 con 0x7fbef0101c90 2024-09-10T21:08:33.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeecff9640 1 -- 172.21.15.33:0/2359157369 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee0004020 con 0x7fbef0101c90 2024-09-10T21:08:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbeecff9640 1 -- 172.21.15.33:0/2359157369 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00047a0 con 0x7fbef0101c90 2024-09-10T21:08:33.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.102+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbef01a5e50 con 0x7fbef0101c90 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.105+0000 7fbeecff9640 1 -- 172.21.15.33:0/2359157369 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbee0009a30 con 0x7fbef0101c90 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.105+0000 7fbef78c4740 1 Processor -- start 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.105+0000 7fbef78c4740 1 -- start start 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4615480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46159c0 0x55e5a4621410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4623d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbef78c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e5a458be50 con 0x55e5a4621950 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbef78c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e5a457b280 con 0x55e5a46150a0 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbef78c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5a457f090 con 0x55e5a46159c0 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4615480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4623d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4615480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41662/0 (socket says 172.21.15.33:41662) 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 -- 172.21.15.33:0/3447227333 learned_addr learned my addr 172.21.15.33:0/3447227333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeeeffd640 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46159c0 0x55e5a4621410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 -- 172.21.15.33:0/3447227333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46159c0 msgr2=0x55e5a4621410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46159c0 0x55e5a4621410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 -- 172.21.15.33:0/3447227333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 msgr2=0x55e5a4623d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4623d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 -- 172.21.15.33:0/3447227333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a42d9650 con 0x55e5a46150a0 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeecff9640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbedc0b5360 0x7fbedc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeeeffd640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbedc0b5360 0x7fbedc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.106+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4615480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbee8035000 tx=0x7fbee8009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75c13dd3ec811c50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:33.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.107+0000 7fbeecff9640 1 -- 172.21.15.33:0/2359157369 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbee00e55b0 con 0x7fbef0101c90 2024-09-10T21:08:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.107+0000 7fbee57fa640 1 -- 172.21.15.33:0/3447227333 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8031cf0 con 0x55e5a46150a0 2024-09-10T21:08:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.107+0000 7fbee57fa640 1 -- 172.21.15.33:0/3447227333 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee8036030 con 0x55e5a46150a0 2024-09-10T21:08:33.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.107+0000 7fbee57fa640 1 -- 172.21.15.33:0/3447227333 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee80042a0 con 0x55e5a46150a0 2024-09-10T21:08:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.107+0000 7fbef78c4740 1 -- 172.21.15.33:0/3447227333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 msgr2=0x55e5a4615480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.107+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4615480 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbee8035000 tx=0x7fbee8009800 comp rx=0 tx=0).stop 2024-09-10T21:08:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.107+0000 7fbeeeffd640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbedc0b5360 0x7fbedc0b7820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbee00a8fa0 tx=0x7fbee0093040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- 172.21.15.33:0/3447227333 shutdown_connections 2024-09-10T21:08:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4623d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46159c0 0x55e5a4621410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3447227333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4615480 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- 172.21.15.33:0/3447227333 >> 172.21.15.33:0/3447227333 conn(0x55e5a4612ed0 msgr2=0x55e5a4614070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- 172.21.15.33:0/3447227333 shutdown_connections 2024-09-10T21:08:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- 172.21.15.33:0/3447227333 wait complete. 2024-09-10T21:08:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- start start 2024-09-10T21:08:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 Processor -- start 2024-09-10T21:08:33.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46150a0 0x55e5a4621280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a461ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e5a43dc700 con 0x55e5a46150a0 2024-09-10T21:08:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e5a4516110 con 0x55e5a46159c0 2024-09-10T21:08:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.108+0000 7fbef78c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5a46243f0 con 0x55e5a4621950 2024-09-10T21:08:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbef78c4740 10 client.0 _ll_register_callbacks cb 0x55e5a452f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:33.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46150a0 0x55e5a4621280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46150a0 0x55e5a4621280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54036/0 (socket says 172.21.15.33:54036) 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbeef7fe640 1 -- 172.21.15.33:0/3358076525 learned_addr learned my addr 172.21.15.33:0/3358076525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a461ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbee7fff640 1 -- 172.21.15.33:0/3358076525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 msgr2=0x55e5a461ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbee7fff640 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a461ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbee7fff640 1 -- 172.21.15.33:0/3358076525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46150a0 msgr2=0x55e5a4621280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbee7fff640 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46150a0 0x55e5a4621280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbee7fff640 1 -- 172.21.15.33:0/3358076525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a4543c40 con 0x55e5a46159c0 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbee7fff640 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461e6d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b9c0 tx=0x7fbed800be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf151f60ab2cd84c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbed67fc640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbed67fc640 1 -- 172.21.15.33:0/3358076525 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003e20 con 0x55e5a46159c0 2024-09-10T21:08:33.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbed67fc640 1 -- 172.21.15.33:0/3358076525 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbed8005020 con 0x55e5a46159c0 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.109+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee8036030 con 0x55e5a46159c0 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbed67fc640 1 -- 172.21.15.33:0/3358076525 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbed8005220 con 0x55e5a46159c0 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 10 client.25706 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e5a462fdf0 con 0x55e5a46159c0 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbed67fc640 1 -- 172.21.15.33:0/3358076525 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbee8036030 con 0x55e5a46159c0 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbed67fc640 1 -- 172.21.15.33:0/3358076525 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fbed801b500 con 0x55e5a46159c0 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 10 client.25706 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 10 client.25706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 20 client.25706 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a46312b0 0x55e5a46336c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 4 client.25706 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5a458dda0 con 0x55e5a46312b0 2024-09-10T21:08:33.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.110+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a46312b0 0x55e5a46336c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.111+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a46312b0 0x55e5a46336c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f38bd698f5401f2e server_cookie=aea4d28ed366caab in_seq=0 out_seq=0 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.111+0000 7fbed67fc640 10 client.25706 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.111+0000 7fbed67fc640 1 -- 172.21.15.33:0/3358076525 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5a458dda0 con 0x55e5a46312b0 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.111+0000 7fbed67fc640 10 client.25706 handle_command_reply: tid=0 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.111+0000 7fbef78c4740 1 client.25706 shutdown 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 msgr2=0x55e5a461e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461e6d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbed800b9c0 tx=0x7fbed800be80 comp rx=0 tx=0).stop 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 shutdown_connections 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a46312b0 0x55e5a46336c0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a461ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461e6d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 --2- 172.21.15.33:0/3358076525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46150a0 0x55e5a4621280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbed67fc640 0 client.25706 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 >> 172.21.15.33:0/3358076525 conn(0x55e5a4612ed0 msgr2=0x55e5a4618a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 shutdown_connections 2024-09-10T21:08:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 1 -- 172.21.15.33:0/3358076525 wait complete. 2024-09-10T21:08:33.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.112+0000 7fbef78c4740 20 client.25706 trim_cache size 0 max 16384 2024-09-10T21:08:33.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 Processor -- start 2024-09-10T21:08:33.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 -- start start 2024-09-10T21:08:33.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46150a0 0x55e5a46146a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4612710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01ac1d0 con 0x55e5a4621950 2024-09-10T21:08:33.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01add10 con 0x55e5a46159c0 2024-09-10T21:08:33.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.134+0000 7fbef78c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01ac8a0 con 0x55e5a46150a0 2024-09-10T21:08:33.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41680/0 (socket says 172.21.15.33:41680) 2024-09-10T21:08:33.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 -- 172.21.15.33:0/631623731 learned_addr learned my addr 172.21.15.33:0/631623731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:33.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46150a0 0x55e5a46146a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4612710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 -- 172.21.15.33:0/631623731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46150a0 msgr2=0x55e5a46146a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46150a0 0x55e5a46146a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 -- 172.21.15.33:0/631623731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 msgr2=0x55e5a4612710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4612710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 -- 172.21.15.33:0/631623731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5a461d630 con 0x55e5a46159c0 2024-09-10T21:08:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4612710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:33.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.135+0000 7fbee7fff640 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461dbc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbed8000c00 tx=0x7fbed8004400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f5ac154e15dbb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.136+0000 7fbee4ff9640 1 -- 172.21.15.33:0/631623731 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8003f60 con 0x55e5a46159c0 2024-09-10T21:08:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.136+0000 7fbee4ff9640 1 -- 172.21.15.33:0/631623731 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed80053c0 con 0x55e5a46159c0 2024-09-10T21:08:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.136+0000 7fbee4ff9640 1 -- 172.21.15.33:0/631623731 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed8004b30 con 0x55e5a46159c0 2024-09-10T21:08:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- 172.21.15.33:0/631623731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 msgr2=0x55e5a461dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461dbc0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbed8000c00 tx=0x7fbed8004400 comp rx=0 tx=0).stop 2024-09-10T21:08:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- 172.21.15.33:0/631623731 shutdown_connections 2024-09-10T21:08:33.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a4621950 0x55e5a4612710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46159c0 0x55e5a461dbc0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 --2- 172.21.15.33:0/631623731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a46150a0 0x55e5a46146a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- 172.21.15.33:0/631623731 >> 172.21.15.33:0/631623731 conn(0x55e5a4612ed0 msgr2=0x55e5a4613f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- 172.21.15.33:0/631623731 shutdown_connections 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- 172.21.15.33:0/631623731 wait complete. 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- start start 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 Processor -- start 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4629950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46159c0 0x55e5a4629eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a4624690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01ad640 con 0x55e5a46159c0 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01acf70 con 0x55e5a46150a0 2024-09-10T21:08:33.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbef01aa190 con 0x55e5a4621950 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.137+0000 7fbef78c4740 10 client.0 _ll_register_callbacks cb 0x7fbed8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4629950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbee7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46159c0 0x55e5a4629eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4629950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41686/0 (socket says 172.21.15.33:41686) 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 -- 172.21.15.33:0/2541123069 learned_addr learned my addr 172.21.15.33:0/2541123069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a4624690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 -- 172.21.15.33:0/2541123069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 msgr2=0x55e5a4624690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a4624690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 -- 172.21.15.33:0/2541123069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46159c0 msgr2=0x55e5a4629eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46159c0 0x55e5a4629eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 -- 172.21.15.33:0/2541123069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5a4624ec0 con 0x55e5a46150a0 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbeef7fe640 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4629950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbee8000f80 tx=0x7fbee8047040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3401e58d01043a67 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:33.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbed67fc640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbed67fc640 1 -- 172.21.15.33:0/2541123069 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee8031660 con 0x55e5a46150a0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbed67fc640 1 -- 172.21.15.33:0/2541123069 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbee8031800 con 0x55e5a46150a0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbed80053c0 con 0x55e5a46150a0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbed67fc640 1 -- 172.21.15.33:0/2541123069 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fbee802f660 con 0x55e5a46150a0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbef78c4740 10 client.25712 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.138+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55e5a469ed20 con 0x55e5a46150a0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbed67fc640 1 -- 172.21.15.33:0/2541123069 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbed80053c0 con 0x55e5a46150a0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbed67fc640 1 -- 172.21.15.33:0/2541123069 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fbee8043070 con 0x55e5a46150a0 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbef78c4740 10 client.25712 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbef78c4740 10 client.25712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbef78c4740 20 client.25712 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbef78c4740 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a469f120 0x55e5a46a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:33.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbef78c4740 4 client.25712 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e5a4589850 con 0x55e5a469f120 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbee7fff640 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a469f120 0x55e5a46a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbee7fff640 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a469f120 0x55e5a46a1510 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11918f029075bef server_cookie=723c7c0178f25a5b in_seq=0 out_seq=0 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.139+0000 7fbed67fc640 10 client.25712 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbed67fc640 1 -- 172.21.15.33:0/2541123069 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55e5a4589850 con 0x55e5a469f120 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbed67fc640 10 client.25712 handle_command_reply: tid=0 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 client.25712 shutdown 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 msgr2=0x55e5a4629950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4629950 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbee8000f80 tx=0x7fbee8047040 comp rx=0 tx=0).stop 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 shutdown_connections 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5a469f120 0x55e5a46a1510 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5a4621950 0x55e5a4624690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbed67fc640 0 client.25712 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5a46159c0 0x55e5a4629eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 --2- 172.21.15.33:0/2541123069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5a46150a0 0x55e5a4629950 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 >> 172.21.15.33:0/2541123069 conn(0x55e5a4612ed0 msgr2=0x55e5a461fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 shutdown_connections 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 1 -- 172.21.15.33:0/2541123069 wait complete. 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.140+0000 7fbef78c4740 20 client.25712 trim_cache size 0 max 16384 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:08:33.151 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:33.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:08:33.153 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:33.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:33.155 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:08:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbedc0b5360 msgr2=0x7fbedc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbedc0b5360 0x7fbedc0b7820 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbee00a8fa0 tx=0x7fbee0093040 comp rx=0 tx=0).stop 2024-09-10T21:08:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c90 msgr2=0x7fbef0199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c90 0x7fbef0199220 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbee000d950 tx=0x7fbee000de10 comp rx=0 tx=0).stop 2024-09-10T21:08:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 shutdown_connections 2024-09-10T21:08:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbedc0b5360 0x7fbedc0b7820 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbef0108020 0x7fbef019eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbef01076a0 0x7fbef0199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 --2- 172.21.15.33:0/2359157369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbef0101c90 0x7fbef0199220 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:33.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 >> 172.21.15.33:0/2359157369 conn(0x7fbef00fd700 msgr2=0x7fbef006ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:33.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 shutdown_connections 2024-09-10T21:08:33.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:33.141+0000 7fbef54ef640 1 -- 172.21.15.33:0/2359157369 wait complete. 2024-09-10T21:08:33.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 429418 10.78 MB/sec execute 94 sec latency 22.951 ms 2024-09-10T21:08:33.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:33 smithi033 ceph-mon[30860]: pgmap v483: 97 pgs: 97 active+clean; 173 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-09-10T21:08:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:33 smithi174 ceph-mon[35635]: pgmap v483: 97 pgs: 97 active+clean; 173 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-09-10T21:08:34.059 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:08:34.059 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:08:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:33 smithi104 ceph-mon[35325]: pgmap v483: 97 pgs: 97 active+clean; 173 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-09-10T21:08:34.344 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:34.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 430998 10.72 MB/sec execute 95 sec latency 19.844 ms 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.876+0000 7fd32d0c8640 1 -- 172.21.15.33:0/1348036768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32810f3b0 msgr2=0x7fd3281117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.876+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/1348036768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32810f3b0 0x7fd3281117a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd320009ca0 tx=0x7fd32002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.876+0000 7fd32d0c8640 1 -- 172.21.15.33:0/1348036768 shutdown_connections 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.876+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/1348036768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd32810f3b0 0x7fd3281117a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.876+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/1348036768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd328075e00 0x7fd32810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.876+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/1348036768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 0x7fd3280758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.876+0000 7fd32d0c8640 1 -- 172.21.15.33:0/1348036768 >> 172.21.15.33:0/1348036768 conn(0x7fd3280fe830 msgr2=0x7fd328100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.877+0000 7fd32d0c8640 1 -- 172.21.15.33:0/1348036768 shutdown_connections 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.877+0000 7fd32d0c8640 1 -- 172.21.15.33:0/1348036768 wait complete. 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.877+0000 7fd32d0c8640 1 Processor -- start 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32d0c8640 1 -- start start 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32d0c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 0x7fd32819b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32d0c8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd328075e00 0x7fd32819ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32d0c8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32810f3b0 0x7fd3281a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32d0c8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd328114fc0 con 0x7fd328075e00 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32d0c8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd328114e40 con 0x7fd32810f3b0 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32d0c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd328115140 con 0x7fd3280754e0 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd328075e00 0x7fd32819ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd326d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 0x7fd32819b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd326d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 0x7fd32819b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46300/0 (socket says 172.21.15.33:46300) 2024-09-10T21:08:34.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd326d9d640 1 -- 172.21.15.33:0/222764314 learned_addr learned my addr 172.21.15.33:0/222764314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32759e640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32810f3b0 0x7fd3281a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32659c640 1 -- 172.21.15.33:0/222764314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 msgr2=0x7fd32819b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32659c640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 0x7fd32819b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32659c640 1 -- 172.21.15.33:0/222764314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32810f3b0 msgr2=0x7fd3281a0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32659c640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32810f3b0 0x7fd3281a0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32659c640 1 -- 172.21.15.33:0/222764314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3281a15e0 con 0x7fd328075e00 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd326d9d640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 0x7fd32819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.878+0000 7fd32759e640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32810f3b0 0x7fd3281a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.879+0000 7fd32659c640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd328075e00 0x7fd32819ba70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00e9f0 tx=0x7fd31c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.879+0000 7fd30ffff640 1 -- 172.21.15.33:0/222764314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c00ccc0 con 0x7fd328075e00 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.879+0000 7fd30ffff640 1 -- 172.21.15.33:0/222764314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd31c0108e0 con 0x7fd328075e00 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.879+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3281a18d0 con 0x7fd328075e00 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.879+0000 7fd30ffff640 1 -- 172.21.15.33:0/222764314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c018880 con 0x7fd328075e00 2024-09-10T21:08:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.879+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3281a7f90 con 0x7fd328075e00 2024-09-10T21:08:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.880+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3280758c0 con 0x7fd328075e00 2024-09-10T21:08:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.882+0000 7fd30ffff640 1 -- 172.21.15.33:0/222764314 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd31c010430 con 0x7fd328075e00 2024-09-10T21:08:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.882+0000 7fd30ffff640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd3100b52e0 0x7fd3100b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.883+0000 7fd326d9d640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd3100b52e0 0x7fd3100b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.883+0000 7fd30ffff640 1 -- 172.21.15.33:0/222764314 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd31c0e6320 con 0x7fd328075e00 2024-09-10T21:08:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.883+0000 7fd326d9d640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd3100b52e0 0x7fd3100b77a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd318011500 tx=0x7fd318005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:34.886+0000 7fd30ffff640 1 -- 172.21.15.33:0/222764314 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd31c0acb50 con 0x7fd328075e00 2024-09-10T21:08:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.131+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd32810a590 con 0x7fd328075e00 2024-09-10T21:08:35.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.132+0000 7fd30ffff640 1 -- 172.21.15.33:0/222764314 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11973 (secure 0 0 0) 0x7fd31c0b1580 con 0x7fd328075e00 2024-09-10T21:08:35.137 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:08:35.137 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":42,"btime":"2024-09-10T21:08:27:601949+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:08:27.083288+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:08:35.138 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 42 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.134+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd3100b52e0 msgr2=0x7fd3100b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.134+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd3100b52e0 0x7fd3100b77a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd318011500 tx=0x7fd318005f90 comp rx=0 tx=0).stop 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.134+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd328075e00 msgr2=0x7fd32819ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.134+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd328075e00 0x7fd32819ba70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00e9f0 tx=0x7fd31c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.134+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 shutdown_connections 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.134+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd3100b52e0 0x7fd3100b77a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.134+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd32810f3b0 0x7fd3281a0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.135+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd328075e00 0x7fd32819ba70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.135+0000 7fd32d0c8640 1 --2- 172.21.15.33:0/222764314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3280754e0 0x7fd32819b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.135+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 >> 172.21.15.33:0/222764314 conn(0x7fd3280fe830 msgr2=0x7fd3280ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:35.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.135+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 shutdown_connections 2024-09-10T21:08:35.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:35.135+0000 7fd32d0c8640 1 -- 172.21.15.33:0/222764314 wait complete. 2024-09-10T21:08:35.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 433191 10.75 MB/sec execute 96 sec latency 11.601 ms 2024-09-10T21:08:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:35 smithi174 ceph-mon[35635]: pgmap v484: 97 pgs: 97 active+clean; 175 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-09-10T21:08:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:35 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/222764314' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:08:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:35 smithi033 ceph-mon[30860]: pgmap v484: 97 pgs: 97 active+clean; 175 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-09-10T21:08:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:35 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/222764314' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:08:35.979 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:08:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:35 smithi104 ceph-mon[35325]: pgmap v484: 97 pgs: 97 active+clean; 175 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.5 MiB/s wr, 374 op/s 2024-09-10T21:08:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:35 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/222764314' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:08:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:36.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 435268 10.72 MB/sec execute 97 sec latency 62.409 ms 2024-09-10T21:08:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 -- 172.21.15.33:0/2714954209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd4103ca0 msgr2=0x7f7cd4104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/2714954209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd4103ca0 0x7f7cd4104080 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8009ca0 tx=0x7f7cc802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 -- 172.21.15.33:0/2714954209 shutdown_connections 2024-09-10T21:08:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/2714954209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd410d100 0x7f7cd410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/2714954209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd41045c0 0x7f7cd410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/2714954209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd4103ca0 0x7f7cd4104080 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 -- 172.21.15.33:0/2714954209 >> 172.21.15.33:0/2714954209 conn(0x7f7cd40fd700 msgr2=0x7f7cd40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 -- 172.21.15.33:0/2714954209 shutdown_connections 2024-09-10T21:08:36.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.923+0000 7f7cdbae0640 1 -- 172.21.15.33:0/2714954209 wait complete. 2024-09-10T21:08:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 Processor -- start 2024-09-10T21:08:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 -- start start 2024-09-10T21:08:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd4103ca0 0x7f7cd41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd41045c0 0x7f7cd4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd410d100 0x7f7cd419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4112d70 con 0x7f7cd410d100 2024-09-10T21:08:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4112bf0 con 0x7f7cd41045c0 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cdbae0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd4112ef0 con 0x7f7cd4103ca0 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd4103ca0 0x7f7cd41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cda07d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd410d100 0x7f7cd419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cd907b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd41045c0 0x7f7cd4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd4103ca0 0x7f7cd41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46316/0 (socket says 172.21.15.33:46316) 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cd907b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd41045c0 0x7f7cd4199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41714/0 (socket says 172.21.15.33:41714) 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.924+0000 7f7cd987c640 1 -- 172.21.15.33:0/3077059903 learned_addr learned my addr 172.21.15.33:0/3077059903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cda07d640 1 -- 172.21.15.33:0/3077059903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd4103ca0 msgr2=0x7f7cd41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cda07d640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd4103ca0 0x7f7cd41992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cda07d640 1 -- 172.21.15.33:0/3077059903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd41045c0 msgr2=0x7f7cd4199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cda07d640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd41045c0 0x7f7cd4199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cda07d640 1 -- 172.21.15.33:0/3077059903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cd419f370 con 0x7f7cd410d100 2024-09-10T21:08:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cda07d640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd410d100 0x7f7cd419eba0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00e970 tx=0x7f7ccc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cd2ffd640 1 -- 172.21.15.33:0/3077059903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00ccc0 con 0x7f7cd410d100 2024-09-10T21:08:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cd2ffd640 1 -- 172.21.15.33:0/3077059903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ccc005020 con 0x7f7cd410d100 2024-09-10T21:08:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cd419f660 con 0x7f7cd410d100 2024-09-10T21:08:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cd2ffd640 1 -- 172.21.15.33:0/3077059903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc004a00 con 0x7f7cd410d100 2024-09-10T21:08:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.925+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cd41a5d20 con 0x7f7cd410d100 2024-09-10T21:08:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.928+0000 7f7cd2ffd640 1 -- 172.21.15.33:0/3077059903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7ccc004ba0 con 0x7f7cd410d100 2024-09-10T21:08:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.928+0000 7f7cddeb5740 1 Processor -- start 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.928+0000 7f7cddeb5740 1 -- start start 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0be460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555d1a028ea0 con 0x555d1a0b20f0 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555d1a0182d0 con 0x555d1a0b2a10 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d1a01c0e0 con 0x555d1a0be9a0 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd907b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0be460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd907b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54110/0 (socket says 172.21.15.33:54110) 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd907b640 1 -- 172.21.15.33:0/3196372608 learned_addr learned my addr 172.21.15.33:0/3196372608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:36.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd907b640 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd987c640 1 -- 172.21.15.33:0/3196372608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 msgr2=0x555d1a0c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd987c640 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd987c640 1 -- 172.21.15.33:0/3196372608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 msgr2=0x555d1a0b24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd987c640 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0b24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd987c640 1 -- 172.21.15.33:0/3196372608 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d19d76650 con 0x555d1a0b2a10 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd907b640 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.929+0000 7f7cd2ffd640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b53e0 0x7f7cc40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.930+0000 7f7cd987c640 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0be460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8000f80 tx=0x7f7cc8005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b4a5fb772a74c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.930+0000 7f7cd907b640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b53e0 0x7f7cc40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.930+0000 7f7cd2ffd640 1 -- 172.21.15.33:0/3077059903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7ccc0e5b00 con 0x7f7cd410d100 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.930+0000 7f7cbffff640 1 -- 172.21.15.33:0/3196372608 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8031cf0 con 0x555d1a0b2a10 2024-09-10T21:08:36.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.930+0000 7f7cbffff640 1 -- 172.21.15.33:0/3196372608 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8036030 con 0x555d1a0b2a10 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.930+0000 7f7cbffff640 1 -- 172.21.15.33:0/3196372608 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc80042a0 con 0x555d1a0b2a10 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3196372608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 msgr2=0x555d1a0be460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cd907b640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b53e0 0x7f7cc40b78a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009fd0 tx=0x7f7cc0015040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0be460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8000f80 tx=0x7f7cc8005f60 comp rx=0 tx=0).stop 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3196372608 shutdown_connections 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0be460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3196372608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0b24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3196372608 >> 172.21.15.33:0/3196372608 conn(0x555d1a0aff20 msgr2=0x555d1a0b10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3196372608 shutdown_connections 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3196372608 wait complete. 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 -- start start 2024-09-10T21:08:36.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.931+0000 7f7cddeb5740 1 Processor -- start 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b20f0 0x555d1a0be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b2a10 0x555d1a0bb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0bbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555d19e79700 con 0x555d1a0b2a10 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555d19fb3160 con 0x555d1a0b20f0 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555d1a0c5850 con 0x555d1a0be9a0 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cddeb5740 10 client.0 _ll_register_callbacks cb 0x555d19fcc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b20f0 0x555d1a0be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd907b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b2a10 0x555d1a0bb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b20f0 0x555d1a0be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41734/0 (socket says 172.21.15.33:41734) 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 -- 172.21.15.33:0/3746809491 learned_addr learned my addr 172.21.15.33:0/3746809491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cda07d640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0bbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 -- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 msgr2=0x555d1a0bbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0bbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 -- 172.21.15.33:0/3746809491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b2a10 msgr2=0x555d1a0bb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b2a10 0x555d1a0bb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 -- 172.21.15.33:0/3746809491 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d19fe0c90 con 0x555d1a0b20f0 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cda07d640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0bbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd907b640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b2a10 0x555d1a0bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cd987c640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b20f0 0x555d1a0be240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7cc802b090 tx=0x7f7cc8031d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84381f0ef054093 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.932+0000 7f7cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/3746809491 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc804a040 con 0x555d1a0b20f0 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/3746809491 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7cc8031650 con 0x555d1a0b20f0 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc8036030 con 0x555d1a0b20f0 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/3746809491 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7cc8031850 con 0x555d1a0b20f0 2024-09-10T21:08:36.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 10 client.25730 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555d1a0d1170 con 0x555d1a0b20f0 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/3746809491 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8036030 con 0x555d1a0b20f0 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/3746809491 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f7cc804d070 con 0x555d1a0b20f0 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 10 client.25730 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 10 client.25730 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 20 client.25730 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a0d24c0 0x555d1a0d48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 4 client.25730 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:36.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.933+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d1a02adf0 con 0x555d1a0d24c0 2024-09-10T21:08:36.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.934+0000 7f7cd907b640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a0d24c0 0x555d1a0d48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.934+0000 7f7cbd7fa640 10 client.25730 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.934+0000 7f7cd907b640 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a0d24c0 0x555d1a0d48b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65196582384a8d86 server_cookie=30ac4b873c749f3f in_seq=0 out_seq=0 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/3746809491 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555d1a02adf0 con 0x555d1a0d24c0 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cbd7fa640 10 client.25730 handle_command_reply: tid=0 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 client.25730 shutdown 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b20f0 msgr2=0x555d1a0be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b20f0 0x555d1a0be240 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7cc802b090 tx=0x7f7cc8031d20 comp rx=0 tx=0).stop 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 shutdown_connections 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a0d24c0 0x555d1a0d48b0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0bbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b2a10 0x555d1a0bb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cbd7fa640 0 client.25730 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/3746809491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b20f0 0x555d1a0be240 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 >> 172.21.15.33:0/3746809491 conn(0x555d1a0aff20 msgr2=0x555d1a0b5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:36.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 shutdown_connections 2024-09-10T21:08:36.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 1 -- 172.21.15.33:0/3746809491 wait complete. 2024-09-10T21:08:36.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.935+0000 7f7cddeb5740 20 client.25730 trim_cache size 0 max 16384 2024-09-10T21:08:36.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 Processor -- start 2024-09-10T21:08:36.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 -- start start 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0bab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0b9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd41ac130 con 0x555d1a0b20f0 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd41adc70 con 0x555d1a0b2a10 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd41ac800 con 0x555d1a0be9a0 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0bab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0bab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54122/0 (socket says 172.21.15.33:54122) 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd907b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 -- 172.21.15.33:0/823876696 learned_addr learned my addr 172.21.15.33:0/823876696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:36.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cda07d640 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0b9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 -- 172.21.15.33:0/823876696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 msgr2=0x555d1a0b9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0b9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 -- 172.21.15.33:0/823876696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 msgr2=0x555d1a0af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.950+0000 7f7cd987c640 1 -- 172.21.15.33:0/823876696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d1a0b2840 con 0x555d1a0b20f0 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.951+0000 7f7cd987c640 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0bab80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8003d40 tx=0x7f7cc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510682114071e745 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.951+0000 7f7cbf7fe640 1 -- 172.21.15.33:0/823876696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8031d90 con 0x555d1a0b20f0 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.951+0000 7f7cbf7fe640 1 -- 172.21.15.33:0/823876696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8036040 con 0x555d1a0b20f0 2024-09-10T21:08:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.951+0000 7f7cbf7fe640 1 -- 172.21.15.33:0/823876696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc80316b0 con 0x555d1a0b20f0 2024-09-10T21:08:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 -- 172.21.15.33:0/823876696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 msgr2=0x555d1a0bab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0bab80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8003d40 tx=0x7f7cc8004a40 comp rx=0 tx=0).stop 2024-09-10T21:08:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 -- 172.21.15.33:0/823876696 shutdown_connections 2024-09-10T21:08:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0b9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/823876696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0bab80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 -- 172.21.15.33:0/823876696 >> 172.21.15.33:0/823876696 conn(0x555d1a0aff20 msgr2=0x555d1a0b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 -- 172.21.15.33:0/823876696 shutdown_connections 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 -- 172.21.15.33:0/823876696 wait complete. 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 -- start start 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.952+0000 7f7cddeb5740 1 Processor -- start 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd41ad5a0 con 0x555d1a0b20f0 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd41aced0 con 0x555d1a0b2a10 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cd41aa0f0 con 0x555d1a0be9a0 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 10 client.0 _ll_register_callbacks cb 0x7f7cc0009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd907b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54128/0 (socket says 172.21.15.33:54128) 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 -- 172.21.15.33:0/495319459 learned_addr learned my addr 172.21.15.33:0/495319459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cda07d640 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 -- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 msgr2=0x555d1a0c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 -- 172.21.15.33:0/495319459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 msgr2=0x555d1a0cb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0cb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 -- 172.21.15.33:0/495319459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d1a0c6f30 con 0x555d1a0b20f0 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd907b640 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cd987c640 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0caf90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8005cf0 tx=0x7f7cc8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b74dd65bd3fec8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/495319459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cc8030eb0 con 0x555d1a0b20f0 2024-09-10T21:08:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/495319459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7cc8030840 con 0x555d1a0b20f0 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.953+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cc8036040 con 0x555d1a0b20f0 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/495319459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f7cc8030a40 con 0x555d1a0b20f0 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 10 client.16173 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x555d1a13eea0 con 0x555d1a0b20f0 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/495319459 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cc8036040 con 0x555d1a0b20f0 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/495319459 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f7cc806b050 con 0x555d1a0b20f0 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 10 client.16173 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 10 client.16173 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 20 client.16173 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a13f350 0x555d1a141770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 4 client.16173 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d1a0268a0 con 0x555d1a13f350 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.954+0000 7f7cd907b640 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a13f350 0x555d1a141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:36.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cd907b640 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a13f350 0x555d1a141770 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3bb88ed8b4762a2 server_cookie=27ca76fb1b6981ab in_seq=0 out_seq=0 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cbd7fa640 10 client.16173 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cbd7fa640 1 -- 172.21.15.33:0/495319459 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d1a0268a0 con 0x555d1a13f350 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cbd7fa640 10 client.16173 handle_command_reply: tid=0 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 client.16173 shutdown 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 msgr2=0x555d1a0caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0caf90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f7cc8005cf0 tx=0x7f7cc8005ef0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 shutdown_connections 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x555d1a13f350 0x555d1a141770 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555d1a0be9a0 0x555d1a0c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555d1a0b2a10 0x555d1a0cb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 --2- 172.21.15.33:0/495319459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555d1a0b20f0 0x555d1a0caf90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 >> 172.21.15.33:0/495319459 conn(0x555d1a0aff20 msgr2=0x555d1a0bccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cbd7fa640 0 client.16173 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:08:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 shutdown_connections 2024-09-10T21:08:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 1 -- 172.21.15.33:0/495319459 wait complete. 2024-09-10T21:08:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.955+0000 7f7cddeb5740 20 client.16173 trim_cache size 0 max 16384 2024-09-10T21:08:36.965 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:08:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b53e0 msgr2=0x7f7cc40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b53e0 0x7f7cc40b78a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009fd0 tx=0x7f7cc0015040 comp rx=0 tx=0).stop 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd410d100 msgr2=0x7f7cd419eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd410d100 0x7f7cd419eba0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00e970 tx=0x7f7ccc00ee30 comp rx=0 tx=0).stop 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 shutdown_connections 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b53e0 0x7f7cc40b78a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cd410d100 0x7f7cd419eba0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cd41045c0 0x7f7cd4199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 --2- 172.21.15.33:0/3077059903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cd4103ca0 0x7f7cd41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 >> 172.21.15.33:0/3077059903 conn(0x7f7cd40fd700 msgr2=0x7f7cd40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 shutdown_connections 2024-09-10T21:08:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:36.956+0000 7f7cdbae0640 1 -- 172.21.15.33:0/3077059903 wait complete. 2024-09-10T21:08:37.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 436870 10.73 MB/sec execute 98 sec latency 18.653 ms 2024-09-10T21:08:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:37 smithi174 ceph-mon[35635]: pgmap v485: 97 pgs: 97 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 308 op/s 2024-09-10T21:08:37.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:37 smithi033 ceph-mon[30860]: pgmap v485: 97 pgs: 97 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 308 op/s 2024-09-10T21:08:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:37 smithi104 ceph-mon[35325]: pgmap v485: 97 pgs: 97 active+clean; 176 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.6 MiB/s wr, 308 op/s 2024-09-10T21:08:38.299 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:08:38.574 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:38.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 439515 10.76 MB/sec execute 99 sec latency 12.175 ms 2024-09-10T21:08:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.080+0000 7fcbc6e53640 1 -- 172.21.15.33:0/2846584805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc010f3e0 msgr2=0x7fcbc01117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.080+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/2846584805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc010f3e0 0x7fcbc01117d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ca0 tx=0x7fcbb802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.080+0000 7fcbc6e53640 1 -- 172.21.15.33:0/2846584805 shutdown_connections 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.080+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/2846584805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc010f3e0 0x7fcbc01117d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.080+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/2846584805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbc0075f40 0x7fcbc010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.080+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/2846584805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbc0075540 0x7fcbc0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.080+0000 7fcbc6e53640 1 -- 172.21.15.33:0/2846584805 >> 172.21.15.33:0/2846584805 conn(0x7fcbc00fe830 msgr2=0x7fcbc0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 -- 172.21.15.33:0/2846584805 shutdown_connections 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 -- 172.21.15.33:0/2846584805 wait complete. 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 Processor -- start 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 -- start start 2024-09-10T21:08:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbc0075540 0x7fcbc019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0075f40 0x7fcbc019b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbc010f3e0 0x7fcbc01a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc0114f50 con 0x7fcbc0075f40 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc0114dd0 con 0x7fcbc010f3e0 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.081+0000 7fcbc6e53640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01150d0 con 0x7fcbc0075540 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0075f40 0x7fcbc019b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc4bef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbc0075540 0x7fcbc019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0075f40 0x7fcbc019b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54144/0 (socket says 172.21.15.33:54144) 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbbffff640 1 -- 172.21.15.33:0/1010809526 learned_addr learned my addr 172.21.15.33:0/1010809526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbc010f3e0 0x7fcbc01a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc4bef640 1 -- 172.21.15.33:0/1010809526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbc010f3e0 msgr2=0x7fcbc01a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc4bef640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbc010f3e0 0x7fcbc01a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc4bef640 1 -- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0075f40 msgr2=0x7fcbc019b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc4bef640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0075f40 0x7fcbc019b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc4bef640 1 -- 172.21.15.33:0/1010809526 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc01a14f0 con 0x7fcbc0075540 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbbffff640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0075f40 0x7fcbc019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc4bef640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbc0075540 0x7fcbc019b440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400e9f0 tx=0x7fcbb400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbbdffb640 1 -- 172.21.15.33:0/1010809526 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb400ccc0 con 0x7fcbc0075540 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc01a17e0 con 0x7fcbc0075540 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbbdffb640 1 -- 172.21.15.33:0/1010809526 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb40108e0 con 0x7fcbc0075540 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.082+0000 7fcbbdffb640 1 -- 172.21.15.33:0/1010809526 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4018880 con 0x7fcbc0075540 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.083+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01a8070 con 0x7fcbc0075540 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.085+0000 7fcbbdffb640 1 -- 172.21.15.33:0/1010809526 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fcbb4010430 con 0x7fcbc0075540 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.085+0000 7fcbc9228740 1 Processor -- start 2024-09-10T21:08:39.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.085+0000 7fcbc9228740 1 -- start start 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d53054d0 0x55c6d53058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d5311840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5314170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc9228740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6d527bea0 con 0x55c6d53054d0 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc9228740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c6d526b2d0 con 0x55c6d5311d80 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc9228740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c6d526f0e0 con 0x55c6d5305df0 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc53f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d53054d0 0x55c6d53058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5314170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc4bef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d5311840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5314170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41776/0 (socket says 172.21.15.33:41776) 2024-09-10T21:08:39.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbbffff640 1 -- 172.21.15.33:0/632044073 learned_addr learned my addr 172.21.15.33:0/632044073 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc53f0640 1 -- 172.21.15.33:0/632044073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 msgr2=0x55c6d5311840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d5311840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc53f0640 1 -- 172.21.15.33:0/632044073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 msgr2=0x55c6d5314170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5314170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc53f0640 1 -- 172.21.15.33:0/632044073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d4fc9650 con 0x55c6d53054d0 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbbffff640 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5314170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbbdffb640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcbb00b52e0 0x7fcbb00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d53054d0 0x55c6d53058b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005e30 tx=0x7fcbb8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90e0d712ef4c8ea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.086+0000 7fcbc4bef640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcbb00b52e0 0x7fcbb00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.087+0000 7fcbbdffb640 1 -- 172.21.15.33:0/1010809526 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcbb40e5f10 con 0x7fcbc0075540 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.087+0000 7fcbaaffd640 1 -- 172.21.15.33:0/632044073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8031cf0 con 0x55c6d53054d0 2024-09-10T21:08:39.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.087+0000 7fcbaaffd640 1 -- 172.21.15.33:0/632044073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb8036030 con 0x55c6d53054d0 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.087+0000 7fcbaaffd640 1 -- 172.21.15.33:0/632044073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb8004040 con 0x55c6d53054d0 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.087+0000 7fcbc9228740 1 -- 172.21.15.33:0/632044073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d53054d0 msgr2=0x55c6d53058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.087+0000 7fcbc4bef640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcbb00b52e0 0x7fcbb00b77a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400e9c0 tx=0x7fcbb40e9950 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.087+0000 7fcbc9228740 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d53054d0 0x55c6d53058b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8005e30 tx=0x7fcbb8005ef0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- 172.21.15.33:0/632044073 shutdown_connections 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5314170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d5311840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 --2- 172.21.15.33:0/632044073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d53054d0 0x55c6d53058b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- 172.21.15.33:0/632044073 >> 172.21.15.33:0/632044073 conn(0x55c6d5303300 msgr2=0x55c6d53044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- 172.21.15.33:0/632044073 shutdown_connections 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- 172.21.15.33:0/632044073 wait complete. 2024-09-10T21:08:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- start start 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 Processor -- start 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d53054d0 0x55c6d53116d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d530eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5311d80 0x55c6d530f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6d50cc700 con 0x55c6d5311d80 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c6d5206160 con 0x55c6d53054d0 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c6d5314820 con 0x55c6d5305df0 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.088+0000 7fcbc9228740 10 client.0 _ll_register_callbacks cb 0x55c6d521f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbc53f0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d53054d0 0x55c6d53116d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbc53f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5311d80 0x55c6d530f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d530eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d530eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46390/0 (socket says 172.21.15.33:46390) 2024-09-10T21:08:39.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 -- 172.21.15.33:0/1283673205 learned_addr learned my addr 172.21.15.33:0/1283673205 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 -- 172.21.15.33:0/1283673205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d53054d0 msgr2=0x55c6d53116d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d53054d0 0x55c6d53116d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 -- 172.21.15.33:0/1283673205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5311d80 msgr2=0x55c6d530f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5311d80 0x55c6d530f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 -- 172.21.15.33:0/1283673205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d5233c90 con 0x55c6d5305df0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbbffff640 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d530eb20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbac012c90 tx=0x7fcbac00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b301e3c4b61c6ac1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcb9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcb9ffff640 1 -- 172.21.15.33:0/1283673205 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010430 con 0x55c6d5305df0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcb9ffff640 1 -- 172.21.15.33:0/1283673205 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcbac0105d0 con 0x55c6d5305df0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.089+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbb8036030 con 0x55c6d5305df0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcb9ffff640 1 -- 172.21.15.33:0/1283673205 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcbac0107d0 con 0x55c6d5305df0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 10 client.25732 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c6d5320260 con 0x55c6d5305df0 2024-09-10T21:08:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcb9ffff640 1 -- 172.21.15.33:0/1283673205 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbb8036030 con 0x55c6d5305df0 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcb9ffff640 1 -- 172.21.15.33:0/1283673205 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fcbac01c960 con 0x55c6d5305df0 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 10 client.25732 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 10 client.25732 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 20 client.25732 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d5321730 0x55c6d5323b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 4 client.25732 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d527ddf0 con 0x55c6d5321730 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.090+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d5321730 0x55c6d5323b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.091+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d5321730 0x55c6d5323b20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e9d1768146340643 server_cookie=554c76cfc8e82638 in_seq=0 out_seq=0 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.091+0000 7fcb9ffff640 10 client.25732 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.091+0000 7fcb9ffff640 1 -- 172.21.15.33:0/1283673205 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c6d527ddf0 con 0x55c6d5321730 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.091+0000 7fcb9ffff640 10 client.25732 handle_command_reply: tid=0 2024-09-10T21:08:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.091+0000 7fcbc9228740 1 client.25732 shutdown 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 msgr2=0x55c6d530eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d530eb20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcbac012c90 tx=0x7fcbac00f790 comp rx=0 tx=0).stop 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 shutdown_connections 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5311d80 0x55c6d530f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d5321730 0x55c6d5323b20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d5305df0 0x55c6d530eb20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 --2- 172.21.15.33:0/1283673205 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d53054d0 0x55c6d53116d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcb9ffff640 0 client.25732 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 >> 172.21.15.33:0/1283673205 conn(0x55c6d5303300 msgr2=0x55c6d5308e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 shutdown_connections 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 1 -- 172.21.15.33:0/1283673205 wait complete. 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.092+0000 7fcbc9228740 20 client.25732 trim_cache size 0 max 16384 2024-09-10T21:08:39.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.105+0000 7fcbc9228740 1 Processor -- start 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.105+0000 7fcbc9228740 1 -- start start 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d5302c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d530d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d530d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc9228740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01b0520 con 0x55c6d5305df0 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc9228740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01b2060 con 0x55c6d5311d80 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc9228740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01b0bf0 con 0x55c6d53054d0 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d530d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d530d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54160/0 (socket says 172.21.15.33:54160) 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 -- 172.21.15.33:0/2364600082 learned_addr learned my addr 172.21.15.33:0/2364600082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d5302c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d530d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 -- 172.21.15.33:0/2364600082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 msgr2=0x55c6d5302c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d5302c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 -- 172.21.15.33:0/2364600082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 msgr2=0x55c6d530d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d530d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 -- 172.21.15.33:0/2364600082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d5324150 con 0x55c6d5305df0 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d530d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbbffff640 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d530d3a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0127e0 tx=0x7fcbac026610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5b1ee451289c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbaa7fc640 1 -- 172.21.15.33:0/2364600082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac010430 con 0x55c6d5305df0 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbaa7fc640 1 -- 172.21.15.33:0/2364600082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbac010e20 con 0x55c6d5305df0 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.106+0000 7fcbaa7fc640 1 -- 172.21.15.33:0/2364600082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac005620 con 0x55c6d5305df0 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 -- 172.21.15.33:0/2364600082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 msgr2=0x55c6d530d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d530d3a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0127e0 tx=0x7fcbac026610 comp rx=0 tx=0).stop 2024-09-10T21:08:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 -- 172.21.15.33:0/2364600082 shutdown_connections 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d530d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d530d3a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 --2- 172.21.15.33:0/2364600082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d5302c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 -- 172.21.15.33:0/2364600082 >> 172.21.15.33:0/2364600082 conn(0x55c6d5303300 msgr2=0x55c6d5309d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 -- 172.21.15.33:0/2364600082 shutdown_connections 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 -- 172.21.15.33:0/2364600082 wait complete. 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 -- start start 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.107+0000 7fcbc9228740 1 Processor -- start 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d530c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d5319ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc9228740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5319ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc9228740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01b1990 con 0x55c6d5305df0 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc9228740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01b12c0 con 0x55c6d5311d80 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc9228740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc01ae4e0 con 0x55c6d53054d0 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc9228740 10 client.0 _ll_register_callbacks cb 0x7fcbac009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d5319ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d5319ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54162/0 (socket says 172.21.15.33:54162) 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 -- 172.21.15.33:0/619291460 learned_addr learned my addr 172.21.15.33:0/619291460 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d530c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5319ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 -- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 msgr2=0x55c6d530c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d530c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 -- 172.21.15.33:0/619291460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 msgr2=0x55c6d5319ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5319ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 -- 172.21.15.33:0/619291460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d531a5e0 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d530c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcbbffff640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d5319ab0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcbac012c90 tx=0x7fcbac012880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688456a66e99f315 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.108+0000 7fcb9ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcb9ffff640 1 -- 172.21.15.33:0/619291460 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbac0054a0 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcb9ffff640 1 -- 172.21.15.33:0/619291460 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcbac02c040 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbac010e20 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcb9ffff640 1 -- 172.21.15.33:0/619291460 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fcbac005640 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 10 client.16191 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55c6d538f590 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcb9ffff640 1 -- 172.21.15.33:0/619291460 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbac010e20 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcb9ffff640 1 -- 172.21.15.33:0/619291460 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fcbac01c990 con 0x55c6d5305df0 2024-09-10T21:08:39.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 10 client.16191 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 10 client.16191 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 20 client.16191 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d538fb00 0x55c6d5391f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 4 client.16191 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c6d52798a0 con 0x55c6d538fb00 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.109+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d538fb00 0x55c6d5391f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc53f0640 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d538fb00 0x55c6d5391f20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=baec79740a844475 server_cookie=9e378f9c2a8f0693 in_seq=0 out_seq=0 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcb9ffff640 10 client.16191 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcb9ffff640 1 -- 172.21.15.33:0/619291460 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c6d52798a0 con 0x55c6d538fb00 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcb9ffff640 10 client.16191 handle_command_reply: tid=0 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 client.16191 shutdown 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 msgr2=0x55c6d5319ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d5319ab0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcbac012c90 tx=0x7fcbac012880 comp rx=0 tx=0).stop 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 shutdown_connections 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c6d5311d80 0x55c6d5319ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c6d5305df0 0x55c6d5319ab0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c6d538fb00 0x55c6d5391f20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 --2- 172.21.15.33:0/619291460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c6d53054d0 0x55c6d530c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 >> 172.21.15.33:0/619291460 conn(0x55c6d5303300 msgr2=0x55c6d5309d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.110+0000 7fcb9ffff640 0 client.16191 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 shutdown_connections 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc9228740 1 -- 172.21.15.33:0/619291460 wait complete. 2024-09-10T21:08:39.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc9228740 20 client.16191 trim_cache size 0 max 16384 2024-09-10T21:08:39.133 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:08:39.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcbb00b52e0 msgr2=0x7fcbb00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcbb00b52e0 0x7fcbb00b77a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400e9c0 tx=0x7fcbb40e9950 comp rx=0 tx=0).stop 2024-09-10T21:08:39.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbc0075540 msgr2=0x7fcbc019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbc0075540 0x7fcbc019b440 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcbb400e9f0 tx=0x7fcbb400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 shutdown_connections 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcbb00b52e0 0x7fcbb00b77a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbc010f3e0 0x7fcbc01a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0075f40 0x7fcbc019b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 --2- 172.21.15.33:0/1010809526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbc0075540 0x7fcbc019b440 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 >> 172.21.15.33:0/1010809526 conn(0x7fcbc00fe830 msgr2=0x7fcbc01115e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.111+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 shutdown_connections 2024-09-10T21:08:39.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:39.112+0000 7fcbc6e53640 1 -- 172.21.15.33:0/1010809526 wait complete. 2024-09-10T21:08:39.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 441034 10.73 MB/sec execute 100 sec latency 10.066 ms 2024-09-10T21:08:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:39 smithi174 ceph-mon[35635]: pgmap v486: 97 pgs: 97 active+clean; 177 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-09-10T21:08:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:39 smithi033 ceph-mon[30860]: pgmap v486: 97 pgs: 97 active+clean; 177 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-09-10T21:08:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:39 smithi104 ceph-mon[35325]: pgmap v486: 97 pgs: 97 active+clean; 177 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.0 MiB/s wr, 348 op/s 2024-09-10T21:08:40.308 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:08:40.577 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:40.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 443764 10.79 MB/sec execute 101 sec latency 11.177 ms 2024-09-10T21:08:41.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 -- 172.21.15.33:0/3168849330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae4101c90 msgr2=0x7fdae4102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/3168849330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae4101c90 0x7fdae4102070 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009900 tx=0x7fdad402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 -- 172.21.15.33:0/3168849330 shutdown_connections 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/3168849330 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4108020 0x7fdae410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/3168849330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae41076a0 0x7fdae4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/3168849330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae4101c90 0x7fdae4102070 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 -- 172.21.15.33:0/3168849330 >> 172.21.15.33:0/3168849330 conn(0x7fdae40fd700 msgr2=0x7fdae40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 -- 172.21.15.33:0/3168849330 shutdown_connections 2024-09-10T21:08:41.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.315+0000 7fdae9c7f640 1 -- 172.21.15.33:0/3168849330 wait complete. 2024-09-10T21:08:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 Processor -- start 2024-09-10T21:08:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 -- start start 2024-09-10T21:08:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4101c90 0x7fdae4199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae41076a0 0x7fdae4199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae4108020 0x7fdae419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdae4111ce0 con 0x7fdae41076a0 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdae4111b60 con 0x7fdae4101c90 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.316+0000 7fdae9c7f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdae4111e60 con 0x7fdae4108020 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae41076a0 0x7fdae4199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4101c90 0x7fdae4199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4101c90 0x7fdae4199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50062/0 (socket says 172.21.15.33:50062) 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 -- 172.21.15.33:0/4064069902 learned_addr learned my addr 172.21.15.33:0/4064069902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae3fff640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae4108020 0x7fdae419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 -- 172.21.15.33:0/4064069902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae4108020 msgr2=0x7fdae419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae4108020 0x7fdae419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 -- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae41076a0 msgr2=0x7fdae4199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae41076a0 0x7fdae4199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 -- 172.21.15.33:0/4064069902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae419f3d0 con 0x7fdae4101c90 2024-09-10T21:08:41.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae41076a0 0x7fdae4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae37fe640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4101c90 0x7fdae4199320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdad4005940 tx=0x7fdad40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae0ff9640 1 -- 172.21.15.33:0/4064069902 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad403f030 con 0x7fdae4101c90 2024-09-10T21:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae419f660 con 0x7fdae4101c90 2024-09-10T21:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.317+0000 7fdae0ff9640 1 -- 172.21.15.33:0/4064069902 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdad4003cb0 con 0x7fdae4101c90 2024-09-10T21:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.318+0000 7fdae0ff9640 1 -- 172.21.15.33:0/4064069902 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4035070 con 0x7fdae4101c90 2024-09-10T21:08:41.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.318+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae41a5d80 con 0x7fdae4101c90 2024-09-10T21:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 Processor -- start 2024-09-10T21:08:41.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 -- start start 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f1023fc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102408840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f10240b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdae0ff9640 1 -- 172.21.15.33:0/4064069902 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdad403bbf0 con 0x7fdae4101c90 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f1023fc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f102372ea0 con 0x55f102408d80 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f1023622d0 con 0x55f1023fc4d0 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdaec054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1023660e0 con 0x55f1023fcdf0 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f1023fc8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50068/0 (socket says 172.21.15.33:50068) 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdae3fff640 1 -- 172.21.15.33:0/52881757 learned_addr learned my addr 172.21.15.33:0/52881757 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f10240b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdae37fe640 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102408840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.321+0000 7fdae3fff640 1 -- 172.21.15.33:0/52881757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 msgr2=0x55f102408840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae3fff640 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102408840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae3fff640 1 -- 172.21.15.33:0/52881757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 msgr2=0x55f10240b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae3fff640 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f10240b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae3fff640 1 -- 172.21.15.33:0/52881757 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1020c0650 con 0x55f1023fc4d0 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae37fe640 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102408840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae3fff640 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f1023fc8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0039f0 tx=0x7fdadc003df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c179038159c95d3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae0ff9640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdacc0b52e0 0x7fdacc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdac9ffb640 1 -- 172.21.15.33:0/52881757 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc01a470 con 0x55f1023fc4d0 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdac9ffb640 1 -- 172.21.15.33:0/52881757 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdadc015070 con 0x55f1023fc4d0 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdac9ffb640 1 -- 172.21.15.33:0/52881757 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc011880 con 0x55f1023fc4d0 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae37fe640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdacc0b52e0 0x7fdacc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.322+0000 7fdae0ff9640 1 -- 172.21.15.33:0/4064069902 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdad41066a0 con 0x7fdae4101c90 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 -- 172.21.15.33:0/52881757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 msgr2=0x55f1023fc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdae37fe640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdacc0b52e0 0x7fdacc0b77a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdad4006010 tx=0x7fdad40045a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f1023fc8b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdadc0039f0 tx=0x7fdadc003df0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 -- 172.21.15.33:0/52881757 shutdown_connections 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f10240b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102408840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 --2- 172.21.15.33:0/52881757 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f1023fc8b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 -- 172.21.15.33:0/52881757 >> 172.21.15.33:0/52881757 conn(0x55f1023fa300 msgr2=0x55f1023fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 -- 172.21.15.33:0/52881757 shutdown_connections 2024-09-10T21:08:41.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 -- 172.21.15.33:0/52881757 wait complete. 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.323+0000 7fdaec054740 1 -- start start 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 1 Processor -- start 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102408640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102405a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f102406070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f1021c3700 con 0x55f102408d80 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f1022fd160 con 0x55f1023fc4d0 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f10240ba00 con 0x55f1023fcdf0 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdaec054740 10 client.0 _ll_register_callbacks cb 0x55f1023169e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f102406070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102405a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f102406070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39172/0 (socket says 172.21.15.33:39172) 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102405a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:43512/0 (socket says 172.21.15.33:43512) 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae3fff640 1 -- 172.21.15.33:0/2750108646 learned_addr learned my addr 172.21.15.33:0/2750108646 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae3fff640 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102408640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae2ffd640 1 -- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 msgr2=0x55f102408640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102408640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae2ffd640 1 -- 172.21.15.33:0/2750108646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 msgr2=0x55f102406070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f102406070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae2ffd640 1 -- 172.21.15.33:0/2750108646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f10232ac90 con 0x55f1023fcdf0 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.324+0000 7fdae3fff640 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f102406070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102405a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005fd0 tx=0x7fdad80039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54623278a46cb2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:08:41.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdac2ffd640 1 -- 172.21.15.33:0/2750108646 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8011650 con 0x55f1023fcdf0 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdac2ffd640 1 -- 172.21.15.33:0/2750108646 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdad80117f0 con 0x55f1023fcdf0 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdadc015070 con 0x55f1023fcdf0 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdac2ffd640 1 -- 172.21.15.33:0/2750108646 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdad80119f0 con 0x55f1023fcdf0 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdaec054740 10 client.25750 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f102417360 con 0x55f1023fcdf0 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.325+0000 7fdac2ffd640 1 -- 172.21.15.33:0/2750108646 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdadc015070 con 0x55f1023fcdf0 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdac2ffd640 1 -- 172.21.15.33:0/2750108646 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fdad801bc40 con 0x55f1023fcdf0 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdaec054740 10 client.25750 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdaec054740 10 client.25750 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdaec054740 20 client.25750 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdaec054740 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f102418830 0x55f10241ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdaec054740 4 client.25750 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:41.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f102374df0 con 0x55f102418830 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdae3fff640 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f102418830 0x55f10241ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdac2ffd640 10 client.25750 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.326+0000 7fdae3fff640 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f102418830 0x55f10241ac20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77a82258a02efa10 server_cookie=4f5dda8bf7f8287a in_seq=0 out_seq=0 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdac2ffd640 1 -- 172.21.15.33:0/2750108646 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f102374df0 con 0x55f102418830 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdac2ffd640 10 client.25750 handle_command_reply: tid=0 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 client.25750 shutdown 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 msgr2=0x55f102405a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102405a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdad8005fd0 tx=0x7fdad80039c0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 shutdown_connections 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f102418830 0x55f10241ac20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f102408d80 0x55f102406070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f102405a90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdac2ffd640 0 client.25750 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:08:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 --2- 172.21.15.33:0/2750108646 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102408640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 >> 172.21.15.33:0/2750108646 conn(0x55f1023fa300 msgr2=0x55f1023ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 shutdown_connections 2024-09-10T21:08:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 1 -- 172.21.15.33:0/2750108646 wait complete. 2024-09-10T21:08:41.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.327+0000 7fdaec054740 20 client.25750 trim_cache size 0 max 16384 2024-09-10T21:08:41.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 Processor -- start 2024-09-10T21:08:41.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 -- start start 2024-09-10T21:08:41.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fc4d0 0x55f102414c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f1023fba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f102408d80 0x55f10240a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41ae470 con 0x55f1023fc4d0 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41affb0 con 0x55f102408d80 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdaec054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41aeb40 con 0x55f1023fcdf0 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fc4d0 0x55f102414c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fc4d0 0x55f102414c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39186/0 (socket says 172.21.15.33:39186) 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdae3fff640 1 -- 172.21.15.33:0/569857571 learned_addr learned my addr 172.21.15.33:0/569857571 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.342+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f1023fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae3fff640 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f102408d80 0x55f10240a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae2ffd640 1 -- 172.21.15.33:0/569857571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f102408d80 msgr2=0x55f10240a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f102408d80 0x55f10240a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae2ffd640 1 -- 172.21.15.33:0/569857571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fc4d0 msgr2=0x55f102414c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fc4d0 0x55f102414c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae2ffd640 1 -- 172.21.15.33:0/569857571 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1023fafe0 con 0x55f1023fcdf0 2024-09-10T21:08:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae3fff640 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fc4d0 0x55f102414c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f1023fba60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdad8000c00 tx=0x7fdad8025900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d6ab6667c3189bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdac97fa640 1 -- 172.21.15.33:0/569857571 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad8011650 con 0x55f1023fcdf0 2024-09-10T21:08:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdac97fa640 1 -- 172.21.15.33:0/569857571 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdad8025e20 con 0x55f1023fcdf0 2024-09-10T21:08:41.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.343+0000 7fdac97fa640 1 -- 172.21.15.33:0/569857571 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad80052a0 con 0x55f1023fcdf0 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 -- 172.21.15.33:0/569857571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 msgr2=0x55f1023fba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f1023fba60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdad8000c00 tx=0x7fdad8025900 comp rx=0 tx=0).stop 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 -- 172.21.15.33:0/569857571 shutdown_connections 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f102408d80 0x55f10240a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1023fcdf0 0x55f1023fba60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 --2- 172.21.15.33:0/569857571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fc4d0 0x55f102414c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 -- 172.21.15.33:0/569857571 >> 172.21.15.33:0/569857571 conn(0x55f1023fa300 msgr2=0x55f1023fb350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:41.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 -- 172.21.15.33:0/569857571 shutdown_connections 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 -- 172.21.15.33:0/569857571 wait complete. 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.344+0000 7fdaec054740 1 -- start start 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 1 Processor -- start 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102410c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fcdf0 0x55f1024111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f102408d80 0x55f102401cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41af8e0 con 0x55f1023fcdf0 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41af210 con 0x55f1023fc4d0 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdae41ac430 con 0x55f102408d80 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdaec054740 10 client.0 _ll_register_callbacks cb 0x7fdad8004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fcdf0 0x55f1024111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102410c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102410c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50100/0 (socket says 172.21.15.33:50100) 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 -- 172.21.15.33:0/3833534701 learned_addr learned my addr 172.21.15.33:0/3833534701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f102408d80 0x55f102401cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 -- 172.21.15.33:0/3833534701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f102408d80 msgr2=0x55f102401cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f102408d80 0x55f102401cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 -- 172.21.15.33:0/3833534701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fcdf0 msgr2=0x55f1024111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fcdf0 0x55f1024111d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae3fff640 1 -- 172.21.15.33:0/3833534701 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f102411960 con 0x55f1023fc4d0 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.345+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fcdf0 0x55f1024111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdae3fff640 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102410c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdadc005fd0 tx=0x7fdadc01c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9988161604a60d63 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdac2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdac2ffd640 1 -- 172.21.15.33:0/3833534701 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc03b060 con 0x55f1023fc4d0 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdac2ffd640 1 -- 172.21.15.33:0/3833534701 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdadc011570 con 0x55f1023fc4d0 2024-09-10T21:08:41.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdad8025e20 con 0x55f1023fc4d0 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdac2ffd640 1 -- 172.21.15.33:0/3833534701 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fdadc023040 con 0x55f1023fc4d0 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 10 client.25772 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f102403250 con 0x55f1023fc4d0 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdac2ffd640 1 -- 172.21.15.33:0/3833534701 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdad8025e20 con 0x55f1023fc4d0 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdac2ffd640 1 -- 172.21.15.33:0/3833534701 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fdadc029040 con 0x55f1023fc4d0 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 10 client.25772 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 10 client.25772 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 20 client.25772 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f1024037a0 0x55f1024886e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 4 client.25772 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:08:41.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.346+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1023708a0 con 0x55f1024037a0 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f1024037a0 0x55f1024886e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdac2ffd640 10 client.25772 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdae2ffd640 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f1024037a0 0x55f1024886e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=791d75ad15bd4f7c server_cookie=4cdff1a15162d2a3 in_seq=0 out_seq=0 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdac2ffd640 1 -- 172.21.15.33:0/3833534701 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1023708a0 con 0x55f1024037a0 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdac2ffd640 10 client.25772 handle_command_reply: tid=0 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 client.25772 shutdown 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 msgr2=0x55f102410c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102410c90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdadc005fd0 tx=0x7fdadc01c040 comp rx=0 tx=0).stop 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 shutdown_connections 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f102408d80 0x55f102401cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55f1024037a0 0x55f1024886e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f1023fcdf0 0x55f1024111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdac2ffd640 0 client.25772 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:08:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 --2- 172.21.15.33:0/3833534701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f1023fc4d0 0x55f102410c90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.347+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 >> 172.21.15.33:0/3833534701 conn(0x55f1023fa300 msgr2=0x55f1024070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 shutdown_connections 2024-09-10T21:08:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdaec054740 1 -- 172.21.15.33:0/3833534701 wait complete. 2024-09-10T21:08:41.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdaec054740 20 client.25772 trim_cache size 0 max 16384 2024-09-10T21:08:41.358 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdacc0b52e0 msgr2=0x7fdacc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdacc0b52e0 0x7fdacc0b77a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdad4006010 tx=0x7fdad40045a0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4101c90 msgr2=0x7fdae4199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4101c90 0x7fdae4199320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fdad4005940 tx=0x7fdad40042f0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 shutdown_connections 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdacc0b52e0 0x7fdacc0b77a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdae4108020 0x7fdae419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdae41076a0 0x7fdae4199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 --2- 172.21.15.33:0/4064069902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdae4101c90 0x7fdae4199320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:41.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 >> 172.21.15.33:0/4064069902 conn(0x7fdae40fd700 msgr2=0x7fdae406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:41.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.348+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 shutdown_connections 2024-09-10T21:08:41.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:41.349+0000 7fdae9c7f640 1 -- 172.21.15.33:0/4064069902 wait complete. 2024-09-10T21:08:41.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 445859 10.75 MB/sec execute 102 sec latency 35.256 ms 2024-09-10T21:08:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:41 smithi174 ceph-mon[35635]: pgmap v487: 97 pgs: 97 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 339 op/s 2024-09-10T21:08:41.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:41 smithi033 ceph-mon[30860]: pgmap v487: 97 pgs: 97 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 339 op/s 2024-09-10T21:08:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:41 smithi104 ceph-mon[35325]: pgmap v487: 97 pgs: 97 active+clean; 179 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.9 MiB/s wr, 339 op/s 2024-09-10T21:08:42.185 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:08:42.449 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:42.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 447623 10.77 MB/sec execute 103 sec latency 20.436 ms 2024-09-10T21:08:42.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 -- 172.21.15.33:0/1690391419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 msgr2=0x7f47e810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 --2- 172.21.15.33:0/1690391419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e810e820 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f47e0009ca0 tx=0x7f47e002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 -- 172.21.15.33:0/1690391419 shutdown_connections 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 --2- 172.21.15.33:0/1690391419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e810e820 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 --2- 172.21.15.33:0/1690391419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e81076a0 0x7f47e8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 --2- 172.21.15.33:0/1690391419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47e8101c90 0x7f47e8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 -- 172.21.15.33:0/1690391419 >> 172.21.15.33:0/1690391419 conn(0x7f47e80fd700 msgr2=0x7f47e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 -- 172.21.15.33:0/1690391419 shutdown_connections 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.982+0000 7f47edb24640 1 -- 172.21.15.33:0/1690391419 wait complete. 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 Processor -- start 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 -- start start 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47e8101c90 0x7f47e8199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e81076a0 0x7f47e81998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:42.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111cf0 con 0x7f47e8108020 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111b70 con 0x7f47e8101c90 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47edb24640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e8111e70 con 0x7f47e81076a0 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e819ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39218/0 (socket says 172.21.15.33:39218) 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47e8101c90 0x7f47e8199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47e7fff640 1 -- 172.21.15.33:0/500709851 learned_addr learned my addr 172.21.15.33:0/500709851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.983+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e81076a0 0x7f47e81998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e7fff640 1 -- 172.21.15.33:0/500709851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e81076a0 msgr2=0x7f47e81998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e7fff640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e81076a0 0x7f47e81998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e7fff640 1 -- 172.21.15.33:0/500709851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47e8101c90 msgr2=0x7f47e8199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e7fff640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47e8101c90 0x7f47e8199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e7fff640 1 -- 172.21.15.33:0/500709851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e819f420 con 0x7f47e8108020 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e77fe640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47e8101c90 0x7f47e8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e7fff640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e819ec50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47e002f540 tx=0x7f47e00316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e4ff9640 1 -- 172.21.15.33:0/500709851 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e0003b70 con 0x7f47e8108020 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e819f6b0 con 0x7f47e8108020 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e4ff9640 1 -- 172.21.15.33:0/500709851 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47e0003d10 con 0x7f47e8108020 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47e4ff9640 1 -- 172.21.15.33:0/500709851 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e002f990 con 0x7f47e8108020 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.984+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e81a5dd0 con 0x7f47e8108020 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47e4ff9640 1 -- 172.21.15.33:0/500709851 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f47e002fb30 con 0x7f47e8108020 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 Processor -- start 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 -- start start 2024-09-10T21:08:43.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30bcd80 0x55eeb30bf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eeb3026ea0 con 0x55eeb30b04d0 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eeb30162d0 con 0x55eeb30bcd80 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47efef9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eeb301a0e0 con 0x55eeb30b0df0 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39222/0 (socket says 172.21.15.33:39222) 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47e6ffd640 1 -- 172.21.15.33:0/2731953895 learned_addr learned my addr 172.21.15.33:0/2731953895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47e77fe640 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30bcd80 0x55eeb30bf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.987+0000 7f47e4ff9640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f47d00b52e0 0x7f47d00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e77fe640 1 -- 172.21.15.33:0/2731953895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30bcd80 msgr2=0x55eeb30bf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e77fe640 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30bcd80 0x55eeb30bf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e77fe640 1 -- 172.21.15.33:0/2731953895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 msgr2=0x55eeb30b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e77fe640 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e7fff640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f47d00b52e0 0x7f47d00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e77fe640 1 -- 172.21.15.33:0/2731953895 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb2d74650 con 0x55eeb30b0df0 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e4ff9640 1 -- 172.21.15.33:0/500709851 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f47e0106d60 con 0x7f47e8108020 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e77fe640 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30bc840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009fd0 tx=0x7f47d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=876ff4eaf0b36366 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47e7fff640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f47d00b52e0 0x7f47d00b77a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47e00cd4b0 tx=0x7f47e0091da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47cdffb640 1 -- 172.21.15.33:0/2731953895 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8004030 con 0x55eeb30b0df0 2024-09-10T21:08:43.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47cdffb640 1 -- 172.21.15.33:0/2731953895 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d8005020 con 0x55eeb30b0df0 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.988+0000 7f47cdffb640 1 -- 172.21.15.33:0/2731953895 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8004cb0 con 0x55eeb30b0df0 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 -- 172.21.15.33:0/2731953895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 msgr2=0x55eeb30bc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30bc840 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47d8009fd0 tx=0x7f47d80118d0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 -- 172.21.15.33:0/2731953895 shutdown_connections 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30bcd80 0x55eeb30bf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30bc840 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 --2- 172.21.15.33:0/2731953895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 -- 172.21.15.33:0/2731953895 >> 172.21.15.33:0/2731953895 conn(0x55eeb30ae300 msgr2=0x55eeb30af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 -- 172.21.15.33:0/2731953895 shutdown_connections 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 -- 172.21.15.33:0/2731953895 wait complete. 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.989+0000 7f47efef9740 1 -- start start 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 1 Processor -- start 2024-09-10T21:08:43.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b04d0 0x55eeb30bc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30b9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30bcd80 0x55eeb30ba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2e77700 con 0x55eeb30bcd80 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eeb2fb1160 con 0x55eeb30b04d0 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eeb30bfa00 con 0x55eeb30b0df0 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47efef9740 10 client.0 _ll_register_callbacks cb 0x55eeb2fca9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30bcd80 0x55eeb30ba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30b9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30bcd80 0x55eeb30ba080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39236/0 (socket says 172.21.15.33:39236) 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e6ffd640 1 -- 172.21.15.33:0/1641136468 learned_addr learned my addr 172.21.15.33:0/1641136468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b04d0 0x55eeb30bc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e77fe640 1 -- 172.21.15.33:0/1641136468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b04d0 msgr2=0x55eeb30bc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e77fe640 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b04d0 0x55eeb30bc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e77fe640 1 -- 172.21.15.33:0/1641136468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30bcd80 msgr2=0x55eeb30ba080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e77fe640 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30bcd80 0x55eeb30ba080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e77fe640 1 -- 172.21.15.33:0/1641136468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb2fdec90 con 0x55eeb30b0df0 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.990+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30bcd80 0x55eeb30ba080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47e77fe640 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30b9aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47d801b040 tx=0x7f47d8009d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574931a32f9063e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47c6ffd640 1 -- 172.21.15.33:0/1641136468 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8028020 con 0x55eeb30b0df0 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47c6ffd640 1 -- 172.21.15.33:0/1641136468 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f47d8016040 con 0x55eeb30b0df0 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d8005020 con 0x55eeb30b0df0 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47c6ffd640 1 -- 172.21.15.33:0/1641136468 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f47d8005c50 con 0x55eeb30b0df0 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47efef9740 10 client.25768 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:43.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55eeb30cb340 con 0x55eeb30b0df0 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47c6ffd640 1 -- 172.21.15.33:0/1641136468 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d8005020 con 0x55eeb30b0df0 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47c6ffd640 1 -- 172.21.15.33:0/1641136468 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f47d8003800 con 0x55eeb30b0df0 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47efef9740 10 client.25768 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.991+0000 7f47efef9740 10 client.25768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.992+0000 7f47efef9740 20 client.25768 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.992+0000 7f47efef9740 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb30cc750 0x55eeb30ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.992+0000 7f47efef9740 4 client.25768 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.992+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eeb3028df0 con 0x55eeb30cc750 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.992+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb30cc750 0x55eeb30ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.992+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb30cc750 0x55eeb30ceb40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2d54e19109f718b6 server_cookie=c2f802bd451ada1c in_seq=0 out_seq=0 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.992+0000 7f47c6ffd640 10 client.25768 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:08:43.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47c6ffd640 1 -- 172.21.15.33:0/1641136468 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eeb3028df0 con 0x55eeb30cc750 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47c6ffd640 10 client.25768 handle_command_reply: tid=0 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 client.25768 shutdown 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 msgr2=0x55eeb30b9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30b9aa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47d801b040 tx=0x7f47d8009d20 comp rx=0 tx=0).stop 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 shutdown_connections 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb30cc750 0x55eeb30ceb40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30bcd80 0x55eeb30ba080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30b0df0 0x55eeb30b9aa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47c6ffd640 0 client.25768 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 --2- 172.21.15.33:0/1641136468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b04d0 0x55eeb30bc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 >> 172.21.15.33:0/1641136468 conn(0x55eeb30ae300 msgr2=0x55eeb30b3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 shutdown_connections 2024-09-10T21:08:43.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.993+0000 7f47efef9740 1 -- 172.21.15.33:0/1641136468 wait complete. 2024-09-10T21:08:43.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:42.994+0000 7f47efef9740 20 client.25768 trim_cache size 0 max 16384 2024-09-10T21:08:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 Processor -- start 2024-09-10T21:08:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 -- start start 2024-09-10T21:08:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30adc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30be170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81ac1c0 con 0x55eeb30b04d0 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81add00 con 0x55eeb30b0df0 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47efef9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81ac890 con 0x55eeb30bcd80 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39248/0 (socket says 172.21.15.33:39248) 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47e6ffd640 1 -- 172.21.15.33:0/3011508399 learned_addr learned my addr 172.21.15.33:0/3011508399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30be170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47e77fe640 1 -- 172.21.15.33:0/3011508399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 msgr2=0x55eeb30be170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.007+0000 7f47e77fe640 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30be170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47e77fe640 1 -- 172.21.15.33:0/3011508399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 msgr2=0x55eeb30b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47e77fe640 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47e77fe640 1 -- 172.21.15.33:0/3011508399 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eeb30aed80 con 0x55eeb30b0df0 2024-09-10T21:08:43.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:43.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47e77fe640 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30adc10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47d8003dc0 tx=0x7f47d8009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c7367c6c777335c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:43.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47cd7fa640 1 -- 172.21.15.33:0/3011508399 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8028040 con 0x55eeb30b0df0 2024-09-10T21:08:43.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47cd7fa640 1 -- 172.21.15.33:0/3011508399 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d8016030 con 0x55eeb30b0df0 2024-09-10T21:08:43.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.008+0000 7f47cd7fa640 1 -- 172.21.15.33:0/3011508399 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8019450 con 0x55eeb30b0df0 2024-09-10T21:08:43.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 -- 172.21.15.33:0/3011508399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 msgr2=0x55eeb30adc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:43.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30adc10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f47d8003dc0 tx=0x7f47d8009b90 comp rx=0 tx=0).stop 2024-09-10T21:08:43.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 -- 172.21.15.33:0/3011508399 shutdown_connections 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30be170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30adc10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 --2- 172.21.15.33:0/3011508399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 -- 172.21.15.33:0/3011508399 >> 172.21.15.33:0/3011508399 conn(0x55eeb30ae300 msgr2=0x55eeb30af800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 -- 172.21.15.33:0/3011508399 shutdown_connections 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 -- 172.21.15.33:0/3011508399 wait complete. 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 -- start start 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.009+0000 7f47efef9740 1 Processor -- start 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47efef9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30c5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47efef9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30d40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47efef9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30d4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47efef9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81ad630 con 0x55eeb30b04d0 2024-09-10T21:08:43.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47efef9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81acf60 con 0x55eeb30b0df0 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47efef9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47e81aa180 con 0x55eeb30bcd80 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47efef9740 10 client.0 _ll_register_callbacks cb 0x7f47dc009a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30d40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39262/0 (socket says 172.21.15.33:39262) 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30d40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50152/0 (socket says 172.21.15.33:50152) 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 -- 172.21.15.33:0/225362734 learned_addr learned my addr 172.21.15.33:0/225362734 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30d4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 -- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 msgr2=0x55eeb30d4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30d4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 -- 172.21.15.33:0/225362734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 msgr2=0x55eeb30d40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30d40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 -- 172.21.15.33:0/225362734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eeb30c10d0 con 0x55eeb30b04d0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e77fe640 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47e6ffd640 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30c5660 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47dc004480 tx=0x7f47dc009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beee090fff749fa9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.010+0000 7f47c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47c6ffd640 1 -- 172.21.15.33:0/225362734 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc005220 con 0x55eeb30b04d0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47c6ffd640 1 -- 172.21.15.33:0/225362734 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f47dc0053c0 con 0x55eeb30b04d0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d8016030 con 0x55eeb30b04d0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47c6ffd640 1 -- 172.21.15.33:0/225362734 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f47dc0055c0 con 0x55eeb30b04d0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 10 client.16227 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55eeb31397a0 con 0x55eeb30b04d0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47c6ffd640 1 -- 172.21.15.33:0/225362734 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47d8016030 con 0x55eeb30b04d0 2024-09-10T21:08:43.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47c6ffd640 1 -- 172.21.15.33:0/225362734 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f47dc013a40 con 0x55eeb30b04d0 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 10 client.16227 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 10 client.16227 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 20 client.16227 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb3139f00 0x55eeb313c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 4 client.16227 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eeb30248a0 con 0x55eeb3139f00 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.011+0000 7f47e77fe640 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb3139f00 0x55eeb313c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47e77fe640 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb3139f00 0x55eeb313c320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8770bb955695ce8a server_cookie=3214765ef27d8b21 in_seq=0 out_seq=0 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47c6ffd640 10 client.16227 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47c6ffd640 1 -- 172.21.15.33:0/225362734 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eeb30248a0 con 0x55eeb3139f00 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47c6ffd640 10 client.16227 handle_command_reply: tid=0 2024-09-10T21:08:43.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 client.16227 shutdown 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 msgr2=0x55eeb30c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30c5660 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f47dc004480 tx=0x7f47dc009b70 comp rx=0 tx=0).stop 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 shutdown_connections 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eeb30bcd80 0x55eeb30d4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eeb30b0df0 0x55eeb30d40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55eeb3139f00 0x55eeb313c320 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 --2- 172.21.15.33:0/225362734 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eeb30b04d0 0x55eeb30c5660 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47c6ffd640 0 client.16227 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 >> 172.21.15.33:0/225362734 conn(0x55eeb30ae300 msgr2=0x55eeb30bb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 shutdown_connections 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.012+0000 7f47efef9740 1 -- 172.21.15.33:0/225362734 wait complete. 2024-09-10T21:08:43.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47efef9740 20 client.16227 trim_cache size 0 max 16384 2024-09-10T21:08:43.023 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:08:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f47d00b52e0 msgr2=0x7f47d00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:43.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f47d00b52e0 0x7f47d00b77a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f47e00cd4b0 tx=0x7f47e0091da0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 msgr2=0x7f47e819ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e819ec50 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f47e002f540 tx=0x7f47e00316f0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 shutdown_connections 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f47d00b52e0 0x7f47d00b77a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47e8108020 0x7f47e819ec50 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47e81076a0 0x7f47e81998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 --2- 172.21.15.33:0/500709851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47e8101c90 0x7f47e8199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 >> 172.21.15.33:0/500709851 conn(0x7f47e80fd700 msgr2=0x7f47e806ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 shutdown_connections 2024-09-10T21:08:43.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:43.013+0000 7f47edb24640 1 -- 172.21.15.33:0/500709851 wait complete. 2024-09-10T21:08:43.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 450334 10.80 MB/sec execute 104 sec latency 38.835 ms 2024-09-10T21:08:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:43 smithi174 ceph-mon[35635]: pgmap v488: 97 pgs: 97 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-09-10T21:08:43.919 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:43 smithi033 ceph-mon[30860]: pgmap v488: 97 pgs: 97 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-09-10T21:08:43.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:08:44.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:43 smithi104 ceph-mon[35325]: pgmap v488: 97 pgs: 97 active+clean; 181 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 304 op/s 2024-09-10T21:08:44.211 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:44.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 451930 10.79 MB/sec execute 105 sec latency 37.153 ms 2024-09-10T21:08:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 -- 172.21.15.33:0/1719260088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd840104530 msgr2=0x7fd84010cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 --2- 172.21.15.33:0/1719260088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd840104530 0x7fd84010cb10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd834009900 tx=0x7fd83402a410 comp rx=0 tx=0).stop 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 -- 172.21.15.33:0/1719260088 shutdown_connections 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 --2- 172.21.15.33:0/1719260088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd84010d0e0 0x7fd84010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 --2- 172.21.15.33:0/1719260088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd840104530 0x7fd84010cb10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 --2- 172.21.15.33:0/1719260088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd840103b30 0x7fd840103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 -- 172.21.15.33:0/1719260088 >> 172.21.15.33:0/1719260088 conn(0x7fd8400fd700 msgr2=0x7fd8400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 -- 172.21.15.33:0/1719260088 shutdown_connections 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 -- 172.21.15.33:0/1719260088 wait complete. 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 Processor -- start 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.805+0000 7fd848080640 1 -- start start 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd848080640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd840103b30 0x7fd84019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd848080640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd840104530 0x7fd84019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd848080640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84010d0e0 0x7fd8401a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd848080640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd840112d60 con 0x7fd84010d0e0 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd848080640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd840112be0 con 0x7fd840103b30 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd848080640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd840112ee0 con 0x7fd840104530 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd84661d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84010d0e0 0x7fd8401a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd845e1c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd840103b30 0x7fd84019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd84661d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84010d0e0 0x7fd8401a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39274/0 (socket says 172.21.15.33:39274) 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd84661d640 1 -- 172.21.15.33:0/326045298 learned_addr learned my addr 172.21.15.33:0/326045298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd84561b640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd840104530 0x7fd84019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd845e1c640 1 -- 172.21.15.33:0/326045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd840104530 msgr2=0x7fd84019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd845e1c640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd840104530 0x7fd84019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd845e1c640 1 -- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84010d0e0 msgr2=0x7fd8401a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd845e1c640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84010d0e0 0x7fd8401a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd845e1c640 1 -- 172.21.15.33:0/326045298 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8401a1560 con 0x7fd840103b30 2024-09-10T21:08:44.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd84661d640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84010d0e0 0x7fd8401a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.806+0000 7fd845e1c640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd840103b30 0x7fd84019b4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd830005fd0 tx=0x7fd83000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.807+0000 7fd83effd640 1 -- 172.21.15.33:0/326045298 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830009c30 con 0x7fd840103b30 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.807+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8401a1850 con 0x7fd840103b30 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.807+0000 7fd83effd640 1 -- 172.21.15.33:0/326045298 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd830009dd0 con 0x7fd840103b30 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.807+0000 7fd83effd640 1 -- 172.21.15.33:0/326045298 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830003e20 con 0x7fd840103b30 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.807+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8401a80e0 con 0x7fd840103b30 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.809+0000 7fd84a455740 1 Processor -- start 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.809+0000 7fd84a455740 1 -- start start 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.809+0000 7fd83effd640 1 -- 172.21.15.33:0/326045298 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd830004000 con 0x7fd840103b30 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.809+0000 7fd84a455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c54d0 0x56448e4c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.809+0000 7fd84a455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84a455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84a455740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56448e43bea0 con 0x56448e4c5df0 2024-09-10T21:08:44.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84a455740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56448e42b2d0 con 0x56448e4c54d0 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84a455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56448e42f0e0 con 0x56448e4d1d80 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd845e1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84661d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c54d0 0x56448e4c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd845e1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4d1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39290/0 (socket says 172.21.15.33:39290) 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd845e1c640 1 -- 172.21.15.33:0/2557715780 learned_addr learned my addr 172.21.15.33:0/2557715780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84561b640 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84661d640 1 -- 172.21.15.33:0/2557715780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 msgr2=0x56448e4d4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84661d640 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84661d640 1 -- 172.21.15.33:0/2557715780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 msgr2=0x56448e4d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84661d640 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84661d640 1 -- 172.21.15.33:0/2557715780 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448e189650 con 0x56448e4c54d0 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd845e1c640 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:44.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd84661d640 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c54d0 0x56448e4c58b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd838011780 tx=0x7fd838011c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17333a9a6bcc599c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.810+0000 7fd827fff640 1 -- 172.21.15.33:0/2557715780 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838004d20 con 0x56448e4c54d0 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd827fff640 1 -- 172.21.15.33:0/2557715780 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83800fdf0 con 0x56448e4c54d0 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd827fff640 1 -- 172.21.15.33:0/2557715780 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8380057a0 con 0x56448e4c54d0 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd83effd640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8280b52e0 0x7fd8280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd845e1c640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8280b52e0 0x7fd8280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd83effd640 1 -- 172.21.15.33:0/326045298 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd8300e5710 con 0x7fd840103b30 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 -- 172.21.15.33:0/2557715780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c54d0 msgr2=0x56448e4c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c54d0 0x56448e4c58b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd838011780 tx=0x7fd838011c40 comp rx=0 tx=0).stop 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 -- 172.21.15.33:0/2557715780 shutdown_connections 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd845e1c640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8280b52e0 0x7fd8280b77a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd830065bc0 tx=0x7fd8300eb530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 --2- 172.21.15.33:0/2557715780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c54d0 0x56448e4c58b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 -- 172.21.15.33:0/2557715780 >> 172.21.15.33:0/2557715780 conn(0x56448e4c3300 msgr2=0x56448e4c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 -- 172.21.15.33:0/2557715780 shutdown_connections 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.811+0000 7fd84a455740 1 -- 172.21.15.33:0/2557715780 wait complete. 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 -- start start 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 Processor -- start 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56448e28c700 con 0x56448e4c54d0 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56448e3c6160 con 0x56448e4c5df0 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56448e4d4a00 con 0x56448e4d1d80 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84a455740 10 client.0 _ll_register_callbacks cb 0x56448e3df9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84661d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84661d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39304/0 (socket says 172.21.15.33:39304) 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84661d640 1 -- 172.21.15.33:0/1018266968 learned_addr learned my addr 172.21.15.33:0/1018266968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.812+0000 7fd84561b640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84661d640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84661d640 1 -- 172.21.15.33:0/1018266968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 msgr2=0x56448e4cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84661d640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4cefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84661d640 1 -- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 msgr2=0x56448e4cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84661d640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84661d640 1 -- 172.21.15.33:0/1018266968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448e3f3c90 con 0x56448e4c54d0 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84561b640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84661d640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d15f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd838005c10 tx=0x7fd8380053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a439f52922c2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd8257fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd8257fa640 1 -- 172.21.15.33:0/1018266968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83801f040 con 0x56448e4c54d0 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd8257fa640 1 -- 172.21.15.33:0/1018266968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd838004c40 con 0x56448e4c54d0 2024-09-10T21:08:44.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83800fdf0 con 0x56448e4c54d0 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd8257fa640 1 -- 172.21.15.33:0/1018266968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd838004e40 con 0x56448e4c54d0 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84a455740 10 client.16239 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.813+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56448e4e02b0 con 0x56448e4c54d0 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd8257fa640 1 -- 172.21.15.33:0/1018266968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd83800fdf0 con 0x56448e4c54d0 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd8257fa640 1 -- 172.21.15.33:0/1018266968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd83800f040 con 0x56448e4c54d0 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84a455740 10 client.16239 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84a455740 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84a455740 20 client.16239 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84a455740 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e4e1600 0x56448e4e39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84a455740 4 client.16239 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56448e43ddf0 con 0x56448e4e1600 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84561b640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e4e1600 0x56448e4e39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd84561b640 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e4e1600 0x56448e4e39f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6a2a2af444641e2 server_cookie=88fe56d43ff5ed58 in_seq=0 out_seq=0 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.814+0000 7fd8257fa640 10 client.16239 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd8257fa640 1 -- 172.21.15.33:0/1018266968 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56448e43ddf0 con 0x56448e4e1600 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd8257fa640 10 client.16239 handle_command_reply: tid=0 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 client.16239 shutdown 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 msgr2=0x56448e4d15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d15f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd838005c10 tx=0x7fd8380053c0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 shutdown_connections 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e4e1600 0x56448e4e39f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4cefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd8257fa640 0 client.16239 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 --2- 172.21.15.33:0/1018266968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d15f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 >> 172.21.15.33:0/1018266968 conn(0x56448e4c3300 msgr2=0x56448e4c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 shutdown_connections 2024-09-10T21:08:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 1 -- 172.21.15.33:0/1018266968 wait complete. 2024-09-10T21:08:44.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.816+0000 7fd84a455740 20 client.16239 trim_cache size 0 max 16384 2024-09-10T21:08:44.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.830+0000 7fd84a455740 1 Processor -- start 2024-09-10T21:08:44.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.830+0000 7fd84a455740 1 -- start start 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84a455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4c54d0 0x56448e4ddc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84a455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4c2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84a455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4d1d80 0x56448e4cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84a455740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b0630 con 0x56448e4c5df0 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84a455740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b2170 con 0x56448e4d1d80 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84a455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b0d00 con 0x56448e4c54d0 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39316/0 (socket says 172.21.15.33:39316) 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 -- 172.21.15.33:0/1841950593 learned_addr learned my addr 172.21.15.33:0/1841950593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84661d640 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4d1d80 0x56448e4cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84661d640 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4c54d0 0x56448e4ddc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 -- 172.21.15.33:0/1841950593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4c54d0 msgr2=0x56448e4ddc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4c54d0 0x56448e4ddc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 -- 172.21.15.33:0/1841950593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4d1d80 msgr2=0x56448e4cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4d1d80 0x56448e4cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 -- 172.21.15.33:0/1841950593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448e4d1220 con 0x56448e4c5df0 2024-09-10T21:08:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd84561b640 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4c2310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd834009900 tx=0x7fd834005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad6e41658d6213f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd8277fe640 1 -- 172.21.15.33:0/1841950593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834035040 con 0x56448e4c5df0 2024-09-10T21:08:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd8277fe640 1 -- 172.21.15.33:0/1841950593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8340054f0 con 0x56448e4c5df0 2024-09-10T21:08:44.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.831+0000 7fd8277fe640 1 -- 172.21.15.33:0/1841950593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834004cb0 con 0x56448e4c5df0 2024-09-10T21:08:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 -- 172.21.15.33:0/1841950593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 msgr2=0x56448e4c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4c2310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd834009900 tx=0x7fd834005c00 comp rx=0 tx=0).stop 2024-09-10T21:08:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 -- 172.21.15.33:0/1841950593 shutdown_connections 2024-09-10T21:08:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4d1d80 0x56448e4cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c5df0 0x56448e4c2310 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 --2- 172.21.15.33:0/1841950593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4c54d0 0x56448e4ddc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 -- 172.21.15.33:0/1841950593 >> 172.21.15.33:0/1841950593 conn(0x56448e4c3300 msgr2=0x56448e4c1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 -- 172.21.15.33:0/1841950593 shutdown_connections 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.832+0000 7fd84a455740 1 -- 172.21.15.33:0/1841950593 wait complete. 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 -- start start 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 Processor -- start 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4da1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b1aa0 con 0x56448e4c54d0 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401b13d0 con 0x56448e4c5df0 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401ae5f0 con 0x56448e4d1d80 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84a455740 10 client.0 _ll_register_callbacks cb 0x7fd8300eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:44.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84661d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84561b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4da1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84561b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4da1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50218/0 (socket says 172.21.15.33:50218) 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84561b640 1 -- 172.21.15.33:0/1530346249 learned_addr learned my addr 172.21.15.33:0/1530346249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.833+0000 7fd84661d640 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84561b640 1 -- 172.21.15.33:0/1530346249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 msgr2=0x56448e4d6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84561b640 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84561b640 1 -- 172.21.15.33:0/1530346249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 msgr2=0x56448e4d9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84561b640 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84561b640 1 -- 172.21.15.33:0/1530346249 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448e4d71a0 con 0x56448e4c5df0 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84661d640 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84561b640 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4da1d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd83402a930 tx=0x7fd834004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa0c82878e6bcd49 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd8257fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd8257fa640 1 -- 172.21.15.33:0/1530346249 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834053070 con 0x56448e4c5df0 2024-09-10T21:08:44.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd8257fa640 1 -- 172.21.15.33:0/1530346249 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd8340035f0 con 0x56448e4c5df0 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8340054f0 con 0x56448e4c5df0 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd8257fa640 1 -- 172.21.15.33:0/1530346249 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7fd83405f050 con 0x56448e4c5df0 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84a455740 10 client.25808 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.834+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56448e54eac0 con 0x56448e4c5df0 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd8257fa640 1 -- 172.21.15.33:0/1530346249 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8340054f0 con 0x56448e4c5df0 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd8257fa640 1 -- 172.21.15.33:0/1530346249 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7fd834004ef0 con 0x56448e4c5df0 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84a455740 10 client.25808 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84a455740 10 client.25808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84a455740 20 client.25808 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84a455740 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e54f1c0 0x56448e5515e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84a455740 4 client.25808 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56448e4398a0 con 0x56448e54f1c0 2024-09-10T21:08:44.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84661d640 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e54f1c0 0x56448e5515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd84661d640 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e54f1c0 0x56448e5515e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b040e67828b4c8ab server_cookie=8777f9d18d86c18f in_seq=0 out_seq=0 2024-09-10T21:08:44.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.835+0000 7fd8257fa640 10 client.25808 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.836+0000 7fd8257fa640 1 -- 172.21.15.33:0/1530346249 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56448e4398a0 con 0x56448e54f1c0 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.836+0000 7fd8257fa640 10 client.25808 handle_command_reply: tid=0 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.836+0000 7fd84a455740 1 client.25808 shutdown 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 msgr2=0x56448e4da1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4da1d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd83402a930 tx=0x7fd834004750 comp rx=0 tx=0).stop 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 shutdown_connections 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56448e54f1c0 0x56448e5515e0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56448e4d1d80 0x56448e4d6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56448e4c5df0 0x56448e4da1d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd8257fa640 0 client.25808 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 --2- 172.21.15.33:0/1530346249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56448e4c54d0 0x56448e4d9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 >> 172.21.15.33:0/1530346249 conn(0x56448e4c3300 msgr2=0x56448e4d0890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 shutdown_connections 2024-09-10T21:08:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 1 -- 172.21.15.33:0/1530346249 wait complete. 2024-09-10T21:08:44.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd84a455740 20 client.25808 trim_cache size 0 max 16384 2024-09-10T21:08:44.846 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8280b52e0 msgr2=0x7fd8280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.837+0000 7fd848080640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8280b52e0 0x7fd8280b77a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd830065bc0 tx=0x7fd8300eb530 comp rx=0 tx=0).stop 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd840103b30 msgr2=0x7fd84019b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd840103b30 0x7fd84019b4b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd830005fd0 tx=0x7fd83000dde0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 shutdown_connections 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd8280b52e0 0x7fd8280b77a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd84010d0e0 0x7fd8401a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd840104530 0x7fd84019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 --2- 172.21.15.33:0/326045298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd840103b30 0x7fd84019b4b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:44.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 >> 172.21.15.33:0/326045298 conn(0x7fd8400fd700 msgr2=0x7fd84010f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:44.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 shutdown_connections 2024-09-10T21:08:44.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:44.838+0000 7fd848080640 1 -- 172.21.15.33:0/326045298 wait complete. 2024-09-10T21:08:45.722 INFO:tasks.workunit.client.0.smithi033.stdout: 1 454400 10.82 MB/sec execute 106 sec latency 11.394 ms 2024-09-10T21:08:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:45 smithi174 ceph-mon[35635]: pgmap v489: 97 pgs: 97 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-10T21:08:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:45 smithi033 ceph-mon[30860]: pgmap v489: 97 pgs: 97 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-10T21:08:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:45 smithi104 ceph-mon[35325]: pgmap v489: 97 pgs: 97 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 363 op/s 2024-09-10T21:08:46.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 456506 10.79 MB/sec execute 107 sec latency 27.301 ms 2024-09-10T21:08:46.752 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:08:46.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:08:47.021 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.494+0000 7f81f1738640 1 -- 172.21.15.33:0/204941287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec104590 msgr2=0x7f81ec10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.494+0000 7f81f1738640 1 --2- 172.21.15.33:0/204941287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec104590 0x7f81ec10cb70 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009900 tx=0x7f81d402a410 comp rx=0 tx=0).stop 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.494+0000 7f81f1738640 1 -- 172.21.15.33:0/204941287 shutdown_connections 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.494+0000 7f81f1738640 1 --2- 172.21.15.33:0/204941287 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec10d140 0x7f81ec10f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.494+0000 7f81f1738640 1 --2- 172.21.15.33:0/204941287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec104590 0x7f81ec10cb70 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.494+0000 7f81f1738640 1 --2- 172.21.15.33:0/204941287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81ec103b90 0x7f81ec103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.494+0000 7f81f1738640 1 -- 172.21.15.33:0/204941287 >> 172.21.15.33:0/204941287 conn(0x7f81ec0fd700 msgr2=0x7f81ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.495+0000 7f81f1738640 1 -- 172.21.15.33:0/204941287 shutdown_connections 2024-09-10T21:08:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.495+0000 7f81f1738640 1 -- 172.21.15.33:0/204941287 wait complete. 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.495+0000 7f81f1738640 1 Processor -- start 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.495+0000 7f81f1738640 1 -- start start 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81f1738640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81ec103b90 0x7f81ec199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81f1738640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec104590 0x7f81ec1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81f1738640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec10d140 0x7f81ec19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81f1738640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec112d80 con 0x7f81ec10d140 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81f1738640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec112c00 con 0x7f81ec104590 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81f1738640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec112f00 con 0x7f81ec103b90 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec10d140 0x7f81ec19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81ea7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec104590 0x7f81ec1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec10d140 0x7f81ec19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39336/0 (socket says 172.21.15.33:39336) 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 -- 172.21.15.33:0/3042282463 learned_addr learned my addr 172.21.15.33:0/3042282463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:47.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eaffd640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81ec103b90 0x7f81ec199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 -- 172.21.15.33:0/3042282463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81ec103b90 msgr2=0x7f81ec199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81ec103b90 0x7f81ec199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 -- 172.21.15.33:0/3042282463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec104590 msgr2=0x7f81ec1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec104590 0x7f81ec1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 -- 172.21.15.33:0/3042282463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81ec19f340 con 0x7f81ec10d140 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec104590 0x7f81ec1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81eb7fe640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec10d140 0x7f81ec19eb70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00eb10 tx=0x7f81dc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81e3fff640 1 -- 172.21.15.33:0/3042282463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc004b20 con 0x7f81ec10d140 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.496+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81ec19f630 con 0x7f81ec10d140 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.497+0000 7f81e3fff640 1 -- 172.21.15.33:0/3042282463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81dc004cc0 con 0x7f81ec10d140 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.497+0000 7f81e3fff640 1 -- 172.21.15.33:0/3042282463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc010670 con 0x7f81ec10d140 2024-09-10T21:08:47.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.497+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81ec1a5cf0 con 0x7f81ec10d140 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.499+0000 7f81f3b0d740 1 Processor -- start 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.499+0000 7f81e3fff640 1 -- 172.21.15.33:0/3042282463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f81dc005020 con 0x7f81ec10d140 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.499+0000 7f81f3b0d740 1 -- start start 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23eee0 0x55734d23f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23f800 0x55734d24f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d251f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55734d1b5e60 con 0x55734d23eee0 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55734d1a5290 con 0x55734d24fba0 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55734d1a90a0 con 0x55734d23f800 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23eee0 0x55734d23f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23eee0 0x55734d23f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39348/0 (socket says 172.21.15.33:39348) 2024-09-10T21:08:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 -- 172.21.15.33:0/2276483557 learned_addr learned my addr 172.21.15.33:0/2276483557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81eaffd640 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23f800 0x55734d24f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d251f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 -- 172.21.15.33:0/2276483557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23f800 msgr2=0x55734d24f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23f800 0x55734d24f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 -- 172.21.15.33:0/2276483557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 msgr2=0x55734d251f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d251f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81ea7fc640 1 -- 172.21.15.33:0/2276483557 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55734cf03930 con 0x55734d23eee0 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.500+0000 7f81eaffd640 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23f800 0x55734d24f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.501+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23eee0 0x55734d23f2c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009900 tx=0x7f81d4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9e0de29ccb38897 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.501+0000 7f81e3fff640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d80b52e0 0x7f81d80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.501+0000 7f81e17fa640 1 -- 172.21.15.33:0/2276483557 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4035040 con 0x55734d23eee0 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.501+0000 7f81eaffd640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d80b52e0 0x7f81d80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.501+0000 7f81e3fff640 1 -- 172.21.15.33:0/3042282463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f81dc0e6200 con 0x7f81ec10d140 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.501+0000 7f81e17fa640 1 -- 172.21.15.33:0/2276483557 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d4045070 con 0x55734d23eee0 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.501+0000 7f81e17fa640 1 -- 172.21.15.33:0/2276483557 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d403a3f0 con 0x55734d23eee0 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2276483557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23eee0 msgr2=0x55734d23f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23eee0 0x55734d23f2c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f81d4009900 tx=0x7f81d4032f40 comp rx=0 tx=0).stop 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81eaffd640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d80b52e0 0x7f81d80b77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55734d242060 tx=0x7f81e400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2276483557 shutdown_connections 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d251f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23f800 0x55734d24f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2276483557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23eee0 0x55734d23f2c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2276483557 >> 172.21.15.33:0/2276483557 conn(0x55734d23cd10 msgr2=0x55734d23deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:47.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2276483557 shutdown_connections 2024-09-10T21:08:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2276483557 wait complete. 2024-09-10T21:08:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 -- start start 2024-09-10T21:08:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.502+0000 7f81f3b0d740 1 Processor -- start 2024-09-10T21:08:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23eee0 0x55734d24caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d24fba0 0x55734d24d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55734d006890 con 0x55734d23f800 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55734d13fd40 con 0x55734d23eee0 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55734d2527f0 con 0x55734d24fba0 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 10 client.0 _ll_register_callbacks cb 0x55734d1a3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81ea7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23eee0 0x55734d24caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81eaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23eee0 0x55734d24caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50266/0 (socket says 172.21.15.33:50266) 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81eb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d24fba0 0x55734d24d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81eaffd640 1 -- 172.21.15.33:0/2249423947 learned_addr learned my addr 172.21.15.33:0/2249423947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81ea7fc640 1 -- 172.21.15.33:0/2249423947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d24fba0 msgr2=0x55734d24d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d24fba0 0x55734d24d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81ea7fc640 1 -- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23eee0 msgr2=0x55734d24caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23eee0 0x55734d24caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81ea7fc640 1 -- 172.21.15.33:0/2249423947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55734d25bb80 con 0x55734d23f800 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81eaffd640 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23eee0 0x55734d24caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24d030 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f81d4032fd0 tx=0x7f81d4032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b59a57d67b00e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81d27fc640 1 -- 172.21.15.33:0/2249423947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d404e060 con 0x55734d23f800 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81d27fc640 1 -- 172.21.15.33:0/2249423947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f81d4005240 con 0x55734d23f800 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.503+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d4045070 con 0x55734d23f800 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81d27fc640 1 -- 172.21.15.33:0/2249423947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f81d4035070 con 0x55734d23f800 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 10 client.16257 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55734d25e530 con 0x55734d23f800 2024-09-10T21:08:47.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81d27fc640 1 -- 172.21.15.33:0/2249423947 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d4045070 con 0x55734d23f800 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81d27fc640 1 -- 172.21.15.33:0/2249423947 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f81d4053070 con 0x55734d23f800 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 10 client.16257 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 20 client.16257 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d25f880 0x55734d261c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 4 client.16257 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55734d1b7db0 con 0x55734d25f880 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.504+0000 7f81eaffd640 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d25f880 0x55734d261c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81eaffd640 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d25f880 0x55734d261c70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=740bea93a47d086a server_cookie=ce2d415d418adfdb in_seq=0 out_seq=0 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81d27fc640 10 client.16257 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81d27fc640 1 -- 172.21.15.33:0/2249423947 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55734d1b7db0 con 0x55734d25f880 2024-09-10T21:08:47.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81d27fc640 10 client.16257 handle_command_reply: tid=0 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 client.16257 shutdown 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 msgr2=0x55734d24d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24d030 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f81d4032fd0 tx=0x7f81d4032ed0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 shutdown_connections 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d25f880 0x55734d261c70 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d24fba0 0x55734d24d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24d030 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81d27fc640 0 client.16257 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/2249423947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23eee0 0x55734d24caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.505+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 >> 172.21.15.33:0/2249423947 conn(0x55734d23cd10 msgr2=0x55734d23bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.506+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 shutdown_connections 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.506+0000 7f81f3b0d740 1 -- 172.21.15.33:0/2249423947 wait complete. 2024-09-10T21:08:47.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.506+0000 7f81f3b0d740 20 client.16257 trim_cache size 0 max 16384 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 Processor -- start 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 -- start start 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d24b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d264c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1ac110 con 0x55734d23f800 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1adc50 con 0x55734d24fba0 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1ac7e0 con 0x55734d23eee0 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81ea7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81eb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d264c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81ea7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39368/0 (socket says 172.21.15.33:39368) 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81ea7fc640 1 -- 172.21.15.33:0/1915724029 learned_addr learned my addr 172.21.15.33:0/1915724029 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.524+0000 7f81eaffd640 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d24b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81ea7fc640 1 -- 172.21.15.33:0/1915724029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 msgr2=0x55734d24b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d24b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81ea7fc640 1 -- 172.21.15.33:0/1915724029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 msgr2=0x55734d264c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d264c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81ea7fc640 1 -- 172.21.15.33:0/1915724029 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55734d23f630 con 0x55734d23f800 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81eb7fe640 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d264c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24b8b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f81d402a930 tx=0x7f81d4031600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f944525483e4dbbf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81e0ff9640 1 -- 172.21.15.33:0/1915724029 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d4059070 con 0x55734d23f800 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81e0ff9640 1 -- 172.21.15.33:0/1915724029 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d4045040 con 0x55734d23f800 2024-09-10T21:08:47.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.525+0000 7f81e0ff9640 1 -- 172.21.15.33:0/1915724029 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81d40040f0 con 0x55734d23f800 2024-09-10T21:08:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1915724029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 msgr2=0x55734d24b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24b8b0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f81d402a930 tx=0x7f81d4031600 comp rx=0 tx=0).stop 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1915724029 shutdown_connections 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d24fba0 0x55734d264c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d23f800 0x55734d24b8b0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1915724029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d24b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1915724029 >> 172.21.15.33:0/1915724029 conn(0x55734d23cd10 msgr2=0x55734d24bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1915724029 shutdown_connections 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1915724029 wait complete. 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- start start 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 Processor -- start 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d261350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23f800 0x55734d261890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d24fba0 0x55734d2cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1ad580 con 0x55734d24fba0 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1aceb0 con 0x55734d23f800 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81ec1aa0d0 con 0x55734d23eee0 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.526+0000 7f81f3b0d740 10 client.0 _ll_register_callbacks cb 0x7f81e400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d24fba0 0x55734d2cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d24fba0 0x55734d2cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39384/0 (socket says 172.21.15.33:39384) 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d261350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 -- 172.21.15.33:0/1533304929 learned_addr learned my addr 172.21.15.33:0/1533304929 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81ea7fc640 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23f800 0x55734d261890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 -- 172.21.15.33:0/1533304929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 msgr2=0x55734d261350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d261350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 -- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23f800 msgr2=0x55734d261890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:47.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23f800 0x55734d261890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 -- 172.21.15.33:0/1533304929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55734d2cb9a0 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eaffd640 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d261350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81eb7fe640 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d24fba0 0x55734d2cb3e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0e3cc0 tx=0x7f81dc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a2ac7d3f4a0a25e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81d27fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81d27fc640 1 -- 172.21.15.33:0/1533304929 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc0c7060 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81d27fc640 1 -- 172.21.15.33:0/1533304929 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f81dc0b5c80 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81d4045040 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81d27fc640 1 -- 172.21.15.33:0/1533304929 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f81dc0ce050 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81f3b0d740 10 client.16263 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.527+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55734d2ccd40 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81d27fc640 1 -- 172.21.15.33:0/1533304929 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81d4045040 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81d27fc640 1 -- 172.21.15.33:0/1533304929 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f81dc0a82d0 con 0x55734d24fba0 2024-09-10T21:08:47.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81f3b0d740 10 client.16263 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81f3b0d740 10 client.16263 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81f3b0d740 20 client.16263 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d2cd4a0 0x55734d2cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81f3b0d740 4 client.16263 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55734d1b3860 con 0x55734d2cd4a0 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81eaffd640 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d2cd4a0 0x55734d2cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81eaffd640 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d2cd4a0 0x55734d2cf8c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c16541a0b810fd server_cookie=2d10999a0566e60a in_seq=0 out_seq=0 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.528+0000 7f81d27fc640 10 client.16263 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81d27fc640 1 -- 172.21.15.33:0/1533304929 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55734d1b3860 con 0x55734d2cd4a0 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81d27fc640 10 client.16263 handle_command_reply: tid=0 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 client.16263 shutdown 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d24fba0 msgr2=0x55734d2cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d24fba0 0x55734d2cb3e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0e3cc0 tx=0x7f81dc005800 comp rx=0 tx=0).stop 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 shutdown_connections 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55734d2cd4a0 0x55734d2cf8c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55734d24fba0 0x55734d2cb3e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81d27fc640 0 client.16263 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55734d23f800 0x55734d261890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 --2- 172.21.15.33:0/1533304929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55734d23eee0 0x55734d261350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 >> 172.21.15.33:0/1533304929 conn(0x55734d23cd10 msgr2=0x55734d24e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 shutdown_connections 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 1 -- 172.21.15.33:0/1533304929 wait complete. 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.530+0000 7f81f3b0d740 20 client.16263 trim_cache size 0 max 16384 2024-09-10T21:08:47.549 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:08:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:08:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "38b6b1ae-0098-4918-bd6a-8bcb16725aa7", 2024-09-10T21:08:47.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:08:47.550 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:08:47.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d80b52e0 msgr2=0x7f81d80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d80b52e0 0x7f81d80b77a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x55734d242060 tx=0x7f81e400a2d0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec10d140 msgr2=0x7f81ec19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec10d140 0x7f81ec19eb70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00eb10 tx=0x7f81dc00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 shutdown_connections 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81d80b52e0 0x7f81d80b77a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f81ec10d140 0x7f81ec19eb70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81ec104590 0x7f81ec1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 --2- 172.21.15.33:0/3042282463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81ec103b90 0x7f81ec199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 >> 172.21.15.33:0/3042282463 conn(0x7f81ec0fd700 msgr2=0x7f81ec0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:47.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 shutdown_connections 2024-09-10T21:08:47.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:47.531+0000 7f81f1738640 1 -- 172.21.15.33:0/3042282463 wait complete. 2024-09-10T21:08:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:47 smithi033 ceph-mon[30860]: pgmap v490: 97 pgs: 97 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-09-10T21:08:47.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 458140 10.80 MB/sec execute 108 sec latency 29.178 ms 2024-09-10T21:08:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:47 smithi174 ceph-mon[35635]: pgmap v490: 97 pgs: 97 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-09-10T21:08:48.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:47 smithi104 ceph-mon[35325]: pgmap v490: 97 pgs: 97 active+clean; 183 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 304 op/s 2024-09-10T21:08:48.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:08:48.616 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:08:48.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 459790 10.94 MB/sec execute 109 sec latency 62.199 ms 2024-09-10T21:08:48.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:08:48.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:48.905 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:48.906 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:48 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:08:49.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:48 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 -- 172.21.15.33:0/2460094798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f08108020 msgr2=0x7f9f0810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 --2- 172.21.15.33:0/2460094798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f08108020 0x7f9f0810e820 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009ca0 tx=0x7f9f0002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 -- 172.21.15.33:0/2460094798 shutdown_connections 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 --2- 172.21.15.33:0/2460094798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f08108020 0x7f9f0810e820 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 --2- 172.21.15.33:0/2460094798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f081076a0 0x7f9f08107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 --2- 172.21.15.33:0/2460094798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f08101c90 0x7f9f08102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 -- 172.21.15.33:0/2460094798 >> 172.21.15.33:0/2460094798 conn(0x7f9f080fd700 msgr2=0x7f9f080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 -- 172.21.15.33:0/2460094798 shutdown_connections 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.096+0000 7f9f0f947640 1 -- 172.21.15.33:0/2460094798 wait complete. 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 Processor -- start 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 -- start start 2024-09-10T21:08:49.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f08101c90 0x7f9f08197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f081076a0 0x7f9f08198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f08108020 0x7f9f0819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f08111ca0 con 0x7f9f081076a0 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f08111b20 con 0x7f9f08108020 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0f947640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f08111e20 con 0x7f9f08101c90 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f081076a0 0x7f9f08198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f081076a0 0x7f9f08198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39404/0 (socket says 172.21.15.33:39404) 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 -- 172.21.15.33:0/443024206 learned_addr learned my addr 172.21.15.33:0/443024206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0dee4640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f08108020 0x7f9f0819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f08101c90 0x7f9f08197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 -- 172.21.15.33:0/443024206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f08101c90 msgr2=0x7f9f08197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f08101c90 0x7f9f08197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 -- 172.21.15.33:0/443024206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f08108020 msgr2=0x7f9f0819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f08108020 0x7f9f0819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0cee2640 1 -- 172.21.15.33:0/443024206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f0819d370 con 0x7f9f081076a0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f08101c90 0x7f9f08197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.097+0000 7f9f0dee4640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f08108020 0x7f9f0819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.098+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f081076a0 0x7f9f08198320 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800e9f0 tx=0x7f9ef800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.098+0000 7f9efe7fc640 1 -- 172.21.15.33:0/443024206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef800cd80 con 0x7f9f081076a0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.098+0000 7f9efe7fc640 1 -- 172.21.15.33:0/443024206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef80108e0 con 0x7f9f081076a0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.098+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f0819d660 con 0x7f9f081076a0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.098+0000 7f9efe7fc640 1 -- 172.21.15.33:0/443024206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8018980 con 0x7f9f081076a0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.098+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f0819dbf0 con 0x7f9f081076a0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9efe7fc640 1 -- 172.21.15.33:0/443024206 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9ef8010430 con 0x7f9f081076a0 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 Processor -- start 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 -- start start 2024-09-10T21:08:49.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617cc480 0x55fe617cc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617ccda0 0x55fe617d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617db120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fe61742e50 con 0x55fe617d8d30 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fe61732280 con 0x55fe617ccda0 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe61736090 con 0x55fe617cc480 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0dee4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617cc480 0x55fe617cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617db120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0d6e3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617ccda0 0x55fe617d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0dee4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617cc480 0x55fe617cc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:43728/0 (socket says 172.21.15.33:43728) 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0dee4640 1 -- 172.21.15.33:0/1506338427 learned_addr learned my addr 172.21.15.33:0/1506338427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/1506338427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617cc480 msgr2=0x55fe617cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/1506338427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617cc480 0x55fe617cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/1506338427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 msgr2=0x55fe617db120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.101+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/1506338427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617db120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/1506338427 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe61490650 con 0x55fe617ccda0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9efe7fc640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ef40b5360 0x7f9ef40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/1506338427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617ccda0 0x55fe617d87f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009c30 tx=0x7f9f040117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffb72a5625a058f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9f0dee4640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ef40b5360 0x7f9ef40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9efe7fc640 1 -- 172.21.15.33:0/443024206 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9ef80e5f90 con 0x7f9f081076a0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9ef37fe640 1 -- 172.21.15.33:0/1506338427 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04012450 con 0x55fe617ccda0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9ef37fe640 1 -- 172.21.15.33:0/1506338427 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f04012aa0 con 0x55fe617ccda0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.102+0000 7f9ef37fe640 1 -- 172.21.15.33:0/1506338427 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f0400fa90 con 0x55fe617ccda0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1506338427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617ccda0 msgr2=0x55fe617d87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f0dee4640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ef40b5360 0x7f9ef40b7820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f00004af0 tx=0x7f9f00005c30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:49.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1506338427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617ccda0 0x55fe617d87f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009c30 tx=0x7f9f040117b0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1506338427 shutdown_connections 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1506338427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617db120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1506338427 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617ccda0 0x55fe617d87f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1506338427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617cc480 0x55fe617cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1506338427 >> 172.21.15.33:0/1506338427 conn(0x55fe617ca2b0 msgr2=0x55fe617cb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1506338427 shutdown_connections 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1506338427 wait complete. 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 -- start start 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.103+0000 7f9f11d1c740 1 Processor -- start 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fe61593700 con 0x55fe617ccda0 2024-09-10T21:08:49.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fe616cd110 con 0x55fe617cc480 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe617db9b0 con 0x55fe617d8d30 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f11d1c740 10 client.0 _ll_register_callbacks cb 0x55fe616e6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0dee4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617d5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39414/0 (socket says 172.21.15.33:39414) 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 -- 172.21.15.33:0/432242444 learned_addr learned my addr 172.21.15.33:0/432242444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 -- 172.21.15.33:0/432242444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 msgr2=0x55fe617d5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 -- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 msgr2=0x55fe617d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 -- 172.21.15.33:0/432242444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe616fac40 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617d5a20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80a99c0 tx=0x7f9ef80e9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f59f40178028bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.104+0000 7f9ef0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.105+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/432242444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef80ac0a0 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.105+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/432242444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9ef80ab9b0 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.105+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f04012aa0 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.105+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/432242444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9ef80abec0 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.105+0000 7f9f11d1c740 10 client.16275 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.105+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fe617e7300 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.105+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/432242444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f04012aa0 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/432242444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f9ef80a80a0 con 0x55fe617ccda0 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f11d1c740 10 client.16275 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:49.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f11d1c740 10 client.16275 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f11d1c740 20 client.16275 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe617e8810 0x55fe617eac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f11d1c740 4 client.16275 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe61744da0 con 0x55fe617e8810 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe617e8810 0x55fe617eac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe617e8810 0x55fe617eac90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875132c9aa782117 server_cookie=6a20762ac9db9ce3 in_seq=0 out_seq=0 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.106+0000 7f9ef0ff9640 10 client.16275 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/432242444 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fe61744da0 con 0x55fe617e8810 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9ef0ff9640 10 client.16275 handle_command_reply: tid=0 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 client.16275 shutdown 2024-09-10T21:08:49.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 msgr2=0x55fe617d5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617d5a20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f9ef80a99c0 tx=0x7f9ef80e9540 comp rx=0 tx=0).stop 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 shutdown_connections 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe617e8810 0x55fe617eac90 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617d5a20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9ef0ff9640 0 client.16275 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/432242444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 >> 172.21.15.33:0/432242444 conn(0x55fe617ca2b0 msgr2=0x55fe617cfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 shutdown_connections 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 1 -- 172.21.15.33:0/432242444 wait complete. 2024-09-10T21:08:49.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.107+0000 7f9f11d1c740 20 client.16275 trim_cache size 0 max 16384 2024-09-10T21:08:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 Processor -- start 2024-09-10T21:08:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 -- start start 2024-09-10T21:08:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081b49d0 con 0x55fe617ccda0 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081b6510 con 0x55fe617cc480 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.125+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081b50a0 con 0x55fe617d8d30 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0d6e3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617c52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39424/0 (socket says 172.21.15.33:39424) 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0cee2640 1 -- 172.21.15.33:0/1403977484 learned_addr learned my addr 172.21.15.33:0/1403977484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0dee4640 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/1403977484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 msgr2=0x55fe617d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/1403977484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 msgr2=0x55fe617c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/1403977484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe617eb310 con 0x55fe617cc480 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:08:49.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e4be0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9f04005830 tx=0x7f9f04009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8974c5ce7cff459 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9ef2ffd640 1 -- 172.21.15.33:0/1403977484 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04009d20 con 0x55fe617cc480 2024-09-10T21:08:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9ef2ffd640 1 -- 172.21.15.33:0/1403977484 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f040054f0 con 0x55fe617cc480 2024-09-10T21:08:49.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.126+0000 7f9ef2ffd640 1 -- 172.21.15.33:0/1403977484 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f04012920 con 0x55fe617cc480 2024-09-10T21:08:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1403977484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 msgr2=0x55fe617e4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e4be0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9f04005830 tx=0x7f9f04009b90 comp rx=0 tx=0).stop 2024-09-10T21:08:49.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1403977484 shutdown_connections 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617d8d30 0x55fe617d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617ccda0 0x55fe617c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/1403977484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e4be0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1403977484 >> 172.21.15.33:0/1403977484 conn(0x55fe617ca2b0 msgr2=0x55fe617cb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1403977484 shutdown_connections 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 -- 172.21.15.33:0/1403977484 wait complete. 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.127+0000 7f9f11d1c740 1 -- start start 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 1 Processor -- start 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617ccda0 0x55fe617f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617f0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081b5e40 con 0x55fe617d8d30 2024-09-10T21:08:49.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081b5770 con 0x55fe617cc480 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f081b2990 con 0x55fe617ccda0 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f11d1c740 10 client.0 _ll_register_callbacks cb 0x7f9ef80e9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617ccda0 0x55fe617f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f0d6e3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f0cee2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617ccda0 0x55fe617f0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:43752/0 (socket says 172.21.15.33:43752) 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f0cee2640 1 -- 172.21.15.33:0/852093146 learned_addr learned my addr 172.21.15.33:0/852093146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.128+0000 7f9f0dee4640 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617f0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/852093146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617ccda0 msgr2=0x55fe617f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617ccda0 0x55fe617f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/852093146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 msgr2=0x55fe617f0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617f0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f0d6e3640 1 -- 172.21.15.33:0/852093146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe617dd080 con 0x55fe617cc480 2024-09-10T21:08:49.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f0d6e3640 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e1610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009c30 tx=0x7f9f04009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=964594e98b52edae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9ef0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/852093146 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f040050b0 con 0x55fe617cc480 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f040054f0 con 0x55fe617cc480 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/852093146 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9f04005250 con 0x55fe617cc480 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/852093146 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f9f04012450 con 0x55fe617cc480 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f11d1c740 10 client.25844 fetch_fsmap learned FSMap version 42 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.129+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55fe61855a20 con 0x55fe617cc480 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/852093146 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f040054f0 con 0x55fe617cc480 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/852093146 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 42) ==== 8362+0+0 (secure 0 0 0) 0x7f9f04003a10 con 0x55fe617cc480 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f11d1c740 10 client.25844 fetch_fsmap finished waiting for FSMap version 42 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f11d1c740 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f11d1c740 20 client.25844 populate_metadata read hostname 'smithi033' 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe61855f70 0x55fe61858390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:08:49.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f11d1c740 4 client.25844 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe61740850 con 0x55fe61855f70 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe61855f70 0x55fe61858390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9ef0ff9640 10 client.25844 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.130+0000 7f9f0cee2640 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe61855f70 0x55fe61858390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb4b420da6910f5b server_cookie=ef460a78819e77e7 in_seq=0 out_seq=0 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9ef0ff9640 1 -- 172.21.15.33:0/852093146 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55fe61740850 con 0x55fe61855f70 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9ef0ff9640 10 client.25844 handle_command_reply: tid=0 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 client.25844 shutdown 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 msgr2=0x55fe617e1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e1610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9f04009c30 tx=0x7f9f04009530 comp rx=0 tx=0).stop 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 shutdown_connections 2024-09-10T21:08:49.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fe61855f70 0x55fe61858390 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fe617d8d30 0x55fe617f0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe617ccda0 0x55fe617f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9ef0ff9640 0 client.25844 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 --2- 172.21.15.33:0/852093146 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fe617cc480 0x55fe617e1610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 >> 172.21.15.33:0/852093146 conn(0x55fe617ca2b0 msgr2=0x55fe617d7050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 shutdown_connections 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 1 -- 172.21.15.33:0/852093146 wait complete. 2024-09-10T21:08:49.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f11d1c740 20 client.25844 trim_cache size 0 max 16384 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "38b6b1ae-0098-4918-bd6a-8bcb16725aa7", 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:08:49.141 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:08:49.142 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:49.143 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:49.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:08:49.145 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:08:49.146 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:08:49.147 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:08:49.147 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:08:49.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:49.147 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:08:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:08:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.131+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ef40b5360 msgr2=0x7f9ef40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ef40b5360 0x7f9ef40b7820 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9f00004af0 tx=0x7f9f00005c30 comp rx=0 tx=0).stop 2024-09-10T21:08:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f081076a0 msgr2=0x7f9f08198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:08:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f081076a0 0x7f9f08198320 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f9ef800e9f0 tx=0x7f9ef800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 shutdown_connections 2024-09-10T21:08:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ef40b5360 0x7f9ef40b7820 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f08108020 0x7f9f0819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f081076a0 0x7f9f08198320 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 --2- 172.21.15.33:0/443024206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f08101c90 0x7f9f08197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:08:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 >> 172.21.15.33:0/443024206 conn(0x7f9f080fd700 msgr2=0x7f9f080fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:08:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 shutdown_connections 2024-09-10T21:08:49.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:08:49.132+0000 7f9f0f947640 1 -- 172.21.15.33:0/443024206 wait complete. 2024-09-10T21:08:49.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:08:49.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:49.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:49.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:49.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:49.150 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.151 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:48 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:08:49.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 462259 10.97 MB/sec execute 110 sec latency 38.694 ms 2024-09-10T21:08:49.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:49 smithi033 ceph-mon[30860]: pgmap v491: 97 pgs: 97 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 341 op/s 2024-09-10T21:08:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:49 smithi104 ceph-mon[35325]: pgmap v491: 97 pgs: 97 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 341 op/s 2024-09-10T21:08:50.075 INFO:tasks.cephfs.filesystem:scrub status for tag:38b6b1ae-0098-4918-bd6a-8bcb16725aa7 - {'path': '/', 'tag': '38b6b1ae-0098-4918-bd6a-8bcb16725aa7', 'options': 'recursive,force'} 2024-09-10T21:08:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:49 smithi174 ceph-mon[35635]: pgmap v491: 97 pgs: 97 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 341 op/s 2024-09-10T21:08:50.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 464675 10.97 MB/sec execute 111 sec latency 12.038 ms 2024-09-10T21:08:50.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:50 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:51.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:50 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:50 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:51.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 466358 10.95 MB/sec execute 112 sec latency 27.125 ms 2024-09-10T21:08:51.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:51 smithi033 ceph-mon[30860]: pgmap v492: 97 pgs: 97 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 312 op/s 2024-09-10T21:08:52.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:51 smithi104 ceph-mon[35325]: pgmap v492: 97 pgs: 97 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 312 op/s 2024-09-10T21:08:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:51 smithi174 ceph-mon[35635]: pgmap v492: 97 pgs: 97 active+clean; 185 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.9 MiB/s wr, 312 op/s 2024-09-10T21:08:52.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 466581 10.86 MB/sec execute 113 sec latency 892.611 ms 2024-09-10T21:08:53.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 468808 10.88 MB/sec execute 114 sec latency 321.366 ms 2024-09-10T21:08:53.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:53 smithi033 ceph-mon[30860]: pgmap v493: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-09-10T21:08:53.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:53 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:53 smithi104 ceph-mon[35325]: pgmap v493: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-09-10T21:08:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:53 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:53 smithi174 ceph-mon[35635]: pgmap v493: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.8 MiB/s wr, 327 op/s 2024-09-10T21:08:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:53 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:54.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 468821 10.78 MB/sec execute 115 sec latency 1065.081 ms 2024-09-10T21:08:55.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 468821 10.69 MB/sec execute 116 sec latency 1240.750 ms 2024-09-10T21:08:55.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:55 smithi033 ceph-mon[30860]: pgmap v494: 97 pgs: 97 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 345 op/s 2024-09-10T21:08:55.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:55 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:55 smithi104 ceph-mon[35325]: pgmap v494: 97 pgs: 97 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 345 op/s 2024-09-10T21:08:56.085 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:55 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:55 smithi174 ceph-mon[35635]: pgmap v494: 97 pgs: 97 active+clean; 176 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.2 MiB/s wr, 345 op/s 2024-09-10T21:08:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:55 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:56.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469053 10.63 MB/sec execute 117 sec latency 1514.929 ms 2024-09-10T21:08:57.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:57.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:57.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:57.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:57.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469202 10.55 MB/sec execute 118 sec latency 903.900 ms 2024-09-10T21:08:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:57 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:08:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:57 smithi104 ceph-mon[35325]: pgmap v495: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-09-10T21:08:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:57 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:57 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:08:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:57 smithi174 ceph-mon[35635]: pgmap v495: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-09-10T21:08:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:57 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:57 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:08:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:57 smithi033 ceph-mon[30860]: pgmap v495: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.5 MiB/s wr, 290 op/s 2024-09-10T21:08:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:57 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:58.723 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469202 10.46 MB/sec execute 119 sec latency 1903.984 ms 2024-09-10T21:08:59.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:58 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:59.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:58 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:59.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:58 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:08:59.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469318 10.37 MB/sec execute 120 sec latency 2651.320 ms 2024-09-10T21:08:59.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:08:59 smithi104 ceph-mon[35325]: pgmap v496: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-09-10T21:09:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:08:59 smithi174 ceph-mon[35635]: pgmap v496: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-09-10T21:09:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:08:59 smithi033 ceph-mon[30860]: pgmap v496: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 304 op/s 2024-09-10T21:09:00.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469318 10.29 MB/sec execute 121 sec latency 1111.437 ms 2024-09-10T21:09:01.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 469366 10.20 MB/sec execute 122 sec latency 1386.385 ms 2024-09-10T21:09:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:01 smithi104 ceph-mon[35325]: pgmap v497: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-10T21:09:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:01 smithi174 ceph-mon[35635]: pgmap v497: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-10T21:09:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:01 smithi033 ceph-mon[30860]: pgmap v497: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 2.6 MiB/s wr, 224 op/s 2024-09-10T21:09:02.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 470061 10.14 MB/sec execute 123 sec latency 700.520 ms 2024-09-10T21:09:03.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:02 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:09:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:02 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:09:03.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:02 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:09:03.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 470061 10.05 MB/sec execute 124 sec latency 1456.687 ms 2024-09-10T21:09:04.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:03 smithi104 ceph-mon[35325]: pgmap v498: 97 pgs: 97 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-09-10T21:09:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:03 smithi174 ceph-mon[35635]: pgmap v498: 97 pgs: 97 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-09-10T21:09:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:03 smithi033 ceph-mon[30860]: pgmap v498: 97 pgs: 97 active+clean; 172 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 514 KiB/s rd, 2.2 MiB/s wr, 207 op/s 2024-09-10T21:09:04.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 471925 10.06 MB/sec execute 125 sec latency 1813.252 ms 2024-09-10T21:09:05.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 472523 10.02 MB/sec execute 126 sec latency 790.766 ms 2024-09-10T21:09:06.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:05 smithi104 ceph-mon[35325]: pgmap v499: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-10T21:09:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:05 smithi174 ceph-mon[35635]: pgmap v499: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-10T21:09:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:05 smithi033 ceph-mon[30860]: pgmap v499: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 2.4 MiB/s wr, 158 op/s 2024-09-10T21:09:06.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 472523 9.94 MB/sec execute 127 sec latency 1790.818 ms 2024-09-10T21:09:07.724 INFO:tasks.workunit.client.0.smithi033.stdout: 1 473320 9.88 MB/sec execute 128 sec latency 2168.894 ms 2024-09-10T21:09:07.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:07 smithi033 ceph-mon[30860]: pgmap v500: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-09-10T21:09:07.979 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:07 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:09:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:07 smithi104 ceph-mon[35325]: pgmap v500: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-09-10T21:09:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:07 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:09:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:07 smithi174 ceph-mon[35635]: pgmap v500: 97 pgs: 97 active+clean; 173 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 2.0 MiB/s wr, 112 op/s 2024-09-10T21:09:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:07 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:09:08.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 475888 9.91 MB/sec execute 129 sec latency 47.507 ms 2024-09-10T21:09:09.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 475888 9.83 MB/sec execute 130 sec latency 1047.827 ms 2024-09-10T21:09:10.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:09 smithi104 ceph-mon[35325]: pgmap v501: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-09-10T21:09:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:09 smithi174 ceph-mon[35635]: pgmap v501: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-09-10T21:09:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:09 smithi033 ceph-mon[30860]: pgmap v501: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 2.3 MiB/s wr, 133 op/s 2024-09-10T21:09:10.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 475888 9.76 MB/sec execute 131 sec latency 2047.859 ms 2024-09-10T21:09:11.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 475888 9.68 MB/sec execute 132 sec latency 3047.964 ms 2024-09-10T21:09:12.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:11 smithi104 ceph-mon[35325]: pgmap v502: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T21:09:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:11 smithi174 ceph-mon[35635]: pgmap v502: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T21:09:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:11 smithi033 ceph-mon[30860]: pgmap v502: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 1.9 MiB/s wr, 113 op/s 2024-09-10T21:09:12.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476242 9.64 MB/sec execute 133 sec latency 3092.723 ms 2024-09-10T21:09:13.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476242 9.57 MB/sec execute 134 sec latency 1544.049 ms 2024-09-10T21:09:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:13 smithi104 ceph-mon[35325]: pgmap v503: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.9 MiB/s wr, 112 op/s 2024-09-10T21:09:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:13 smithi174 ceph-mon[35635]: pgmap v503: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.9 MiB/s wr, 112 op/s 2024-09-10T21:09:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:13 smithi033 ceph-mon[30860]: pgmap v503: 97 pgs: 97 active+clean; 177 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 1.9 MiB/s wr, 112 op/s 2024-09-10T21:09:14.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476242 9.49 MB/sec execute 135 sec latency 2544.161 ms 2024-09-10T21:09:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:09:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:09:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:09:15.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:09:15.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:09:15.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:09:15.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:09:15.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:09:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:09:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:09:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:09:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:09:15.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476242 9.42 MB/sec execute 136 sec latency 3544.273 ms 2024-09-10T21:09:16.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:15 smithi104 ceph-mon[35325]: pgmap v504: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 2.4 MiB/s wr, 152 op/s 2024-09-10T21:09:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:15 smithi174 ceph-mon[35635]: pgmap v504: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 2.4 MiB/s wr, 152 op/s 2024-09-10T21:09:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:15 smithi033 ceph-mon[30860]: pgmap v504: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 259 KiB/s rd, 2.4 MiB/s wr, 152 op/s 2024-09-10T21:09:16.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476242 9.36 MB/sec execute 137 sec latency 4544.381 ms 2024-09-10T21:09:17.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476295 9.30 MB/sec execute 138 sec latency 4965.106 ms 2024-09-10T21:09:18.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:17 smithi104 ceph-mon[35325]: pgmap v505: 97 pgs: 97 active+clean; 175 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-10T21:09:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:17 smithi174 ceph-mon[35635]: pgmap v505: 97 pgs: 97 active+clean; 175 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-10T21:09:18.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:17 smithi033 ceph-mon[30860]: pgmap v505: 97 pgs: 97 active+clean; 175 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 2.1 MiB/s wr, 129 op/s 2024-09-10T21:09:18.725 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476295 9.23 MB/sec execute 139 sec latency 1563.206 ms 2024-09-10T21:09:19.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476324 9.16 MB/sec execute 140 sec latency 1927.998 ms 2024-09-10T21:09:20.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:19 smithi104 ceph-mon[35325]: pgmap v506: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-10T21:09:20.076 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:09:20.105 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:19 smithi033 ceph-mon[30860]: pgmap v506: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-10T21:09:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:19 smithi174 ceph-mon[35635]: pgmap v506: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-09-10T21:09:20.353 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:20.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476324 9.10 MB/sec execute 141 sec latency 1582.677 ms 2024-09-10T21:09:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1265056092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac101cd0 msgr2=0x7fcbac1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1265056092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac101cd0 0x7fcbac1020b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009900 tx=0x7fcb9c02a410 comp rx=0 tx=0).stop 2024-09-10T21:09:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1265056092 shutdown_connections 2024-09-10T21:09:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1265056092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac108060 0x7fcbac10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1265056092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbac1076e0 0x7fcbac107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1265056092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac101cd0 0x7fcbac1020b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1265056092 >> 172.21.15.33:0/1265056092 conn(0x7fcbac0fd700 msgr2=0x7fcbac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1265056092 shutdown_connections 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1265056092 wait complete. 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.841+0000 7fcbb1dec640 1 Processor -- start 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbb1dec640 1 -- start start 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbb1dec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbac101cd0 0x7fcbac0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbb1dec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac1076e0 0x7fcbac0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbb1dec640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac108060 0x7fcbac0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbb1dec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac111d20 con 0x7fcbac1076e0 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbb1dec640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac111ba0 con 0x7fcbac108060 2024-09-10T21:09:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbb1dec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac111ea0 con 0x7fcbac101cd0 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac1076e0 0x7fcbac0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac1076e0 0x7fcbac0fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49444/0 (socket says 172.21.15.33:49444) 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1458813616 learned_addr learned my addr 172.21.15.33:0/1458813616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbabfff640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac108060 0x7fcbac0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbac101cd0 0x7fcbac0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1458813616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbac101cd0 msgr2=0x7fcbac0fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbac101cd0 0x7fcbac0fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1458813616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac108060 msgr2=0x7fcbac0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac108060 0x7fcbac0fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbaaffd640 1 -- 172.21.15.33:0/1458813616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbac1a76e0 con 0x7fcbac1076e0 2024-09-10T21:09:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbabfff640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac108060 0x7fcbac0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:09:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.842+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbac101cd0 0x7fcbac0fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.843+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac1076e0 0x7fcbac0fe140 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c970 tx=0x7fcba000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.843+0000 7fcba8ff9640 1 -- 172.21.15.33:0/1458813616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0007bb0 con 0x7fcbac1076e0 2024-09-10T21:09:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.843+0000 7fcba8ff9640 1 -- 172.21.15.33:0/1458813616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba0007d50 con 0x7fcbac1076e0 2024-09-10T21:09:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.843+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbac1a79d0 con 0x7fcbac1076e0 2024-09-10T21:09:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.843+0000 7fcba8ff9640 1 -- 172.21.15.33:0/1458813616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba000f450 con 0x7fcbac1076e0 2024-09-10T21:09:20.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.843+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbac1a7f10 con 0x7fcbac1076e0 2024-09-10T21:09:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 Processor -- start 2024-09-10T21:09:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 -- start start 2024-09-10T21:09:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb40a0 0x556d01bb4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb49c0 0x556d01bc0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bc2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d01b2ae50 con 0x556d01bb40a0 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d01b1a280 con 0x556d01bc0950 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556d01b1e090 con 0x556d01bb49c0 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb40a0 0x556d01bb4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb40a0 0x556d01bb4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49454/0 (socket says 172.21.15.33:49454) 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbabfff640 1 -- 172.21.15.33:0/841458566 learned_addr learned my addr 172.21.15.33:0/841458566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.846+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb49c0 0x556d01bc0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcbabfff640 1 -- 172.21.15.33:0/841458566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb49c0 msgr2=0x556d01bc0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcbabfff640 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb49c0 0x556d01bc0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcbabfff640 1 -- 172.21.15.33:0/841458566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 msgr2=0x556d01bc2d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:09:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcbabfff640 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bc2d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcbabfff640 1 -- 172.21.15.33:0/841458566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d01878650 con 0x556d01bb40a0 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcbabfff640 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb40a0 0x556d01bb4480 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcba40107a0 tx=0x7fcba4010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12551c1459f8efe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcb91ffb640 1 -- 172.21.15.33:0/841458566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4004d20 con 0x556d01bb40a0 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcb91ffb640 1 -- 172.21.15.33:0/841458566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba400ede0 con 0x556d01bb40a0 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcb91ffb640 1 -- 172.21.15.33:0/841458566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba4005890 con 0x556d01bb40a0 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.847+0000 7fcba8ff9640 1 -- 172.21.15.33:0/1458813616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fcba0004f20 con 0x7fcbac1076e0 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 -- 172.21.15.33:0/841458566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb40a0 msgr2=0x556d01bb4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb40a0 0x556d01bb4480 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fcba40107a0 tx=0x7fcba4010c60 comp rx=0 tx=0).stop 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 -- 172.21.15.33:0/841458566 shutdown_connections 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bc2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb49c0 0x556d01bc0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/841458566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb40a0 0x556d01bb4480 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 -- 172.21.15.33:0/841458566 >> 172.21.15.33:0/841458566 conn(0x556d01bb1ed0 msgr2=0x556d01bb3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 -- 172.21.15.33:0/841458566 shutdown_connections 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 -- 172.21.15.33:0/841458566 wait complete. 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcba8ff9640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb940b5270 0x7fcb940b7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb940b5270 0x7fcb940b7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcbb41c1740 1 -- start start 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.848+0000 7fcba8ff9640 1 -- 172.21.15.33:0/1458813616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcba00e58a0 con 0x7fcbac1076e0 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 1 Processor -- start 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb940b5270 0x7fcb940b7730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c031440 tx=0x7fcb9c0058b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb49c0 0x556d01bbd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bc0950 0x556d01bbdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556d0197b700 con 0x556d01bc0950 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556d01ab5110 con 0x556d01bb49c0 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556d01bc35d0 con 0x556d01bb40a0 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbb41c1740 10 client.0 _ll_register_callbacks cb 0x556d01ace990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bc0950 0x556d01bbdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bc0950 0x556d01bbdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49458/0 (socket says 172.21.15.33:49458) 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 -- 172.21.15.33:0/594752243 learned_addr learned my addr 172.21.15.33:0/594752243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb49c0 0x556d01bbd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 -- 172.21.15.33:0/594752243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 msgr2=0x556d01bc0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 -- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb49c0 msgr2=0x556d01bbd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb49c0 0x556d01bbd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.849+0000 7fcbabfff640 1 -- 172.21.15.33:0/594752243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d01ae2c40 con 0x556d01bc0950 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcbabfff640 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bc0950 0x556d01bbdc10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcba4000f80 tx=0x7fcba4010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e96bdb5dac922e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcb8affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcb8affd640 1 -- 172.21.15.33:0/594752243 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba40055d0 con 0x556d01bc0950 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcb8affd640 1 -- 172.21.15.33:0/594752243 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcba4005770 con 0x556d01bc0950 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba400ede0 con 0x556d01bc0950 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcb8affd640 1 -- 172.21.15.33:0/594752243 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcba4005970 con 0x556d01bc0950 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcbb41c1740 10 client.16293 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556d01bceec0 con 0x556d01bc0950 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.850+0000 7fcb8affd640 1 -- 172.21.15.33:0/594752243 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba400ede0 con 0x556d01bc0950 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcb8affd640 1 -- 172.21.15.33:0/594752243 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7fcba401aa20 con 0x556d01bc0950 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbb41c1740 10 client.16293 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbb41c1740 10 client.16293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbb41c1740 20 client.16293 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01bd02c0 0x556d01bd26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbb41c1740 4 client.16293 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d01b2cda0 con 0x556d01bd02c0 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01bd02c0 0x556d01bd26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcb8affd640 10 client.16293 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.851+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01bd02c0 0x556d01bd26b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26f16de9cd08ed3d server_cookie=96496de80479e2c3 in_seq=0 out_seq=0 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcb8affd640 1 -- 172.21.15.33:0/594752243 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d01b2cda0 con 0x556d01bd02c0 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcb8affd640 10 client.16293 handle_command_reply: tid=0 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 client.16293 shutdown 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bc0950 msgr2=0x556d01bbdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bc0950 0x556d01bbdc10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fcba4000f80 tx=0x7fcba4010eb0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 shutdown_connections 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01bd02c0 0x556d01bd26b0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bc0950 0x556d01bbdc10 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb49c0 0x556d01bbd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/594752243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 >> 172.21.15.33:0/594752243 conn(0x556d01bb1ed0 msgr2=0x556d01bb7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcb8affd640 0 client.16293 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.852+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 shutdown_connections 2024-09-10T21:09:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.853+0000 7fcbb41c1740 1 -- 172.21.15.33:0/594752243 wait complete. 2024-09-10T21:09:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.853+0000 7fcbb41c1740 20 client.16293 trim_cache size 0 max 16384 2024-09-10T21:09:20.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 Processor -- start 2024-09-10T21:09:20.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 -- start start 2024-09-10T21:09:20.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb40a0 0x556d01bcc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bd22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bc0950 0x556d01bb0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b0650 con 0x556d01bb49c0 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b2310 con 0x556d01bb40a0 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b0d80 con 0x556d01bc0950 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bd22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bd22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49464/0 (socket says 172.21.15.33:49464) 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.870+0000 7fcbaaffd640 1 -- 172.21.15.33:0/2953572794 learned_addr learned my addr 172.21.15.33:0/2953572794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb40a0 0x556d01bcc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbabfff640 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bc0950 0x556d01bb0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbaaffd640 1 -- 172.21.15.33:0/2953572794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bc0950 msgr2=0x556d01bb0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bc0950 0x556d01bb0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbaaffd640 1 -- 172.21.15.33:0/2953572794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb40a0 msgr2=0x556d01bcc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb40a0 0x556d01bcc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbaaffd640 1 -- 172.21.15.33:0/2953572794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d01bbfe30 con 0x556d01bb49c0 2024-09-10T21:09:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bd22d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcba00a8630 tx=0x7fcba00a8020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddaa83ee55042db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcb917fa640 1 -- 172.21.15.33:0/2953572794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba001d040 con 0x556d01bb49c0 2024-09-10T21:09:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcb917fa640 1 -- 172.21.15.33:0/2953572794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba00cf0c0 con 0x556d01bb49c0 2024-09-10T21:09:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.871+0000 7fcb917fa640 1 -- 172.21.15.33:0/2953572794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0005240 con 0x556d01bb49c0 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 -- 172.21.15.33:0/2953572794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 msgr2=0x556d01bd22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bd22d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fcba00a8630 tx=0x7fcba00a8020 comp rx=0 tx=0).stop 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 -- 172.21.15.33:0/2953572794 shutdown_connections 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bc0950 0x556d01bb0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bd22d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/2953572794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bb40a0 0x556d01bcc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 -- 172.21.15.33:0/2953572794 >> 172.21.15.33:0/2953572794 conn(0x556d01bb1ed0 msgr2=0x556d01bb0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 -- 172.21.15.33:0/2953572794 shutdown_connections 2024-09-10T21:09:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 -- 172.21.15.33:0/2953572794 wait complete. 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 -- start start 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.872+0000 7fcbb41c1740 1 Processor -- start 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bc8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbb41c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bb9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b1b80 con 0x556d01bb49c0 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1b1480 con 0x556d01bc0950 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbb41c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcbac1ae610 con 0x556d01bb40a0 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbb41c1740 10 client.0 _ll_register_callbacks cb 0x7fcb9c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:20.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bc8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bc8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49470/0 (socket says 172.21.15.33:49470) 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 -- 172.21.15.33:0/484797257 learned_addr learned my addr 172.21.15.33:0/484797257 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbabfff640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bb9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 -- 172.21.15.33:0/484797257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 msgr2=0x556d01bc8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 -- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 msgr2=0x556d01bb9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bb9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 -- 172.21.15.33:0/484797257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d01bb9db0 con 0x556d01bb49c0 2024-09-10T21:09:20.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbabfff640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bb9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcbaaffd640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bc8da0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcba00b50c0 tx=0x7fcba00b0b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c83808768ce2ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcb8affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.873+0000 7fcb8affd640 1 -- 172.21.15.33:0/484797257 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba001d070 con 0x556d01bb49c0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcb8affd640 1 -- 172.21.15.33:0/484797257 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcba00ab4c0 con 0x556d01bb49c0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba00cf0c0 con 0x556d01bb49c0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcb8affd640 1 -- 172.21.15.33:0/484797257 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fcba00ac760 con 0x556d01bb49c0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 10 client.16299 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x556d01bbb0c0 con 0x556d01bb49c0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcb8affd640 1 -- 172.21.15.33:0/484797257 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcba00cf0c0 con 0x556d01bb49c0 2024-09-10T21:09:20.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcb8affd640 1 -- 172.21.15.33:0/484797257 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7fcba00aad30 con 0x556d01bb49c0 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 10 client.16299 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 10 client.16299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 20 client.16299 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01c3df50 0x556d01c40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 4 client.16299 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.874+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556d01b28850 con 0x556d01c3df50 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01c3df50 0x556d01c40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbab7fe640 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01c3df50 0x556d01c40340 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63d76204136cd4d server_cookie=effa0fd2546512b in_seq=0 out_seq=0 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcb8affd640 10 client.16299 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcb8affd640 1 -- 172.21.15.33:0/484797257 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x556d01b28850 con 0x556d01c3df50 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcb8affd640 10 client.16299 handle_command_reply: tid=0 2024-09-10T21:09:20.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 client.16299 shutdown 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 msgr2=0x556d01bc8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bc8da0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fcba00b50c0 tx=0x7fcba00b0b40 comp rx=0 tx=0).stop 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 shutdown_connections 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556d01c3df50 0x556d01c40340 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556d01bc0950 0x556d01bb9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556d01bb49c0 0x556d01bc8da0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 --2- 172.21.15.33:0/484797257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d01bb40a0 0x556d01bc8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcb8affd640 0 client.16299 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.875+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 >> 172.21.15.33:0/484797257 conn(0x556d01bb1ed0 msgr2=0x556d01bbf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 shutdown_connections 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb41c1740 1 -- 172.21.15.33:0/484797257 wait complete. 2024-09-10T21:09:20.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb41c1740 20 client.16299 trim_cache size 0 max 16384 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:09:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:09:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:09:20.894 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb940b5270 msgr2=0x7fcb940b7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb940b5270 0x7fcb940b7730 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c031440 tx=0x7fcb9c0058b0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac1076e0 msgr2=0x7fcbac0fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac1076e0 0x7fcbac0fe140 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c970 tx=0x7fcba000ce30 comp rx=0 tx=0).stop 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 shutdown_connections 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcb940b5270 0x7fcb940b7730 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcbac108060 0x7fcbac0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbac1076e0 0x7fcbac0fe140 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 --2- 172.21.15.33:0/1458813616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcbac101cd0 0x7fcbac0fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 >> 172.21.15.33:0/1458813616 conn(0x7fcbac0fd700 msgr2=0x7fcbac06b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 shutdown_connections 2024-09-10T21:09:20.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:20.876+0000 7fcbb1dec640 1 -- 172.21.15.33:0/1458813616 wait complete. 2024-09-10T21:09:21.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476324 9.03 MB/sec execute 142 sec latency 2582.714 ms 2024-09-10T21:09:21.735 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:09:21.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:09:21.799 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:21 smithi033 ceph-mon[30860]: pgmap v507: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-09-10T21:09:22.049 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:22.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:21 smithi104 ceph-mon[35325]: pgmap v507: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-09-10T21:09:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:21 smithi174 ceph-mon[35635]: pgmap v507: 97 pgs: 97 active+clean; 174 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 169 KiB/s rd, 1.8 MiB/s wr, 115 op/s 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 -- 172.21.15.33:0/210998531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd60101c90 msgr2=0x7efd60102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 --2- 172.21.15.33:0/210998531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd60101c90 0x7efd60102070 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7efd58009900 tx=0x7efd5802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 -- 172.21.15.33:0/210998531 shutdown_connections 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 --2- 172.21.15.33:0/210998531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd60108020 0x7efd6010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 --2- 172.21.15.33:0/210998531 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd601076a0 0x7efd60107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 --2- 172.21.15.33:0/210998531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd60101c90 0x7efd60102070 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 -- 172.21.15.33:0/210998531 >> 172.21.15.33:0/210998531 conn(0x7efd600fd700 msgr2=0x7efd600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 -- 172.21.15.33:0/210998531 shutdown_connections 2024-09-10T21:09:22.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.514+0000 7efd67897640 1 -- 172.21.15.33:0/210998531 wait complete. 2024-09-10T21:09:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 Processor -- start 2024-09-10T21:09:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 -- start start 2024-09-10T21:09:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd60101c90 0x7efd60198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:22.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd601076a0 0x7efd601986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60108020 0x7efd6019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efd60111ce0 con 0x7efd601076a0 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efd60111b60 con 0x7efd60108020 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.515+0000 7efd67897640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efd60111e60 con 0x7efd60101c90 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd601076a0 0x7efd601986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd601076a0 0x7efd601986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49490/0 (socket says 172.21.15.33:49490) 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 -- 172.21.15.33:0/2670027375 learned_addr learned my addr 172.21.15.33:0/2670027375 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd65633640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd60101c90 0x7efd60198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd65e34640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60108020 0x7efd6019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 -- 172.21.15.33:0/2670027375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd60101c90 msgr2=0x7efd60198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd60101c90 0x7efd60198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 -- 172.21.15.33:0/2670027375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60108020 msgr2=0x7efd6019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60108020 0x7efd6019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 -- 172.21.15.33:0/2670027375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6019e220 con 0x7efd601076a0 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd64e32640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd601076a0 0x7efd601986b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d950 tx=0x7efd5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.516+0000 7efd5e7fc640 1 -- 172.21.15.33:0/2670027375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd54017030 con 0x7efd601076a0 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.517+0000 7efd5e7fc640 1 -- 172.21.15.33:0/2670027375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd54004020 con 0x7efd601076a0 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.517+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6019e510 con 0x7efd601076a0 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.517+0000 7efd5e7fc640 1 -- 172.21.15.33:0/2670027375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd540047a0 con 0x7efd601076a0 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.517+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd601a4bd0 con 0x7efd601076a0 2024-09-10T21:09:22.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.518+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd60102070 con 0x7efd601076a0 2024-09-10T21:09:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.519+0000 7efd5e7fc640 1 -- 172.21.15.33:0/2670027375 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efd54009a30 con 0x7efd601076a0 2024-09-10T21:09:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.520+0000 7efd5e7fc640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efd480b52e0 0x7efd480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.520+0000 7efd65633640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efd480b52e0 0x7efd480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:22.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.520+0000 7efd5e7fc640 1 -- 172.21.15.33:0/2670027375 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efd540e5c90 con 0x7efd601076a0 2024-09-10T21:09:22.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.520+0000 7efd65633640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efd480b52e0 0x7efd480b77a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd58031440 tx=0x7efd58005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:22.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.523+0000 7efd5e7fc640 1 -- 172.21.15.33:0/2670027375 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efd540acb10 con 0x7efd601076a0 2024-09-10T21:09:22.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476324 8.97 MB/sec execute 143 sec latency 3583.029 ms 2024-09-10T21:09:22.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.744+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd601992a0 con 0x7efd601076a0 2024-09-10T21:09:22.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.746+0000 7efd5e7fc640 1 -- 172.21.15.33:0/2670027375 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11988 (secure 0 0 0) 0x7efd601992a0 con 0x7efd601076a0 2024-09-10T21:09:22.752 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:09:22.752 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"btime":"2024-09-10T21:09:06:710981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:09:05.985999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:09:22.752 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 49 2024-09-10T21:09:22.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.748+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efd480b52e0 msgr2=0x7efd480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.748+0000 7efd67897640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efd480b52e0 0x7efd480b77a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd58031440 tx=0x7efd58005b50 comp rx=0 tx=0).stop 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.748+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd601076a0 msgr2=0x7efd601986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.748+0000 7efd67897640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd601076a0 0x7efd601986b0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7efd5400d950 tx=0x7efd5400de10 comp rx=0 tx=0).stop 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.748+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 shutdown_connections 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.749+0000 7efd67897640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efd480b52e0 0x7efd480b77a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.749+0000 7efd67897640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efd60108020 0x7efd6019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.749+0000 7efd67897640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efd601076a0 0x7efd601986b0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.749+0000 7efd67897640 1 --2- 172.21.15.33:0/2670027375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efd60101c90 0x7efd60198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.749+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 >> 172.21.15.33:0/2670027375 conn(0x7efd600fd700 msgr2=0x7efd60105940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.749+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 shutdown_connections 2024-09-10T21:09:22.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:22.749+0000 7efd67897640 1 -- 172.21.15.33:0/2670027375 wait complete. 2024-09-10T21:09:23.709 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:09:23.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476324 8.91 MB/sec execute 144 sec latency 4583.102 ms 2024-09-10T21:09:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:23 smithi033 ceph-mon[30860]: pgmap v508: 97 pgs: 97 active+clean; 175 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-09-10T21:09:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:23 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2670027375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:09:23.993 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:23 smithi104 ceph-mon[35325]: pgmap v508: 97 pgs: 97 active+clean; 175 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-09-10T21:09:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:23 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2670027375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:09:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:23 smithi174 ceph-mon[35635]: pgmap v508: 97 pgs: 97 active+clean; 175 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-09-10T21:09:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:23 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2670027375' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:09:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1687408750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78101c90 msgr2=0x7faa78102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1687408750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78101c90 0x7faa78102070 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7faa68009900 tx=0x7faa6802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1687408750 shutdown_connections 2024-09-10T21:09:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1687408750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa78108020 0x7faa7810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1687408750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa781076a0 0x7faa78107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1687408750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa78101c90 0x7faa78102070 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1687408750 >> 172.21.15.33:0/1687408750 conn(0x7faa780fd700 msgr2=0x7faa780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1687408750 shutdown_connections 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.477+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1687408750 wait complete. 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 Processor -- start 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 -- start start 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa78101c90 0x7faa78199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa781076a0 0x7faa781997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa78108020 0x7faa7819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112e00 con 0x7faa781076a0 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112c80 con 0x7faa78108020 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa7e0d2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa78112f80 con 0x7faa78101c90 2024-09-10T21:09:24.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa781076a0 0x7faa781997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa781076a0 0x7faa781997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49508/0 (socket says 172.21.15.33:49508) 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 -- 172.21.15.33:0/1899209721 learned_addr learned my addr 172.21.15.33:0/1899209721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa77fff640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa78108020 0x7faa7819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa777fe640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa78101c90 0x7faa78199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 -- 172.21.15.33:0/1899209721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa78101c90 msgr2=0x7faa78199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa78101c90 0x7faa78199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 -- 172.21.15.33:0/1899209721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa78108020 msgr2=0x7faa7819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa78108020 0x7faa7819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa76ffd640 1 -- 172.21.15.33:0/1899209721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa7819f320 con 0x7faa781076a0 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa77fff640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa78108020 0x7faa7819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.478+0000 7faa777fe640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa78101c90 0x7faa78199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:09:24.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.479+0000 7faa76ffd640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa781076a0 0x7faa781997b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c920 tx=0x7faa6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.479+0000 7faa74ff9640 1 -- 172.21.15.33:0/1899209721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c007c70 con 0x7faa781076a0 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.479+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa7819f610 con 0x7faa781076a0 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.479+0000 7faa74ff9640 1 -- 172.21.15.33:0/1899209721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa6c007e10 con 0x7faa781076a0 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.479+0000 7faa74ff9640 1 -- 172.21.15.33:0/1899209721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c018820 con 0x7faa781076a0 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.479+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa781a5d50 con 0x7faa781076a0 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.481+0000 7faa804a7740 1 Processor -- start 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.481+0000 7faa804a7740 1 -- start start 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.481+0000 7faa74ff9640 1 -- 172.21.15.33:0/1899209721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7faa6c00f450 con 0x7faa781076a0 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.481+0000 7faa804a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.481+0000 7faa804a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.481+0000 7faa804a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5e89a0 0x561abd5ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa804a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561abd552ea0 con 0x561abd5dca10 2024-09-10T21:09:24.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa804a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561abd5422d0 con 0x561abd5e89a0 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa804a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561abd5460e0 con 0x561abd5dc0f0 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5e8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49518/0 (socket says 172.21.15.33:49518) 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 -- 172.21.15.33:0/146615866 learned_addr learned my addr 172.21.15.33:0/146615866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa77fff640 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa76ffd640 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5e89a0 0x561abd5ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 -- 172.21.15.33:0/146615866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 msgr2=0x561abd5dc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5dc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 -- 172.21.15.33:0/146615866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5e89a0 msgr2=0x561abd5ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5e89a0 0x561abd5ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 -- 172.21.15.33:0/146615866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abd2a0650 con 0x561abd5dca10 2024-09-10T21:09:24.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa77fff640 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa777fe640 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5e8460 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faa68000c00 tx=0x7faa68032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ed10f8c837aea3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.482+0000 7faa5dffb640 1 -- 172.21.15.33:0/146615866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6803f070 con 0x561abd5dca10 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa5dffb640 1 -- 172.21.15.33:0/146615866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa68031760 con 0x561abd5dca10 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa5dffb640 1 -- 172.21.15.33:0/146615866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa68035070 con 0x561abd5dca10 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa74ff9640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7faa600b52e0 0x7faa600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa77fff640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7faa600b52e0 0x7faa600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa74ff9640 1 -- 172.21.15.33:0/1899209721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7faa6c0e5e00 con 0x7faa781076a0 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa77fff640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7faa600b52e0 0x7faa600b77a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faa7000e390 tx=0x7faa700075e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 -- 172.21.15.33:0/146615866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 msgr2=0x561abd5e8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5e8460 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faa68000c00 tx=0x7faa68032e90 comp rx=0 tx=0).stop 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 -- 172.21.15.33:0/146615866 shutdown_connections 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5e89a0 0x561abd5ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5e8460 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 --2- 172.21.15.33:0/146615866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5dc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 -- 172.21.15.33:0/146615866 >> 172.21.15.33:0/146615866 conn(0x561abd5d9f20 msgr2=0x561abd5db0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 -- 172.21.15.33:0/146615866 shutdown_connections 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.483+0000 7faa804a7740 1 -- 172.21.15.33:0/146615866 wait complete. 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 -- start start 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 Processor -- start 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dc0f0 0x561abd5e8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5e56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5e5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561abd3a3700 con 0x561abd5dc0f0 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561abd4dd160 con 0x561abd5dca10 2024-09-10T21:09:24.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561abd5eb5c0 con 0x561abd5e89a0 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa804a7740 10 client.0 _ll_register_callbacks cb 0x561abd4f69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dc0f0 0x561abd5e8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dc0f0 0x561abd5e8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49520/0 (socket says 172.21.15.33:49520) 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa777fe640 1 -- 172.21.15.33:0/191897651 learned_addr learned my addr 172.21.15.33:0/191897651 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa76ffd640 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5e56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.484+0000 7faa77fff640 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5e5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa777fe640 1 -- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 msgr2=0x561abd5e5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa777fe640 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5e5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa777fe640 1 -- 172.21.15.33:0/191897651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 msgr2=0x561abd5e56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa777fe640 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5e56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa777fe640 1 -- 172.21.15.33:0/191897651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561abd50ac90 con 0x561abd5dc0f0 2024-09-10T21:09:24.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa777fe640 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dc0f0 0x561abd5e8270 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faa68005ab0 tx=0x7faa68032bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80e0afcbbaea4310 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa56ffd640 1 -- 172.21.15.33:0/191897651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6803f070 con 0x561abd5dc0f0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa56ffd640 1 -- 172.21.15.33:0/191897651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7faa68004260 con 0x561abd5dc0f0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa68031760 con 0x561abd5dc0f0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa56ffd640 1 -- 172.21.15.33:0/191897651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faa68035040 con 0x561abd5dc0f0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa804a7740 10 client.16317 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561abd5f6f00 con 0x561abd5dc0f0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.485+0000 7faa56ffd640 1 -- 172.21.15.33:0/191897651 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa68031760 con 0x561abd5dc0f0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa56ffd640 1 -- 172.21.15.33:0/191897651 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7faa6803b4d0 con 0x561abd5dc0f0 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa804a7740 10 client.16317 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa804a7740 10 client.16317 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa804a7740 20 client.16317 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa804a7740 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd5f8250 0x561abd5fa640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa804a7740 4 client.16317 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561abd554df0 con 0x561abd5f8250 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa76ffd640 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd5f8250 0x561abd5fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa56ffd640 10 client.16317 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.486+0000 7faa76ffd640 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd5f8250 0x561abd5fa640 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8048a62ac20a16ea server_cookie=6ec51e07635a3e89 in_seq=0 out_seq=0 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa56ffd640 1 -- 172.21.15.33:0/191897651 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561abd554df0 con 0x561abd5f8250 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa56ffd640 10 client.16317 handle_command_reply: tid=0 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 client.16317 shutdown 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dc0f0 msgr2=0x561abd5e8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dc0f0 0x561abd5e8270 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faa68005ab0 tx=0x7faa68032bf0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 shutdown_connections 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd5f8250 0x561abd5fa640 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5e5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5e56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 --2- 172.21.15.33:0/191897651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dc0f0 0x561abd5e8270 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa56ffd640 0 client.16317 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.487+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 >> 172.21.15.33:0/191897651 conn(0x561abd5d9f20 msgr2=0x561abd5dfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.488+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 shutdown_connections 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.488+0000 7faa804a7740 1 -- 172.21.15.33:0/191897651 wait complete. 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.488+0000 7faa804a7740 20 client.16317 trim_cache size 0 max 16384 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 Processor -- start 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 -- start start 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5d8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5e89a0 0x561abd5e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa781ae490 con 0x561abd5e89a0 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa781b0150 con 0x561abd5dca10 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa804a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa781aebc0 con 0x561abd5dc0f0 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5e89a0 0x561abd5e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5e89a0 0x561abd5e28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49530/0 (socket says 172.21.15.33:49530) 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa77fff640 1 -- 172.21.15.33:0/3557778513 learned_addr learned my addr 172.21.15.33:0/3557778513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa76ffd640 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5d8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.501+0000 7faa777fe640 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa77fff640 1 -- 172.21.15.33:0/3557778513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 msgr2=0x561abd5f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa77fff640 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa77fff640 1 -- 172.21.15.33:0/3557778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 msgr2=0x561abd5d8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa77fff640 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5d8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa77fff640 1 -- 172.21.15.33:0/3557778513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561abd5dacb0 con 0x561abd5e89a0 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa777fe640 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa77fff640 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5e89a0 0x561abd5e28d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7faa70010770 tx=0x7faa70011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7282cf1009c6c854 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa5d7fa640 1 -- 172.21.15.33:0/3557778513 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa7000fa00 con 0x561abd5e89a0 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa5d7fa640 1 -- 172.21.15.33:0/3557778513 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa7000fba0 con 0x561abd5e89a0 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.502+0000 7faa5d7fa640 1 -- 172.21.15.33:0/3557778513 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa70026600 con 0x561abd5e89a0 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- 172.21.15.33:0/3557778513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5e89a0 msgr2=0x561abd5e28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5e89a0 0x561abd5e28d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7faa70010770 tx=0x7faa70011000 comp rx=0 tx=0).stop 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- 172.21.15.33:0/3557778513 shutdown_connections 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5e89a0 0x561abd5e28d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dca10 0x561abd5d8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 --2- 172.21.15.33:0/3557778513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5dc0f0 0x561abd5f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- 172.21.15.33:0/3557778513 >> 172.21.15.33:0/3557778513 conn(0x561abd5d9f20 msgr2=0x561abd5d8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- 172.21.15.33:0/3557778513 shutdown_connections 2024-09-10T21:09:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- 172.21.15.33:0/3557778513 wait complete. 2024-09-10T21:09:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- start start 2024-09-10T21:09:24.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 Processor -- start 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dc0f0 0x561abd5f0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5ed970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faa781af9c0 con 0x561abd5dca10 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faa781af2c0 con 0x561abd5dc0f0 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.503+0000 7faa804a7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa781ac450 con 0x561abd5e89a0 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa804a7740 10 client.0 _ll_register_callbacks cb 0x7faa68032d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5ed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49534/0 (socket says 172.21.15.33:49534) 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 -- 172.21.15.33:0/2796609263 learned_addr learned my addr 172.21.15.33:0/2796609263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa77fff640 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5ed970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa777fe640 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dc0f0 0x561abd5f0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 -- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 msgr2=0x561abd5ed970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5ed970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 -- 172.21.15.33:0/2796609263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dc0f0 msgr2=0x561abd5f0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dc0f0 0x561abd5f0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 -- 172.21.15.33:0/2796609263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561abd5ee0e0 con 0x561abd5dca10 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa76ffd640 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5ed430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0ac550 tx=0x7faa6c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325478f946535b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa56ffd640 1 -- 172.21.15.33:0/2796609263 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0ac5c0 con 0x561abd5dca10 2024-09-10T21:09:24.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa56ffd640 1 -- 172.21.15.33:0/2796609263 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7faa6c0a8830 con 0x561abd5dca10 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa7000fba0 con 0x561abd5dca10 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.504+0000 7faa56ffd640 1 -- 172.21.15.33:0/2796609263 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7faa6c0b10a0 con 0x561abd5dca10 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 10 client.16323 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x561abd665e00 con 0x561abd5dca10 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa56ffd640 1 -- 172.21.15.33:0/2796609263 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa7000fba0 con 0x561abd5dca10 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa56ffd640 1 -- 172.21.15.33:0/2796609263 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7faa6c0b56c0 con 0x561abd5dca10 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 10 client.16323 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 10 client.16323 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 20 client.16323 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd666550 0x561abd668970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 4 client.16323 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561abd5508a0 con 0x561abd666550 2024-09-10T21:09:24.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.505+0000 7faa777fe640 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd666550 0x561abd668970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa56ffd640 10 client.16323 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa777fe640 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd666550 0x561abd668970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eac54884d12686c0 server_cookie=8b802548e6217cbd in_seq=0 out_seq=0 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa56ffd640 1 -- 172.21.15.33:0/2796609263 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561abd5508a0 con 0x561abd666550 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa56ffd640 10 client.16323 handle_command_reply: tid=0 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 client.16323 shutdown 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 msgr2=0x561abd5ed430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5ed430 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0ac550 tx=0x7faa6c000bc0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 shutdown_connections 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x561abd666550 0x561abd668970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561abd5e89a0 0x561abd5ed970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561abd5dca10 0x561abd5ed430 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa56ffd640 0 client.16323 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:09:24.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 --2- 172.21.15.33:0/2796609263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561abd5dc0f0 0x561abd5f0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 >> 172.21.15.33:0/2796609263 conn(0x561abd5d9f20 msgr2=0x561abd5d8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 shutdown_connections 2024-09-10T21:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 1 -- 172.21.15.33:0/2796609263 wait complete. 2024-09-10T21:09:24.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.506+0000 7faa804a7740 20 client.16323 trim_cache size 0 max 16384 2024-09-10T21:09:24.528 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7faa600b52e0 msgr2=0x7faa600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7faa600b52e0 0x7faa600b77a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7faa7000e390 tx=0x7faa700075e0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa781076a0 msgr2=0x7faa781997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa781076a0 0x7faa781997b0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faa6c00c920 tx=0x7faa6c00cde0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 shutdown_connections 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7faa600b52e0 0x7faa600b77a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faa78108020 0x7faa7819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa781076a0 0x7faa781997b0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 --2- 172.21.15.33:0/1899209721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa78101c90 0x7faa78199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 >> 172.21.15.33:0/1899209721 conn(0x7faa780fd700 msgr2=0x7faa780ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 shutdown_connections 2024-09-10T21:09:24.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:24.507+0000 7faa7e0d2640 1 -- 172.21.15.33:0/1899209721 wait complete. 2024-09-10T21:09:24.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476328 8.85 MB/sec execute 145 sec latency 4953.717 ms 2024-09-10T21:09:25.216 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:09:25.474 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:25.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476328 8.79 MB/sec execute 146 sec latency 1627.014 ms 2024-09-10T21:09:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:25 smithi033 ceph-mon[30860]: pgmap v509: 97 pgs: 97 active+clean; 182 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2024-09-10T21:09:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:25 smithi104 ceph-mon[35325]: pgmap v509: 97 pgs: 97 active+clean; 182 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2024-09-10T21:09:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 -- 172.21.15.33:0/2972214426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c1076e0 msgr2=0x7f993c107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 --2- 172.21.15.33:0/2972214426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c1076e0 0x7f993c107b20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f992c009900 tx=0x7f992c02a410 comp rx=0 tx=0).stop 2024-09-10T21:09:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 -- 172.21.15.33:0/2972214426 shutdown_connections 2024-09-10T21:09:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 --2- 172.21.15.33:0/2972214426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f993c108060 0x7f993c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 --2- 172.21.15.33:0/2972214426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c1076e0 0x7f993c107b20 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 --2- 172.21.15.33:0/2972214426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f993c101cd0 0x7f993c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 -- 172.21.15.33:0/2972214426 >> 172.21.15.33:0/2972214426 conn(0x7f993c0fd700 msgr2=0x7f993c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 -- 172.21.15.33:0/2972214426 shutdown_connections 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.078+0000 7f9943e1c640 1 -- 172.21.15.33:0/2972214426 wait complete. 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 Processor -- start 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 -- start start 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c101cd0 0x7f993c074a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f993c1076e0 0x7f993c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f993c108060 0x7f993c06ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f993c111d20 con 0x7f993c101cd0 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f993c111ba0 con 0x7f993c108060 2024-09-10T21:09:26.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.079+0000 7f9943e1c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f993c111ea0 con 0x7f993c1076e0 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c101cd0 0x7f993c074a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c101cd0 0x7f993c074a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49540/0 (socket says 172.21.15.33:49540) 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 -- 172.21.15.33:0/1409556549 learned_addr learned my addr 172.21.15.33:0/1409556549 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f99413b7640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f993c1076e0 0x7f993c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f99423b9640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f993c108060 0x7f993c06ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 -- 172.21.15.33:0/1409556549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f993c1076e0 msgr2=0x7f993c06fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f993c1076e0 0x7f993c06fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 -- 172.21.15.33:0/1409556549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f993c108060 msgr2=0x7f993c06ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f993c108060 0x7f993c06ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 -- 172.21.15.33:0/1409556549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993c0704c0 con 0x7f993c101cd0 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f99423b9640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f993c108060 0x7f993c06ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9941bb8640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c101cd0 0x7f993c074a60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f993800d770 tx=0x7f993800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:26.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9932ffd640 1 -- 172.21.15.33:0/1409556549 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938017070 con 0x7f993c101cd0 2024-09-10T21:09:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f993c0707b0 con 0x7f993c101cd0 2024-09-10T21:09:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9932ffd640 1 -- 172.21.15.33:0/1409556549 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99380054d0 con 0x7f993c101cd0 2024-09-10T21:09:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9932ffd640 1 -- 172.21.15.33:0/1409556549 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938004c30 con 0x7f993c101cd0 2024-09-10T21:09:26.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.080+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f993c1ae4b0 con 0x7f993c101cd0 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 Processor -- start 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 -- start start 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f9932ffd640 1 -- 172.21.15.33:0/1409556549 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9938004ee0 con 0x7f993c101cd0 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ec055fea0 con 0x562ec05e9a10 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562ec054f2d0 con 0x562ec05f59a0 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.083+0000 7f99461f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ec05530e0 con 0x562ec05e90f0 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49556/0 (socket says 172.21.15.33:49556) 2024-09-10T21:09:26.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 -- 172.21.15.33:0/3032546622 learned_addr learned my addr 172.21.15.33:0/3032546622 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f99423b9640 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f99413b7640 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 -- 172.21.15.33:0/3032546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 msgr2=0x562ec05e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 -- 172.21.15.33:0/3032546622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 msgr2=0x562ec05f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 -- 172.21.15.33:0/3032546622 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec02ad650 con 0x562ec05e9a10 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f99413b7640 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9941bb8640 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f5460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f99380ebc20 tx=0x7f99380ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e429e0dae22e8ad5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.084+0000 7f9932ffd640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99280b52e0 0x7f99280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f99423b9640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99280b52e0 0x7f99280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f9927fff640 1 -- 172.21.15.33:0/3032546622 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380f4070 con 0x562ec05e9a10 2024-09-10T21:09:26.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f9927fff640 1 -- 172.21.15.33:0/3032546622 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993801d1f0 con 0x562ec05e9a10 2024-09-10T21:09:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f9927fff640 1 -- 172.21.15.33:0/3032546622 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380a8580 con 0x562ec05e9a10 2024-09-10T21:09:26.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f9932ffd640 1 -- 172.21.15.33:0/1409556549 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f99380e5770 con 0x7f993c101cd0 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f99461f1740 1 -- 172.21.15.33:0/3032546622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 msgr2=0x562ec05f5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f99423b9640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99280b52e0 0x7f99280b77a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9934005fd0 tx=0x7f9934009800 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.085+0000 7f99461f1740 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f5460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f99380ebc20 tx=0x7f99380ed670 comp rx=0 tx=0).stop 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 -- 172.21.15.33:0/3032546622 shutdown_connections 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f5460 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 --2- 172.21.15.33:0/3032546622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 -- 172.21.15.33:0/3032546622 >> 172.21.15.33:0/3032546622 conn(0x562ec05e6f20 msgr2=0x562ec05e80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 -- 172.21.15.33:0/3032546622 shutdown_connections 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 -- 172.21.15.33:0/3032546622 wait complete. 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 -- start start 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.086+0000 7f99461f1740 1 Processor -- start 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99461f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99461f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e9a10 0x562ec05f2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99461f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05f2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99461f1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562ec03b0700 con 0x562ec05f59a0 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99461f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562ec04ea160 con 0x562ec05e9a10 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99461f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ec05f8620 con 0x562ec05e90f0 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99461f1740 10 client.0 _ll_register_callbacks cb 0x562ec05039e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05f2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05f2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49564/0 (socket says 172.21.15.33:49564) 2024-09-10T21:09:26.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 -- 172.21.15.33:0/3863051262 learned_addr learned my addr 172.21.15.33:0/3863051262 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f9941bb8640 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99413b7640 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e9a10 0x562ec05f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 -- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 msgr2=0x562ec05f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 -- 172.21.15.33:0/3863051262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e9a10 msgr2=0x562ec05f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e9a10 0x562ec05f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 -- 172.21.15.33:0/3863051262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec0517c90 con 0x562ec05f59a0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99423b9640 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05f2c20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9934011ce0 tx=0x7f9934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6a40e4b3efc953 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.087+0000 7f99257fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99257fa640 1 -- 172.21.15.33:0/3863051262 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993401eb20 con 0x562ec05f59a0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99257fa640 1 -- 172.21.15.33:0/3863051262 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f993401ecc0 con 0x562ec05f59a0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f993801d1f0 con 0x562ec05f59a0 2024-09-10T21:09:26.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99257fa640 1 -- 172.21.15.33:0/3863051262 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f993402a480 con 0x562ec05f59a0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 10 client.16335 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562ec0603f50 con 0x562ec05f59a0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99257fa640 1 -- 172.21.15.33:0/3863051262 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f993801d1f0 con 0x562ec05f59a0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99257fa640 1 -- 172.21.15.33:0/3863051262 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f993400f040 con 0x562ec05f59a0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 10 client.16335 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 10 client.16335 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 20 client.16335 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec06054b0 0x562ec06078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 4 client.16335 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.088+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ec0561df0 con 0x562ec06054b0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.089+0000 7f9941bb8640 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec06054b0 0x562ec06078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.089+0000 7f9941bb8640 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec06054b0 0x562ec06078c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=616bd020d4ac7711 server_cookie=d484f36877004030 in_seq=0 out_seq=0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.089+0000 7f99257fa640 10 client.16335 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99257fa640 1 -- 172.21.15.33:0/3863051262 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562ec0561df0 con 0x562ec06054b0 2024-09-10T21:09:26.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99257fa640 10 client.16335 handle_command_reply: tid=0 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 client.16335 shutdown 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 msgr2=0x562ec05f2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05f2c20 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9934011ce0 tx=0x7f9934004a40 comp rx=0 tx=0).stop 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 shutdown_connections 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05f2c20 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e9a10 0x562ec05f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec06054b0 0x562ec06078c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 --2- 172.21.15.33:0/3863051262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 >> 172.21.15.33:0/3863051262 conn(0x562ec05e6f20 msgr2=0x562ec05ecac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99257fa640 0 client.16335 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 shutdown_connections 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 1 -- 172.21.15.33:0/3863051262 wait complete. 2024-09-10T21:09:26.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.090+0000 7f99461f1740 20 client.16335 trim_cache size 0 max 16384 2024-09-10T21:09:26.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 Processor -- start 2024-09-10T21:09:26.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 -- start start 2024-09-10T21:09:26.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1b6b90 con 0x562ec05e9a10 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1b8850 con 0x562ec05f59a0 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99461f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1b72c0 con 0x562ec05e90f0 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99413b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99413b7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49570/0 (socket says 172.21.15.33:49570) 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.103+0000 7f99413b7640 1 -- 172.21.15.33:0/312157990 learned_addr learned my addr 172.21.15.33:0/312157990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f9941bb8640 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99413b7640 1 -- 172.21.15.33:0/312157990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 msgr2=0x562ec05f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99413b7640 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99413b7640 1 -- 172.21.15.33:0/312157990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 msgr2=0x562ec05f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99423b9640 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99413b7640 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99413b7640 1 -- 172.21.15.33:0/312157990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ec05e9840 con 0x562ec05e9a10 2024-09-10T21:09:26.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99423b9640 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:09:26.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99413b7640 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f6d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f992c02a930 tx=0x7f992c032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98eb96aac23576d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:26.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99277fe640 1 -- 172.21.15.33:0/312157990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c035040 con 0x562ec05e9a10 2024-09-10T21:09:26.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99277fe640 1 -- 172.21.15.33:0/312157990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c0316a0 con 0x562ec05e9a10 2024-09-10T21:09:26.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.104+0000 7f99277fe640 1 -- 172.21.15.33:0/312157990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c0046c0 con 0x562ec05e9a10 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 -- 172.21.15.33:0/312157990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 msgr2=0x562ec05f6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f6d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f992c02a930 tx=0x7f992c032df0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 -- 172.21.15.33:0/312157990 shutdown_connections 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05f59a0 0x562ec05f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05e9a10 0x562ec05f6d90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 --2- 172.21.15.33:0/312157990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e90f0 0x562ec05f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 -- 172.21.15.33:0/312157990 >> 172.21.15.33:0/312157990 conn(0x562ec05e6f20 msgr2=0x562ec05f3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:26.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 -- 172.21.15.33:0/312157990 shutdown_connections 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 -- 172.21.15.33:0/312157990 wait complete. 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 -- start start 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.105+0000 7f99461f1740 1 Processor -- start 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99461f1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e90f0 0x562ec05f0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99461f1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e9a10 0x562ec05fd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99461f1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05fddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99461f1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1b80c0 con 0x562ec05f59a0 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99461f1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1b79c0 con 0x562ec05e90f0 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99461f1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f993c1b4b50 con 0x562ec05e9a10 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99461f1740 10 client.0 _ll_register_callbacks cb 0x7f99340093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05fddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05fddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49578/0 (socket says 172.21.15.33:49578) 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 -- 172.21.15.33:0/1689714816 learned_addr learned my addr 172.21.15.33:0/1689714816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:26.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99413b7640 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e9a10 0x562ec05fd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f9941bb8640 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e90f0 0x562ec05f0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 -- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e9a10 msgr2=0x562ec05fd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e9a10 0x562ec05fd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 -- 172.21.15.33:0/1689714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e90f0 msgr2=0x562ec05f0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e90f0 0x562ec05f0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 -- 172.21.15.33:0/1689714816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ec05fe600 con 0x562ec05f59a0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99423b9640 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05fddf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9934014fd0 tx=0x7f993401fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b653eb5ba98671 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.106+0000 7f99257fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99257fa640 1 -- 172.21.15.33:0/1689714816 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993401e970 con 0x562ec05f59a0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99257fa640 1 -- 172.21.15.33:0/1689714816 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f993401eb10 con 0x562ec05f59a0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f992c0316a0 con 0x562ec05f59a0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99257fa640 1 -- 172.21.15.33:0/1689714816 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f993401ed10 con 0x562ec05f59a0 2024-09-10T21:09:26.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 10 client.16341 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x562ec0673460 con 0x562ec05f59a0 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99257fa640 1 -- 172.21.15.33:0/1689714816 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c0316a0 con 0x562ec05f59a0 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99257fa640 1 -- 172.21.15.33:0/1689714816 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f9934035620 con 0x562ec05f59a0 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 10 client.16341 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 10 client.16341 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 20 client.16341 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec0673c60 0x562ec0676080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 4 client.16341 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ec055d8a0 con 0x562ec0673c60 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.107+0000 7f9941bb8640 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec0673c60 0x562ec0676080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f9941bb8640 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec0673c60 0x562ec0676080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6dc9971be4c79b7 server_cookie=ac1b156e9bce7b2b in_seq=0 out_seq=0 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99257fa640 10 client.16341 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:09:26.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99257fa640 1 -- 172.21.15.33:0/1689714816 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ec055d8a0 con 0x562ec0673c60 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99257fa640 10 client.16341 handle_command_reply: tid=0 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 client.16341 shutdown 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 msgr2=0x562ec05fddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05fddf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9934014fd0 tx=0x7f993401fcd0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 shutdown_connections 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562ec05f59a0 0x562ec05fddf0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x562ec0673c60 0x562ec0676080 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ec05e9a10 0x562ec05fd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 --2- 172.21.15.33:0/1689714816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562ec05e90f0 0x562ec05f0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 >> 172.21.15.33:0/1689714816 conn(0x562ec05e6f20 msgr2=0x562ec05f44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99257fa640 0 client.16341 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 shutdown_connections 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.108+0000 7f99461f1740 1 -- 172.21.15.33:0/1689714816 wait complete. 2024-09-10T21:09:26.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f99461f1740 20 client.16341 trim_cache size 0 max 16384 2024-09-10T21:09:26.119 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:09:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99280b52e0 msgr2=0x7f99280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99280b52e0 0x7f99280b77a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9934005fd0 tx=0x7f9934009800 comp rx=0 tx=0).stop 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c101cd0 msgr2=0x7f993c074a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c101cd0 0x7f993c074a60 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f993800d770 tx=0x7f993800dc30 comp rx=0 tx=0).stop 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 shutdown_connections 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99280b52e0 0x7f99280b77a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f993c108060 0x7f993c06ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f993c1076e0 0x7f993c06fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 --2- 172.21.15.33:0/1409556549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f993c101cd0 0x7f993c074a60 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 >> 172.21.15.33:0/1409556549 conn(0x7f993c0fd700 msgr2=0x7f993c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 shutdown_connections 2024-09-10T21:09:26.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:26.109+0000 7f9943e1c640 1 -- 172.21.15.33:0/1409556549 wait complete. 2024-09-10T21:09:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:25 smithi174 ceph-mon[35635]: pgmap v509: 97 pgs: 97 active+clean; 182 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 2.8 MiB/s wr, 117 op/s 2024-09-10T21:09:26.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476328 8.73 MB/sec execute 147 sec latency 2627.112 ms 2024-09-10T21:09:26.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:09:27.226 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:27.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476328 8.67 MB/sec execute 148 sec latency 3627.145 ms 2024-09-10T21:09:27.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/2873236891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0113b20 msgr2=0x7ff3c0115f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/2873236891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0113b20 0x7ff3c0115f10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8009ca0 tx=0x7ff3b802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/2873236891 shutdown_connections 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/2873236891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0113b20 0x7ff3c0115f10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/2873236891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3c0075e60 0x7ff3c01135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/2873236891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3c0075540 0x7ff3c0075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/2873236891 >> 172.21.15.33:0/2873236891 conn(0x7ff3c00fe830 msgr2=0x7ff3c0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/2873236891 shutdown_connections 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.715+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/2873236891 wait complete. 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 Processor -- start 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 -- start start 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0075540 0x7ff3c019f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3c0075e60 0x7ff3c019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3c0113b20 0x7ff3c01a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0119380 con 0x7ff3c0075540 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0119200 con 0x7ff3c0113b20 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.716+0000 7ff3c7ef8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c0119500 con 0x7ff3c0075e60 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0075540 0x7ff3c019f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0075540 0x7ff3c019f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49604/0 (socket says 172.21.15.33:49604) 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1453901006 learned_addr learned my addr 172.21.15.33:0/1453901006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5493640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3c0075e60 0x7ff3c019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c6495640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3c0113b20 0x7ff3c01a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1453901006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3c0075e60 msgr2=0x7ff3c019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3c0075e60 0x7ff3c019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1453901006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3c0113b20 msgr2=0x7ff3c01a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3c0113b20 0x7ff3c01a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1453901006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3c01a58a0 con 0x7ff3c0075540 2024-09-10T21:09:27.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0075540 0x7ff3c019f7f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c920 tx=0x7ff3bc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3b6ffd640 1 -- 172.21.15.33:0/1453901006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007c70 con 0x7ff3c0075540 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3c01a5b90 con 0x7ff3c0075540 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3b6ffd640 1 -- 172.21.15.33:0/1453901006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc007e10 con 0x7ff3c0075540 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3b6ffd640 1 -- 172.21.15.33:0/1453901006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc018820 con 0x7ff3c0075540 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.717+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3c01ac250 con 0x7ff3c0075540 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3b6ffd640 1 -- 172.21.15.33:0/1453901006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff3bc00f450 con 0x7ff3c0075540 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 Processor -- start 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 -- start start 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56482b53fea0 con 0x56482b5c94d0 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56482b52f2d0 con 0x56482b5d5d80 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.720+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56482b5330e0 con 0x56482b5c9df0 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49606/0 (socket says 172.21.15.33:49606) 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 -- 172.21.15.33:0/1764112560 learned_addr learned my addr 172.21.15.33:0/1764112560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c5493640 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3b6ffd640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff3ac0b52e0 0x7ff3ac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 -- 172.21.15.33:0/1764112560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 msgr2=0x56482b5d5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 -- 172.21.15.33:0/1764112560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 msgr2=0x56482b5d8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 -- 172.21.15.33:0/1764112560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482b28d650 con 0x56482b5c94d0 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff3ac0b52e0 0x7ff3ac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3b6ffd640 1 -- 172.21.15.33:0/1453901006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff3bc0e5d50 con 0x7ff3c0075540 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.721+0000 7ff3c6495640 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c98b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005cc0 tx=0x7ff3b8031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cbf72ec4754427 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3abfff640 1 -- 172.21.15.33:0/1764112560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8003b30 con 0x56482b5c94d0 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3abfff640 1 -- 172.21.15.33:0/1764112560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b8036040 con 0x56482b5c94d0 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3abfff640 1 -- 172.21.15.33:0/1764112560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b802f860 con 0x56482b5c94d0 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1764112560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 msgr2=0x56482b5c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c98b0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005cc0 tx=0x7ff3b8031af0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3c6495640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff3ac0b52e0 0x7ff3ac0b77a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff3b804c000 tx=0x7ff3b8049000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1764112560 shutdown_connections 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.722+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1764112560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c98b0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1764112560 >> 172.21.15.33:0/1764112560 conn(0x56482b5c7300 msgr2=0x56482b5c84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1764112560 shutdown_connections 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1764112560 wait complete. 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 -- start start 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 Processor -- start 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5c94d0 0x56482b5d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c9df0 0x56482b5d2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.723+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5d5d80 0x56482b5d3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56482b390700 con 0x56482b5c9df0 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56482b4ca160 con 0x56482b5c94d0 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56482b5d8880 con 0x56482b5d5d80 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3ca2cd740 10 client.0 _ll_register_callbacks cb 0x56482b4e39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c9df0 0x56482b5d2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c9df0 0x56482b5d2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49616/0 (socket says 172.21.15.33:49616) 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 -- 172.21.15.33:0/2046605730 learned_addr learned my addr 172.21.15.33:0/2046605730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5c94d0 0x56482b5d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c6495640 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5d5d80 0x56482b5d3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 -- 172.21.15.33:0/2046605730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5d5d80 msgr2=0x56482b5d3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5d5d80 0x56482b5d3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 -- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5c94d0 msgr2=0x56482b5d5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5c94d0 0x56482b5d5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 -- 172.21.15.33:0/2046605730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56482b4f7c90 con 0x56482b5c9df0 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3c5493640 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c9df0 0x56482b5d2a90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000bad0 tx=0x7ff3b000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39867143ecd347f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3a97fa640 1 -- 172.21.15.33:0/2046605730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0005020 con 0x56482b5c9df0 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3a97fa640 1 -- 172.21.15.33:0/2046605730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff3b00051c0 con 0x56482b5c9df0 2024-09-10T21:09:27.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.724+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3b8036040 con 0x56482b5c9df0 2024-09-10T21:09:27.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3a97fa640 1 -- 172.21.15.33:0/2046605730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff3b00053c0 con 0x56482b5c9df0 2024-09-10T21:09:27.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 10 client.16353 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:27.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56482b5e4210 con 0x56482b5c9df0 2024-09-10T21:09:27.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3a97fa640 1 -- 172.21.15.33:0/2046605730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b8036040 con 0x56482b5c9df0 2024-09-10T21:09:27.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3a97fa640 1 -- 172.21.15.33:0/2046605730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7ff3b001b670 con 0x56482b5c9df0 2024-09-10T21:09:27.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 10 client.16353 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 10 client.16353 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 20 client.16353 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b5e56d0 0x56482b5e7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 4 client.16353 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56482b541df0 con 0x56482b5e56d0 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.725+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b5e56d0 0x56482b5e7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.726+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b5e56d0 0x56482b5e7b60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34890addd02d775c server_cookie=a62472c82d483d6c in_seq=0 out_seq=0 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.726+0000 7ff3a97fa640 10 client.16353 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3a97fa640 1 -- 172.21.15.33:0/2046605730 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56482b541df0 con 0x56482b5e56d0 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3a97fa640 10 client.16353 handle_command_reply: tid=0 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 client.16353 shutdown 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c9df0 msgr2=0x56482b5d2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c9df0 0x56482b5d2a90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ff3b000bad0 tx=0x7ff3b000bed0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 shutdown_connections 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5d5d80 0x56482b5d3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b5e56d0 0x56482b5e7b60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c9df0 0x56482b5d2a90 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/2046605730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5c94d0 0x56482b5d5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 >> 172.21.15.33:0/2046605730 conn(0x56482b5c7300 msgr2=0x56482b5cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3a97fa640 0 client.16353 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 shutdown_connections 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/2046605730 wait complete. 2024-09-10T21:09:27.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.727+0000 7ff3ca2cd740 20 client.16353 trim_cache size 0 max 16384 2024-09-10T21:09:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 Processor -- start 2024-09-10T21:09:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 -- start start 2024-09-10T21:09:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b4980 con 0x56482b5c94d0 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b6640 con 0x56482b5d5d80 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.740+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b50b0 con 0x56482b5c9df0 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49630/0 (socket says 172.21.15.33:49630) 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 -- 172.21.15.33:0/3101775281 learned_addr learned my addr 172.21.15.33:0/3101775281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c6495640 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5493640 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 -- 172.21.15.33:0/3101775281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 msgr2=0x56482b5d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 -- 172.21.15.33:0/3101775281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 msgr2=0x56482b5d1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 -- 172.21.15.33:0/3101775281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482b5e81e0 con 0x56482b5c94d0 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5493640 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c6495640 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c6c10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c8f0 tx=0x7ff3bc0be040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0d2ecd58c46a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3ab7fe640 1 -- 172.21.15.33:0/3101775281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0ac020 con 0x56482b5c94d0 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3ab7fe640 1 -- 172.21.15.33:0/3101775281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc0e7040 con 0x56482b5c94d0 2024-09-10T21:09:27.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.741+0000 7ff3ab7fe640 1 -- 172.21.15.33:0/3101775281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc00f770 con 0x56482b5c94d0 2024-09-10T21:09:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/3101775281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 msgr2=0x56482b5c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c6c10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c8f0 tx=0x7ff3bc0be040 comp rx=0 tx=0).stop 2024-09-10T21:09:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/3101775281 shutdown_connections 2024-09-10T21:09:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5d1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5d1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/3101775281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5c6c10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/3101775281 >> 172.21.15.33:0/3101775281 conn(0x56482b5c7300 msgr2=0x56482b5c8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/3101775281 shutdown_connections 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/3101775281 wait complete. 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 -- start start 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.742+0000 7ff3ca2cd740 1 Processor -- start 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5dddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5de350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3ca2cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5cecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b5eb0 con 0x56482b5c94d0 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b57b0 con 0x56482b5d5d80 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3ca2cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3c01b2940 con 0x56482b5c9df0 2024-09-10T21:09:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3ca2cd740 10 client.0 _ll_register_callbacks cb 0x7ff3b00040a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5dddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5dddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49636/0 (socket says 172.21.15.33:49636) 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1153394174 learned_addr learned my addr 172.21.15.33:0/1153394174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c6495640 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5cecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5493640 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5de350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1153394174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 msgr2=0x56482b5de350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5de350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 msgr2=0x56482b5cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5cecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 -- 172.21.15.33:0/1153394174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56482b5cf500 con 0x56482b5c94d0 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3c5c94640 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5dddd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0e9430 tx=0x7ff3bc00fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712d5411e9213f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.743+0000 7ff3a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3a97fa640 1 -- 172.21.15.33:0/1153394174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0ac510 con 0x56482b5c94d0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3a97fa640 1 -- 172.21.15.33:0/1153394174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff3bc0a8780 con 0x56482b5c94d0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3bc0e7040 con 0x56482b5c94d0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3a97fa640 1 -- 172.21.15.33:0/1153394174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7ff3bc0c6020 con 0x56482b5c94d0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 10 client.16359 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56482b5d0770 con 0x56482b5c94d0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3a97fa640 1 -- 172.21.15.33:0/1153394174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3bc06d020 con 0x56482b5c94d0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3a97fa640 1 -- 172.21.15.33:0/1153394174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7ff3bc0b5610 con 0x56482b5c94d0 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 10 client.16359 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 10 client.16359 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 20 client.16359 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b6538f0 0x56482b655d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 4 client.16359 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.744+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56482b53d8a0 con 0x56482b6538f0 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3c5493640 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b6538f0 0x56482b655d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3c5493640 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b6538f0 0x56482b655d10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9adac05033a73328 server_cookie=ae2204b4be40797b in_seq=0 out_seq=0 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3a97fa640 10 client.16359 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3a97fa640 1 -- 172.21.15.33:0/1153394174 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56482b53d8a0 con 0x56482b6538f0 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3a97fa640 10 client.16359 handle_command_reply: tid=0 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3ca2cd740 1 client.16359 shutdown 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 msgr2=0x56482b5dddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.745+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5dddd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc0e9430 tx=0x7ff3bc00fe60 comp rx=0 tx=0).stop 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 shutdown_connections 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56482b5d5d80 0x56482b5cecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56482b5c9df0 0x56482b5de350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56482b6538f0 0x56482b655d10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 --2- 172.21.15.33:0/1153394174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56482b5c94d0 0x56482b5dddd0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3a97fa640 0 client.16359 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:09:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 >> 172.21.15.33:0/1153394174 conn(0x56482b5c7300 msgr2=0x56482b5d1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 shutdown_connections 2024-09-10T21:09:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 1 -- 172.21.15.33:0/1153394174 wait complete. 2024-09-10T21:09:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3ca2cd740 20 client.16359 trim_cache size 0 max 16384 2024-09-10T21:09:27.756 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:09:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff3ac0b52e0 msgr2=0x7ff3ac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff3ac0b52e0 0x7ff3ac0b77a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff3b804c000 tx=0x7ff3b8049000 comp rx=0 tx=0).stop 2024-09-10T21:09:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0075540 msgr2=0x7ff3c019f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0075540 0x7ff3c019f7f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c920 tx=0x7ff3bc00cde0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 shutdown_connections 2024-09-10T21:09:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff3ac0b52e0 0x7ff3ac0b77a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff3c0113b20 0x7ff3c01a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3c0075e60 0x7ff3c019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 --2- 172.21.15.33:0/1453901006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3c0075540 0x7ff3c019f7f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 >> 172.21.15.33:0/1453901006 conn(0x7ff3c00fe830 msgr2=0x7ff3c00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.746+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 shutdown_connections 2024-09-10T21:09:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:27.747+0000 7ff3c7ef8640 1 -- 172.21.15.33:0/1453901006 wait complete. 2024-09-10T21:09:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:27 smithi033 ceph-mon[30860]: pgmap v510: 97 pgs: 97 active+clean; 185 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.8 MiB/s wr, 79 op/s 2024-09-10T21:09:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:27 smithi104 ceph-mon[35325]: pgmap v510: 97 pgs: 97 active+clean; 185 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.8 MiB/s wr, 79 op/s 2024-09-10T21:09:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:27 smithi174 ceph-mon[35635]: pgmap v510: 97 pgs: 97 active+clean; 185 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.8 MiB/s wr, 79 op/s 2024-09-10T21:09:28.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:09:28.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476328 8.61 MB/sec execute 149 sec latency 4627.180 ms 2024-09-10T21:09:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 -- 172.21.15.33:0/1160730434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4630103ca0 msgr2=0x7f4630104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 --2- 172.21.15.33:0/1160730434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4630103ca0 0x7f4630104080 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4620009900 tx=0x7f462002a410 comp rx=0 tx=0).stop 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 -- 172.21.15.33:0/1160730434 shutdown_connections 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 --2- 172.21.15.33:0/1160730434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463010d100 0x7f463010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 --2- 172.21.15.33:0/1160730434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46301045c0 0x7f463010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 --2- 172.21.15.33:0/1160730434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4630103ca0 0x7f4630104080 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 -- 172.21.15.33:0/1160730434 >> 172.21.15.33:0/1160730434 conn(0x7f46300fd700 msgr2=0x7f46300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 -- 172.21.15.33:0/1160730434 shutdown_connections 2024-09-10T21:09:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.230+0000 7f46351c6640 1 -- 172.21.15.33:0/1160730434 wait complete. 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.231+0000 7f46351c6640 1 Processor -- start 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.231+0000 7f46351c6640 1 -- start start 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.231+0000 7f46351c6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4630103ca0 0x7f4630199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.231+0000 7f46351c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46301045c0 0x7f46301997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.231+0000 7f46351c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463010d100 0x7f463019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f46351c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4630112dc0 con 0x7f46301045c0 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f46351c6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4630112c40 con 0x7f4630103ca0 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f46351c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4630112f40 con 0x7f463010d100 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46301045c0 0x7f46301997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46301045c0 0x7f46301997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49646/0 (socket says 172.21.15.33:49646) 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462ed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4630103ca0 0x7f4630199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 -- 172.21.15.33:0/2261171662 learned_addr learned my addr 172.21.15.33:0/2261171662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:29.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462f59e640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463010d100 0x7f463019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 -- 172.21.15.33:0/2261171662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463010d100 msgr2=0x7f463019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463010d100 0x7f463019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 -- 172.21.15.33:0/2261171662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4630103ca0 msgr2=0x7f4630199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4630103ca0 0x7f4630199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 -- 172.21.15.33:0/2261171662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463019f340 con 0x7f46301045c0 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462f59e640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463010d100 0x7f463019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462ed9d640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4630103ca0 0x7f4630199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f462e59c640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46301045c0 0x7f46301997d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f462400e9c0 tx=0x7f462400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f4617fff640 1 -- 172.21.15.33:0/2261171662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400ccc0 con 0x7f46301045c0 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f4617fff640 1 -- 172.21.15.33:0/2261171662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46240035f0 con 0x7f46301045c0 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463019f630 con 0x7f46301045c0 2024-09-10T21:09:29.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.232+0000 7f4617fff640 1 -- 172.21.15.33:0/2261171662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462400ce60 con 0x7f46301045c0 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.233+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46301a5cf0 con 0x7f46301045c0 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.235+0000 7f463759b740 1 Processor -- start 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.235+0000 7f463759b740 1 -- start start 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.235+0000 7f4617fff640 1 -- 172.21.15.33:0/2261171662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4624004000 con 0x7f46301045c0 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f463759b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e5fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f463759b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e607460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f463759b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e6079a0 0x56143e609d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f463759b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56143e571ea0 con 0x56143e6079a0 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f463759b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56143e5612d0 con 0x56143e5fba10 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f463759b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56143e5650e0 con 0x56143e5fb0f0 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462f59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e5fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462ed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e607460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e6079a0 0x56143e609d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462f59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e5fb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:41204/0 (socket says 172.21.15.33:41204) 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462f59e640 1 -- 172.21.15.33:0/6469382 learned_addr learned my addr 172.21.15.33:0/6469382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462ed9d640 1 -- 172.21.15.33:0/6469382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 msgr2=0x56143e5fb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462ed9d640 1 --2- 172.21.15.33:0/6469382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e5fb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462ed9d640 1 -- 172.21.15.33:0/6469382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e6079a0 msgr2=0x56143e609d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462ed9d640 1 --2- 172.21.15.33:0/6469382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e6079a0 0x56143e609d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462ed9d640 1 -- 172.21.15.33:0/6469382 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143e2bf650 con 0x56143e5fba10 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.236+0000 7f462ed9d640 1 --2- 172.21.15.33:0/6469382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e607460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f462002aa00 tx=0x7f46200043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dacdb3f082b0a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f4617fff640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46180b52e0 0x7f46180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f46157fa640 1 -- 172.21.15.33:0/6469382 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620035040 con 0x56143e5fba10 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f462f59e640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46180b52e0 0x7f46180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f46157fa640 1 -- 172.21.15.33:0/6469382 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4620004770 con 0x56143e5fba10 2024-09-10T21:09:29.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f46157fa640 1 -- 172.21.15.33:0/6469382 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620003920 con 0x56143e5fba10 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f4617fff640 1 -- 172.21.15.33:0/2261171662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f46240e5e70 con 0x7f46301045c0 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f462f59e640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46180b52e0 0x7f46180b77a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fd0 tx=0x7f4628009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f463759b740 1 -- 172.21.15.33:0/6469382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 msgr2=0x56143e607460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.237+0000 7f463759b740 1 --2- 172.21.15.33:0/6469382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e607460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f462002aa00 tx=0x7f46200043f0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- 172.21.15.33:0/6469382 shutdown_connections 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 --2- 172.21.15.33:0/6469382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e6079a0 0x56143e609d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 --2- 172.21.15.33:0/6469382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e607460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 --2- 172.21.15.33:0/6469382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e5fb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- 172.21.15.33:0/6469382 >> 172.21.15.33:0/6469382 conn(0x56143e5f8f20 msgr2=0x56143e5fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- 172.21.15.33:0/6469382 shutdown_connections 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- 172.21.15.33:0/6469382 wait complete. 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- start start 2024-09-10T21:09:29.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 Processor -- start 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e607250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e6046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e6079a0 0x56143e604c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56143e3c2700 con 0x56143e5fb0f0 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56143e4fc160 con 0x56143e5fba10 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56143e60a5c0 con 0x56143e6079a0 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f463759b740 10 client.0 _ll_register_callbacks cb 0x56143e5159e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.238+0000 7f462ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e607250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e607250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49672/0 (socket says 172.21.15.33:49672) 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 -- 172.21.15.33:0/3586436954 learned_addr learned my addr 172.21.15.33:0/3586436954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462e59c640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e6046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462f59e640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e6079a0 0x56143e604c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 -- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e6079a0 msgr2=0x56143e604c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e6079a0 0x56143e604c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 -- 172.21.15.33:0/3586436954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 msgr2=0x56143e6046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e6046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 -- 172.21.15.33:0/3586436954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143e529c90 con 0x56143e5fb0f0 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462e59c640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e6046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f462ed9d640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e607250 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4620005ab0 tx=0x7f4620005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd72e825d01f223 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f4612ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f4612ff5640 1 -- 172.21.15.33:0/3586436954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620035040 con 0x56143e5fb0f0 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4620004770 con 0x56143e5fb0f0 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f4612ff5640 1 -- 172.21.15.33:0/3586436954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4620004b20 con 0x56143e5fb0f0 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f4612ff5640 1 -- 172.21.15.33:0/3586436954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4620004d20 con 0x56143e5fb0f0 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.239+0000 7f463759b740 10 client.16371 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:29.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56143e615ee0 con 0x56143e5fb0f0 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f4612ff5640 1 -- 172.21.15.33:0/3586436954 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4620004770 con 0x56143e5fb0f0 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f4612ff5640 1 -- 172.21.15.33:0/3586436954 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f4620055030 con 0x56143e5fb0f0 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f463759b740 10 client.16371 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f463759b740 10 client.16371 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f463759b740 20 client.16371 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f463759b740 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e617230 0x56143e619620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f463759b740 4 client.16371 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56143e573df0 con 0x56143e617230 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.240+0000 7f462e59c640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e617230 0x56143e619620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.241+0000 7f462e59c640 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e617230 0x56143e619620 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4886e2edfd5617e server_cookie=37580d088e1fda06 in_seq=0 out_seq=0 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.241+0000 7f4612ff5640 10 client.16371 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.241+0000 7f4612ff5640 1 -- 172.21.15.33:0/3586436954 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56143e573df0 con 0x56143e617230 2024-09-10T21:09:29.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.241+0000 7f4612ff5640 10 client.16371 handle_command_reply: tid=0 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.241+0000 7f463759b740 1 client.16371 shutdown 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.241+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 msgr2=0x56143e607250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.241+0000 7f463759b740 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e607250 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f4620005ab0 tx=0x7f4620005900 comp rx=0 tx=0).stop 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 shutdown_connections 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e6079a0 0x56143e604c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e5fba10 0x56143e6046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e617230 0x56143e619620 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 --2- 172.21.15.33:0/3586436954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e607250 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f4612ff5640 0 client.16371 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 >> 172.21.15.33:0/3586436954 conn(0x56143e5f8f20 msgr2=0x56143e5fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 shutdown_connections 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 1 -- 172.21.15.33:0/3586436954 wait complete. 2024-09-10T21:09:29.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.242+0000 7f463759b740 20 client.16371 trim_cache size 0 max 16384 2024-09-10T21:09:29.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.255+0000 7f463759b740 1 Processor -- start 2024-09-10T21:09:29.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.255+0000 7f463759b740 1 -- start start 2024-09-10T21:09:29.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f463759b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e5f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f463759b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fba10 0x56143e5fa140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f463759b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e608d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f463759b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46301ae3e0 con 0x56143e5fb0f0 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f463759b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46301b00a0 con 0x56143e6079a0 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f463759b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46301aeb10 con 0x56143e5fba10 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e5f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e5f9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49686/0 (socket says 172.21.15.33:49686) 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 -- 172.21.15.33:0/3539338499 learned_addr learned my addr 172.21.15.33:0/3539338499 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462e59c640 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fba10 0x56143e5fa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462f59e640 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e608d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 -- 172.21.15.33:0/3539338499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fba10 msgr2=0x56143e5fa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fba10 0x56143e5fa140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 -- 172.21.15.33:0/3539338499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 msgr2=0x56143e608d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e608d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 -- 172.21.15.33:0/3539338499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56143e604fd0 con 0x56143e5fb0f0 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.256+0000 7f462ed9d640 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e5f9c00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4620005e00 tx=0x7f462002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e48de706af8b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f4614ff9640 1 -- 172.21.15.33:0/3539338499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620035070 con 0x56143e5fb0f0 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f4614ff9640 1 -- 172.21.15.33:0/3539338499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46200048e0 con 0x56143e5fb0f0 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f4614ff9640 1 -- 172.21.15.33:0/3539338499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620003d20 con 0x56143e5fb0f0 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 -- 172.21.15.33:0/3539338499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 msgr2=0x56143e5f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e5f9c00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f4620005e00 tx=0x7f462002a410 comp rx=0 tx=0).stop 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 -- 172.21.15.33:0/3539338499 shutdown_connections 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e608d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fba10 0x56143e5fa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 --2- 172.21.15.33:0/3539338499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fb0f0 0x56143e5f9c00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 -- 172.21.15.33:0/3539338499 >> 172.21.15.33:0/3539338499 conn(0x56143e5f8f20 msgr2=0x56143e5ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 -- 172.21.15.33:0/3539338499 shutdown_connections 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.257+0000 7f463759b740 1 -- 172.21.15.33:0/3539338499 wait complete. 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 -- start start 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 Processor -- start 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e61c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fba10 0x56143e61cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e619620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f462e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fba10 0x56143e61cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46301af910 con 0x56143e5fba10 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f462e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fba10 0x56143e61cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49690/0 (socket says 172.21.15.33:49690) 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46301af210 con 0x56143e6079a0 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f462e59c640 1 -- 172.21.15.33:0/1775789506 learned_addr learned my addr 172.21.15.33:0/1775789506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46301ac3a0 con 0x56143e5fb0f0 2024-09-10T21:09:29.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f462ed9d640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e61c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f462f59e640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e619620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.258+0000 7f463759b740 10 client.0 _ll_register_callbacks cb 0x7f46240e97c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f462ed9d640 1 -- 172.21.15.33:0/1775789506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 msgr2=0x56143e619620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f462ed9d640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e619620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f462ed9d640 1 -- 172.21.15.33:0/1775789506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fba10 msgr2=0x56143e61cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f462ed9d640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fba10 0x56143e61cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f462ed9d640 1 -- 172.21.15.33:0/1775789506 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56143e619c90 con 0x56143e5fb0f0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f462e59c640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fba10 0x56143e61cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f462ed9d640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e61c830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4620005e00 tx=0x7f46200045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d32e5b41c6fd92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f4612ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f4612ff5640 1 -- 172.21.15.33:0/1775789506 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620035030 con 0x56143e5fb0f0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46200048e0 con 0x56143e5fb0f0 2024-09-10T21:09:29.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f4612ff5640 1 -- 172.21.15.33:0/1775789506 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4620003cc0 con 0x56143e5fb0f0 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f4612ff5640 1 -- 172.21.15.33:0/1775789506 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f4620070050 con 0x56143e5fb0f0 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f463759b740 10 client.25903 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.259+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56143e688ab0 con 0x56143e5fb0f0 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f4612ff5640 1 -- 172.21.15.33:0/1775789506 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46200048e0 con 0x56143e5fb0f0 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f4612ff5640 1 -- 172.21.15.33:0/1775789506 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f4620068020 con 0x56143e5fb0f0 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f463759b740 10 client.25903 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f463759b740 10 client.25903 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f463759b740 20 client.25903 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f463759b740 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e689110 0x56143e68b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f463759b740 4 client.25903 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56143e56f8a0 con 0x56143e689110 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f462e59c640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e689110 0x56143e68b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f462e59c640 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e689110 0x56143e68b530 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a12713ec94807a5b server_cookie=5e016370e24af8b0 in_seq=0 out_seq=0 2024-09-10T21:09:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f4612ff5640 10 client.25903 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f4612ff5640 1 -- 172.21.15.33:0/1775789506 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56143e56f8a0 con 0x56143e689110 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.260+0000 7f4612ff5640 10 client.25903 handle_command_reply: tid=0 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 client.25903 shutdown 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 msgr2=0x56143e61c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e61c830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4620005e00 tx=0x7f46200045e0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 shutdown_connections 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56143e6079a0 0x56143e619620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56143e689110 0x56143e68b530 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56143e5fba10 0x56143e61cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 --2- 172.21.15.33:0/1775789506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56143e5fb0f0 0x56143e61c830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f4612ff5640 0 client.25903 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 >> 172.21.15.33:0/1775789506 conn(0x56143e5f8f20 msgr2=0x56143e5f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 shutdown_connections 2024-09-10T21:09:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 1 -- 172.21.15.33:0/1775789506 wait complete. 2024-09-10T21:09:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f463759b740 20 client.25903 trim_cache size 0 max 16384 2024-09-10T21:09:29.276 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46180b52e0 msgr2=0x7f46180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f46351c6640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46180b52e0 0x7f46180b77a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f4628009fd0 tx=0x7f4628009500 comp rx=0 tx=0).stop 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46301045c0 msgr2=0x7f46301997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.261+0000 7f46351c6640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46301045c0 0x7f46301997d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f462400e9c0 tx=0x7f462400ee80 comp rx=0 tx=0).stop 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 shutdown_connections 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46180b52e0 0x7f46180b77a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f463010d100 0x7f463019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46301045c0 0x7f46301997d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 --2- 172.21.15.33:0/2261171662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4630103ca0 0x7f4630199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 >> 172.21.15.33:0/2261171662 conn(0x7f46300fd700 msgr2=0x7f46300fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 shutdown_connections 2024-09-10T21:09:29.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:29.262+0000 7f46351c6640 1 -- 172.21.15.33:0/2261171662 wait complete. 2024-09-10T21:09:29.726 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476471 8.55 MB/sec execute 150 sec latency 4990.160 ms 2024-09-10T21:09:29.898 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:09:30.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:29 smithi104 ceph-mon[35325]: pgmap v511: 97 pgs: 97 active+clean; 188 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 3.0 MiB/s wr, 85 op/s 2024-09-10T21:09:30.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:29 smithi174 ceph-mon[35635]: pgmap v511: 97 pgs: 97 active+clean; 188 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 3.0 MiB/s wr, 85 op/s 2024-09-10T21:09:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:29 smithi033 ceph-mon[30860]: pgmap v511: 97 pgs: 97 active+clean; 188 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 3.0 MiB/s wr, 85 op/s 2024-09-10T21:09:30.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 -- 172.21.15.33:0/1972669437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 msgr2=0x7f03cc107b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/1972669437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc107b20 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009900 tx=0x7f03b802a410 comp rx=0 tx=0).stop 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 -- 172.21.15.33:0/1972669437 shutdown_connections 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/1972669437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03cc108060 0x7f03cc10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/1972669437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc107b20 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/1972669437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03cc101cd0 0x7f03cc1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 -- 172.21.15.33:0/1972669437 >> 172.21.15.33:0/1972669437 conn(0x7f03cc0fd700 msgr2=0x7f03cc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 -- 172.21.15.33:0/1972669437 shutdown_connections 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.692+0000 7f03d2a9a640 1 -- 172.21.15.33:0/1972669437 wait complete. 2024-09-10T21:09:30.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 Processor -- start 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 -- start start 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03cc101cd0 0x7f03cc197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03cc108060 0x7f03cc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc111cb0 con 0x7f03cc1076e0 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc111b30 con 0x7f03cc101cd0 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d2a9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc111e30 con 0x7f03cc108060 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41174/0 (socket says 172.21.15.33:41174) 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03cbfff640 1 -- 172.21.15.33:0/2088699674 learned_addr learned my addr 172.21.15.33:0/2088699674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d1037640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03cc108060 0x7f03cc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.693+0000 7f03d0836640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03cc101cd0 0x7f03cc197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03cbfff640 1 -- 172.21.15.33:0/2088699674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03cc108060 msgr2=0x7f03cc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03cbfff640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03cc108060 0x7f03cc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03cbfff640 1 -- 172.21.15.33:0/2088699674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03cc101cd0 msgr2=0x7f03cc197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03cbfff640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03cc101cd0 0x7f03cc197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03cbfff640 1 -- 172.21.15.33:0/2088699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03cc1a7670 con 0x7f03cc1076e0 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03d1037640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03cc108060 0x7f03cc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03cbfff640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc1983d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005940 tx=0x7f03b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03c9ffb640 1 -- 172.21.15.33:0/2088699674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b803f030 con 0x7f03cc1076e0 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03c9ffb640 1 -- 172.21.15.33:0/2088699674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b8003cb0 con 0x7f03cc1076e0 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03c9ffb640 1 -- 172.21.15.33:0/2088699674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8035070 con 0x7f03cc1076e0 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03cc1a7900 con 0x7f03cc1076e0 2024-09-10T21:09:30.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.694+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03cc1a7e30 con 0x7f03cc1076e0 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 Processor -- start 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03c9ffb640 1 -- 172.21.15.33:0/2088699674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f03b803bb70 con 0x7f03cc1076e0 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 -- start start 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 0x56132023fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5613201a7ea0 con 0x5613202310f0 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5613201972d0 con 0x561320231a10 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56132019b0e0 con 0x56132023d9a0 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d1037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d1037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202314d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41186/0 (socket says 172.21.15.33:41186) 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d1037640 1 -- 172.21.15.33:0/3646735787 learned_addr learned my addr 172.21.15.33:0/3646735787 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:30.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.697+0000 7f03d0836640 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d1037640 1 -- 172.21.15.33:0/3646735787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 msgr2=0x56132023fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d1037640 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 0x56132023fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d1037640 1 -- 172.21.15.33:0/3646735787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 msgr2=0x56132023d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d1037640 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d1037640 1 -- 172.21.15.33:0/3646735787 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56131fef5650 con 0x5613202310f0 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d0836640 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d1037640 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202314d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009f30 tx=0x7f03c4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0bbbdea888ded6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03c9ffb640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03bc0b5360 0x7f03bc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03d0836640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03bc0b5360 0x7f03bc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03b6ffd640 1 -- 172.21.15.33:0/3646735787 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4016070 con 0x5613202310f0 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03b6ffd640 1 -- 172.21.15.33:0/3646735787 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c401bb10 con 0x5613202310f0 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03b6ffd640 1 -- 172.21.15.33:0/3646735787 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4012bb0 con 0x5613202310f0 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.698+0000 7f03c9ffb640 1 -- 172.21.15.33:0/2088699674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f03b8106780 con 0x7f03cc1076e0 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3646735787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 msgr2=0x5613202314d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d0836640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03bc0b5360 0x7f03bc0b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03c000d370 tx=0x7f03c000d300 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202314d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009f30 tx=0x7f03c4010800 comp rx=0 tx=0).stop 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3646735787 shutdown_connections 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 0x56132023fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3646735787 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202314d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3646735787 >> 172.21.15.33:0/3646735787 conn(0x56132022ef20 msgr2=0x5613202300c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:30.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3646735787 shutdown_connections 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3646735787 wait complete. 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 -- start start 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.699+0000 7f03d4e6f740 1 Processor -- start 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613202310f0 0x56132023d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56132023d9a0 0x56132023ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56131fff8700 con 0x56132023d9a0 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561320132160 con 0x561320231a10 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561320240620 con 0x5613202310f0 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d4e6f740 10 client.0 _ll_register_callbacks cb 0x56132014b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56132023d9a0 0x56132023ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56132023d9a0 0x56132023ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41196/0 (socket says 172.21.15.33:41196) 2024-09-10T21:09:30.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 -- 172.21.15.33:0/2114486530 learned_addr learned my addr 172.21.15.33:0/2114486530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 -- 172.21.15.33:0/2114486530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613202310f0 msgr2=0x56132023d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613202310f0 0x56132023d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 -- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 msgr2=0x56132023a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03cbfff640 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 -- 172.21.15.33:0/2114486530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56132015fc90 con 0x56132023d9a0 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03cbfff640 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03d1037640 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56132023d9a0 0x56132023ae70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f03c4012830 tx=0x7f03c4009c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a6a0a89e647437 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.700+0000 7f03abfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03abfff640 1 -- 172.21.15.33:0/2114486530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4029060 con 0x56132023d9a0 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03abfff640 1 -- 172.21.15.33:0/2114486530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f03c4004d10 con 0x56132023d9a0 2024-09-10T21:09:30.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c401bb10 con 0x56132023d9a0 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03abfff640 1 -- 172.21.15.33:0/2114486530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f03c4016070 con 0x56132023d9a0 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 10 client.16389 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56132024c030 con 0x56132023d9a0 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03abfff640 1 -- 172.21.15.33:0/2114486530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c401bb10 con 0x56132023d9a0 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03abfff640 1 -- 172.21.15.33:0/2114486530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f03c402e070 con 0x56132023d9a0 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 10 client.16389 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 10 client.16389 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 20 client.16389 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56132024d430 0x56132024f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 4 client.16389 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.701+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613201a9df0 con 0x56132024d430 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.702+0000 7f03d0836640 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56132024d430 0x56132024f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.702+0000 7f03d0836640 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56132024d430 0x56132024f820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bb55b40f05a0d7 server_cookie=c9a4952281e722cc in_seq=0 out_seq=0 2024-09-10T21:09:30.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.702+0000 7f03abfff640 10 client.16389 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03abfff640 1 -- 172.21.15.33:0/2114486530 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5613201a9df0 con 0x56132024d430 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03abfff640 10 client.16389 handle_command_reply: tid=0 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 client.16389 shutdown 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56132023d9a0 msgr2=0x56132023ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56132023d9a0 0x56132023ae70 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f03c4012830 tx=0x7f03c4009c70 comp rx=0 tx=0).stop 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 shutdown_connections 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56132024d430 0x56132024f820 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56132023d9a0 0x56132023ae70 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x56132023a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2114486530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5613202310f0 0x56132023d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 >> 172.21.15.33:0/2114486530 conn(0x56132022ef20 msgr2=0x561320234ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03abfff640 0 client.16389 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 shutdown_connections 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2114486530 wait complete. 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.703+0000 7f03d4e6f740 20 client.16389 trim_cache size 0 max 16384 2024-09-10T21:09:30.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 Processor -- start 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 -- start start 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x561320230760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x561320235160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 0x561320249850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b0670 con 0x5613202310f0 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b2330 con 0x561320231a10 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.716+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b0da0 con 0x56132023d9a0 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x561320230760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x561320230760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41210/0 (socket says 172.21.15.33:41210) 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 -- 172.21.15.33:0/2719781924 learned_addr learned my addr 172.21.15.33:0/2719781924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 -- 172.21.15.33:0/2719781924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 msgr2=0x561320249850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 0x561320249850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 -- 172.21.15.33:0/2719781924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 msgr2=0x561320235160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03cbfff640 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x561320235160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x561320235160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 -- 172.21.15.33:0/2719781924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561320231840 con 0x5613202310f0 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03cbfff640 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x561320235160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03d0836640 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x561320230760 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f03c000dac0 tx=0x7f03c0014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e074ce4a275737c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.717+0000 7f03b67fc640 1 -- 172.21.15.33:0/2719781924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0012de0 con 0x5613202310f0 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.718+0000 7f03b67fc640 1 -- 172.21.15.33:0/2719781924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c002c070 con 0x5613202310f0 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.718+0000 7f03b67fc640 1 -- 172.21.15.33:0/2719781924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c001e770 con 0x5613202310f0 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2719781924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 msgr2=0x561320230760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x561320230760 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f03c000dac0 tx=0x7f03c0014070 comp rx=0 tx=0).stop 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2719781924 shutdown_connections 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56132023d9a0 0x561320249850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561320231a10 0x561320235160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/2719781924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x561320230760 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2719781924 >> 172.21.15.33:0/2719781924 conn(0x56132022ef20 msgr2=0x561320230050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2719781924 shutdown_connections 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 -- 172.21.15.33:0/2719781924 wait complete. 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 -- start start 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.719+0000 7f03d4e6f740 1 Processor -- start 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202462d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561320231a10 0x56132023ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d4e6f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132023d9a0 0x56132023f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b1ba0 con 0x5613202310f0 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1b14a0 con 0x56132023d9a0 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d4e6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03cc1ae630 con 0x561320231a10 2024-09-10T21:09:30.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d4e6f740 10 client.0 _ll_register_callbacks cb 0x7f03c4009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202462d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41224/0 (socket says 172.21.15.33:41224) 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 -- 172.21.15.33:0/3591857175 learned_addr learned my addr 172.21.15.33:0/3591857175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d1037640 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132023d9a0 0x56132023f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03cbfff640 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561320231a10 0x56132023ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 -- 172.21.15.33:0/3591857175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561320231a10 msgr2=0x56132023ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561320231a10 0x56132023ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 -- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132023d9a0 msgr2=0x56132023f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132023d9a0 0x56132023f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 -- 172.21.15.33:0/3591857175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561320241d40 con 0x5613202310f0 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03d0836640 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202462d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f03c000de60 tx=0x7f03c000de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf3735a8867fc2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:30.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.720+0000 7f03abfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03abfff640 1 -- 172.21.15.33:0/3591857175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c0005100 con 0x5613202310f0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03abfff640 1 -- 172.21.15.33:0/3591857175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f03c00052a0 con 0x5613202310f0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c002c070 con 0x5613202310f0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03abfff640 1 -- 172.21.15.33:0/3591857175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f03c00054a0 con 0x5613202310f0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 10 client.16395 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5613202ba340 con 0x5613202310f0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03abfff640 1 -- 172.21.15.33:0/3591857175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c002c070 con 0x5613202310f0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03abfff640 1 -- 172.21.15.33:0/3591857175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f03c0004b00 con 0x5613202310f0 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 10 client.16395 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:30.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 20 client.16395 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613202baad0 0x5613202bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 4 client.16395 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:09:30.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.721+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5613201a58a0 con 0x5613202baad0 2024-09-10T21:09:30.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476471 8.50 MB/sec execute 151 sec latency 1445.799 ms 2024-09-10T21:09:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03cbfff640 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613202baad0 0x5613202bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03cbfff640 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613202baad0 0x5613202bcef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0bc19c8979be3e server_cookie=885241edff86302c in_seq=0 out_seq=0 2024-09-10T21:09:30.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03abfff640 10 client.16395 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03abfff640 1 -- 172.21.15.33:0/3591857175 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5613201a58a0 con 0x5613202baad0 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03abfff640 10 client.16395 handle_command_reply: tid=0 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 client.16395 shutdown 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 msgr2=0x5613202462d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202462d0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f03c000de60 tx=0x7f03c000de90 comp rx=0 tx=0).stop 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 shutdown_connections 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5613202baad0 0x5613202bcef0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56132023d9a0 0x56132023f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561320231a10 0x56132023ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 --2- 172.21.15.33:0/3591857175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5613202310f0 0x5613202462d0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 >> 172.21.15.33:0/3591857175 conn(0x56132022ef20 msgr2=0x56132023bdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03abfff640 0 client.16395 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:30.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.722+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 shutdown_connections 2024-09-10T21:09:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d4e6f740 1 -- 172.21.15.33:0/3591857175 wait complete. 2024-09-10T21:09:30.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d4e6f740 20 client.16395 trim_cache size 0 max 16384 2024-09-10T21:09:30.744 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03bc0b5360 msgr2=0x7f03bc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03bc0b5360 0x7f03bc0b7820 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03c000d370 tx=0x7f03c000d300 comp rx=0 tx=0).stop 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 msgr2=0x7f03cc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc1983d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f03b8005940 tx=0x7f03b80042f0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 shutdown_connections 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03bc0b5360 0x7f03bc0b7820 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03cc108060 0x7f03cc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc1076e0 0x7f03cc1983d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 --2- 172.21.15.33:0/2088699674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03cc101cd0 0x7f03cc197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:30.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 >> 172.21.15.33:0/2088699674 conn(0x7f03cc0fd700 msgr2=0x7f03cc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:30.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 shutdown_connections 2024-09-10T21:09:30.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:30.723+0000 7f03d2a9a640 1 -- 172.21.15.33:0/2088699674 wait complete. 2024-09-10T21:09:31.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476471 8.44 MB/sec execute 152 sec latency 2445.887 ms 2024-09-10T21:09:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:31 smithi033 ceph-mon[30860]: pgmap v512: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2024-09-10T21:09:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:31 smithi104 ceph-mon[35325]: pgmap v512: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2024-09-10T21:09:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:31 smithi174 ceph-mon[35635]: pgmap v512: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 2.8 MiB/s wr, 72 op/s 2024-09-10T21:09:32.392 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:09:32.396 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:09:32.663 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:32.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 476471 8.39 MB/sec execute 153 sec latency 3445.969 ms 2024-09-10T21:09:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 -- 172.21.15.33:0/1511714221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc102da0 msgr2=0x7f98dc103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 --2- 172.21.15.33:0/1511714221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc102da0 0x7f98dc103180 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009900 tx=0x7f98d402a410 comp rx=0 tx=0).stop 2024-09-10T21:09:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 -- 172.21.15.33:0/1511714221 shutdown_connections 2024-09-10T21:09:33.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 --2- 172.21.15.33:0/1511714221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98dc109130 0x7f98dc10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 --2- 172.21.15.33:0/1511714221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98dc1087b0 0x7f98dc108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 --2- 172.21.15.33:0/1511714221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc102da0 0x7f98dc103180 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 -- 172.21.15.33:0/1511714221 >> 172.21.15.33:0/1511714221 conn(0x7f98dc0fe810 msgr2=0x7f98dc100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 -- 172.21.15.33:0/1511714221 shutdown_connections 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.140+0000 7f98e38be640 1 -- 172.21.15.33:0/1511714221 wait complete. 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 Processor -- start 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 -- start start 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98dc102da0 0x7f98dc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98dc1087b0 0x7f98dc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc109130 0x7f98dc19eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc112df0 con 0x7f98dc109130 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc112c70 con 0x7f98dc1087b0 2024-09-10T21:09:33.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e38be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc112f70 con 0x7f98dc102da0 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e1e5b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc109130 0x7f98dc19eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e165a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98dc102da0 0x7f98dc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e1e5b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc109130 0x7f98dc19eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41246/0 (socket says 172.21.15.33:41246) 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.141+0000 7f98e1e5b640 1 -- 172.21.15.33:0/960020013 learned_addr learned my addr 172.21.15.33:0/960020013 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e0e59640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98dc1087b0 0x7f98dc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e1e5b640 1 -- 172.21.15.33:0/960020013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98dc102da0 msgr2=0x7f98dc199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e1e5b640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98dc102da0 0x7f98dc199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e1e5b640 1 -- 172.21.15.33:0/960020013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98dc1087b0 msgr2=0x7f98dc1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e1e5b640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98dc1087b0 0x7f98dc1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e1e5b640 1 -- 172.21.15.33:0/960020013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98dc19f310 con 0x7f98dc109130 2024-09-10T21:09:33.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e1e5b640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc109130 0x7f98dc19eb40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00cce0 tx=0x7f98cc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98da7fc640 1 -- 172.21.15.33:0/960020013 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00f450 con 0x7f98dc109130 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98da7fc640 1 -- 172.21.15.33:0/960020013 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98cc00faa0 con 0x7f98dc109130 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98dc19f600 con 0x7f98dc109130 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98da7fc640 1 -- 172.21.15.33:0/960020013 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98cc018b70 con 0x7f98dc109130 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.142+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98dc1a5d40 con 0x7f98dc109130 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98da7fc640 1 -- 172.21.15.33:0/960020013 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f98cc00f5f0 con 0x7f98dc109130 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 Processor -- start 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 -- start start 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc2ee0 0x564d4adc32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adcf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564d4ad39e60 con 0x564d4adcf790 2024-09-10T21:09:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564d4ad29290 con 0x564d4adc3800 2024-09-10T21:09:33.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.145+0000 7f98e5c93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d4ad2d0a0 con 0x564d4adc2ee0 2024-09-10T21:09:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41260/0 (socket says 172.21.15.33:41260) 2024-09-10T21:09:33.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 -- 172.21.15.33:0/3808522779 learned_addr learned my addr 172.21.15.33:0/3808522779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e165a640 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adcf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc2ee0 0x564d4adc32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98da7fc640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98c40b5360 0x7f98c40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 -- 172.21.15.33:0/3808522779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc2ee0 msgr2=0x564d4adc32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc2ee0 0x564d4adc32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 -- 172.21.15.33:0/3808522779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 msgr2=0x564d4adcf250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adcf250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 -- 172.21.15.33:0/3808522779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4aa87930 con 0x564d4adcf790 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e1e5b640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98c40b5360 0x7f98c40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98da7fc640 1 -- 172.21.15.33:0/960020013 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f98cc0e62e0 con 0x7f98dc109130 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98e0e59640 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add1b80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f98d000adb0 tx=0x7f98d000a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233c56970fc0f417 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.146+0000 7f98c37fe640 1 -- 172.21.15.33:0/3808522779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d001b070 con 0x564d4adcf790 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98c37fe640 1 -- 172.21.15.33:0/3808522779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d000fdf0 con 0x564d4adcf790 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98c37fe640 1 -- 172.21.15.33:0/3808522779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d00145f0 con 0x564d4adcf790 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 -- 172.21.15.33:0/3808522779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 msgr2=0x564d4add1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e1e5b640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98c40b5360 0x7f98c40b7820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0aca30 tx=0x7f98cc071600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add1b80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f98d000adb0 tx=0x7f98d000a490 comp rx=0 tx=0).stop 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 -- 172.21.15.33:0/3808522779 shutdown_connections 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add1b80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adcf250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 --2- 172.21.15.33:0/3808522779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc2ee0 0x564d4adc32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 -- 172.21.15.33:0/3808522779 >> 172.21.15.33:0/3808522779 conn(0x564d4adc0d10 msgr2=0x564d4adc1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 -- 172.21.15.33:0/3808522779 shutdown_connections 2024-09-10T21:09:33.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.147+0000 7f98e5c93740 1 -- 172.21.15.33:0/3808522779 wait complete. 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 -- start start 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 Processor -- start 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc2ee0 0x564d4adcc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adccd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 0x564d4adcd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564d4ab8a890 con 0x564d4adc2ee0 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564d4acc3d40 con 0x564d4adc3800 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d4add2440 con 0x564d4adcf790 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e5c93740 10 client.0 _ll_register_callbacks cb 0x564d4ad27c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e0e59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc2ee0 0x564d4adcc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e0e59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc2ee0 0x564d4adcc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41262/0 (socket says 172.21.15.33:41262) 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.148+0000 7f98e0e59640 1 -- 172.21.15.33:0/2580759491 learned_addr learned my addr 172.21.15.33:0/2580759491 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:33.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e0e59640 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 0x564d4adcd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e165a640 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adccd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e0e59640 1 -- 172.21.15.33:0/2580759491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 msgr2=0x564d4adcd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e0e59640 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 0x564d4adcd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e0e59640 1 -- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 msgr2=0x564d4adccd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e0e59640 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adccd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e0e59640 1 -- 172.21.15.33:0/2580759491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4addb8f0 con 0x564d4adc2ee0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e0e59640 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc2ee0 0x564d4adcc820 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98d0016fd0 tx=0x7f98d0006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccfaab0f8634cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98c0ff9640 1 -- 172.21.15.33:0/2580759491 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d002a060 con 0x564d4adc2ee0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98c0ff9640 1 -- 172.21.15.33:0/2580759491 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f98d0025420 con 0x564d4adc2ee0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d000fdf0 con 0x564d4adc2ee0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98c0ff9640 1 -- 172.21.15.33:0/2580759491 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f98d001b070 con 0x564d4adc2ee0 2024-09-10T21:09:33.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e5c93740 10 client.16407 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.149+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564d4adde2a0 con 0x564d4adc2ee0 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98c0ff9640 1 -- 172.21.15.33:0/2580759491 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d000fdf0 con 0x564d4adc2ee0 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98c0ff9640 1 -- 172.21.15.33:0/2580759491 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f98d002f070 con 0x564d4adc2ee0 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98e5c93740 10 client.16407 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98e5c93740 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98e5c93740 20 client.16407 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98e5c93740 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4addf650 0x564d4ade1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98e5c93740 4 client.16407 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:33.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d4ad3bdb0 con 0x564d4addf650 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.150+0000 7f98e165a640 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4addf650 0x564d4ade1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.151+0000 7f98e165a640 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4addf650 0x564d4ade1a40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6eea5cf9fefd0 server_cookie=2cabf6d9e9079a23 in_seq=0 out_seq=0 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.151+0000 7f98c0ff9640 10 client.16407 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.151+0000 7f98c0ff9640 1 -- 172.21.15.33:0/2580759491 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d4ad3bdb0 con 0x564d4addf650 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.151+0000 7f98c0ff9640 10 client.16407 handle_command_reply: tid=0 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.151+0000 7f98e5c93740 1 client.16407 shutdown 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.151+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc2ee0 msgr2=0x564d4adcc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.151+0000 7f98e5c93740 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc2ee0 0x564d4adcc820 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f98d0016fd0 tx=0x7f98d0006d80 comp rx=0 tx=0).stop 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 shutdown_connections 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4addf650 0x564d4ade1a40 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 0x564d4adcd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc3800 0x564d4adccd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 --2- 172.21.15.33:0/2580759491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc2ee0 0x564d4adcc820 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 >> 172.21.15.33:0/2580759491 conn(0x564d4adc0d10 msgr2=0x564d4adbff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98c0ff9640 0 client.16407 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 shutdown_connections 2024-09-10T21:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 1 -- 172.21.15.33:0/2580759491 wait complete. 2024-09-10T21:09:33.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.152+0000 7f98e5c93740 20 client.16407 trim_cache size 0 max 16384 2024-09-10T21:09:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.169+0000 7f98e5c93740 1 Processor -- start 2024-09-10T21:09:33.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.169+0000 7f98e5c93740 1 -- start start 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4adc9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc3800 0x564d4adc9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 0x564d4ade70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e5c93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1ae420 con 0x564d4adc3800 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e5c93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1b00e0 con 0x564d4adc2ee0 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e5c93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1aeb50 con 0x564d4adcf790 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc3800 0x564d4adc9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc3800 0x564d4adc9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41272/0 (socket says 172.21.15.33:41272) 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 -- 172.21.15.33:0/1255991000 learned_addr learned my addr 172.21.15.33:0/1255991000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e0e59640 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4adc9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 -- 172.21.15.33:0/1255991000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 msgr2=0x564d4ade70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 0x564d4ade70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 -- 172.21.15.33:0/1255991000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 msgr2=0x564d4adc9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4adc9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 -- 172.21.15.33:0/1255991000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d4adbc640 con 0x564d4adc3800 2024-09-10T21:09:33.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98e165a640 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc3800 0x564d4adc9f70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f98d402a930 tx=0x7f98d4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aeb6a3c9f6c0507 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:33.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.170+0000 7f98c2ffd640 1 -- 172.21.15.33:0/1255991000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4049070 con 0x564d4adc3800 2024-09-10T21:09:33.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98c2ffd640 1 -- 172.21.15.33:0/1255991000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d4031950 con 0x564d4adc3800 2024-09-10T21:09:33.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98c2ffd640 1 -- 172.21.15.33:0/1255991000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d4035040 con 0x564d4adc3800 2024-09-10T21:09:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 -- 172.21.15.33:0/1255991000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc3800 msgr2=0x564d4adc9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc3800 0x564d4adc9f70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f98d402a930 tx=0x7f98d4005e80 comp rx=0 tx=0).stop 2024-09-10T21:09:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 -- 172.21.15.33:0/1255991000 shutdown_connections 2024-09-10T21:09:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adcf790 0x564d4ade70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adc3800 0x564d4adc9f70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 --2- 172.21.15.33:0/1255991000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4adc9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 -- 172.21.15.33:0/1255991000 >> 172.21.15.33:0/1255991000 conn(0x564d4adc0d10 msgr2=0x564d4adcb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 -- 172.21.15.33:0/1255991000 shutdown_connections 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 -- 172.21.15.33:0/1255991000 wait complete. 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.171+0000 7f98e5c93740 1 -- start start 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 1 Processor -- start 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4add3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc3800 0x564d4add3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1af950 con 0x564d4adcf790 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1af250 con 0x564d4adc2ee0 2024-09-10T21:09:33.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98dc1ac3e0 con 0x564d4adc3800 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e5c93740 10 client.0 _ll_register_callbacks cb 0x7f98d0019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41276/0 (socket says 172.21.15.33:41276) 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 -- 172.21.15.33:0/188859190 learned_addr learned my addr 172.21.15.33:0/188859190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4add3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e165a640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc3800 0x564d4add3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 -- 172.21.15.33:0/188859190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc3800 msgr2=0x564d4add3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc3800 0x564d4add3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 -- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 msgr2=0x564d4add3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4add3730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e0e59640 1 -- 172.21.15.33:0/188859190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d4ae4b400 con 0x564d4adcf790 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.172+0000 7f98e165a640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc3800 0x564d4add3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:33.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e0e59640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add41b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f98d00307e0 tx=0x7f98d0006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3b1309cfd408a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98c0ff9640 1 -- 172.21.15.33:0/188859190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d0035070 con 0x564d4adcf790 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98c0ff9640 1 -- 172.21.15.33:0/188859190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f98d000fb70 con 0x564d4adcf790 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d4031950 con 0x564d4adcf790 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98c0ff9640 1 -- 172.21.15.33:0/188859190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f98d002a040 con 0x564d4adcf790 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 10 client.16413 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x564d4ae4c720 con 0x564d4adcf790 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98c0ff9640 1 -- 172.21.15.33:0/188859190 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d4031950 con 0x564d4adcf790 2024-09-10T21:09:33.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98c0ff9640 1 -- 172.21.15.33:0/188859190 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f98d002f070 con 0x564d4adcf790 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 10 client.16413 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 20 client.16413 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4ae4cde0 0x564d4ae4f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 4 client.16413 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.173+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564d4ad37860 con 0x564d4ae4cde0 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.174+0000 7f98e165a640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4ae4cde0 0x564d4ae4f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.174+0000 7f98c0ff9640 10 client.16413 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:33.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.174+0000 7f98e165a640 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4ae4cde0 0x564d4ae4f200 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7f81a3e835112 server_cookie=5fed76f23e83e298 in_seq=0 out_seq=0 2024-09-10T21:09:33.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98c0ff9640 1 -- 172.21.15.33:0/188859190 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564d4ad37860 con 0x564d4ae4cde0 2024-09-10T21:09:33.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98c0ff9640 10 client.16413 handle_command_reply: tid=0 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 client.16413 shutdown 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 msgr2=0x564d4add41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add41b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f98d00307e0 tx=0x7f98d0006d80 comp rx=0 tx=0).stop 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 shutdown_connections 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d4ae4cde0 0x564d4ae4f200 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d4adcf790 0x564d4add41b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d4adc3800 0x564d4add3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 --2- 172.21.15.33:0/188859190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d4adc2ee0 0x564d4add3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98c0ff9640 0 client.16413 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 >> 172.21.15.33:0/188859190 conn(0x564d4adc0d10 msgr2=0x564d4adcde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:33.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 shutdown_connections 2024-09-10T21:09:33.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 1 -- 172.21.15.33:0/188859190 wait complete. 2024-09-10T21:09:33.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.176+0000 7f98e5c93740 20 client.16413 trim_cache size 0 max 16384 2024-09-10T21:09:33.187 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:09:33.187 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:09:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "232d6926-c99d-4411-a1a9-c37d6c417870", 2024-09-10T21:09:33.188 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:09:33.188 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:09:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98c40b5360 msgr2=0x7f98c40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98c40b5360 0x7f98c40b7820 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f98cc0aca30 tx=0x7f98cc071600 comp rx=0 tx=0).stop 2024-09-10T21:09:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc109130 msgr2=0x7f98dc19eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:33.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc109130 0x7f98dc19eb40 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00cce0 tx=0x7f98cc00a620 comp rx=0 tx=0).stop 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 shutdown_connections 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98c40b5360 0x7f98c40b7820 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc109130 0x7f98dc19eb40 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f98dc1087b0 0x7f98dc1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 --2- 172.21.15.33:0/960020013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98dc102da0 0x7f98dc199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 >> 172.21.15.33:0/960020013 conn(0x7f98dc0fe810 msgr2=0x7f98dc100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 shutdown_connections 2024-09-10T21:09:33.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:33.177+0000 7f98e38be640 1 -- 172.21.15.33:0/960020013 wait complete. 2024-09-10T21:09:33.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 477146 8.34 MB/sec execute 154 sec latency 3978.966 ms 2024-09-10T21:09:33.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:33 smithi033 ceph-mon[30860]: pgmap v513: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.6 MiB/s wr, 64 op/s 2024-09-10T21:09:33.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:33 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:33.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:33 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:33.890 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:33 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:33.891 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:33 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:33.918 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:09:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:33 smithi104 ceph-mon[35325]: pgmap v513: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.6 MiB/s wr, 64 op/s 2024-09-10T21:09:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:33 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:34.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:33 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:33 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:34.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:33 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:33 smithi174 ceph-mon[35635]: pgmap v513: 97 pgs: 97 active+clean; 187 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 2.6 MiB/s wr, 64 op/s 2024-09-10T21:09:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:33 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:33 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:33 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:33 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.193 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:09:34.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 -- 172.21.15.33:0/1343806368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c075540 msgr2=0x7f559c075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 --2- 172.21.15.33:0/1343806368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c075540 0x7f559c075920 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f558c009900 tx=0x7f558c02a410 comp rx=0 tx=0).stop 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 -- 172.21.15.33:0/1343806368 shutdown_connections 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 --2- 172.21.15.33:0/1343806368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f559c113b20 0x7f559c115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 --2- 172.21.15.33:0/1343806368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c075e60 0x7f559c1135e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 --2- 172.21.15.33:0/1343806368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c075540 0x7f559c075920 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 -- 172.21.15.33:0/1343806368 >> 172.21.15.33:0/1343806368 conn(0x7f559c0fe830 msgr2=0x7f559c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 -- 172.21.15.33:0/1343806368 shutdown_connections 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.701+0000 7f55a23a2640 1 -- 172.21.15.33:0/1343806368 wait complete. 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 Processor -- start 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 -- start start 2024-09-10T21:09:34.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c075540 0x7f559c19f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f559c075e60 0x7f559c19fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c113b20 0x7f559c1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1193c0 con 0x7f559c113b20 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f559c119240 con 0x7f559c075e60 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.702+0000 7f55a23a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f559c119540 con 0x7f559c075540 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c113b20 0x7f559c1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c113b20 0x7f559c1a5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41296/0 (socket says 172.21.15.33:41296) 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 -- 172.21.15.33:0/3263810496 learned_addr learned my addr 172.21.15.33:0/3263810496 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f559bfff640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c075540 0x7f559c19f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f559b7fe640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f559c075e60 0x7f559c19fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 -- 172.21.15.33:0/3263810496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c075540 msgr2=0x7f559c19f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c075540 0x7f559c19f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 -- 172.21.15.33:0/3263810496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f559c075e60 msgr2=0x7f559c19fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f559c075e60 0x7f559c19fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 -- 172.21.15.33:0/3263810496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559c1a5920 con 0x7f559c113b20 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f559bfff640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c075540 0x7f559c19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a093f640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c113b20 0x7f559c1a5150 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f559400ccb0 tx=0x7f559400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55997fa640 1 -- 172.21.15.33:0/3263810496 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559400f450 con 0x7f559c113b20 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559c1a5c10 con 0x7f559c113b20 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55997fa640 1 -- 172.21.15.33:0/3263810496 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559400faa0 con 0x7f559c113b20 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55997fa640 1 -- 172.21.15.33:0/3263810496 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5594018b00 con 0x7f559c113b20 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.703+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559c1ac2d0 con 0x7f559c113b20 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.706+0000 7f55a4777740 1 Processor -- start 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.706+0000 7f55997fa640 1 -- 172.21.15.33:0/3263810496 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f559400f5f0 con 0x7f559c113b20 2024-09-10T21:09:34.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55a4777740 1 -- start start 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55a4777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca90a0 0x559795ca9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55a4777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cb5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55a4777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795cb5950 0x559795cb7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55a4777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559795c1fe50 con 0x559795cb5950 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55a4777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559795c0f280 con 0x559795ca99c0 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55a4777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559795c13090 con 0x559795ca90a0 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795cb5950 0x559795cb7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795cb5950 0x559795cb7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41308/0 (socket says 172.21.15.33:41308) 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 -- 172.21.15.33:0/810267206 learned_addr learned my addr 172.21.15.33:0/810267206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559bfff640 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cb5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca90a0 0x559795ca9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 -- 172.21.15.33:0/810267206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca90a0 msgr2=0x559795ca9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca90a0 0x559795ca9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 -- 172.21.15.33:0/810267206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 msgr2=0x559795cb5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cb5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f559b7fe640 1 -- 172.21.15.33:0/810267206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55979596d650 con 0x559795cb5950 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.707+0000 7f55997fa640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f55840b52e0 0x7f55840b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f559b7fe640 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795cb5950 0x559795cb7d40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5590013620 tx=0x7f5590009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26a8bd2b85c7a42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f559bfff640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f55840b52e0 0x7f55840b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f55997fa640 1 -- 172.21.15.33:0/3263810496 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f55940e62e0 con 0x7f559c113b20 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f55827fc640 1 -- 172.21.15.33:0/810267206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590004050 con 0x559795cb5950 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f55827fc640 1 -- 172.21.15.33:0/810267206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55900041f0 con 0x559795cb5950 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f55827fc640 1 -- 172.21.15.33:0/810267206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559000f7e0 con 0x559795cb5950 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f55a4777740 1 -- 172.21.15.33:0/810267206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795cb5950 msgr2=0x559795cb7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f559bfff640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f55840b52e0 0x7f55840b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f558c031440 tx=0x7f558c032ed0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.708+0000 7f55a4777740 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795cb5950 0x559795cb7d40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f5590013620 tx=0x7f5590009a00 comp rx=0 tx=0).stop 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- 172.21.15.33:0/810267206 shutdown_connections 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795cb5950 0x559795cb7d40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cb5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 --2- 172.21.15.33:0/810267206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca90a0 0x559795ca9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- 172.21.15.33:0/810267206 >> 172.21.15.33:0/810267206 conn(0x559795ca6ed0 msgr2=0x559795ca8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- 172.21.15.33:0/810267206 shutdown_connections 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- 172.21.15.33:0/810267206 wait complete. 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- start start 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 Processor -- start 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca90a0 0x559795cb5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca99c0 0x559795cb26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559795a70700 con 0x559795ca99c0 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559795baa110 con 0x559795ca90a0 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559795cb85d0 con 0x559795cb5950 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.709+0000 7f55a4777740 10 client.0 _ll_register_callbacks cb 0x559795bc3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca99c0 0x559795cb26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca90a0 0x559795cb5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca99c0 0x559795cb26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41316/0 (socket says 172.21.15.33:41316) 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 -- 172.21.15.33:0/1078532632 learned_addr learned my addr 172.21.15.33:0/1078532632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f55a093f640 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 -- 172.21.15.33:0/1078532632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 msgr2=0x559795cb2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 -- 172.21.15.33:0/1078532632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca90a0 msgr2=0x559795cb5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca90a0 0x559795cb5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 -- 172.21.15.33:0/1078532632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559795bd7c40 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f55a093f640 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f559b7fe640 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca99c0 0x559795cb26b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5590006010 tx=0x7f55900056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f17b3c61df000d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f557b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f557b7fe640 1 -- 172.21.15.33:0/1078532632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5590005c50 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55900041f0 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f557b7fe640 1 -- 172.21.15.33:0/1078532632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f559000f580 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f557b7fe640 1 -- 172.21.15.33:0/1078532632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f559000f780 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f55a4777740 10 client.16425 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.710+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559795cc3f30 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f557b7fe640 1 -- 172.21.15.33:0/1078532632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55900041f0 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f557b7fe640 1 -- 172.21.15.33:0/1078532632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f5590028020 con 0x559795ca99c0 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f55a4777740 10 client.16425 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:34.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f55a4777740 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f55a4777740 20 client.16425 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f55a4777740 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795cc7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f55a4777740 4 client.16425 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559795c21da0 con 0x559795cc5320 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.711+0000 7f559bfff640 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795cc7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f559bfff640 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795cc7710 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3c3b6907d027309 server_cookie=9d795b7d1ee817da in_seq=0 out_seq=0 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f557b7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f557b7fe640 1 -- 172.21.15.33:0/1078532632 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559795c21da0 con 0x559795cc5320 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f557b7fe640 10 client.16425 handle_command_reply: tid=0 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 client.16425 shutdown 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca99c0 msgr2=0x559795cb26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca99c0 0x559795cb26b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5590006010 tx=0x7f55900056d0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 shutdown_connections 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795cc7710 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca99c0 0x559795cb26b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f557b7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 --2- 172.21.15.33:0/1078532632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca90a0 0x559795cb5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.712+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 >> 172.21.15.33:0/1078532632 conn(0x559795ca6ed0 msgr2=0x559795caca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.713+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 shutdown_connections 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.713+0000 7f55a4777740 1 -- 172.21.15.33:0/1078532632 wait complete. 2024-09-10T21:09:34.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.713+0000 7f55a4777740 20 client.16425 trim_cache size 0 max 16384 2024-09-10T21:09:34.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479677 8.39 MB/sec execute 155 sec latency 46.725 ms 2024-09-10T21:09:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.730+0000 7f55a4777740 1 Processor -- start 2024-09-10T21:09:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.730+0000 7f55a4777740 1 -- start start 2024-09-10T21:09:34.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f55a4777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795ca6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f55a4777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca99c0 0x559795cb6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f55a4777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795cb5950 0x559795cb7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f55a4777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b49d0 con 0x559795ca90a0 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f55a4777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b6690 con 0x559795cb5950 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f55a4777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b5100 con 0x559795ca99c0 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795ca6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795ca6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41328/0 (socket says 172.21.15.33:41328) 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 -- 172.21.15.33:0/1263920112 learned_addr learned my addr 172.21.15.33:0/1263920112 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f55a093f640 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795cb5950 0x559795cb7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559b7fe640 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca99c0 0x559795cb6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 -- 172.21.15.33:0/1263920112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca99c0 msgr2=0x559795cb6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca99c0 0x559795cb6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 -- 172.21.15.33:0/1263920112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795cb5950 msgr2=0x559795cb7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795cb5950 0x559795cb7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 -- 172.21.15.33:0/1263920112 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559795cc7a80 con 0x559795ca90a0 2024-09-10T21:09:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f559bfff640 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795ca6710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f558c0096c0 tx=0x7f558c03f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc563974907177f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.731+0000 7f5581ffb640 1 -- 172.21.15.33:0/1263920112 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c035040 con 0x559795ca90a0 2024-09-10T21:09:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f5581ffb640 1 -- 172.21.15.33:0/1263920112 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c004770 con 0x559795ca90a0 2024-09-10T21:09:34.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f5581ffb640 1 -- 172.21.15.33:0/1263920112 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c003850 con 0x559795ca90a0 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 -- 172.21.15.33:0/1263920112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 msgr2=0x559795ca6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795ca6710 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f558c0096c0 tx=0x7f558c03f000 comp rx=0 tx=0).stop 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 -- 172.21.15.33:0/1263920112 shutdown_connections 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795cb5950 0x559795cb7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795ca99c0 0x559795cb6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 --2- 172.21.15.33:0/1263920112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795ca6710 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 -- 172.21.15.33:0/1263920112 >> 172.21.15.33:0/1263920112 conn(0x559795ca6ed0 msgr2=0x559795cb0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 -- 172.21.15.33:0/1263920112 shutdown_connections 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 -- 172.21.15.33:0/1263920112 wait complete. 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.732+0000 7f55a4777740 1 -- start start 2024-09-10T21:09:34.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 1 Processor -- start 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795cbd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cbddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b5f00 con 0x559795ca90a0 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b5800 con 0x559795ca99c0 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f559c1b2990 con 0x559795cb5950 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a4777740 10 client.0 _ll_register_callbacks cb 0x7f558c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795cbd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795cbd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41342/0 (socket says 172.21.15.33:41342) 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559bfff640 1 -- 172.21.15.33:0/262986542 learned_addr learned my addr 172.21.15.33:0/262986542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:09:34.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559b7fe640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cbddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f55a093f640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559bfff640 1 -- 172.21.15.33:0/262986542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 msgr2=0x559795cb8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559bfff640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559bfff640 1 -- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 msgr2=0x559795cbddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.733+0000 7f559bfff640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cbddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f559bfff640 1 -- 172.21.15.33:0/262986542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559795cbe4a0 con 0x559795ca90a0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f559b7fe640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cbddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f559bfff640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795cbd870 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f558c0045f0 tx=0x7f558c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1bcc2e1b84d214f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f557b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f557b7fe640 1 -- 172.21.15.33:0/262986542 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558c069060 con 0x559795ca90a0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f557b7fe640 1 -- 172.21.15.33:0/262986542 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f558c0039c0 con 0x559795ca90a0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f558c004770 con 0x559795ca90a0 2024-09-10T21:09:34.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f557b7fe640 1 -- 172.21.15.33:0/262986542 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f558c003bc0 con 0x559795ca90a0 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f55a4777740 10 client.16431 fetch_fsmap learned FSMap version 49 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559795d32690 con 0x559795ca90a0 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.734+0000 7f557b7fe640 1 -- 172.21.15.33:0/262986542 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f558c004770 con 0x559795ca90a0 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f557b7fe640 1 -- 172.21.15.33:0/262986542 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f558c059020 con 0x559795ca90a0 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f55a4777740 10 client.16431 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f55a4777740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f55a4777740 20 client.16431 populate_metadata read hostname 'smithi033' 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f55a4777740 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795d34e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f55a4777740 4 client.16431 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559795c1d850 con 0x559795cc5320 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f559b7fe640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795d34e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f559b7fe640 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795d34e00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3684938299cf2671 server_cookie=858d391d5f0eadea in_seq=0 out_seq=0 2024-09-10T21:09:34.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.735+0000 7f557b7fe640 10 client.16431 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f557b7fe640 1 -- 172.21.15.33:0/262986542 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2558 (crc 0 0 0) 0x559795c1d850 con 0x559795cc5320 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f557b7fe640 10 client.16431 handle_command_reply: tid=0 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 client.16431 shutdown 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 msgr2=0x559795cbd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795cbd870 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f558c0045f0 tx=0x7f558c004040 comp rx=0 tx=0).stop 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 shutdown_connections 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559795cc5320 0x559795d34e00 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559795cb5950 0x559795cb8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559795ca99c0 0x559795cbddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 --2- 172.21.15.33:0/262986542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559795ca90a0 0x559795cbd870 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f557b7fe640 0 client.16431 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 >> 172.21.15.33:0/262986542 conn(0x559795ca6ed0 msgr2=0x559795cb3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 shutdown_connections 2024-09-10T21:09:34.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 1 -- 172.21.15.33:0/262986542 wait complete. 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.736+0000 7f55a4777740 20 client.16431 trim_cache size 0 max 16384 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "232d6926-c99d-4411-a1a9-c37d6c417870", 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:09:34.746 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:09:34.747 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.748 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.749 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.750 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:09:34.751 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:09:34.752 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:09:34.752 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:09:34.752 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:09:34.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f55840b52e0 msgr2=0x7f55840b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f55840b52e0 0x7f55840b77a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f558c031440 tx=0x7f558c032ed0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c113b20 msgr2=0x7f559c1a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c113b20 0x7f559c1a5150 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f559400ccb0 tx=0x7f559400a620 comp rx=0 tx=0).stop 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 shutdown_connections 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f55840b52e0 0x7f55840b77a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f559c113b20 0x7f559c1a5150 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f559c075e60 0x7f559c19fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 --2- 172.21.15.33:0/3263810496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559c075540 0x7f559c19f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 >> 172.21.15.33:0/3263810496 conn(0x7f559c0fe830 msgr2=0x7f559c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 shutdown_connections 2024-09-10T21:09:34.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:09:34.737+0000 7f55a23a2640 1 -- 172.21.15.33:0/3263810496 wait complete. 2024-09-10T21:09:34.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:09:34.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:34.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:34 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:34 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:35.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:09:35.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:34 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:09:35.472 INFO:tasks.cephfs.filesystem:scrub status for tag:232d6926-c99d-4411-a1a9-c37d6c417870 - {'path': '/', 'tag': '232d6926-c99d-4411-a1a9-c37d6c417870', 'options': 'recursive,force'} 2024-09-10T21:09:35.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479677 8.34 MB/sec execute 156 sec latency 1046.818 ms 2024-09-10T21:09:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:35 smithi104 ceph-mon[35325]: pgmap v514: 97 pgs: 97 active+clean; 186 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.5 MiB/s wr, 70 op/s 2024-09-10T21:09:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:35 smithi174 ceph-mon[35635]: pgmap v514: 97 pgs: 97 active+clean; 186 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.5 MiB/s wr, 70 op/s 2024-09-10T21:09:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:35 smithi033 ceph-mon[30860]: pgmap v514: 97 pgs: 97 active+clean; 186 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 2.5 MiB/s wr, 70 op/s 2024-09-10T21:09:36.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479677 8.28 MB/sec execute 157 sec latency 2046.927 ms 2024-09-10T21:09:37.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479742 8.23 MB/sec execute 158 sec latency 2419.659 ms 2024-09-10T21:09:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:37 smithi104 ceph-mon[35325]: pgmap v515: 97 pgs: 97 active+clean; 186 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-09-10T21:09:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:37 smithi174 ceph-mon[35635]: pgmap v515: 97 pgs: 97 active+clean; 186 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-09-10T21:09:38.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:37 smithi033 ceph-mon[30860]: pgmap v515: 97 pgs: 97 active+clean; 186 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 1.3 MiB/s wr, 53 op/s 2024-09-10T21:09:38.727 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479742 8.18 MB/sec execute 159 sec latency 1606.338 ms 2024-09-10T21:09:39.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479742 8.13 MB/sec execute 160 sec latency 2606.393 ms 2024-09-10T21:09:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:39 smithi104 ceph-mon[35325]: pgmap v516: 97 pgs: 97 active+clean; 189 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-09-10T21:09:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:39 smithi174 ceph-mon[35635]: pgmap v516: 97 pgs: 97 active+clean; 189 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-09-10T21:09:40.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:39 smithi033 ceph-mon[30860]: pgmap v516: 97 pgs: 97 active+clean; 189 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 1.1 MiB/s wr, 83 op/s 2024-09-10T21:09:40.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479742 8.08 MB/sec execute 161 sec latency 3606.790 ms 2024-09-10T21:09:41.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479742 8.03 MB/sec execute 162 sec latency 4606.893 ms 2024-09-10T21:09:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:41 smithi104 ceph-mon[35325]: pgmap v517: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 674 KiB/s wr, 74 op/s 2024-09-10T21:09:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:09:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:09:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:09:42.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:09:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:41 smithi174 ceph-mon[35635]: pgmap v517: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 674 KiB/s wr, 74 op/s 2024-09-10T21:09:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:09:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:09:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:09:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:09:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:41 smithi033 ceph-mon[30860]: pgmap v517: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 674 KiB/s wr, 74 op/s 2024-09-10T21:09:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:09:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:09:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:09:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:09:42.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479829 7.98 MB/sec execute 163 sec latency 4980.801 ms 2024-09-10T21:09:43.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:09:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:09:43.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:09:43.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479829 7.93 MB/sec execute 164 sec latency 1530.098 ms 2024-09-10T21:09:44.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:43 smithi104 ceph-mon[35325]: pgmap v518: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 615 KiB/s wr, 72 op/s 2024-09-10T21:09:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:43 smithi174 ceph-mon[35635]: pgmap v518: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 615 KiB/s wr, 72 op/s 2024-09-10T21:09:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:43 smithi033 ceph-mon[30860]: pgmap v518: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 615 KiB/s wr, 72 op/s 2024-09-10T21:09:44.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479829 7.88 MB/sec execute 165 sec latency 2530.206 ms 2024-09-10T21:09:45.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479829 7.84 MB/sec execute 166 sec latency 3530.325 ms 2024-09-10T21:09:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:45 smithi104 ceph-mon[35325]: pgmap v519: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 685 KiB/s wr, 81 op/s 2024-09-10T21:09:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:45 smithi174 ceph-mon[35635]: pgmap v519: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 685 KiB/s wr, 81 op/s 2024-09-10T21:09:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:45 smithi033 ceph-mon[30860]: pgmap v519: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 685 KiB/s wr, 81 op/s 2024-09-10T21:09:46.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 479829 7.79 MB/sec execute 167 sec latency 4530.465 ms 2024-09-10T21:09:47.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 480536 7.76 MB/sec execute 168 sec latency 4902.775 ms 2024-09-10T21:09:48.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:47 smithi104 ceph-mon[35325]: pgmap v520: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 601 KiB/s wr, 72 op/s 2024-09-10T21:09:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:47 smithi174 ceph-mon[35635]: pgmap v520: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 601 KiB/s wr, 72 op/s 2024-09-10T21:09:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:47 smithi033 ceph-mon[30860]: pgmap v520: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 395 KiB/s rd, 601 KiB/s wr, 72 op/s 2024-09-10T21:09:48.728 INFO:tasks.workunit.client.0.smithi033.stdout: 1 483160 7.81 MB/sec execute 169 sec latency 12.706 ms 2024-09-10T21:09:49.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 483288 7.76 MB/sec execute 170 sec latency 939.917 ms 2024-09-10T21:09:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:49 smithi104 ceph-mon[35325]: pgmap v521: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 602 KiB/s wr, 79 op/s 2024-09-10T21:09:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:49 smithi174 ceph-mon[35635]: pgmap v521: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 602 KiB/s wr, 79 op/s 2024-09-10T21:09:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:49 smithi033 ceph-mon[30860]: pgmap v521: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 317 KiB/s rd, 602 KiB/s wr, 79 op/s 2024-09-10T21:09:50.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 483288 7.72 MB/sec execute 171 sec latency 1940.017 ms 2024-09-10T21:09:51.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 483288 7.67 MB/sec execute 172 sec latency 2940.142 ms 2024-09-10T21:09:52.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:51 smithi104 ceph-mon[35325]: pgmap v522: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 310 KiB/s wr, 45 op/s 2024-09-10T21:09:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:51 smithi174 ceph-mon[35635]: pgmap v522: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 310 KiB/s wr, 45 op/s 2024-09-10T21:09:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:51 smithi033 ceph-mon[30860]: pgmap v522: 97 pgs: 97 active+clean; 186 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 310 KiB/s wr, 45 op/s 2024-09-10T21:09:52.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 483953 7.64 MB/sec execute 173 sec latency 3312.296 ms 2024-09-10T21:09:53.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486580 7.68 MB/sec execute 174 sec latency 11.967 ms 2024-09-10T21:09:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:53 smithi104 ceph-mon[35325]: pgmap v523: 97 pgs: 97 active+clean; 187 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 292 KiB/s wr, 46 op/s 2024-09-10T21:09:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:53 smithi174 ceph-mon[35635]: pgmap v523: 97 pgs: 97 active+clean; 187 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 292 KiB/s wr, 46 op/s 2024-09-10T21:09:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:53 smithi033 ceph-mon[30860]: pgmap v523: 97 pgs: 97 active+clean; 187 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 161 KiB/s rd, 292 KiB/s wr, 46 op/s 2024-09-10T21:09:54.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486768 7.65 MB/sec execute 175 sec latency 936.843 ms 2024-09-10T21:09:55.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486768 7.61 MB/sec execute 176 sec latency 1936.941 ms 2024-09-10T21:09:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:55 smithi104 ceph-mon[35325]: pgmap v524: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 350 KiB/s wr, 75 op/s 2024-09-10T21:09:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:55 smithi174 ceph-mon[35635]: pgmap v524: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 350 KiB/s wr, 75 op/s 2024-09-10T21:09:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:55 smithi033 ceph-mon[30860]: pgmap v524: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 405 KiB/s rd, 350 KiB/s wr, 75 op/s 2024-09-10T21:09:56.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486768 7.56 MB/sec execute 177 sec latency 2937.064 ms 2024-09-10T21:09:57.729 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486833 7.52 MB/sec execute 178 sec latency 3304.694 ms 2024-09-10T21:09:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:57 smithi104 ceph-mon[35325]: pgmap v525: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 274 KiB/s wr, 65 op/s 2024-09-10T21:09:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:57 smithi174 ceph-mon[35635]: pgmap v525: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 274 KiB/s wr, 65 op/s 2024-09-10T21:09:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:57 smithi033 ceph-mon[30860]: pgmap v525: 97 pgs: 97 active+clean; 188 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 274 KiB/s wr, 65 op/s 2024-09-10T21:09:58.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486833 7.48 MB/sec execute 179 sec latency 1614.120 ms 2024-09-10T21:09:59.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486833 7.44 MB/sec execute 180 sec latency 2614.261 ms 2024-09-10T21:10:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:09:59 smithi104 ceph-mon[35325]: pgmap v526: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 382 KiB/s wr, 96 op/s 2024-09-10T21:10:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:09:59 smithi174 ceph-mon[35635]: pgmap v526: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 382 KiB/s wr, 96 op/s 2024-09-10T21:10:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:09:59 smithi033 ceph-mon[30860]: pgmap v526: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 737 KiB/s rd, 382 KiB/s wr, 96 op/s 2024-09-10T21:10:00.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486833 7.40 MB/sec execute 181 sec latency 3614.367 ms 2024-09-10T21:10:01.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:00 smithi174 ceph-mon[35635]: overall HEALTH_OK 2024-09-10T21:10:01.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:00 smithi033 ceph-mon[30860]: overall HEALTH_OK 2024-09-10T21:10:01.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:00 smithi104 ceph-mon[35325]: overall HEALTH_OK 2024-09-10T21:10:01.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486833 7.36 MB/sec execute 182 sec latency 4614.457 ms 2024-09-10T21:10:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:01 smithi174 ceph-mon[35635]: pgmap v527: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 342 KiB/s wr, 86 op/s 2024-09-10T21:10:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:01 smithi033 ceph-mon[30860]: pgmap v527: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 342 KiB/s wr, 86 op/s 2024-09-10T21:10:02.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:01 smithi104 ceph-mon[35325]: pgmap v527: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 342 KiB/s wr, 86 op/s 2024-09-10T21:10:02.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 486919 7.32 MB/sec execute 183 sec latency 4983.714 ms 2024-09-10T21:10:03.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 487483 7.29 MB/sec execute 184 sec latency 1043.111 ms 2024-09-10T21:10:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:03 smithi174 ceph-mon[35635]: pgmap v528: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 307 KiB/s wr, 84 op/s 2024-09-10T21:10:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:03 smithi033 ceph-mon[30860]: pgmap v528: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 307 KiB/s wr, 84 op/s 2024-09-10T21:10:04.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:03 smithi104 ceph-mon[35325]: pgmap v528: 97 pgs: 97 active+clean; 189 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 643 KiB/s rd, 307 KiB/s wr, 84 op/s 2024-09-10T21:10:04.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490137 7.32 MB/sec execute 185 sec latency 11.960 ms 2024-09-10T21:10:05.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:10:05.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490314 7.30 MB/sec execute 186 sec latency 943.554 ms 2024-09-10T21:10:05.739 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:06.103 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:05 smithi033 ceph-mon[30860]: pgmap v529: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 673 KiB/s wr, 86 op/s 2024-09-10T21:10:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:05 smithi174 ceph-mon[35635]: pgmap v529: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 673 KiB/s wr, 86 op/s 2024-09-10T21:10:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.185+0000 7f870ecf0640 1 -- 172.21.15.33:0/133949269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708103ca0 msgr2=0x7f8708104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.185+0000 7f870ecf0640 1 --2- 172.21.15.33:0/133949269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708103ca0 0x7f8708104080 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009900 tx=0x7f86fc02a410 comp rx=0 tx=0).stop 2024-09-10T21:10:06.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 -- 172.21.15.33:0/133949269 shutdown_connections 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 --2- 172.21.15.33:0/133949269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870810d100 0x7f870810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 --2- 172.21.15.33:0/133949269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87081045c0 0x7f870810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 --2- 172.21.15.33:0/133949269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8708103ca0 0x7f8708104080 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 -- 172.21.15.33:0/133949269 >> 172.21.15.33:0/133949269 conn(0x7f87080fd700 msgr2=0x7f87080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 -- 172.21.15.33:0/133949269 shutdown_connections 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 -- 172.21.15.33:0/133949269 wait complete. 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 Processor -- start 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 -- start start 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.186+0000 7f870ecf0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8708103ca0 0x7f8708101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ecf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87081045c0 0x7f8708101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ecf0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870810d100 0x7f87081023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ecf0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8708112d70 con 0x7f87081045c0 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ecf0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8708112bf0 con 0x7f8708103ca0 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ecf0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8708112ef0 con 0x7f870810d100 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87081045c0 0x7f8708101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87081045c0 0x7f8708101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37928/0 (socket says 172.21.15.33:37928) 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 -- 172.21.15.33:0/3876055209 learned_addr learned my addr 172.21.15.33:0/3876055209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ca8c640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8708103ca0 0x7f8708101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870d28d640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870810d100 0x7f87081023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 -- 172.21.15.33:0/3876055209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870810d100 msgr2=0x7f87081023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870810d100 0x7f87081023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 -- 172.21.15.33:0/3876055209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8708103ca0 msgr2=0x7f8708101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8708103ca0 0x7f8708101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 -- 172.21.15.33:0/3876055209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87081a7680 con 0x7f87081045c0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ca8c640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8708103ca0 0x7f8708101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8707fff640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87081045c0 0x7f8708101e90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f86f400e9c0 tx=0x7f86f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8705ffb640 1 -- 172.21.15.33:0/3876055209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f400ccc0 con 0x7f87081045c0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8705ffb640 1 -- 172.21.15.33:0/3876055209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f40035f0 con 0x7f87081045c0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87081a7970 con 0x7f87081045c0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.187+0000 7f8705ffb640 1 -- 172.21.15.33:0/3876055209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f400ce60 con 0x7f87081045c0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.188+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87081a7eb0 con 0x7f87081045c0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.190+0000 7f8705ffb640 1 -- 172.21.15.33:0/3876055209 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f86f4004000 con 0x7f87081045c0 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.190+0000 7f87110c5740 1 Processor -- start 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.190+0000 7f87110c5740 1 -- start start 2024-09-10T21:10:06.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.190+0000 7f87110c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051c480 0x55691051c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.190+0000 7f87110c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x5569105287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f87110c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556910528d30 0x55691052b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f87110c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556910492e50 con 0x55691051cda0 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f87110c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556910482280 con 0x55691051c480 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f87110c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556910486090 con 0x556910528d30 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x5569105287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x5569105287f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37932/0 (socket says 172.21.15.33:37932) 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 -- 172.21.15.33:0/2049715359 learned_addr learned my addr 172.21.15.33:0/2049715359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870d28d640 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051c480 0x55691051c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f8707fff640 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556910528d30 0x55691052b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f8705ffb640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86f80b52e0 0x7f86f80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 -- 172.21.15.33:0/2049715359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556910528d30 msgr2=0x55691052b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556910528d30 0x55691052b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 -- 172.21.15.33:0/2049715359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051c480 msgr2=0x55691051c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051c480 0x55691051c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870ca8c640 1 -- 172.21.15.33:0/2049715359 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569101e0650 con 0x55691051cda0 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870d28d640 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051c480 0x55691051c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f8705ffb640 1 -- 172.21.15.33:0/3876055209 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86f40e5eb0 con 0x7f87081045c0 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.191+0000 7f870d28d640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86f80b52e0 0x7f86f80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.192+0000 7f870ca8c640 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x5569105287f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009900 tx=0x7f86fc005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cb98f99c239548 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.192+0000 7f86f2ffd640 1 -- 172.21.15.33:0/2049715359 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc040070 con 0x55691051cda0 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.192+0000 7f86f2ffd640 1 -- 172.21.15.33:0/2049715359 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86fc031860 con 0x55691051cda0 2024-09-10T21:10:06.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.192+0000 7f86f2ffd640 1 -- 172.21.15.33:0/2049715359 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc035040 con 0x55691051cda0 2024-09-10T21:10:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 -- 172.21.15.33:0/2049715359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 msgr2=0x5569105287f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f870d28d640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86f80b52e0 0x7f86f80b77a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8708103a10 tx=0x7f8700004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x5569105287f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f86fc009900 tx=0x7f86fc005d90 comp rx=0 tx=0).stop 2024-09-10T21:10:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 -- 172.21.15.33:0/2049715359 shutdown_connections 2024-09-10T21:10:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556910528d30 0x55691052b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x5569105287f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 --2- 172.21.15.33:0/2049715359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051c480 0x55691051c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 -- 172.21.15.33:0/2049715359 >> 172.21.15.33:0/2049715359 conn(0x55691051a2b0 msgr2=0x55691051b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 -- 172.21.15.33:0/2049715359 shutdown_connections 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 -- 172.21.15.33:0/2049715359 wait complete. 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 -- start start 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.193+0000 7f87110c5740 1 Processor -- start 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f87110c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x5569105285e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f87110c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x556910525a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f87110c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910525ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f87110c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5569102e3700 con 0x55691051cda0 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f87110c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55691041d110 con 0x556910528d30 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f87110c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55691052b950 con 0x55691051c480 2024-09-10T21:10:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f87110c5740 10 client.0 _ll_register_callbacks cb 0x556910436990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x556910525a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x556910525a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37940/0 (socket says 172.21.15.33:37940) 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 -- 172.21.15.33:0/3666663861 learned_addr learned my addr 172.21.15.33:0/3666663861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f870ca8c640 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x5569105285e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f870d28d640 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910525ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 -- 172.21.15.33:0/3666663861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 msgr2=0x5569105285e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x5569105285e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 -- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 msgr2=0x556910525ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910525ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.194+0000 7f8707fff640 1 -- 172.21.15.33:0/3666663861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691044ac40 con 0x55691051cda0 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f8707fff640 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x556910525a30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c6a0 tx=0x7f86f4003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c3444f8e668c027 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f86e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f86e7fff640 1 -- 172.21.15.33:0/3666663861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40acd30 con 0x55691051cda0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f86e7fff640 1 -- 172.21.15.33:0/3666663861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86f4004320 con 0x55691051cda0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86fc031860 con 0x55691051cda0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f86e7fff640 1 -- 172.21.15.33:0/3666663861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f86f40ac630 con 0x55691051cda0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f87110c5740 10 client.16443 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5569105372a0 con 0x55691051cda0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.195+0000 7f86e7fff640 1 -- 172.21.15.33:0/3666663861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86fc031860 con 0x55691051cda0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f86e7fff640 1 -- 172.21.15.33:0/3666663861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f86f40ab300 con 0x55691051cda0 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f87110c5740 10 client.16443 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f87110c5740 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f87110c5740 20 client.16443 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f87110c5740 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556910538800 0x55691053ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f87110c5740 4 client.16443 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556910494da0 con 0x556910538800 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f870ca8c640 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556910538800 0x55691053ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f870ca8c640 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556910538800 0x55691053ac50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e38819d96a953f40 server_cookie=423c51a303374ca8 in_seq=0 out_seq=0 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.196+0000 7f86e7fff640 10 client.16443 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f86e7fff640 1 -- 172.21.15.33:0/3666663861 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556910494da0 con 0x556910538800 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f86e7fff640 10 client.16443 handle_command_reply: tid=0 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 client.16443 shutdown 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 msgr2=0x556910525a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x556910525a30 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c6a0 tx=0x7f86f4003e20 comp rx=0 tx=0).stop 2024-09-10T21:10:06.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 shutdown_connections 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556910538800 0x55691053ac50 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910525ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x556910525a30 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 --2- 172.21.15.33:0/3666663861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x5569105285e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 >> 172.21.15.33:0/3666663861 conn(0x55691051a2b0 msgr2=0x55691051fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f86e7fff640 0 client.16443 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 shutdown_connections 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.197+0000 7f87110c5740 1 -- 172.21.15.33:0/3666663861 wait complete. 2024-09-10T21:10:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.198+0000 7f87110c5740 20 client.16443 trim_cache size 0 max 16384 2024-09-10T21:10:06.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 Processor -- start 2024-09-10T21:10:06.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 -- start start 2024-09-10T21:10:06.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910534be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x55691051b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910527770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b05e0 con 0x55691051cda0 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b22a0 con 0x556910528d30 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f87110c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b0d10 con 0x55691051c480 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f8707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x55691051b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f8707fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x55691051b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37946/0 (socket says 172.21.15.33:37946) 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.215+0000 7f8707fff640 1 -- 172.21.15.33:0/1584219515 learned_addr learned my addr 172.21.15.33:0/1584219515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f870ca8c640 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910534be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f870d28d640 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910527770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f8707fff640 1 -- 172.21.15.33:0/1584219515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 msgr2=0x556910534be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f8707fff640 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910534be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f8707fff640 1 -- 172.21.15.33:0/1584219515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 msgr2=0x556910527770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f8707fff640 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910527770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f8707fff640 1 -- 172.21.15.33:0/1584219515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691051ad30 con 0x55691051cda0 2024-09-10T21:10:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f870d28d640 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910527770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f8707fff640 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x55691051b300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f86f40058c0 tx=0x7f86f4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a60139cc9dbefe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f86f27fc640 1 -- 172.21.15.33:0/1584219515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40acd30 con 0x55691051cda0 2024-09-10T21:10:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f86f27fc640 1 -- 172.21.15.33:0/1584219515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f40b5bc0 con 0x55691051cda0 2024-09-10T21:10:06.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.216+0000 7f86f27fc640 1 -- 172.21.15.33:0/1584219515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f40abaf0 con 0x55691051cda0 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 -- 172.21.15.33:0/1584219515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 msgr2=0x55691051b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x55691051b300 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f86f40058c0 tx=0x7f86f4009f90 comp rx=0 tx=0).stop 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 -- 172.21.15.33:0/1584219515 shutdown_connections 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556910528d30 0x556910527770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55691051cda0 0x55691051b300 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 --2- 172.21.15.33:0/1584219515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910534be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 -- 172.21.15.33:0/1584219515 >> 172.21.15.33:0/1584219515 conn(0x55691051a2b0 msgr2=0x556910527060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 -- 172.21.15.33:0/1584219515 shutdown_connections 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 -- 172.21.15.33:0/1584219515 wait complete. 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 -- start start 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.217+0000 7f87110c5740 1 Processor -- start 2024-09-10T21:10:06.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f87110c5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910530c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f87110c5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051cda0 0x5569105311b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f87110c5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556910528d30 0x55691052bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f87110c5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b1b10 con 0x556910528d30 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f87110c5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87081b1410 con 0x55691051cda0 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f87110c5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87081ae5a0 con 0x55691051c480 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f87110c5740 10 client.0 _ll_register_callbacks cb 0x7f86f40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556910528d30 0x55691052bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556910528d30 0x55691052bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37954/0 (socket says 172.21.15.33:37954) 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 -- 172.21.15.33:0/555739518 learned_addr learned my addr 172.21.15.33:0/555739518 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f8707fff640 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051cda0 0x5569105311b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870ca8c640 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910530c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 -- 172.21.15.33:0/555739518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 msgr2=0x556910530c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910530c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 -- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051cda0 msgr2=0x5569105311b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051cda0 0x5569105311b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.218+0000 7f870d28d640 1 -- 172.21.15.33:0/555739518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691052c1c0 con 0x556910528d30 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f870d28d640 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556910528d30 0x55691052bc00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f870000f960 tx=0x7f8700004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8458aa83a10c65b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f86e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f86e7fff640 1 -- 172.21.15.33:0/555739518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8700010700 con 0x556910528d30 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f86e7fff640 1 -- 172.21.15.33:0/555739518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87000108a0 con 0x556910528d30 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f40b5bc0 con 0x556910528d30 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f86e7fff640 1 -- 172.21.15.33:0/555739518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f8700010aa0 con 0x556910528d30 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f87110c5740 10 client.16449 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x5569105a5fa0 con 0x556910528d30 2024-09-10T21:10:06.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f86e7fff640 1 -- 172.21.15.33:0/555739518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86f40b5bc0 con 0x556910528d30 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.219+0000 7f86e7fff640 1 -- 172.21.15.33:0/555739518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f870001a070 con 0x556910528d30 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f87110c5740 10 client.16449 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f87110c5740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f87110c5740 20 client.16449 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f87110c5740 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5569105a6860 0x5569105a8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f87110c5740 4 client.16449 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556910490850 con 0x5569105a6860 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f870ca8c640 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5569105a6860 0x5569105a8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f870ca8c640 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5569105a6860 0x5569105a8c80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b64a3c14ae57350 server_cookie=9afb59c26b98e43d in_seq=0 out_seq=0 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.220+0000 7f86e7fff640 10 client.16449 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f86e7fff640 1 -- 172.21.15.33:0/555739518 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x556910490850 con 0x5569105a6860 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f86e7fff640 10 client.16449 handle_command_reply: tid=0 2024-09-10T21:10:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 client.16449 shutdown 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556910528d30 msgr2=0x55691052bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556910528d30 0x55691052bc00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f870000f960 tx=0x7f8700004ab0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 shutdown_connections 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5569105a6860 0x5569105a8c80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f86e7fff640 0 client.16449 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556910528d30 0x55691052bc00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55691051cda0 0x5569105311b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 --2- 172.21.15.33:0/555739518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691051c480 0x556910530c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 >> 172.21.15.33:0/555739518 conn(0x55691051a2b0 msgr2=0x556910527880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 shutdown_connections 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 1 -- 172.21.15.33:0/555739518 wait complete. 2024-09-10T21:10:06.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.221+0000 7f87110c5740 20 client.16449 trim_cache size 0 max 16384 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.231 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:06.232 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:06.234 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:10:06.236 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:10:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86f80b52e0 msgr2=0x7f86f80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86f80b52e0 0x7f86f80b77a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f8708103a10 tx=0x7f8700004a40 comp rx=0 tx=0).stop 2024-09-10T21:10:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87081045c0 msgr2=0x7f8708101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87081045c0 0x7f8708101e90 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f86f400e9c0 tx=0x7f86f400ee80 comp rx=0 tx=0).stop 2024-09-10T21:10:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 shutdown_connections 2024-09-10T21:10:06.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86f80b52e0 0x7f86f80b77a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f870810d100 0x7f87081023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87081045c0 0x7f8708101e90 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 --2- 172.21.15.33:0/3876055209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8708103ca0 0x7f8708101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 >> 172.21.15.33:0/3876055209 conn(0x7f87080fd700 msgr2=0x7f87080fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 shutdown_connections 2024-09-10T21:10:06.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:06.222+0000 7f870ecf0640 1 -- 172.21.15.33:0/3876055209 wait complete. 2024-09-10T21:10:06.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:05 smithi104 ceph-mon[35325]: pgmap v529: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 673 KiB/s wr, 86 op/s 2024-09-10T21:10:06.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490314 7.26 MB/sec execute 187 sec latency 1943.603 ms 2024-09-10T21:10:07.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:10:07.064 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:10:07.329 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:07.730 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490395 7.22 MB/sec execute 188 sec latency 2316.123 ms 2024-09-10T21:10:07.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 -- 172.21.15.33:0/359562426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484104600 msgr2=0x7f448410caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 --2- 172.21.15.33:0/359562426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484104600 0x7f448410caf0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f4478009900 tx=0x7f447802a410 comp rx=0 tx=0).stop 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 -- 172.21.15.33:0/359562426 shutdown_connections 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 --2- 172.21.15.33:0/359562426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f448410d140 0x7f448410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 --2- 172.21.15.33:0/359562426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484104600 0x7f448410caf0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 --2- 172.21.15.33:0/359562426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4484103ce0 0x7f44841040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 -- 172.21.15.33:0/359562426 >> 172.21.15.33:0/359562426 conn(0x7f44840fd700 msgr2=0x7f44840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 -- 172.21.15.33:0/359562426 shutdown_connections 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.852+0000 7f448a2f5640 1 -- 172.21.15.33:0/359562426 wait complete. 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 Processor -- start 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 -- start start 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484103ce0 0x7f4484199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4484104600 0x7f44841997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448410d140 0x7f448419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4484112db0 con 0x7f4484103ce0 2024-09-10T21:10:07.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4484112c30 con 0x7f4484104600 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f448a2f5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4484112f30 con 0x7f448410d140 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f4483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484103ce0 0x7f4484199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f4483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484103ce0 0x7f4484199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37966/0 (socket says 172.21.15.33:37966) 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.853+0000 7f4483fff640 1 -- 172.21.15.33:0/4180098175 learned_addr learned my addr 172.21.15.33:0/4180098175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4488892640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448410d140 0x7f448419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f44837fe640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4484104600 0x7f44841997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4483fff640 1 -- 172.21.15.33:0/4180098175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448410d140 msgr2=0x7f448419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4483fff640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448410d140 0x7f448419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4483fff640 1 -- 172.21.15.33:0/4180098175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4484104600 msgr2=0x7f44841997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:07.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4483fff640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4484104600 0x7f44841997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4483fff640 1 -- 172.21.15.33:0/4180098175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f448419f330 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4488892640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448410d140 0x7f448419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f4483fff640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484103ce0 0x7f4484199280 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f447400d770 tx=0x7f447400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f44817fa640 1 -- 172.21.15.33:0/4180098175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474017070 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f44817fa640 1 -- 172.21.15.33:0/4180098175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44740054d0 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f44817fa640 1 -- 172.21.15.33:0/4180098175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4474004c30 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f448419f620 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.854+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44841a5ce0 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.855+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44841040c0 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.857+0000 7f44817fa640 1 -- 172.21.15.33:0/4180098175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4474004dd0 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.857+0000 7f44817fa640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f446c0b5360 0x7f446c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.857+0000 7f44817fa640 1 -- 172.21.15.33:0/4180098175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f44740e5ae0 con 0x7f4484103ce0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.857+0000 7f44837fe640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f446c0b5360 0x7f446c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:07.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.858+0000 7f44837fe640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f446c0b5360 0x7f446c0b7820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f4478000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:07.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:07.861+0000 7f44817fa640 1 -- 172.21.15.33:0/4180098175 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f44740aca20 con 0x7f4484103ce0 2024-09-10T21:10:08.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.083+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44841a60b0 con 0x7f4484103ce0 2024-09-10T21:10:08.088 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:07 smithi033 ceph-mon[30860]: pgmap v530: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-09-10T21:10:08.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.084+0000 7f44817fa640 1 -- 172.21.15.33:0/4180098175 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11988 (secure 0 0 0) 0x7f44740ac320 con 0x7f4484103ce0 2024-09-10T21:10:08.089 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:10:08.089 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":49,"btime":"2024-09-10T21:09:06:710981+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:09:05.985999+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:10:08.089 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 49 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f446c0b5360 msgr2=0x7f446c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f446c0b5360 0x7f446c0b7820 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4478006010 tx=0x7f4478000bc0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484103ce0 msgr2=0x7f4484199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484103ce0 0x7f4484199280 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f447400d770 tx=0x7f447400dc30 comp rx=0 tx=0).stop 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 shutdown_connections 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f446c0b5360 0x7f446c0b7820 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f448410d140 0x7f448419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4484104600 0x7f44841997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 --2- 172.21.15.33:0/4180098175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484103ce0 0x7f4484199280 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:08.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 >> 172.21.15.33:0/4180098175 conn(0x7f44840fd700 msgr2=0x7f44840fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 shutdown_connections 2024-09-10T21:10:08.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:08.087+0000 7f448a2f5640 1 -- 172.21.15.33:0/4180098175 wait complete. 2024-09-10T21:10:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:07 smithi174 ceph-mon[35635]: pgmap v530: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-09-10T21:10:08.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:07 smithi104 ceph-mon[35325]: pgmap v530: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 735 KiB/s wr, 63 op/s 2024-09-10T21:10:08.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490395 7.19 MB/sec execute 189 sec latency 1590.477 ms 2024-09-10T21:10:08.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:10:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:09.132 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:08 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4180098175' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:10:09.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:08 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4180098175' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:10:09.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:08 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4180098175' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:10:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 -- 172.21.15.33:0/2162709568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 msgr2=0x7f1da8108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 --2- 172.21.15.33:0/2162709568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da8108c10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c009900 tx=0x7f1d9c02a410 comp rx=0 tx=0).stop 2024-09-10T21:10:09.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 -- 172.21.15.33:0/2162709568 shutdown_connections 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 --2- 172.21.15.33:0/2162709568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da8109150 0x7f1da810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 --2- 172.21.15.33:0/2162709568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da8108c10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 --2- 172.21.15.33:0/2162709568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da8102dc0 0x7f1da81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 -- 172.21.15.33:0/2162709568 >> 172.21.15.33:0/2162709568 conn(0x7f1da80fe830 msgr2=0x7f1da8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 -- 172.21.15.33:0/2162709568 shutdown_connections 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.525+0000 7f1db06cc640 1 -- 172.21.15.33:0/2162709568 wait complete. 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 Processor -- start 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 -- start start 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da8102dc0 0x7f1da80754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da80759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da8109150 0x7f1da8075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8112e10 con 0x7f1da81087d0 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8112c90 con 0x7f1da8109150 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1db06cc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8112f90 con 0x7f1da8102dc0 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da80759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da80759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34060/0 (socket says 172.21.15.33:34060) 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 -- 172.21.15.33:0/1723766783 learned_addr learned my addr 172.21.15.33:0/1723766783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1daec69640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da8109150 0x7f1da8075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dae468640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da8102dc0 0x7f1da80754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 -- 172.21.15.33:0/1723766783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da8102dc0 msgr2=0x7f1da80754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da8102dc0 0x7f1da80754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 -- 172.21.15.33:0/1723766783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da8109150 msgr2=0x7f1da8075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da8109150 0x7f1da8075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.526+0000 7f1dadc67640 1 -- 172.21.15.33:0/1723766783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da81a7700 con 0x7f1da81087d0 2024-09-10T21:10:09.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.527+0000 7f1daec69640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da8109150 0x7f1da8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.527+0000 7f1dadc67640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da80759e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.527+0000 7f1d9b7fe640 1 -- 172.21.15.33:0/1723766783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c03f030 con 0x7f1da81087d0 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.527+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da81a7990 con 0x7f1da81087d0 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.527+0000 7f1d9b7fe640 1 -- 172.21.15.33:0/1723766783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9c003800 con 0x7f1da81087d0 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.527+0000 7f1d9b7fe640 1 -- 172.21.15.33:0/1723766783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c035070 con 0x7f1da81087d0 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.527+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da81a7ef0 con 0x7f1da81087d0 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.529+0000 7f1d9b7fe640 1 -- 172.21.15.33:0/1723766783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1d9c031b10 con 0x7f1da81087d0 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 Processor -- start 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 -- start start 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89e4d0 0x55bbae89e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89edf0 0x55bbae8aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbae814ea0 con 0x55bbae8aad80 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bbae8042d0 con 0x55bbae89edf0 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bbae8080e0 con 0x55bbae89e4d0 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1dadc67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1d9b7fe640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d940b52e0 0x7f1d940b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1dadc67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34072/0 (socket says 172.21.15.33:34072) 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1dadc67640 1 -- 172.21.15.33:0/2412123620 learned_addr learned my addr 172.21.15.33:0/2412123620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:09.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.530+0000 7f1daec69640 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89e4d0 0x55bbae89e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dae468640 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89edf0 0x55bbae8aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1daec69640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d940b52e0 0x7f1d940b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dadc67640 1 -- 172.21.15.33:0/2412123620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89e4d0 msgr2=0x55bbae89e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dadc67640 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89e4d0 0x55bbae89e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dadc67640 1 -- 172.21.15.33:0/2412123620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89edf0 msgr2=0x55bbae8aa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dadc67640 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89edf0 0x55bbae8aa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1d9b7fe640 1 -- 172.21.15.33:0/1723766783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1d9c106650 con 0x7f1da81087d0 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dadc67640 1 -- 172.21.15.33:0/2412123620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbae562650 con 0x55bbae8aad80 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1daec69640 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89e4d0 0x55bbae89e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dae468640 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89edf0 0x55bbae8aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1dadc67640 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ad170 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c031440 tx=0x7f1d9c078a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ad79d7b5828c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:09.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1d98ff9640 1 -- 172.21.15.33:0/2412123620 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0cc540 con 0x55bbae8aad80 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1daec69640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d940b52e0 0x7f1d940b77a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1da000bfd0 tx=0x7f1da00046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1d98ff9640 1 -- 172.21.15.33:0/2412123620 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d9c0d6360 con 0x55bbae8aad80 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.531+0000 7f1d98ff9640 1 -- 172.21.15.33:0/2412123620 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c07a360 con 0x55bbae8aad80 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 -- 172.21.15.33:0/2412123620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 msgr2=0x55bbae8ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ad170 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c031440 tx=0x7f1d9c078a50 comp rx=0 tx=0).stop 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 -- 172.21.15.33:0/2412123620 shutdown_connections 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ad170 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89edf0 0x55bbae8aa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/2412123620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89e4d0 0x55bbae89e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 -- 172.21.15.33:0/2412123620 >> 172.21.15.33:0/2412123620 conn(0x55bbae89c300 msgr2=0x55bbae89d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 -- 172.21.15.33:0/2412123620 shutdown_connections 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 -- 172.21.15.33:0/2412123620 wait complete. 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.532+0000 7f1db2aa1740 1 -- start start 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 1 Processor -- start 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae89edf0 0x55bbae8a7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae8aad80 0x55bbae8a8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bbae665700 con 0x55bbae89edf0 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bbae79f160 con 0x55bbae89e4d0 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bbae8ad7c0 con 0x55bbae8aad80 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1db2aa1740 10 client.0 _ll_register_callbacks cb 0x55bbae7b89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae89edf0 0x55bbae8a7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae89edf0 0x55bbae8a7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34082/0 (socket says 172.21.15.33:34082) 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 -- 172.21.15.33:0/3812025378 learned_addr learned my addr 172.21.15.33:0/3812025378 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dae468640 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1daec69640 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae8aad80 0x55bbae8a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 -- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae8aad80 msgr2=0x55bbae8a8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae8aad80 0x55bbae8a8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 -- 172.21.15.33:0/3812025378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 msgr2=0x55bbae8aa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8aa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 -- 172.21.15.33:0/3812025378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbae7ccc90 con 0x55bbae89edf0 2024-09-10T21:10:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1dadc67640 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae89edf0 0x55bbae8a7af0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c031ae0 tx=0x7f1d9c07a360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d9d34fd41c7f55 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.533+0000 7f1d91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1d91ffb640 1 -- 172.21.15.33:0/3812025378 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c049040 con 0x55bbae89edf0 2024-09-10T21:10:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1d91ffb640 1 -- 172.21.15.33:0/3812025378 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1d9c0b40e0 con 0x55bbae89edf0 2024-09-10T21:10:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bbae816df0 con 0x55bbae89edf0 2024-09-10T21:10:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1d91ffb640 1 -- 172.21.15.33:0/3812025378 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1d9c0cc8e0 con 0x55bbae89edf0 2024-09-10T21:10:09.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 10 client.16467 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bbae8b9010 con 0x55bbae89edf0 2024-09-10T21:10:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1d91ffb640 1 -- 172.21.15.33:0/3812025378 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bbae816df0 con 0x55bbae89edf0 2024-09-10T21:10:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1d91ffb640 1 -- 172.21.15.33:0/3812025378 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f1d9c0cd030 con 0x55bbae89edf0 2024-09-10T21:10:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 10 client.16467 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 10 client.16467 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:10:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 20 client.16467 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:09.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae8ba360 0x55bbae8bc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 4 client.16467 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.534+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bbae8128a0 con 0x55bbae8ba360 2024-09-10T21:10:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.535+0000 7f1dae468640 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae8ba360 0x55bbae8bc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.535+0000 7f1dae468640 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae8ba360 0x55bbae8bc750 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f8daeb177598ccb1 server_cookie=9bc6cd63c293f63d in_seq=0 out_seq=0 2024-09-10T21:10:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.535+0000 7f1d91ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:09.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1d91ffb640 1 -- 172.21.15.33:0/3812025378 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bbae8128a0 con 0x55bbae8ba360 2024-09-10T21:10:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1d91ffb640 10 client.16467 handle_command_reply: tid=0 2024-09-10T21:10:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 client.16467 shutdown 2024-09-10T21:10:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae89edf0 msgr2=0x55bbae8a7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae89edf0 0x55bbae8a7af0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c031ae0 tx=0x7f1d9c07a360 comp rx=0 tx=0).stop 2024-09-10T21:10:09.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 shutdown_connections 2024-09-10T21:10:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae8ba360 0x55bbae8bc750 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae8aad80 0x55bbae8a8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae89edf0 0x55bbae8a7af0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1d91ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3812025378 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8aa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 >> 172.21.15.33:0/3812025378 conn(0x55bbae89c300 msgr2=0x55bbae8a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 shutdown_connections 2024-09-10T21:10:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3812025378 wait complete. 2024-09-10T21:10:09.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.536+0000 7f1db2aa1740 20 client.16467 trim_cache size 0 max 16384 2024-09-10T21:10:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.549+0000 7f1db2aa1740 1 Processor -- start 2024-09-10T21:10:09.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.549+0000 7f1db2aa1740 1 -- start start 2024-09-10T21:10:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae89bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8a9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b06a0 con 0x55bbae8aad80 2024-09-10T21:10:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b2360 con 0x55bbae89e4d0 2024-09-10T21:10:09.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b0dd0 con 0x55bbae89edf0 2024-09-10T21:10:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34088/0 (socket says 172.21.15.33:34088) 2024-09-10T21:10:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 -- 172.21.15.33:0/3490205318 learned_addr learned my addr 172.21.15.33:0/3490205318 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1dadc67640 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1dae468640 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae89bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 -- 172.21.15.33:0/3490205318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 msgr2=0x55bbae8a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 -- 172.21.15.33:0/3490205318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 msgr2=0x55bbae89bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae89bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 -- 172.21.15.33:0/3490205318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbae8aa2d0 con 0x55bbae8aad80 2024-09-10T21:10:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1dadc67640 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1dae468640 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae89bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:09.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.550+0000 7f1daec69640 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ac170 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1da0007fd0 tx=0x7f1da0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a5b1af79d53fae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1d93fff640 1 -- 172.21.15.33:0/3490205318 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da000f040 con 0x55bbae8aad80 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1d93fff640 1 -- 172.21.15.33:0/3490205318 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da0027070 con 0x55bbae8aad80 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1d93fff640 1 -- 172.21.15.33:0/3490205318 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0021620 con 0x55bbae8aad80 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3490205318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 msgr2=0x55bbae8ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ac170 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1da0007fd0 tx=0x7f1da0009000 comp rx=0 tx=0).stop 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3490205318 shutdown_connections 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8ac170 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/3490205318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae89bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3490205318 >> 172.21.15.33:0/3490205318 conn(0x55bbae89c300 msgr2=0x55bbae8a9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3490205318 shutdown_connections 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 -- 172.21.15.33:0/3490205318 wait complete. 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.551+0000 7f1db2aa1740 1 -- start start 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 1 Processor -- start 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8ad920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b1bd0 con 0x55bbae8aad80 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81b14d0 con 0x55bbae89e4d0 2024-09-10T21:10:09.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81ae660 con 0x55bbae89edf0 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1db2aa1740 10 client.0 _ll_register_callbacks cb 0x7f1da000b210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34102/0 (socket says 172.21.15.33:34102) 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 -- 172.21.15.33:0/1598851505 learned_addr learned my addr 172.21.15.33:0/1598851505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1dadc67640 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1dae468640 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8ad920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 -- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 msgr2=0x55bbae8adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 -- 172.21.15.33:0/1598851505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 msgr2=0x55bbae8ad920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8ad920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.552+0000 7f1daec69640 1 -- 172.21.15.33:0/1598851505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbae8b2ea0 con 0x55bbae8aad80 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1daec69640 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8b2790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006c70 tx=0x7f1da0022690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2cac8a5f35212a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:09.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1d91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1d91ffb640 1 -- 172.21.15.33:0/1598851505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da0030070 con 0x55bbae8aad80 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1d91ffb640 1 -- 172.21.15.33:0/1598851505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1da0035460 con 0x55bbae8aad80 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1da0027070 con 0x55bbae8aad80 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1d91ffb640 1 -- 172.21.15.33:0/1598851505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f1da0035660 con 0x55bbae8aad80 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 10 client.16473 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bbae9278f0 con 0x55bbae8aad80 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1d91ffb640 1 -- 172.21.15.33:0/1598851505 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1da0027070 con 0x55bbae8aad80 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1d91ffb640 1 -- 172.21.15.33:0/1598851505 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f1da002c700 con 0x55bbae8aad80 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 10 client.16473 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 10 client.16473 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 20 client.16473 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:09.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae928070 0x55bbae92a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 4 client.16473 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.553+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bbae80fcd0 con 0x55bbae928070 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1dae468640 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae928070 0x55bbae92a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1dae468640 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae928070 0x55bbae92a490 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b54d10eed7f3f93f server_cookie=f836c95748e1ca89 in_seq=0 out_seq=0 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1d91ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1d91ffb640 1 -- 172.21.15.33:0/1598851505 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bbae80fcd0 con 0x55bbae928070 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1d91ffb640 10 client.16473 handle_command_reply: tid=0 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 client.16473 shutdown 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 msgr2=0x55bbae8b2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8b2790 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1da0006c70 tx=0x7f1da0022690 comp rx=0 tx=0).stop 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 shutdown_connections 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55bbae928070 0x55bbae92a490 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1d91ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bbae8aad80 0x55bbae8b2790 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bbae89edf0 0x55bbae8adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 --2- 172.21.15.33:0/1598851505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bbae89e4d0 0x55bbae8ad920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.554+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 >> 172.21.15.33:0/1598851505 conn(0x55bbae89c300 msgr2=0x55bbae8a9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 shutdown_connections 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db2aa1740 1 -- 172.21.15.33:0/1598851505 wait complete. 2024-09-10T21:10:09.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db2aa1740 20 client.16473 trim_cache size 0 max 16384 2024-09-10T21:10:09.584 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d940b52e0 msgr2=0x7f1d940b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d940b52e0 0x7f1d940b77a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1da000bfd0 tx=0x7f1da00046c0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 msgr2=0x7f1da80759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da80759e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c006010 tx=0x7f1d9c0042f0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 shutdown_connections 2024-09-10T21:10:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d940b52e0 0x7f1d940b77a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1da8109150 0x7f1da8075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1da81087d0 0x7f1da80759e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 --2- 172.21.15.33:0/1723766783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1da8102dc0 0x7f1da80754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 >> 172.21.15.33:0/1723766783 conn(0x7f1da80fe830 msgr2=0x7f1da8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 shutdown_connections 2024-09-10T21:10:09.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:09.555+0000 7f1db06cc640 1 -- 172.21.15.33:0/1723766783 wait complete. 2024-09-10T21:10:09.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490395 7.15 MB/sec execute 190 sec latency 2590.882 ms 2024-09-10T21:10:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:09 smithi174 ceph-mon[35635]: pgmap v531: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-10T21:10:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:09 smithi033 ceph-mon[30860]: pgmap v531: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-10T21:10:10.271 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:10:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:09 smithi104 ceph-mon[35325]: pgmap v531: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 1.1 MiB/s wr, 110 op/s 2024-09-10T21:10:10.533 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:10.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490395 7.11 MB/sec execute 191 sec latency 3590.968 ms 2024-09-10T21:10:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 -- 172.21.15.33:0/1593137851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610104570 msgr2=0x7f961010cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 --2- 172.21.15.33:0/1593137851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610104570 0x7f961010cb50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f9600009900 tx=0x7f960002a410 comp rx=0 tx=0).stop 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 -- 172.21.15.33:0/1593137851 shutdown_connections 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 --2- 172.21.15.33:0/1593137851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961010d120 0x7f961010f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 --2- 172.21.15.33:0/1593137851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610104570 0x7f961010cb50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 --2- 172.21.15.33:0/1593137851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9610103b70 0x7f9610103f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 -- 172.21.15.33:0/1593137851 >> 172.21.15.33:0/1593137851 conn(0x7f96100fd700 msgr2=0x7f96100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 -- 172.21.15.33:0/1593137851 shutdown_connections 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.049+0000 7f9617f0b640 1 -- 172.21.15.33:0/1593137851 wait complete. 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 Processor -- start 2024-09-10T21:10:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 -- start start 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610103b70 0x7f9610199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9610104570 0x7f96101997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961010d120 0x7f961019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9610112da0 con 0x7f9610103b70 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9610112c20 con 0x7f9610104570 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9617f0b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9610112f20 con 0x7f961010d120 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9615ca7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610103b70 0x7f9610199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9615ca7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610103b70 0x7f9610199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34110/0 (socket says 172.21.15.33:34110) 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f9615ca7640 1 -- 172.21.15.33:0/3639441900 learned_addr learned my addr 172.21.15.33:0/3639441900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.050+0000 7f96154a6640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9610104570 0x7f96101997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f96164a8640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961010d120 0x7f961019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9615ca7640 1 -- 172.21.15.33:0/3639441900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961010d120 msgr2=0x7f961019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9615ca7640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961010d120 0x7f961019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9615ca7640 1 -- 172.21.15.33:0/3639441900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9610104570 msgr2=0x7f96101997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9615ca7640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9610104570 0x7f96101997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9615ca7640 1 -- 172.21.15.33:0/3639441900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f961019f340 con 0x7f9610103b70 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f96154a6640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9610104570 0x7f96101997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f96164a8640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961010d120 0x7f961019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9615ca7640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610103b70 0x7f9610199290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f960c00e9f0 tx=0x7f960c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9606ffd640 1 -- 172.21.15.33:0/3639441900 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c00cd80 con 0x7f9610103b70 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9606ffd640 1 -- 172.21.15.33:0/3639441900 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960c0108e0 con 0x7f9610103b70 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f961019f630 con 0x7f9610103b70 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9606ffd640 1 -- 172.21.15.33:0/3639441900 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c018980 con 0x7f9610103b70 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.051+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96101a5ec0 con 0x7f9610103b70 2024-09-10T21:10:11.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.053+0000 7f961a2e0740 1 Processor -- start 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f961a2e0740 1 -- start start 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f9606ffd640 1 -- 172.21.15.33:0/3639441900 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f960c010430 con 0x7f9610103b70 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aabc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aac8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 0x55f20aacb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f961a2e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f20aa2eea0 con 0x55f20aabc500 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f961a2e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f20aa1e2d0 con 0x55f20aac8db0 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f961a2e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f20aa220e0 con 0x55f20aabce20 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f96164a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aabc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f96164a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aabc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34112/0 (socket says 172.21.15.33:34112) 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f96164a8640 1 -- 172.21.15.33:0/2157476488 learned_addr learned my addr 172.21.15.33:0/2157476488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:11.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f9615ca7640 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aac8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.054+0000 7f96154a6640 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 0x55f20aacb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f96164a8640 1 -- 172.21.15.33:0/2157476488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 msgr2=0x55f20aac8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f96164a8640 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aac8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f96164a8640 1 -- 172.21.15.33:0/2157476488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 msgr2=0x55f20aacb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f96164a8640 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 0x55f20aacb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f96164a8640 1 -- 172.21.15.33:0/2157476488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20a77c650 con 0x55f20aabc500 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f96164a8640 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aabc8e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9608009f10 tx=0x7f960800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dde76ac19090411 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f9606ffd640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95fc0b5360 0x7f95fc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f95fbfff640 1 -- 172.21.15.33:0/2157476488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608016070 con 0x55f20aabc500 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f95fbfff640 1 -- 172.21.15.33:0/2157476488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9608004d20 con 0x55f20aabc500 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f9615ca7640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95fc0b5360 0x7f95fc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f95fbfff640 1 -- 172.21.15.33:0/2157476488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608011740 con 0x55f20aabc500 2024-09-10T21:10:11.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.055+0000 7f9606ffd640 1 -- 172.21.15.33:0/3639441900 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f960c0e6000 con 0x7f9610103b70 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 -- 172.21.15.33:0/2157476488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 msgr2=0x55f20aabc8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f9615ca7640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95fc0b5360 0x7f95fc0b7820 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f960c0ac750 tx=0x7f960c0e9a80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aabc8e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f9608009f10 tx=0x7f960800e6c0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 -- 172.21.15.33:0/2157476488 shutdown_connections 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 0x55f20aacb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aac8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 --2- 172.21.15.33:0/2157476488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aabc8e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 -- 172.21.15.33:0/2157476488 >> 172.21.15.33:0/2157476488 conn(0x55f20aaba330 msgr2=0x55f20aabb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 -- 172.21.15.33:0/2157476488 shutdown_connections 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 -- 172.21.15.33:0/2157476488 wait complete. 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 -- start start 2024-09-10T21:10:11.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.056+0000 7f961a2e0740 1 Processor -- start 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aac8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabce20 0x55f20aac5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aac6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f961a2e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f20a87f700 con 0x55f20aabc500 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f961a2e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f20a9b9160 con 0x55f20aabce20 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f961a2e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f20aacba30 con 0x55f20aac8db0 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f961a2e0740 10 client.0 _ll_register_callbacks cb 0x55f20a9d29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aac8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aac8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34126/0 (socket says 172.21.15.33:34126) 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 -- 172.21.15.33:0/3241345056 learned_addr learned my addr 172.21.15.33:0/3241345056 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96154a6640 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabce20 0x55f20aac5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aac6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 -- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 msgr2=0x55f20aac6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aac6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 -- 172.21.15.33:0/3241345056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabce20 msgr2=0x55f20aac5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabce20 0x55f20aac5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.057+0000 7f96164a8640 1 -- 172.21.15.33:0/3241345056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20a9e6c90 con 0x55f20aabc500 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f96164a8640 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aac8630 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9608005b40 tx=0x7f96080094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f527efe24e670f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f95f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f95f97fa640 1 -- 172.21.15.33:0/3241345056 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608016040 con 0x55f20aabc500 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f95f97fa640 1 -- 172.21.15.33:0/3241345056 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f96080114e0 con 0x55f20aabc500 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9608004d20 con 0x55f20aabc500 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f95f97fa640 1 -- 172.21.15.33:0/3241345056 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9608005790 con 0x55f20aabc500 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 10 client.16485 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:11.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f20aad7330 con 0x55f20aabc500 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f95f97fa640 1 -- 172.21.15.33:0/3241345056 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9608004d20 con 0x55f20aabc500 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f95f97fa640 1 -- 172.21.15.33:0/3241345056 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f9608027020 con 0x55f20aabc500 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 10 client.16485 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 10 client.16485 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 20 client.16485 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20aad8730 0x55f20aadab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 4 client.16485 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.058+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f20aa30df0 con 0x55f20aad8730 2024-09-10T21:10:11.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.059+0000 7f96154a6640 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20aad8730 0x55f20aadab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.059+0000 7f95f97fa640 10 client.16485 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.059+0000 7f96154a6640 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20aad8730 0x55f20aadab20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c87a8a32a827283a server_cookie=5c42e34365a2a13f in_seq=0 out_seq=0 2024-09-10T21:10:11.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f95f97fa640 1 -- 172.21.15.33:0/3241345056 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f20aa30df0 con 0x55f20aad8730 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f95f97fa640 10 client.16485 handle_command_reply: tid=0 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 client.16485 shutdown 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 msgr2=0x55f20aac8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aac8630 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9608005b40 tx=0x7f96080094d0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 shutdown_connections 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20aad8730 0x55f20aadab20 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aac6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabce20 0x55f20aac5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f95f97fa640 0 client.16485 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 --2- 172.21.15.33:0/3241345056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aac8630 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 >> 172.21.15.33:0/3241345056 conn(0x55f20aaba330 msgr2=0x55f20aabfea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:11.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 shutdown_connections 2024-09-10T21:10:11.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 1 -- 172.21.15.33:0/3241345056 wait complete. 2024-09-10T21:10:11.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.060+0000 7f961a2e0740 20 client.16485 trim_cache size 0 max 16384 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 Processor -- start 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 -- start start 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aab4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aaca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 0x55f20aaca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96101ac2e0 con 0x55f20aabc500 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f96101adfa0 con 0x55f20aac8db0 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f961a2e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96101aca10 con 0x55f20aabce20 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aab4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aab4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34142/0 (socket says 172.21.15.33:34142) 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 -- 172.21.15.33:0/4280373807 learned_addr learned my addr 172.21.15.33:0/4280373807 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:11.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96154a6640 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aaca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 -- 172.21.15.33:0/4280373807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 msgr2=0x55f20aaca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aaca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 -- 172.21.15.33:0/4280373807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 msgr2=0x55f20aaca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 0x55f20aaca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 -- 172.21.15.33:0/4280373807 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f20aabcc50 con 0x55f20aabc500 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.074+0000 7f96164a8640 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aab4e60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9608029980 tx=0x7f9608005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947cb9ea3f332124 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f95fb7fe640 1 -- 172.21.15.33:0/4280373807 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9608016070 con 0x55f20aabc500 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f95fb7fe640 1 -- 172.21.15.33:0/4280373807 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960801e070 con 0x55f20aabc500 2024-09-10T21:10:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f95fb7fe640 1 -- 172.21.15.33:0/4280373807 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96080095b0 con 0x55f20aabc500 2024-09-10T21:10:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 -- 172.21.15.33:0/4280373807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 msgr2=0x55f20aab4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aab4e60 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9608029980 tx=0x7f9608005e30 comp rx=0 tx=0).stop 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 -- 172.21.15.33:0/4280373807 shutdown_connections 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aac8db0 0x55f20aaca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aabce20 0x55f20aaca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 --2- 172.21.15.33:0/4280373807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabc500 0x55f20aab4e60 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 -- 172.21.15.33:0/4280373807 >> 172.21.15.33:0/4280373807 conn(0x55f20aaba330 msgr2=0x55f20aab4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 -- 172.21.15.33:0/4280373807 shutdown_connections 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.075+0000 7f961a2e0740 1 -- 172.21.15.33:0/4280373807 wait complete. 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 -- start start 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 Processor -- start 2024-09-10T21:10:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabc500 0x55f20aad0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabce20 0x55f20aad1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aacbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f96101ad810 con 0x55f20aabce20 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f96101ad110 con 0x55f20aabc500 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96101aa2a0 con 0x55f20aac8db0 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f961a2e0740 10 client.0 _ll_register_callbacks cb 0x7f960c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f96154a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabce20 0x55f20aad1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f96164a8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabc500 0x55f20aad0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f96154a6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabce20 0x55f20aad1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34148/0 (socket says 172.21.15.33:34148) 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.076+0000 7f96154a6640 1 -- 172.21.15.33:0/1140302407 learned_addr learned my addr 172.21.15.33:0/1140302407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f96164a8640 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aacbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f96154a6640 1 -- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 msgr2=0x55f20aacbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f96154a6640 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aacbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f96154a6640 1 -- 172.21.15.33:0/1140302407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabc500 msgr2=0x55f20aad0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f96154a6640 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabc500 0x55f20aad0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f96154a6640 1 -- 172.21.15.33:0/1140302407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f20aacc1f0 con 0x55f20aabce20 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f96154a6640 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabce20 0x55f20aad1240 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9600009900 tx=0x7f9600004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a1c8ff809f91a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f95f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f95f97fa640 1 -- 172.21.15.33:0/1140302407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9600035040 con 0x55f20aabce20 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f95f97fa640 1 -- 172.21.15.33:0/1140302407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9600005040 con 0x55f20aabce20 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f960801e070 con 0x55f20aabce20 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f95f97fa640 1 -- 172.21.15.33:0/1140302407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f9600053050 con 0x55f20aabce20 2024-09-10T21:10:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f961a2e0740 10 client.16491 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.077+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f20ab45b90 con 0x55f20aabce20 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f95f97fa640 1 -- 172.21.15.33:0/1140302407 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960801e070 con 0x55f20aabce20 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f95f97fa640 1 -- 172.21.15.33:0/1140302407 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f96000035f0 con 0x55f20aabce20 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f961a2e0740 10 client.16491 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f961a2e0740 10 client.16491 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f961a2e0740 20 client.16491 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f961a2e0740 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20ab463f0 0x55f20ab48810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f961a2e0740 4 client.16491 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f20aa2c8a0 con 0x55f20ab463f0 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f96164a8640 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20ab463f0 0x55f20ab48810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:11.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f96164a8640 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20ab463f0 0x55f20ab48810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=988f43ce691ead39 server_cookie=b1ca668ff700192f in_seq=0 out_seq=0 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.078+0000 7f95f97fa640 10 client.16491 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f95f97fa640 1 -- 172.21.15.33:0/1140302407 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f20aa2c8a0 con 0x55f20ab463f0 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f95f97fa640 10 client.16491 handle_command_reply: tid=0 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 client.16491 shutdown 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabce20 msgr2=0x55f20aad1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabce20 0x55f20aad1240 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9600009900 tx=0x7f9600004180 comp rx=0 tx=0).stop 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 shutdown_connections 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55f20ab463f0 0x55f20ab48810 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f20aac8db0 0x55f20aacbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f20aabce20 0x55f20aad1240 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 --2- 172.21.15.33:0/1140302407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f20aabc500 0x55f20aad0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f95f97fa640 0 client.16491 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:11.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 >> 172.21.15.33:0/1140302407 conn(0x55f20aaba330 msgr2=0x55f20aac78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 shutdown_connections 2024-09-10T21:10:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 1 -- 172.21.15.33:0/1140302407 wait complete. 2024-09-10T21:10:11.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f961a2e0740 20 client.16491 trim_cache size 0 max 16384 2024-09-10T21:10:11.089 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95fc0b5360 msgr2=0x7f95fc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f9617f0b640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95fc0b5360 0x7f95fc0b7820 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f960c0ac750 tx=0x7f960c0e9a80 comp rx=0 tx=0).stop 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610103b70 msgr2=0x7f9610199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.079+0000 7f9617f0b640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610103b70 0x7f9610199290 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f960c00e9f0 tx=0x7f960c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 shutdown_connections 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f95fc0b5360 0x7f95fc0b7820 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f961010d120 0x7f961019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9610104570 0x7f96101997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 --2- 172.21.15.33:0/3639441900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9610103b70 0x7f9610199290 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 >> 172.21.15.33:0/3639441900 conn(0x7f96100fd700 msgr2=0x7f96100fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 shutdown_connections 2024-09-10T21:10:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:11.080+0000 7f9617f0b640 1 -- 172.21.15.33:0/3639441900 wait complete. 2024-09-10T21:10:11.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 490395 7.07 MB/sec execute 192 sec latency 4591.072 ms 2024-09-10T21:10:11.859 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:10:12.118 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:11 smithi174 ceph-mon[35635]: pgmap v532: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1011 KiB/s wr, 84 op/s 2024-09-10T21:10:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:11 smithi033 ceph-mon[30860]: pgmap v532: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1011 KiB/s wr, 84 op/s 2024-09-10T21:10:12.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:11 smithi104 ceph-mon[35325]: pgmap v532: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1011 KiB/s wr, 84 op/s 2024-09-10T21:10:12.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 -- 172.21.15.33:0/3253090129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09601045c0 msgr2=0x7f096010cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 --2- 172.21.15.33:0/3253090129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09601045c0 0x7f096010cab0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0954009900 tx=0x7f095402a410 comp rx=0 tx=0).stop 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 -- 172.21.15.33:0/3253090129 shutdown_connections 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 --2- 172.21.15.33:0/3253090129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f096010d100 0x7f096010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 --2- 172.21.15.33:0/3253090129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09601045c0 0x7f096010cab0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 --2- 172.21.15.33:0/3253090129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0960103ca0 0x7f0960104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 -- 172.21.15.33:0/3253090129 >> 172.21.15.33:0/3253090129 conn(0x7f09600fd700 msgr2=0x7f09600ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 -- 172.21.15.33:0/3253090129 shutdown_connections 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.603+0000 7f096815e640 1 -- 172.21.15.33:0/3253090129 wait complete. 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 Processor -- start 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 -- start start 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0960103ca0 0x7f09601992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09601045c0 0x7f0960199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f096010d100 0x7f096019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0960112dc0 con 0x7f0960103ca0 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0960112c40 con 0x7f09601045c0 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f096815e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0960112f40 con 0x7f096010d100 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f0965efa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0960103ca0 0x7f09601992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f0965efa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0960103ca0 0x7f09601992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34160/0 (socket says 172.21.15.33:34160) 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f0965efa640 1 -- 172.21.15.33:0/1639152690 learned_addr learned my addr 172.21.15.33:0/1639152690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f09656f9640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09601045c0 0x7f0960199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.604+0000 7f09666fb640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f096010d100 0x7f096019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f0965efa640 1 -- 172.21.15.33:0/1639152690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f096010d100 msgr2=0x7f096019ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f0965efa640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f096010d100 0x7f096019ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f0965efa640 1 -- 172.21.15.33:0/1639152690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09601045c0 msgr2=0x7f0960199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f0965efa640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09601045c0 0x7f0960199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f0965efa640 1 -- 172.21.15.33:0/1639152690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f096019f380 con 0x7f0960103ca0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f0965efa640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0960103ca0 0x7f09601992d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f095000ebc0 tx=0x7f095000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f095effd640 1 -- 172.21.15.33:0/1639152690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950010630 con 0x7f0960103ca0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f095effd640 1 -- 172.21.15.33:0/1639152690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0950010c80 con 0x7f0960103ca0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f096019f670 con 0x7f0960103ca0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f095effd640 1 -- 172.21.15.33:0/1639152690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950018b60 con 0x7f0960103ca0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.605+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09601a5d30 con 0x7f0960103ca0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f095effd640 1 -- 172.21.15.33:0/1639152690 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f09500107d0 con 0x7f0960103ca0 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 Processor -- start 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 -- start start 2024-09-10T21:10:12.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12b0f0 0x56446f12b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f13b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f13e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56446f0a1ea0 con 0x56446f12b0f0 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56446f0912d0 con 0x56446f12ba10 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f096a533740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56446f0950e0 con 0x56446f13bdb0 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f0965efa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f13b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.608+0000 7f09666fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12b0f0 0x56446f12b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f13b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:42104/0 (socket says 172.21.15.33:42104) 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 -- 172.21.15.33:0/2163052671 learned_addr learned my addr 172.21.15.33:0/2163052671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f09656f9640 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f13e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 -- 172.21.15.33:0/2163052671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 msgr2=0x56446f13e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f13e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 -- 172.21.15.33:0/2163052671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12b0f0 msgr2=0x56446f12b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12b0f0 0x56446f12b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 -- 172.21.15.33:0/2163052671 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56446edef650 con 0x56446f12ba10 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f09666fb640 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12b0f0 0x56446f12b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f095effd640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09480b52e0 0x7f09480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0965efa640 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f13b870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09500ee4b0 tx=0x7f09500ee970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa9caa3b7c6863e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f09666fb640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09480b52e0 0x7f09480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0947fff640 1 -- 172.21.15.33:0/2163052671 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09500a9040 con 0x56446f12ba10 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f095effd640 1 -- 172.21.15.33:0/1639152690 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f09500e61c0 con 0x7f0960103ca0 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0947fff640 1 -- 172.21.15.33:0/2163052671 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0950080070 con 0x56446f12ba10 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.609+0000 7f0947fff640 1 -- 172.21.15.33:0/2163052671 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0950026060 con 0x56446f12ba10 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 -- 172.21.15.33:0/2163052671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 msgr2=0x56446f13b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f09666fb640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09480b52e0 0x7f09480b77a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0958009f70 tx=0x7f0958005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f13b870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f09500ee4b0 tx=0x7f09500ee970 comp rx=0 tx=0).stop 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 -- 172.21.15.33:0/2163052671 shutdown_connections 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f13e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f13b870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 --2- 172.21.15.33:0/2163052671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12b0f0 0x56446f12b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 -- 172.21.15.33:0/2163052671 >> 172.21.15.33:0/2163052671 conn(0x56446f128f20 msgr2=0x56446f12a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 -- 172.21.15.33:0/2163052671 shutdown_connections 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.610+0000 7f096a533740 1 -- 172.21.15.33:0/2163052671 wait complete. 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 -- start start 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 Processor -- start 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12b0f0 0x56446f13b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f138b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f139110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56446eef2700 con 0x56446f13bdb0 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56446f02c160 con 0x56446f12ba10 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56446f13ea60 con 0x56446f12b0f0 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f096a533740 10 client.0 _ll_register_callbacks cb 0x56446f0459e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f0965efa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12b0f0 0x56446f13b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f09656f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f09666fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f139110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f0965efa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12b0f0 0x56446f13b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:58740/0 (socket says 172.21.15.33:58740) 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.611+0000 7f0965efa640 1 -- 172.21.15.33:0/2279988484 learned_addr learned my addr 172.21.15.33:0/2279988484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09656f9640 1 -- 172.21.15.33:0/2279988484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12b0f0 msgr2=0x56446f13b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09656f9640 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12b0f0 0x56446f13b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09656f9640 1 -- 172.21.15.33:0/2279988484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 msgr2=0x56446f139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09656f9640 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f139110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09656f9640 1 -- 172.21.15.33:0/2279988484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446f059c90 con 0x56446f12ba10 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09666fb640 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f0965efa640 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12b0f0 0x56446f13b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09656f9640 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f138b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f095402a930 tx=0x7f09540043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3f4a32374357b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09457fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09457fa640 1 -- 172.21.15.33:0/2279988484 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954035040 con 0x56446f12ba10 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f09457fa640 1 -- 172.21.15.33:0/2279988484 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0954005450 con 0x56446f12ba10 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.612+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0950080070 con 0x56446f12ba10 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f09457fa640 1 -- 172.21.15.33:0/2279988484 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f0954005650 con 0x56446f12ba10 2024-09-10T21:10:12.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 10 client.26021 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56446f14a410 con 0x56446f12ba10 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f09457fa640 1 -- 172.21.15.33:0/2279988484 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0950080070 con 0x56446f12ba10 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f09457fa640 1 -- 172.21.15.33:0/2279988484 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f0954003a80 con 0x56446f12ba10 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 10 client.26021 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 10 client.26021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 20 client.26021 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f14b920 0x56446f14dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 4 client.26021 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56446f0a3df0 con 0x56446f14b920 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.613+0000 7f0965efa640 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f14b920 0x56446f14dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.614+0000 7f0965efa640 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f14b920 0x56446f14dd10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dd0d3277a6a771ec server_cookie=84bc7e5ad327295b in_seq=0 out_seq=0 2024-09-10T21:10:12.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.614+0000 7f09457fa640 10 client.26021 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.614+0000 7f09457fa640 1 -- 172.21.15.33:0/2279988484 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56446f0a3df0 con 0x56446f14b920 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.614+0000 7f09457fa640 10 client.26021 handle_command_reply: tid=0 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.614+0000 7f096a533740 1 client.26021 shutdown 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 msgr2=0x56446f138b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f138b00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f095402a930 tx=0x7f09540043b0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 shutdown_connections 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f14b920 0x56446f14dd10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12ba10 0x56446f138b00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f09457fa640 0 client.26021 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 --2- 172.21.15.33:0/2279988484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12b0f0 0x56446f13b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 >> 172.21.15.33:0/2279988484 conn(0x56446f128f20 msgr2=0x56446f12eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:12.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 shutdown_connections 2024-09-10T21:10:12.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 1 -- 172.21.15.33:0/2279988484 wait complete. 2024-09-10T21:10:12.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.615+0000 7f096a533740 20 client.26021 trim_cache size 0 max 16384 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 Processor -- start 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 -- start start 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12ba10 0x56446f128760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09601ac1b0 con 0x56446f13bdb0 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09601ade70 con 0x56446f12b0f0 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f096a533740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09601ac8e0 con 0x56446f12ba10 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f09666fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f09666fb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f13a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34184/0 (socket says 172.21.15.33:34184) 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f09666fb640 1 -- 172.21.15.33:0/1165170338 learned_addr learned my addr 172.21.15.33:0/1165170338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f0965efa640 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f09656f9640 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12ba10 0x56446f128760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f09666fb640 1 -- 172.21.15.33:0/1165170338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12ba10 msgr2=0x56446f128760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.630+0000 7f09666fb640 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12ba10 0x56446f128760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f09666fb640 1 -- 172.21.15.33:0/1165170338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 msgr2=0x56446f147cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f09666fb640 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f147cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f09666fb640 1 -- 172.21.15.33:0/1165170338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56446f139480 con 0x56446f13bdb0 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f0965efa640 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f147cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f09666fb640 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f13a840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f095801eb20 tx=0x7f095801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d7684ff6a56d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f09477fe640 1 -- 172.21.15.33:0/1165170338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095800f500 con 0x56446f13bdb0 2024-09-10T21:10:12.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f09477fe640 1 -- 172.21.15.33:0/1165170338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f095802f070 con 0x56446f13bdb0 2024-09-10T21:10:12.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.631+0000 7f09477fe640 1 -- 172.21.15.33:0/1165170338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f095802ab20 con 0x56446f13bdb0 2024-09-10T21:10:12.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 -- 172.21.15.33:0/1165170338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 msgr2=0x56446f13a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f13a840 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f095801eb20 tx=0x7f095801b000 comp rx=0 tx=0).stop 2024-09-10T21:10:12.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 -- 172.21.15.33:0/1165170338 shutdown_connections 2024-09-10T21:10:12.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f13bdb0 0x56446f13a840 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f12ba10 0x56446f128760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 --2- 172.21.15.33:0/1165170338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f147cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 -- 172.21.15.33:0/1165170338 >> 172.21.15.33:0/1165170338 conn(0x56446f128f20 msgr2=0x56446f13a130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 -- 172.21.15.33:0/1165170338 shutdown_connections 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 -- 172.21.15.33:0/1165170338 wait complete. 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 -- start start 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.632+0000 7f096a533740 1 Processor -- start 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f096a533740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f143d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f096a533740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12ba10 0x56446f144290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f096a533740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f133fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f096a533740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09601ad6e0 con 0x56446f12ba10 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f096a533740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09601acfe0 con 0x56446f12b0f0 2024-09-10T21:10:12.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f096a533740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09601aa170 con 0x56446f13bdb0 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f096a533740 10 client.0 _ll_register_callbacks cb 0x7f0958005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12ba10 0x56446f144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12ba10 0x56446f144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34188/0 (socket says 172.21.15.33:34188) 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f0965efa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 -- 172.21.15.33:0/3986025724 learned_addr learned my addr 172.21.15.33:0/3986025724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09666fb640 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f133fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 -- 172.21.15.33:0/3986025724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 msgr2=0x56446f133fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f133fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 -- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 msgr2=0x56446f143d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f143d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 -- 172.21.15.33:0/3986025724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446f144b00 con 0x56446f12ba10 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f0965efa640 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:12.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09656f9640 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12ba10 0x56446f144290 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0954009900 tx=0x7f0954005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d9b0ee94780ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.633+0000 7f09457fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f09457fa640 1 -- 172.21.15.33:0/3986025724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954035030 con 0x56446f12ba10 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f09457fa640 1 -- 172.21.15.33:0/3986025724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0954031d50 con 0x56446f12ba10 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f095802f070 con 0x56446f12ba10 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f09457fa640 1 -- 172.21.15.33:0/3986025724 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f09540316d0 con 0x56446f12ba10 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 10 client.16509 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x56446f1bd620 con 0x56446f12ba10 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f09457fa640 1 -- 172.21.15.33:0/3986025724 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f095802f070 con 0x56446f12ba10 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f09457fa640 1 -- 172.21.15.33:0/3986025724 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f0954004930 con 0x56446f12ba10 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 10 client.16509 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 10 client.16509 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 20 client.16509 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:12.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f1bde50 0x56446f1c0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 4 client.16509 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.634+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56446f09f8a0 con 0x56446f1bde50 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f0965efa640 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f1bde50 0x56446f1c0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f09457fa640 10 client.16509 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f0965efa640 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f1bde50 0x56446f1c0270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1dd0fc668fb9474 server_cookie=7f375cb69a743ca3 in_seq=0 out_seq=0 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f09457fa640 1 -- 172.21.15.33:0/3986025724 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56446f09f8a0 con 0x56446f1bde50 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f09457fa640 10 client.16509 handle_command_reply: tid=0 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 client.16509 shutdown 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12ba10 msgr2=0x56446f144290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12ba10 0x56446f144290 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0954009900 tx=0x7f0954005850 comp rx=0 tx=0).stop 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 shutdown_connections 2024-09-10T21:10:12.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56446f13bdb0 0x56446f133fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56446f1bde50 0x56446f1c0270 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56446f12ba10 0x56446f144290 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f09457fa640 0 client.16509 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.635+0000 7f096a533740 1 --2- 172.21.15.33:0/3986025724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56446f12b0f0 0x56446f143d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 >> 172.21.15.33:0/3986025724 conn(0x56446f128f20 msgr2=0x56446f13a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 shutdown_connections 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096a533740 1 -- 172.21.15.33:0/3986025724 wait complete. 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096a533740 20 client.16509 trim_cache size 0 max 16384 2024-09-10T21:10:12.645 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09480b52e0 msgr2=0x7f09480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09480b52e0 0x7f09480b77a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0958009f70 tx=0x7f0958005f90 comp rx=0 tx=0).stop 2024-09-10T21:10:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0960103ca0 msgr2=0x7f09601992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0960103ca0 0x7f09601992d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f095000ebc0 tx=0x7f095000c6a0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 shutdown_connections 2024-09-10T21:10:12.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09480b52e0 0x7f09480b77a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f096010d100 0x7f096019ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09601045c0 0x7f0960199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.636+0000 7f096815e640 1 --2- 172.21.15.33:0/1639152690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0960103ca0 0x7f09601992d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.637+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 >> 172.21.15.33:0/1639152690 conn(0x7f09600fd700 msgr2=0x7f09600fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.637+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 shutdown_connections 2024-09-10T21:10:12.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:12.637+0000 7f096815e640 1 -- 172.21.15.33:0/1639152690 wait complete. 2024-09-10T21:10:12.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 491114 7.05 MB/sec execute 193 sec latency 4960.188 ms 2024-09-10T21:10:13.366 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:10:13.622 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:13.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 491377 7.02 MB/sec execute 194 sec latency 892.293 ms 2024-09-10T21:10:14.008 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:13 smithi033 ceph-mon[30860]: pgmap v533: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1014 KiB/s wr, 85 op/s 2024-09-10T21:10:14.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 -- 172.21.15.33:0/461530652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 msgr2=0x7fea9010f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 --2- 172.21.15.33:0/461530652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea9010f550 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fea88009ca0 tx=0x7fea8802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 -- 172.21.15.33:0/461530652 shutdown_connections 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 --2- 172.21.15.33:0/461530652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea9010f550 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 --2- 172.21.15.33:0/461530652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea90104550 0x7fea9010cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 --2- 172.21.15.33:0/461530652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90103b50 0x7fea90103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 -- 172.21.15.33:0/461530652 >> 172.21.15.33:0/461530652 conn(0x7fea900fd700 msgr2=0x7fea900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 -- 172.21.15.33:0/461530652 shutdown_connections 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.089+0000 7fea98645640 1 -- 172.21.15.33:0/461530652 wait complete. 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 Processor -- start 2024-09-10T21:10:14.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 -- start start 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90103b50 0x7fea9019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea90104550 0x7fea9019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea901a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea901037c0 con 0x7fea9010d100 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea90103640 con 0x7fea90104550 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea98645640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea90103940 con 0x7fea90103b50 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea901a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea901a0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34212/0 (socket says 172.21.15.33:34212) 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 -- 172.21.15.33:0/3116622702 learned_addr learned my addr 172.21.15.33:0/3116622702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea95be0640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea90104550 0x7fea9019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea963e1640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90103b50 0x7fea9019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 -- 172.21.15.33:0/3116622702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90103b50 msgr2=0x7fea9019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90103b50 0x7fea9019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 -- 172.21.15.33:0/3116622702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea90104550 msgr2=0x7fea9019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea90104550 0x7fea9019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.090+0000 7fea96be2640 1 -- 172.21.15.33:0/3116622702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea901a14b0 con 0x7fea9010d100 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.091+0000 7fea96be2640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea901a0ce0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fea88005cc0 tx=0x7fea88031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.091+0000 7fea877fe640 1 -- 172.21.15.33:0/3116622702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea88003b70 con 0x7fea9010d100 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.091+0000 7fea877fe640 1 -- 172.21.15.33:0/3116622702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea880318f0 con 0x7fea9010d100 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.091+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea901a1740 con 0x7fea9010d100 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.091+0000 7fea877fe640 1 -- 172.21.15.33:0/3116622702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8802f950 con 0x7fea9010d100 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.091+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901a7e60 con 0x7fea9010d100 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea877fe640 1 -- 172.21.15.33:0/3116622702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fea8802fb50 con 0x7fea9010d100 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 Processor -- start 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 -- start start 2024-09-10T21:10:14.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0068714d0 0x55a0068718b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00687dd80 0x55a006880170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a0067e7ea0 con 0x55a006871df0 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a0067d72d0 con 0x55a00687dd80 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0067db0e0 con 0x55a0068714d0 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea963e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea963e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34224/0 (socket says 172.21.15.33:34224) 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea963e1640 1 -- 172.21.15.33:0/1234179826 learned_addr learned my addr 172.21.15.33:0/1234179826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.094+0000 7fea95be0640 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00687dd80 0x55a006880170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea95be0640 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0068714d0 0x55a0068718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea95be0640 1 -- 172.21.15.33:0/1234179826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0068714d0 msgr2=0x55a0068718b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea95be0640 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0068714d0 0x55a0068718b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea95be0640 1 -- 172.21.15.33:0/1234179826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 msgr2=0x55a00687d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea95be0640 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea95be0640 1 -- 172.21.15.33:0/1234179826 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a006535650 con 0x55a00687dd80 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea963e1640 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea877fe640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea7c0b52e0 0x7fea7c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea877fe640 1 -- 172.21.15.33:0/3116622702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fea88106cf0 con 0x7fea9010d100 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea963e1640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea7c0b52e0 0x7fea7c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.095+0000 7fea95be0640 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00687dd80 0x55a006880170 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea80009730 tx=0x7fea800096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5343ec01d392a8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea84ff9640 1 -- 172.21.15.33:0/1234179826 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80004050 con 0x55a00687dd80 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea84ff9640 1 -- 172.21.15.33:0/1234179826 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea800054d0 con 0x55a00687dd80 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea84ff9640 1 -- 172.21.15.33:0/1234179826 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80004b50 con 0x55a00687dd80 2024-09-10T21:10:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1234179826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00687dd80 msgr2=0x55a006880170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea963e1640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea7c0b52e0 0x7fea7c0b77a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55a0068701e0 tx=0x7fea8c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00687dd80 0x55a006880170 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fea80009730 tx=0x7fea800096c0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1234179826 shutdown_connections 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a00687dd80 0x55a006880170 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1234179826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0068714d0 0x55a0068718b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.096+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1234179826 >> 172.21.15.33:0/1234179826 conn(0x55a00686f300 msgr2=0x55a0068704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1234179826 shutdown_connections 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1234179826 wait complete. 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 -- start start 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 Processor -- start 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a00687d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a006638700 con 0x55a006871df0 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a006772160 con 0x55a0068714d0 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a006880a00 con 0x55a00687dd80 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea9aa1a740 10 client.0 _ll_register_callbacks cb 0x55a00678b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea95be0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea95be0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34240/0 (socket says 172.21.15.33:34240) 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea95be0640 1 -- 172.21.15.33:0/3096408602 learned_addr learned my addr 172.21.15.33:0/3096408602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea963e1640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a00687d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.097+0000 7fea96be2640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea95be0640 1 -- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 msgr2=0x55a00687b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea95be0640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea95be0640 1 -- 172.21.15.33:0/3096408602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 msgr2=0x55a00687d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea95be0640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a00687d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea95be0640 1 -- 172.21.15.33:0/3096408602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a00679fc90 con 0x55a006871df0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea963e1640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a00687d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea95be0640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687ab00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fea80005bc0 tx=0x7fea80005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca0b44fe2bc7183 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea79ffb640 1 -- 172.21.15.33:0/3096408602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80004d00 con 0x55a006871df0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea79ffb640 1 -- 172.21.15.33:0/3096408602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fea8000f790 con 0x55a006871df0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea800054d0 con 0x55a006871df0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea79ffb640 1 -- 172.21.15.33:0/3096408602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fea800046c0 con 0x55a006871df0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea9aa1a740 10 client.16521 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.098+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a00688c3a0 con 0x55a006871df0 2024-09-10T21:10:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea79ffb640 1 -- 172.21.15.33:0/3096408602 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea800054d0 con 0x55a006871df0 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea79ffb640 1 -- 172.21.15.33:0/3096408602 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7fea80027070 con 0x55a006871df0 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea9aa1a740 10 client.16521 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea9aa1a740 10 client.16521 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea9aa1a740 20 client.16521 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a00688fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea9aa1a740 4 client.16521 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0067e9df0 con 0x55a00688d6f0 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.099+0000 7fea963e1640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a00688fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea963e1640 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a00688fae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1ef7ddd94a2cceca server_cookie=dc3b7ff81bb299ba in_seq=0 out_seq=0 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea79ffb640 10 client.16521 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea79ffb640 1 -- 172.21.15.33:0/3096408602 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a0067e9df0 con 0x55a00688d6f0 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea79ffb640 10 client.16521 handle_command_reply: tid=0 2024-09-10T21:10:14.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 client.16521 shutdown 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 msgr2=0x55a00687ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687ab00 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fea80005bc0 tx=0x7fea80005c50 comp rx=0 tx=0).stop 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 shutdown_connections 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a00688fae0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea79ffb640 0 client.16521 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a00687ab00 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3096408602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a00687d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.100+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 >> 172.21.15.33:0/3096408602 conn(0x55a00686f300 msgr2=0x55a006874ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.101+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 shutdown_connections 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.101+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3096408602 wait complete. 2024-09-10T21:10:14.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.101+0000 7fea9aa1a740 20 client.16521 trim_cache size 0 max 16384 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 Processor -- start 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 -- start start 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006889c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a006870a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b0570 con 0x55a006871df0 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b2230 con 0x55a0068714d0 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b0ca0 con 0x55a00687dd80 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea95be0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a006870a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea95be0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a006870a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34246/0 (socket says 172.21.15.33:34246) 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea95be0640 1 -- 172.21.15.33:0/1126739778 learned_addr learned my addr 172.21.15.33:0/1126739778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea963e1640 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006889c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.114+0000 7fea96be2640 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea95be0640 1 -- 172.21.15.33:0/1126739778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 msgr2=0x55a00687f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea95be0640 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea95be0640 1 -- 172.21.15.33:0/1126739778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 msgr2=0x55a006889c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea95be0640 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006889c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea95be0640 1 -- 172.21.15.33:0/1126739778 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a00687b4f0 con 0x55a006871df0 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea95be0640 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a006870a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fea80009730 tx=0x7fea80029ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57df2dfe1b0da627 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea7bfff640 1 -- 172.21.15.33:0/1126739778 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800055a0 con 0x55a006871df0 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea7bfff640 1 -- 172.21.15.33:0/1126739778 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea80029440 con 0x55a006871df0 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea7bfff640 1 -- 172.21.15.33:0/1126739778 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80004330 con 0x55a006871df0 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.115+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1126739778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 msgr2=0x55a006870a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a006870a60 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fea80009730 tx=0x7fea80029ee0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1126739778 shutdown_connections 2024-09-10T21:10:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a00687dd80 0x55a00687f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a006871df0 0x55a006870a60 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/1126739778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006889c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1126739778 >> 172.21.15.33:0/1126739778 conn(0x55a00686f300 msgr2=0x55a006870350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1126739778 shutdown_connections 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- 172.21.15.33:0/1126739778 wait complete. 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- start start 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 Processor -- start 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006885ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a006871df0 0x55a006886420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a00687dd80 0x55a006876d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b1aa0 con 0x55a00687dd80 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b13a0 con 0x55a0068714d0 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea901ae530 con 0x55a006871df0 2024-09-10T21:10:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.116+0000 7fea9aa1a740 10 client.0 _ll_register_callbacks cb 0x7fea8c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a00687dd80 0x55a006876d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a00687dd80 0x55a006876d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34260/0 (socket says 172.21.15.33:34260) 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea963e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006885ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 -- 172.21.15.33:0/3166334873 learned_addr learned my addr 172.21.15.33:0/3166334873 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea95be0640 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a006871df0 0x55a006886420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 -- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a006871df0 msgr2=0x55a006886420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a006871df0 0x55a006886420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 -- 172.21.15.33:0/3166334873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 msgr2=0x55a006885ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006885ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 -- 172.21.15.33:0/3166334873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a006877570 con 0x55a00687dd80 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea963e1640 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006885ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea96be2640 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a00687dd80 0x55a006876d40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fea88006010 tx=0x7fea88003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498d43818c6219fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea79ffb640 1 -- 172.21.15.33:0/3166334873 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea880cdbb0 con 0x55a00687dd80 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea79ffb640 1 -- 172.21.15.33:0/3166334873 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fea880cd710 con 0x55a00687dd80 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea80029440 con 0x55a00687dd80 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea79ffb640 1 -- 172.21.15.33:0/3166334873 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7fea880cd4b0 con 0x55a00687dd80 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea9aa1a740 10 client.16527 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.117+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55a006878840 con 0x55a00687dd80 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea79ffb640 1 -- 172.21.15.33:0/3166334873 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea80029440 con 0x55a00687dd80 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea79ffb640 1 -- 172.21.15.33:0/3166334873 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7fea880d65b0 con 0x55a00687dd80 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea9aa1a740 10 client.16527 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea9aa1a740 10 client.16527 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea9aa1a740 20 client.16527 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a0068fd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea9aa1a740 4 client.16527 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0067e58a0 con 0x55a00688d6f0 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea963e1640 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a0068fd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea963e1640 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a0068fd6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=258183e19ded2d74 server_cookie=1d0cf65687cdd7f4 in_seq=0 out_seq=0 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.118+0000 7fea79ffb640 10 client.16527 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea79ffb640 1 -- 172.21.15.33:0/3166334873 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0067e58a0 con 0x55a00688d6f0 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea79ffb640 10 client.16527 handle_command_reply: tid=0 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 client.16527 shutdown 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a00687dd80 msgr2=0x55a006876d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a00687dd80 0x55a006876d40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fea88006010 tx=0x7fea88003f90 comp rx=0 tx=0).stop 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 shutdown_connections 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55a00688d6f0 0x55a0068fd6d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a00687dd80 0x55a006876d40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a006871df0 0x55a006886420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 --2- 172.21.15.33:0/3166334873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a0068714d0 0x55a006885ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 >> 172.21.15.33:0/3166334873 conn(0x55a00686f300 msgr2=0x55a00687c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:14.144 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:13 smithi174 ceph-mon[35635]: pgmap v533: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1014 KiB/s wr, 85 op/s 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea79ffb640 0 client.16527 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 shutdown_connections 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 1 -- 172.21.15.33:0/3166334873 wait complete. 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea9aa1a740 20 client.16527 trim_cache size 0 max 16384 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea7c0b52e0 msgr2=0x7fea7c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.119+0000 7fea98645640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea7c0b52e0 0x7fea7c0b77a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x55a0068701e0 tx=0x7fea8c005f90 comp rx=0 tx=0).stop 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 msgr2=0x7fea901a0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea901a0ce0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fea88005cc0 tx=0x7fea88031700 comp rx=0 tx=0).stop 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 shutdown_connections 2024-09-10T21:10:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea7c0b52e0 0x7fea7c0b77a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea9010d100 0x7fea901a0ce0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea90104550 0x7fea9019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 --2- 172.21.15.33:0/3116622702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea90103b50 0x7fea9019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:14.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 >> 172.21.15.33:0/3116622702 conn(0x7fea900fd700 msgr2=0x7fea900fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:14.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 shutdown_connections 2024-09-10T21:10:14.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:14.120+0000 7fea98645640 1 -- 172.21.15.33:0/3116622702 wait complete. 2024-09-10T21:10:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:13 smithi104 ceph-mon[35325]: pgmap v533: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1014 KiB/s wr, 85 op/s 2024-09-10T21:10:14.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 493295 7.04 MB/sec execute 195 sec latency 1241.743 ms 2024-09-10T21:10:14.945 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:10:15.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:14 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:10:15.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:14 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:10:15.239 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:15.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:14 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:10:15.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 493657 7.01 MB/sec execute 196 sec latency 893.473 ms 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 -- 172.21.15.33:0/1779643924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c104550 msgr2=0x7f366c10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 --2- 172.21.15.33:0/1779643924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c104550 0x7f366c10cb30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f3660009900 tx=0x7f366002a410 comp rx=0 tx=0).stop 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 -- 172.21.15.33:0/1779643924 shutdown_connections 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 --2- 172.21.15.33:0/1779643924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c10d100 0x7f366c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 --2- 172.21.15.33:0/1779643924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c104550 0x7f366c10cb30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 --2- 172.21.15.33:0/1779643924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f366c103b50 0x7f366c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 -- 172.21.15.33:0/1779643924 >> 172.21.15.33:0/1779643924 conn(0x7f366c0fd700 msgr2=0x7f366c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 -- 172.21.15.33:0/1779643924 shutdown_connections 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 -- 172.21.15.33:0/1779643924 wait complete. 2024-09-10T21:10:15.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 Processor -- start 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.767+0000 7f36732ef640 1 -- start start 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f36732ef640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c103b50 0x7f366c1992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f36732ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f366c104550 0x7f366c199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f36732ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c10d100 0x7f366c19eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f36732ef640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f366c112d60 con 0x7f366c10d100 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f36732ef640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f366c112be0 con 0x7f366c104550 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f36732ef640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f366c112ee0 con 0x7f366c103b50 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c10d100 0x7f366c19eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c10d100 0x7f366c19eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34272/0 (socket says 172.21.15.33:34272) 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 -- 172.21.15.33:0/369211081 learned_addr learned my addr 172.21.15.33:0/369211081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367108b640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c103b50 0x7f366c1992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367088a640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f366c104550 0x7f366c199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 -- 172.21.15.33:0/369211081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c103b50 msgr2=0x7f366c1992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c103b50 0x7f366c1992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 -- 172.21.15.33:0/369211081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f366c104550 msgr2=0x7f366c199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f366c104550 0x7f366c199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 -- 172.21.15.33:0/369211081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c19f370 con 0x7f366c10d100 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.768+0000 7f367188c640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c10d100 0x7f366c19eba0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f365c00cad0 tx=0x7f365c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.769+0000 7f366a7fc640 1 -- 172.21.15.33:0/369211081 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c00ae20 con 0x7f366c10d100 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.769+0000 7f366a7fc640 1 -- 172.21.15.33:0/369211081 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f365c005020 con 0x7f366c10d100 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.769+0000 7f366a7fc640 1 -- 172.21.15.33:0/369211081 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c00f6b0 con 0x7f366c10d100 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.769+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c19f660 con 0x7f366c10d100 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.769+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a5d20 con 0x7f366c10d100 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.771+0000 7f366a7fc640 1 -- 172.21.15.33:0/369211081 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f365c0054f0 con 0x7f366c10d100 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.771+0000 7f36756c4740 1 Processor -- start 2024-09-10T21:10:15.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.771+0000 7f36756c4740 1 -- start start 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f36756c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f36756c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7fdf0 0x559a7ff8b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f36756c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff8e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f36756c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a7fef5ea0 con 0x559a7ff8bd80 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f36756c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a7fee52d0 con 0x559a7ff7f4d0 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f36756c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a7fee90e0 con 0x559a7ff7fdf0 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff8e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff8e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34274/0 (socket says 172.21.15.33:34274) 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 -- 172.21.15.33:0/308698973 learned_addr learned my addr 172.21.15.33:0/308698973 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367108b640 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7fdf0 0x559a7ff8b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f366a7fc640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f36540b52e0 0x7f36540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 -- 172.21.15.33:0/308698973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7fdf0 msgr2=0x559a7ff8b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7fdf0 0x559a7ff8b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 -- 172.21.15.33:0/308698973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7f4d0 msgr2=0x559a7ff7f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367088a640 1 -- 172.21.15.33:0/308698973 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7fc43650 con 0x559a7ff8bd80 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.772+0000 7f367188c640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f36540b52e0 0x7f36540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f367088a640 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff8e170 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f36600057c0 tx=0x7f36600058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16864477b7d98b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f366a7fc640 1 -- 172.21.15.33:0/369211081 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f365c0e5b30 con 0x7f366c10d100 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f36537fe640 1 -- 172.21.15.33:0/308698973 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660035040 con 0x559a7ff8bd80 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f36537fe640 1 -- 172.21.15.33:0/308698973 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36600035f0 con 0x559a7ff8bd80 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f36537fe640 1 -- 172.21.15.33:0/308698973 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36600044e0 con 0x559a7ff8bd80 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f367188c640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f36540b52e0 0x7f36540b77a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f365c00a6e0 tx=0x7f365c0797c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:15.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f36756c4740 1 -- 172.21.15.33:0/308698973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 msgr2=0x559a7ff8e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.773+0000 7f36756c4740 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff8e170 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f36600057c0 tx=0x7f36600058d0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 -- 172.21.15.33:0/308698973 shutdown_connections 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff8e170 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7fdf0 0x559a7ff8b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 --2- 172.21.15.33:0/308698973 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 -- 172.21.15.33:0/308698973 >> 172.21.15.33:0/308698973 conn(0x559a7ff7d300 msgr2=0x559a7ff7e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 -- 172.21.15.33:0/308698973 shutdown_connections 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 -- 172.21.15.33:0/308698973 wait complete. 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 -- start start 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.774+0000 7f36756c4740 1 Processor -- start 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f36756c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff8b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f36756c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ff88a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f36756c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff89070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f36756c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a7fd46700 con 0x559a7ff8bd80 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f36756c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a7fe80160 con 0x559a7ff7fdf0 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f36756c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a7ff8e9a0 con 0x559a7ff7f4d0 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f36756c4740 10 client.0 _ll_register_callbacks cb 0x559a7fe999e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff89070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff89070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34284/0 (socket says 172.21.15.33:34284) 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 -- 172.21.15.33:0/2479254766 learned_addr learned my addr 172.21.15.33:0/2479254766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff8b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367108b640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ff88a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 -- 172.21.15.33:0/2479254766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 msgr2=0x559a7ff8b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff8b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 -- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 msgr2=0x559a7ff88a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ff88a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 -- 172.21.15.33:0/2479254766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a7feadc90 con 0x559a7ff8bd80 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367108b640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ff88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f367088a640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff89070 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3660032db0 tx=0x7f3660003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94648c3cd8d62464 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.775+0000 7f3650ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f3650ff9640 1 -- 172.21.15.33:0/2479254766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366004d020 con 0x559a7ff8bd80 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f3650ff9640 1 -- 172.21.15.33:0/2479254766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f36600037a0 con 0x559a7ff8bd80 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36600035f0 con 0x559a7ff8bd80 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f3650ff9640 1 -- 172.21.15.33:0/2479254766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3660035070 con 0x559a7ff8bd80 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 10 client.16539 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559a7ff9e6b0 con 0x559a7ff8bd80 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f3650ff9640 1 -- 172.21.15.33:0/2479254766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36600035f0 con 0x559a7ff8bd80 2024-09-10T21:10:15.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f3650ff9640 1 -- 172.21.15.33:0/2479254766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f366003a040 con 0x559a7ff8bd80 2024-09-10T21:10:15.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 10 client.16539 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:15.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:15.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 20 client.16539 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:15.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a7ff9fa00 0x559a7ffa1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 4 client.16539 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:15.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.776+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a7fef7df0 con 0x559a7ff9fa00 2024-09-10T21:10:15.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.777+0000 7f367108b640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a7ff9fa00 0x559a7ffa1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.777+0000 7f3650ff9640 10 client.16539 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:15.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.777+0000 7f367108b640 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a7ff9fa00 0x559a7ffa1df0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de1e1c5a9d3a37a2 server_cookie=610508cb1ef61771 in_seq=0 out_seq=0 2024-09-10T21:10:15.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f3650ff9640 1 -- 172.21.15.33:0/2479254766 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a7fef7df0 con 0x559a7ff9fa00 2024-09-10T21:10:15.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f3650ff9640 10 client.16539 handle_command_reply: tid=0 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 client.16539 shutdown 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 msgr2=0x559a7ff89070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff89070 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3660032db0 tx=0x7f3660003ef0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 shutdown_connections 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a7ff9fa00 0x559a7ffa1df0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ff89070 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ff88a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 --2- 172.21.15.33:0/2479254766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff8b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 >> 172.21.15.33:0/2479254766 conn(0x559a7ff7d300 msgr2=0x559a7ff82e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f3650ff9640 0 client.16539 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 shutdown_connections 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 1 -- 172.21.15.33:0/2479254766 wait complete. 2024-09-10T21:10:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.778+0000 7f36756c4740 20 client.16539 trim_cache size 0 max 16384 2024-09-10T21:10:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.791+0000 7f36756c4740 1 Processor -- start 2024-09-10T21:10:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.791+0000 7f36756c4740 1 -- start start 2024-09-10T21:10:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f36756c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f36756c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff7fdf0 0x559a7ff85ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f36756c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff8bd80 0x559a7ff861e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f36756c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1ac260 con 0x559a7ff7fdf0 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f36756c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1adf20 con 0x559a7ff8bd80 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f36756c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1ac990 con 0x559a7ff7f4d0 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff7fdf0 0x559a7ff85ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff7fdf0 0x559a7ff85ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34300/0 (socket says 172.21.15.33:34300) 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 -- 172.21.15.33:0/4247216835 learned_addr learned my addr 172.21.15.33:0/4247216835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367088a640 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff8bd80 0x559a7ff861e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367088a640 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 -- 172.21.15.33:0/4247216835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 msgr2=0x559a7ff7cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 -- 172.21.15.33:0/4247216835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff8bd80 msgr2=0x559a7ff861e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff8bd80 0x559a7ff861e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 -- 172.21.15.33:0/4247216835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a7ff7dd80 con 0x559a7ff7fdf0 2024-09-10T21:10:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367088a640 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f367108b640 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff7fdf0 0x559a7ff85ca0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3664005fd0 tx=0x7f3664013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49ca546bdde9683 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f3652ffd640 1 -- 172.21.15.33:0/4247216835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664013190 con 0x559a7ff7fdf0 2024-09-10T21:10:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f3652ffd640 1 -- 172.21.15.33:0/4247216835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36640054d0 con 0x559a7ff7fdf0 2024-09-10T21:10:15.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.792+0000 7f3652ffd640 1 -- 172.21.15.33:0/4247216835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664011440 con 0x559a7ff7fdf0 2024-09-10T21:10:15.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 -- 172.21.15.33:0/4247216835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff7fdf0 msgr2=0x559a7ff85ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff7fdf0 0x559a7ff85ca0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3664005fd0 tx=0x7f3664013830 comp rx=0 tx=0).stop 2024-09-10T21:10:15.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 -- 172.21.15.33:0/4247216835 shutdown_connections 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff8bd80 0x559a7ff861e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff7fdf0 0x559a7ff85ca0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 --2- 172.21.15.33:0/4247216835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ff7cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 -- 172.21.15.33:0/4247216835 >> 172.21.15.33:0/4247216835 conn(0x559a7ff7d300 msgr2=0x559a7ff87310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 -- 172.21.15.33:0/4247216835 shutdown_connections 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 -- 172.21.15.33:0/4247216835 wait complete. 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.793+0000 7f36756c4740 1 -- start start 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 1 Processor -- start 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ffa2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ffa2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ffa34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1ad790 con 0x559a7ff8bd80 2024-09-10T21:10:15.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1ad090 con 0x559a7ff7fdf0 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1aa220 con 0x559a7ff7f4d0 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f36756c4740 10 client.0 _ll_register_callbacks cb 0x7f3660032de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ffa34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ffa34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34312/0 (socket says 172.21.15.33:34312) 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 -- 172.21.15.33:0/2175330803 learned_addr learned my addr 172.21.15.33:0/2175330803 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ffa2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367108b640 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ffa2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 -- 172.21.15.33:0/2175330803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 msgr2=0x559a7ffa2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ffa2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 -- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 msgr2=0x559a7ffa2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ffa2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.794+0000 7f367088a640 1 -- 172.21.15.33:0/2175330803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a8000b5f0 con 0x559a7ff8bd80 2024-09-10T21:10:15.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f367088a640 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ffa34b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3660031440 tx=0x7f366002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3938f0e6adf91ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f3650ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f3650ff9640 1 -- 172.21.15.33:0/2175330803 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366004d040 con 0x559a7ff8bd80 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f3650ff9640 1 -- 172.21.15.33:0/2175330803 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f36600317f0 con 0x559a7ff8bd80 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36640054d0 con 0x559a7ff8bd80 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f3650ff9640 1 -- 172.21.15.33:0/2175330803 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f3660005d50 con 0x559a7ff8bd80 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 10 client.16545 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x559a8000c8d0 con 0x559a7ff8bd80 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f3650ff9640 1 -- 172.21.15.33:0/2175330803 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36640054d0 con 0x559a7ff8bd80 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f3650ff9640 1 -- 172.21.15.33:0/2175330803 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f3660032070 con 0x559a7ff8bd80 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 10 client.16545 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 20 client.16545 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a8000cfd0 0x559a8000f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 4 client.16545 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.795+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a7fef38a0 con 0x559a8000cfd0 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.796+0000 7f367108b640 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a8000cfd0 0x559a8000f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.796+0000 7f3650ff9640 10 client.16545 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.796+0000 7f367108b640 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a8000cfd0 0x559a8000f3f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0fa5d9ba64853c3 server_cookie=96df3a88d77d6ce7 in_seq=0 out_seq=0 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.796+0000 7f3650ff9640 1 -- 172.21.15.33:0/2175330803 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a7fef38a0 con 0x559a8000cfd0 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.796+0000 7f3650ff9640 10 client.16545 handle_command_reply: tid=0 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.796+0000 7f36756c4740 1 client.16545 shutdown 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 msgr2=0x559a7ffa34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ffa34b0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3660031440 tx=0x7f366002a410 comp rx=0 tx=0).stop 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 shutdown_connections 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a8000cfd0 0x559a8000f3f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a7ff8bd80 0x559a7ffa34b0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a7ff7fdf0 0x559a7ffa2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 --2- 172.21.15.33:0/2175330803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a7ff7f4d0 0x559a7ffa2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 >> 172.21.15.33:0/2175330803 conn(0x559a7ff7d300 msgr2=0x559a7ff8a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f3650ff9640 0 client.16545 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 shutdown_connections 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 1 -- 172.21.15.33:0/2175330803 wait complete. 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36756c4740 20 client.16545 trim_cache size 0 max 16384 2024-09-10T21:10:15.806 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f36540b52e0 msgr2=0x7f36540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f36540b52e0 0x7f36540b77a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f365c00a6e0 tx=0x7f365c0797c0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c10d100 msgr2=0x7f366c19eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c10d100 0x7f366c19eba0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f365c00cad0 tx=0x7f365c00cf90 comp rx=0 tx=0).stop 2024-09-10T21:10:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 shutdown_connections 2024-09-10T21:10:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f36540b52e0 0x7f36540b77a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f366c10d100 0x7f366c19eba0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f366c104550 0x7f366c199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 --2- 172.21.15.33:0/369211081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f366c103b50 0x7f366c1992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.797+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 >> 172.21.15.33:0/369211081 conn(0x7f366c0fd700 msgr2=0x7f366c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.798+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 shutdown_connections 2024-09-10T21:10:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:15.798+0000 7f36732ef640 1 -- 172.21.15.33:0/369211081 wait complete. 2024-09-10T21:10:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:15 smithi174 ceph-mon[35635]: pgmap v534: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-10T21:10:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:15 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:10:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:15 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:10:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:15 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:10:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:15 smithi033 ceph-mon[30860]: pgmap v534: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-10T21:10:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:15 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:10:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:15 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:10:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:15 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:10:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:15 smithi104 ceph-mon[35325]: pgmap v534: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.0 MiB/s wr, 95 op/s 2024-09-10T21:10:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:15 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:10:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:15 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:10:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:15 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:10:16.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 493657 6.97 MB/sec execute 197 sec latency 1893.577 ms 2024-09-10T21:10:17.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:10:17.617 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:10:17.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 493992 6.96 MB/sec execute 198 sec latency 1933.801 ms 2024-09-10T21:10:17.892 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:17 smithi033 ceph-mon[30860]: pgmap v535: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 681 KiB/s wr, 88 op/s 2024-09-10T21:10:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:17 smithi174 ceph-mon[35635]: pgmap v535: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 681 KiB/s wr, 88 op/s 2024-09-10T21:10:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:17 smithi104 ceph-mon[35325]: pgmap v535: 97 pgs: 97 active+clean; 190 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 513 KiB/s rd, 681 KiB/s wr, 88 op/s 2024-09-10T21:10:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 -- 172.21.15.33:0/1495615204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 msgr2=0x7f7d7c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1495615204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c108bf0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009900 tx=0x7f7d7002a410 comp rx=0 tx=0).stop 2024-09-10T21:10:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 -- 172.21.15.33:0/1495615204 shutdown_connections 2024-09-10T21:10:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1495615204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109130 0x7f7d7c10f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1495615204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c108bf0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1495615204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c102da0 0x7f7d7c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 -- 172.21.15.33:0/1495615204 >> 172.21.15.33:0/1495615204 conn(0x7f7d7c0fe810 msgr2=0x7f7d7c100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:18.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 -- 172.21.15.33:0/1495615204 shutdown_connections 2024-09-10T21:10:18.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.406+0000 7f7d8249c640 1 -- 172.21.15.33:0/1495615204 wait complete. 2024-09-10T21:10:18.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 Processor -- start 2024-09-10T21:10:18.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 -- start start 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c102da0 0x7f7d7c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c109130 0x7f7d7c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112df0 con 0x7f7d7c1087b0 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112c70 con 0x7f7d7c109130 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.407+0000 7f7d8249c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112f70 con 0x7f7d7c102da0 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34324/0 (socket says 172.21.15.33:34324) 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/1864909574 learned_addr learned my addr 172.21.15.33:0/1864909574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:18.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d80a39640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c109130 0x7f7d7c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c102da0 0x7f7d7c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/1864909574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c102da0 msgr2=0x7f7d7c1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c102da0 0x7f7d7c1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/1864909574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c109130 msgr2=0x7f7d7c19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c109130 0x7f7d7c19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/1864909574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c19f350 con 0x7f7d7c1087b0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c1997e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7d70005940 tx=0x7f7d700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d797fa640 1 -- 172.21.15.33:0/1864909574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7003f030 con 0x7f7d7c1087b0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c19f5e0 con 0x7f7d7c1087b0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d797fa640 1 -- 172.21.15.33:0/1864909574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d70003cb0 con 0x7f7d7c1087b0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d797fa640 1 -- 172.21.15.33:0/1864909574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70035070 con 0x7f7d7c1087b0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.408+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1a5d00 con 0x7f7d7c1087b0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d797fa640 1 -- 172.21.15.33:0/1864909574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7d7003bbf0 con 0x7f7d7c1087b0 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 Processor -- start 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 -- start start 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c8fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c90b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c90bba0 0x55f83c90df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f83c871e60 con 0x55f83c8fb800 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f83c861290 con 0x55f83c90bba0 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d84871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f83c8650a0 con 0x55f83c8faee0 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.411+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c90b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c90b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34326/0 (socket says 172.21.15.33:34326) 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d80a39640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c8fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 -- 172.21.15.33:0/1366304682 learned_addr learned my addr 172.21.15.33:0/1366304682 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c90bba0 0x55f83c90df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d797fa640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d640b52e0 0x7f7d640b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 -- 172.21.15.33:0/1366304682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 msgr2=0x55f83c8fb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c8fb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 -- 172.21.15.33:0/1366304682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c90bba0 msgr2=0x55f83c90df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c90bba0 0x55f83c90df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 -- 172.21.15.33:0/1366304682 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83c5bf930 con 0x55f83c8fb800 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d80a39640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d640b52e0 0x7f7d640b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c90b660 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00cc90 tx=0x7f7d6c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63505c1ed45fea6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d797fa640 1 -- 172.21.15.33:0/1864909574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d70106750 con 0x7f7d7c1087b0 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d627fc640 1 -- 172.21.15.33:0/1366304682 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c014070 con 0x55f83c8fb800 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d627fc640 1 -- 172.21.15.33:0/1366304682 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d6c0047a0 con 0x55f83c8fb800 2024-09-10T21:10:18.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.412+0000 7f7d627fc640 1 -- 172.21.15.33:0/1366304682 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c0106b0 con 0x55f83c8fb800 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 -- 172.21.15.33:0/1366304682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 msgr2=0x55f83c90b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d80a39640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d640b52e0 0x7f7d640b77a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005fd0 tx=0x7f7d74004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c90b660 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c00cc90 tx=0x7f7d6c00e720 comp rx=0 tx=0).stop 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 -- 172.21.15.33:0/1366304682 shutdown_connections 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c90bba0 0x55f83c90df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c90b660 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 --2- 172.21.15.33:0/1366304682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c8fb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 -- 172.21.15.33:0/1366304682 >> 172.21.15.33:0/1366304682 conn(0x55f83c8f8d10 msgr2=0x55f83c8f9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 -- 172.21.15.33:0/1366304682 shutdown_connections 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 -- 172.21.15.33:0/1366304682 wait complete. 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.413+0000 7f7d84871740 1 -- start start 2024-09-10T21:10:18.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 1 Processor -- start 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c908b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c909080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c9095c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f83c6c2890 con 0x55f83c8fb800 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f83c7fbd40 con 0x55f83c8faee0 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f83c90e850 con 0x55f83c90bba0 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d84871740 10 client.0 _ll_register_callbacks cb 0x55f83c85fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c909080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c909080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34332/0 (socket says 172.21.15.33:34332) 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/4135109650 learned_addr learned my addr 172.21.15.33:0/4135109650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c908b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/4135109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 msgr2=0x55f83c9095c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d80a39640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c9095c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c9095c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 msgr2=0x55f83c908b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c908b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/4135109650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83c917c00 con 0x55f83c8fb800 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.414+0000 7f7d80a39640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c9095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c909080 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ca180 tx=0x7f7d7003b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af19e22a5c39e096 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/4135109650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70049070 con 0x55f83c8fb800 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/4135109650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d700cd370 con 0x55f83c8fb800 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d6c0047a0 con 0x55f83c8fb800 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/4135109650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7d700cc680 con 0x55f83c8fb800 2024-09-10T21:10:18.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 10 client.16557 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f83c91a5b0 con 0x55f83c8fb800 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/4135109650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d6c0047a0 con 0x55f83c8fb800 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/4135109650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f7d700d6010 con 0x55f83c8fb800 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 10 client.16557 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 10 client.16557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 20 client.16557 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c91ba10 0x55f83c91de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 4 client.16557 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.415+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83c873db0 con 0x55f83c91ba10 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.416+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c91ba10 0x55f83c91de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.416+0000 7f7d5b7fe640 10 client.16557 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.416+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c91ba10 0x55f83c91de00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=199431dfba5c66a4 server_cookie=dafe71dd81d4b3a6 in_seq=0 out_seq=0 2024-09-10T21:10:18.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/4135109650 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f83c873db0 con 0x55f83c91ba10 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d5b7fe640 10 client.16557 handle_command_reply: tid=0 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 client.16557 shutdown 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 msgr2=0x55f83c909080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c909080 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ca180 tx=0x7f7d7003b6b0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 shutdown_connections 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c91ba10 0x55f83c91de00 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c9095c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c909080 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 --2- 172.21.15.33:0/4135109650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c908b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 >> 172.21.15.33:0/4135109650 conn(0x55f83c8f8d10 msgr2=0x55f83c8f7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d5b7fe640 0 client.16557 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 shutdown_connections 2024-09-10T21:10:18.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 1 -- 172.21.15.33:0/4135109650 wait complete. 2024-09-10T21:10:18.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.417+0000 7f7d84871740 20 client.16557 trim_cache size 0 max 16384 2024-09-10T21:10:18.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.435+0000 7f7d84871740 1 Processor -- start 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.435+0000 7f7d84871740 1 -- start start 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d84871740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c8fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d84871740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c906630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d84871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c909060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d84871740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ac160 con 0x55f83c8fb800 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d84871740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ade20 con 0x55f83c8faee0 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d84871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ac890 con 0x55f83c90bba0 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c906630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c906630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34348/0 (socket says 172.21.15.33:34348) 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/286098755 learned_addr learned my addr 172.21.15.33:0/286098755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d80a39640 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c909060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c8fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/286098755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 msgr2=0x55f83c909060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c909060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/286098755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 msgr2=0x55f83c8fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c8fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 -- 172.21.15.33:0/286098755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83c907ac0 con 0x55f83c8fb800 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c906630 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7d700f0110 tx=0x7f7d70106a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0425747c5da8956 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d61ffb640 1 -- 172.21.15.33:0/286098755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70049070 con 0x55f83c8fb800 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.436+0000 7f7d61ffb640 1 -- 172.21.15.33:0/286098755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d700b8020 con 0x55f83c8fb800 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d61ffb640 1 -- 172.21.15.33:0/286098755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70106e60 con 0x55f83c8fb800 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 -- 172.21.15.33:0/286098755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 msgr2=0x55f83c906630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c906630 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7d700f0110 tx=0x7f7d70106a80 comp rx=0 tx=0).stop 2024-09-10T21:10:18.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 -- 172.21.15.33:0/286098755 shutdown_connections 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c90bba0 0x55f83c909060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c8fb800 0x55f83c906630 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 --2- 172.21.15.33:0/286098755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8faee0 0x55f83c8fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 -- 172.21.15.33:0/286098755 >> 172.21.15.33:0/286098755 conn(0x55f83c8f8d10 msgr2=0x55f83c8f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 -- 172.21.15.33:0/286098755 shutdown_connections 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 -- 172.21.15.33:0/286098755 wait complete. 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.437+0000 7f7d84871740 1 -- start start 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 1 Processor -- start 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c913ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8fb800 0x55f83c914020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c90bba0 0x55f83c922700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ad690 con 0x55f83c90bba0 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1acf90 con 0x55f83c8fb800 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1aa120 con 0x55f83c8faee0 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d80a39640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c90bba0 0x55f83c922700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d84871740 10 client.0 _ll_register_callbacks cb 0x7f7d7003bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d80a39640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c90bba0 0x55f83c922700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34358/0 (socket says 172.21.15.33:34358) 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d80a39640 1 -- 172.21.15.33:0/3617558417 learned_addr learned my addr 172.21.15.33:0/3617558417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c913ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.438+0000 7f7d7b7fe640 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8fb800 0x55f83c914020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d80a39640 1 -- 172.21.15.33:0/3617558417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 msgr2=0x55f83c913ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d80a39640 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c913ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d80a39640 1 -- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8fb800 msgr2=0x55f83c914020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d80a39640 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8fb800 0x55f83c914020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d80a39640 1 -- 172.21.15.33:0/3617558417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f83c922c40 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d80a39640 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c90bba0 0x55f83c922700 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009990 tx=0x7f7d74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8650d0c42eaf5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/3617558417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74011910 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/3617558417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d74018070 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d700b8020 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/3617558417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f7d74011ab0 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d84871740 10 client.16563 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55f83c923fe0 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.439+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/3617558417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d700b8020 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/3617558417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f7d7401ec10 con 0x55f83c90bba0 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d84871740 10 client.16563 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:18.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d84871740 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d84871740 20 client.16563 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d84871740 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c989990 0x55f83c98bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d84871740 4 client.16563 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f83c86f860 con 0x55f83c989990 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c989990 0x55f83c98bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c989990 0x55f83c98bdb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1cedf058229e60 server_cookie=14954be8d900831e in_seq=0 out_seq=0 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.440+0000 7f7d5b7fe640 10 client.16563 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d5b7fe640 1 -- 172.21.15.33:0/3617558417 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f83c86f860 con 0x55f83c989990 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d5b7fe640 10 client.16563 handle_command_reply: tid=0 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 client.16563 shutdown 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c90bba0 msgr2=0x55f83c922700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c90bba0 0x55f83c922700 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009990 tx=0x7f7d74004ab0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 shutdown_connections 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f83c989990 0x55f83c98bdb0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f83c90bba0 0x55f83c922700 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f83c8fb800 0x55f83c914020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d5b7fe640 0 client.16563 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 --2- 172.21.15.33:0/3617558417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f83c8faee0 0x55f83c913ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.442+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 >> 172.21.15.33:0/3617558417 conn(0x55f83c8f8d10 msgr2=0x55f83c90a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 shutdown_connections 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d84871740 1 -- 172.21.15.33:0/3617558417 wait complete. 2024-09-10T21:10:18.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d84871740 20 client.16563 trim_cache size 0 max 16384 2024-09-10T21:10:18.459 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:10:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:10:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "ee25ff54-3f06-4448-8815-95355590c2e1", 2024-09-10T21:10:18.459 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:10:18.459 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d640b52e0 msgr2=0x7f7d640b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d640b52e0 0x7f7d640b77a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005fd0 tx=0x7f7d74004a40 comp rx=0 tx=0).stop 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 msgr2=0x7f7d7c1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c1997e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7d70005940 tx=0x7f7d700042f0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 shutdown_connections 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d640b52e0 0x7f7d640b77a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c109130 0x7f7d7c19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087b0 0x7f7d7c1997e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 --2- 172.21.15.33:0/1864909574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c102da0 0x7f7d7c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:18.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 >> 172.21.15.33:0/1864909574 conn(0x7f7d7c0fe810 msgr2=0x7f7d7c100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 shutdown_connections 2024-09-10T21:10:18.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:18.443+0000 7f7d8249c640 1 -- 172.21.15.33:0/1864909574 wait complete. 2024-09-10T21:10:18.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 494179 6.93 MB/sec execute 199 sec latency 1316.426 ms 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.113 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:18 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:10:19.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:19.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:19.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:19.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:19.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:18 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:19.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:19.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:19.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:19.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:18 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:19.409 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:19.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 496451 6.93 MB/sec execute 200 sec latency 18.964 ms 2024-09-10T21:10:19.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 -- 172.21.15.33:0/3019214971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc1076a0 msgr2=0x7f2ffc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 --2- 172.21.15.33:0/3019214971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc1076a0 0x7f2ffc107ae0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009900 tx=0x7f2fec02a410 comp rx=0 tx=0).stop 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 -- 172.21.15.33:0/3019214971 shutdown_connections 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 --2- 172.21.15.33:0/3019214971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ffc108020 0x7f2ffc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 --2- 172.21.15.33:0/3019214971 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc1076a0 0x7f2ffc107ae0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 --2- 172.21.15.33:0/3019214971 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ffc101c90 0x7f2ffc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 -- 172.21.15.33:0/3019214971 >> 172.21.15.33:0/3019214971 conn(0x7f2ffc0fd700 msgr2=0x7f2ffc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 -- 172.21.15.33:0/3019214971 shutdown_connections 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.883+0000 7f3003d46640 1 -- 172.21.15.33:0/3019214971 wait complete. 2024-09-10T21:10:19.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 Processor -- start 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 -- start start 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ffc101c90 0x7f2ffc197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ffc1076a0 0x7f2ffc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc108020 0x7f2ffc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111cf0 con 0x7f2ffc108020 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111b70 con 0x7f2ffc1076a0 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3003d46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc111e70 con 0x7f2ffc101c90 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f30022e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc108020 0x7f2ffc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f30022e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc108020 0x7f2ffc198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39864/0 (socket says 172.21.15.33:39864) 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f30022e3640 1 -- 172.21.15.33:0/3541813958 learned_addr learned my addr 172.21.15.33:0/3541813958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f3001ae2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ffc101c90 0x7f2ffc197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.884+0000 7f30012e1640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ffc1076a0 0x7f2ffc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f30022e3640 1 -- 172.21.15.33:0/3541813958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ffc101c90 msgr2=0x7f2ffc197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f30022e3640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ffc101c90 0x7f2ffc197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f30022e3640 1 -- 172.21.15.33:0/3541813958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ffc1076a0 msgr2=0x7f2ffc1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f30022e3640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ffc1076a0 0x7f2ffc1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f30022e3640 1 -- 172.21.15.33:0/3541813958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ffc1a7690 con 0x7f2ffc108020 2024-09-10T21:10:19.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ffc101c90 0x7f2ffc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f30012e1640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ffc1076a0 0x7f2ffc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f30022e3640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc108020 0x7f2ffc198930 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400eb10 tx=0x7f2ff400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f2ff2ffd640 1 -- 172.21.15.33:0/3541813958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4004b20 con 0x7f2ffc108020 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ffc1a7980 con 0x7f2ffc108020 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f2ff2ffd640 1 -- 172.21.15.33:0/3541813958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff4004cc0 con 0x7f2ffc108020 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f2ff2ffd640 1 -- 172.21.15.33:0/3541813958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff4010690 con 0x7f2ffc108020 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.885+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ffc1a7ee0 con 0x7f2ffc108020 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.888+0000 7f300611b740 1 Processor -- start 2024-09-10T21:10:19.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f2ff2ffd640 1 -- 172.21.15.33:0/3541813958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2ff4005020 con 0x7f2ffc108020 2024-09-10T21:10:19.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f300611b740 1 -- start start 2024-09-10T21:10:19.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f300611b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f300611b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f300611b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f300611b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf86b2fe50 con 0x55bf86bc5950 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f300611b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf86b1f280 con 0x55bf86bb99c0 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f300611b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf86b23090 con 0x55bf86bb90a0 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f30012e1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f3001ae2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f3001ae2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59924/0 (socket says 172.21.15.33:59924) 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f3001ae2640 1 -- 172.21.15.33:0/3259239447 learned_addr learned my addr 172.21.15.33:0/3259239447 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.889+0000 7f30012e1640 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f3001ae2640 1 -- 172.21.15.33:0/3259239447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 msgr2=0x55bf86bb9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f3001ae2640 1 -- 172.21.15.33:0/3259239447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 msgr2=0x55bf86bc7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f3001ae2640 1 -- 172.21.15.33:0/3259239447 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8687d650 con 0x55bf86bb99c0 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f2ff2ffd640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2fe80b52e0 0x7f2fe80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f30012e1640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2fe80b52e0 0x7f2fe80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc5410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80115f0 tx=0x7f2ff8011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76b40835f03fc39d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f2ff2ffd640 1 -- 172.21.15.33:0/3541813958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2ff40e6150 con 0x7f2ffc108020 2024-09-10T21:10:19.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f2fe7fff640 1 -- 172.21.15.33:0/3259239447 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff801e070 con 0x55bf86bb99c0 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f2fe7fff640 1 -- 172.21.15.33:0/3259239447 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff800fa70 con 0x55bf86bb99c0 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.890+0000 7f2fe7fff640 1 -- 172.21.15.33:0/3259239447 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8019a40 con 0x55bf86bb99c0 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 -- 172.21.15.33:0/3259239447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 msgr2=0x55bf86bc5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f30012e1640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2fe80b52e0 0x7f2fe80b77a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005ab0 tx=0x7f2fec005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc5410 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80115f0 tx=0x7f2ff8011ab0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 -- 172.21.15.33:0/3259239447 shutdown_connections 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc5410 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 --2- 172.21.15.33:0/3259239447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 -- 172.21.15.33:0/3259239447 >> 172.21.15.33:0/3259239447 conn(0x55bf86bb6ed0 msgr2=0x55bf86bb8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 -- 172.21.15.33:0/3259239447 shutdown_connections 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.891+0000 7f300611b740 1 -- 172.21.15.33:0/3259239447 wait complete. 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 -- start start 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 Processor -- start 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bc5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bc26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf86980700 con 0x55bf86bc5950 2024-09-10T21:10:19.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf86aba110 con 0x55bf86bb90a0 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf86bc85d0 con 0x55bf86bb99c0 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f300611b740 10 client.0 _ll_register_callbacks cb 0x55bf86ad3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f30022e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f3001ae2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bc5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f30022e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39878/0 (socket says 172.21.15.33:39878) 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f30022e3640 1 -- 172.21.15.33:0/3041597794 learned_addr learned my addr 172.21.15.33:0/3041597794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bc5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59930/0 (socket says 172.21.15.33:59930) 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.892+0000 7f30012e1640 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bc26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f30022e3640 1 -- 172.21.15.33:0/3041597794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 msgr2=0x55bf86bc26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f30022e3640 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bc26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f30022e3640 1 -- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 msgr2=0x55bf86bc5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f30022e3640 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bc5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f30022e3640 1 -- 172.21.15.33:0/3041597794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf86ae7c40 con 0x55bf86bc5950 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f30022e3640 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc2ce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40a8ee0 tx=0x7f2ff40ac6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bfe0a24d6de6bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f2fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3041597794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40a8080 con 0x55bf86bc5950 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3041597794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2ff40b5a10 con 0x55bf86bc5950 2024-09-10T21:10:19.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff800fa70 con 0x55bf86bc5950 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3041597794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2ff4005540 con 0x55bf86bc5950 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f300611b740 10 client.16575 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bf86bd3f90 con 0x55bf86bc5950 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.893+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3041597794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff800fa70 con 0x55bf86bc5950 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3041597794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f2ff40cf040 con 0x55bf86bc5950 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f300611b740 10 client.16575 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f300611b740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f300611b740 20 client.16575 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:19.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f300611b740 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86bd5590 0x55bf86bd79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f300611b740 4 client.16575 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf86b31da0 con 0x55bf86bd5590 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86bd5590 0x55bf86bd79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86bd5590 0x55bf86bd79f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44f74642786fa410 server_cookie=25063d19e5125f25 in_seq=0 out_seq=0 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.894+0000 7f2fe57fa640 10 client.16575 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3041597794 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf86b31da0 con 0x55bf86bd5590 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f2fe57fa640 10 client.16575 handle_command_reply: tid=0 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 client.16575 shutdown 2024-09-10T21:10:19.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 msgr2=0x55bf86bc2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc2ce0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2ff40a8ee0 tx=0x7f2ff40ac6e0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 shutdown_connections 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86bd5590 0x55bf86bd79f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc2ce0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bc26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f2fe57fa640 0 client.16575 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 --2- 172.21.15.33:0/3041597794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bc5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 >> 172.21.15.33:0/3041597794 conn(0x55bf86bb6ed0 msgr2=0x55bf86bbca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 shutdown_connections 2024-09-10T21:10:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 1 -- 172.21.15.33:0/3041597794 wait complete. 2024-09-10T21:10:19.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.896+0000 7f300611b740 20 client.16575 trim_cache size 0 max 16384 2024-09-10T21:10:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 Processor -- start 2024-09-10T21:10:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 -- start start 2024-09-10T21:10:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b0660 con 0x55bf86bc5950 2024-09-10T21:10:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b2320 con 0x55bf86bb99c0 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f300611b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b0d90 con 0x55bf86bb90a0 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f30022e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f30022e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39890/0 (socket says 172.21.15.33:39890) 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f30012e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f30022e3640 1 -- 172.21.15.33:0/1313962905 learned_addr learned my addr 172.21.15.33:0/1313962905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.914+0000 7f3001ae2640 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f30022e3640 1 -- 172.21.15.33:0/1313962905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 msgr2=0x55bf86bb6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f30022e3640 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f30022e3640 1 -- 172.21.15.33:0/1313962905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 msgr2=0x55bf86bc0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f30022e3640 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f30022e3640 1 -- 172.21.15.33:0/1313962905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf86bb7950 con 0x55bf86bc5950 2024-09-10T21:10:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f30022e3640 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc1470 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4004f90 tx=0x7f2ff40ac810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d65d0d4f94db927 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f2fe77fe640 1 -- 172.21.15.33:0/1313962905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40ac260 con 0x55bf86bc5950 2024-09-10T21:10:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f2fe77fe640 1 -- 172.21.15.33:0/1313962905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff40ad010 con 0x55bf86bc5950 2024-09-10T21:10:19.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.915+0000 7f2fe77fe640 1 -- 172.21.15.33:0/1313962905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40a8080 con 0x55bf86bc5950 2024-09-10T21:10:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 -- 172.21.15.33:0/1313962905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 msgr2=0x55bf86bc1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc1470 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4004f90 tx=0x7f2ff40ac810 comp rx=0 tx=0).stop 2024-09-10T21:10:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 -- 172.21.15.33:0/1313962905 shutdown_connections 2024-09-10T21:10:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc1470 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb99c0 0x55bf86bc0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 --2- 172.21.15.33:0/1313962905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb90a0 0x55bf86bb6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 -- 172.21.15.33:0/1313962905 >> 172.21.15.33:0/1313962905 conn(0x55bf86bb6ed0 msgr2=0x55bf86bbd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 -- 172.21.15.33:0/1313962905 shutdown_connections 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 -- 172.21.15.33:0/1313962905 wait complete. 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 -- start start 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.916+0000 7f300611b740 1 Processor -- start 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f300611b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bcd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f300611b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bcddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f300611b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f300611b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b1b90 con 0x55bf86bc5950 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f300611b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1b1490 con 0x55bf86bb90a0 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f300611b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc1ae620 con 0x55bf86bb99c0 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f300611b740 10 client.0 _ll_register_callbacks cb 0x7f2ff40a9030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bcd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f30022e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bcd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59952/0 (socket says 172.21.15.33:59952) 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f30012e1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bcddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 -- 172.21.15.33:0/3594106359 learned_addr learned my addr 172.21.15.33:0/3594106359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 -- 172.21.15.33:0/3594106359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 msgr2=0x55bf86bcddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bcddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 -- 172.21.15.33:0/3594106359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 msgr2=0x55bf86bc8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 -- 172.21.15.33:0/3594106359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf86bce450 con 0x55bf86bb90a0 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f30022e3640 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f3001ae2640 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bcd870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8011510 tx=0x7f2ff8009f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d77ca18390b06809 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.917+0000 7f2fe57fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:10:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3594106359 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8031060 con 0x55bf86bb90a0 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ff40ad010 con 0x55bf86bb90a0 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3594106359 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2ff800fca0 con 0x55bf86bb90a0 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3594106359 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f2ff801e040 con 0x55bf86bb90a0 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 10 client.26096 fetch_fsmap learned FSMap version 49 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) -- 0x55bf86c42970 con 0x55bf86bb90a0 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3594106359 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ff40ad010 con 0x55bf86bb90a0 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3594106359 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 49) ==== 8402+0+0 (secure 0 0 0) 0x7f2ff8036070 con 0x55bf86bb90a0 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 10 client.26096 fetch_fsmap finished waiting for FSMap version 49 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 20 client.26096 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86c43030 0x55bf86c45450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 4 client.26096 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.918+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf86b2d850 con 0x55bf86c43030 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f30012e1640 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86c43030 0x55bf86c45450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f30012e1640 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86c43030 0x55bf86c45450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=350e1df88f6b8d0d server_cookie=d42e834922a4bee in_seq=0 out_seq=0 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f2fe57fa640 10 client.26096 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f2fe57fa640 1 -- 172.21.15.33:0/3594106359 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x55bf86b2d850 con 0x55bf86c43030 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f2fe57fa640 10 client.26096 handle_command_reply: tid=0 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f300611b740 1 client.26096 shutdown 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 msgr2=0x55bf86bcd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.919+0000 7f300611b740 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bcd870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8011510 tx=0x7f2ff8009f60 comp rx=0 tx=0).stop 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 shutdown_connections 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf86c43030 0x55bf86c45450 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf86bc5950 0x55bf86bc8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf86bb99c0 0x55bf86bcddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 --2- 172.21.15.33:0/3594106359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf86bb90a0 0x55bf86bcd870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 >> 172.21.15.33:0/3594106359 conn(0x55bf86bb6ed0 msgr2=0x55bf86bc0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f2fe57fa640 0 client.26096 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 shutdown_connections 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 1 -- 172.21.15.33:0/3594106359 wait complete. 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f300611b740 20 client.26096 trim_cache size 0 max 16384 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:10:19.929 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "ee25ff54-3f06-4448-8815-95355590c2e1", 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.930 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.931 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.932 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:10:19.933 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.934 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:19.935 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:19.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:19.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:19.936 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:19.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:10:19.936 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:10:19.936 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2fe80b52e0 msgr2=0x7f2fe80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f3003d46640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2fe80b52e0 0x7f2fe80b77a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2fec005ab0 tx=0x7f2fec005d90 comp rx=0 tx=0).stop 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc108020 msgr2=0x7f2ffc198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.920+0000 7f3003d46640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc108020 0x7f2ffc198930 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400eb10 tx=0x7f2ff400c2d0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 shutdown_connections 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2fe80b52e0 0x7f2fe80b77a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc108020 0x7f2ffc198930 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2ffc1076a0 0x7f2ffc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 --2- 172.21.15.33:0/3541813958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ffc101c90 0x7f2ffc197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 >> 172.21.15.33:0/3541813958 conn(0x7f2ffc0fd700 msgr2=0x7f2ffc0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 shutdown_connections 2024-09-10T21:10:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:19.921+0000 7f3003d46640 1 -- 172.21.15.33:0/3541813958 wait complete. 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: pgmap v536: 97 pgs: 97 active+clean; 191 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 581 KiB/s wr, 103 op/s 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:10:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: pgmap v536: 97 pgs: 97 active+clean; 191 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 581 KiB/s wr, 103 op/s 2024-09-10T21:10:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:10:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:20.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:20.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:10:20.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: pgmap v536: 97 pgs: 97 active+clean; 191 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 581 KiB/s wr, 103 op/s 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:10:20.654 INFO:tasks.cephfs.filesystem:scrub status for tag:ee25ff54-3f06-4448-8815-95355590c2e1 - {'path': '/', 'tag': 'ee25ff54-3f06-4448-8815-95355590c2e1', 'options': 'recursive,force'} 2024-09-10T21:10:20.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497200 6.93 MB/sec execute 201 sec latency 768.869 ms 2024-09-10T21:10:21.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497200 6.90 MB/sec execute 202 sec latency 1768.982 ms 2024-09-10T21:10:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:21 smithi174 ceph-mon[35635]: pgmap v537: 97 pgs: 97 active+clean; 191 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 239 KiB/s wr, 60 op/s 2024-09-10T21:10:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:21 smithi033 ceph-mon[30860]: pgmap v537: 97 pgs: 97 active+clean; 191 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 239 KiB/s wr, 60 op/s 2024-09-10T21:10:22.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:21 smithi104 ceph-mon[35325]: pgmap v537: 97 pgs: 97 active+clean; 191 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 239 KiB/s wr, 60 op/s 2024-09-10T21:10:22.731 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497513 6.88 MB/sec execute 203 sec latency 1809.837 ms 2024-09-10T21:10:23.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497513 6.85 MB/sec execute 204 sec latency 1613.666 ms 2024-09-10T21:10:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:23 smithi174 ceph-mon[35635]: pgmap v538: 97 pgs: 97 active+clean; 192 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 78 op/s 2024-09-10T21:10:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:23 smithi033 ceph-mon[30860]: pgmap v538: 97 pgs: 97 active+clean; 192 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 78 op/s 2024-09-10T21:10:24.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:23 smithi104 ceph-mon[35325]: pgmap v538: 97 pgs: 97 active+clean; 192 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 283 KiB/s wr, 78 op/s 2024-09-10T21:10:24.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497513 6.81 MB/sec execute 205 sec latency 2613.792 ms 2024-09-10T21:10:25.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497513 6.78 MB/sec execute 206 sec latency 3613.867 ms 2024-09-10T21:10:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:25 smithi174 ceph-mon[35635]: pgmap v539: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 373 KiB/s wr, 96 op/s 2024-09-10T21:10:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:25 smithi033 ceph-mon[30860]: pgmap v539: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 373 KiB/s wr, 96 op/s 2024-09-10T21:10:26.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:25 smithi104 ceph-mon[35325]: pgmap v539: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 704 KiB/s rd, 373 KiB/s wr, 96 op/s 2024-09-10T21:10:26.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497513 6.75 MB/sec execute 207 sec latency 4613.947 ms 2024-09-10T21:10:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:10:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:10:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:10:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:10:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:10:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:10:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:10:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:10:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:10:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:10:27.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:10:27.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:10:27.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:10:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:10:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:10:27.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497535 6.71 MB/sec execute 208 sec latency 4980.381 ms 2024-09-10T21:10:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:27 smithi174 ceph-mon[35635]: pgmap v540: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 320 KiB/s wr, 86 op/s 2024-09-10T21:10:28.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:27 smithi033 ceph-mon[30860]: pgmap v540: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 320 KiB/s wr, 86 op/s 2024-09-10T21:10:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:27 smithi104 ceph-mon[35325]: pgmap v540: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 320 KiB/s wr, 86 op/s 2024-09-10T21:10:28.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 497535 6.68 MB/sec execute 209 sec latency 1617.621 ms 2024-09-10T21:10:29.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 498853 6.68 MB/sec execute 210 sec latency 1710.564 ms 2024-09-10T21:10:30.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:29 smithi174 ceph-mon[35635]: pgmap v541: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 323 KiB/s wr, 90 op/s 2024-09-10T21:10:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:29 smithi033 ceph-mon[30860]: pgmap v541: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 323 KiB/s wr, 90 op/s 2024-09-10T21:10:30.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:29 smithi104 ceph-mon[35325]: pgmap v541: 97 pgs: 97 active+clean; 193 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 323 KiB/s wr, 90 op/s 2024-09-10T21:10:30.732 INFO:tasks.workunit.client.0.smithi033.stdout: 1 501061 6.71 MB/sec execute 211 sec latency 228.792 ms 2024-09-10T21:10:31.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 501061 6.68 MB/sec execute 212 sec latency 1228.899 ms 2024-09-10T21:10:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:31 smithi174 ceph-mon[35635]: pgmap v542: 97 pgs: 97 active+clean; 194 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 539 KiB/s wr, 74 op/s 2024-09-10T21:10:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:31 smithi033 ceph-mon[30860]: pgmap v542: 97 pgs: 97 active+clean; 194 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 539 KiB/s wr, 74 op/s 2024-09-10T21:10:32.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:31 smithi104 ceph-mon[35325]: pgmap v542: 97 pgs: 97 active+clean; 194 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 539 KiB/s wr, 74 op/s 2024-09-10T21:10:32.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 501744 6.65 MB/sec execute 213 sec latency 1598.576 ms 2024-09-10T21:10:33.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 504486 6.70 MB/sec execute 214 sec latency 15.249 ms 2024-09-10T21:10:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:33 smithi174 ceph-mon[35635]: pgmap v543: 97 pgs: 97 active+clean; 195 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 677 KiB/s wr, 95 op/s 2024-09-10T21:10:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:33 smithi033 ceph-mon[30860]: pgmap v543: 97 pgs: 97 active+clean; 195 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 677 KiB/s wr, 95 op/s 2024-09-10T21:10:34.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:33 smithi104 ceph-mon[35325]: pgmap v543: 97 pgs: 97 active+clean; 195 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 677 KiB/s wr, 95 op/s 2024-09-10T21:10:34.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 506022 6.70 MB/sec execute 215 sec latency 16.057 ms 2024-09-10T21:10:35.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508150 6.73 MB/sec execute 216 sec latency 255.083 ms 2024-09-10T21:10:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:35 smithi174 ceph-mon[35635]: pgmap v544: 97 pgs: 97 active+clean; 195 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-10T21:10:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:35 smithi033 ceph-mon[30860]: pgmap v544: 97 pgs: 97 active+clean; 195 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-10T21:10:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:35 smithi104 ceph-mon[35325]: pgmap v544: 97 pgs: 97 active+clean; 195 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.1 MiB/s wr, 116 op/s 2024-09-10T21:10:36.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508150 6.70 MB/sec execute 217 sec latency 1255.154 ms 2024-09-10T21:10:37.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508172 6.67 MB/sec execute 218 sec latency 1620.827 ms 2024-09-10T21:10:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:37 smithi174 ceph-mon[35635]: pgmap v545: 97 pgs: 97 active+clean; 196 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T21:10:38.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:37 smithi033 ceph-mon[30860]: pgmap v545: 97 pgs: 97 active+clean; 196 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T21:10:38.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:37 smithi104 ceph-mon[35325]: pgmap v545: 97 pgs: 97 active+clean; 196 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.1 MiB/s wr, 104 op/s 2024-09-10T21:10:38.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508172 6.64 MB/sec execute 219 sec latency 1618.524 ms 2024-09-10T21:10:39.733 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508172 6.61 MB/sec execute 220 sec latency 2618.628 ms 2024-09-10T21:10:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:39 smithi174 ceph-mon[35635]: pgmap v546: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-09-10T21:10:40.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:39 smithi033 ceph-mon[30860]: pgmap v546: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-09-10T21:10:40.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:39 smithi104 ceph-mon[35325]: pgmap v546: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-09-10T21:10:40.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508172 6.58 MB/sec execute 221 sec latency 3618.993 ms 2024-09-10T21:10:41.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508172 6.55 MB/sec execute 222 sec latency 4619.100 ms 2024-09-10T21:10:42.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:41 smithi174 ceph-mon[35635]: pgmap v547: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-09-10T21:10:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:41 smithi033 ceph-mon[30860]: pgmap v547: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-09-10T21:10:42.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:41 smithi104 ceph-mon[35325]: pgmap v547: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-09-10T21:10:42.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508191 6.52 MB/sec execute 223 sec latency 4986.263 ms 2024-09-10T21:10:43.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508191 6.49 MB/sec execute 224 sec latency 1617.630 ms 2024-09-10T21:10:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:43 smithi174 ceph-mon[35635]: pgmap v548: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-09-10T21:10:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:43 smithi033 ceph-mon[30860]: pgmap v548: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-09-10T21:10:44.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:43 smithi104 ceph-mon[35325]: pgmap v548: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 152 op/s 2024-09-10T21:10:44.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508191 6.46 MB/sec execute 225 sec latency 2617.749 ms 2024-09-10T21:10:45.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508191 6.43 MB/sec execute 226 sec latency 3617.873 ms 2024-09-10T21:10:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:45 smithi033 ceph-mon[30860]: pgmap v549: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 906 KiB/s wr, 128 op/s 2024-09-10T21:10:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:45 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:46.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:45 smithi104 ceph-mon[35325]: pgmap v549: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 906 KiB/s wr, 128 op/s 2024-09-10T21:10:46.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:45 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:46.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:45 smithi174 ceph-mon[35635]: pgmap v549: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 906 KiB/s wr, 128 op/s 2024-09-10T21:10:46.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:45 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:46.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508191 6.40 MB/sec execute 227 sec latency 4617.972 ms 2024-09-10T21:10:47.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 508916 6.39 MB/sec execute 228 sec latency 4987.639 ms 2024-09-10T21:10:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:47 smithi033 ceph-mon[30860]: pgmap v550: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 343 KiB/s wr, 82 op/s 2024-09-10T21:10:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:47 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:48.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:47 smithi104 ceph-mon[35325]: pgmap v550: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 343 KiB/s wr, 82 op/s 2024-09-10T21:10:48.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:47 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:47 smithi174 ceph-mon[35635]: pgmap v550: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 343 KiB/s wr, 82 op/s 2024-09-10T21:10:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:47 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:48.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 511667 6.43 MB/sec execute 229 sec latency 33.708 ms 2024-09-10T21:10:49.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 511667 6.40 MB/sec execute 230 sec latency 1033.805 ms 2024-09-10T21:10:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:49 smithi033 ceph-mon[30860]: pgmap v551: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 313 KiB/s wr, 82 op/s 2024-09-10T21:10:50.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:49 smithi104 ceph-mon[35325]: pgmap v551: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 313 KiB/s wr, 82 op/s 2024-09-10T21:10:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:49 smithi174 ceph-mon[35635]: pgmap v551: 97 pgs: 97 active+clean; 197 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 313 KiB/s wr, 82 op/s 2024-09-10T21:10:50.656 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:10:50.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 511667 6.38 MB/sec execute 231 sec latency 2033.870 ms 2024-09-10T21:10:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:51.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 -- 172.21.15.33:0/733032870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630101c90 msgr2=0x7f5630102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 --2- 172.21.15.33:0/733032870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630101c90 0x7f5630102070 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f562c009900 tx=0x7f562c02a410 comp rx=0 tx=0).stop 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 -- 172.21.15.33:0/733032870 shutdown_connections 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 --2- 172.21.15.33:0/733032870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5630108020 0x7f563010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 --2- 172.21.15.33:0/733032870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56301076a0 0x7f5630107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 --2- 172.21.15.33:0/733032870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630101c90 0x7f5630102070 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 -- 172.21.15.33:0/733032870 >> 172.21.15.33:0/733032870 conn(0x7f56300fd700 msgr2=0x7f56300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 -- 172.21.15.33:0/733032870 shutdown_connections 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 -- 172.21.15.33:0/733032870 wait complete. 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.412+0000 7f5637653640 1 Processor -- start 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5637653640 1 -- start start 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5637653640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5630101c90 0x7f56301982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5637653640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56301076a0 0x7f5630198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5637653640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630108020 0x7f563019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5637653640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5630111d20 con 0x7f5630108020 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5637653640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5630111ba0 con 0x7f56301076a0 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5637653640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5630111ea0 con 0x7f5630101c90 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5635bf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630108020 0x7f563019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5635bf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630108020 0x7f563019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33384/0 (socket says 172.21.15.33:33384) 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5635bf0640 1 -- 172.21.15.33:0/2968397874 learned_addr learned my addr 172.21.15.33:0/2968397874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f5634bee640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56301076a0 0x7f5630198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.413+0000 7f56353ef640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5630101c90 0x7f56301982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5635bf0640 1 -- 172.21.15.33:0/2968397874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5630101c90 msgr2=0x7f56301982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5635bf0640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5630101c90 0x7f56301982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5635bf0640 1 -- 172.21.15.33:0/2968397874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56301076a0 msgr2=0x7f5630198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5635bf0640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56301076a0 0x7f5630198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5635bf0640 1 -- 172.21.15.33:0/2968397874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563019e380 con 0x7f5630108020 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5634bee640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56301076a0 0x7f5630198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5635bf0640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630108020 0x7f563019dbb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f562800db90 tx=0x7f562800dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f56267fc640 1 -- 172.21.15.33:0/2968397874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562800e980 con 0x7f5630108020 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563019e610 con 0x7f5630108020 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f56267fc640 1 -- 172.21.15.33:0/2968397874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5628009580 con 0x7f5630108020 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563006ab30 con 0x7f5630108020 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.414+0000 7f56267fc640 1 -- 172.21.15.33:0/2968397874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5628016650 con 0x7f5630108020 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.417+0000 7f5639a28740 1 Processor -- start 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.417+0000 7f56267fc640 1 -- 172.21.15.33:0/2968397874 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f56280167f0 con 0x7f5630108020 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.417+0000 7f5639a28740 1 -- start start 2024-09-10T21:10:51.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5639a28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5639a28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa08dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5639a28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa090530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5639a28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5572f9ff3e50 con 0x5572fa08e140 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5639a28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5572f9fe3280 con 0x5572fa07dda0 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5639a28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5572f9fe7090 con 0x5572fa07d480 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5634bee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa090530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa08dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa08dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:60342/0 (socket says 172.21.15.33:60342) 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5634bee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa090530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33386/0 (socket says 172.21.15.33:33386) 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 -- 172.21.15.33:0/1460920037 learned_addr learned my addr 172.21.15.33:0/1460920037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5634bee640 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 -- 172.21.15.33:0/1460920037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 msgr2=0x5572fa07d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 -- 172.21.15.33:0/1460920037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 msgr2=0x5572fa090530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa090530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56353ef640 1 -- 172.21.15.33:0/1460920037 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572f9d41650 con 0x5572fa07dda0 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f5634bee640 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa090530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.418+0000 7f56267fc640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f561c0b5360 0x7f561c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.419+0000 7f5634bee640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f561c0b5360 0x7f561c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.419+0000 7f56353ef640 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa08dc00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f562c02a930 tx=0x7f562c031cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8e225b80aaec99c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.419+0000 7f56267fc640 1 -- 172.21.15.33:0/2968397874 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f56280e70e0 con 0x7f5630108020 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.419+0000 7f561b7fe640 1 -- 172.21.15.33:0/1460920037 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c03f070 con 0x5572fa07dda0 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.419+0000 7f561b7fe640 1 -- 172.21.15.33:0/1460920037 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c0316a0 con 0x5572fa07dda0 2024-09-10T21:10:51.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.419+0000 7f561b7fe640 1 -- 172.21.15.33:0/1460920037 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c035070 con 0x5572fa07dda0 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 -- 172.21.15.33:0/1460920037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 msgr2=0x5572fa08dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5634bee640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f561c0b5360 0x7f561c0b7820 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5572fa07d430 tx=0x7f562000a2b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa08dc00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f562c02a930 tx=0x7f562c031cd0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 -- 172.21.15.33:0/1460920037 shutdown_connections 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa090530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa08dc00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 --2- 172.21.15.33:0/1460920037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 -- 172.21.15.33:0/1460920037 >> 172.21.15.33:0/1460920037 conn(0x5572fa07b2b0 msgr2=0x5572fa07c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:51.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 -- 172.21.15.33:0/1460920037 shutdown_connections 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 -- 172.21.15.33:0/1460920037 wait complete. 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 -- start start 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.420+0000 7f5639a28740 1 Processor -- start 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5639a28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07d480 0x5572fa08d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5639a28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07dda0 0x5572fa08ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5639a28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa08e140 0x5572fa08b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5639a28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5572f9e44700 con 0x5572fa07dda0 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5639a28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5572f9f7e110 con 0x5572fa07d480 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5639a28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5572fa090d90 con 0x5572fa08e140 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07dda0 0x5572fa08ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5639a28740 10 client.0 _ll_register_callbacks cb 0x5572f9f97990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07dda0 0x5572fa08ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33398/0 (socket says 172.21.15.33:33398) 2024-09-10T21:10:51.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 -- 172.21.15.33:0/3142205342 learned_addr learned my addr 172.21.15.33:0/3142205342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f56353ef640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07d480 0x5572fa08d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5635bf0640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa08e140 0x5572fa08b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 -- 172.21.15.33:0/3142205342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa08e140 msgr2=0x5572fa08b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa08e140 0x5572fa08b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 -- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07d480 msgr2=0x5572fa08d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07d480 0x5572fa08d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 -- 172.21.15.33:0/3142205342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572f9fabc40 con 0x5572fa07dda0 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5635bf0640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa08e140 0x5572fa08b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5634bee640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07dda0 0x5572fa08ada0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f562001abf0 tx=0x7f56200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2030445480eba23e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.421+0000 7f5618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5618ff9640 1 -- 172.21.15.33:0/3142205342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620028060 con 0x5572fa07dda0 2024-09-10T21:10:51.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5618ff9640 1 -- 172.21.15.33:0/3142205342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f562000f050 con 0x5572fa07dda0 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f562c0316a0 con 0x5572fa07dda0 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5618ff9640 1 -- 172.21.15.33:0/3142205342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f5620015a60 con 0x5572fa07dda0 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 10 client.16593 fetch_fsmap learned FSMap version 51 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5572fa09c6a0 con 0x5572fa07dda0 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5618ff9640 1 -- 172.21.15.33:0/3142205342 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f562c0316a0 con 0x5572fa07dda0 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5618ff9640 1 -- 172.21.15.33:0/3142205342 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f5620015040 con 0x5572fa07dda0 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 10 client.16593 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 10 client.16593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 20 client.16593 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa09dbd0 0x5572fa0a0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 4 client.16593 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572f9ff5da0 con 0x5572fa09dbd0 2024-09-10T21:10:51.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.422+0000 7f56353ef640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa09dbd0 0x5572fa0a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.423+0000 7f56353ef640 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa09dbd0 0x5572fa0a0060 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc32165d47efc1e2 server_cookie=59a2a6a2244ab504 in_seq=0 out_seq=0 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.423+0000 7f5618ff9640 10 client.16593 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5618ff9640 1 -- 172.21.15.33:0/3142205342 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5572f9ff5da0 con 0x5572fa09dbd0 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5618ff9640 10 client.16593 handle_command_reply: tid=0 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 client.16593 shutdown 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07dda0 msgr2=0x5572fa08ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07dda0 0x5572fa08ada0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f562001abf0 tx=0x7f56200046c0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 shutdown_connections 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa09dbd0 0x5572fa0a0060 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa08e140 0x5572fa08b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07dda0 0x5572fa08ada0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 --2- 172.21.15.33:0/3142205342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07d480 0x5572fa08d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 >> 172.21.15.33:0/3142205342 conn(0x5572fa07b2b0 msgr2=0x5572fa07a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:51.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5618ff9640 0 client.16593 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:51.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 shutdown_connections 2024-09-10T21:10:51.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 1 -- 172.21.15.33:0/3142205342 wait complete. 2024-09-10T21:10:51.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.424+0000 7f5639a28740 20 client.16593 trim_cache size 0 max 16384 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.442+0000 7f5639a28740 1 Processor -- start 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.442+0000 7f5639a28740 1 -- start start 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5639a28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5639a28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa0a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5639a28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa0a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5639a28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f561400e4c0 con 0x5572fa08e140 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5639a28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5614010150 con 0x5572fa07dda0 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5639a28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f561400ebc0 con 0x5572fa07d480 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa0a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa0a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33402/0 (socket says 172.21.15.33:33402) 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 -- 172.21.15.33:0/1145051232 learned_addr learned my addr 172.21.15.33:0/1145051232 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:51.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f56353ef640 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5634bee640 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa0a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 -- 172.21.15.33:0/1145051232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 msgr2=0x5572fa07ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 -- 172.21.15.33:0/1145051232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 msgr2=0x5572fa0a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa0a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 -- 172.21.15.33:0/1145051232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572fa0892a0 con 0x5572fa08e140 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.443+0000 7f5635bf0640 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa0a5f00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f562800bc10 tx=0x7f5628004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5c2e1d084b8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f561affd640 1 -- 172.21.15.33:0/1145051232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56280adfa0 con 0x5572fa08e140 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f561affd640 1 -- 172.21.15.33:0/1145051232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56280ad8a0 con 0x5572fa08e140 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f561affd640 1 -- 172.21.15.33:0/1145051232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562800eb20 con 0x5572fa08e140 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f5639a28740 1 -- 172.21.15.33:0/1145051232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 msgr2=0x5572fa0a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f5639a28740 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa0a5f00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f562800bc10 tx=0x7f5628004a40 comp rx=0 tx=0).stop 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f5639a28740 1 -- 172.21.15.33:0/1145051232 shutdown_connections 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f5639a28740 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa08e140 0x5572fa0a5f00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f5639a28740 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa07dda0 0x5572fa0a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f5639a28740 1 --2- 172.21.15.33:0/1145051232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07d480 0x5572fa07ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.444+0000 7f5639a28740 1 -- 172.21.15.33:0/1145051232 >> 172.21.15.33:0/1145051232 conn(0x5572fa07b2b0 msgr2=0x5572fa07c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 -- 172.21.15.33:0/1145051232 shutdown_connections 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 -- 172.21.15.33:0/1145051232 wait complete. 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 -- start start 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 Processor -- start 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07d480 0x5572fa0960b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07dda0 0x5572fa0965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa08e140 0x5572fa093080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f561400f9c0 con 0x5572fa07d480 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f561400f2c0 con 0x5572fa08e140 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f561400c480 con 0x5572fa07dda0 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.445+0000 7f5639a28740 10 client.0 _ll_register_callbacks cb 0x7f562000a5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07d480 0x5572fa0960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07d480 0x5572fa0960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33408/0 (socket says 172.21.15.33:33408) 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 -- 172.21.15.33:0/797655827 learned_addr learned my addr 172.21.15.33:0/797655827 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5634bee640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07dda0 0x5572fa0965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5635bf0640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa08e140 0x5572fa093080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 -- 172.21.15.33:0/797655827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07dda0 msgr2=0x5572fa0965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07dda0 0x5572fa0965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 -- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa08e140 msgr2=0x5572fa093080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:51.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa08e140 0x5572fa093080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 -- 172.21.15.33:0/797655827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572fa096d80 con 0x5572fa07d480 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5634bee640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07dda0 0x5572fa0965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5635bf0640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa08e140 0x5572fa093080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f56353ef640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07d480 0x5572fa0960b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f562c009900 tx=0x7f562c0051c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4b70af667c393c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5618ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5618ff9640 1 -- 172.21.15.33:0/797655827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c03f040 con 0x5572fa07d480 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5618ff9640 1 -- 172.21.15.33:0/797655827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f562c0052d0 con 0x5572fa07d480 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.446+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56280ad8a0 con 0x5572fa07d480 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5618ff9640 1 -- 172.21.15.33:0/797655827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f562c035040 con 0x5572fa07d480 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 10 client.16599 fetch_fsmap learned FSMap version 51 2024-09-10T21:10:51.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x5572fa10b170 con 0x5572fa07d480 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5618ff9640 1 -- 172.21.15.33:0/797655827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56280ad8a0 con 0x5572fa07d480 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5618ff9640 1 -- 172.21.15.33:0/797655827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f562c003890 con 0x5572fa07d480 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 10 client.16599 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 10 client.16599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 20 client.16599 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa10b510 0x5572fa10d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 4 client.16599 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572f9ff1850 con 0x5572fa10b510 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.447+0000 7f5634bee640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa10b510 0x5572fa10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5634bee640 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa10b510 0x5572fa10d900 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e6c7f95b0a7b8c1 server_cookie=9627e3fc5134820e in_seq=0 out_seq=0 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5618ff9640 10 client.16599 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5618ff9640 1 -- 172.21.15.33:0/797655827 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x5572f9ff1850 con 0x5572fa10b510 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5618ff9640 10 client.16599 handle_command_reply: tid=0 2024-09-10T21:10:51.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 client.16599 shutdown 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07d480 msgr2=0x5572fa0960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07d480 0x5572fa0960b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f562c009900 tx=0x7f562c0051c0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 shutdown_connections 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5572fa10b510 0x5572fa10d900 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5572fa08e140 0x5572fa093080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5618ff9640 0 client.16599 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572fa07dda0 0x5572fa0965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 --2- 172.21.15.33:0/797655827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5572fa07d480 0x5572fa0960b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 >> 172.21.15.33:0/797655827 conn(0x5572fa07b2b0 msgr2=0x5572fa08c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 shutdown_connections 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.448+0000 7f5639a28740 1 -- 172.21.15.33:0/797655827 wait complete. 2024-09-10T21:10:51.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5639a28740 20 client.16599 trim_cache size 0 max 16384 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.468 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.469 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:10:51.470 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:10:51.471 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:10:51.473 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:10:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f561c0b5360 msgr2=0x7f561c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f561c0b5360 0x7f561c0b7820 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x5572fa07d430 tx=0x7f562000a2b0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630108020 msgr2=0x7f563019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630108020 0x7f563019dbb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f562800db90 tx=0x7f562800dc80 comp rx=0 tx=0).stop 2024-09-10T21:10:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 shutdown_connections 2024-09-10T21:10:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f561c0b5360 0x7f561c0b7820 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5630108020 0x7f563019dbb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56301076a0 0x7f5630198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 --2- 172.21.15.33:0/2968397874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5630101c90 0x7f56301982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:51.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 >> 172.21.15.33:0/2968397874 conn(0x7f56300fd700 msgr2=0x7f56300ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:51.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 shutdown_connections 2024-09-10T21:10:51.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:51.449+0000 7f5637653640 1 -- 172.21.15.33:0/2968397874 wait complete. 2024-09-10T21:10:51.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 511667 6.35 MB/sec execute 232 sec latency 3033.906 ms 2024-09-10T21:10:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:51 smithi033 ceph-mon[30860]: pgmap v552: 97 pgs: 97 active+clean; 198 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 162 KiB/s wr, 50 op/s 2024-09-10T21:10:52.228 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:10:52.229 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:10:52.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:51 smithi104 ceph-mon[35325]: pgmap v552: 97 pgs: 97 active+clean; 198 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 162 KiB/s wr, 50 op/s 2024-09-10T21:10:52.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:51 smithi174 ceph-mon[35635]: pgmap v552: 97 pgs: 97 active+clean; 198 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 162 KiB/s wr, 50 op/s 2024-09-10T21:10:52.499 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:52.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 512303 6.33 MB/sec execute 233 sec latency 3399.809 ms 2024-09-10T21:10:53.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/1246340036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc10d140 msgr2=0x7fa2bc10f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:53.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/1246340036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc10d140 0x7fa2bc10f5a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009ca0 tx=0x7fa2b402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/1246340036 shutdown_connections 2024-09-10T21:10:53.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/1246340036 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc10d140 0x7fa2bc10f5a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/1246340036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2bc104600 0x7fa2bc10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/1246340036 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2bc103ce0 0x7fa2bc1040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/1246340036 >> 172.21.15.33:0/1246340036 conn(0x7fa2bc0fd700 msgr2=0x7fa2bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/1246340036 shutdown_connections 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.988+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/1246340036 wait complete. 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 Processor -- start 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 -- start start 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2bc103ce0 0x7fa2bc101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc104600 0x7fa2bc101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2bc10d140 0x7fa2bc102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc112db0 con 0x7fa2bc104600 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc112c30 con 0x7fa2bc10d140 2024-09-10T21:10:53.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2c1f1d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2bc112f30 con 0x7fa2bc103ce0 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc104600 0x7fa2bc101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc104600 0x7fa2bc101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33428/0 (socket says 172.21.15.33:33428) 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 -- 172.21.15.33:0/852899357 learned_addr learned my addr 172.21.15.33:0/852899357 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2bb7fe640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2bc103ce0 0x7fa2bc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2bbfff640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2bc10d140 0x7fa2bc102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 -- 172.21.15.33:0/852899357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2bc103ce0 msgr2=0x7fa2bc101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2bc103ce0 0x7fa2bc101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 -- 172.21.15.33:0/852899357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2bc10d140 msgr2=0x7fa2bc102410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2bc10d140 0x7fa2bc102410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2b3fff640 1 -- 172.21.15.33:0/852899357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2bc1a7680 con 0x7fa2bc104600 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.989+0000 7fa2bbfff640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2bc10d140 0x7fa2bc102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:53.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.990+0000 7fa2bb7fe640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2bc103ce0 0x7fa2bc101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.990+0000 7fa2b3fff640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc104600 0x7fa2bc101ed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9c0 tx=0x7fa2a400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.990+0000 7fa2b97fa640 1 -- 172.21.15.33:0/852899357 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400ccc0 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.990+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2bc1a7970 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.990+0000 7fa2b97fa640 1 -- 172.21.15.33:0/852899357 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2a40035f0 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.990+0000 7fa2b97fa640 1 -- 172.21.15.33:0/852899357 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400ce60 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.990+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2bc1a7eb0 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.991+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2bc1040c0 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.993+0000 7fa2b97fa640 1 -- 172.21.15.33:0/852899357 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa2a4004000 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.993+0000 7fa2b97fa640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2a80b52e0 0x7fa2a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.994+0000 7fa2bb7fe640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2a80b52e0 0x7fa2a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.994+0000 7fa2b97fa640 1 -- 172.21.15.33:0/852899357 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa2a40e5fc0 con 0x7fa2bc104600 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.994+0000 7fa2bb7fe640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2a80b52e0 0x7fa2a80b77a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009fc0 tx=0x7fa2ac005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:53.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:52.997+0000 7fa2b97fa640 1 -- 172.21.15.33:0/852899357 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa2a40acf10 con 0x7fa2bc104600 2024-09-10T21:10:53.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.230+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2bc02e5c0 con 0x7fa2bc104600 2024-09-10T21:10:53.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.231+0000 7fa2b97fa640 1 -- 172.21.15.33:0/852899357 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+11984 (secure 0 0 0) 0x7fa2a40ac810 con 0x7fa2bc104600 2024-09-10T21:10:53.235 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:10:53.235 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":51,"btime":"2024-09-10T21:10:46:893994+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:10:46.759422+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:10:53.236 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 51 2024-09-10T21:10:53.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.233+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2a80b52e0 msgr2=0x7fa2a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:53.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.233+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2a80b52e0 0x7fa2a80b77a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac009fc0 tx=0x7fa2ac005f90 comp rx=0 tx=0).stop 2024-09-10T21:10:53.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.233+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc104600 msgr2=0x7fa2bc101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:53.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.233+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc104600 0x7fa2bc101ed0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fa2a400e9c0 tx=0x7fa2a400ee80 comp rx=0 tx=0).stop 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.233+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 shutdown_connections 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.234+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2a80b52e0 0x7fa2a80b77a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.234+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2bc10d140 0x7fa2bc102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.234+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2bc104600 0x7fa2bc101ed0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.234+0000 7fa2c1f1d640 1 --2- 172.21.15.33:0/852899357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2bc103ce0 0x7fa2bc101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.234+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 >> 172.21.15.33:0/852899357 conn(0x7fa2bc0fd700 msgr2=0x7fa2bc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.234+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 shutdown_connections 2024-09-10T21:10:53.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:53.234+0000 7fa2c1f1d640 1 -- 172.21.15.33:0/852899357 wait complete. 2024-09-10T21:10:53.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 514957 6.36 MB/sec execute 234 sec latency 17.931 ms 2024-09-10T21:10:53.957 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:10:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:53 smithi033 ceph-mon[30860]: pgmap v553: 97 pgs: 97 active+clean; 199 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 179 KiB/s wr, 50 op/s 2024-09-10T21:10:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:53 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/852899357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:10:54.216 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:54.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:53 smithi104 ceph-mon[35325]: pgmap v553: 97 pgs: 97 active+clean; 199 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 179 KiB/s wr, 50 op/s 2024-09-10T21:10:54.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:53 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/852899357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:10:54.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:53 smithi174 ceph-mon[35635]: pgmap v553: 97 pgs: 97 active+clean; 199 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 179 KiB/s wr, 50 op/s 2024-09-10T21:10:54.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:53 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/852899357' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:10:54.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 -- 172.21.15.33:0/1356954534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727410d130 msgr2=0x7f727410f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 --2- 172.21.15.33:0/1356954534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727410d130 0x7f727410f590 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f726c009900 tx=0x7f726c02a410 comp rx=0 tx=0).stop 2024-09-10T21:10:54.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 -- 172.21.15.33:0/1356954534 shutdown_connections 2024-09-10T21:10:54.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 --2- 172.21.15.33:0/1356954534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f727410d130 0x7f727410f590 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 --2- 172.21.15.33:0/1356954534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72741045f0 0x7f727410cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 --2- 172.21.15.33:0/1356954534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7274103cd0 0x7f72741040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 -- 172.21.15.33:0/1356954534 >> 172.21.15.33:0/1356954534 conn(0x7f72740fd770 msgr2=0x7f72740ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 -- 172.21.15.33:0/1356954534 shutdown_connections 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 -- 172.21.15.33:0/1356954534 wait complete. 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.713+0000 7f72793e4640 1 Processor -- start 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72793e4640 1 -- start start 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72793e4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7274103cd0 0x7f7274101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72793e4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72741045f0 0x7f7274101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72793e4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727410d130 0x7f7274102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72793e4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7274112da0 con 0x7f7274103cd0 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72793e4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7274112c20 con 0x7f72741045f0 2024-09-10T21:10:54.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72793e4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7274112f20 con 0x7f727410d130 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7274103cd0 0x7f7274101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72727fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72741045f0 0x7f7274101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7274103cd0 0x7f7274101980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33448/0 (socket says 172.21.15.33:33448) 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f7272ffd640 1 -- 172.21.15.33:0/2386876218 learned_addr learned my addr 172.21.15.33:0/2386876218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72737fe640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727410d130 0x7f7274102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72727fc640 1 -- 172.21.15.33:0/2386876218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727410d130 msgr2=0x7f7274102400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72727fc640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727410d130 0x7f7274102400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72727fc640 1 -- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7274103cd0 msgr2=0x7f7274101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72727fc640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7274103cd0 0x7f7274101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72727fc640 1 -- 172.21.15.33:0/2386876218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72741a7650 con 0x7f72741045f0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f7272ffd640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7274103cd0 0x7f7274101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.714+0000 7f72727fc640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72741045f0 0x7f7274101ec0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e770 tx=0x7f725c00ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.715+0000 7f726bfff640 1 -- 172.21.15.33:0/2386876218 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c00ccc0 con 0x7f72741045f0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.715+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741a7940 con 0x7f72741045f0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.715+0000 7f726bfff640 1 -- 172.21.15.33:0/2386876218 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f725c00ee80 con 0x7f72741045f0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.715+0000 7f726bfff640 1 -- 172.21.15.33:0/2386876218 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c003e40 con 0x7f72741045f0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.715+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72741a7ed0 con 0x7f72741045f0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.717+0000 7f727b7b9740 1 Processor -- start 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.717+0000 7f727b7b9740 1 -- start start 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.717+0000 7f726bfff640 1 -- 172.21.15.33:0/2386876218 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f725c004000 con 0x7f72741045f0 2024-09-10T21:10:54.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.717+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b61a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b626840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b629170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f727b7b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55846b590ea0 con 0x55846b61adf0 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f727b7b9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55846b5802d0 con 0x55846b61a4d0 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f727b7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55846b5840e0 con 0x55846b626d80 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b626840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b626840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33454/0 (socket says 172.21.15.33:33454) 2024-09-10T21:10:54.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 -- 172.21.15.33:0/888001200 learned_addr learned my addr 172.21.15.33:0/888001200 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f72737fe640 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b61a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f72727fc640 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b629170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 -- 172.21.15.33:0/888001200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 msgr2=0x55846b629170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b629170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 -- 172.21.15.33:0/888001200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 msgr2=0x55846b61a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b61a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 -- 172.21.15.33:0/888001200 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846b2de650 con 0x55846b61adf0 2024-09-10T21:10:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f72727fc640 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b629170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:10:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f7272ffd640 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b626840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f72640118b0 tx=0x7f7264011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff635cbfab5c310 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f72697fa640 1 -- 172.21.15.33:0/888001200 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726400fc00 con 0x55846b61adf0 2024-09-10T21:10:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f726bfff640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f72600b52e0 0x7f72600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f72697fa640 1 -- 172.21.15.33:0/888001200 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264016070 con 0x55846b61adf0 2024-09-10T21:10:54.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.718+0000 7f72697fa640 1 -- 172.21.15.33:0/888001200 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264012690 con 0x55846b61adf0 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f72737fe640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f72600b52e0 0x7f72600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f726bfff640 1 -- 172.21.15.33:0/2386876218 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f725c0e5e20 con 0x7f72741045f0 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 -- 172.21.15.33:0/888001200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 msgr2=0x55846b626840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f72737fe640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f72600b52e0 0x7f72600b77a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7274103a40 tx=0x7f726c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b626840 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f72640118b0 tx=0x7f7264011d70 comp rx=0 tx=0).stop 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 -- 172.21.15.33:0/888001200 shutdown_connections 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b629170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b626840 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 --2- 172.21.15.33:0/888001200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b61a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 -- 172.21.15.33:0/888001200 >> 172.21.15.33:0/888001200 conn(0x55846b618300 msgr2=0x55846b6194a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 -- 172.21.15.33:0/888001200 shutdown_connections 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.719+0000 7f727b7b9740 1 -- 172.21.15.33:0/888001200 wait complete. 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 -- start start 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 Processor -- start 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61a4d0 0x55846b6266a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b623af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b626d80 0x55846b624120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55846b3e1700 con 0x55846b61adf0 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55846b51b160 con 0x55846b626d80 2024-09-10T21:10:54.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55846b629a00 con 0x55846b61a4d0 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f727b7b9740 10 client.0 _ll_register_callbacks cb 0x55846b5349e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f72727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b623af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f72727fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b623af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33464/0 (socket says 172.21.15.33:33464) 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f72727fc640 1 -- 172.21.15.33:0/2795903903 learned_addr learned my addr 172.21.15.33:0/2795903903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f7272ffd640 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61a4d0 0x55846b6266a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f72737fe640 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b626d80 0x55846b624120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.720+0000 7f72727fc640 1 -- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61a4d0 msgr2=0x55846b6266a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f72727fc640 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61a4d0 0x55846b6266a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f72727fc640 1 -- 172.21.15.33:0/2795903903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b626d80 msgr2=0x55846b624120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f72727fc640 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b626d80 0x55846b624120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f72727fc640 1 -- 172.21.15.33:0/2795903903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846b548c90 con 0x55846b61adf0 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f72727fc640 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b623af0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f725c0a9810 tx=0x7f725c0a8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defd2166fd46d1fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f725a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f725a7fc640 1 -- 172.21.15.33:0/2795903903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c0abd10 con 0x55846b61adf0 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f725a7fc640 1 -- 172.21.15.33:0/2795903903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f725c0aba00 con 0x55846b61adf0 2024-09-10T21:10:54.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7264016070 con 0x55846b61adf0 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f725a7fc640 1 -- 172.21.15.33:0/2795903903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f725c0035f0 con 0x55846b61adf0 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f727b7b9740 10 client.16617 fetch_fsmap learned FSMap version 51 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55846b6353b0 con 0x55846b61adf0 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f725a7fc640 1 -- 172.21.15.33:0/2795903903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264016070 con 0x55846b61adf0 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f725a7fc640 1 -- 172.21.15.33:0/2795903903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f725c0d31a0 con 0x55846b61adf0 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f727b7b9740 10 client.16617 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f727b7b9740 10 client.16617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.721+0000 7f727b7b9740 20 client.16617 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.722+0000 7f727b7b9740 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b636870 0x55846b638c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.722+0000 7f727b7b9740 4 client.16617 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.722+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55846b592df0 con 0x55846b636870 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.722+0000 7f7272ffd640 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b636870 0x55846b638c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.722+0000 7f7272ffd640 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b636870 0x55846b638c60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cf154de543081819 server_cookie=42c3705c4efed9ac in_seq=0 out_seq=0 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.722+0000 7f725a7fc640 10 client.16617 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f725a7fc640 1 -- 172.21.15.33:0/2795903903 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55846b592df0 con 0x55846b636870 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f725a7fc640 10 client.16617 handle_command_reply: tid=0 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 client.16617 shutdown 2024-09-10T21:10:54.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 msgr2=0x55846b623af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b623af0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f725c0a9810 tx=0x7f725c0a8cb0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 shutdown_connections 2024-09-10T21:10:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b636870 0x55846b638c60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b626d80 0x55846b624120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61adf0 0x55846b623af0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f725a7fc640 0 client.16617 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:54.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 --2- 172.21.15.33:0/2795903903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61a4d0 0x55846b6266a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 >> 172.21.15.33:0/2795903903 conn(0x55846b618300 msgr2=0x55846b61dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 shutdown_connections 2024-09-10T21:10:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 1 -- 172.21.15.33:0/2795903903 wait complete. 2024-09-10T21:10:54.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.723+0000 7f727b7b9740 20 client.16617 trim_cache size 0 max 16384 2024-09-10T21:10:54.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515243 6.35 MB/sec execute 235 sec latency 887.822 ms 2024-09-10T21:10:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.737+0000 7f727b7b9740 1 Processor -- start 2024-09-10T21:10:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.737+0000 7f727b7b9740 1 -- start start 2024-09-10T21:10:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.737+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b632c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.737+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61adf0 0x55846b617310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.737+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b626d80 0x55846b628170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.737+0000 7f727b7b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72741b0630 con 0x55846b626d80 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.737+0000 7f727b7b9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f72741b22c0 con 0x55846b61a4d0 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f727b7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72741b0d30 con 0x55846b61adf0 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f72737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b626d80 0x55846b628170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f72737fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b626d80 0x55846b628170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33478/0 (socket says 172.21.15.33:33478) 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f72737fe640 1 -- 172.21.15.33:0/3391933035 learned_addr learned my addr 172.21.15.33:0/3391933035 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7272ffd640 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b632c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f72727fc640 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61adf0 0x55846b617310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7272ffd640 1 -- 172.21.15.33:0/3391933035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61adf0 msgr2=0x55846b617310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7272ffd640 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61adf0 0x55846b617310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7272ffd640 1 -- 172.21.15.33:0/3391933035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b626d80 msgr2=0x55846b628170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7272ffd640 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b626d80 0x55846b628170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7272ffd640 1 -- 172.21.15.33:0/3391933035 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55846b6238a0 con 0x55846b61a4d0 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7272ffd640 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b632c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7264011880 tx=0x7f726401bd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85d1e2ca95bb82d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7268ff9640 1 -- 172.21.15.33:0/3391933035 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726400fc80 con 0x55846b61a4d0 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7268ff9640 1 -- 172.21.15.33:0/3391933035 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264016070 con 0x55846b61a4d0 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.738+0000 7f7268ff9640 1 -- 172.21.15.33:0/3391933035 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726401b970 con 0x55846b61a4d0 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 -- 172.21.15.33:0/3391933035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 msgr2=0x55846b632c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b632c30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7264011880 tx=0x7f726401bd20 comp rx=0 tx=0).stop 2024-09-10T21:10:54.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 -- 172.21.15.33:0/3391933035 shutdown_connections 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b626d80 0x55846b628170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b61adf0 0x55846b617310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3391933035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61a4d0 0x55846b632c30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 -- 172.21.15.33:0/3391933035 >> 172.21.15.33:0/3391933035 conn(0x55846b618300 msgr2=0x55846b616c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 -- 172.21.15.33:0/3391933035 shutdown_connections 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.739+0000 7f727b7b9740 1 -- 172.21.15.33:0/3391933035 wait complete. 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 -- start start 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 Processor -- start 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61a4d0 0x55846b62f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61adf0 0x55846b62cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b62d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f72741b1b30 con 0x55846b61a4d0 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f72741b1430 con 0x55846b61adf0 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72741ae5f0 con 0x55846b626d80 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f727b7b9740 10 client.0 _ll_register_callbacks cb 0x7f725c0037e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61a4d0 0x55846b62f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f7272ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61a4d0 0x55846b62f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33484/0 (socket says 172.21.15.33:33484) 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f7272ffd640 1 -- 172.21.15.33:0/3763960376 learned_addr learned my addr 172.21.15.33:0/3763960376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.740+0000 7f72727fc640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61adf0 0x55846b62cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f72737fe640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b62d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f7272ffd640 1 -- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 msgr2=0x55846b62d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f7272ffd640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b62d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f7272ffd640 1 -- 172.21.15.33:0/3763960376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61adf0 msgr2=0x55846b62cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f7272ffd640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61adf0 0x55846b62cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f7272ffd640 1 -- 172.21.15.33:0/3763960376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55846b62d890 con 0x55846b61a4d0 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f72737fe640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b62d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f7272ffd640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61a4d0 0x55846b62f280 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f726400f730 tx=0x7f726400fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3db116c577cb8c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f725a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f725a7fc640 1 -- 172.21.15.33:0/3763960376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726401b8c0 con 0x55846b61a4d0 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f725a7fc640 1 -- 172.21.15.33:0/3763960376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f726401ba60 con 0x55846b61a4d0 2024-09-10T21:10:54.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7264016070 con 0x55846b61a4d0 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f725a7fc640 1 -- 172.21.15.33:0/3763960376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) ==== 24+0+0 (secure 0 0 0) 0x7f7264003970 con 0x55846b61a4d0 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f727b7b9740 10 client.16623 fetch_fsmap learned FSMap version 51 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) -- 0x55846b6a45c0 con 0x55846b61a4d0 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.741+0000 7f725a7fc640 1 -- 172.21.15.33:0/3763960376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264016070 con 0x55846b61a4d0 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f725a7fc640 1 -- 172.21.15.33:0/3763960376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 51) ==== 8394+0+0 (secure 0 0 0) 0x7f726400f040 con 0x55846b61a4d0 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f727b7b9740 10 client.16623 fetch_fsmap finished waiting for FSMap version 51 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f727b7b9740 10 client.16623 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f727b7b9740 20 client.16623 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b6a4af0 0x55846b6a6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f727b7b9740 4 client.16623 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:54.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55846b58e8a0 con 0x55846b6a4af0 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f72727fc640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b6a4af0 0x55846b6a6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f72727fc640 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b6a4af0 0x55846b6a6f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63b7b89f5696697d server_cookie=663502b52bb71867 in_seq=0 out_seq=0 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.742+0000 7f725a7fc640 10 client.16623 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f725a7fc640 1 -- 172.21.15.33:0/3763960376 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55846b58e8a0 con 0x55846b6a4af0 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f725a7fc640 10 client.16623 handle_command_reply: tid=0 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 client.16623 shutdown 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61a4d0 msgr2=0x55846b62f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61a4d0 0x55846b62f280 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f726400f730 tx=0x7f726400fcb0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 shutdown_connections 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55846b6a4af0 0x55846b6a6f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55846b626d80 0x55846b62d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55846b61adf0 0x55846b62cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f725a7fc640 0 client.16623 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 --2- 172.21.15.33:0/3763960376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55846b61a4d0 0x55846b62f280 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 >> 172.21.15.33:0/3763960376 conn(0x55846b618300 msgr2=0x55846b617420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 shutdown_connections 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 1 -- 172.21.15.33:0/3763960376 wait complete. 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f727b7b9740 20 client.16623 trim_cache size 0 max 16384 2024-09-10T21:10:54.762 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.743+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f72600b52e0 msgr2=0x7f72600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f72600b52e0 0x7f72600b77a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7274103a40 tx=0x7f726c0058d0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72741045f0 msgr2=0x7f7274101ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72741045f0 0x7f7274101ec0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e770 tx=0x7f725c00ec30 comp rx=0 tx=0).stop 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 shutdown_connections 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f72600b52e0 0x7f72600b77a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727410d130 0x7f7274102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f72741045f0 0x7f7274101ec0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 --2- 172.21.15.33:0/2386876218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7274103cd0 0x7f7274101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 >> 172.21.15.33:0/2386876218 conn(0x7f72740fd770 msgr2=0x7f72740fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 shutdown_connections 2024-09-10T21:10:54.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:54.744+0000 7f72793e4640 1 -- 172.21.15.33:0/2386876218 wait complete. 2024-09-10T21:10:55.433 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:10:55.689 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:55.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515243 6.32 MB/sec execute 236 sec latency 1887.906 ms 2024-09-10T21:10:56.076 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:55 smithi033 ceph-mon[30860]: pgmap v554: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 294 KiB/s wr, 80 op/s 2024-09-10T21:10:56.076 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:55 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:56.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.155+0000 7f6a697be640 1 -- 172.21.15.33:0/847823231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64102dc0 msgr2=0x7f6a641031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.155+0000 7f6a697be640 1 --2- 172.21.15.33:0/847823231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64102dc0 0x7f6a641031a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009900 tx=0x7f6a5c02a410 comp rx=0 tx=0).stop 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.155+0000 7f6a697be640 1 -- 172.21.15.33:0/847823231 shutdown_connections 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.155+0000 7f6a697be640 1 --2- 172.21.15.33:0/847823231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a64109150 0x7f6a6410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.155+0000 7f6a697be640 1 --2- 172.21.15.33:0/847823231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a641087d0 0x7f6a64108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.155+0000 7f6a697be640 1 --2- 172.21.15.33:0/847823231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64102dc0 0x7f6a641031a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.155+0000 7f6a697be640 1 -- 172.21.15.33:0/847823231 >> 172.21.15.33:0/847823231 conn(0x7f6a640fe830 msgr2=0x7f6a64100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.156+0000 7f6a697be640 1 -- 172.21.15.33:0/847823231 shutdown_connections 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.156+0000 7f6a697be640 1 -- 172.21.15.33:0/847823231 wait complete. 2024-09-10T21:10:56.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.156+0000 7f6a697be640 1 Processor -- start 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.156+0000 7f6a697be640 1 -- start start 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a697be640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a64102dc0 0x7f6a64199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a697be640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a641087d0 0x7f6a64199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a697be640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64109150 0x7f6a6419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a697be640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64112e10 con 0x7f6a64109150 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a697be640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64112c90 con 0x7f6a641087d0 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a697be640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a64112f90 con 0x7f6a64102dc0 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64109150 0x7f6a6419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64109150 0x7f6a6419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33500/0 (socket says 172.21.15.33:33500) 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 -- 172.21.15.33:0/146320656 learned_addr learned my addr 172.21.15.33:0/146320656 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a627fc640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a641087d0 0x7f6a64199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a64102dc0 0x7f6a64199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 -- 172.21.15.33:0/146320656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a64102dc0 msgr2=0x7f6a64199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a64102dc0 0x7f6a64199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 -- 172.21.15.33:0/146320656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a641087d0 msgr2=0x7f6a64199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a641087d0 0x7f6a64199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 -- 172.21.15.33:0/146320656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6419f3d0 con 0x7f6a64109150 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.157+0000 7f6a637fe640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64109150 0x7f6a6419ec00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009fd0 tx=0x7f6a5400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.158+0000 7f6a5bfff640 1 -- 172.21.15.33:0/146320656 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54017070 con 0x7f6a64109150 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.158+0000 7f6a5bfff640 1 -- 172.21.15.33:0/146320656 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a54003e20 con 0x7f6a64109150 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.158+0000 7f6a5bfff640 1 -- 172.21.15.33:0/146320656 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54010460 con 0x7f6a64109150 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.158+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6419f6c0 con 0x7f6a64109150 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.158+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a641a5d80 con 0x7f6a64109150 2024-09-10T21:10:56.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.160+0000 7f6a6bb93740 1 Processor -- start 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.160+0000 7f6a5bfff640 1 -- 172.21.15.33:0/146320656 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6a54003970 con 0x7f6a64109150 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.160+0000 7f6a6bb93740 1 -- start start 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b0508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b050df0 0x555c1b060c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b063580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c1afc6ea0 con 0x555c1b061190 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555c1afb62d0 con 0x555c1b0504d0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c1afba0e0 con 0x555c1b050df0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b063580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b0508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b0508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:60456/0 (socket says 172.21.15.33:60456) 2024-09-10T21:10:56.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 -- 172.21.15.33:0/2283783623 learned_addr learned my addr 172.21.15.33:0/2283783623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b050df0 0x555c1b060c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 -- 172.21.15.33:0/2283783623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b050df0 msgr2=0x555c1b060c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b050df0 0x555c1b060c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 -- 172.21.15.33:0/2283783623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 msgr2=0x555c1b063580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b063580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 -- 172.21.15.33:0/2283783623 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1ad14650 con 0x555c1b0504d0 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a5bfff640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6a500b52e0 0x7f6a500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6a500b52e0 0x7f6a500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.161+0000 7f6a627fc640 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b0508b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c017760 tx=0x7f6a4c009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10401c3981345f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.162+0000 7f6a5bfff640 1 -- 172.21.15.33:0/146320656 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6a540e59c0 con 0x7f6a64109150 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.162+0000 7f6a597fa640 1 -- 172.21.15.33:0/2283783623 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c01c020 con 0x555c1b0504d0 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.162+0000 7f6a597fa640 1 -- 172.21.15.33:0/2283783623 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a4c015e10 con 0x555c1b0504d0 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.162+0000 7f6a597fa640 1 -- 172.21.15.33:0/2283783623 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c018d40 con 0x555c1b0504d0 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.162+0000 7f6a6bb93740 1 -- 172.21.15.33:0/2283783623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 msgr2=0x555c1b0508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.162+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6a500b52e0 0x7f6a500b77a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c031440 tx=0x7f6a5c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.162+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b0508b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c017760 tx=0x7f6a4c009110 comp rx=0 tx=0).stop 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- 172.21.15.33:0/2283783623 shutdown_connections 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b063580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b050df0 0x555c1b060c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/2283783623 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b0508b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- 172.21.15.33:0/2283783623 >> 172.21.15.33:0/2283783623 conn(0x555c1b04e300 msgr2=0x555c1b04f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- 172.21.15.33:0/2283783623 shutdown_connections 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- 172.21.15.33:0/2283783623 wait complete. 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- start start 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 Processor -- start 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b060ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555c1ae17700 con 0x555c1b050df0 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555c1af51160 con 0x555c1b0504d0 2024-09-10T21:10:56.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c1b063e40 con 0x555c1b061190 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.163+0000 7f6a6bb93740 10 client.0 _ll_register_callbacks cb 0x555c1af6a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33510/0 (socket says 172.21.15.33:33510) 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a62ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b060ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 -- 172.21.15.33:0/1652369821 learned_addr learned my addr 172.21.15.33:0/1652369821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a637fe640 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 -- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 msgr2=0x555c1b05e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 -- 172.21.15.33:0/1652369821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 msgr2=0x555c1b060ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b060ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 -- 172.21.15.33:0/1652369821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1af7ec90 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a627fc640 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05df10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c018960 tx=0x7f6a4c0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca17c33893df90f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/1652369821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c01c030 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/1652369821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6a4c018cc0 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a4c015e10 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/1652369821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6a4c006d80 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a6bb93740 10 client.16635 fetch_fsmap learned FSMap version 52 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.164+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555c1b06f7b0 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/1652369821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a4c015e10 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/1652369821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7f6a4c02b750 con 0x555c1b050df0 2024-09-10T21:10:56.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a6bb93740 10 client.16635 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a6bb93740 10 client.16635 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a6bb93740 20 client.16635 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b070c10 0x555c1b073000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a6bb93740 4 client.16635 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1afc8df0 con 0x555c1b070c10 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.165+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b070c10 0x555c1b073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.166+0000 7f6a4a7fc640 10 client.16635 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.166+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b070c10 0x555c1b073000 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5da72d8c1ea376c7 server_cookie=4b9f4fd561e8a862 in_seq=0 out_seq=0 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.166+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/1652369821 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555c1afc8df0 con 0x555c1b070c10 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.166+0000 7f6a4a7fc640 10 client.16635 handle_command_reply: tid=0 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.166+0000 7f6a6bb93740 1 client.16635 shutdown 2024-09-10T21:10:56.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 msgr2=0x555c1b05df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05df10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c018960 tx=0x7f6a4c0092f0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 shutdown_connections 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b070c10 0x555c1b073000 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05df10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a4a7fc640 0 client.16635 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/1652369821 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b060ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 >> 172.21.15.33:0/1652369821 conn(0x555c1b04e300 msgr2=0x555c1b053e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 shutdown_connections 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 1 -- 172.21.15.33:0/1652369821 wait complete. 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.167+0000 7f6a6bb93740 20 client.16635 trim_cache size 0 max 16384 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 Processor -- start 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 -- start start 2024-09-10T21:10:56.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b06d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a641ac1b0 con 0x555c1b050df0 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a641ade10 con 0x555c1b0504d0 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.180+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a641ac8b0 con 0x555c1b061190 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a62ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b06d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a627fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33524/0 (socket says 172.21.15.33:33524) 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a627fc640 1 -- 172.21.15.33:0/3014164825 learned_addr learned my addr 172.21.15.33:0/3014164825 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a637fe640 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a62ffd640 1 -- 172.21.15.33:0/3014164825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 msgr2=0x555c1b05ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a62ffd640 1 -- 172.21.15.33:0/3014164825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 msgr2=0x555c1b05c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a62ffd640 1 -- 172.21.15.33:0/3014164825 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1b04df70 con 0x555c1b0504d0 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a627fc640 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b06d070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009230 tx=0x7f6a5c040000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=433f86327209ac07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a58ff9640 1 -- 172.21.15.33:0/3014164825 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c035050 con 0x555c1b0504d0 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a58ff9640 1 -- 172.21.15.33:0/3014164825 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a5c054070 con 0x555c1b0504d0 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.181+0000 7f6a58ff9640 1 -- 172.21.15.33:0/3014164825 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c004cc0 con 0x555c1b0504d0 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 -- 172.21.15.33:0/3014164825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 msgr2=0x555c1b06d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b06d070 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009230 tx=0x7f6a5c040000 comp rx=0 tx=0).stop 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 -- 172.21.15.33:0/3014164825 shutdown_connections 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b061190 0x555c1b05ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b050df0 0x555c1b05c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/3014164825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b0504d0 0x555c1b06d070 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 -- 172.21.15.33:0/3014164825 >> 172.21.15.33:0/3014164825 conn(0x555c1b04e300 msgr2=0x555c1b062070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 -- 172.21.15.33:0/3014164825 shutdown_connections 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 -- 172.21.15.33:0/3014164825 wait complete. 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.182+0000 7f6a6bb93740 1 -- start start 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 1 Processor -- start 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b0504d0 0x555c1b0690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b050df0 0x555c1b069620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b064090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6a641ad680 con 0x555c1b061190 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6a641acf80 con 0x555c1b050df0 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a641aa170 con 0x555c1b0504d0 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a6bb93740 10 client.0 _ll_register_callbacks cb 0x7f6a4c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b064090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b064090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33530/0 (socket says 172.21.15.33:33530) 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 -- 172.21.15.33:0/974578955 learned_addr learned my addr 172.21.15.33:0/974578955 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b0504d0 0x555c1b0690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a627fc640 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b050df0 0x555c1b069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 -- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b0504d0 msgr2=0x555c1b0690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b0504d0 0x555c1b0690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 -- 172.21.15.33:0/974578955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b050df0 msgr2=0x555c1b069620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b050df0 0x555c1b069620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 -- 172.21.15.33:0/974578955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1b069d10 con 0x555c1b061190 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a637fe640 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b064090 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6a54005c40 tx=0x7f6a540cf330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e939f112112b30c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.183+0000 7f6a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/974578955 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54020040 con 0x555c1b061190 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a5c054070 con 0x555c1b061190 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/974578955 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6a540cdb60 con 0x555c1b061190 2024-09-10T21:10:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/974578955 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6a540ac880 con 0x555c1b061190 2024-09-10T21:10:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 10 client.16641 fetch_fsmap learned FSMap version 52 2024-09-10T21:10:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x555c1b0ddd00 con 0x555c1b061190 2024-09-10T21:10:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/974578955 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a5c054070 con 0x555c1b061190 2024-09-10T21:10:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/974578955 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7f6a540a7ad0 con 0x555c1b061190 2024-09-10T21:10:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 10 client.16641 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:10:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 10 client.16641 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 20 client.16641 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b0de430 0x555c1b0e0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 4 client.16641 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555c1afc48a0 con 0x555c1b0de430 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.184+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b0de430 0x555c1b0e0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a4a7fc640 10 client.16641 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a62ffd640 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b0de430 0x555c1b0e0850 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c5da9fe0910573d5 server_cookie=3fdff76e125bd643 in_seq=0 out_seq=0 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a4a7fc640 1 -- 172.21.15.33:0/974578955 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555c1afc48a0 con 0x555c1b0de430 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a4a7fc640 10 client.16641 handle_command_reply: tid=0 2024-09-10T21:10:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 client.16641 shutdown 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 msgr2=0x555c1b064090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b064090 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f6a54005c40 tx=0x7f6a540cf330 comp rx=0 tx=0).stop 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 shutdown_connections 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555c1b061190 0x555c1b064090 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555c1b050df0 0x555c1b069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555c1b0de430 0x555c1b0e0850 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a4a7fc640 0 client.16641 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 --2- 172.21.15.33:0/974578955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c1b0504d0 0x555c1b0690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 >> 172.21.15.33:0/974578955 conn(0x555c1b04e300 msgr2=0x555c1b05c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 shutdown_connections 2024-09-10T21:10:56.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.185+0000 7f6a6bb93740 1 -- 172.21.15.33:0/974578955 wait complete. 2024-09-10T21:10:56.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a6bb93740 20 client.16641 trim_cache size 0 max 16384 2024-09-10T21:10:56.211 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6a500b52e0 msgr2=0x7f6a500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6a500b52e0 0x7f6a500b77a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c031440 tx=0x7f6a5c005a20 comp rx=0 tx=0).stop 2024-09-10T21:10:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64109150 msgr2=0x7f6a6419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64109150 0x7f6a6419ec00 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6a54009fd0 tx=0x7f6a5400def0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 shutdown_connections 2024-09-10T21:10:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6a500b52e0 0x7f6a500b77a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a64109150 0x7f6a6419ec00 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a641087d0 0x7f6a64199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 --2- 172.21.15.33:0/146320656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a64102dc0 0x7f6a64199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 >> 172.21.15.33:0/146320656 conn(0x7f6a640fe830 msgr2=0x7f6a64100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 shutdown_connections 2024-09-10T21:10:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:56.186+0000 7f6a697be640 1 -- 172.21.15.33:0/146320656 wait complete. 2024-09-10T21:10:56.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:55 smithi104 ceph-mon[35325]: pgmap v554: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 294 KiB/s wr, 80 op/s 2024-09-10T21:10:56.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:55 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:56.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:55 smithi174 ceph-mon[35635]: pgmap v554: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 449 KiB/s rd, 294 KiB/s wr, 80 op/s 2024-09-10T21:10:56.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:55 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:10:56.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515243 6.30 MB/sec execute 237 sec latency 2887.957 ms 2024-09-10T21:10:57.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:10:57.290 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:57.734 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515265 6.27 MB/sec execute 238 sec latency 3253.371 ms 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.793+0000 7fdce89fa640 1 -- 172.21.15.33:0/2617994728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410a920 msgr2=0x7fdce410cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.793+0000 7fdce89fa640 1 --2- 172.21.15.33:0/2617994728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410a920 0x7fdce410cd10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8009900 tx=0x7fdcd802a410 comp rx=0 tx=0).stop 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 -- 172.21.15.33:0/2617994728 shutdown_connections 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 --2- 172.21.15.33:0/2617994728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdce410d250 0x7fdce410f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 --2- 172.21.15.33:0/2617994728 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410a920 0x7fdce410cd10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 --2- 172.21.15.33:0/2617994728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce4101810 0x7fdce4101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 -- 172.21.15.33:0/2617994728 >> 172.21.15.33:0/2617994728 conn(0x7fdce4076f50 msgr2=0x7fdce4077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 -- 172.21.15.33:0/2617994728 shutdown_connections 2024-09-10T21:10:57.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 -- 172.21.15.33:0/2617994728 wait complete. 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 Processor -- start 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.794+0000 7fdce89fa640 1 -- start start 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce89fa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce4101810 0x7fdce41027e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce89fa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdce410a920 0x7fdce4102d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce89fa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410d250 0x7fdce4103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce89fa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4112db0 con 0x7fdce410d250 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce89fa640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4112c30 con 0x7fdce4101810 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce89fa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdce4112f30 con 0x7fdce410a920 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410d250 0x7fdce4103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410d250 0x7fdce4103260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33546/0 (socket says 172.21.15.33:33546) 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 -- 172.21.15.33:0/3053091345 learned_addr learned my addr 172.21.15.33:0/3053091345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdce410a920 0x7fdce4102d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce259c640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce4101810 0x7fdce41027e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 -- 172.21.15.33:0/3053091345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdce410a920 msgr2=0x7fdce4102d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdce410a920 0x7fdce4102d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 -- 172.21.15.33:0/3053091345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce4101810 msgr2=0x7fdce41027e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce4101810 0x7fdce41027e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 -- 172.21.15.33:0/3053091345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdce41a7780 con 0x7fdce410d250 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce2d9d640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410d250 0x7fdce4103260 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009fd0 tx=0x7fdcdc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdcd37fe640 1 -- 172.21.15.33:0/3053091345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc011830 con 0x7fdce410d250 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdcd37fe640 1 -- 172.21.15.33:0/3053091345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcdc016070 con 0x7fdce410d250 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.795+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdce41a7a70 con 0x7fdce410d250 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.796+0000 7fdcd37fe640 1 -- 172.21.15.33:0/3053091345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc010de0 con 0x7fdce410d250 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.796+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdce41a7f30 con 0x7fdce410d250 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.798+0000 7fdceadcf740 1 Processor -- start 2024-09-10T21:10:57.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.798+0000 7fdcd37fe640 1 -- 172.21.15.33:0/3053091345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdcdc0119d0 con 0x7fdce410d250 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdceadcf740 1 -- start start 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa02f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa03b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa03e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdceadcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561a9fa5ea0 con 0x5561aa02fdf0 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdceadcf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561a9f952d0 con 0x5561aa02f4d0 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdceadcf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561a9f990e0 con 0x5561aa03bd80 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa03b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa03b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33556/0 (socket says 172.21.15.33:33556) 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 -- 172.21.15.33:0/1898356529 learned_addr learned my addr 172.21.15.33:0/1898356529 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa02f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 -- 172.21.15.33:0/1898356529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 msgr2=0x5561aa03e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa03e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 -- 172.21.15.33:0/1898356529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 msgr2=0x5561aa02f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa02f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.799+0000 7fdce259c640 1 -- 172.21.15.33:0/1898356529 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a9cf3650 con 0x5561aa02fdf0 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.800+0000 7fdcd37fe640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdccc0b52e0 0x7fdccc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.800+0000 7fdce259c640 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa03b840 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400af40 tx=0x7fdcd400f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f703448b93090255 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.800+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdccc0b52e0 0x7fdccc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.800+0000 7fdcd37fe640 1 -- 172.21.15.33:0/3053091345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdcdc0e5100 con 0x7fdce410d250 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.800+0000 7fdcd0ff9640 1 -- 172.21.15.33:0/1898356529 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd401d070 con 0x5561aa02fdf0 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.800+0000 7fdcd0ff9640 1 -- 172.21.15.33:0/1898356529 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4010db0 con 0x5561aa02fdf0 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.800+0000 7fdcd0ff9640 1 -- 172.21.15.33:0/1898356529 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4018da0 con 0x5561aa02fdf0 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 -- 172.21.15.33:0/1898356529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 msgr2=0x5561aa03b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdccc0b52e0 0x7fdccc0b77a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8031440 tx=0x7fdcd8032b20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa03b840 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400af40 tx=0x7fdcd400f850 comp rx=0 tx=0).stop 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 -- 172.21.15.33:0/1898356529 shutdown_connections 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa03e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa03b840 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 --2- 172.21.15.33:0/1898356529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa02f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 -- 172.21.15.33:0/1898356529 >> 172.21.15.33:0/1898356529 conn(0x5561aa02d300 msgr2=0x5561aa02e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 -- 172.21.15.33:0/1898356529 shutdown_connections 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 -- 172.21.15.33:0/1898356529 wait complete. 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 -- start start 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 Processor -- start 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa03b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02fdf0 0x5561aa038ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa0390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.801+0000 7fdceadcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5561a9df6700 con 0x5561aa02f4d0 2024-09-10T21:10:57.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdceadcf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5561a9f30160 con 0x5561aa02fdf0 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdceadcf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561aa03ea00 con 0x5561aa03bd80 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdceadcf740 10 client.0 _ll_register_callbacks cb 0x5561a9f499e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa03b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33572/0 (socket says 172.21.15.33:33572) 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 -- 172.21.15.33:0/2582258909 learned_addr learned my addr 172.21.15.33:0/2582258909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02fdf0 0x5561aa038ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce2d9d640 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa0390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 -- 172.21.15.33:0/2582258909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 msgr2=0x5561aa0390e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa0390e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 -- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02fdf0 msgr2=0x5561aa038ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02fdf0 0x5561aa038ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 -- 172.21.15.33:0/2582258909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a9f5dc90 con 0x5561aa02f4d0 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdce259c640 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa03b660 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4000f80 tx=0x7fdcd400a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06cfc9b44c86034 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdcc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/2582258909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4027030 con 0x5561aa02f4d0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/2582258909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdcd4018500 con 0x5561aa02f4d0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.802+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd4010db0 con 0x5561aa02f4d0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/2582258909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdcd401d040 con 0x5561aa02f4d0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 10 client.16653 fetch_fsmap learned FSMap version 52 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5561aa04a350 con 0x5561aa02f4d0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/2582258909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4010db0 con 0x5561aa02f4d0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/2582258909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7fdcd4010700 con 0x5561aa02f4d0 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 10 client.16653 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 10 client.16653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 20 client.16653 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:57.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa04b6a0 0x5561aa04da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 4 client.16653 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561a9fa7df0 con 0x5561aa04b6a0 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.803+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa04b6a0 0x5561aa04da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.804+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa04b6a0 0x5561aa04da90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8aeb6f05a1e04221 server_cookie=36ae03885a1c9a58 in_seq=0 out_seq=0 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.804+0000 7fdcc9ffb640 10 client.16653 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/2582258909 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5561a9fa7df0 con 0x5561aa04b6a0 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdcc9ffb640 10 client.16653 handle_command_reply: tid=0 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 client.16653 shutdown 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 msgr2=0x5561aa03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa03b660 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdcd4000f80 tx=0x7fdcd400a920 comp rx=0 tx=0).stop 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 shutdown_connections 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa0390e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02fdf0 0x5561aa038ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa04b6a0 0x5561aa04da90 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 --2- 172.21.15.33:0/2582258909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa03b660 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdcc9ffb640 0 client.16653 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:57.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 >> 172.21.15.33:0/2582258909 conn(0x5561aa02d300 msgr2=0x5561aa032ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 shutdown_connections 2024-09-10T21:10:57.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 1 -- 172.21.15.33:0/2582258909 wait complete. 2024-09-10T21:10:57.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.805+0000 7fdceadcf740 20 client.16653 trim_cache size 0 max 16384 2024-09-10T21:10:57.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 Processor -- start 2024-09-10T21:10:57.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 -- start start 2024-09-10T21:10:57.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa047c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa02fdf0 0x5561aa02cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa03bd80 0x5561aa03d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b06c0 con 0x5561aa02f4d0 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b2320 con 0x5561aa03bd80 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdceadcf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b0dc0 con 0x5561aa02fdf0 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdce259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa047c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdce259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa047c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33574/0 (socket says 172.21.15.33:33574) 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.818+0000 7fdce259c640 1 -- 172.21.15.33:0/3514792365 learned_addr learned my addr 172.21.15.33:0/3514792365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce2d9d640 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa03bd80 0x5561aa03d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa02fdf0 0x5561aa02cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce259c640 1 -- 172.21.15.33:0/3514792365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa02fdf0 msgr2=0x5561aa02cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce259c640 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa02fdf0 0x5561aa02cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce259c640 1 -- 172.21.15.33:0/3514792365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa03bd80 msgr2=0x5561aa03d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce259c640 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa03bd80 0x5561aa03d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce259c640 1 -- 172.21.15.33:0/3514792365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561aa02fc20 con 0x5561aa02f4d0 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdce259c640 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa047c60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400a8f0 tx=0x7fdcd400ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b12f139e514828 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdccbfff640 1 -- 172.21.15.33:0/3514792365 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd4027040 con 0x5561aa02f4d0 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdccbfff640 1 -- 172.21.15.33:0/3514792365 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4010700 con 0x5561aa02f4d0 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.819+0000 7fdccbfff640 1 -- 172.21.15.33:0/3514792365 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd40046f0 con 0x5561aa02f4d0 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 -- 172.21.15.33:0/3514792365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 msgr2=0x5561aa047c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa047c60 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdcd400a8f0 tx=0x7fdcd400ae40 comp rx=0 tx=0).stop 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 -- 172.21.15.33:0/3514792365 shutdown_connections 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa03bd80 0x5561aa03d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa02fdf0 0x5561aa02cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 --2- 172.21.15.33:0/3514792365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02f4d0 0x5561aa047c60 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 -- 172.21.15.33:0/3514792365 >> 172.21.15.33:0/3514792365 conn(0x5561aa02d300 msgr2=0x5561aa02e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 -- 172.21.15.33:0/3514792365 shutdown_connections 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 -- 172.21.15.33:0/3514792365 wait complete. 2024-09-10T21:10:57.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 -- start start 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 Processor -- start 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa043cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa044210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa03ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.820+0000 7fdceadcf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b1b90 con 0x5561aa02fdf0 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdceadcf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41b1490 con 0x5561aa02f4d0 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdceadcf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdce41ae680 con 0x5561aa03bd80 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdceadcf740 10 client.0 _ll_register_callbacks cb 0x7fdcd401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa044210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa044210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33588/0 (socket says 172.21.15.33:33588) 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 -- 172.21.15.33:0/830917644 learned_addr learned my addr 172.21.15.33:0/830917644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce259c640 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa043cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce2d9d640 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa03ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 -- 172.21.15.33:0/830917644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 msgr2=0x5561aa03ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa03ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 -- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 msgr2=0x5561aa043cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa043cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 -- 172.21.15.33:0/830917644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561aa044900 con 0x5561aa02fdf0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdce1d9b640 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa044210 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8032f00 tx=0x7fdcd803e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391d59cc077016da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdcc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/830917644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8055070 con 0x5561aa02fdf0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/830917644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdcd80035f0 con 0x5561aa02fdf0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.821+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdcd4010700 con 0x5561aa02fdf0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/830917644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fdcd8061050 con 0x5561aa02fdf0 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 10 client.16659 fetch_fsmap learned FSMap version 52 2024-09-10T21:10:57.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x5561aa0b87d0 con 0x5561aa02fdf0 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/830917644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcd4010700 con 0x5561aa02fdf0 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/830917644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7fdcd803c7e0 con 0x5561aa02fdf0 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 10 client.16659 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 10 client.16659 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 20 client.16659 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa0b8f30 0x5561aa0bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 4 client.16659 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5561a9fa38a0 con 0x5561aa0b8f30 2024-09-10T21:10:57.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.822+0000 7fdce259c640 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa0b8f30 0x5561aa0bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdce259c640 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa0b8f30 0x5561aa0bb350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc432a9aca3a701b server_cookie=b95845303eee9cd4 in_seq=0 out_seq=0 2024-09-10T21:10:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdcc9ffb640 10 client.16659 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdcc9ffb640 1 -- 172.21.15.33:0/830917644 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5561a9fa38a0 con 0x5561aa0b8f30 2024-09-10T21:10:57.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdcc9ffb640 10 client.16659 handle_command_reply: tid=0 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 client.16659 shutdown 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 msgr2=0x5561aa044210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa044210 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8032f00 tx=0x7fdcd803e070 comp rx=0 tx=0).stop 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 shutdown_connections 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa03bd80 0x5561aa03ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5561aa0b8f30 0x5561aa0bb350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5561aa02fdf0 0x5561aa044210 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 --2- 172.21.15.33:0/830917644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5561aa02f4d0 0x5561aa043cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdcc9ffb640 0 client.16659 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 >> 172.21.15.33:0/830917644 conn(0x5561aa02d300 msgr2=0x5561aa032530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 shutdown_connections 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 1 -- 172.21.15.33:0/830917644 wait complete. 2024-09-10T21:10:57.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.823+0000 7fdceadcf740 20 client.16659 trim_cache size 0 max 16384 2024-09-10T21:10:57.834 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:57.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdccc0b52e0 msgr2=0x7fdccc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdccc0b52e0 0x7fdccc0b77a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fdcd8031440 tx=0x7fdcd8032b20 comp rx=0 tx=0).stop 2024-09-10T21:10:57.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410d250 msgr2=0x7fdce4103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410d250 0x7fdce4103260 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009fd0 tx=0x7fdcdc00b8e0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 shutdown_connections 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdccc0b52e0 0x7fdccc0b77a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdce410d250 0x7fdce4103260 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdce410a920 0x7fdce4102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 --2- 172.21.15.33:0/3053091345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce4101810 0x7fdce41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 >> 172.21.15.33:0/3053091345 conn(0x7fdce4076f50 msgr2=0x7fdce41005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 shutdown_connections 2024-09-10T21:10:57.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:57.824+0000 7fdce89fa640 1 -- 172.21.15.33:0/3053091345 wait complete. 2024-09-10T21:10:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:57 smithi033 ceph-mon[30860]: pgmap v555: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 323 KiB/s wr, 80 op/s 2024-09-10T21:10:58.195 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:57 smithi174 ceph-mon[35635]: pgmap v555: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 323 KiB/s wr, 80 op/s 2024-09-10T21:10:58.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:57 smithi104 ceph-mon[35325]: pgmap v555: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 323 KiB/s wr, 80 op/s 2024-09-10T21:10:58.703 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:10:58.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515265 6.24 MB/sec execute 239 sec latency 1620.306 ms 2024-09-10T21:10:58.964 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:10:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 -- 172.21.15.33:0/645347652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd304102dc0 msgr2=0x7fd3041031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/645347652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd304102dc0 0x7fd3041031a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009900 tx=0x7fd2f402a410 comp rx=0 tx=0).stop 2024-09-10T21:10:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 -- 172.21.15.33:0/645347652 shutdown_connections 2024-09-10T21:10:59.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/645347652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd304109150 0x7fd30410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/645347652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3041087d0 0x7fd304108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/645347652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd304102dc0 0x7fd3041031a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 -- 172.21.15.33:0/645347652 >> 172.21.15.33:0/645347652 conn(0x7fd3040fe830 msgr2=0x7fd304100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 -- 172.21.15.33:0/645347652 shutdown_connections 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.454+0000 7fd30a6d8640 1 -- 172.21.15.33:0/645347652 wait complete. 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 Processor -- start 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 -- start start 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd304102dc0 0x7fd3041992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3041087d0 0x7fd304199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd304109150 0x7fd30419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd304112e60 con 0x7fd3041087d0 2024-09-10T21:10:59.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd304112ce0 con 0x7fd304102dc0 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd30a6d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd304112fe0 con 0x7fd304109150 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3041087d0 0x7fd304199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd3037fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3041087d0 0x7fd304199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55614/0 (socket says 172.21.15.33:55614) 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd3037fe640 1 -- 172.21.15.33:0/2160492118 learned_addr learned my addr 172.21.15.33:0/2160492118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd303fff640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd304102dc0 0x7fd3041992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd308c75640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd304109150 0x7fd30419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd3037fe640 1 -- 172.21.15.33:0/2160492118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd304109150 msgr2=0x7fd30419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd3037fe640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd304109150 0x7fd30419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd3037fe640 1 -- 172.21.15.33:0/2160492118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd304102dc0 msgr2=0x7fd3041992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.455+0000 7fd3037fe640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd304102dc0 0x7fd3041992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.456+0000 7fd3037fe640 1 -- 172.21.15.33:0/2160492118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd30419f3a0 con 0x7fd3041087d0 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.456+0000 7fd3037fe640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3041087d0 0x7fd304199830 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800d770 tx=0x7fd2f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.456+0000 7fd3017fa640 1 -- 172.21.15.33:0/2160492118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8017070 con 0x7fd3041087d0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.456+0000 7fd3017fa640 1 -- 172.21.15.33:0/2160492118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f80054d0 con 0x7fd3041087d0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.456+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd30419f690 con 0x7fd3041087d0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.456+0000 7fd3017fa640 1 -- 172.21.15.33:0/2160492118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8004c30 con 0x7fd3041087d0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.456+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3041a5d50 con 0x7fd3041087d0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 Processor -- start 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd3017fa640 1 -- 172.21.15.33:0/2160492118 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd2f8004e10 con 0x7fd3041087d0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 -- start start 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c58170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560732bbfea0 con 0x560732c494d0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560732baf2d0 con 0x560732c49df0 2024-09-10T21:10:59.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd30caad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560732bb30e0 con 0x560732c55d80 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd308c75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd308c75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55626/0 (socket says 172.21.15.33:55626) 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd303fff640 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd308c75640 1 -- 172.21.15.33:0/1682860651 learned_addr learned my addr 172.21.15.33:0/1682860651 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd3037fe640 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c58170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd308c75640 1 -- 172.21.15.33:0/1682860651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 msgr2=0x560732c58170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.459+0000 7fd308c75640 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c58170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd308c75640 1 -- 172.21.15.33:0/1682860651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 msgr2=0x560732c55840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd308c75640 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c55840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd308c75640 1 -- 172.21.15.33:0/1682860651 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073290d650 con 0x560732c494d0 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd3017fa640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd2ec0b52e0 0x7fd2ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd308c75640 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c498b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009fa0 tx=0x7fd2fc00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2d83aaa58b7e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd303fff640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd2ec0b52e0 0x7fd2ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd3017fa640 1 -- 172.21.15.33:0/2160492118 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd2f80e57c0 con 0x7fd3041087d0 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd2ea7fc640 1 -- 172.21.15.33:0/1682860651 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc015070 con 0x560732c494d0 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd2ea7fc640 1 -- 172.21.15.33:0/1682860651 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc01ab10 con 0x560732c494d0 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.460+0000 7fd2ea7fc640 1 -- 172.21.15.33:0/1682860651 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc011d00 con 0x560732c494d0 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 -- 172.21.15.33:0/1682860651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 msgr2=0x560732c498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd303fff640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd2ec0b52e0 0x7fd2ec0b77a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4000c00 tx=0x7fd2f4005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c498b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009fa0 tx=0x7fd2fc00f7f0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 -- 172.21.15.33:0/1682860651 shutdown_connections 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c58170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 --2- 172.21.15.33:0/1682860651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c498b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 -- 172.21.15.33:0/1682860651 >> 172.21.15.33:0/1682860651 conn(0x560732c47300 msgr2=0x560732c484a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 -- 172.21.15.33:0/1682860651 shutdown_connections 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 -- 172.21.15.33:0/1682860651 wait complete. 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.461+0000 7fd30caad740 1 -- start start 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 1 Processor -- start 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c52af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c53120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560732a10700 con 0x560732c494d0 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560732b4a160 con 0x560732c49df0 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560732c58a00 con 0x560732c55d80 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd30caad740 10 client.0 _ll_register_callbacks cb 0x560732b639e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55630/0 (socket says 172.21.15.33:55630) 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 -- 172.21.15.33:0/159168347 learned_addr learned my addr 172.21.15.33:0/159168347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd3037fe640 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c52af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd308c75640 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c53120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 -- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 msgr2=0x560732c53120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c53120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 -- 172.21.15.33:0/159168347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 msgr2=0x560732c52af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c52af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.462+0000 7fd303fff640 1 -- 172.21.15.33:0/159168347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560732b77c90 con 0x560732c494d0 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd303fff640 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c556a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4033000 tx=0x7fd2f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=816595df6310bfbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd2e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd2e37fe640 1 -- 172.21.15.33:0/159168347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f404d070 con 0x560732c494d0 2024-09-10T21:10:59.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd2e37fe640 1 -- 172.21.15.33:0/159168347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd2f4004c10 con 0x560732c494d0 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2fc01ab10 con 0x560732c494d0 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd2e37fe640 1 -- 172.21.15.33:0/159168347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd2f4059050 con 0x560732c494d0 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 10 client.16671 fetch_fsmap learned FSMap version 52 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560732c64360 con 0x560732c494d0 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd2e37fe640 1 -- 172.21.15.33:0/159168347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc01ab10 con 0x560732c494d0 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd2e37fe640 1 -- 172.21.15.33:0/159168347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7fd2f40325b0 con 0x560732c494d0 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 10 client.16671 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 10 client.16671 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 20 client.16671 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732c657c0 0x560732c67c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 4 client.16671 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.463+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560732bc1df0 con 0x560732c657c0 2024-09-10T21:10:59.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.464+0000 7fd3037fe640 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732c657c0 0x560732c67c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.464+0000 7fd2e37fe640 10 client.16671 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.464+0000 7fd3037fe640 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732c657c0 0x560732c67c30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bba60e4cb75456ee server_cookie=e3a3ac3c0e7638fb in_seq=0 out_seq=0 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd2e37fe640 1 -- 172.21.15.33:0/159168347 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560732bc1df0 con 0x560732c657c0 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd2e37fe640 10 client.16671 handle_command_reply: tid=0 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 client.16671 shutdown 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 msgr2=0x560732c556a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c556a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4033000 tx=0x7fd2f4009260 comp rx=0 tx=0).stop 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 shutdown_connections 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c53120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c52af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732c657c0 0x560732c67c30 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 --2- 172.21.15.33:0/159168347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c556a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd2e37fe640 0 client.16671 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:59.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 >> 172.21.15.33:0/159168347 conn(0x560732c47300 msgr2=0x560732c4cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 shutdown_connections 2024-09-10T21:10:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 1 -- 172.21.15.33:0/159168347 wait complete. 2024-09-10T21:10:59.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.465+0000 7fd30caad740 20 client.16671 trim_cache size 0 max 16384 2024-09-10T21:10:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.484+0000 7fd30caad740 1 Processor -- start 2024-09-10T21:10:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.484+0000 7fd30caad740 1 -- start start 2024-09-10T21:10:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.485+0000 7fd30caad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c61c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.485+0000 7fd30caad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.485+0000 7fd30caad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c57170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.485+0000 7fd30caad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3041ae4d0 con 0x560732c494d0 2024-09-10T21:10:59.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.485+0000 7fd30caad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3041b0130 con 0x560732c49df0 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.485+0000 7fd30caad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3041aebd0 con 0x560732c55d80 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c57170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c57170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:38802/0 (socket says 172.21.15.33:38802) 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 -- 172.21.15.33:0/3354654276 learned_addr learned my addr 172.21.15.33:0/3354654276 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd303fff640 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c61c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 -- 172.21.15.33:0/3354654276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 msgr2=0x560732c46310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c46310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 -- 172.21.15.33:0/3354654276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 msgr2=0x560732c61c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c61c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.486+0000 7fd308c75640 1 -- 172.21.15.33:0/3354654276 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560732c528a0 con 0x560732c55d80 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.487+0000 7fd308c75640 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c57170 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009fa0 tx=0x7fd2fc009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467c99822293347a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.488+0000 7fd2e9ffb640 1 -- 172.21.15.33:0/3354654276 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2fc015040 con 0x560732c55d80 2024-09-10T21:10:59.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.488+0000 7fd2e9ffb640 1 -- 172.21.15.33:0/3354654276 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc003e20 con 0x560732c55d80 2024-09-10T21:10:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.489+0000 7fd30caad740 1 -- 172.21.15.33:0/3354654276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 msgr2=0x560732c57170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.489+0000 7fd30caad740 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c57170 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc009fa0 tx=0x7fd2fc009520 comp rx=0 tx=0).stop 2024-09-10T21:10:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.489+0000 7fd30caad740 1 -- 172.21.15.33:0/3354654276 shutdown_connections 2024-09-10T21:10:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.489+0000 7fd30caad740 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c57170 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.489+0000 7fd30caad740 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c49df0 0x560732c46310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.489+0000 7fd30caad740 1 --2- 172.21.15.33:0/3354654276 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c494d0 0x560732c61c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.489+0000 7fd30caad740 1 -- 172.21.15.33:0/3354654276 >> 172.21.15.33:0/3354654276 conn(0x560732c47300 msgr2=0x560732c45c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.497+0000 7fd30caad740 1 -- 172.21.15.33:0/3354654276 shutdown_connections 2024-09-10T21:10:59.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.501+0000 7fd30caad740 1 -- 172.21.15.33:0/3354654276 wait complete. 2024-09-10T21:10:59.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.501+0000 7fd30caad740 1 -- start start 2024-09-10T21:10:59.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.501+0000 7fd30caad740 1 Processor -- start 2024-09-10T21:10:59.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd30caad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c494d0 0x560732c5e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd30caad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c5bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd30caad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c5c4d0 0x560732c5c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd30caad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd3041af9a0 con 0x560732c5c4d0 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd30caad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd3041af2a0 con 0x560732c494d0 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd30caad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3041ac490 con 0x560732c55d80 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd30caad740 10 client.0 _ll_register_callbacks cb 0x7fd2f4005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd308c75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c5c4d0 0x560732c5c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd308c75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c5c4d0 0x560732c5c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55654/0 (socket says 172.21.15.33:55654) 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.502+0000 7fd308c75640 1 -- 172.21.15.33:0/1199831931 learned_addr learned my addr 172.21.15.33:0/1199831931 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.503+0000 7fd3037fe640 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c5bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.504+0000 7fd3037fe640 1 -- 172.21.15.33:0/1199831931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c494d0 msgr2=0x560732c5e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.504+0000 7fd3037fe640 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c494d0 0x560732c5e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.504+0000 7fd3037fe640 1 -- 172.21.15.33:0/1199831931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c5c4d0 msgr2=0x560732c5c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.504+0000 7fd3037fe640 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c5c4d0 0x560732c5c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.504+0000 7fd3037fe640 1 -- 172.21.15.33:0/1199831931 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560732ccc800 con 0x560732c55d80 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.505+0000 7fd3037fe640 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c5bf90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80e3280 tx=0x7fd2f80cf040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ce888e9d949be22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.505+0000 7fd2e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.506+0000 7fd2e37fe640 1 -- 172.21.15.33:0/1199831931 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f8020040 con 0x560732c55d80 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.506+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2fc003e20 con 0x560732c55d80 2024-09-10T21:10:59.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.506+0000 7fd2e37fe640 1 -- 172.21.15.33:0/1199831931 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd2f8005130 con 0x560732c55d80 2024-09-10T21:10:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.508+0000 7fd2e37fe640 1 -- 172.21.15.33:0/1199831931 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd2f80ab6b0 con 0x560732c55d80 2024-09-10T21:10:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.508+0000 7fd30caad740 10 client.26149 fetch_fsmap learned FSMap version 52 2024-09-10T21:10:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.508+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560732ccdb00 con 0x560732c55d80 2024-09-10T21:10:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.509+0000 7fd2e37fe640 1 -- 172.21.15.33:0/1199831931 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2fc003e20 con 0x560732c55d80 2024-09-10T21:10:59.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.511+0000 7fd2e37fe640 1 -- 172.21.15.33:0/1199831931 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 52) ==== 8390+0+0 (secure 0 0 0) 0x7fd2f80e5bb0 con 0x560732c55d80 2024-09-10T21:10:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.511+0000 7fd30caad740 10 client.26149 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:10:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.511+0000 7fd30caad740 10 client.26149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:10:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.511+0000 7fd30caad740 20 client.26149 populate_metadata read hostname 'smithi033' 2024-09-10T21:10:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.511+0000 7fd30caad740 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732cce2f0 0x560732cd0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:10:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.511+0000 7fd30caad740 4 client.26149 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:10:59.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.511+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560732bbd8a0 con 0x560732cce2f0 2024-09-10T21:10:59.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.516+0000 7fd303fff640 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732cce2f0 0x560732cd0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:10:59.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.526+0000 7fd2e37fe640 10 client.26149 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:59.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.526+0000 7fd303fff640 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732cce2f0 0x560732cd0710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fea420a418297463 server_cookie=4c4cee1b6c163b50 in_seq=0 out_seq=0 2024-09-10T21:10:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.528+0000 7fd2e37fe640 1 -- 172.21.15.33:0/1199831931 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560732bbd8a0 con 0x560732cce2f0 2024-09-10T21:10:59.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.528+0000 7fd2e37fe640 10 client.26149 handle_command_reply: tid=0 2024-09-10T21:10:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.528+0000 7fd30caad740 1 client.26149 shutdown 2024-09-10T21:10:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.529+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 msgr2=0x560732c5bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.529+0000 7fd30caad740 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c5bf90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd2f80e3280 tx=0x7fd2f80cf040 comp rx=0 tx=0).stop 2024-09-10T21:10:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 shutdown_connections 2024-09-10T21:10:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560732cce2f0 0x560732cd0710 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560732c5c4d0 0x560732c5c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560732c55d80 0x560732c5bf90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 --2- 172.21.15.33:0/1199831931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560732c494d0 0x560732c5e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd2e37fe640 0 client.26149 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:10:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 >> 172.21.15.33:0/1199831931 conn(0x560732c47300 msgr2=0x560732c54940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 shutdown_connections 2024-09-10T21:10:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.530+0000 7fd30caad740 1 -- 172.21.15.33:0/1199831931 wait complete. 2024-09-10T21:10:59.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.531+0000 7fd30caad740 20 client.26149 trim_cache size 0 max 16384 2024-09-10T21:10:59.536 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:10:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.532+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd2ec0b52e0 msgr2=0x7fd2ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.532+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd2ec0b52e0 0x7fd2ec0b77a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4000c00 tx=0x7fd2f4005b50 comp rx=0 tx=0).stop 2024-09-10T21:10:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3041087d0 msgr2=0x7fd304199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:10:59.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3041087d0 0x7fd304199830 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800d770 tx=0x7fd2f800dc30 comp rx=0 tx=0).stop 2024-09-10T21:10:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 shutdown_connections 2024-09-10T21:10:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd2ec0b52e0 0x7fd2ec0b77a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd304109150 0x7fd30419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3041087d0 0x7fd304199830 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 --2- 172.21.15.33:0/2160492118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd304102dc0 0x7fd3041992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:10:59.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.533+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 >> 172.21.15.33:0/2160492118 conn(0x7fd3040fe830 msgr2=0x7fd304100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:10:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.534+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 shutdown_connections 2024-09-10T21:10:59.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:10:59.534+0000 7fd30a6d8640 1 -- 172.21.15.33:0/2160492118 wait complete. 2024-09-10T21:10:59.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 515408 6.22 MB/sec execute 240 sec latency 2461.733 ms 2024-09-10T21:11:00.297 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:10:59 smithi033 ceph-mon[30860]: pgmap v556: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 372 KiB/s wr, 90 op/s 2024-09-10T21:11:00.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:10:59 smithi104 ceph-mon[35325]: pgmap v556: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 372 KiB/s wr, 90 op/s 2024-09-10T21:11:00.325 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:11:00.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:10:59 smithi174 ceph-mon[35635]: pgmap v556: 97 pgs: 97 active+clean; 200 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 692 KiB/s rd, 372 KiB/s wr, 90 op/s 2024-09-10T21:11:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:00.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 517525 6.23 MB/sec execute 241 sec latency 24.318 ms 2024-09-10T21:11:01.035 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:00 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 -- 172.21.15.33:0/724690014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d9410d100 msgr2=0x7f5d9410f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/724690014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d9410d100 0x7f5d9410f560 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009be0 tx=0x7f5d8402a6c0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 -- 172.21.15.33:0/724690014 shutdown_connections 2024-09-10T21:11:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/724690014 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d9410d100 0x7f5d9410f560 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/724690014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d941045c0 0x7f5d9410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/724690014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d94103ca0 0x7f5d94104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 -- 172.21.15.33:0/724690014 >> 172.21.15.33:0/724690014 conn(0x7f5d940fd700 msgr2=0x7f5d940ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 -- 172.21.15.33:0/724690014 shutdown_connections 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.256+0000 7f5d9c61b640 1 -- 172.21.15.33:0/724690014 wait complete. 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 Processor -- start 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 -- start start 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d94103ca0 0x7f5d94199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d941045c0 0x7f5d94199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9410d100 0x7f5d9419eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94112d70 con 0x7f5d941045c0 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94112bf0 con 0x7f5d94103ca0 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.257+0000 7f5d9c61b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d94112ef0 con 0x7f5d9410d100 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d94103ca0 0x7f5d94199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d94103ca0 0x7f5d94199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57970/0 (socket says 172.21.15.33:57970) 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 -- 172.21.15.33:0/4217528869 learned_addr learned my addr 172.21.15.33:0/4217528869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9410d100 0x7f5d9419eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d941045c0 0x7f5d94199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 -- 172.21.15.33:0/4217528869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9410d100 msgr2=0x7f5d9419eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9410d100 0x7f5d9419eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 -- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d941045c0 msgr2=0x7f5d94199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d941045c0 0x7f5d94199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 -- 172.21.15.33:0/4217528869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9419f2f0 con 0x7f5d94103ca0 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9410d100 0x7f5d9419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9a3b7640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d94103ca0 0x7f5d94199240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d937fe640 1 -- 172.21.15.33:0/4217528869 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00cd80 con 0x7f5d94103ca0 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9419f5e0 con 0x7f5d94103ca0 2024-09-10T21:11:01.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d937fe640 1 -- 172.21.15.33:0/4217528869 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8c0108e0 con 0x7f5d94103ca0 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d937fe640 1 -- 172.21.15.33:0/4217528869 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c018980 con 0x7f5d94103ca0 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.258+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d94101a00 con 0x7f5d94103ca0 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 Processor -- start 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 -- start start 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d937fe640 1 -- 172.21.15.33:0/4217528869 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5d8c010430 con 0x7f5d94103ca0 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cdb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdbdf0 0x557b35ce7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35cea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b35c51ea0 con 0x557b35ce7d80 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557b35c412d0 con 0x557b35cdb4d0 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.261+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b35c450e0 con 0x557b35cdbdf0 2024-09-10T21:11:01.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cdb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d99bb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35cea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cdb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57978/0 (socket says 172.21.15.33:57978) 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 -- 172.21.15.33:0/1909361165 learned_addr learned my addr 172.21.15.33:0/1909361165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9a3b7640 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdbdf0 0x557b35ce7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 -- 172.21.15.33:0/1909361165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdbdf0 msgr2=0x557b35ce7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdbdf0 0x557b35ce7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 -- 172.21.15.33:0/1909361165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 msgr2=0x557b35cea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35cea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 -- 172.21.15.33:0/1909361165 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b3599f650 con 0x557b35cdb4d0 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cdb8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009be0 tx=0x7f5d840307d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=867baff52acd914f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d937fe640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d7c0b5360 0x7f5d7c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d90ff9640 1 -- 172.21.15.33:0/1909361165 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84041030 con 0x557b35cdb4d0 2024-09-10T21:11:01.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d9a3b7640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d7c0b5360 0x7f5d7c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d90ff9640 1 -- 172.21.15.33:0/1909361165 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d84030970 con 0x557b35cdb4d0 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d90ff9640 1 -- 172.21.15.33:0/1909361165 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84036040 con 0x557b35cdb4d0 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.262+0000 7f5d937fe640 1 -- 172.21.15.33:0/4217528869 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5d8c0e5e30 con 0x7f5d94103ca0 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1909361165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 msgr2=0x557b35cdb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9a3b7640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d7c0b5360 0x7f5d7c0b7820 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0a8c70 tx=0x7f5d8c018420 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cdb8b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009be0 tx=0x7f5d840307d0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1909361165 shutdown_connections 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35cea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdbdf0 0x557b35ce7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1909361165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cdb8b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1909361165 >> 172.21.15.33:0/1909361165 conn(0x557b35cd9300 msgr2=0x557b35cda4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1909361165 shutdown_connections 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.263+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1909361165 wait complete. 2024-09-10T21:11:01.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 -- start start 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 Processor -- start 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35ce75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdbdf0 0x557b35ce4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35ce4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b35aa2700 con 0x557b35ce7d80 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557b35bdc160 con 0x557b35cdbdf0 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b35cea9a0 con 0x557b35cdb4d0 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35ce4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35ce4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55686/0 (socket says 172.21.15.33:55686) 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 -- 172.21.15.33:0/2002376403 learned_addr learned my addr 172.21.15.33:0/2002376403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9e9f0740 10 client.0 _ll_register_callbacks cb 0x557b35bf59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdbdf0 0x557b35ce4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35ce75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 -- 172.21.15.33:0/2002376403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 msgr2=0x557b35ce75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35ce75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 -- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdbdf0 msgr2=0x557b35ce4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdbdf0 0x557b35ce4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.264+0000 7f5d9abb8640 1 -- 172.21.15.33:0/2002376403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b35c09c90 con 0x557b35ce7d80 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35ce4f80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5d84034040 tx=0x7f5d84004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9c83ca10b1fbb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d79ffb640 1 -- 172.21.15.33:0/2002376403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84036050 con 0x557b35ce7d80 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d79ffb640 1 -- 172.21.15.33:0/2002376403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5d840044c0 con 0x557b35ce7d80 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d84030970 con 0x557b35ce7d80 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d79ffb640 1 -- 172.21.15.33:0/2002376403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5d8405a050 con 0x557b35ce7d80 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d9e9f0740 10 client.16689 fetch_fsmap learned FSMap version 53 2024-09-10T21:11:01.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557b35cf6250 con 0x557b35ce7d80 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d79ffb640 1 -- 172.21.15.33:0/2002376403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d84030970 con 0x557b35ce7d80 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d79ffb640 1 -- 172.21.15.33:0/2002376403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f5d8403d620 con 0x557b35ce7d80 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d9e9f0740 10 client.16689 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.265+0000 7f5d9e9f0740 10 client.16689 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.266+0000 7f5d9e9f0740 20 client.16689 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.266+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35cf75a0 0x557b35cf9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.266+0000 7f5d9e9f0740 4 client.16689 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.266+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b35c53df0 con 0x557b35cf75a0 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.266+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35cf75a0 0x557b35cf9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.266+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35cf75a0 0x557b35cf9990 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8ddedd180dbbde0 server_cookie=4275a6caea5d20cd in_seq=0 out_seq=0 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.266+0000 7f5d79ffb640 10 client.16689 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d79ffb640 1 -- 172.21.15.33:0/2002376403 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b35c53df0 con 0x557b35cf75a0 2024-09-10T21:11:01.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d79ffb640 10 client.16689 handle_command_reply: tid=0 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 client.16689 shutdown 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 msgr2=0x557b35ce4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35ce4f80 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f5d84034040 tx=0x7f5d84004180 comp rx=0 tx=0).stop 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 shutdown_connections 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35cf75a0 0x557b35cf9990 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35ce7d80 0x557b35ce4f80 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdbdf0 0x557b35ce4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/2002376403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35ce75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d79ffb640 0 client.16689 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 >> 172.21.15.33:0/2002376403 conn(0x557b35cd9300 msgr2=0x557b35cdee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 shutdown_connections 2024-09-10T21:11:01.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/2002376403 wait complete. 2024-09-10T21:11:01.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.267+0000 7f5d9e9f0740 20 client.16689 trim_cache size 0 max 16384 2024-09-10T21:11:01.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 Processor -- start 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 -- start start 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35cd8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35ce32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35ce7d80 0x557b35ce37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b05e0 con 0x557b35cdbdf0 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b2210 con 0x557b35ce7d80 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.282+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b0cb0 con 0x557b35cdb4d0 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d99bb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35ce32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35cd8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d99bb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35ce32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55696/0 (socket says 172.21.15.33:55696) 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d99bb6640 1 -- 172.21.15.33:0/3887947989 learned_addr learned my addr 172.21.15.33:0/3887947989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:01.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35ce7d80 0x557b35ce37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 -- 172.21.15.33:0/3887947989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35ce7d80 msgr2=0x557b35ce37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35ce7d80 0x557b35ce37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 -- 172.21.15.33:0/3887947989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 msgr2=0x557b35ce32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35ce32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 -- 172.21.15.33:0/3887947989 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b35ce2f20 con 0x557b35cdb4d0 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35ce32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35cd8c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d8403dbe0 tx=0x7f5d8403f040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da19e434cb468687 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d7bfff640 1 -- 172.21.15.33:0/3887947989 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84036040 con 0x557b35cdb4d0 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d7bfff640 1 -- 172.21.15.33:0/3887947989 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d84057070 con 0x557b35cdb4d0 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.283+0000 7f5d7bfff640 1 -- 172.21.15.33:0/3887947989 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d84041040 con 0x557b35cdb4d0 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/3887947989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 msgr2=0x557b35cd8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35cd8c10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5d8403dbe0 tx=0x7f5d8403f040 comp rx=0 tx=0).stop 2024-09-10T21:11:01.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/3887947989 shutdown_connections 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35ce7d80 0x557b35ce37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35ce32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/3887947989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35cdb4d0 0x557b35cd8c10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/3887947989 >> 172.21.15.33:0/3887947989 conn(0x557b35cd9300 msgr2=0x557b35cda3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/3887947989 shutdown_connections 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.284+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/3887947989 wait complete. 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 -- start start 2024-09-10T21:11:01.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 Processor -- start 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cefee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35cf0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35ce7d80 0x557b35ce0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b1a80 con 0x557b35cdbdf0 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941b1380 con 0x557b35cdb4d0 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d941ae5a0 con 0x557b35ce7d80 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9e9f0740 10 client.0 _ll_register_callbacks cb 0x7f5d880094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:01.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d99bb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35cf0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9abb8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35ce7d80 0x557b35ce0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d99bb6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35cf0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55710/0 (socket says 172.21.15.33:55710) 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d99bb6640 1 -- 172.21.15.33:0/1372372063 learned_addr learned my addr 172.21.15.33:0/1372372063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.285+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cefee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d99bb6640 1 -- 172.21.15.33:0/1372372063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35ce7d80 msgr2=0x557b35ce0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35ce7d80 0x557b35ce0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d99bb6640 1 -- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 msgr2=0x557b35cefee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cefee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d99bb6640 1 -- 172.21.15.33:0/1372372063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b35ce1450 con 0x557b35cdbdf0 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d99bb6640 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35cf0460 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5d88005fd0 tx=0x7f5d880128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fc46b9d1210ed4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d79ffb640 1 -- 172.21.15.33:0/1372372063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8802a060 con 0x557b35cdbdf0 2024-09-10T21:11:01.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d79ffb640 1 -- 172.21.15.33:0/1372372063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5d88004250 con 0x557b35cdbdf0 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d84057070 con 0x557b35cdbdf0 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d79ffb640 1 -- 172.21.15.33:0/1372372063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f5d88004450 con 0x557b35cdbdf0 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d9e9f0740 10 client.16695 fetch_fsmap learned FSMap version 53 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.286+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x557b35ce27b0 con 0x557b35cdbdf0 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d79ffb640 1 -- 172.21.15.33:0/1372372063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d84057070 con 0x557b35cdbdf0 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d79ffb640 1 -- 172.21.15.33:0/1372372063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7f5d8802f070 con 0x557b35cdbdf0 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9e9f0740 10 client.16695 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9e9f0740 10 client.16695 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9e9f0740 20 client.16695 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35d65600 0x557b35d67a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9e9f0740 4 client.16695 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b35c4f8a0 con 0x557b35d65600 2024-09-10T21:11:01.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35d65600 0x557b35d67a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d9abb8640 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35d65600 0x557b35d67a20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa82eb1be82b1d31 server_cookie=7c87b443f07aafa3 in_seq=0 out_seq=0 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.287+0000 7f5d79ffb640 10 client.16695 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d79ffb640 1 -- 172.21.15.33:0/1372372063 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b35c4f8a0 con 0x557b35d65600 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d79ffb640 10 client.16695 handle_command_reply: tid=0 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 client.16695 shutdown 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 msgr2=0x557b35cf0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35cf0460 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5d88005fd0 tx=0x7f5d880128e0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 shutdown_connections 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557b35d65600 0x557b35d67a20 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b35ce7d80 0x557b35ce0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b35cdbdf0 0x557b35cf0460 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 --2- 172.21.15.33:0/1372372063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b35cdb4d0 0x557b35cefee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 >> 172.21.15.33:0/1372372063 conn(0x557b35cd9300 msgr2=0x557b35ce3260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d79ffb640 0 client.16695 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:01.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 shutdown_connections 2024-09-10T21:11:01.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 1 -- 172.21.15.33:0/1372372063 wait complete. 2024-09-10T21:11:01.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.288+0000 7f5d9e9f0740 20 client.16695 trim_cache size 0 max 16384 2024-09-10T21:11:01.298 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d7c0b5360 msgr2=0x7f5d7c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d7c0b5360 0x7f5d7c0b7820 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0a8c70 tx=0x7f5d8c018420 comp rx=0 tx=0).stop 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d94103ca0 msgr2=0x7f5d94199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d94103ca0 0x7f5d94199240 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00e9f0 tx=0x7f5d8c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 shutdown_connections 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d7c0b5360 0x7f5d7c0b7820 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d9410d100 0x7f5d9419eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d941045c0 0x7f5d94199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 --2- 172.21.15.33:0/4217528869 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d94103ca0 0x7f5d94199240 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 >> 172.21.15.33:0/4217528869 conn(0x7f5d940fd700 msgr2=0x7f5d9410f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 shutdown_connections 2024-09-10T21:11:01.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:01.289+0000 7f5d9c61b640 1 -- 172.21.15.33:0/4217528869 wait complete. 2024-09-10T21:11:01.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:00 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:01.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:00 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:01.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 518756 6.24 MB/sec execute 242 sec latency 459.855 ms 2024-09-10T21:11:02.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:01 smithi104 ceph-mon[35325]: pgmap v557: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 491 KiB/s wr, 88 op/s 2024-09-10T21:11:02.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:01 smithi174 ceph-mon[35635]: pgmap v557: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 491 KiB/s wr, 88 op/s 2024-09-10T21:11:02.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:01 smithi033 ceph-mon[30860]: pgmap v557: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 491 KiB/s wr, 88 op/s 2024-09-10T21:11:02.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 518785 6.22 MB/sec execute 243 sec latency 824.669 ms 2024-09-10T21:11:03.008 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:11:03.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:11:03.281 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:03.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 518785 6.19 MB/sec execute 244 sec latency 1617.992 ms 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 -- 172.21.15.33:0/1696637651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630103bd0 msgr2=0x7fa630103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 --2- 172.21.15.33:0/1696637651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630103bd0 0x7fa630103fb0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa620009900 tx=0x7fa62002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 -- 172.21.15.33:0/1696637651 shutdown_connections 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 --2- 172.21.15.33:0/1696637651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa63010d180 0x7fa63010f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 --2- 172.21.15.33:0/1696637651 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6301045d0 0x7fa63010cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 --2- 172.21.15.33:0/1696637651 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa630103bd0 0x7fa630103fb0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 -- 172.21.15.33:0/1696637651 >> 172.21.15.33:0/1696637651 conn(0x7fa6300fd700 msgr2=0x7fa6300ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 -- 172.21.15.33:0/1696637651 shutdown_connections 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.735+0000 7fa636055640 1 -- 172.21.15.33:0/1696637651 wait complete. 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 Processor -- start 2024-09-10T21:11:03.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 -- start start 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630103bd0 0x7fa6301991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6301045d0 0x7fa630199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d180 0x7fa63019eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa630112da0 con 0x7fa63010d180 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa630112c20 con 0x7fa630103bd0 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.736+0000 7fa636055640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa630112f20 con 0x7fa6301045d0 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d180 0x7fa63019eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d180 0x7fa63019eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55742/0 (socket says 172.21.15.33:55742) 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 -- 172.21.15.33:0/3051292296 learned_addr learned my addr 172.21.15.33:0/3051292296 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630103bd0 0x7fa6301991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62effd640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6301045d0 0x7fa630199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 -- 172.21.15.33:0/3051292296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6301045d0 msgr2=0x7fa630199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6301045d0 0x7fa630199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 -- 172.21.15.33:0/3051292296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630103bd0 msgr2=0x7fa6301991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630103bd0 0x7fa6301991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 -- 172.21.15.33:0/3051292296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa63019f280 con 0x7fa63010d180 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62ffff640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d180 0x7fa63019eab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa62800ccb0 tx=0x7fa62800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62cff9640 1 -- 172.21.15.33:0/3051292296 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62800f450 con 0x7fa63010d180 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62cff9640 1 -- 172.21.15.33:0/3051292296 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa62800faa0 con 0x7fa63010d180 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa63019f570 con 0x7fa63010d180 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa62cff9640 1 -- 172.21.15.33:0/3051292296 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280189d0 con 0x7fa63010d180 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.737+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6301a5c30 con 0x7fa63010d180 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.740+0000 7fa63842a740 1 Processor -- start 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62cff9640 1 -- 172.21.15.33:0/3051292296 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa628020030 con 0x7fa63010d180 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa63842a740 1 -- start start 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa63842a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040cee0 0x55c65040d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa63842a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650419250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa63842a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c650419790 0x55c65041bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa63842a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c650383e60 con 0x55c65040d800 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa63842a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c650373290 con 0x55c650419790 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa63842a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c6503770a0 con 0x55c65040cee0 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650419250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650419250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55750/0 (socket says 172.21.15.33:55750) 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1257711791 learned_addr learned my addr 172.21.15.33:0/1257711791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62effd640 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040cee0 0x55c65040d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62effd640 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c650419790 0x55c65041bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1257711791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040cee0 msgr2=0x55c65040d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040cee0 0x55c65040d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1257711791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c650419790 msgr2=0x55c65041bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.741+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c650419790 0x55c65041bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1257711791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6500d1930 con 0x55c65040d800 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa62effd640 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040cee0 0x55c65040d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa62cff9640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b52e0 0x7fa6180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa62effd640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b52e0 0x7fa6180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650419250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa620031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856dfbedd9bf62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa62cff9640 1 -- 172.21.15.33:0/3051292296 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa6280e5be0 con 0x7fa63010d180 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa615ffb640 1 -- 172.21.15.33:0/1257711791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620040070 con 0x55c65040d800 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa615ffb640 1 -- 172.21.15.33:0/1257711791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6200316a0 con 0x55c65040d800 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.742+0000 7fa615ffb640 1 -- 172.21.15.33:0/1257711791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620035040 con 0x55c65040d800 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 -- 172.21.15.33:0/1257711791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 msgr2=0x55c650419250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa62effd640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b52e0 0x7fa6180b77a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55c65040ce90 tx=0x7fa624012040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650419250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa620031de0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 -- 172.21.15.33:0/1257711791 shutdown_connections 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c650419790 0x55c65041bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650419250 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 --2- 172.21.15.33:0/1257711791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040cee0 0x55c65040d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 -- 172.21.15.33:0/1257711791 >> 172.21.15.33:0/1257711791 conn(0x55c65040ad10 msgr2=0x55c65040beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 -- 172.21.15.33:0/1257711791 shutdown_connections 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 -- 172.21.15.33:0/1257711791 wait complete. 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 -- start start 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.743+0000 7fa63842a740 1 Processor -- start 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa63842a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650416710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa63842a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650416c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa63842a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650417190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa63842a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c6501d4890 con 0x55c65040d800 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa63842a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c65030dd40 con 0x55c65040cee0 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa63842a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c65041c3e0 con 0x55c650419790 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa63842a740 10 client.0 _ll_register_callbacks cb 0x55c650371c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650416c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650416c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55758/0 (socket says 172.21.15.33:55758) 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 -- 172.21.15.33:0/668816550 learned_addr learned my addr 172.21.15.33:0/668816550 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650416710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 -- 172.21.15.33:0/668816550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 msgr2=0x55c650417190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650417190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 -- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 msgr2=0x55c650416710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650416710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 -- 172.21.15.33:0/668816550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6504257f0 con 0x55c65040d800 2024-09-10T21:11:03.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa62effd640 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650416c50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa62400fd60 tx=0x7fa62400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67e5a2780a861181 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa60effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.744+0000 7fa60effd640 1 -- 172.21.15.33:0/668816550 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624023070 con 0x55c65040d800 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6200316a0 con 0x55c65040d800 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa60effd640 1 -- 172.21.15.33:0/668816550 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa624005220 con 0x55c65040d800 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa60effd640 1 -- 172.21.15.33:0/668816550 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa62400fe50 con 0x55c65040d800 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 10 client.16707 fetch_fsmap learned FSMap version 53 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55c650428230 con 0x55c65040d800 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa60effd640 1 -- 172.21.15.33:0/668816550 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6200316a0 con 0x55c65040d800 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa60effd640 1 -- 172.21.15.33:0/668816550 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7fa624027530 con 0x55c65040d800 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 10 client.16707 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 10 client.16707 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:03.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 20 client.16707 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c650429700 0x55c65042bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 4 client.16707 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c650385db0 con 0x55c650429700 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.745+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c650429700 0x55c65042bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.746+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c650429700 0x55c65042bb70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abebbc0ed362c6e0 server_cookie=a8a5272042c5696d in_seq=0 out_seq=0 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.746+0000 7fa60effd640 10 client.16707 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.746+0000 7fa60effd640 1 -- 172.21.15.33:0/668816550 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c650385db0 con 0x55c650429700 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.746+0000 7fa60effd640 10 client.16707 handle_command_reply: tid=0 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 client.16707 shutdown 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 msgr2=0x55c650416c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650416c50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa62400fd60 tx=0x7fa62400a2d0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 shutdown_connections 2024-09-10T21:11:03.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c650429700 0x55c65042bb70 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650417190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040d800 0x55c650416c50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 --2- 172.21.15.33:0/668816550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650416710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa60effd640 0 client.16707 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 >> 172.21.15.33:0/668816550 conn(0x55c65040ad10 msgr2=0x55c650409f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 shutdown_connections 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 1 -- 172.21.15.33:0/668816550 wait complete. 2024-09-10T21:11:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.747+0000 7fa63842a740 20 client.16707 trim_cache size 0 max 16384 2024-09-10T21:11:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 Processor -- start 2024-09-10T21:11:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 -- start start 2024-09-10T21:11:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650415fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040d800 0x55c650416d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c650419790 0x55c650413920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301ae2c0 con 0x55c650419790 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301afef0 con 0x55c65040cee0 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa63842a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301ae990 con 0x55c65040d800 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c650419790 0x55c650413920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa62ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c650419790 0x55c650413920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55768/0 (socket says 172.21.15.33:55768) 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.765+0000 7fa62ffff640 1 -- 172.21.15.33:0/2520385680 learned_addr learned my addr 172.21.15.33:0/2520385680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650415fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62effd640 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040d800 0x55c650416d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62ffff640 1 -- 172.21.15.33:0/2520385680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040d800 msgr2=0x55c650416d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62ffff640 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040d800 0x55c650416d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62ffff640 1 -- 172.21.15.33:0/2520385680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 msgr2=0x55c650415fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62ffff640 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650415fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62ffff640 1 -- 172.21.15.33:0/2520385680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6504122c0 con 0x55c650419790 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650415fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62effd640 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040d800 0x55c650416d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa62ffff640 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c650419790 0x55c650413920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa6280a8970 tx=0x7fa6280abb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e07eef1bd914869 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa6157fa640 1 -- 172.21.15.33:0/2520385680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280ac3a0 con 0x55c650419790 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa6157fa640 1 -- 172.21.15.33:0/2520385680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6280b0e80 con 0x55c650419790 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.766+0000 7fa6157fa640 1 -- 172.21.15.33:0/2520385680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6280e3e00 con 0x55c650419790 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 -- 172.21.15.33:0/2520385680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c650419790 msgr2=0x55c650413920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c650419790 0x55c650413920 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa6280a8970 tx=0x7fa6280abb10 comp rx=0 tx=0).stop 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 -- 172.21.15.33:0/2520385680 shutdown_connections 2024-09-10T21:11:03.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c650419790 0x55c650413920 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c65040d800 0x55c650416d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 --2- 172.21.15.33:0/2520385680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040cee0 0x55c650415fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 -- 172.21.15.33:0/2520385680 >> 172.21.15.33:0/2520385680 conn(0x55c65040ad10 msgr2=0x55c650416680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 -- 172.21.15.33:0/2520385680 shutdown_connections 2024-09-10T21:11:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 -- 172.21.15.33:0/2520385680 wait complete. 2024-09-10T21:11:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 -- start start 2024-09-10T21:11:03.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.767+0000 7fa63842a740 1 Processor -- start 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa63842a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040cee0 0x55c65041de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa63842a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040d800 0x55c65041e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa63842a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650495640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa63842a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301af760 con 0x55c65040cee0 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa63842a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301af060 con 0x55c65040d800 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa63842a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6301ac280 con 0x55c650419790 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa63842a740 10 client.0 _ll_register_callbacks cb 0x7fa62400fa10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040cee0 0x55c65041de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040cee0 0x55c65041de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55780/0 (socket says 172.21.15.33:55780) 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1128141773 learned_addr learned my addr 172.21.15.33:0/1128141773 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62effd640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040d800 0x55c65041e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62ffff640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650495640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1128141773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 msgr2=0x55c650495640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650495640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040d800 msgr2=0x55c65041e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040d800 0x55c65041e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 -- 172.21.15.33:0/1128141773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c65040d630 con 0x55c65040cee0 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62ffff640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650495640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62effd640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040d800 0x55c65041e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa62f7fe640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040cee0 0x55c65041de70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa620003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b5525cd991cc04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:03.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.768+0000 7fa60effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa60effd640 1 -- 172.21.15.33:0/1128141773 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62005a060 con 0x55c65040cee0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6280b0e80 con 0x55c65040cee0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa60effd640 1 -- 172.21.15.33:0/1128141773 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa6200316b0 con 0x55c65040cee0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa60effd640 1 -- 172.21.15.33:0/1128141773 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa620035040 con 0x55c65040cee0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 10 client.16713 fetch_fsmap learned FSMap version 53 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x55c650496d70 con 0x55c65040cee0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa60effd640 1 -- 172.21.15.33:0/1128141773 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6280b0e80 con 0x55c65040cee0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa60effd640 1 -- 172.21.15.33:0/1128141773 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7fa62005f070 con 0x55c65040cee0 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 10 client.16713 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 10 client.16713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 20 client.16713 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c6504971c0 0x55c6504995b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:03.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 4 client.16713 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:11:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.769+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c650381860 con 0x55c6504971c0 2024-09-10T21:11:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.770+0000 7fa62effd640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c6504971c0 0x55c6504995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.770+0000 7fa60effd640 10 client.16713 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.770+0000 7fa62effd640 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c6504971c0 0x55c6504995b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bee3d848458c6f74 server_cookie=1a95da135de9610f in_seq=0 out_seq=0 2024-09-10T21:11:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa60effd640 1 -- 172.21.15.33:0/1128141773 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c650381860 con 0x55c6504971c0 2024-09-10T21:11:03.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa60effd640 10 client.16713 handle_command_reply: tid=0 2024-09-10T21:11:03.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 client.16713 shutdown 2024-09-10T21:11:03.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040cee0 msgr2=0x55c65041de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040cee0 0x55c65041de70 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa620033000 tx=0x7fa620003910 comp rx=0 tx=0).stop 2024-09-10T21:11:03.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 shutdown_connections 2024-09-10T21:11:03.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c6504971c0 0x55c6504995b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c650419790 0x55c650495640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c65040d800 0x55c65041e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 --2- 172.21.15.33:0/1128141773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c65040cee0 0x55c65041de70 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 >> 172.21.15.33:0/1128141773 conn(0x55c65040ad10 msgr2=0x55c650409f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa60effd640 0 client.16713 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 shutdown_connections 2024-09-10T21:11:03.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 1 -- 172.21.15.33:0/1128141773 wait complete. 2024-09-10T21:11:03.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.772+0000 7fa63842a740 20 client.16713 trim_cache size 0 max 16384 2024-09-10T21:11:03.784 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:11:03.784 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:11:03.784 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0038ceab-3214-49d2-b703-dad66105eb96", 2024-09-10T21:11:03.785 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:11:03.785 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:11:03.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b52e0 msgr2=0x7fa6180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b52e0 0x7fa6180b77a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x55c65040ce90 tx=0x7fa624012040 comp rx=0 tx=0).stop 2024-09-10T21:11:03.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d180 msgr2=0x7fa63019eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d180 0x7fa63019eab0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa62800ccb0 tx=0x7fa62800a620 comp rx=0 tx=0).stop 2024-09-10T21:11:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 shutdown_connections 2024-09-10T21:11:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa6180b52e0 0x7fa6180b77a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa63010d180 0x7fa63019eab0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6301045d0 0x7fa630199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 --2- 172.21.15.33:0/3051292296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa630103bd0 0x7fa6301991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 >> 172.21.15.33:0/3051292296 conn(0x7fa6300fd700 msgr2=0x7fa63010f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 shutdown_connections 2024-09-10T21:11:03.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:03.773+0000 7fa636055640 1 -- 172.21.15.33:0/3051292296 wait complete. 2024-09-10T21:11:04.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:03 smithi104 ceph-mon[35325]: pgmap v558: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 890 KiB/s wr, 81 op/s 2024-09-10T21:11:04.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:03 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:04.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:03 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:04.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:03 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:04.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:03 smithi174 ceph-mon[35635]: pgmap v558: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 890 KiB/s wr, 81 op/s 2024-09-10T21:11:04.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:03 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:04.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:03 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:04.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:03 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:04.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:03 smithi033 ceph-mon[30860]: pgmap v558: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 890 KiB/s wr, 81 op/s 2024-09-10T21:11:04.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:03 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:04.420 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:03 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:04.421 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:03 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:04.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:11:04.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 520108 6.19 MB/sec execute 245 sec latency 1703.046 ms 2024-09-10T21:11:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:11:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:05.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:04 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.286+0000 7fdca877b640 1 -- 172.21.15.33:0/4061512190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0108020 msgr2=0x7fdca010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.286+0000 7fdca877b640 1 --2- 172.21.15.33:0/4061512190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0108020 0x7fdca010e820 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009900 tx=0x7fdc9802a410 comp rx=0 tx=0).stop 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 -- 172.21.15.33:0/4061512190 shutdown_connections 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 --2- 172.21.15.33:0/4061512190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0108020 0x7fdca010e820 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 --2- 172.21.15.33:0/4061512190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca01076a0 0x7fdca0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 --2- 172.21.15.33:0/4061512190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca0101c90 0x7fdca0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 -- 172.21.15.33:0/4061512190 >> 172.21.15.33:0/4061512190 conn(0x7fdca00fd700 msgr2=0x7fdca00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 -- 172.21.15.33:0/4061512190 shutdown_connections 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 -- 172.21.15.33:0/4061512190 wait complete. 2024-09-10T21:11:05.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 Processor -- start 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.287+0000 7fdca877b640 1 -- start start 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca877b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0101c90 0x7fdca019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca877b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca01076a0 0x7fdca019b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca877b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0108020 0x7fdca01a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca877b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111ca0 con 0x7fdca0101c90 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca877b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111b20 con 0x7fdca0108020 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca877b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdca0111e20 con 0x7fdca01076a0 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0101c90 0x7fdca019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0101c90 0x7fdca019b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55798/0 (socket says 172.21.15.33:55798) 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 -- 172.21.15.33:0/1975709547 learned_addr learned my addr 172.21.15.33:0/1975709547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca5d16640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca01076a0 0x7fdca019b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6d18640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0108020 0x7fdca01a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 -- 172.21.15.33:0/1975709547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca01076a0 msgr2=0x7fdca019b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca01076a0 0x7fdca019b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 -- 172.21.15.33:0/1975709547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0108020 msgr2=0x7fdca01a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0108020 0x7fdca01a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 -- 172.21.15.33:0/1975709547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca01a1550 con 0x7fdca0101c90 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca5d16640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca01076a0 0x7fdca019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdca6517640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0101c90 0x7fdca019b4a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00ebc0 tx=0x7fdc9c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.288+0000 7fdc977fe640 1 -- 172.21.15.33:0/1975709547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c010630 con 0x7fdca0101c90 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.289+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca01a1840 con 0x7fdca0101c90 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.289+0000 7fdc977fe640 1 -- 172.21.15.33:0/1975709547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9c010c80 con 0x7fdca0101c90 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.289+0000 7fdc977fe640 1 -- 172.21.15.33:0/1975709547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c018b60 con 0x7fdca0101c90 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.289+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca01a80d0 con 0x7fdca0101c90 2024-09-10T21:11:05.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.291+0000 7fdc977fe640 1 -- 172.21.15.33:0/1975709547 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdc9c0107d0 con 0x7fdca0101c90 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.292+0000 7fdc977fe640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdc8c0b5320 0x7fdc8c0b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.292+0000 7fdcaab50740 1 Processor -- start 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.292+0000 7fdca5d16640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdc8c0b5320 0x7fdc8c0b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.292+0000 7fdc977fe640 1 -- 172.21.15.33:0/1975709547 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdc9c0e61c0 con 0x7fdca0101c90 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdcaab50740 1 -- start start 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0eba5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebb17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdcaab50740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564d0eb1be50 con 0x564d0eba5da0 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdcaab50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564d0eb0b280 con 0x564d0ebb1d30 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdcaab50740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d0eb0f090 con 0x564d0eba5480 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebb17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca6d18640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebb17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55800/0 (socket says 172.21.15.33:55800) 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 -- 172.21.15.33:0/2164504742 learned_addr learned my addr 172.21.15.33:0/2164504742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca6517640 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0eba5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdc8c0b5320 0x7fdc8c0b77e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdca010ec10 tx=0x7fdc90005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 -- 172.21.15.33:0/2164504742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 msgr2=0x564d0eba5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0eba5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 -- 172.21.15.33:0/2164504742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 msgr2=0x564d0ebb4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca5d16640 1 -- 172.21.15.33:0/2164504742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0e869650 con 0x564d0eba5da0 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.293+0000 7fdca6d18640 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.294+0000 7fdca5d16640 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebb17f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdc900042c0 tx=0x7fdc90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98a101d282a3a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.294+0000 7fdc94ff9640 1 -- 172.21.15.33:0/2164504742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90026470 con 0x564d0eba5da0 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.294+0000 7fdc94ff9640 1 -- 172.21.15.33:0/2164504742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9001d8a0 con 0x564d0eba5da0 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.294+0000 7fdc94ff9640 1 -- 172.21.15.33:0/2164504742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9001c960 con 0x564d0eba5da0 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.294+0000 7fdcaab50740 1 -- 172.21.15.33:0/2164504742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 msgr2=0x564d0ebb17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.294+0000 7fdcaab50740 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebb17f0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fdc900042c0 tx=0x7fdc90014000 comp rx=0 tx=0).stop 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 -- 172.21.15.33:0/2164504742 shutdown_connections 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebb17f0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 --2- 172.21.15.33:0/2164504742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0eba5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 -- 172.21.15.33:0/2164504742 >> 172.21.15.33:0/2164504742 conn(0x564d0eba32b0 msgr2=0x564d0eba4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 -- 172.21.15.33:0/2164504742 shutdown_connections 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 -- 172.21.15.33:0/2164504742 wait complete. 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 -- start start 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.295+0000 7fdcaab50740 1 Processor -- start 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebb16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebaeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebaf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdcaab50740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564d0e96c700 con 0x564d0eba5da0 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdcaab50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564d0eaa6110 con 0x564d0ebb1d30 2024-09-10T21:11:05.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdcaab50740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564d0ebb49b0 con 0x564d0eba5480 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdcaab50740 10 client.0 _ll_register_callbacks cb 0x564d0eabf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca6517640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebb16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca5d16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebaeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca6517640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebb16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:38956/0 (socket says 172.21.15.33:38956) 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca6517640 1 -- 172.21.15.33:0/1211774915 learned_addr learned my addr 172.21.15.33:0/1211774915 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca6d18640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebaf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca5d16640 1 -- 172.21.15.33:0/1211774915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 msgr2=0x564d0ebb16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca5d16640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebb16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca5d16640 1 -- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 msgr2=0x564d0ebaf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca5d16640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebaf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca5d16640 1 -- 172.21.15.33:0/1211774915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0ead3c40 con 0x564d0eba5da0 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca6d18640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebaf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:05.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca6517640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebb16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdca5d16640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebaeaf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdc90004000 tx=0x7fdc9001c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a4ad180831f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.296+0000 7fdc89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdc89ffb640 1 -- 172.21.15.33:0/1211774915 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9001dc50 con 0x564d0eba5da0 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdc89ffb640 1 -- 172.21.15.33:0/1211774915 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdc9001c6a0 con 0x564d0eba5da0 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc9001d8a0 con 0x564d0eba5da0 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdc89ffb640 1 -- 172.21.15.33:0/1211774915 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdc9001ddf0 con 0x564d0eba5da0 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdcaab50740 10 client.16725 fetch_fsmap learned FSMap version 53 2024-09-10T21:11:05.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564d0ebc0390 con 0x564d0eba5da0 2024-09-10T21:11:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdc89ffb640 1 -- 172.21.15.33:0/1211774915 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc9001d8a0 con 0x564d0eba5da0 2024-09-10T21:11:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdc89ffb640 1 -- 172.21.15.33:0/1211774915 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7fdc90037050 con 0x564d0eba5da0 2024-09-10T21:11:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdcaab50740 10 client.16725 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:11:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.297+0000 7fdcaab50740 10 client.16725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:05.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.298+0000 7fdcaab50740 20 client.16725 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.298+0000 7fdcaab50740 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ebc16e0 0x564d0ebc3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.298+0000 7fdcaab50740 4 client.16725 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.298+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d0eb1dda0 con 0x564d0ebc16e0 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.298+0000 7fdca6517640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ebc16e0 0x564d0ebc3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.298+0000 7fdca6517640 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ebc16e0 0x564d0ebc3ad0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=badcb0f6f25c5943 server_cookie=2fedd4a60e911e72 in_seq=0 out_seq=0 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.298+0000 7fdc89ffb640 10 client.16725 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.299+0000 7fdc89ffb640 1 -- 172.21.15.33:0/1211774915 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564d0eb1dda0 con 0x564d0ebc16e0 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.299+0000 7fdc89ffb640 10 client.16725 handle_command_reply: tid=0 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.299+0000 7fdcaab50740 1 client.16725 shutdown 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.299+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 msgr2=0x564d0ebaeaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.299+0000 7fdcaab50740 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebaeaf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fdc90004000 tx=0x7fdc9001c5c0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.299+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 shutdown_connections 2024-09-10T21:11:05.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ebc16e0 0x564d0ebc3ad0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebaf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebaeaf0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdc89ffb640 0 client.16725 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 1 --2- 172.21.15.33:0/1211774915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebb16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 >> 172.21.15.33:0/1211774915 conn(0x564d0eba32b0 msgr2=0x564d0eba8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 shutdown_connections 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 1 -- 172.21.15.33:0/1211774915 wait complete. 2024-09-10T21:11:05.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.300+0000 7fdcaab50740 20 client.16725 trim_cache size 0 max 16384 2024-09-10T21:11:05.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:05.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:05.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:04 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.318+0000 7fdcaab50740 1 Processor -- start 2024-09-10T21:11:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.318+0000 7fdcaab50740 1 -- start start 2024-09-10T21:11:05.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.318+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebbdc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.318+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebadfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdcaab50740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01b06d0 con 0x564d0eba5da0 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdcaab50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01b2300 con 0x564d0ebb1d30 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdcaab50740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01b0da0 con 0x564d0eba5480 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebadfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebadfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55808/0 (socket says 172.21.15.33:55808) 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 -- 172.21.15.33:0/3789826441 learned_addr learned my addr 172.21.15.33:0/3789826441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca6d18640 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca6517640 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebbdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 -- 172.21.15.33:0/3789826441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 msgr2=0x564d0ebbdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebbdc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 -- 172.21.15.33:0/3789826441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 msgr2=0x564d0ebb3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 -- 172.21.15.33:0/3789826441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d0ebacff0 con 0x564d0eba5da0 2024-09-10T21:11:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdca5d16640 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebadfe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011be0 tx=0x7fdc90026c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4589cb1d9cc61cf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdc8bfff640 1 -- 172.21.15.33:0/3789826441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90003e50 con 0x564d0eba5da0 2024-09-10T21:11:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdc8bfff640 1 -- 172.21.15.33:0/3789826441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc90005c50 con 0x564d0eba5da0 2024-09-10T21:11:05.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.319+0000 7fdc8bfff640 1 -- 172.21.15.33:0/3789826441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc90005540 con 0x564d0eba5da0 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 -- 172.21.15.33:0/3789826441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 msgr2=0x564d0ebadfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebadfe0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fdc90011be0 tx=0x7fdc90026c30 comp rx=0 tx=0).stop 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 -- 172.21.15.33:0/3789826441 shutdown_connections 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebb3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5da0 0x564d0ebadfe0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 --2- 172.21.15.33:0/3789826441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5480 0x564d0ebbdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 -- 172.21.15.33:0/3789826441 >> 172.21.15.33:0/3789826441 conn(0x564d0eba32b0 msgr2=0x564d0ebad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 -- 172.21.15.33:0/3789826441 shutdown_connections 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.320+0000 7fdcaab50740 1 -- 172.21.15.33:0/3789826441 wait complete. 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 -- start start 2024-09-10T21:11:05.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 Processor -- start 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5480 0x564d0ebba520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5da0 0x564d0ebc8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebc94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01b1b70 con 0x564d0eba5480 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01b1470 con 0x564d0ebb1d30 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdca01ae690 con 0x564d0eba5da0 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdcaab50740 10 client.0 _ll_register_callbacks cb 0x7fdc9c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdca6517640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5480 0x564d0ebba520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdca6517640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5480 0x564d0ebba520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55820/0 (socket says 172.21.15.33:55820) 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdca6517640 1 -- 172.21.15.33:0/4279161388 learned_addr learned my addr 172.21.15.33:0/4279161388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdca6d18640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebc94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdca5d16640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5da0 0x564d0ebc8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdca6517640 1 -- 172.21.15.33:0/4279161388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5da0 msgr2=0x564d0ebc8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.321+0000 7fdca6517640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5da0 0x564d0ebc8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdca6517640 1 -- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 msgr2=0x564d0ebc94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdca6517640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebc94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdca6517640 1 -- 172.21.15.33:0/4279161388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d0ebb5f90 con 0x564d0eba5480 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdca6d18640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebc94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdca6517640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5480 0x564d0ebba520 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a9c60 tx=0x7fdc9c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dec4a883072dc4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdc89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdc89ffb640 1 -- 172.21.15.33:0/4279161388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0a8160 con 0x564d0eba5480 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdc89ffb640 1 -- 172.21.15.33:0/4279161388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdc9c0abc50 con 0x564d0eba5480 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc90005c50 con 0x564d0eba5480 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdc89ffb640 1 -- 172.21.15.33:0/4279161388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fdc9c018420 con 0x564d0eba5480 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdcaab50740 10 client.16731 fetch_fsmap learned FSMap version 53 2024-09-10T21:11:05.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) -- 0x564d0ec2e4e0 con 0x564d0eba5480 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.322+0000 7fdc89ffb640 1 -- 172.21.15.33:0/4279161388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc90005c50 con 0x564d0eba5480 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdc89ffb640 1 -- 172.21.15.33:0/4279161388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 53) ==== 8386+0+0 (secure 0 0 0) 0x7fdc9c0ab6c0 con 0x564d0eba5480 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdcaab50740 10 client.16731 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdcaab50740 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdcaab50740 20 client.16731 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdcaab50740 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ec2eca0 0x564d0ec310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdcaab50740 4 client.16731 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d0eb19850 con 0x564d0ec2eca0 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdca5d16640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ec2eca0 0x564d0ec310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdca5d16640 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ec2eca0 0x564d0ec310c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=574bcc310567794e server_cookie=349548879748dfb in_seq=0 out_seq=0 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.323+0000 7fdc89ffb640 10 client.16731 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdc89ffb640 1 -- 172.21.15.33:0/4279161388 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x564d0eb19850 con 0x564d0ec2eca0 2024-09-10T21:11:05.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdc89ffb640 10 client.16731 handle_command_reply: tid=0 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 client.16731 shutdown 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5480 msgr2=0x564d0ebba520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5480 0x564d0ebba520 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0a9c60 tx=0x7fdc9c000f40 comp rx=0 tx=0).stop 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 shutdown_connections 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564d0ec2eca0 0x564d0ec310c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564d0ebb1d30 0x564d0ebc94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564d0eba5da0 0x564d0ebc8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 --2- 172.21.15.33:0/4279161388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564d0eba5480 0x564d0ebba520 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdc89ffb640 0 client.16731 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 >> 172.21.15.33:0/4279161388 conn(0x564d0eba32b0 msgr2=0x564d0eba3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 shutdown_connections 2024-09-10T21:11:05.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 1 -- 172.21.15.33:0/4279161388 wait complete. 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdcaab50740 20 client.16731 trim_cache size 0 max 16384 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0038ceab-3214-49d2-b703-dad66105eb96", 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:05.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.337 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.338 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:11:05.339 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:05.341 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:05.342 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:05.342 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:11:05.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdc8c0b5320 msgr2=0x7fdc8c0b77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdca877b640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdc8c0b5320 0x7fdc8c0b77e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdca010ec10 tx=0x7fdc90005f90 comp rx=0 tx=0).stop 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0101c90 msgr2=0x7fdca019b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.324+0000 7fdca877b640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0101c90 0x7fdca019b4a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00ebc0 tx=0x7fdc9c00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 shutdown_connections 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdc8c0b5320 0x7fdc8c0b77e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdca0108020 0x7fdca01a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdca01076a0 0x7fdca019b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 --2- 172.21.15.33:0/1975709547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca0101c90 0x7fdca019b4a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 >> 172.21.15.33:0/1975709547 conn(0x7fdca00fd700 msgr2=0x7fdca006ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 shutdown_connections 2024-09-10T21:11:05.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:05.325+0000 7fdca877b640 1 -- 172.21.15.33:0/1975709547 wait complete. 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:05.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:04 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:05.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522304 6.22 MB/sec execute 246 sec latency 182.636 ms 2024-09-10T21:11:06.033 INFO:tasks.cephfs.filesystem:scrub status for tag:0038ceab-3214-49d2-b703-dad66105eb96 - {'path': '/', 'tag': '0038ceab-3214-49d2-b703-dad66105eb96', 'options': 'recursive,force'} 2024-09-10T21:11:06.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:05 smithi104 ceph-mon[35325]: pgmap v559: 97 pgs: 97 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-10T21:11:06.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:05 smithi174 ceph-mon[35635]: pgmap v559: 97 pgs: 97 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-10T21:11:06.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:05 smithi033 ceph-mon[30860]: pgmap v559: 97 pgs: 97 active+clean; 203 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 144 op/s 2024-09-10T21:11:06.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522304 6.20 MB/sec execute 247 sec latency 1182.733 ms 2024-09-10T21:11:07.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:06 smithi104 ceph-mon[35325]: pgmap v560: 97 pgs: 97 active+clean; 201 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-09-10T21:11:07.329 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:06 smithi033 ceph-mon[30860]: pgmap v560: 97 pgs: 97 active+clean; 201 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-09-10T21:11:07.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:06 smithi174 ceph-mon[35635]: pgmap v560: 97 pgs: 97 active+clean; 201 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 1.6 MiB/s wr, 118 op/s 2024-09-10T21:11:07.735 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522355 6.17 MB/sec execute 248 sec latency 1551.265 ms 2024-09-10T21:11:08.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522355 6.15 MB/sec execute 249 sec latency 1587.775 ms 2024-09-10T21:11:09.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:08 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:09.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:08 smithi104 ceph-mon[35325]: pgmap v561: 97 pgs: 97 active+clean; 201 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-10T21:11:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:08 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:08 smithi174 ceph-mon[35635]: pgmap v561: 97 pgs: 97 active+clean; 201 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-10T21:11:09.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:08 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:09.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:08 smithi033 ceph-mon[30860]: pgmap v561: 97 pgs: 97 active+clean; 201 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 986 KiB/s rd, 1.6 MiB/s wr, 132 op/s 2024-09-10T21:11:09.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522355 6.12 MB/sec execute 250 sec latency 2588.163 ms 2024-09-10T21:11:10.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522355 6.10 MB/sec execute 251 sec latency 3588.243 ms 2024-09-10T21:11:11.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:11 smithi104 ceph-mon[35325]: pgmap v562: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-09-10T21:11:11.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:11 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:11.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:11 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:11.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:11 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:11 smithi174 ceph-mon[35635]: pgmap v562: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-09-10T21:11:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:11 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:11 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:11.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:11 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:11 smithi033 ceph-mon[30860]: pgmap v562: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 957 KiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-09-10T21:11:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:11 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:11 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:11 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:11.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 522355 6.07 MB/sec execute 252 sec latency 4588.374 ms 2024-09-10T21:11:12.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:12 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:12.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:12 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:12 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:12.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:12 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:12 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:12.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:12 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:12.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 523116 6.06 MB/sec execute 253 sec latency 4952.046 ms 2024-09-10T21:11:13.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:13 smithi104 ceph-mon[35325]: pgmap v563: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-10T21:11:13.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:13 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:13 smithi174 ceph-mon[35635]: pgmap v563: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-10T21:11:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:13 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:13.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:13 smithi033 ceph-mon[30860]: pgmap v563: 97 pgs: 97 active+clean; 202 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 1.5 MiB/s wr, 140 op/s 2024-09-10T21:11:13.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:13 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:13.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 525879 6.10 MB/sec execute 254 sec latency 10.830 ms 2024-09-10T21:11:14.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 527468 6.10 MB/sec execute 255 sec latency 12.557 ms 2024-09-10T21:11:15.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:15 smithi104 ceph-mon[35325]: pgmap v564: 97 pgs: 97 active+clean; 203 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-09-10T21:11:15.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:15 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:11:15.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:15 smithi174 ceph-mon[35635]: pgmap v564: 97 pgs: 97 active+clean; 203 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-09-10T21:11:15.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:15 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:11:15.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:15 smithi033 ceph-mon[30860]: pgmap v564: 97 pgs: 97 active+clean; 203 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 1.1 MiB/s wr, 158 op/s 2024-09-10T21:11:15.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:15 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:11:15.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529395 6.13 MB/sec execute 256 sec latency 351.636 ms 2024-09-10T21:11:16.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.563 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: Unable to set osd_memory_target on smithi104 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: Unable to set osd_memory_target on smithi174 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: Unable to set osd_memory_target on smithi033 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:11:16.564 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:11:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: Unable to set osd_memory_target on smithi104 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-10T21:11:16.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: Unable to set osd_memory_target on smithi174 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-10T21:11:16.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: Unable to set osd_memory_target on smithi033 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-10T21:11:16.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:11:16.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:11:16.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:11:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: Unable to set osd_memory_target on smithi104 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-10T21:11:16.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: Unable to set osd_memory_target on smithi174 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-10T21:11:16.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: Unable to set osd_memory_target on smithi033 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-10T21:11:16.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:11:16.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:11:16.679 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:11:16.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529395 6.10 MB/sec execute 257 sec latency 1351.741 ms 2024-09-10T21:11:17.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529465 6.08 MB/sec execute 258 sec latency 1718.786 ms 2024-09-10T21:11:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:17 smithi033 ceph-mon[30860]: pgmap v565: 97 pgs: 97 active+clean; 203 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 355 KiB/s wr, 92 op/s 2024-09-10T21:11:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:17 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:18.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:17 smithi104 ceph-mon[35325]: pgmap v565: 97 pgs: 97 active+clean; 203 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 355 KiB/s wr, 92 op/s 2024-09-10T21:11:18.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:17 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:17 smithi174 ceph-mon[35635]: pgmap v565: 97 pgs: 97 active+clean; 203 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 708 KiB/s rd, 355 KiB/s wr, 92 op/s 2024-09-10T21:11:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:17 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:18.736 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529465 6.06 MB/sec execute 259 sec latency 1559.831 ms 2024-09-10T21:11:19.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529465 6.03 MB/sec execute 260 sec latency 2559.890 ms 2024-09-10T21:11:19.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:19 smithi033 ceph-mon[30860]: pgmap v566: 97 pgs: 97 active+clean; 204 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 443 KiB/s wr, 106 op/s 2024-09-10T21:11:20.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:19 smithi104 ceph-mon[35325]: pgmap v566: 97 pgs: 97 active+clean; 204 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 443 KiB/s wr, 106 op/s 2024-09-10T21:11:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:19 smithi174 ceph-mon[35635]: pgmap v566: 97 pgs: 97 active+clean; 204 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 443 KiB/s wr, 106 op/s 2024-09-10T21:11:20.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529465 6.01 MB/sec execute 261 sec latency 3559.987 ms 2024-09-10T21:11:21.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 529465 5.99 MB/sec execute 262 sec latency 4560.095 ms 2024-09-10T21:11:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:21 smithi033 ceph-mon[30860]: pgmap v567: 97 pgs: 97 active+clean; 205 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 465 KiB/s wr, 112 op/s 2024-09-10T21:11:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:21 smithi104 ceph-mon[35325]: pgmap v567: 97 pgs: 97 active+clean; 205 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 465 KiB/s wr, 112 op/s 2024-09-10T21:11:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:21 smithi174 ceph-mon[35635]: pgmap v567: 97 pgs: 97 active+clean; 205 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 465 KiB/s wr, 112 op/s 2024-09-10T21:11:22.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 530220 5.97 MB/sec execute 263 sec latency 4924.395 ms 2024-09-10T21:11:23.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:22 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:22 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:23.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:22 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:23.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 532973 6.01 MB/sec execute 264 sec latency 22.089 ms 2024-09-10T21:11:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:23 smithi104 ceph-mon[35325]: pgmap v568: 97 pgs: 97 active+clean; 205 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 375 KiB/s wr, 88 op/s 2024-09-10T21:11:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:23 smithi174 ceph-mon[35635]: pgmap v568: 97 pgs: 97 active+clean; 205 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 375 KiB/s wr, 88 op/s 2024-09-10T21:11:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:23 smithi033 ceph-mon[30860]: pgmap v568: 97 pgs: 97 active+clean; 205 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 711 KiB/s rd, 375 KiB/s wr, 88 op/s 2024-09-10T21:11:24.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 532973 5.99 MB/sec execute 265 sec latency 1022.235 ms 2024-09-10T21:11:25.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 532973 5.97 MB/sec execute 266 sec latency 2022.340 ms 2024-09-10T21:11:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:25 smithi104 ceph-mon[35325]: pgmap v569: 97 pgs: 97 active+clean; 206 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 482 KiB/s wr, 117 op/s 2024-09-10T21:11:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:25 smithi174 ceph-mon[35635]: pgmap v569: 97 pgs: 97 active+clean; 206 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 482 KiB/s wr, 117 op/s 2024-09-10T21:11:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:25 smithi033 ceph-mon[30860]: pgmap v569: 97 pgs: 97 active+clean; 206 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 482 KiB/s wr, 117 op/s 2024-09-10T21:11:26.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 532973 5.95 MB/sec execute 267 sec latency 3022.401 ms 2024-09-10T21:11:27.737 INFO:tasks.workunit.client.0.smithi033.stdout: 1 533668 5.93 MB/sec execute 268 sec latency 3387.354 ms 2024-09-10T21:11:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:27 smithi104 ceph-mon[35325]: pgmap v570: 97 pgs: 97 active+clean; 207 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 419 KiB/s wr, 87 op/s 2024-09-10T21:11:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:27 smithi174 ceph-mon[35635]: pgmap v570: 97 pgs: 97 active+clean; 207 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 419 KiB/s wr, 87 op/s 2024-09-10T21:11:28.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:27 smithi033 ceph-mon[30860]: pgmap v570: 97 pgs: 97 active+clean; 207 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 803 KiB/s rd, 419 KiB/s wr, 87 op/s 2024-09-10T21:11:28.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 536414 5.97 MB/sec execute 269 sec latency 13.657 ms 2024-09-10T21:11:29.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 536520 5.95 MB/sec execute 270 sec latency 945.901 ms 2024-09-10T21:11:29.946 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:29 smithi174 ceph-mon[35635]: pgmap v571: 97 pgs: 97 active+clean; 207 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 429 KiB/s wr, 96 op/s 2024-09-10T21:11:30.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:29 smithi104 ceph-mon[35325]: pgmap v571: 97 pgs: 97 active+clean; 207 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 429 KiB/s wr, 96 op/s 2024-09-10T21:11:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:29 smithi033 ceph-mon[30860]: pgmap v571: 97 pgs: 97 active+clean; 207 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 786 KiB/s rd, 429 KiB/s wr, 96 op/s 2024-09-10T21:11:30.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 536520 5.93 MB/sec execute 271 sec latency 1945.994 ms 2024-09-10T21:11:31.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 536520 5.91 MB/sec execute 272 sec latency 2946.093 ms 2024-09-10T21:11:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:31 smithi104 ceph-mon[35325]: pgmap v572: 97 pgs: 97 active+clean; 208 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 390 KiB/s wr, 92 op/s 2024-09-10T21:11:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:31 smithi174 ceph-mon[35635]: pgmap v572: 97 pgs: 97 active+clean; 208 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 390 KiB/s wr, 92 op/s 2024-09-10T21:11:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:31 smithi033 ceph-mon[30860]: pgmap v572: 97 pgs: 97 active+clean; 208 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 390 KiB/s wr, 92 op/s 2024-09-10T21:11:32.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 537211 5.89 MB/sec execute 273 sec latency 3309.963 ms 2024-09-10T21:11:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:32 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:32 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:33.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:32 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:11:33.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 540046 5.93 MB/sec execute 274 sec latency 13.630 ms 2024-09-10T21:11:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:33 smithi104 ceph-mon[35325]: pgmap v573: 97 pgs: 97 active+clean; 208 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 323 KiB/s wr, 75 op/s 2024-09-10T21:11:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:33 smithi174 ceph-mon[35635]: pgmap v573: 97 pgs: 97 active+clean; 208 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 323 KiB/s wr, 75 op/s 2024-09-10T21:11:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:33 smithi033 ceph-mon[30860]: pgmap v573: 97 pgs: 97 active+clean; 208 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 323 KiB/s wr, 75 op/s 2024-09-10T21:11:34.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 540092 5.91 MB/sec execute 275 sec latency 963.000 ms 2024-09-10T21:11:35.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 540092 5.89 MB/sec execute 276 sec latency 1963.118 ms 2024-09-10T21:11:36.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:11:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:35 smithi104 ceph-mon[35325]: pgmap v574: 97 pgs: 97 active+clean; 209 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 543 KiB/s wr, 107 op/s 2024-09-10T21:11:36.063 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:35 smithi033 ceph-mon[30860]: pgmap v574: 97 pgs: 97 active+clean; 209 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 543 KiB/s wr, 107 op/s 2024-09-10T21:11:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:35 smithi174 ceph-mon[35635]: pgmap v574: 97 pgs: 97 active+clean; 209 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 543 KiB/s wr, 107 op/s 2024-09-10T21:11:36.320 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:36.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 540092 5.87 MB/sec execute 277 sec latency 2963.154 ms 2024-09-10T21:11:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 -- 172.21.15.33:0/3808808661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f228810d100 msgr2=0x7f228810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 --2- 172.21.15.33:0/3808808661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f228810d100 0x7f228810f550 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f2280009ca0 tx=0x7f228002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 -- 172.21.15.33:0/3808808661 shutdown_connections 2024-09-10T21:11:36.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 --2- 172.21.15.33:0/3808808661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f228810d100 0x7f228810f550 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 --2- 172.21.15.33:0/3808808661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288104550 0x7f228810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 --2- 172.21.15.33:0/3808808661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2288103b50 0x7f2288103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 -- 172.21.15.33:0/3808808661 >> 172.21.15.33:0/3808808661 conn(0x7f22880fd700 msgr2=0x7f22880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 -- 172.21.15.33:0/3808808661 shutdown_connections 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.808+0000 7f228d01d640 1 -- 172.21.15.33:0/3808808661 wait complete. 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 Processor -- start 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 -- start start 2024-09-10T21:11:36.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288103b50 0x7f22881992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2288104550 0x7f22881997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f228810d100 0x7f228819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2288112dc0 con 0x7f2288104550 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2288112c40 con 0x7f228810d100 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228d01d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2288112f40 con 0x7f2288103b50 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2288104550 0x7f22881997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2288104550 0x7f22881997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55628/0 (socket says 172.21.15.33:55628) 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f228659c640 1 -- 172.21.15.33:0/2592704525 learned_addr learned my addr 172.21.15.33:0/2592704525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.809+0000 7f2286d9d640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288103b50 0x7f22881992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228759e640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f228810d100 0x7f228819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228659c640 1 -- 172.21.15.33:0/2592704525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288103b50 msgr2=0x7f22881992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228659c640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288103b50 0x7f22881992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228659c640 1 -- 172.21.15.33:0/2592704525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f228810d100 msgr2=0x7f228819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228659c640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f228810d100 0x7f228819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228659c640 1 -- 172.21.15.33:0/2592704525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228819f350 con 0x7f2288104550 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228659c640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2288104550 0x7f22881997e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f227c00b4a0 tx=0x7f227c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f226ffff640 1 -- 172.21.15.33:0/2592704525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c014070 con 0x7f2288104550 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f226ffff640 1 -- 172.21.15.33:0/2592704525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f227c010ac0 con 0x7f2288104550 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228819f640 con 0x7f2288104550 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f226ffff640 1 -- 172.21.15.33:0/2592704525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c017c30 con 0x7f2288104550 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.810+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22881a5d80 con 0x7f2288104550 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.814+0000 7f228f3f2740 1 Processor -- start 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.814+0000 7f228f3f2740 1 -- start start 2024-09-10T21:11:36.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.814+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb892f90a0 0x55fb892f9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb89303400 0x55fb892f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb893103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f228f3f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb8926fe50 con 0x55fb892f90a0 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f228f3f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fb8925f280 con 0x55fb89303400 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f228f3f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb89263090 con 0x55fb8930e000 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb89303400 0x55fb892f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f228759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb892f90a0 0x55fb892f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb89303400 0x55fb892f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:45488/0 (socket says 172.21.15.33:45488) 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 -- 172.21.15.33:0/3446390665 learned_addr learned my addr 172.21.15.33:0/3446390665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f226ffff640 1 -- 172.21.15.33:0/2592704525 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f227c010610 con 0x7f2288104550 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 -- 172.21.15.33:0/3446390665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 msgr2=0x55fb893103f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 --2- 172.21.15.33:0/3446390665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb893103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 -- 172.21.15.33:0/3446390665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb892f90a0 msgr2=0x55fb892f9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 --2- 172.21.15.33:0/3446390665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb892f90a0 0x55fb892f9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.815+0000 7f2286d9d640 1 -- 172.21.15.33:0/3446390665 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb88fbd650 con 0x55fb89303400 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.816+0000 7f2286d9d640 1 --2- 172.21.15.33:0/3446390665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb89303400 0x55fb892f99c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f227800ac10 tx=0x7f227800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80adae8412c1146 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.816+0000 7f226d7fa640 1 -- 172.21.15.33:0/3446390665 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227801d070 con 0x55fb89303400 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.816+0000 7f226d7fa640 1 -- 172.21.15.33:0/3446390665 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2278018af0 con 0x55fb89303400 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.816+0000 7f226ffff640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22700b52e0 0x7f22700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.816+0000 7f228759e640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22700b52e0 0x7f22700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.816+0000 7f226d7fa640 1 -- 172.21.15.33:0/3446390665 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278010a60 con 0x55fb89303400 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f226ffff640 1 -- 172.21.15.33:0/2592704525 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f227c0e5500 con 0x7f2288104550 2024-09-10T21:11:36.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 -- 172.21.15.33:0/3446390665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb89303400 msgr2=0x55fb892f99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3446390665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb89303400 0x55fb892f99c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f227800ac10 tx=0x7f227800f790 comp rx=0 tx=0).stop 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 -- 172.21.15.33:0/3446390665 shutdown_connections 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3446390665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb893103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3446390665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb89303400 0x55fb892f99c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3446390665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb892f90a0 0x55fb892f9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 -- 172.21.15.33:0/3446390665 >> 172.21.15.33:0/3446390665 conn(0x55fb892f6ed0 msgr2=0x55fb892f8070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228f3f2740 1 -- 172.21.15.33:0/3446390665 shutdown_connections 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.817+0000 7f228759e640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22700b52e0 0x7f22700b77a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2280005fa0 tx=0x7f2280005c30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.818+0000 7f228f3f2740 1 -- 172.21.15.33:0/3446390665 wait complete. 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.818+0000 7f228f3f2740 1 -- start start 2024-09-10T21:11:36.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.818+0000 7f228f3f2740 1 Processor -- start 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb89302cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb89300100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb893006c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228f3f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb890c0700 con 0x55fb89303400 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228f3f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fb891fa110 con 0x55fb892f90a0 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228f3f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb89310e40 con 0x55fb8930e000 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228f3f2740 10 client.0 _ll_register_callbacks cb 0x55fb89213990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb89302cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb89302cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:45500/0 (socket says 172.21.15.33:45500) 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 -- 172.21.15.33:0/2226590609 learned_addr learned my addr 172.21.15.33:0/2226590609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228759e640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb893006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228659c640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb89300100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 -- 172.21.15.33:0/2226590609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 msgr2=0x55fb893006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb893006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 -- 172.21.15.33:0/2226590609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 msgr2=0x55fb89300100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb89300100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f2286d9d640 1 -- 172.21.15.33:0/2226590609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb89227c40 con 0x55fb892f90a0 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.819+0000 7f228659c640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb89300100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f2286d9d640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb89302cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f227800abe0 tx=0x7f22780186d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50ff7f5ca8087feb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f226aff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f226aff5640 1 -- 172.21.15.33:0/2226590609 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278027020 con 0x55fb892f90a0 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2278018af0 con 0x55fb892f90a0 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f226aff5640 1 -- 172.21.15.33:0/2226590609 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2278005040 con 0x55fb892f90a0 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f226aff5640 1 -- 172.21.15.33:0/2226590609 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f227801d040 con 0x55fb892f90a0 2024-09-10T21:11:36.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f228f3f2740 10 client.26237 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.820+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fb8931c7b0 con 0x55fb892f90a0 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.821+0000 7f226aff5640 1 -- 172.21.15.33:0/2226590609 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2278018af0 con 0x55fb892f90a0 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f226aff5640 1 -- 172.21.15.33:0/2226590609 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f22780046c0 con 0x55fb892f90a0 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f228f3f2740 10 client.26237 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f228f3f2740 10 client.26237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f228f3f2740 20 client.26237 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f228f3f2740 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8931ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f228f3f2740 4 client.26237 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:36.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb89271da0 con 0x55fb8931db60 2024-09-10T21:11:36.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f228659c640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8931ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.822+0000 7f226aff5640 10 client.26237 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:36.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.823+0000 7f228659c640 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8931ff50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a32d9aa5816f060 server_cookie=de5f6ac79b3caa67 in_seq=0 out_seq=0 2024-09-10T21:11:36.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.823+0000 7f226aff5640 1 -- 172.21.15.33:0/2226590609 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb89271da0 con 0x55fb8931db60 2024-09-10T21:11:36.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.823+0000 7f226aff5640 10 client.26237 handle_command_reply: tid=0 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.823+0000 7f228f3f2740 1 client.26237 shutdown 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 msgr2=0x55fb89302cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb89302cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f227800abe0 tx=0x7f22780186d0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 shutdown_connections 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8931ff50 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb893006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f226aff5640 0 client.26237 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb89300100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 --2- 172.21.15.33:0/2226590609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb89302cb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 >> 172.21.15.33:0/2226590609 conn(0x55fb892f6ed0 msgr2=0x55fb892fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 shutdown_connections 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 1 -- 172.21.15.33:0/2226590609 wait complete. 2024-09-10T21:11:36.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.824+0000 7f228f3f2740 20 client.26237 trim_cache size 0 max 16384 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 Processor -- start 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 -- start start 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb892f90a0 0x55fb892f8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb892fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb8930e000 0x55fb892fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ac320 con 0x55fb89303400 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ade60 con 0x55fb8930e000 2024-09-10T21:11:36.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.853+0000 7f228f3f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ac9f0 con 0x55fb892f90a0 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb892fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb892fc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55660/0 (socket says 172.21.15.33:55660) 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 -- 172.21.15.33:0/4243960958 learned_addr learned my addr 172.21.15.33:0/4243960958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f2286d9d640 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb892f90a0 0x55fb892f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228759e640 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb8930e000 0x55fb892fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 -- 172.21.15.33:0/4243960958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb892f90a0 msgr2=0x55fb892f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb892f90a0 0x55fb892f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 -- 172.21.15.33:0/4243960958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb8930e000 msgr2=0x55fb892fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb8930e000 0x55fb892fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 -- 172.21.15.33:0/4243960958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb892f6710 con 0x55fb89303400 2024-09-10T21:11:36.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f228659c640 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb892fc1e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f227c0a8250 tx=0x7f227c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e70c04aff070fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:36.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f226cff9640 1 -- 172.21.15.33:0/4243960958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c01e040 con 0x55fb89303400 2024-09-10T21:11:36.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f226cff9640 1 -- 172.21.15.33:0/4243960958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f227c0ab790 con 0x55fb89303400 2024-09-10T21:11:36.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.854+0000 7f226cff9640 1 -- 172.21.15.33:0/4243960958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c0e30d0 con 0x55fb89303400 2024-09-10T21:11:36.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 -- 172.21.15.33:0/4243960958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 msgr2=0x55fb892fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb892fc1e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f227c0a8250 tx=0x7f227c0047d0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 -- 172.21.15.33:0/4243960958 shutdown_connections 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb8930e000 0x55fb892fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb892fc1e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 --2- 172.21.15.33:0/4243960958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb892f90a0 0x55fb892f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 -- 172.21.15.33:0/4243960958 >> 172.21.15.33:0/4243960958 conn(0x55fb892f6ed0 msgr2=0x55fb892f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 -- 172.21.15.33:0/4243960958 shutdown_connections 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.855+0000 7f228f3f2740 1 -- 172.21.15.33:0/4243960958 wait complete. 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 -- start start 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 Processor -- start 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb893236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb8931eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb8931f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ad790 con 0x55fb89303400 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22881ad0c0 con 0x55fb892f90a0 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22881aa2e0 con 0x55fb8930e000 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228f3f2740 10 client.0 _ll_register_callbacks cb 0x7f227801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb8931eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f2286d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb893236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb8931eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55662/0 (socket says 172.21.15.33:55662) 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228659c640 1 -- 172.21.15.33:0/3690777169 learned_addr learned my addr 172.21.15.33:0/3690777169 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.856+0000 7f228759e640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb8931f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f2286d9d640 1 -- 172.21.15.33:0/3690777169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 msgr2=0x55fb8931f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f2286d9d640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb8931f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f2286d9d640 1 -- 172.21.15.33:0/3690777169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 msgr2=0x55fb8931eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f2286d9d640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb8931eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f2286d9d640 1 -- 172.21.15.33:0/3690777169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb8931f8e0 con 0x55fb892f90a0 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f228759e640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb8931f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f228659c640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb8931eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f2286d9d640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb893236a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2278004c80 tx=0x7f22780199b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f302d750b3043a2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f226aff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f226aff5640 1 -- 172.21.15.33:0/3690777169 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278027070 con 0x55fb892f90a0 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f226aff5640 1 -- 172.21.15.33:0/3690777169 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2278004220 con 0x55fb892f90a0 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f227c0ab790 con 0x55fb892f90a0 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f226aff5640 1 -- 172.21.15.33:0/3690777169 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2278005cc0 con 0x55fb892f90a0 2024-09-10T21:11:36.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f228f3f2740 10 client.26243 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.857+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fb8938a6a0 con 0x55fb892f90a0 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f226aff5640 1 -- 172.21.15.33:0/3690777169 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f227c0ab790 con 0x55fb892f90a0 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f226aff5640 1 -- 172.21.15.33:0/3690777169 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f227803f050 con 0x55fb892f90a0 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228f3f2740 10 client.26243 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228f3f2740 10 client.26243 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228f3f2740 20 client.26243 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8938cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228f3f2740 4 client.26243 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fb8926d850 con 0x55fb8931db60 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228659c640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8938cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:36.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f228659c640 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8938cd10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ba589e853c0e6 server_cookie=fb34ec8c325c19ed in_seq=0 out_seq=0 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.858+0000 7f226aff5640 10 client.26243 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f226aff5640 1 -- 172.21.15.33:0/3690777169 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x55fb8926d850 con 0x55fb8931db60 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f226aff5640 10 client.26243 handle_command_reply: tid=0 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 client.26243 shutdown 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 msgr2=0x55fb893236a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb893236a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2278004c80 tx=0x7f22780199b0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 shutdown_connections 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb8931db60 0x55fb8938cd10 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb8930e000 0x55fb8931f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb89303400 0x55fb8931eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 --2- 172.21.15.33:0/3690777169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb892f90a0 0x55fb893236a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 >> 172.21.15.33:0/3690777169 conn(0x55fb892f6ed0 msgr2=0x55fb892f7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f226aff5640 0 client.26243 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:36.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 shutdown_connections 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 1 -- 172.21.15.33:0/3690777169 wait complete. 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.859+0000 7f228f3f2740 20 client.26243 trim_cache size 0 max 16384 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:11:36.870 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:11:36.874 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:36.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:36.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:36.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:36.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:36.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:36.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:36.877 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22700b52e0 msgr2=0x7f22700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22700b52e0 0x7f22700b77a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2280005fa0 tx=0x7f2280005c30 comp rx=0 tx=0).stop 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2288104550 msgr2=0x7f22881997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2288104550 0x7f22881997e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f227c00b4a0 tx=0x7f227c00b960 comp rx=0 tx=0).stop 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 shutdown_connections 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22700b52e0 0x7f22700b77a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f228810d100 0x7f228819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2288104550 0x7f22881997e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 --2- 172.21.15.33:0/2592704525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2288103b50 0x7f22881992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 >> 172.21.15.33:0/2592704525 conn(0x7f22880fd700 msgr2=0x7f22880fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 shutdown_connections 2024-09-10T21:11:36.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:36.860+0000 7f228d01d640 1 -- 172.21.15.33:0/2592704525 wait complete. 2024-09-10T21:11:37.680 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:11:37.680 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:11:37.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 540713 5.85 MB/sec execute 278 sec latency 3330.041 ms 2024-09-10T21:11:37.745 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:37 smithi033 ceph-mon[30860]: pgmap v575: 97 pgs: 97 active+clean; 211 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 579 KiB/s wr, 84 op/s 2024-09-10T21:11:38.013 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:37 smithi104 ceph-mon[35325]: pgmap v575: 97 pgs: 97 active+clean; 211 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 579 KiB/s wr, 84 op/s 2024-09-10T21:11:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:37 smithi174 ceph-mon[35635]: pgmap v575: 97 pgs: 97 active+clean; 211 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 763 KiB/s rd, 579 KiB/s wr, 84 op/s 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 -- 172.21.15.33:0/1094236081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 msgr2=0x7f98741117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 --2- 172.21.15.33:0/1094236081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741117d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f986c009ca0 tx=0x7f986c02a950 comp rx=0 tx=0).stop 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 -- 172.21.15.33:0/1094236081 shutdown_connections 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 --2- 172.21.15.33:0/1094236081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741117d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 --2- 172.21.15.33:0/1094236081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9874102150 0x7f987410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 --2- 172.21.15.33:0/1094236081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9874101830 0x7f9874101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 -- 172.21.15.33:0/1094236081 >> 172.21.15.33:0/1094236081 conn(0x7f98740fd700 msgr2=0x7f98740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 -- 172.21.15.33:0/1094236081 shutdown_connections 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.552+0000 7f987c243640 1 -- 172.21.15.33:0/1094236081 wait complete. 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 Processor -- start 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 -- start start 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9874101830 0x7f987419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9874102150 0x7f987419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9874114f20 con 0x7f987410f3e0 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9874114da0 con 0x7f9874102150 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987c243640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98741150a0 con 0x7f9874101830 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987a7e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987a7e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741a0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55674/0 (socket says 172.21.15.33:55674) 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987a7e0640 1 -- 172.21.15.33:0/1901713068 learned_addr learned my addr 172.21.15.33:0/1901713068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f98797de640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9874102150 0x7f987419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f9879fdf640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9874101830 0x7f987419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987a7e0640 1 -- 172.21.15.33:0/1901713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9874101830 msgr2=0x7f987419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987a7e0640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9874101830 0x7f987419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987a7e0640 1 -- 172.21.15.33:0/1901713068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9874102150 msgr2=0x7f987419b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.553+0000 7f987a7e0640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9874102150 0x7f987419b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f987a7e0640 1 -- 172.21.15.33:0/1901713068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98741a14e0 con 0x7f987410f3e0 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f9879fdf640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9874101830 0x7f987419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f987a7e0640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741a0d10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f986c004af0 tx=0x7f986c031f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f986affd640 1 -- 172.21.15.33:0/1901713068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c034040 con 0x7f987410f3e0 2024-09-10T21:11:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f986affd640 1 -- 172.21.15.33:0/1901713068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f986c0308f0 con 0x7f987410f3e0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98741a1770 con 0x7f987410f3e0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f986affd640 1 -- 172.21.15.33:0/1901713068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986c004600 con 0x7f987410f3e0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.554+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98741a7e90 con 0x7f987410f3e0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.555+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9874101c10 con 0x7f987410f3e0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.557+0000 7f986affd640 1 -- 172.21.15.33:0/1901713068 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f986c04d030 con 0x7f987410f3e0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.558+0000 7f986affd640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98600b5360 0x7f98600b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.558+0000 7f9879fdf640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98600b5360 0x7f98600b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.558+0000 7f986affd640 1 -- 172.21.15.33:0/1901713068 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f986c106840 con 0x7f987410f3e0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.558+0000 7f9879fdf640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98600b5360 0x7f98600b7820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9870009c20 tx=0x7f9870005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.561+0000 7f986affd640 1 -- 172.21.15.33:0/1901713068 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f986c0cd700 con 0x7f987410f3e0 2024-09-10T21:11:38.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 543487 5.90 MB/sec execute 279 sec latency 9.496 ms 2024-09-10T21:11:38.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.792+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98741a8200 con 0x7f987410f3e0 2024-09-10T21:11:38.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.792+0000 7f986affd640 1 -- 172.21.15.33:0/1901713068 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11973 (secure 0 0 0) 0x7f986c0cd000 con 0x7f987410f3e0 2024-09-10T21:11:38.798 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:11:38.798 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"btime":"2024-09-10T21:11:31:688744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:11:30.989029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:11:38.798 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2024-09-10T21:11:38.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98600b5360 msgr2=0x7f98600b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:38.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98600b5360 0x7f98600b7820 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9870009c20 tx=0x7f9870005f90 comp rx=0 tx=0).stop 2024-09-10T21:11:38.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 msgr2=0x7f98741a0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741a0d10 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f986c004af0 tx=0x7f986c031f50 comp rx=0 tx=0).stop 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 shutdown_connections 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f98600b5360 0x7f98600b7820 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f987410f3e0 0x7f98741a0d10 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9874102150 0x7f987419b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 --2- 172.21.15.33:0/1901713068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9874101830 0x7f987419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 >> 172.21.15.33:0/1901713068 conn(0x7f98740fd700 msgr2=0x7f98740ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 shutdown_connections 2024-09-10T21:11:38.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:38.795+0000 7f987c243640 1 -- 172.21.15.33:0/1901713068 wait complete. 2024-09-10T21:11:39.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:11:39.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 545039 5.90 MB/sec execute 280 sec latency 10.981 ms 2024-09-10T21:11:39.769 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:40.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:39 smithi033 ceph-mon[30860]: pgmap v576: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 784 KiB/s wr, 119 op/s 2024-09-10T21:11:40.057 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:39 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1901713068' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:11:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:39 smithi104 ceph-mon[35325]: pgmap v576: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 784 KiB/s wr, 119 op/s 2024-09-10T21:11:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:39 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1901713068' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:11:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:39 smithi174 ceph-mon[35635]: pgmap v576: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 896 KiB/s rd, 784 KiB/s wr, 119 op/s 2024-09-10T21:11:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:39 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1901713068' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:11:40.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 -- 172.21.15.33:0/189515295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 msgr2=0x7f13bc1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 --2- 172.21.15.33:0/189515295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc1040c0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ca0 tx=0x7f13ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 -- 172.21.15.33:0/189515295 shutdown_connections 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 --2- 172.21.15.33:0/189515295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc10d140 0x7f13bc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 --2- 172.21.15.33:0/189515295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13bc104600 0x7f13bc10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 --2- 172.21.15.33:0/189515295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc1040c0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 -- 172.21.15.33:0/189515295 >> 172.21.15.33:0/189515295 conn(0x7f13bc0fd700 msgr2=0x7f13bc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 -- 172.21.15.33:0/189515295 shutdown_connections 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 -- 172.21.15.33:0/189515295 wait complete. 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.278+0000 7f13c166b640 1 Processor -- start 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13c166b640 1 -- start start 2024-09-10T21:11:40.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13c166b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13c166b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc104600 0x7f13bc1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13c166b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13bc10d140 0x7f13bc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13c166b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc112db0 con 0x7f13bc103ce0 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13c166b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc112c30 con 0x7f13bc104600 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13c166b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc112f30 con 0x7f13bc10d140 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13baffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34064/0 (socket says 172.21.15.33:34064) 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13ba7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc104600 0x7f13bc1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13baffd640 1 -- 172.21.15.33:0/3830587805 learned_addr learned my addr 172.21.15.33:0/3830587805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13bc10d140 0x7f13bc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13ba7fc640 1 -- 172.21.15.33:0/3830587805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13bc10d140 msgr2=0x7f13bc19eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13bc10d140 0x7f13bc19eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13ba7fc640 1 -- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 msgr2=0x7f13bc199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13ba7fc640 1 -- 172.21.15.33:0/3830587805 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc19f330 con 0x7f13bc104600 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.279+0000 7f13baffd640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.280+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc104600 0x7f13bc1997c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f13b000d650 tx=0x7f13b000db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.280+0000 7f13a3fff640 1 -- 172.21.15.33:0/3830587805 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0017070 con 0x7f13bc104600 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.280+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc19f620 con 0x7f13bc104600 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.280+0000 7f13a3fff640 1 -- 172.21.15.33:0/3830587805 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b0009c70 con 0x7f13bc104600 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.280+0000 7f13a3fff640 1 -- 172.21.15.33:0/3830587805 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0003b30 con 0x7f13bc104600 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.280+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc1a5eb0 con 0x7f13bc104600 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.282+0000 7f13c3a40740 1 Processor -- start 2024-09-10T21:11:40.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.282+0000 7f13c3a40740 1 -- start start 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.282+0000 7f13a3fff640 1 -- 172.21.15.33:0/3830587805 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f13b0003cd0 con 0x7f13bc104600 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e7424d0 0x55751e7428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e751170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13c3a40740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55751e6b8ea0 con 0x55751e7424d0 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13c3a40740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55751e6a82d0 con 0x55751e74ed80 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13c3a40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55751e6ac0e0 con 0x55751e742df0 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e7424d0 0x55751e7428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e7424d0 0x55751e7428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34080/0 (socket says 172.21.15.33:34080) 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 -- 172.21.15.33:0/3477083891 learned_addr learned my addr 172.21.15.33:0/3477083891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e751170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13baffd640 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 -- 172.21.15.33:0/3477083891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 msgr2=0x55751e74e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 -- 172.21.15.33:0/3477083891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 msgr2=0x55751e751170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e751170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 -- 172.21.15.33:0/3477083891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751e406650 con 0x55751e7424d0 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.283+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e7424d0 0x55751e7428b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009ca0 tx=0x7f13b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f06936bd558494a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13a17fa640 1 -- 172.21.15.33:0/3477083891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b401e070 con 0x55751e7424d0 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13a3fff640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f13a40b52e0 0x7f13a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13a17fa640 1 -- 172.21.15.33:0/3477083891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b4009e50 con 0x55751e7424d0 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13a17fa640 1 -- 172.21.15.33:0/3477083891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b400e9f0 con 0x55751e7424d0 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13baffd640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f13a40b52e0 0x7f13a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13a3fff640 1 -- 172.21.15.33:0/3830587805 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f13b00e5740 con 0x7f13bc104600 2024-09-10T21:11:40.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 -- 172.21.15.33:0/3477083891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e7424d0 msgr2=0x55751e7428b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13baffd640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f13a40b52e0 0x7f13a40b77a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009430 tx=0x7f13ac0093c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e7424d0 0x55751e7428b0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f13b4009ca0 tx=0x7f13b4011810 comp rx=0 tx=0).stop 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 -- 172.21.15.33:0/3477083891 shutdown_connections 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e751170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3477083891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e7424d0 0x55751e7428b0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 -- 172.21.15.33:0/3477083891 >> 172.21.15.33:0/3477083891 conn(0x55751e740300 msgr2=0x55751e7414a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.284+0000 7f13c3a40740 1 -- 172.21.15.33:0/3477083891 shutdown_connections 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 -- 172.21.15.33:0/3477083891 wait complete. 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 -- start start 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 Processor -- start 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e74e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e74c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55751e509700 con 0x55751e74ed80 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55751e643160 con 0x55751e7424d0 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55751e755e10 con 0x55751e742df0 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.285+0000 7f13c3a40740 10 client.0 _ll_register_callbacks cb 0x55751e65c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13ba7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e74c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13ba7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34912/0 (socket says 172.21.15.33:34912) 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13ba7fc640 1 -- 172.21.15.33:0/3106455899 learned_addr learned my addr 172.21.15.33:0/3106455899 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13baffd640 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e74e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13bb7fe640 1 -- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 msgr2=0x55751e74bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13bb7fe640 1 -- 172.21.15.33:0/3106455899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 msgr2=0x55751e74e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e74e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13bb7fe640 1 -- 172.21.15.33:0/3106455899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55751e670c90 con 0x55751e74ed80 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e74c110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e630 tx=0x7f13b400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681160e7799e2c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f139eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f139eff5640 1 -- 172.21.15.33:0/3106455899 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b401e030 con 0x55751e74ed80 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b4009e50 con 0x55751e74ed80 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f139eff5640 1 -- 172.21.15.33:0/3106455899 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f13b400edc0 con 0x55751e74ed80 2024-09-10T21:11:40.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f139eff5640 1 -- 172.21.15.33:0/3106455899 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f13b40098f0 con 0x55751e74ed80 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13c3a40740 10 client.16767 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.286+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55751e761770 con 0x55751e74ed80 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f139eff5640 1 -- 172.21.15.33:0/3106455899 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b4009e50 con 0x55751e74ed80 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f139eff5640 1 -- 172.21.15.33:0/3106455899 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f13b402c020 con 0x55751e74ed80 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f13c3a40740 10 client.16767 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f13c3a40740 10 client.16767 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f13c3a40740 20 client.16767 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e762b20 0x55751e764f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f13c3a40740 4 client.16767 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55751e6badf0 con 0x55751e762b20 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.287+0000 7f13baffd640 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e762b20 0x55751e764f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f139eff5640 10 client.16767 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13baffd640 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e762b20 0x55751e764f10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=824231fd86b97cef server_cookie=f298363a03c3677 in_seq=0 out_seq=0 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f139eff5640 1 -- 172.21.15.33:0/3106455899 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55751e6badf0 con 0x55751e762b20 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f139eff5640 10 client.16767 handle_command_reply: tid=0 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 client.16767 shutdown 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 msgr2=0x55751e74c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e74c110 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f13b400e630 tx=0x7f13b400e9c0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 shutdown_connections 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e762b20 0x55751e764f10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e74c110 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e74bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f139eff5640 0 client.16767 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:11:40.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 --2- 172.21.15.33:0/3106455899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e74e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.288+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 >> 172.21.15.33:0/3106455899 conn(0x55751e740300 msgr2=0x55751e745e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.289+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 shutdown_connections 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.289+0000 7f13c3a40740 1 -- 172.21.15.33:0/3106455899 wait complete. 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.289+0000 7f13c3a40740 20 client.16767 trim_cache size 0 max 16384 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 Processor -- start 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 -- start start 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e7424d0 0x55751e74c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e742df0 0x55751e74aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e74d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1ac240 con 0x55751e742df0 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1add80 con 0x55751e74ed80 2024-09-10T21:11:40.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13c3a40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1ac910 con 0x55751e7424d0 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e742df0 0x55751e74aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13ba7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e742df0 0x55751e74aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34108/0 (socket says 172.21.15.33:34108) 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13ba7fc640 1 -- 172.21.15.33:0/2489619335 learned_addr learned my addr 172.21.15.33:0/2489619335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.303+0000 7f13baffd640 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e7424d0 0x55751e74c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e74d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13ba7fc640 1 -- 172.21.15.33:0/2489619335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e7424d0 msgr2=0x55751e74c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e7424d0 0x55751e74c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13ba7fc640 1 -- 172.21.15.33:0/2489619335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 msgr2=0x55751e74d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e74d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13ba7fc640 1 -- 172.21.15.33:0/2489619335 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55751e740fe0 con 0x55751e742df0 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e742df0 0x55751e74aff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f13b00abf40 tx=0x7f13b00eb530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e304e8f1a2a2a54f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:40.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13a0ff9640 1 -- 172.21.15.33:0/2489619335 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b00eb020 con 0x55751e742df0 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13a0ff9640 1 -- 172.21.15.33:0/2489619335 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b00abac0 con 0x55751e742df0 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.304+0000 7f13a0ff9640 1 -- 172.21.15.33:0/2489619335 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0020040 con 0x55751e742df0 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 -- 172.21.15.33:0/2489619335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e742df0 msgr2=0x55751e74aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e742df0 0x55751e74aff0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f13b00abf40 tx=0x7f13b00eb530 comp rx=0 tx=0).stop 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 -- 172.21.15.33:0/2489619335 shutdown_connections 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e74ed80 0x55751e74d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e742df0 0x55751e74aff0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 --2- 172.21.15.33:0/2489619335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e7424d0 0x55751e74c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 -- 172.21.15.33:0/2489619335 >> 172.21.15.33:0/2489619335 conn(0x55751e740300 msgr2=0x55751e74d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 -- 172.21.15.33:0/2489619335 shutdown_connections 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 -- 172.21.15.33:0/2489619335 wait complete. 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.305+0000 7f13c3a40740 1 -- start start 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 1 Processor -- start 2024-09-10T21:11:40.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e75b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e75b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e756060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1ad6b0 con 0x55751e74ed80 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1acfe0 con 0x55751e7424d0 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc1aa200 con 0x55751e742df0 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13c3a40740 10 client.0 _ll_register_callbacks cb 0x7f13b401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13baffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e75b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e756060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13baffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e75b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:34266/0 (socket says 172.21.15.33:34266) 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13baffd640 1 -- 172.21.15.33:0/1730714752 learned_addr learned my addr 172.21.15.33:0/1730714752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:40.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13ba7fc640 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e75b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13bb7fe640 1 -- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 msgr2=0x55751e75b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e75b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13bb7fe640 1 -- 172.21.15.33:0/1730714752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 msgr2=0x55751e75b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e75b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.306+0000 7f13bb7fe640 1 -- 172.21.15.33:0/1730714752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55751e7567a0 con 0x55751e74ed80 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13bb7fe640 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e756060 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f13b4004b50 tx=0x7f13b4005990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce26b7bfe73bc173 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f139eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f139eff5640 1 -- 172.21.15.33:0/1730714752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b401e040 con 0x55751e74ed80 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f139eff5640 1 -- 172.21.15.33:0/1730714752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f13b4004660 con 0x55751e74ed80 2024-09-10T21:11:40.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13b00abac0 con 0x55751e74ed80 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f139eff5640 1 -- 172.21.15.33:0/1730714752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f13b4009e50 con 0x55751e74ed80 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 10 client.16773 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55751e7d0330 con 0x55751e74ed80 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f139eff5640 1 -- 172.21.15.33:0/1730714752 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b00abac0 con 0x55751e74ed80 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f139eff5640 1 -- 172.21.15.33:0/1730714752 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f13b4003620 con 0x55751e74ed80 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 10 client.16773 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 10 client.16773 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 20 client.16773 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e7d09f0 0x55751e7d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 4 client.16773 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.307+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55751e6b68a0 con 0x55751e7d09f0 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.308+0000 7f13baffd640 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e7d09f0 0x55751e7d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.311+0000 7f13baffd640 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e7d09f0 0x55751e7d2e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d4089e5fb38c51bc server_cookie=36687420ad9f2fa6 in_seq=0 out_seq=0 2024-09-10T21:11:40.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.311+0000 7f139eff5640 10 client.16773 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:11:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f139eff5640 1 -- 172.21.15.33:0/1730714752 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55751e6b68a0 con 0x55751e7d09f0 2024-09-10T21:11:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f139eff5640 10 client.16773 handle_command_reply: tid=0 2024-09-10T21:11:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 client.16773 shutdown 2024-09-10T21:11:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 msgr2=0x55751e756060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e756060 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f13b4004b50 tx=0x7f13b4005990 comp rx=0 tx=0).stop 2024-09-10T21:11:40.422 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 shutdown_connections 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55751e7d09f0 0x55751e7d2e10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55751e74ed80 0x55751e756060 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55751e742df0 0x55751e75b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f139eff5640 0 client.16773 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 --2- 172.21.15.33:0/1730714752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55751e7424d0 0x55751e75b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 >> 172.21.15.33:0/1730714752 conn(0x55751e740300 msgr2=0x55751e74d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 shutdown_connections 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 1 -- 172.21.15.33:0/1730714752 wait complete. 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.312+0000 7f13c3a40740 20 client.16773 trim_cache size 0 max 16384 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f13a40b52e0 msgr2=0x7f13a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f13a40b52e0 0x7f13a40b77a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009430 tx=0x7f13ac0093c0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc104600 msgr2=0x7f13bc1997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc104600 0x7f13bc1997c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f13b000d650 tx=0x7f13b000db10 comp rx=0 tx=0).stop 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 shutdown_connections 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f13a40b52e0 0x7f13a40b77a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13bc10d140 0x7f13bc19eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13bc104600 0x7f13bc1997c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 --2- 172.21.15.33:0/3830587805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13bc103ce0 0x7f13bc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 >> 172.21.15.33:0/3830587805 conn(0x7f13bc0fd700 msgr2=0x7f13bc0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 shutdown_connections 2024-09-10T21:11:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:40.313+0000 7f13c166b640 1 -- 172.21.15.33:0/3830587805 wait complete. 2024-09-10T21:11:40.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547206 5.92 MB/sec execute 281 sec latency 187.411 ms 2024-09-10T21:11:41.207 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:11:41.470 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:41.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547206 5.90 MB/sec execute 282 sec latency 1187.493 ms 2024-09-10T21:11:41.853 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:41 smithi033 ceph-mon[30860]: pgmap v577: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 969 KiB/s wr, 123 op/s 2024-09-10T21:11:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:41 smithi104 ceph-mon[35325]: pgmap v577: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 969 KiB/s wr, 123 op/s 2024-09-10T21:11:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 -- 172.21.15.33:0/3999471974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 msgr2=0x7f5ac810cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 --2- 172.21.15.33:0/3999471974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac810cb90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009900 tx=0x7f5ac402a410 comp rx=0 tx=0).stop 2024-09-10T21:11:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 -- 172.21.15.33:0/3999471974 shutdown_connections 2024-09-10T21:11:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 --2- 172.21.15.33:0/3999471974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ac810d160 0x7f5ac810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 --2- 172.21.15.33:0/3999471974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac810cb90 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 --2- 172.21.15.33:0/3999471974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ac8103bb0 0x7f5ac8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 -- 172.21.15.33:0/3999471974 >> 172.21.15.33:0/3999471974 conn(0x7f5ac80fd700 msgr2=0x7f5ac80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 -- 172.21.15.33:0/3999471974 shutdown_connections 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.053+0000 7f5acfab9640 1 -- 172.21.15.33:0/3999471974 wait complete. 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 Processor -- start 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 -- start start 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ac8103bb0 0x7f5ac8101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac8101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ac810d160 0x7f5ac81024b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8112df0 con 0x7f5ac81045b0 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8112c70 con 0x7f5ac810d160 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acfab9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac8112f70 con 0x7f5ac8103bb0 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acd054640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac8101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acd855640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ac8103bb0 0x7f5ac8101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acd054640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac8101f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34134/0 (socket says 172.21.15.33:34134) 2024-09-10T21:11:42.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.054+0000 7f5acd054640 1 -- 172.21.15.33:0/2079241631 learned_addr learned my addr 172.21.15.33:0/2079241631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5ace056640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ac810d160 0x7f5ac81024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acd054640 1 -- 172.21.15.33:0/2079241631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ac8103bb0 msgr2=0x7f5ac8101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acd054640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ac8103bb0 0x7f5ac8101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acd054640 1 -- 172.21.15.33:0/2079241631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ac810d160 msgr2=0x7f5ac81024b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acd054640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ac810d160 0x7f5ac81024b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acd054640 1 -- 172.21.15.33:0/2079241631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac81a76a0 con 0x7f5ac81045b0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acd054640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac8101f70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005ab0 tx=0x7f5ac4003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5abeffd640 1 -- 172.21.15.33:0/2079241631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac403f070 con 0x7f5ac81045b0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac81a7930 con 0x7f5ac81045b0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5abeffd640 1 -- 172.21.15.33:0/2079241631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac4031d60 con 0x7f5ac81045b0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5abeffd640 1 -- 172.21.15.33:0/2079241631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4035030 con 0x7f5ac81045b0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.055+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac81a7e10 con 0x7f5ac81045b0 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.058+0000 7f5ad1e8e740 1 Processor -- start 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.058+0000 7f5ad1e8e740 1 -- start start 2024-09-10T21:11:42.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.058+0000 7f5abeffd640 1 -- 172.21.15.33:0/2079241631 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5ac403b580 con 0x7f5ac81045b0 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.058+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afaf24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.058+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afafe460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb00d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606afa68ea0 con 0x5606afaf20f0 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5606afa582d0 con 0x5606afaf2a10 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606afa5c0e0 con 0x5606afafe9a0 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afaf24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afaf24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34140/0 (socket says 172.21.15.33:34140) 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 -- 172.21.15.33:0/748128216 learned_addr learned my addr 172.21.15.33:0/748128216 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5acd855640 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afafe460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5acd054640 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb00d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 -- 172.21.15.33:0/748128216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 msgr2=0x5606afb00d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb00d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 -- 172.21.15.33:0/748128216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 msgr2=0x5606afafe460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afafe460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 -- 172.21.15.33:0/748128216 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606af7b6650 con 0x5606afaf20f0 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5acd054640 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5acd855640 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afafe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ace056640 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afaf24d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009fd0 tx=0x7f5ac0010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef9f460fc50011e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5abeffd640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ab40b52e0 0x7f5ab40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5ab3fff640 1 -- 172.21.15.33:0/748128216 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0016070 con 0x5606afaf20f0 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.059+0000 7f5acd855640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ab40b52e0 0x7f5ab40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ab3fff640 1 -- 172.21.15.33:0/748128216 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac0012bc0 con 0x5606afaf20f0 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ab3fff640 1 -- 172.21.15.33:0/748128216 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0011b20 con 0x5606afaf20f0 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5abeffd640 1 -- 172.21.15.33:0/2079241631 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5ac4106730 con 0x7f5ac81045b0 2024-09-10T21:11:42.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/748128216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 msgr2=0x5606afaf24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5acd855640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ab40b52e0 0x7f5ab40b77a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5606afaf0e00 tx=0x7f5ab800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afaf24d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009fd0 tx=0x7f5ac0010800 comp rx=0 tx=0).stop 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/748128216 shutdown_connections 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb00d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afafe460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/748128216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afaf24d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/748128216 >> 172.21.15.33:0/748128216 conn(0x5606afaeff20 msgr2=0x5606afaf10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/748128216 shutdown_connections 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.060+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/748128216 wait complete. 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 -- start start 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 Processor -- start 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afafe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afaf2a10 0x5606afafb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afafe9a0 0x5606afafbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5606af8b9700 con 0x5606afaf20f0 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5606af9f3160 con 0x5606afafe9a0 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606afb01620 con 0x5606afaf2a10 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5ad1e8e740 10 client.0 _ll_register_callbacks cb 0x5606afa0c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5acd855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afafe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5acd855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afafe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34152/0 (socket says 172.21.15.33:34152) 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5acd855640 1 -- 172.21.15.33:0/392269604 learned_addr learned my addr 172.21.15.33:0/392269604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.061+0000 7f5acd054640 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afaf2a10 0x5606afafb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ace056640 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afafe9a0 0x5606afafbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5acd855640 1 -- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afaf2a10 msgr2=0x5606afafb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5acd855640 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afaf2a10 0x5606afafb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5acd855640 1 -- 172.21.15.33:0/392269604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afafe9a0 msgr2=0x5606afafbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5acd855640 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afafe9a0 0x5606afafbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5acd855640 1 -- 172.21.15.33:0/392269604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606afa20c90 con 0x5606afaf20f0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5acd855640 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afafe2b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8003d00 tx=0x7f5ab8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84b61f1d8ecf6c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ab17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ab17fa640 1 -- 172.21.15.33:0/392269604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab802c060 con 0x5606afaf20f0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ac0012bc0 con 0x5606afaf20f0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ab17fa640 1 -- 172.21.15.33:0/392269604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5ab8017030 con 0x5606afaf20f0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ab17fa640 1 -- 172.21.15.33:0/392269604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5ab8027410 con 0x5606afaf20f0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ad1e8e740 10 client.16785 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.062+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5606afb0cfb0 con 0x5606afaf20f0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ab17fa640 1 -- 172.21.15.33:0/392269604 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ac0012bc0 con 0x5606afaf20f0 2024-09-10T21:11:42.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ab17fa640 1 -- 172.21.15.33:0/392269604 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f5ab8031070 con 0x5606afaf20f0 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ad1e8e740 10 client.16785 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ad1e8e740 10 client.16785 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ad1e8e740 20 client.16785 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb0e4c0 0x5606afb108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ad1e8e740 4 client.16785 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606afa6adf0 con 0x5606afb0e4c0 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5acd054640 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb0e4c0 0x5606afb108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5ab17fa640 10 client.16785 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.063+0000 7f5acd054640 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb0e4c0 0x5606afb108e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f13f6bdf308c150e server_cookie=25fba5e140a0c04c in_seq=0 out_seq=0 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ab17fa640 1 -- 172.21.15.33:0/392269604 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5606afa6adf0 con 0x5606afb0e4c0 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ab17fa640 10 client.16785 handle_command_reply: tid=0 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 client.16785 shutdown 2024-09-10T21:11:42.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 msgr2=0x5606afafe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afafe2b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8003d00 tx=0x7f5ab8004730 comp rx=0 tx=0).stop 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 shutdown_connections 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afafe9a0 0x5606afafbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb0e4c0 0x5606afb108e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afaf2a10 0x5606afafb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/392269604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afafe2b0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ab17fa640 0 client.16785 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.064+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 >> 172.21.15.33:0/392269604 conn(0x5606afaeff20 msgr2=0x5606afaf5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.065+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 shutdown_connections 2024-09-10T21:11:42.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.065+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/392269604 wait complete. 2024-09-10T21:11:42.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.065+0000 7f5ad1e8e740 20 client.16785 trim_cache size 0 max 16384 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 Processor -- start 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 -- start start 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb0a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afaffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b0640 con 0x5606afaf20f0 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b2180 con 0x5606afaf2a10 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b0d10 con 0x5606afafe9a0 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5acd855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb0a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5acd855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb0a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34158/0 (socket says 172.21.15.33:34158) 2024-09-10T21:11:42.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5acd855640 1 -- 172.21.15.33:0/14017606 learned_addr learned my addr 172.21.15.33:0/14017606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.078+0000 7f5acd054640 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afaffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5ace056640 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5acd855640 1 -- 172.21.15.33:0/14017606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 msgr2=0x5606afb002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5acd855640 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5acd855640 1 -- 172.21.15.33:0/14017606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 msgr2=0x5606afaffd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5acd855640 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afaffd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5acd855640 1 -- 172.21.15.33:0/14017606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606afaf2840 con 0x5606afaf20f0 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5acd855640 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb0a8b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800b980 tx=0x7f5ab8004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7131572077da8d3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5ab37fe640 1 -- 172.21.15.33:0/14017606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8037020 con 0x5606afaf20f0 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5ab37fe640 1 -- 172.21.15.33:0/14017606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab8004350 con 0x5606afaf20f0 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.079+0000 7f5ab37fe640 1 -- 172.21.15.33:0/14017606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8027d50 con 0x5606afaf20f0 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/14017606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 msgr2=0x5606afb0a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb0a8b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ab800b980 tx=0x7f5ab8004650 comp rx=0 tx=0).stop 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/14017606 shutdown_connections 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afaffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/14017606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb0a8b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/14017606 >> 172.21.15.33:0/14017606 conn(0x5606afaeff20 msgr2=0x5606afaf9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/14017606 shutdown_connections 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/14017606 wait complete. 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 -- start start 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.080+0000 7f5ad1e8e740 1 Processor -- start 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb06e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afb04940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ad1e8e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb04e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b1ab0 con 0x5606afaf20f0 2024-09-10T21:11:42.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81b13e0 con 0x5606afaf2a10 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ad1e8e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac81ae600 con 0x5606afafe9a0 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ad1e8e740 10 client.0 _ll_register_callbacks cb 0x7f5ab8003bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb06e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb06e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34168/0 (socket says 172.21.15.33:34168) 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 -- 172.21.15.33:0/2786957390 learned_addr learned my addr 172.21.15.33:0/2786957390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd054640 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afb04940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ace056640 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb04e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 -- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 msgr2=0x5606afb04e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb04e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 -- 172.21.15.33:0/2786957390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 msgr2=0x5606afb04940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afb04940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 -- 172.21.15.33:0/2786957390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606afb05470 con 0x5606afaf20f0 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5acd855640 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb06e70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005860 tx=0x7f5ab800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2b615bb75f58dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:42.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.081+0000 7f5ab17fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ab17fa640 1 -- 172.21.15.33:0/2786957390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab8037030 con 0x5606afaf20f0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ab17fa640 1 -- 172.21.15.33:0/2786957390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5ab8005460 con 0x5606afaf20f0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ab8004350 con 0x5606afaf20f0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ab17fa640 1 -- 172.21.15.33:0/2786957390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5ab802c020 con 0x5606afaf20f0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 10 client.16791 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5606afb7c1d0 con 0x5606afaf20f0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ab17fa640 1 -- 172.21.15.33:0/2786957390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ab8004350 con 0x5606afaf20f0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ab17fa640 1 -- 172.21.15.33:0/2786957390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f5ab8031040 con 0x5606afaf20f0 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 10 client.16791 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 10 client.16791 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 20 client.16791 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb7c890 0x5606afb7ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 4 client.16791 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:11:42.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.082+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606afa668a0 con 0x5606afb7c890 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5acd054640 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb7c890 0x5606afb7ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ab17fa640 10 client.16791 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5acd054640 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb7c890 0x5606afb7ecb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=192ea445a58ba9b8 server_cookie=7704b209ce785738 in_seq=0 out_seq=0 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ab17fa640 1 -- 172.21.15.33:0/2786957390 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606afa668a0 con 0x5606afb7c890 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ab17fa640 10 client.16791 handle_command_reply: tid=0 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 client.16791 shutdown 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 msgr2=0x5606afb06e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb06e70 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5ab8005860 tx=0x7f5ab800b480 comp rx=0 tx=0).stop 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 shutdown_connections 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5606afb7c890 0x5606afb7ecb0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606afafe9a0 0x5606afb04e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5606afaf2a10 0x5606afb04940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 --2- 172.21.15.33:0/2786957390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5606afaf20f0 0x5606afb06e70 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 >> 172.21.15.33:0/2786957390 conn(0x5606afaeff20 msgr2=0x5606afaee750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:42.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ab17fa640 0 client.16791 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:11:42.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 shutdown_connections 2024-09-10T21:11:42.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.083+0000 7f5ad1e8e740 1 -- 172.21.15.33:0/2786957390 wait complete. 2024-09-10T21:11:42.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5ad1e8e740 20 client.16791 trim_cache size 0 max 16384 2024-09-10T21:11:42.094 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ab40b52e0 msgr2=0x7f5ab40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ab40b52e0 0x7f5ab40b77a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x5606afaf0e00 tx=0x7f5ab800b390 comp rx=0 tx=0).stop 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 msgr2=0x7f5ac8101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac8101f70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005ab0 tx=0x7f5ac4003cd0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 shutdown_connections 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ab40b52e0 0x7f5ab40b77a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ac810d160 0x7f5ac81024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ac81045b0 0x7f5ac8101f70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 --2- 172.21.15.33:0/2079241631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ac8103bb0 0x7f5ac8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 >> 172.21.15.33:0/2079241631 conn(0x7f5ac80fd700 msgr2=0x7f5ac80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:42.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 shutdown_connections 2024-09-10T21:11:42.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:42.084+0000 7f5acfab9640 1 -- 172.21.15.33:0/2079241631 wait complete. 2024-09-10T21:11:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:41 smithi174 ceph-mon[35635]: pgmap v577: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 969 KiB/s wr, 123 op/s 2024-09-10T21:11:42.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547225 5.88 MB/sec execute 283 sec latency 1549.883 ms 2024-09-10T21:11:42.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:11:43.143 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 -- 172.21.15.33:0/79609855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0108060 msgr2=0x7ff7f010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 --2- 172.21.15.33:0/79609855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0108060 0x7ff7f010e860 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8009900 tx=0x7ff7e802a410 comp rx=0 tx=0).stop 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 -- 172.21.15.33:0/79609855 shutdown_connections 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 --2- 172.21.15.33:0/79609855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0108060 0x7ff7f010e860 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 --2- 172.21.15.33:0/79609855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f01076e0 0x7ff7f0107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 --2- 172.21.15.33:0/79609855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0101cd0 0x7ff7f01020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 -- 172.21.15.33:0/79609855 >> 172.21.15.33:0/79609855 conn(0x7ff7f00fd700 msgr2=0x7ff7f00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 -- 172.21.15.33:0/79609855 shutdown_connections 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.652+0000 7ff7f7106640 1 -- 172.21.15.33:0/79609855 wait complete. 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 Processor -- start 2024-09-10T21:11:43.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 -- start start 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0101cd0 0x7ff7f00fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f01076e0 0x7ff7f00fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0108060 0x7ff7f00fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0111d20 con 0x7ff7f0101cd0 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0111ba0 con 0x7ff7f01076e0 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f7106640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0111ea0 con 0x7ff7f0108060 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f4ea2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0101cd0 0x7ff7f00fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f4ea2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0101cd0 0x7ff7f00fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34188/0 (socket says 172.21.15.33:34188) 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f4ea2640 1 -- 172.21.15.33:0/674966331 learned_addr learned my addr 172.21.15.33:0/674966331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.653+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0108060 0x7ff7f00fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7effff640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f01076e0 0x7ff7f00fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f4ea2640 1 -- 172.21.15.33:0/674966331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0108060 msgr2=0x7ff7f00fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f4ea2640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0108060 0x7ff7f00fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f4ea2640 1 -- 172.21.15.33:0/674966331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f01076e0 msgr2=0x7ff7f00fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f4ea2640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f01076e0 0x7ff7f00fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f4ea2640 1 -- 172.21.15.33:0/674966331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f01a7620 con 0x7ff7f0101cd0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f4ea2640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0101cd0 0x7ff7f00fdc00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800d770 tx=0x7ff7d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7edffb640 1 -- 172.21.15.33:0/674966331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8017070 con 0x7ff7f0101cd0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7edffb640 1 -- 172.21.15.33:0/674966331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7d80054d0 con 0x7ff7f0101cd0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f01a7910 con 0x7ff7f0101cd0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7edffb640 1 -- 172.21.15.33:0/674966331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8004c30 con 0x7ff7f0101cd0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.654+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f01a7e00 con 0x7ff7f0101cd0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 Processor -- start 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7edffb640 1 -- 172.21.15.33:0/674966331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff7d8004e10 con 0x7ff7f0101cd0 2024-09-10T21:11:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 -- start start 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362a4d0 0x561e8362a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83636840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 0x561e83639170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e835a0ea0 con 0x561e8362a4d0 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561e835902d0 con 0x561e83636d80 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f94db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e835940e0 con 0x561e8362adf0 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f56a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362a4d0 0x561e8362a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f56a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362a4d0 0x561e8362a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34190/0 (socket says 172.21.15.33:34190) 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f56a3640 1 -- 172.21.15.33:0/4290349849 learned_addr learned my addr 172.21.15.33:0/4290349849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7f4ea2640 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83636840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.657+0000 7ff7effff640 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 0x561e83639170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7f56a3640 1 -- 172.21.15.33:0/4290349849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 msgr2=0x561e83636840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83636840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7f56a3640 1 -- 172.21.15.33:0/4290349849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 msgr2=0x561e83639170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 0x561e83639170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7f56a3640 1 -- 172.21.15.33:0/4290349849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e832ee650 con 0x561e8362a4d0 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7edffb640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff7dc0b52e0 0x7ff7dc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7f4ea2640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff7dc0b52e0 0x7ff7dc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362a4d0 0x561e8362a8b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7e802a930 tx=0x7ff7e8032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4778e65886d32efe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7edffb640 1 -- 172.21.15.33:0/674966331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff7d80e56f0 con 0x7ff7f0101cd0 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7e6ffd640 1 -- 172.21.15.33:0/4290349849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8040070 con 0x561e8362a4d0 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7e6ffd640 1 -- 172.21.15.33:0/4290349849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e80316a0 con 0x561e8362a4d0 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.658+0000 7ff7e6ffd640 1 -- 172.21.15.33:0/4290349849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8035040 con 0x561e8362a4d0 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 -- 172.21.15.33:0/4290349849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362a4d0 msgr2=0x561e8362a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f4ea2640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff7dc0b52e0 0x7ff7dc0b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800d740 tx=0x7ff7d80eb530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362a4d0 0x561e8362a8b0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff7e802a930 tx=0x7ff7e8032c50 comp rx=0 tx=0).stop 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 -- 172.21.15.33:0/4290349849 shutdown_connections 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 0x561e83639170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83636840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 --2- 172.21.15.33:0/4290349849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362a4d0 0x561e8362a8b0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 -- 172.21.15.33:0/4290349849 >> 172.21.15.33:0/4290349849 conn(0x561e83628300 msgr2=0x561e836294a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 -- 172.21.15.33:0/4290349849 shutdown_connections 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 -- 172.21.15.33:0/4290349849 wait complete. 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 -- start start 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.659+0000 7ff7f94db740 1 Processor -- start 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362a4d0 0x561e83636670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83633ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e836340f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f94db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561e833f1700 con 0x561e83636d80 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f94db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561e8352b160 con 0x561e8362a4d0 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f94db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e836399a0 con 0x561e8362adf0 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f94db740 10 client.0 _ll_register_callbacks cb 0x561e835449e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e836340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e836340f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34206/0 (socket says 172.21.15.33:34206) 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 -- 172.21.15.33:0/1918501765 learned_addr learned my addr 172.21.15.33:0/1918501765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7effff640 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83633ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362a4d0 0x561e83636670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 -- 172.21.15.33:0/1918501765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 msgr2=0x561e83633ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83633ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 -- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362a4d0 msgr2=0x561e83636670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362a4d0 0x561e83636670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 -- 172.21.15.33:0/1918501765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e83558c90 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e836340f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80318f0 tx=0x7ff7e8005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b994ed6e5a8ef8fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.660+0000 7ff7cffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7cffff640 1 -- 172.21.15.33:0/1918501765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8040030 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7cffff640 1 -- 172.21.15.33:0/1918501765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff7e8005040 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e80316a0 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7cffff640 1 -- 172.21.15.33:0/1918501765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff7e8059050 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 10 client.16803 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561e83645300 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7cffff640 1 -- 172.21.15.33:0/1918501765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e80316a0 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7cffff640 1 -- 172.21.15.33:0/1918501765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff7e803c4d0 con 0x561e83636d80 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 10 client.16803 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:43.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 10 client.16803 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 20 client.16803 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e83646650 0x561e83648a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 4 client.16803 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.661+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e835a2df0 con 0x561e83646650 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.662+0000 7ff7effff640 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e83646650 0x561e83648a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.662+0000 7ff7effff640 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e83646650 0x561e83648a40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f7fb833262da949a server_cookie=88bb0bff3d3a1eb in_seq=0 out_seq=0 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.662+0000 7ff7cffff640 10 client.16803 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7cffff640 1 -- 172.21.15.33:0/1918501765 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561e835a2df0 con 0x561e83646650 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7cffff640 10 client.16803 handle_command_reply: tid=0 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 client.16803 shutdown 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 msgr2=0x561e836340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e836340f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80318f0 tx=0x7ff7e8005d90 comp rx=0 tx=0).stop 2024-09-10T21:11:43.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 shutdown_connections 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e83646650 0x561e83648a40 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e836340f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362adf0 0x561e83633ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7cffff640 0 client.16803 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 --2- 172.21.15.33:0/1918501765 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362a4d0 0x561e83636670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 >> 172.21.15.33:0/1918501765 conn(0x561e83628300 msgr2=0x561e8362de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 shutdown_connections 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 1 -- 172.21.15.33:0/1918501765 wait complete. 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.663+0000 7ff7f94db740 20 client.16803 trim_cache size 0 max 16384 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.676+0000 7ff7f94db740 1 Processor -- start 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.676+0000 7ff7f94db740 1 -- start start 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 0x561e83627310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362adf0 0x561e83638170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 0x561e836386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7f94db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01b05a0 con 0x561e8362adf0 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7f94db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01b20e0 con 0x561e83636d80 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7f94db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01b0c70 con 0x561e8362a4d0 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362adf0 0x561e83638170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362adf0 0x561e83638170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34210/0 (socket says 172.21.15.33:34210) 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 -- 172.21.15.33:0/3213791562 learned_addr learned my addr 172.21.15.33:0/3213791562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 -- 172.21.15.33:0/3213791562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 msgr2=0x561e83627310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 --2- 172.21.15.33:0/3213791562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 0x561e83627310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 -- 172.21.15.33:0/3213791562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 msgr2=0x561e836386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 --2- 172.21.15.33:0/3213791562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 0x561e836386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 -- 172.21.15.33:0/3213791562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e83631fc0 con 0x561e8362adf0 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7effff640 1 --2- 172.21.15.33:0/3213791562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362adf0 0x561e83638170 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x561e83627cc0 tx=0x7ff7e0012990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc99e107c2a5f98e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7e67fc640 1 -- 172.21.15.33:0/3213791562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e001e5a0 con 0x561e8362adf0 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7e67fc640 1 -- 172.21.15.33:0/3213791562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e001ebf0 con 0x561e8362adf0 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.677+0000 7ff7e67fc640 1 -- 172.21.15.33:0/3213791562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0023cc0 con 0x561e8362adf0 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 -- 172.21.15.33:0/3213791562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362adf0 msgr2=0x561e83638170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 --2- 172.21.15.33:0/3213791562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362adf0 0x561e83638170 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x561e83627cc0 tx=0x7ff7e0012990 comp rx=0 tx=0).stop 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 -- 172.21.15.33:0/3213791562 shutdown_connections 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 --2- 172.21.15.33:0/3213791562 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e83636d80 0x561e836386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 --2- 172.21.15.33:0/3213791562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e8362adf0 0x561e83638170 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 --2- 172.21.15.33:0/3213791562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 0x561e83627310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 -- 172.21.15.33:0/3213791562 >> 172.21.15.33:0/3213791562 conn(0x561e83628300 msgr2=0x561e83626c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 -- 172.21.15.33:0/3213791562 shutdown_connections 2024-09-10T21:11:43.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 -- 172.21.15.33:0/3213791562 wait complete. 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 -- start start 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.678+0000 7ff7f94db740 1 Processor -- start 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 0x561e8364c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362adf0 0x561e83648700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f94db740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e83648c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f94db740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01b1a10 con 0x561e83636d80 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f94db740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01b1340 con 0x561e8362adf0 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f94db740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01ae560 con 0x561e8362a4d0 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f94db740 10 client.0 _ll_register_callbacks cb 0x7ff7e0009660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e83648c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e83648c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34220/0 (socket says 172.21.15.33:34220) 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7effff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362adf0 0x561e83648700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 -- 172.21.15.33:0/2874641530 learned_addr learned my addr 172.21.15.33:0/2874641530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:43.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 0x561e8364c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 -- 172.21.15.33:0/2874641530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 msgr2=0x561e8364c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 0x561e8364c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 -- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362adf0 msgr2=0x561e83648700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362adf0 0x561e83648700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7f56a3640 1 -- 172.21.15.33:0/2874641530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e83649470 con 0x561e83636d80 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.679+0000 7ff7effff640 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362adf0 0x561e83648700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f56a3640 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e83648c40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8006010 tx=0x7ff7e8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ebbdb1023a3794 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7cffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7cffff640 1 -- 172.21.15.33:0/2874641530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8040070 con 0x561e83636d80 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e001ebf0 con 0x561e83636d80 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7cffff640 1 -- 172.21.15.33:0/2874641530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff7e80314a0 con 0x561e83636d80 2024-09-10T21:11:43.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7cffff640 1 -- 172.21.15.33:0/2874641530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7ff7e8035070 con 0x561e83636d80 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 10 client.16809 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561e836b4220 con 0x561e83636d80 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7cffff640 1 -- 172.21.15.33:0/2874641530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e001ebf0 con 0x561e83636d80 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7cffff640 1 -- 172.21.15.33:0/2874641530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7ff7e80040d0 con 0x561e83636d80 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 10 client.16809 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 10 client.16809 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 20 client.16809 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e836b47e0 0x561e836b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 4 client.16809 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.680+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e8359e8a0 con 0x561e836b47e0 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7effff640 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e836b47e0 0x561e836b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7effff640 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e836b47e0 0x561e836b6bd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c10da3eac8d5b7cc server_cookie=2daf64dff5527ae9 in_seq=0 out_seq=0 2024-09-10T21:11:43.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7cffff640 10 client.16809 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7cffff640 1 -- 172.21.15.33:0/2874641530 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e8359e8a0 con 0x561e836b47e0 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7cffff640 10 client.16809 handle_command_reply: tid=0 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7f94db740 1 client.16809 shutdown 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 msgr2=0x561e83648c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.681+0000 7ff7f94db740 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e83648c40 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8006010 tx=0x7ff7e8000bc0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 shutdown_connections 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561e836b47e0 0x561e836b6bd0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561e83636d80 0x561e83648c40 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561e8362adf0 0x561e83648700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 --2- 172.21.15.33:0/2874641530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e8362a4d0 0x561e8364c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 >> 172.21.15.33:0/2874641530 conn(0x561e83628300 msgr2=0x561e83635910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7cffff640 0 client.16809 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 shutdown_connections 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 1 -- 172.21.15.33:0/2874641530 wait complete. 2024-09-10T21:11:43.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f94db740 20 client.16809 trim_cache size 0 max 16384 2024-09-10T21:11:43.703 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:11:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff7dc0b52e0 msgr2=0x7ff7dc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff7dc0b52e0 0x7ff7dc0b77a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800d740 tx=0x7ff7d80eb530 comp rx=0 tx=0).stop 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0101cd0 msgr2=0x7ff7f00fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0101cd0 0x7ff7f00fdc00 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff7d800d770 tx=0x7ff7d800dc30 comp rx=0 tx=0).stop 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 shutdown_connections 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff7dc0b52e0 0x7ff7dc0b77a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0108060 0x7ff7f00fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7f01076e0 0x7ff7f00fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 --2- 172.21.15.33:0/674966331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7f0101cd0 0x7ff7f00fdc00 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.682+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 >> 172.21.15.33:0/674966331 conn(0x7ff7f00fd700 msgr2=0x7ff7f006b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.683+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 shutdown_connections 2024-09-10T21:11:43.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:43.683+0000 7ff7f7106640 1 -- 172.21.15.33:0/674966331 wait complete. 2024-09-10T21:11:43.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547225 5.86 MB/sec execute 284 sec latency 1622.375 ms 2024-09-10T21:11:43.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:43 smithi033 ceph-mon[30860]: pgmap v578: 97 pgs: 97 active+clean; 211 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 966 KiB/s wr, 115 op/s 2024-09-10T21:11:44.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:43 smithi104 ceph-mon[35325]: pgmap v578: 97 pgs: 97 active+clean; 211 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 966 KiB/s wr, 115 op/s 2024-09-10T21:11:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:43 smithi174 ceph-mon[35635]: pgmap v578: 97 pgs: 97 active+clean; 211 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 966 KiB/s wr, 115 op/s 2024-09-10T21:11:44.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:11:44.646 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:44.738 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547225 5.84 MB/sec execute 285 sec latency 2622.421 ms 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 -- 172.21.15.33:0/150279288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed8103b70 msgr2=0x7f6ed8103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/150279288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed8103b70 0x7f6ed8103f50 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009900 tx=0x7f6ec802a410 comp rx=0 tx=0).stop 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 -- 172.21.15.33:0/150279288 shutdown_connections 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/150279288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ed810d120 0x7f6ed810f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/150279288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ed8104570 0x7f6ed810cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/150279288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed8103b70 0x7f6ed8103f50 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 -- 172.21.15.33:0/150279288 >> 172.21.15.33:0/150279288 conn(0x7f6ed80fd700 msgr2=0x7f6ed80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 -- 172.21.15.33:0/150279288 shutdown_connections 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.114+0000 7f6edcd7d640 1 -- 172.21.15.33:0/150279288 wait complete. 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 Processor -- start 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 -- start start 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ed8103b70 0x7f6ed81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ed8104570 0x7f6ed8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed810d120 0x7f6ed819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8112da0 con 0x7f6ed810d120 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8112c20 con 0x7f6ed8104570 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6edcd7d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed8112f20 con 0x7f6ed8103b70 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.115+0000 7f6ed6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed810d120 0x7f6ed819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed810d120 0x7f6ed819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34246/0 (socket says 172.21.15.33:34246) 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ed8103b70 0x7f6ed81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 -- 172.21.15.33:0/2207682544 learned_addr learned my addr 172.21.15.33:0/2207682544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ed8104570 0x7f6ed8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 -- 172.21.15.33:0/2207682544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ed8103b70 msgr2=0x7f6ed81991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ed8103b70 0x7f6ed81991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 -- 172.21.15.33:0/2207682544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ed8104570 msgr2=0x7f6ed8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ed8104570 0x7f6ed8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 -- 172.21.15.33:0/2207682544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed819f2a0 con 0x7f6ed810d120 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ed8104570 0x7f6ed8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ed6d9d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed810d120 0x7f6ed819ead0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000cad0 tx=0x7f6ed000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ecffff640 1 -- 172.21.15.33:0/2207682544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000ae20 con 0x7f6ed810d120 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed819f590 con 0x7f6ed810d120 2024-09-10T21:11:45.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ecffff640 1 -- 172.21.15.33:0/2207682544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ed0005020 con 0x7f6ed810d120 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6ecffff640 1 -- 172.21.15.33:0/2207682544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000f6b0 con 0x7f6ed810d120 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.116+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed81a5c50 con 0x7f6ed810d120 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 Processor -- start 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6ecffff640 1 -- 172.21.15.33:0/2207682544 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6ed00054f0 con 0x7f6ed810d120 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 -- start start 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e041b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e042bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e042e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5612e0391ea0 con 0x5612e041bdf0 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5612e03812d0 con 0x5612e042c190 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6edf152740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612e03850e0 con 0x5612e041b4d0 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6ed659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e042bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6ed659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e042bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34260/0 (socket says 172.21.15.33:34260) 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.119+0000 7f6ed659c640 1 -- 172.21.15.33:0/1423306191 learned_addr learned my addr 172.21.15.33:0/1423306191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e041b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e042e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ed659c640 1 -- 172.21.15.33:0/1423306191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 msgr2=0x5612e041b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ed659c640 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e041b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ed659c640 1 -- 172.21.15.33:0/1423306191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 msgr2=0x5612e042e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ed659c640 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e042e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ed659c640 1 -- 172.21.15.33:0/1423306191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e00df650 con 0x5612e041bdf0 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ecffff640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ec40b52e0 0x7f6ec40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ed659c640 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e042bc50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8033000 tx=0x7f6ec8031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ee398a964fdc2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ec40b52e0 0x7f6ec40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6ecffff640 1 -- 172.21.15.33:0/2207682544 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6ed00e5c20 con 0x7f6ed810d120 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6eccd99640 1 -- 172.21.15.33:0/1423306191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8040070 con 0x5612e041bdf0 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6eccd99640 1 -- 172.21.15.33:0/1423306191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec80316a0 con 0x5612e041bdf0 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.120+0000 7f6eccd99640 1 -- 172.21.15.33:0/1423306191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8035040 con 0x5612e041bdf0 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 -- 172.21.15.33:0/1423306191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 msgr2=0x5612e042bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ec40b52e0 0x7f6ec40b77a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6ed810f800 tx=0x7f6ec0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e042bc50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8033000 tx=0x7f6ec8031de0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 -- 172.21.15.33:0/1423306191 shutdown_connections 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e042e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e042bc50 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 --2- 172.21.15.33:0/1423306191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e041b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 -- 172.21.15.33:0/1423306191 >> 172.21.15.33:0/1423306191 conn(0x5612e0419300 msgr2=0x5612e041a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 -- 172.21.15.33:0/1423306191 shutdown_connections 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.121+0000 7f6edf152740 1 -- 172.21.15.33:0/1423306191 wait complete. 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 -- start start 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 Processor -- start 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041b4d0 0x5612e042ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041bdf0 0x5612e0428ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0429480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5612e01e2700 con 0x5612e041b4d0 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5612e031c160 con 0x5612e042c190 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612e042ede0 con 0x5612e041bdf0 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6edf152740 10 client.0 _ll_register_callbacks cb 0x5612e03359e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041b4d0 0x5612e042ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041b4d0 0x5612e042ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34274/0 (socket says 172.21.15.33:34274) 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 -- 172.21.15.33:0/2598178868 learned_addr learned my addr 172.21.15.33:0/2598178868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041bdf0 0x5612e0428ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 -- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041bdf0 msgr2=0x5612e0428ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041bdf0 0x5612e0428ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 -- 172.21.15.33:0/2598178868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 msgr2=0x5612e0429480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0429480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ed659c640 1 -- 172.21.15.33:0/2598178868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e0349c90 con 0x5612e041b4d0 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.122+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041bdf0 0x5612e0428ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6ed659c640 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041b4d0 0x5612e042ba50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006010 tx=0x7f6ec802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85a1a8845a303c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6ebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6ebdffb640 1 -- 172.21.15.33:0/2598178868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8040070 con 0x5612e041b4d0 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6ebdffb640 1 -- 172.21.15.33:0/2598178868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6ec80314a0 con 0x5612e041b4d0 2024-09-10T21:11:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec80316a0 con 0x5612e041b4d0 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6ebdffb640 1 -- 172.21.15.33:0/2598178868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6ec8035040 con 0x5612e041b4d0 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 10 client.16821 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5612e043a710 con 0x5612e041b4d0 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6ebdffb640 1 -- 172.21.15.33:0/2598178868 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec80316a0 con 0x5612e041b4d0 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6ebdffb640 1 -- 172.21.15.33:0/2598178868 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f6ec8051020 con 0x5612e041b4d0 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 10 client.16821 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 10 client.16821 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 20 client.16821 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e043de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 4 client.16821 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.123+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612e0393df0 con 0x5612e043ba60 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.124+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e043de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.124+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e043de50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6569fb428c38e2f server_cookie=ddef589621881325 in_seq=0 out_seq=0 2024-09-10T21:11:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.124+0000 7f6ebdffb640 10 client.16821 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6ebdffb640 1 -- 172.21.15.33:0/2598178868 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5612e0393df0 con 0x5612e043ba60 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6ebdffb640 10 client.16821 handle_command_reply: tid=0 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 client.16821 shutdown 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041b4d0 msgr2=0x5612e042ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041b4d0 0x5612e042ba50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006010 tx=0x7f6ec802a410 comp rx=0 tx=0).stop 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 shutdown_connections 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0429480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041bdf0 0x5612e0428ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e043de50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6ebdffb640 0 client.16821 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 --2- 172.21.15.33:0/2598178868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041b4d0 0x5612e042ba50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 >> 172.21.15.33:0/2598178868 conn(0x5612e0419300 msgr2=0x5612e0418420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:45.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 shutdown_connections 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 1 -- 172.21.15.33:0/2598178868 wait complete. 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.125+0000 7f6edf152740 20 client.16821 trim_cache size 0 max 16384 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.138+0000 7f6edf152740 1 Processor -- start 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.138+0000 7f6edf152740 1 -- start start 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6edf152740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0438040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6edf152740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e0428390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6edf152740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0427720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6edf152740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ac130 con 0x5612e041bdf0 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6edf152740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81adc70 con 0x5612e042c190 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6edf152740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ac800 con 0x5612e041b4d0 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e0428390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e0428390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34286/0 (socket says 172.21.15.33:34286) 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/3501738660 learned_addr learned my addr 172.21.15.33:0/3501738660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ed6d9d640 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0427720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ed659c640 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0438040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/3501738660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 msgr2=0x5612e0438040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0438040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/3501738660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 msgr2=0x5612e0427720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0427720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/3501738660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612e04297f0 con 0x5612e041bdf0 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ed6d9d640 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0427720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e0428390 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0011e40 tx=0x7f6ec0013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54ac3d129d45b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ebffff640 1 -- 172.21.15.33:0/3501738660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00204c0 con 0x5612e041bdf0 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ebffff640 1 -- 172.21.15.33:0/3501738660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec0020b10 con 0x5612e041bdf0 2024-09-10T21:11:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.139+0000 7f6ebffff640 1 -- 172.21.15.33:0/3501738660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0029c30 con 0x5612e041bdf0 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 -- 172.21.15.33:0/3501738660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 msgr2=0x5612e0428390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e0428390 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0011e40 tx=0x7f6ec0013040 comp rx=0 tx=0).stop 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 -- 172.21.15.33:0/3501738660 shutdown_connections 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0427720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e0428390 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 --2- 172.21.15.33:0/3501738660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0438040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 -- 172.21.15.33:0/3501738660 >> 172.21.15.33:0/3501738660 conn(0x5612e0419300 msgr2=0x5612e042cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 -- 172.21.15.33:0/3501738660 shutdown_connections 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 -- 172.21.15.33:0/3501738660 wait complete. 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 -- start start 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.140+0000 7f6edf152740 1 Processor -- start 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6edf152740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0434280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6edf152740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e04347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6edf152740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0431070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6edf152740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81ad5a0 con 0x5612e041bdf0 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6edf152740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81aced0 con 0x5612e042c190 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6edf152740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed81aa0f0 con 0x5612e041b4d0 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6edf152740 10 client.0 _ll_register_callbacks cb 0x7f6ec000b4e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e04347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e04347c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34302/0 (socket says 172.21.15.33:34302) 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/258654467 learned_addr learned my addr 172.21.15.33:0/258654467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ed659c640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0434280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ed6d9d640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0431070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 msgr2=0x5612e0434280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0434280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/258654467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 msgr2=0x5612e0431070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0431070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 -- 172.21.15.33:0/258654467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612e0431710 con 0x5612e041bdf0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ed6d9d640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0431070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ecdd9b640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e04347c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0029910 tx=0x7f6ec0020f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65767fcd77a8ca39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.141+0000 7f6ebdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6ebdffb640 1 -- 172.21.15.33:0/258654467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00299b0 con 0x5612e041bdf0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6ebdffb640 1 -- 172.21.15.33:0/258654467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6ec0019040 con 0x5612e041bdf0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ec0020b10 con 0x5612e041bdf0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6ebdffb640 1 -- 172.21.15.33:0/258654467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6ec0029b50 con 0x5612e041bdf0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 10 client.16827 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5612e04a90b0 con 0x5612e041bdf0 2024-09-10T21:11:45.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6ebdffb640 1 -- 172.21.15.33:0/258654467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ec0020b10 con 0x5612e041bdf0 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6ebdffb640 1 -- 172.21.15.33:0/258654467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f6ec002f020 con 0x5612e041bdf0 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 10 client.16827 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 10 client.16827 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 20 client.16827 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e04ab790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 4 client.16827 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5612e038f8a0 con 0x5612e043ba60 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.142+0000 7f6ed659c640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e04ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6ed659c640 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e04ab790 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d3264b69a1bd609 server_cookie=7e451024573b21d1 in_seq=0 out_seq=0 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6ebdffb640 10 client.16827 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6ebdffb640 1 -- 172.21.15.33:0/258654467 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5612e038f8a0 con 0x5612e043ba60 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6ebdffb640 10 client.16827 handle_command_reply: tid=0 2024-09-10T21:11:45.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 client.16827 shutdown 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 msgr2=0x5612e04347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e04347c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0029910 tx=0x7f6ec0020f60 comp rx=0 tx=0).stop 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 shutdown_connections 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5612e042c190 0x5612e0431070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5612e043ba60 0x5612e04ab790 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5612e041bdf0 0x5612e04347c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 --2- 172.21.15.33:0/258654467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612e041b4d0 0x5612e0434280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6ebdffb640 0 client.16827 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 >> 172.21.15.33:0/258654467 conn(0x5612e0419300 msgr2=0x5612e04272f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 shutdown_connections 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 1 -- 172.21.15.33:0/258654467 wait complete. 2024-09-10T21:11:45.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.143+0000 7f6edf152740 20 client.16827 trim_cache size 0 max 16384 2024-09-10T21:11:45.165 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:11:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ec40b52e0 msgr2=0x7f6ec40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ec40b52e0 0x7f6ec40b77a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6ed810f800 tx=0x7f6ec0005f90 comp rx=0 tx=0).stop 2024-09-10T21:11:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed810d120 msgr2=0x7f6ed819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:45.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed810d120 0x7f6ed819ead0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000cad0 tx=0x7f6ed000cf90 comp rx=0 tx=0).stop 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 shutdown_connections 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ec40b52e0 0x7f6ec40b77a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ed810d120 0x7f6ed819ead0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6ed8104570 0x7f6ed8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 --2- 172.21.15.33:0/2207682544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ed8103b70 0x7f6ed81991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 >> 172.21.15.33:0/2207682544 conn(0x7f6ed80fd700 msgr2=0x7f6ed80fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 shutdown_connections 2024-09-10T21:11:45.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:45.145+0000 7f6edcd7d640 1 -- 172.21.15.33:0/2207682544 wait complete. 2024-09-10T21:11:45.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547225 5.82 MB/sec execute 286 sec latency 3622.465 ms 2024-09-10T21:11:45.852 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:11:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:45 smithi104 ceph-mon[35325]: pgmap v579: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-09-10T21:11:46.112 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:45 smithi174 ceph-mon[35635]: pgmap v579: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-09-10T21:11:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:45 smithi033 ceph-mon[30860]: pgmap v579: 97 pgs: 97 active+clean; 213 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-09-10T21:11:46.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 -- 172.21.15.33:0/820470228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410d250 msgr2=0x7fa48410f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 --2- 172.21.15.33:0/820470228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410d250 0x7fa48410f640 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009ca0 tx=0x7fa47c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 -- 172.21.15.33:0/820470228 shutdown_connections 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 --2- 172.21.15.33:0/820470228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410d250 0x7fa48410f640 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 --2- 172.21.15.33:0/820470228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa48410a920 0x7fa48410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 --2- 172.21.15.33:0/820470228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa484101810 0x7fa484101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 -- 172.21.15.33:0/820470228 >> 172.21.15.33:0/820470228 conn(0x7fa484076f50 msgr2=0x7fa484077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 -- 172.21.15.33:0/820470228 shutdown_connections 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.708+0000 7fa48b24d640 1 -- 172.21.15.33:0/820470228 wait complete. 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 Processor -- start 2024-09-10T21:11:46.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 -- start start 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa484101810 0x7fa484199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410a920 0x7fa484199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa48410d250 0x7fa48419eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa484112d90 con 0x7fa48410a920 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa484112c10 con 0x7fa484101810 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa48b24d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa484112f10 con 0x7fa48410d250 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410a920 0x7fa484199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410a920 0x7fa484199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34328/0 (socket says 172.21.15.33:34328) 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 -- 172.21.15.33:0/3104703966 learned_addr learned my addr 172.21.15.33:0/3104703966 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:46.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa4897ea640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa48410d250 0x7fa48419eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa488fe9640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa484101810 0x7fa484199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 -- 172.21.15.33:0/3104703966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa48410d250 msgr2=0x7fa48419eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa48410d250 0x7fa48419eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 -- 172.21.15.33:0/3104703966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa484101810 msgr2=0x7fa484199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa484101810 0x7fa484199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa483fff640 1 -- 172.21.15.33:0/3104703966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48419f2b0 con 0x7fa48410a920 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.709+0000 7fa488fe9640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa484101810 0x7fa484199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.710+0000 7fa483fff640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410a920 0x7fa484199740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa47400d950 tx=0x7fa47400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.710+0000 7fa481ffb640 1 -- 172.21.15.33:0/3104703966 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474017030 con 0x7fa48410a920 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.710+0000 7fa481ffb640 1 -- 172.21.15.33:0/3104703966 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa474004020 con 0x7fa48410a920 2024-09-10T21:11:46.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.710+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48419f5a0 con 0x7fa48410a920 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.710+0000 7fa481ffb640 1 -- 172.21.15.33:0/3104703966 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740047a0 con 0x7fa48410a920 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.710+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4841a5c60 con 0x7fa48410a920 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.712+0000 7fa481ffb640 1 -- 172.21.15.33:0/3104703966 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa474009a30 con 0x7fa48410a920 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.712+0000 7fa48d622740 1 Processor -- start 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa48d622740 1 -- start start 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa48d622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 0x55834c77e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa48d622740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77ea10 0x55834c78e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa48d622740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c7911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa48d622740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55834c6f4ea0 con 0x55834c78edb0 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa48d622740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55834c6e42d0 con 0x55834c77ea10 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa48d622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55834c6e80e0 con 0x55834c77e0f0 2024-09-10T21:11:46.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c7911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c7911a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34332/0 (socket says 172.21.15.33:34332) 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 -- 172.21.15.33:0/2803663095 learned_addr learned my addr 172.21.15.33:0/2803663095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa488fe9640 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77ea10 0x55834c78e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa481ffb640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa4700b52e0 0x7fa4700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 -- 172.21.15.33:0/2803663095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 msgr2=0x55834c77e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 0x55834c77e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 -- 172.21.15.33:0/2803663095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77ea10 msgr2=0x55834c78e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77ea10 0x55834c78e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa483fff640 1 -- 172.21.15.33:0/2803663095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834c442650 con 0x55834c78edb0 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.713+0000 7fa4897ea640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa4700b52e0 0x7fa4700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.714+0000 7fa481ffb640 1 -- 172.21.15.33:0/3104703966 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa4740e5640 con 0x7fa48410a920 2024-09-10T21:11:46.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.714+0000 7fa483fff640 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c7911a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa47409fb60 tx=0x7fa474021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0990d3dde1c15d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.714+0000 7fa4897ea640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa4700b52e0 0x7fa4700b77a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005cc0 tx=0x7fa47c005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.714+0000 7fa47affd640 1 -- 172.21.15.33:0/2803663095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740abb90 con 0x55834c78edb0 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.714+0000 7fa47affd640 1 -- 172.21.15.33:0/2803663095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47407f390 con 0x55834c78edb0 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.714+0000 7fa47affd640 1 -- 172.21.15.33:0/2803663095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740171d0 con 0x55834c78edb0 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 -- 172.21.15.33:0/2803663095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 msgr2=0x55834c7911a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c7911a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa47409fb60 tx=0x7fa474021000 comp rx=0 tx=0).stop 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 -- 172.21.15.33:0/2803663095 shutdown_connections 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c7911a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77ea10 0x55834c78e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 --2- 172.21.15.33:0/2803663095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 0x55834c77e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 -- 172.21.15.33:0/2803663095 >> 172.21.15.33:0/2803663095 conn(0x55834c77bf20 msgr2=0x55834c77d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 -- 172.21.15.33:0/2803663095 shutdown_connections 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 -- 172.21.15.33:0/2803663095 wait complete. 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 -- start start 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.715+0000 7fa48d622740 1 Processor -- start 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 0x55834c78e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77ea10 0x55834c78ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c78bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55834c545700 con 0x55834c77ea10 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55834c67f160 con 0x55834c78edb0 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55834c791a00 con 0x55834c77e0f0 2024-09-10T21:11:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 10 client.0 _ll_register_callbacks cb 0x55834c6989e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77ea10 0x55834c78ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77ea10 0x55834c78ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34348/0 (socket says 172.21.15.33:34348) 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 -- 172.21.15.33:0/4202589187 learned_addr learned my addr 172.21.15.33:0/4202589187 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa4897ea640 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c78bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 -- 172.21.15.33:0/4202589187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 msgr2=0x55834c78e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 0x55834c78e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 -- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 msgr2=0x55834c78bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c78bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 -- 172.21.15.33:0/4202589187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834c6acc90 con 0x55834c77ea10 2024-09-10T21:11:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa4897ea640 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c78bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa483fff640 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77ea10 0x55834c78ba70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa4740045f0 tx=0x7fa4740ab940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a291be1b94b091 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa463fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa463fff640 1 -- 172.21.15.33:0/4202589187 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4740ab530 con 0x55834c77ea10 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa463fff640 1 -- 172.21.15.33:0/4202589187 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa4740a8030 con 0x55834c77ea10 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.716+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa47407f390 con 0x55834c77ea10 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa463fff640 1 -- 172.21.15.33:0/4202589187 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa4740ab710 con 0x55834c77ea10 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 10 client.16839 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55834c79d2b0 con 0x55834c77ea10 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa463fff640 1 -- 172.21.15.33:0/4202589187 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa47407f390 con 0x55834c77ea10 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa463fff640 1 -- 172.21.15.33:0/4202589187 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fa4740930e0 con 0x55834c77ea10 2024-09-10T21:11:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 10 client.16839 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 10 client.16839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 20 client.16839 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c7a09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 4 client.16839 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55834c6f6df0 con 0x55834c79e600 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.717+0000 7fa488fe9640 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c7a09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.718+0000 7fa488fe9640 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c7a09f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d58519d754b62a4d server_cookie=668bfaabc5bde981 in_seq=0 out_seq=0 2024-09-10T21:11:46.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.718+0000 7fa463fff640 10 client.16839 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.718+0000 7fa463fff640 1 -- 172.21.15.33:0/4202589187 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55834c6f6df0 con 0x55834c79e600 2024-09-10T21:11:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.718+0000 7fa463fff640 10 client.16839 handle_command_reply: tid=0 2024-09-10T21:11:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.718+0000 7fa48d622740 1 client.16839 shutdown 2024-09-10T21:11:46.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77ea10 msgr2=0x55834c78ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77ea10 0x55834c78ba70 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa4740045f0 tx=0x7fa4740ab940 comp rx=0 tx=0).stop 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 shutdown_connections 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c7a09f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c78bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa463fff640 0 client.16839 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77ea10 0x55834c78ba70 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 --2- 172.21.15.33:0/4202589187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77e0f0 0x55834c78e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 >> 172.21.15.33:0/4202589187 conn(0x55834c77bf20 msgr2=0x55834c781a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 shutdown_connections 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 1 -- 172.21.15.33:0/4202589187 wait complete. 2024-09-10T21:11:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.719+0000 7fa48d622740 20 client.16839 trim_cache size 0 max 16384 2024-09-10T21:11:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 Processor -- start 2024-09-10T21:11:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 -- start start 2024-09-10T21:11:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77e0f0 0x55834c79ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c77b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c790ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841ac210 con 0x55834c77e0f0 2024-09-10T21:11:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841add50 con 0x55834c78edb0 2024-09-10T21:11:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.734+0000 7fa48d622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841ac8e0 con 0x55834c77ea10 2024-09-10T21:11:46.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547225 5.80 MB/sec execute 287 sec latency 4622.506 ms 2024-09-10T21:11:46.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77e0f0 0x55834c79ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77e0f0 0x55834c79ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34362/0 (socket says 172.21.15.33:34362) 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 -- 172.21.15.33:0/2925770190 learned_addr learned my addr 172.21.15.33:0/2925770190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa483fff640 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c77b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa4897ea640 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c790ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 -- 172.21.15.33:0/2925770190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 msgr2=0x55834c77b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c77b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 -- 172.21.15.33:0/2925770190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 msgr2=0x55834c790ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c790ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 -- 172.21.15.33:0/2925770190 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834c789f70 con 0x55834c77e0f0 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa483fff640 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c77b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:46.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa488fe9640 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77e0f0 0x55834c79ac30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00bc00 tx=0x7fa46c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed861da64a2f04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa47a7fc640 1 -- 172.21.15.33:0/2925770190 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005020 con 0x55834c77e0f0 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa47a7fc640 1 -- 172.21.15.33:0/2925770190 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c01e030 con 0x55834c77e0f0 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.735+0000 7fa47a7fc640 1 -- 172.21.15.33:0/2925770190 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa46c011440 con 0x55834c77e0f0 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 -- 172.21.15.33:0/2925770190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77e0f0 msgr2=0x55834c79ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77e0f0 0x55834c79ac30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00bc00 tx=0x7fa46c00bf90 comp rx=0 tx=0).stop 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 -- 172.21.15.33:0/2925770190 shutdown_connections 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c78edb0 0x55834c790ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c77b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 --2- 172.21.15.33:0/2925770190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c77e0f0 0x55834c79ac30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 -- 172.21.15.33:0/2925770190 >> 172.21.15.33:0/2925770190 conn(0x55834c77bf20 msgr2=0x55834c78fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 -- 172.21.15.33:0/2925770190 shutdown_connections 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 -- 172.21.15.33:0/2925770190 wait complete. 2024-09-10T21:11:46.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.736+0000 7fa48d622740 1 -- start start 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 1 Processor -- start 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77e0f0 0x55834c796c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c7971d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c793c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841ad680 con 0x55834c78edb0 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841acfb0 con 0x55834c77e0f0 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4841aa1d0 con 0x55834c77ea10 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa48d622740 10 client.0 _ll_register_callbacks cb 0x7fa46c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c793c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c793c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34374/0 (socket says 172.21.15.33:34374) 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 -- 172.21.15.33:0/2928400998 learned_addr learned my addr 172.21.15.33:0/2928400998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa488fe9640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77e0f0 0x55834c796c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa483fff640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c7971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 -- 172.21.15.33:0/2928400998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 msgr2=0x55834c7971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c7971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 -- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77e0f0 msgr2=0x55834c796c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:46.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77e0f0 0x55834c796c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.737+0000 7fa4897ea640 1 -- 172.21.15.33:0/2928400998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55834c797960 con 0x55834c78edb0 2024-09-10T21:11:46.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa483fff640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c7971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa4897ea640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c793c60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa47c035000 tx=0x7fa47c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac1a76cb4aa4b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa463fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa463fff640 1 -- 172.21.15.33:0/2928400998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c004040 con 0x55834c78edb0 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa46c01e030 con 0x55834c78edb0 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa463fff640 1 -- 172.21.15.33:0/2928400998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa47c0041e0 con 0x55834c78edb0 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa463fff640 1 -- 172.21.15.33:0/2928400998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa47c0043e0 con 0x55834c78edb0 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa48d622740 10 client.16845 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55834c80b7c0 con 0x55834c78edb0 2024-09-10T21:11:46.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa463fff640 1 -- 172.21.15.33:0/2928400998 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa46c01e030 con 0x55834c78edb0 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.738+0000 7fa463fff640 1 -- 172.21.15.33:0/2928400998 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fa47c0305c0 con 0x55834c78edb0 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa48d622740 10 client.16845 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa48d622740 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa48d622740 20 client.16845 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa48d622740 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c80e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa48d622740 4 client.16845 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55834c6f28a0 con 0x55834c79e600 2024-09-10T21:11:46.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa488fe9640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c80e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa488fe9640 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c80e000 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65f7231ef06ab4cc server_cookie=69610a751e689928 in_seq=0 out_seq=0 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa463fff640 10 client.16845 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa463fff640 1 -- 172.21.15.33:0/2928400998 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55834c6f28a0 con 0x55834c79e600 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa463fff640 10 client.16845 handle_command_reply: tid=0 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.739+0000 7fa48d622740 1 client.16845 shutdown 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 msgr2=0x55834c793c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c793c60 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa47c035000 tx=0x7fa47c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 shutdown_connections 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55834c79e600 0x55834c80e000 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55834c78edb0 0x55834c793c60 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834c77ea10 0x55834c7971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 --2- 172.21.15.33:0/2928400998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55834c77e0f0 0x55834c796c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 >> 172.21.15.33:0/2928400998 conn(0x55834c77bf20 msgr2=0x55834c790ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa463fff640 0 client.16845 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 shutdown_connections 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 1 -- 172.21.15.33:0/2928400998 wait complete. 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.740+0000 7fa48d622740 20 client.16845 trim_cache size 0 max 16384 2024-09-10T21:11:46.751 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:11:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa4700b52e0 msgr2=0x7fa4700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa4700b52e0 0x7fa4700b77a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa47c005cc0 tx=0x7fa47c005c50 comp rx=0 tx=0).stop 2024-09-10T21:11:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410a920 msgr2=0x7fa484199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:46.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410a920 0x7fa484199740 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fa47400d950 tx=0x7fa47400de10 comp rx=0 tx=0).stop 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 shutdown_connections 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa4700b52e0 0x7fa4700b77a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa48410d250 0x7fa48419eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa48410a920 0x7fa484199740 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 --2- 172.21.15.33:0/3104703966 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa484101810 0x7fa484199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 >> 172.21.15.33:0/3104703966 conn(0x7fa484076f50 msgr2=0x7fa4841005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 shutdown_connections 2024-09-10T21:11:46.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:46.741+0000 7fa48b24d640 1 -- 172.21.15.33:0/3104703966 wait complete. 2024-09-10T21:11:47.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 547982 5.79 MB/sec execute 288 sec latency 4985.282 ms 2024-09-10T21:11:47.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:47 smithi033 ceph-mon[30860]: pgmap v580: 97 pgs: 97 active+clean; 214 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-09-10T21:11:48.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:47 smithi104 ceph-mon[35325]: pgmap v580: 97 pgs: 97 active+clean; 214 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-09-10T21:11:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:47 smithi174 ceph-mon[35635]: pgmap v580: 97 pgs: 97 active+clean; 214 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-09-10T21:11:48.588 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:11:48.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:11:48.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 550730 5.82 MB/sec execute 289 sec latency 14.129 ms 2024-09-10T21:11:48.852 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:49.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 -- 172.21.15.33:0/3992693739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c81076a0 msgr2=0x7f50c8107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 --2- 172.21.15.33:0/3992693739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c81076a0 0x7f50c8107ae0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009900 tx=0x7f50b802a410 comp rx=0 tx=0).stop 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 -- 172.21.15.33:0/3992693739 shutdown_connections 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 --2- 172.21.15.33:0/3992693739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50c8108020 0x7f50c810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 --2- 172.21.15.33:0/3992693739 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c81076a0 0x7f50c8107ae0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 --2- 172.21.15.33:0/3992693739 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c8101c90 0x7f50c8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 -- 172.21.15.33:0/3992693739 >> 172.21.15.33:0/3992693739 conn(0x7f50c80fd700 msgr2=0x7f50c80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 -- 172.21.15.33:0/3992693739 shutdown_connections 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.321+0000 7f50d0613640 1 -- 172.21.15.33:0/3992693739 wait complete. 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 Processor -- start 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 -- start start 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c8101c90 0x7f50c8199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c81076a0 0x7f50c8199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50c8108020 0x7f50c819eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8112e00 con 0x7f50c8101c90 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8112c80 con 0x7f50c81076a0 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50d0613640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50c8112f80 con 0x7f50c8108020 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c8101c90 0x7f50c8199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c8101c90 0x7f50c8199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34384/0 (socket says 172.21.15.33:34384) 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 -- 172.21.15.33:0/4281012415 learned_addr learned my addr 172.21.15.33:0/4281012415 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50cdbae640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c81076a0 0x7f50c8199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50cebb0640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50c8108020 0x7f50c819eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 -- 172.21.15.33:0/4281012415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50c8108020 msgr2=0x7f50c819eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50c8108020 0x7f50c819eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 -- 172.21.15.33:0/4281012415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c81076a0 msgr2=0x7f50c8199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c81076a0 0x7f50c8199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.322+0000 7f50ce3af640 1 -- 172.21.15.33:0/4281012415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c819f2c0 con 0x7f50c8101c90 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.323+0000 7f50ce3af640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c8101c90 0x7f50c8199210 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f50c400d950 tx=0x7f50c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.323+0000 7f50bf7fe640 1 -- 172.21.15.33:0/4281012415 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c4017070 con 0x7f50c8101c90 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.323+0000 7f50bf7fe640 1 -- 172.21.15.33:0/4281012415 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c4009c30 con 0x7f50c8101c90 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.323+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c819f5b0 con 0x7f50c8101c90 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.323+0000 7f50bf7fe640 1 -- 172.21.15.33:0/4281012415 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c400f620 con 0x7f50c8101c90 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.323+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c81a5e40 con 0x7f50c8101c90 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.325+0000 7f50bf7fe640 1 -- 172.21.15.33:0/4281012415 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f50c4010460 con 0x7f50c8101c90 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.325+0000 7f50d29e8740 1 Processor -- start 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50d29e8740 1 -- start start 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14aea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14afa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14afcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50d29e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f14a60e60 con 0x556f14aea800 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50d29e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f14a50290 con 0x556f14ae9ee0 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50d29e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f14a540a0 con 0x556f14afaba0 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50ce3af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14afa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14aea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14aea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:34498/0 (socket says 172.21.15.33:34498) 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 -- 172.21.15.33:0/1521836856 learned_addr learned my addr 172.21.15.33:0/1521836856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cdbae640 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14afcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 -- 172.21.15.33:0/1521836856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 msgr2=0x556f14afcf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14afcf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 -- 172.21.15.33:0/1521836856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 msgr2=0x556f14afa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14afa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50cebb0640 1 -- 172.21.15.33:0/1521836856 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f147ae930 con 0x556f14ae9ee0 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.326+0000 7f50bf7fe640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50b40b5360 0x7f50b40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50cebb0640 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14aea2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007e50 tx=0x7f50c000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be05f1eaf885ff09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50ce3af640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50b40b5360 0x7f50b40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50bf7fe640 1 -- 172.21.15.33:0/4281012415 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50c40e57f0 con 0x7f50c8101c90 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50bcff9640 1 -- 172.21.15.33:0/1521836856 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0014070 con 0x556f14ae9ee0 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50bcff9640 1 -- 172.21.15.33:0/1521836856 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c000ee90 con 0x556f14ae9ee0 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50bcff9640 1 -- 172.21.15.33:0/1521836856 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0005340 con 0x556f14ae9ee0 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50d29e8740 1 -- 172.21.15.33:0/1521836856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 msgr2=0x556f14aea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50d29e8740 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14aea2c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f50c0007e50 tx=0x7f50c000e720 comp rx=0 tx=0).stop 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50ce3af640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50b40b5360 0x7f50b40b7820 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f50c40abf40 tx=0x7f50c4093040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50d29e8740 1 -- 172.21.15.33:0/1521836856 shutdown_connections 2024-09-10T21:11:49.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50d29e8740 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14afcf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50d29e8740 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14afa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50d29e8740 1 --2- 172.21.15.33:0/1521836856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14aea2c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.327+0000 7f50d29e8740 1 -- 172.21.15.33:0/1521836856 >> 172.21.15.33:0/1521836856 conn(0x556f14ae7d10 msgr2=0x556f14ae8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 -- 172.21.15.33:0/1521836856 shutdown_connections 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 -- 172.21.15.33:0/1521836856 wait complete. 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 -- start start 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 Processor -- start 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14afa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14af7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14af8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f148b1890 con 0x556f14ae9ee0 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f149ead40 con 0x556f14afaba0 2024-09-10T21:11:49.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f14afd850 con 0x556f14aea800 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50d29e8740 10 client.0 _ll_register_callbacks cb 0x556f14a4ec80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50cebb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14afa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50cdbae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14af7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50cebb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14afa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34410/0 (socket says 172.21.15.33:34410) 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.328+0000 7f50cebb0640 1 -- 172.21.15.33:0/3376008695 learned_addr learned my addr 172.21.15.33:0/3376008695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50cebb0640 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14af8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50cebb0640 1 -- 172.21.15.33:0/3376008695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 msgr2=0x556f14af7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50cebb0640 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14af7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50cebb0640 1 -- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 msgr2=0x556f14af8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50cebb0640 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14af8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50cebb0640 1 -- 172.21.15.33:0/3376008695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f14b06bb0 con 0x556f14ae9ee0 2024-09-10T21:11:49.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50cebb0640 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14afa660 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000f80 tx=0x7f50c00072a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde002af84246bc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50b1ffb640 1 -- 172.21.15.33:0/3376008695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0014070 con 0x556f14ae9ee0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50c000ee90 con 0x556f14ae9ee0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50b1ffb640 1 -- 172.21.15.33:0/3376008695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50c0010640 con 0x556f14ae9ee0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50b1ffb640 1 -- 172.21.15.33:0/3376008695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f50c0010a50 con 0x556f14ae9ee0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50d29e8740 10 client.16857 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.329+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556f14b09560 con 0x556f14ae9ee0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50b1ffb640 1 -- 172.21.15.33:0/3376008695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50c000ee90 con 0x556f14ae9ee0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50b1ffb640 1 -- 172.21.15.33:0/3376008695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f50c0005240 con 0x556f14ae9ee0 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50d29e8740 10 client.16857 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50d29e8740 10 client.16857 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50d29e8740 20 client.16857 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:49.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50d29e8740 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b0a970 0x556f14b0cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50d29e8740 4 client.16857 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f14a62db0 con 0x556f14b0a970 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50cdbae640 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b0a970 0x556f14b0cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50b1ffb640 10 client.16857 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.330+0000 7f50cdbae640 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b0a970 0x556f14b0cd60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fee7b03a3e2ab81 server_cookie=18c6b8f7da18bac1 in_seq=0 out_seq=0 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50b1ffb640 1 -- 172.21.15.33:0/3376008695 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f14a62db0 con 0x556f14b0a970 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50b1ffb640 10 client.16857 handle_command_reply: tid=0 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 client.16857 shutdown 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 msgr2=0x556f14afa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14afa660 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f50c0000f80 tx=0x7f50c00072a0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 shutdown_connections 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b0a970 0x556f14b0cd60 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14af8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14af7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 --2- 172.21.15.33:0/3376008695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14afa660 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 >> 172.21.15.33:0/3376008695 conn(0x556f14ae7d10 msgr2=0x556f14ae6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50b1ffb640 0 client.16857 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 shutdown_connections 2024-09-10T21:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.331+0000 7f50d29e8740 1 -- 172.21.15.33:0/3376008695 wait complete. 2024-09-10T21:11:49.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.332+0000 7f50d29e8740 20 client.16857 trim_cache size 0 max 16384 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 Processor -- start 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 -- start start 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14af5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14af6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14aebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81ac210 con 0x556f14aea800 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81add50 con 0x556f14ae9ee0 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50d29e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81ac8e0 con 0x556f14afaba0 2024-09-10T21:11:49.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50cdbae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14af6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50cdbae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14af6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34424/0 (socket says 172.21.15.33:34424) 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50cdbae640 1 -- 172.21.15.33:0/922282830 learned_addr learned my addr 172.21.15.33:0/922282830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50cebb0640 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14af5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50cebb0640 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14aebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.350+0000 7f50cdbae640 1 -- 172.21.15.33:0/922282830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 msgr2=0x556f14aebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50cdbae640 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14aebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50cdbae640 1 -- 172.21.15.33:0/922282830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 msgr2=0x556f14af5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50cdbae640 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14af5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50cdbae640 1 -- 172.21.15.33:0/922282830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f14ae86e0 con 0x556f14aea800 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50cebb0640 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14aebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50cdbae640 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14af6fe0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f50b802a930 tx=0x7f50b8005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be1b30a2ddc2484f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50b3fff640 1 -- 172.21.15.33:0/922282830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8035040 con 0x556f14aea800 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50b3fff640 1 -- 172.21.15.33:0/922282830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b80054f0 con 0x556f14aea800 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.351+0000 7f50b3fff640 1 -- 172.21.15.33:0/922282830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8003850 con 0x556f14aea800 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- 172.21.15.33:0/922282830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 msgr2=0x556f14af6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14af6fe0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f50b802a930 tx=0x7f50b8005e60 comp rx=0 tx=0).stop 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- 172.21.15.33:0/922282830 shutdown_connections 2024-09-10T21:11:49.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14afaba0 0x556f14aebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14aea800 0x556f14af6fe0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 --2- 172.21.15.33:0/922282830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14ae9ee0 0x556f14af5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- 172.21.15.33:0/922282830 >> 172.21.15.33:0/922282830 conn(0x556f14ae7d10 msgr2=0x556f14ae8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- 172.21.15.33:0/922282830 shutdown_connections 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- 172.21.15.33:0/922282830 wait complete. 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- start start 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 Processor -- start 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14b02cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14b03230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14b11350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81ad680 con 0x556f14ae9ee0 2024-09-10T21:11:49.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81acfb0 con 0x556f14afaba0 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50c81aa1d0 con 0x556f14aea800 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.352+0000 7f50d29e8740 10 client.0 _ll_register_callbacks cb 0x7f50c40eb400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14b02cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14b02cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34432/0 (socket says 172.21.15.33:34432) 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 -- 172.21.15.33:0/2413996967 learned_addr learned my addr 172.21.15.33:0/2413996967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cdbae640 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14b03230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14b11350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 -- 172.21.15.33:0/2413996967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 msgr2=0x556f14b03230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14b03230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 -- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 msgr2=0x556f14b11350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14b11350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 -- 172.21.15.33:0/2413996967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f14b119c0 con 0x556f14ae9ee0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50cebb0640 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14b02cf0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f50c0005ef0 tx=0x7f50c0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9cd4fc12543a72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50b1ffb640 1 -- 172.21.15.33:0/2413996967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0038020 con 0x556f14ae9ee0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50b1ffb640 1 -- 172.21.15.33:0/2413996967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50c0004e90 con 0x556f14ae9ee0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.353+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b80054f0 con 0x556f14ae9ee0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50b1ffb640 1 -- 172.21.15.33:0/2413996967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f50c0014040 con 0x556f14ae9ee0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 10 client.16863 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556f14b12c50 con 0x556f14ae9ee0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50b1ffb640 1 -- 172.21.15.33:0/2413996967 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b80054f0 con 0x556f14ae9ee0 2024-09-10T21:11:49.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50b1ffb640 1 -- 172.21.15.33:0/2413996967 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f50c0004400 con 0x556f14ae9ee0 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 10 client.16863 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 20 client.16863 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b78630 0x556f14b7aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 4 client.16863 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556f14b0d150 con 0x556f14b78630 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.354+0000 7f50cdbae640 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b78630 0x556f14b7aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.355+0000 7f50cdbae640 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b78630 0x556f14b7aa20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e365950cb6b0787 server_cookie=115d8effe53c824 in_seq=0 out_seq=0 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.355+0000 7f50b1ffb640 10 client.16863 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50b1ffb640 1 -- 172.21.15.33:0/2413996967 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556f14b0d150 con 0x556f14b78630 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50b1ffb640 10 client.16863 handle_command_reply: tid=0 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 client.16863 shutdown 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 msgr2=0x556f14b02cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14b02cf0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f50c0005ef0 tx=0x7f50c0010e90 comp rx=0 tx=0).stop 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 shutdown_connections 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556f14b78630 0x556f14b7aa20 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f14afaba0 0x556f14b11350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f14aea800 0x556f14b03230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 --2- 172.21.15.33:0/2413996967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f14ae9ee0 0x556f14b02cf0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50b1ffb640 0 client.16863 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 >> 172.21.15.33:0/2413996967 conn(0x556f14ae7d10 msgr2=0x556f14aed020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 shutdown_connections 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.356+0000 7f50d29e8740 1 -- 172.21.15.33:0/2413996967 wait complete. 2024-09-10T21:11:49.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d29e8740 20 client.16863 trim_cache size 0 max 16384 2024-09-10T21:11:49.370 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:11:49.370 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:11:49.370 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "13db7707-2d44-4d9b-bcfc-6a0f10383258", 2024-09-10T21:11:49.370 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:11:49.370 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50b40b5360 msgr2=0x7f50b40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50b40b5360 0x7f50b40b7820 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f50c40abf40 tx=0x7f50c4093040 comp rx=0 tx=0).stop 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c8101c90 msgr2=0x7f50c8199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c8101c90 0x7f50c8199210 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f50c400d950 tx=0x7f50c400de10 comp rx=0 tx=0).stop 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 shutdown_connections 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50b40b5360 0x7f50b40b7820 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50c8108020 0x7f50c819eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c81076a0 0x7f50c8199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 --2- 172.21.15.33:0/4281012415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50c8101c90 0x7f50c8199210 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 >> 172.21.15.33:0/4281012415 conn(0x7f50c80fd700 msgr2=0x7f50c80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 shutdown_connections 2024-09-10T21:11:49.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:49.357+0000 7f50d0613640 1 -- 172.21.15.33:0/4281012415 wait complete. 2024-09-10T21:11:49.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 551030 5.81 MB/sec execute 290 sec latency 671.798 ms 2024-09-10T21:11:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:49 smithi033 ceph-mon[30860]: pgmap v581: 97 pgs: 97 active+clean; 216 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-10T21:11:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:49 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:49 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:49 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:49.989 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:49 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:49 smithi104 ceph-mon[35325]: pgmap v581: 97 pgs: 97 active+clean; 216 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-10T21:11:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:49 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:49 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:49 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:50.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:49 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:11:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:49 smithi174 ceph-mon[35635]: pgmap v581: 97 pgs: 97 active+clean; 216 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1010 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-09-10T21:11:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:49 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:49 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:49 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:49 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.375 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:11:50.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 553373 5.83 MB/sec execute 291 sec latency 58.166 ms 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.833 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:50.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:11:50.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.834 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:50 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 -- 172.21.15.33:0/1278417912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2010101c90 msgr2=0x7f2010102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1278417912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2010101c90 0x7f2010102070 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f2000009ca0 tx=0x7f200002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 -- 172.21.15.33:0/1278417912 shutdown_connections 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1278417912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2010108020 0x7f201010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1278417912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20101076a0 0x7f2010107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1278417912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2010101c90 0x7f2010102070 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 -- 172.21.15.33:0/1278417912 >> 172.21.15.33:0/1278417912 conn(0x7f20100fd700 msgr2=0x7f20100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.916+0000 7f2015fc4640 1 -- 172.21.15.33:0/1278417912 shutdown_connections 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.917+0000 7f2015fc4640 1 -- 172.21.15.33:0/1278417912 wait complete. 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.917+0000 7f2015fc4640 1 Processor -- start 2024-09-10T21:11:50.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f2015fc4640 1 -- start start 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f2015fc4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2010101c90 0x7f20101992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f2015fc4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20101076a0 0x7f2010199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f2015fc4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2010108020 0x7f201019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f2015fc4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111ce0 con 0x7f20101076a0 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f2015fc4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111b60 con 0x7f2010101c90 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f2015fc4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111e60 con 0x7f2010108020 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20101076a0 0x7f2010199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2010101c90 0x7f20101992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20101076a0 0x7f2010199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57952/0 (socket says 172.21.15.33:57952) 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 -- 172.21.15.33:0/1590444463 learned_addr learned my addr 172.21.15.33:0/1590444463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200ffff640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2010108020 0x7f201019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 -- 172.21.15.33:0/1590444463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2010108020 msgr2=0x7f201019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2010108020 0x7f201019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 -- 172.21.15.33:0/1590444463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2010101c90 msgr2=0x7f20101992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2010101c90 0x7f20101992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.918+0000 7f200effd640 1 -- 172.21.15.33:0/1590444463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201019f370 con 0x7f20101076a0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.919+0000 7f200effd640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20101076a0 0x7f2010199800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800e9f0 tx=0x7f1ff800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.919+0000 7f200cff9640 1 -- 172.21.15.33:0/1590444463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800cd80 con 0x7f20101076a0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.919+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201019f660 con 0x7f20101076a0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.919+0000 7f200cff9640 1 -- 172.21.15.33:0/1590444463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff80054d0 con 0x7f20101076a0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.919+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101a5d20 con 0x7f20101076a0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.919+0000 7f200cff9640 1 -- 172.21.15.33:0/1590444463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8010690 con 0x7f20101076a0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 Processor -- start 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 -- start start 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f200cff9640 1 -- 172.21.15.33:0/1590444463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1ff8004e90 con 0x7f20101076a0 2024-09-10T21:11:50.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc0a0 0x560cb60fc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc9c0 0x560cb6108410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb610ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560cb6072e50 con 0x560cb60fc0a0 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560cb6062280 con 0x560cb60fc9c0 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f2018399740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560cb6066090 con 0x560cb6108950 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f200ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc0a0 0x560cb60fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f200f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc9c0 0x560cb6108410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f200effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb610ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f200ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc0a0 0x560cb60fc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57960/0 (socket says 172.21.15.33:57960) 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.922+0000 7f200ffff640 1 -- 172.21.15.33:0/2633438237 learned_addr learned my addr 172.21.15.33:0/2633438237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200ffff640 1 -- 172.21.15.33:0/2633438237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 msgr2=0x560cb610ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200ffff640 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb610ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200ffff640 1 -- 172.21.15.33:0/2633438237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc9c0 msgr2=0x560cb6108410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200ffff640 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc9c0 0x560cb6108410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200ffff640 1 -- 172.21.15.33:0/2633438237 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb5dc0650 con 0x560cb60fc0a0 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200f7fe640 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc9c0 0x560cb6108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200cff9640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ffc0b52e0 0x7f1ffc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200ffff640 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc0a0 0x560cb60fc480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2008009ca0 tx=0x7f2008011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42418f72b35e24c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200f7fe640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ffc0b52e0 0x7f1ffc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f2005ffb640 1 -- 172.21.15.33:0/2633438237 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200801e070 con 0x560cb60fc0a0 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f200cff9640 1 -- 172.21.15.33:0/1590444463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1ff80e6010 con 0x7f20101076a0 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f2005ffb640 1 -- 172.21.15.33:0/2633438237 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2008009e50 con 0x560cb60fc0a0 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.923+0000 7f2005ffb640 1 -- 172.21.15.33:0/2633438237 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200800ea10 con 0x560cb60fc0a0 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 -- 172.21.15.33:0/2633438237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc0a0 msgr2=0x560cb60fc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc0a0 0x560cb60fc480 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f2008009ca0 tx=0x7f2008011810 comp rx=0 tx=0).stop 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f200f7fe640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ffc0b52e0 0x7f1ffc0b77a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x560cb60fadb0 tx=0x7f2000009360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 -- 172.21.15.33:0/2633438237 shutdown_connections 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb610ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc9c0 0x560cb6108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 --2- 172.21.15.33:0/2633438237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc0a0 0x560cb60fc480 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 -- 172.21.15.33:0/2633438237 >> 172.21.15.33:0/2633438237 conn(0x560cb60f9ed0 msgr2=0x560cb60fb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 -- 172.21.15.33:0/2633438237 shutdown_connections 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 -- 172.21.15.33:0/2633438237 wait complete. 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 -- start start 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.924+0000 7f2018399740 1 Processor -- start 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f2018399740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb61082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f2018399740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb60fc9c0 0x560cb6105710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f2018399740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb6108950 0x560cb6105d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f2018399740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560cb5ec3700 con 0x560cb6108950 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f2018399740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560cb5ffd110 con 0x560cb60fc0a0 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f2018399740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560cb610b5d0 con 0x560cb60fc9c0 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f2018399740 10 client.0 _ll_register_callbacks cb 0x560cb6016990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb6108950 0x560cb6105d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb61082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb6108950 0x560cb6105d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57976/0 (socket says 172.21.15.33:57976) 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 -- 172.21.15.33:0/2609247268 learned_addr learned my addr 172.21.15.33:0/2609247268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb61082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:60090/0 (socket says 172.21.15.33:60090) 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200effd640 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb60fc9c0 0x560cb6105710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 -- 172.21.15.33:0/2609247268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb60fc9c0 msgr2=0x560cb6105710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb60fc9c0 0x560cb6105710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 -- 172.21.15.33:0/2609247268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 msgr2=0x560cb61082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb61082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 -- 172.21.15.33:0/2609247268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb602ac40 con 0x560cb6108950 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f200ffff640 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb6108950 0x560cb6105d90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f200800e590 tx=0x7f2008005520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22b629013fde7c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.925+0000 7f1feeffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f1feeffd640 1 -- 172.21.15.33:0/2609247268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008026020 con 0x560cb6108950 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f1feeffd640 1 -- 172.21.15.33:0/2609247268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f200800eba0 con 0x560cb6108950 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2008009e50 con 0x560cb6108950 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f1feeffd640 1 -- 172.21.15.33:0/2609247268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f200801e040 con 0x560cb6108950 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 10 client.16875 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x560cb611b390 con 0x560cb6108950 2024-09-10T21:11:50.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f1feeffd640 1 -- 172.21.15.33:0/2609247268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2008009e50 con 0x560cb6108950 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f1feeffd640 1 -- 172.21.15.33:0/2609247268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f2008019430 con 0x560cb6108950 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 10 client.16875 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 10 client.16875 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 20 client.16875 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb611c6e0 0x560cb611ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 4 client.16875 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.926+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cb6074da0 con 0x560cb611c6e0 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.927+0000 7f200f7fe640 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb611c6e0 0x560cb611ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.927+0000 7f1feeffd640 10 client.16875 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.927+0000 7f200f7fe640 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb611c6e0 0x560cb611ead0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba6b0934a8a994bb server_cookie=9e03eb3f2e85ec7c in_seq=0 out_seq=0 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f1feeffd640 1 -- 172.21.15.33:0/2609247268 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560cb6074da0 con 0x560cb611c6e0 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f1feeffd640 10 client.16875 handle_command_reply: tid=0 2024-09-10T21:11:50.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 client.16875 shutdown 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb6108950 msgr2=0x560cb6105d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb6108950 0x560cb6105d90 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f200800e590 tx=0x7f2008005520 comp rx=0 tx=0).stop 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 shutdown_connections 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb611c6e0 0x560cb611ead0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb6108950 0x560cb6105d90 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb60fc9c0 0x560cb6105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 --2- 172.21.15.33:0/2609247268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb61082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 >> 172.21.15.33:0/2609247268 conn(0x560cb60f9ed0 msgr2=0x560cb60ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f1feeffd640 0 client.16875 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 shutdown_connections 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 1 -- 172.21.15.33:0/2609247268 wait complete. 2024-09-10T21:11:50.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.928+0000 7f2018399740 20 client.16875 trim_cache size 0 max 16384 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 Processor -- start 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 -- start start 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6100110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb6118c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6109d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20101ac270 con 0x560cb60fc9c0 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20101addb0 con 0x560cb60fc0a0 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f2018399740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20101ac940 con 0x560cb6108950 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb6118c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6109d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb6118c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57990/0 (socket says 172.21.15.33:57990) 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6109d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:43904/0 (socket says 172.21.15.33:43904) 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 -- 172.21.15.33:0/718146356 learned_addr learned my addr 172.21.15.33:0/718146356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 -- 172.21.15.33:0/718146356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 msgr2=0x560cb6109d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 --2- 172.21.15.33:0/718146356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6109d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 -- 172.21.15.33:0/718146356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 msgr2=0x560cb6100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 --2- 172.21.15.33:0/718146356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.947+0000 7f200effd640 1 -- 172.21.15.33:0/718146356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cb60fc7f0 con 0x560cb60fc9c0 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.948+0000 7f200effd640 1 --2- 172.21.15.33:0/718146356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb6118c90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80e9830 tx=0x7f1ff80e9a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9221601555887f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.948+0000 7f20057fa640 1 -- 172.21.15.33:0/718146356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80a8670 con 0x560cb60fc9c0 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.948+0000 7f20057fa640 1 -- 172.21.15.33:0/718146356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff80ac2e0 con 0x560cb60fc9c0 2024-09-10T21:11:50.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.948+0000 7f20057fa640 1 -- 172.21.15.33:0/718146356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80abf40 con 0x560cb60fc9c0 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 -- 172.21.15.33:0/718146356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 msgr2=0x560cb6118c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 --2- 172.21.15.33:0/718146356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb6118c90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80e9830 tx=0x7f1ff80e9a90 comp rx=0 tx=0).stop 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 -- 172.21.15.33:0/718146356 shutdown_connections 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 --2- 172.21.15.33:0/718146356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 --2- 172.21.15.33:0/718146356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb6118c90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 --2- 172.21.15.33:0/718146356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 -- 172.21.15.33:0/718146356 >> 172.21.15.33:0/718146356 conn(0x560cb60f9ed0 msgr2=0x560cb61056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 -- 172.21.15.33:0/718146356 shutdown_connections 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 -- 172.21.15.33:0/718146356 wait complete. 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 -- start start 2024-09-10T21:11:50.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.949+0000 7f2018399740 1 Processor -- start 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f2018399740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6115160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f2018399740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb61156e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f2018399740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6123d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f2018399740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20101ad6e0 con 0x560cb60fc9c0 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f2018399740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20101ad010 con 0x560cb60fc0a0 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f2018399740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20101aa230 con 0x560cb6108950 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f2018399740 10 client.0 _ll_register_callbacks cb 0x7f200801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb61156e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6123d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6123d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:43908/0 (socket says 172.21.15.33:43908) 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200ffff640 1 -- 172.21.15.33:0/423823914 learned_addr learned my addr 172.21.15.33:0/423823914 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200f7fe640 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6115160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200effd640 1 -- 172.21.15.33:0/423823914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 msgr2=0x560cb6123d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200effd640 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6123d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200effd640 1 -- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 msgr2=0x560cb6115160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200effd640 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6115160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200effd640 1 -- 172.21.15.33:0/423823914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cb6124550 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.950+0000 7f200ffff640 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6123d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f200effd640 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb61156e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80e9830 tx=0x7f1ff80b1390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8580d5cf84dd1e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f1feeffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f1feeffd640 1 -- 172.21.15.33:0/423823914 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80ac120 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f1feeffd640 1 -- 172.21.15.33:0/423823914 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1ff80e8040 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ff80ac2e0 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f1feeffd640 1 -- 172.21.15.33:0/423823914 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f1ff80ac5a0 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f2018399740 10 client.16881 fetch_fsmap learned FSMap version 58 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x560cb6113a20 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f1feeffd640 1 -- 172.21.15.33:0/423823914 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ff80aced0 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.951+0000 7f1feeffd640 1 -- 172.21.15.33:0/423823914 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f1ff80b58d0 con 0x560cb60fc9c0 2024-09-10T21:11:50.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f2018399740 10 client.16881 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f2018399740 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f2018399740 20 client.16881 populate_metadata read hostname 'smithi033' 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f2018399740 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb6189750 0x560cb618bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f2018399740 4 client.16881 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1ff80ac2e0 con 0x560cb6189750 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f200f7fe640 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb6189750 0x560cb618bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f1feeffd640 10 client.16881 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.952+0000 7f200f7fe640 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb6189750 0x560cb618bb70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc3d97c48e9df1 server_cookie=57ae1f36f24f8b6f in_seq=0 out_seq=0 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f1feeffd640 1 -- 172.21.15.33:0/423823914 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3410 (crc 0 0 0) 0x7f1ff80ac2e0 con 0x560cb6189750 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f1feeffd640 10 client.16881 handle_command_reply: tid=0 2024-09-10T21:11:50.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 client.16881 shutdown 2024-09-10T21:11:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 msgr2=0x560cb61156e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb61156e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1ff80e9830 tx=0x7f1ff80b1390 comp rx=0 tx=0).stop 2024-09-10T21:11:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 shutdown_connections 2024-09-10T21:11:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x560cb6189750 0x560cb618bb70 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cb6108950 0x560cb6123d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560cb60fc9c0 0x560cb61156e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f1feeffd640 0 client.16881 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 --2- 172.21.15.33:0/423823914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560cb60fc0a0 0x560cb6115160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 >> 172.21.15.33:0/423823914 conn(0x560cb60f9ed0 msgr2=0x560cb6106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 shutdown_connections 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.953+0000 7f2018399740 1 -- 172.21.15.33:0/423823914 wait complete. 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2018399740 20 client.16881 trim_cache size 0 max 16384 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:11:50.962 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "13db7707-2d44-4d9b-bcfc-6a0f10383258", 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.963 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:11:50.965 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.966 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:11:50.969 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:11:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ffc0b52e0 msgr2=0x7f1ffc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ffc0b52e0 0x7f1ffc0b77a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x560cb60fadb0 tx=0x7f2000009360 comp rx=0 tx=0).stop 2024-09-10T21:11:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20101076a0 msgr2=0x7f2010199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:11:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20101076a0 0x7f2010199800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800e9f0 tx=0x7f1ff800edf0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 shutdown_connections 2024-09-10T21:11:50.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1ffc0b52e0 0x7f1ffc0b77a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2010108020 0x7f201019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20101076a0 0x7f2010199800 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 --2- 172.21.15.33:0/1590444463 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2010101c90 0x7f20101992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:11:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 >> 172.21.15.33:0/1590444463 conn(0x7f20100fd700 msgr2=0x7f201006adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:11:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 shutdown_connections 2024-09-10T21:11:50.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:11:50.954+0000 7f2015fc4640 1 -- 172.21.15.33:0/1590444463 wait complete. 2024-09-10T21:11:51.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:11:51.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:51.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:51.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:50 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:51.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:11:51.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:50 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:11:51.727 INFO:tasks.cephfs.filesystem:scrub status for tag:13db7707-2d44-4d9b-bcfc-6a0f10383258 - {'path': '/', 'tag': '13db7707-2d44-4d9b-bcfc-6a0f10383258', 'options': 'recursive,force'} 2024-09-10T21:11:51.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 554247 5.83 MB/sec execute 292 sec latency 635.401 ms 2024-09-10T21:11:52.065 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:51 smithi104 ceph-mon[35325]: pgmap v582: 97 pgs: 97 active+clean; 216 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-10T21:11:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:51 smithi174 ceph-mon[35635]: pgmap v582: 97 pgs: 97 active+clean; 216 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-10T21:11:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:51 smithi033 ceph-mon[30860]: pgmap v582: 97 pgs: 97 active+clean; 216 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 908 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-09-10T21:11:52.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 554317 5.81 MB/sec execute 293 sec latency 999.173 ms 2024-09-10T21:11:53.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 554317 5.79 MB/sec execute 294 sec latency 1574.137 ms 2024-09-10T21:11:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:53 smithi104 ceph-mon[35325]: pgmap v583: 97 pgs: 97 active+clean; 214 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1001 KiB/s wr, 122 op/s 2024-09-10T21:11:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:53 smithi174 ceph-mon[35635]: pgmap v583: 97 pgs: 97 active+clean; 214 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1001 KiB/s wr, 122 op/s 2024-09-10T21:11:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:53 smithi033 ceph-mon[30860]: pgmap v583: 97 pgs: 97 active+clean; 214 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1001 KiB/s wr, 122 op/s 2024-09-10T21:11:54.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 554317 5.77 MB/sec execute 295 sec latency 2574.260 ms 2024-09-10T21:11:55.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 554317 5.75 MB/sec execute 296 sec latency 3574.364 ms 2024-09-10T21:11:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:55 smithi104 ceph-mon[35325]: pgmap v584: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-10T21:11:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:55 smithi174 ceph-mon[35635]: pgmap v584: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-10T21:11:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:55 smithi033 ceph-mon[30860]: pgmap v584: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 172 op/s 2024-09-10T21:11:56.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 554317 5.73 MB/sec execute 297 sec latency 4574.411 ms 2024-09-10T21:11:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:57.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:57.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:57.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:57.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 555139 5.72 MB/sec execute 298 sec latency 4937.508 ms 2024-09-10T21:11:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:57 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:11:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:57 smithi104 ceph-mon[35325]: pgmap v585: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 910 KiB/s wr, 115 op/s 2024-09-10T21:11:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:57 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:11:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:57 smithi174 ceph-mon[35635]: pgmap v585: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 910 KiB/s wr, 115 op/s 2024-09-10T21:11:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:57 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:11:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:57 smithi033 ceph-mon[30860]: pgmap v585: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 764 KiB/s rd, 910 KiB/s wr, 115 op/s 2024-09-10T21:11:58.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557795 5.76 MB/sec execute 299 sec latency 52.458 ms 2024-09-10T21:11:59.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:11:59 smithi033 ceph-mon[30860]: pgmap v586: 97 pgs: 97 active+clean; 217 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 966 KiB/s wr, 127 op/s 2024-09-10T21:11:59.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:11:59 smithi104 ceph-mon[35325]: pgmap v586: 97 pgs: 97 active+clean; 217 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 966 KiB/s wr, 127 op/s 2024-09-10T21:11:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:11:59 smithi174 ceph-mon[35635]: pgmap v586: 97 pgs: 97 active+clean; 217 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 966 KiB/s wr, 127 op/s 2024-09-10T21:11:59.739 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557795 5.74 MB/sec execute 300 sec latency 1008.649 ms 2024-09-10T21:12:00.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557795 5.72 MB/sec execute 301 sec latency 2008.755 ms 2024-09-10T21:12:01.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:01 smithi104 ceph-mon[35325]: pgmap v587: 97 pgs: 97 active+clean; 217 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 884 KiB/s wr, 123 op/s 2024-09-10T21:12:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:01 smithi174 ceph-mon[35635]: pgmap v587: 97 pgs: 97 active+clean; 217 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 884 KiB/s wr, 123 op/s 2024-09-10T21:12:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:01 smithi033 ceph-mon[30860]: pgmap v587: 97 pgs: 97 active+clean; 217 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 851 KiB/s rd, 884 KiB/s wr, 123 op/s 2024-09-10T21:12:01.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557795 5.70 MB/sec execute 302 sec latency 3008.875 ms 2024-09-10T21:12:02.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557846 5.68 MB/sec execute 303 sec latency 3372.824 ms 2024-09-10T21:12:03.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:03 smithi104 ceph-mon[35325]: pgmap v588: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 668 KiB/s wr, 112 op/s 2024-09-10T21:12:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:03 smithi174 ceph-mon[35635]: pgmap v588: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 668 KiB/s wr, 112 op/s 2024-09-10T21:12:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:03 smithi033 ceph-mon[30860]: pgmap v588: 97 pgs: 97 active+clean; 216 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 777 KiB/s rd, 668 KiB/s wr, 112 op/s 2024-09-10T21:12:03.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557846 5.66 MB/sec execute 304 sec latency 1593.262 ms 2024-09-10T21:12:04.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557846 5.64 MB/sec execute 305 sec latency 2593.380 ms 2024-09-10T21:12:05.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:05 smithi104 ceph-mon[35325]: pgmap v589: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 753 KiB/s wr, 132 op/s 2024-09-10T21:12:05.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:05 smithi174 ceph-mon[35635]: pgmap v589: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 753 KiB/s wr, 132 op/s 2024-09-10T21:12:05.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:05 smithi033 ceph-mon[30860]: pgmap v589: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 865 KiB/s rd, 753 KiB/s wr, 132 op/s 2024-09-10T21:12:05.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557846 5.63 MB/sec execute 306 sec latency 3593.487 ms 2024-09-10T21:12:06.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 557846 5.61 MB/sec execute 307 sec latency 4593.594 ms 2024-09-10T21:12:07.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:07 smithi104 ceph-mon[35325]: pgmap v590: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 686 KiB/s wr, 76 op/s 2024-09-10T21:12:07.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:07 smithi174 ceph-mon[35635]: pgmap v590: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 686 KiB/s wr, 76 op/s 2024-09-10T21:12:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:07 smithi033 ceph-mon[30860]: pgmap v590: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 686 KiB/s wr, 76 op/s 2024-09-10T21:12:07.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 558629 5.60 MB/sec execute 308 sec latency 4955.669 ms 2024-09-10T21:12:08.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 561407 5.63 MB/sec execute 309 sec latency 19.988 ms 2024-09-10T21:12:09.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:09 smithi104 ceph-mon[35325]: pgmap v591: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 856 KiB/s wr, 88 op/s 2024-09-10T21:12:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:09 smithi174 ceph-mon[35635]: pgmap v591: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 856 KiB/s wr, 88 op/s 2024-09-10T21:12:09.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:09 smithi033 ceph-mon[30860]: pgmap v591: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 856 KiB/s wr, 88 op/s 2024-09-10T21:12:09.740 INFO:tasks.workunit.client.0.smithi033.stdout: 1 561407 5.62 MB/sec execute 310 sec latency 1020.075 ms 2024-09-10T21:12:10.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 561407 5.60 MB/sec execute 311 sec latency 2020.183 ms 2024-09-10T21:12:11.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:11 smithi104 ceph-mon[35325]: pgmap v592: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 951 KiB/s wr, 89 op/s 2024-09-10T21:12:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:11 smithi174 ceph-mon[35635]: pgmap v592: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 951 KiB/s wr, 89 op/s 2024-09-10T21:12:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:11 smithi033 ceph-mon[30860]: pgmap v592: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 510 KiB/s rd, 951 KiB/s wr, 89 op/s 2024-09-10T21:12:11.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 561407 5.58 MB/sec execute 312 sec latency 3020.297 ms 2024-09-10T21:12:12.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 562121 5.57 MB/sec execute 313 sec latency 3389.244 ms 2024-09-10T21:12:13.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:13 smithi104 ceph-mon[35325]: pgmap v593: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 926 KiB/s wr, 90 op/s 2024-09-10T21:12:13.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:13 smithi174 ceph-mon[35635]: pgmap v593: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 926 KiB/s wr, 90 op/s 2024-09-10T21:12:13.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:13 smithi033 ceph-mon[30860]: pgmap v593: 97 pgs: 97 active+clean; 218 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 453 KiB/s rd, 926 KiB/s wr, 90 op/s 2024-09-10T21:12:13.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 564880 5.60 MB/sec execute 314 sec latency 10.612 ms 2024-09-10T21:12:14.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 564955 5.59 MB/sec execute 315 sec latency 932.025 ms 2024-09-10T21:12:15.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:15 smithi174 ceph-mon[35635]: pgmap v594: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T21:12:15.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:15 smithi033 ceph-mon[30860]: pgmap v594: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T21:12:15.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 564955 5.57 MB/sec execute 316 sec latency 1932.120 ms 2024-09-10T21:12:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:15 smithi104 ceph-mon[35325]: pgmap v594: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.2 MiB/s wr, 117 op/s 2024-09-10T21:12:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:12:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:12:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:12:16.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:12:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:12:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:12:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:12:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:12:16.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 564955 5.55 MB/sec execute 317 sec latency 2932.213 ms 2024-09-10T21:12:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:12:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:12:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:12:16.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:12:17.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:17 smithi174 ceph-mon[35635]: pgmap v595: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T21:12:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:17 smithi033 ceph-mon[30860]: pgmap v595: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T21:12:17.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 565773 5.54 MB/sec execute 318 sec latency 3292.144 ms 2024-09-10T21:12:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:17 smithi104 ceph-mon[35325]: pgmap v595: 97 pgs: 97 active+clean; 219 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 600 KiB/s rd, 1.2 MiB/s wr, 102 op/s 2024-09-10T21:12:18.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568500 5.58 MB/sec execute 319 sec latency 15.807 ms 2024-09-10T21:12:19.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:19 smithi174 ceph-mon[35635]: pgmap v596: 97 pgs: 97 active+clean; 219 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T21:12:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:19 smithi033 ceph-mon[30860]: pgmap v596: 97 pgs: 97 active+clean; 219 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T21:12:19.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568500 5.56 MB/sec execute 320 sec latency 1002.945 ms 2024-09-10T21:12:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:19 smithi104 ceph-mon[35325]: pgmap v596: 97 pgs: 97 active+clean; 219 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-09-10T21:12:20.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568500 5.54 MB/sec execute 321 sec latency 2003.062 ms 2024-09-10T21:12:21.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:21 smithi174 ceph-mon[35635]: pgmap v597: 97 pgs: 97 active+clean; 219 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-10T21:12:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:21 smithi033 ceph-mon[30860]: pgmap v597: 97 pgs: 97 active+clean; 219 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-10T21:12:21.728 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:12:21.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 568500 5.52 MB/sec execute 322 sec latency 3003.110 ms 2024-09-10T21:12:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:21 smithi104 ceph-mon[35325]: pgmap v597: 97 pgs: 97 active+clean; 219 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 948 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-09-10T21:12:22.015 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:22.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 -- 172.21.15.33:0/306158228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e01076a0 msgr2=0x7f33e0107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 --2- 172.21.15.33:0/306158228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e01076a0 0x7f33e0107ae0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009900 tx=0x7f33c802a410 comp rx=0 tx=0).stop 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 -- 172.21.15.33:0/306158228 shutdown_connections 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 --2- 172.21.15.33:0/306158228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33e0108020 0x7f33e010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 --2- 172.21.15.33:0/306158228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e01076a0 0x7f33e0107ae0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 --2- 172.21.15.33:0/306158228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e0101c90 0x7f33e0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 -- 172.21.15.33:0/306158228 >> 172.21.15.33:0/306158228 conn(0x7f33e00fd700 msgr2=0x7f33e00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 -- 172.21.15.33:0/306158228 shutdown_connections 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.516+0000 7f33e5230640 1 -- 172.21.15.33:0/306158228 wait complete. 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 Processor -- start 2024-09-10T21:12:22.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 -- start start 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e0101c90 0x7f33e00fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e01076a0 0x7f33e00fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33e0108020 0x7f33e00fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33e0111ce0 con 0x7f33e0101c90 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f33e0111b60 con 0x7f33e01076a0 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33e5230640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33e0111e60 con 0x7f33e0108020 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e0101c90 0x7f33e00fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e01076a0 0x7f33e00fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e01076a0 0x7f33e00fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37588/0 (socket says 172.21.15.33:37588) 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e0101c90 0x7f33e00fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42346/0 (socket says 172.21.15.33:42346) 2024-09-10T21:12:22.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33d7fff640 1 -- 172.21.15.33:0/2098023906 learned_addr learned my addr 172.21.15.33:0/2098023906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.517+0000 7f33df59e640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33e0108020 0x7f33e00fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33d7fff640 1 -- 172.21.15.33:0/2098023906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33e0108020 msgr2=0x7f33e00fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33d7fff640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33e0108020 0x7f33e00fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33d7fff640 1 -- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e0101c90 msgr2=0x7f33e00fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33d7fff640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e0101c90 0x7f33e00fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33d7fff640 1 -- 172.21.15.33:0/2098023906 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33e01a74a0 con 0x7f33e01076a0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33d7fff640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e01076a0 0x7f33e00fe140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f33c8030650 tx=0x7f33c8030680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33dcd99640 1 -- 172.21.15.33:0/2098023906 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c803f070 con 0x7f33e01076a0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33e01a7730 con 0x7f33e01076a0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33dcd99640 1 -- 172.21.15.33:0/2098023906 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c8004460 con 0x7f33e01076a0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33dcd99640 1 -- 172.21.15.33:0/2098023906 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8035030 con 0x7f33e01076a0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.518+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33e01a7c60 con 0x7f33e01076a0 2024-09-10T21:12:22.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.521+0000 7f33e7605740 1 Processor -- start 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33e7605740 1 -- start start 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33dcd99640 1 -- 172.21.15.33:0/2098023906 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f33c803b460 con 0x7f33e01076a0 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33e7605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61ad480 0x556cb61ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33e7605740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33e7605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61bc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33e7605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cb6123e50 con 0x556cb61b9d30 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33e7605740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556cb6113280 con 0x556cb61adda0 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33e7605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cb6117090 con 0x556cb61ad480 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33df59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61ad480 0x556cb61ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33df59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61ad480 0x556cb61ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34694/0 (socket says 172.21.15.33:34694) 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61b97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37596/0 (socket says 172.21.15.33:37596) 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33df59e640 1 -- 172.21.15.33:0/3001887603 learned_addr learned my addr 172.21.15.33:0/3001887603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.522+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61bc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33df59e640 1 -- 172.21.15.33:0/3001887603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 msgr2=0x556cb61b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33df59e640 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33df59e640 1 -- 172.21.15.33:0/3001887603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 msgr2=0x556cb61bc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33df59e640 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61bc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33df59e640 1 -- 172.21.15.33:0/3001887603 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb5e71650 con 0x556cb61ad480 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33d7fff640 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33df59e640 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61ad480 0x556cb61ad860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f90 tx=0x7f33d80118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f57ccfefd4a2db3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33d57fa640 1 -- 172.21.15.33:0/3001887603 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8004d20 con 0x556cb61ad480 2024-09-10T21:12:22.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33dcd99640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33cc0b52e0 0x7f33cc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33d57fa640 1 -- 172.21.15.33:0/3001887603 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d8004050 con 0x556cb61ad480 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33d57fa640 1 -- 172.21.15.33:0/3001887603 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8005280 con 0x556cb61ad480 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33ded9d640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33cc0b52e0 0x7f33cc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.523+0000 7f33dcd99640 1 -- 172.21.15.33:0/2098023906 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f33c8106740 con 0x7f33e01076a0 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 -- 172.21.15.33:0/3001887603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61ad480 msgr2=0x556cb61ad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33ded9d640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33cc0b52e0 0x7f33cc0b77a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x556cb61ac190 tx=0x7f33d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61ad480 0x556cb61ad860 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009f90 tx=0x7f33d80118d0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 -- 172.21.15.33:0/3001887603 shutdown_connections 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61bc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 --2- 172.21.15.33:0/3001887603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61ad480 0x556cb61ad860 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 -- 172.21.15.33:0/3001887603 >> 172.21.15.33:0/3001887603 conn(0x556cb61ab2b0 msgr2=0x556cb61ac450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 -- 172.21.15.33:0/3001887603 shutdown_connections 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.524+0000 7f33e7605740 1 -- 172.21.15.33:0/3001887603 wait complete. 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 -- start start 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 Processor -- start 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61ad480 0x556cb61b9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:12:22.659 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:12:22.660 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.661 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:12:22.662 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.663 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.664 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.665 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:12:22.666 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:12:22.667 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:12:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556cb5f74700 con 0x556cb61b9d30 2024-09-10T21:12:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556cb60ae110 con 0x556cb61ad480 2024-09-10T21:12:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cb61bc9b0 con 0x556cb61adda0 2024-09-10T21:12:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61ad480 0x556cb61b9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61ad480 0x556cb61b9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37606/0 (socket says 172.21.15.33:37606) 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33ded9d640 1 -- 172.21.15.33:0/2413795588 learned_addr learned my addr 172.21.15.33:0/2413795588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33d7fff640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33e7605740 10 client.0 _ll_register_callbacks cb 0x556cb60c7990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.525+0000 7f33df59e640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33ded9d640 1 -- 172.21.15.33:0/2413795588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 msgr2=0x556cb61b6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33ded9d640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61b6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33ded9d640 1 -- 172.21.15.33:0/2413795588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 msgr2=0x556cb61b70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33ded9d640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61b70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33ded9d640 1 -- 172.21.15.33:0/2413795588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb60dbc40 con 0x556cb61ad480 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33df59e640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33ded9d640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61ad480 0x556cb61b9620 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009730 tx=0x7f33d0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6589232586a209bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:12:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33c67fc640 1 -- 172.21.15.33:0/2413795588 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d001f640 con 0x556cb61ad480 2024-09-10T21:12:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33c67fc640 1 -- 172.21.15.33:0/2413795588 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f33d001f7e0 con 0x556cb61ad480 2024-09-10T21:12:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d8004050 con 0x556cb61ad480 2024-09-10T21:12:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33c67fc640 1 -- 172.21.15.33:0/2413795588 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f33d001f9e0 con 0x556cb61ad480 2024-09-10T21:12:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33e7605740 10 client.26375 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.526+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556cb61c8320 con 0x556cb61ad480 2024-09-10T21:12:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33c67fc640 1 -- 172.21.15.33:0/2413795588 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d8004050 con 0x556cb61ad480 2024-09-10T21:12:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33c67fc640 1 -- 172.21.15.33:0/2413795588 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f33d0027070 con 0x556cb61ad480 2024-09-10T21:12:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33e7605740 10 client.26375 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33e7605740 10 client.26375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33e7605740 20 client.26375 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33e7605740 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb61c97d0 0x556cb61cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33e7605740 4 client.26375 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cb6125da0 con 0x556cb61c97d0 2024-09-10T21:12:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33d7fff640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb61c97d0 0x556cb61cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33c67fc640 10 client.26375 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.527+0000 7f33d7fff640 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb61c97d0 0x556cb61cbbf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9add43abfd993fd server_cookie=c3922c0d54cebbda in_seq=0 out_seq=0 2024-09-10T21:12:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33c67fc640 1 -- 172.21.15.33:0/2413795588 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556cb6125da0 con 0x556cb61c97d0 2024-09-10T21:12:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33c67fc640 10 client.26375 handle_command_reply: tid=0 2024-09-10T21:12:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 client.26375 shutdown 2024-09-10T21:12:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61ad480 msgr2=0x556cb61b9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61ad480 0x556cb61b9620 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009730 tx=0x7f33d0004730 comp rx=0 tx=0).stop 2024-09-10T21:12:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 shutdown_connections 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb61c97d0 0x556cb61cbbf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61b9d30 0x556cb61b70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61b6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 --2- 172.21.15.33:0/2413795588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61ad480 0x556cb61b9620 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33c67fc640 0 client.26375 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 >> 172.21.15.33:0/2413795588 conn(0x556cb61ab2b0 msgr2=0x556cb61b0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 shutdown_connections 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.528+0000 7f33e7605740 1 -- 172.21.15.33:0/2413795588 wait complete. 2024-09-10T21:12:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.529+0000 7f33e7605740 20 client.26375 trim_cache size 0 max 16384 2024-09-10T21:12:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.548+0000 7f33e7605740 1 Processor -- start 2024-09-10T21:12:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.548+0000 7f33e7605740 1 -- start start 2024-09-10T21:12:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33e7605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33e7605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33e7605740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61b9d30 0x556cb61bb660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33e7605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01b0590 con 0x556cb61ad480 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33e7605740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01b20d0 con 0x556cb61b9d30 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33e7605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01b0c60 con 0x556cb61adda0 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42390/0 (socket says 172.21.15.33:42390) 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33df59e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61b9d30 0x556cb61bb660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33ded9d640 1 -- 172.21.15.33:0/1539474578 learned_addr learned my addr 172.21.15.33:0/1539474578 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33d7fff640 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61bb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33df59e640 1 -- 172.21.15.33:0/1539474578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 msgr2=0x556cb61bb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33df59e640 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61bb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33df59e640 1 -- 172.21.15.33:0/1539474578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 msgr2=0x556cb61aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33df59e640 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33df59e640 1 -- 172.21.15.33:0/1539474578 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb61ac990 con 0x556cb61b9d30 2024-09-10T21:12:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33ded9d640 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.549+0000 7f33df59e640 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61b9d30 0x556cb61bb660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33d801b040 tx=0x7f33d80045c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f3b5667845faac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.550+0000 7f33d4ff9640 1 -- 172.21.15.33:0/1539474578 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8005bd0 con 0x556cb61b9d30 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.550+0000 7f33d4ff9640 1 -- 172.21.15.33:0/1539474578 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33d80047b0 con 0x556cb61b9d30 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.550+0000 7f33d4ff9640 1 -- 172.21.15.33:0/1539474578 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d8005280 con 0x556cb61b9d30 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 -- 172.21.15.33:0/1539474578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61b9d30 msgr2=0x556cb61bb660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61b9d30 0x556cb61bb660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f33d801b040 tx=0x7f33d80045c0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 -- 172.21.15.33:0/1539474578 shutdown_connections 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61b9d30 0x556cb61bb660 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61adda0 0x556cb61bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 --2- 172.21.15.33:0/1539474578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 -- 172.21.15.33:0/1539474578 >> 172.21.15.33:0/1539474578 conn(0x556cb61ab2b0 msgr2=0x556cb61b80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 -- 172.21.15.33:0/1539474578 shutdown_connections 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 -- 172.21.15.33:0/1539474578 wait complete. 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 -- start start 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.551+0000 7f33e7605740 1 Processor -- start 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33e7605740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61c1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33e7605740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61c2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33e7605740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61b9d30 0x556cb61bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33e7605740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01b1a00 con 0x556cb61ad480 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33e7605740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01b1330 con 0x556cb61adda0 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33e7605740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33e01ae550 con 0x556cb61b9d30 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33e7605740 10 client.0 _ll_register_callbacks cb 0x7f33d0009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61c1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33d7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61c2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33ded9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61c1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42404/0 (socket says 172.21.15.33:42404) 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33ded9d640 1 -- 172.21.15.33:0/545325231 learned_addr learned my addr 172.21.15.33:0/545325231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33df59e640 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61b9d30 0x556cb61bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33d7fff640 1 -- 172.21.15.33:0/545325231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61b9d30 msgr2=0x556cb61bcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33d7fff640 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61b9d30 0x556cb61bcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33d7fff640 1 -- 172.21.15.33:0/545325231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 msgr2=0x556cb61c1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33d7fff640 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61c1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33d7fff640 1 -- 172.21.15.33:0/545325231 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb61bd390 con 0x556cb61adda0 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33ded9d640 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33d7fff640 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61c2390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33c80306f0 tx=0x7f33c802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1014bb0f41b7533 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.552+0000 7f33c67fc640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33c67fc640 1 -- 172.21.15.33:0/545325231 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c80cd600 con 0x556cb61adda0 2024-09-10T21:12:22.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33c67fc640 1 -- 172.21.15.33:0/545325231 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f33c8003d30 con 0x556cb61adda0 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d80047b0 con 0x556cb61adda0 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33c67fc640 1 -- 172.21.15.33:0/545325231 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f33c80ccf00 con 0x556cb61adda0 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 10 client.26381 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x556cb62371b0 con 0x556cb61adda0 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33c67fc640 1 -- 172.21.15.33:0/545325231 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c80d19e0 con 0x556cb61adda0 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33c67fc640 1 -- 172.21.15.33:0/545325231 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f33c80cd1d0 con 0x556cb61adda0 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 10 client.26381 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 10 client.26381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 20 client.26381 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb6237890 0x556cb6239cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 4 client.26381 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f33d80047b0 con 0x556cb6237890 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.553+0000 7f33ded9d640 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb6237890 0x556cb6239cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33c67fc640 10 client.26381 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33ded9d640 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb6237890 0x556cb6239cb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caad0bcca3126eff server_cookie=1127b762c852fede in_seq=0 out_seq=0 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33c67fc640 1 -- 172.21.15.33:0/545325231 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x7f33d80047b0 con 0x556cb6237890 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33c67fc640 10 client.26381 handle_command_reply: tid=0 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 client.26381 shutdown 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 msgr2=0x556cb61c2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61c2390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f33c80306f0 tx=0x7f33c802a410 comp rx=0 tx=0).stop 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 shutdown_connections 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556cb6237890 0x556cb6239cb0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cb61b9d30 0x556cb61bcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556cb61adda0 0x556cb61c2390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 --2- 172.21.15.33:0/545325231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556cb61ad480 0x556cb61c1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 >> 172.21.15.33:0/545325231 conn(0x556cb61ab2b0 msgr2=0x556cb61b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33c67fc640 0 client.26381 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 shutdown_connections 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.554+0000 7f33e7605740 1 -- 172.21.15.33:0/545325231 wait complete. 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e7605740 20 client.26381 trim_cache size 0 max 16384 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33cc0b52e0 msgr2=0x7f33cc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33cc0b52e0 0x7f33cc0b77a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x556cb61ac190 tx=0x7f33d0005f90 comp rx=0 tx=0).stop 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e01076a0 msgr2=0x7f33e00fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e01076a0 0x7f33e00fe140 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f33c8030650 tx=0x7f33c8030680 comp rx=0 tx=0).stop 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 shutdown_connections 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33cc0b52e0 0x7f33cc0b77a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33e0108020 0x7f33e00fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f33e01076a0 0x7f33e00fe140 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 --2- 172.21.15.33:0/2098023906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f33e0101c90 0x7f33e00fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 >> 172.21.15.33:0/2098023906 conn(0x7f33e00fd700 msgr2=0x7f33e006ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 shutdown_connections 2024-09-10T21:12:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:22.555+0000 7f33e5230640 1 -- 172.21.15.33:0/2098023906 wait complete. 2024-09-10T21:12:22.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 569171 5.51 MB/sec execute 323 sec latency 3370.066 ms 2024-09-10T21:12:23.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:23 smithi033 ceph-mon[30860]: pgmap v598: 97 pgs: 97 active+clean; 221 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-10T21:12:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:23 smithi174 ceph-mon[35635]: pgmap v598: 97 pgs: 97 active+clean; 221 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-10T21:12:23.656 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:12:23.657 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:12:23.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 571974 5.55 MB/sec execute 324 sec latency 30.062 ms 2024-09-10T21:12:23.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:23 smithi104 ceph-mon[35325]: pgmap v598: 97 pgs: 97 active+clean; 221 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 911 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-09-10T21:12:23.961 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:24.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 -- 172.21.15.33:0/3750100234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 msgr2=0x7faca01031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 --2- 172.21.15.33:0/3750100234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca01031a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009900 tx=0x7fac9c02a410 comp rx=0 tx=0).stop 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 -- 172.21.15.33:0/3750100234 shutdown_connections 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 --2- 172.21.15.33:0/3750100234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0109150 0x7faca010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 --2- 172.21.15.33:0/3750100234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faca01087d0 0x7faca0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 --2- 172.21.15.33:0/3750100234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca01031a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 -- 172.21.15.33:0/3750100234 >> 172.21.15.33:0/3750100234 conn(0x7faca00fe830 msgr2=0x7faca0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 -- 172.21.15.33:0/3750100234 shutdown_connections 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.426+0000 7faca7ecb640 1 -- 172.21.15.33:0/3750100234 wait complete. 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 Processor -- start 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 -- start start 2024-09-10T21:12:24.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca00754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faca01087d0 0x7faca00759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0109150 0x7faca0075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7faca0112e10 con 0x7faca0102dc0 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7faca0112c90 con 0x7faca01087d0 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca7ecb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faca0112f90 con 0x7faca0109150 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca00754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca00754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42418/0 (socket says 172.21.15.33:42418) 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 -- 172.21.15.33:0/614185147 learned_addr learned my addr 172.21.15.33:0/614185147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca6468640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0109150 0x7faca0075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5466640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faca01087d0 0x7faca00759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 -- 172.21.15.33:0/614185147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0109150 msgr2=0x7faca0075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0109150 0x7faca0075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 -- 172.21.15.33:0/614185147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faca01087d0 msgr2=0x7faca00759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faca01087d0 0x7faca00759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca5c67640 1 -- 172.21.15.33:0/614185147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca01a76a0 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.427+0000 7faca6468640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0109150 0x7faca0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.428+0000 7faca5c67640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca00754a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fac9c005940 tx=0x7fac9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.428+0000 7fac96ffd640 1 -- 172.21.15.33:0/614185147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c03f030 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.428+0000 7fac96ffd640 1 -- 172.21.15.33:0/614185147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac9c003cb0 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.428+0000 7fac96ffd640 1 -- 172.21.15.33:0/614185147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9c035070 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.428+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca01a7930 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.428+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca01a7e10 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.429+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca01031a0 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.430+0000 7fac96ffd640 1 -- 172.21.15.33:0/614185147 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fac9c03bb50 con 0x7faca0102dc0 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.431+0000 7fac96ffd640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac8c0b52e0 0x7fac8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:24.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.431+0000 7faca5466640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac8c0b52e0 0x7fac8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:24.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.431+0000 7fac96ffd640 1 -- 172.21.15.33:0/614185147 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fac9c106a20 con 0x7faca0102dc0 2024-09-10T21:12:24.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.432+0000 7faca5466640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac8c0b52e0 0x7fac8c0b77a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac90005fd0 tx=0x7fac90005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:24.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.434+0000 7fac96ffd640 1 -- 172.21.15.33:0/614185147 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fac9c0cd8a0 con 0x7faca0102dc0 2024-09-10T21:12:24.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.659+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faca01a81e0 con 0x7faca0102dc0 2024-09-10T21:12:24.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.659+0000 7fac96ffd640 1 -- 172.21.15.33:0/614185147 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11973 (secure 0 0 0) 0x7fac9c0cd1a0 con 0x7faca0102dc0 2024-09-10T21:12:24.664 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:12:24.664 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"btime":"2024-09-10T21:11:31:688744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:11:30.989029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:12:24.664 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2024-09-10T21:12:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac8c0b52e0 msgr2=0x7fac8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac8c0b52e0 0x7fac8c0b77a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fac90005fd0 tx=0x7fac90005850 comp rx=0 tx=0).stop 2024-09-10T21:12:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 msgr2=0x7faca00754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca00754a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fac9c005940 tx=0x7fac9c0042f0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 shutdown_connections 2024-09-10T21:12:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac8c0b52e0 0x7fac8c0b77a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0109150 0x7faca0075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faca01087d0 0x7faca00759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 --2- 172.21.15.33:0/614185147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faca0102dc0 0x7faca00754a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 >> 172.21.15.33:0/614185147 conn(0x7faca00fe830 msgr2=0x7faca01000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 shutdown_connections 2024-09-10T21:12:24.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:24.662+0000 7faca7ecb640 1 -- 172.21.15.33:0/614185147 wait complete. 2024-09-10T21:12:24.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 571974 5.53 MB/sec execute 325 sec latency 1030.155 ms 2024-09-10T21:12:25.356 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:12:25.617 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:25 smithi174 ceph-mon[35635]: pgmap v599: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-10T21:12:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:25 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/614185147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:12:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:25 smithi033 ceph-mon[30860]: pgmap v599: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-10T21:12:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:25 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/614185147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:12:25.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 571974 5.52 MB/sec execute 326 sec latency 2030.225 ms 2024-09-10T21:12:25.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:25 smithi104 ceph-mon[35325]: pgmap v599: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-09-10T21:12:25.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:25 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/614185147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 -- 172.21.15.33:0/3312330790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa18810d140 msgr2=0x7fa18810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 --2- 172.21.15.33:0/3312330790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa18810d140 0x7fa18810f5a0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa180009ca0 tx=0x7fa18002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 -- 172.21.15.33:0/3312330790 shutdown_connections 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 --2- 172.21.15.33:0/3312330790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa18810d140 0x7fa18810f5a0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 --2- 172.21.15.33:0/3312330790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa188104600 0x7fa18810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 --2- 172.21.15.33:0/3312330790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa188103ce0 0x7fa1881040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 -- 172.21.15.33:0/3312330790 >> 172.21.15.33:0/3312330790 conn(0x7fa1880fd700 msgr2=0x7fa1880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 -- 172.21.15.33:0/3312330790 shutdown_connections 2024-09-10T21:12:26.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.077+0000 7fa18f24b640 1 -- 172.21.15.33:0/3312330790 wait complete. 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 Processor -- start 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 -- start start 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa188103ce0 0x7fa188199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa188104600 0x7fa188199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa18810d140 0x7fa18819ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa188112e00 con 0x7fa188103ce0 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa188112c80 con 0x7fa18810d140 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18f24b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa188112f80 con 0x7fa188104600 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18cfe7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa188103ce0 0x7fa188199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.078+0000 7fa18cfe7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa188103ce0 0x7fa188199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42434/0 (socket says 172.21.15.33:42434) 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18cfe7640 1 -- 172.21.15.33:0/608314050 learned_addr learned my addr 172.21.15.33:0/608314050 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:26.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa187fff640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa188104600 0x7fa188199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa18810d140 0x7fa18819ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18cfe7640 1 -- 172.21.15.33:0/608314050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa188104600 msgr2=0x7fa188199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa188104600 0x7fa188199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18cfe7640 1 -- 172.21.15.33:0/608314050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa18810d140 msgr2=0x7fa18819ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa18810d140 0x7fa18819ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18cfe7640 1 -- 172.21.15.33:0/608314050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18819f1e0 con 0x7fa188103ce0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa187fff640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa188104600 0x7fa188199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa188103ce0 0x7fa188199130 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c920 tx=0x7fa17c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa185ffb640 1 -- 172.21.15.33:0/608314050 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c007c70 con 0x7fa188103ce0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa185ffb640 1 -- 172.21.15.33:0/608314050 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa17c007e10 con 0x7fa188103ce0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa18819f4d0 con 0x7fa188103ce0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.079+0000 7fa185ffb640 1 -- 172.21.15.33:0/608314050 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c018820 con 0x7fa188103ce0 2024-09-10T21:12:26.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.080+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1881a5b90 con 0x7fa188103ce0 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 Processor -- start 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa185ffb640 1 -- 172.21.15.33:0/608314050 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa17c00f450 con 0x7fa188103ce0 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 -- start start 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce340f0 0x560ecce344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce34a10 0x560ecce40460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560eccdaaea0 con 0x560ecce34a10 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560eccd9a2d0 con 0x560ecce340f0 2024-09-10T21:12:26.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa191620740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560eccd9e0e0 con 0x560ecce409a0 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa18cfe7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce34a10 0x560ecce40460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa18cfe7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce34a10 0x560ecce40460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42444/0 (socket says 172.21.15.33:42444) 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.082+0000 7fa18cfe7640 1 -- 172.21.15.33:0/1286911737 learned_addr learned my addr 172.21.15.33:0/1286911737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce340f0 0x560ecce344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa187fff640 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18cfe7640 1 -- 172.21.15.33:0/1286911737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 msgr2=0x560ecce42d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce42d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18cfe7640 1 -- 172.21.15.33:0/1286911737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce340f0 msgr2=0x560ecce344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce340f0 0x560ecce344d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18cfe7640 1 -- 172.21.15.33:0/1286911737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eccaf8650 con 0x560ecce34a10 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce340f0 0x560ecce344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa185ffb640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1780b52e0 0x7fa1780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1780b52e0 0x7fa1780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce34a10 0x560ecce40460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0e9ef0 tx=0x7fa17c0ec440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67865e192effb0a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa185ffb640 1 -- 172.21.15.33:0/608314050 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa17c0e5e00 con 0x7fa188103ce0 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa172ffd640 1 -- 172.21.15.33:0/1286911737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0a8c10 con 0x560ecce34a10 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa172ffd640 1 -- 172.21.15.33:0/1286911737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa17c0ac0d0 con 0x560ecce34a10 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.083+0000 7fa172ffd640 1 -- 172.21.15.33:0/1286911737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0abd30 con 0x560ecce34a10 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 -- 172.21.15.33:0/1286911737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce34a10 msgr2=0x560ecce40460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1780b52e0 0x7fa1780b77a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa180009430 tx=0x7fa180009360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce34a10 0x560ecce40460 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0e9ef0 tx=0x7fa17c0ec440 comp rx=0 tx=0).stop 2024-09-10T21:12:26.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 -- 172.21.15.33:0/1286911737 shutdown_connections 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce42d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce34a10 0x560ecce40460 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 --2- 172.21.15.33:0/1286911737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce340f0 0x560ecce344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 -- 172.21.15.33:0/1286911737 >> 172.21.15.33:0/1286911737 conn(0x560ecce31f20 msgr2=0x560ecce330c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 -- 172.21.15.33:0/1286911737 shutdown_connections 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.084+0000 7fa191620740 1 -- 172.21.15.33:0/1286911737 wait complete. 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 -- start start 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 Processor -- start 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce340f0 0x560ecce40300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce3d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce3ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560eccbfb700 con 0x560ecce340f0 2024-09-10T21:12:26.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560eccd35160 con 0x560ecce34a10 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ecce435c0 con 0x560ecce409a0 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa191620740 10 client.0 _ll_register_callbacks cb 0x560eccd4e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa18cfe7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce340f0 0x560ecce40300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa18cfe7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce340f0 0x560ecce40300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42454/0 (socket says 172.21.15.33:42454) 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa18cfe7640 1 -- 172.21.15.33:0/201821272 learned_addr learned my addr 172.21.15.33:0/201821272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce3ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.085+0000 7fa187fff640 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce3d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa18cfe7640 1 -- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 msgr2=0x560ecce3ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce3ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa18cfe7640 1 -- 172.21.15.33:0/201821272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 msgr2=0x560ecce3d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce3d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa18cfe7640 1 -- 172.21.15.33:0/201821272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eccd62c90 con 0x560ecce340f0 2024-09-10T21:12:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce340f0 0x560ecce40300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0e9ec0 tx=0x7fa17c0e9470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6ce04a5a426593 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa167fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:26.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa167fff640 1 -- 172.21.15.33:0/201821272 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0abf10 con 0x560ecce340f0 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa167fff640 1 -- 172.21.15.33:0/201821272 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa17c01f300 con 0x560ecce340f0 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa17c0ac0d0 con 0x560ecce340f0 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa167fff640 1 -- 172.21.15.33:0/201821272 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa17c0ac390 con 0x560ecce340f0 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa191620740 10 client.16917 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x560ecce4eff0 con 0x560ecce340f0 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa167fff640 1 -- 172.21.15.33:0/201821272 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa17c0ac0d0 con 0x560ecce340f0 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa167fff640 1 -- 172.21.15.33:0/201821272 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fa17c0ec0c0 con 0x560ecce340f0 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa191620740 10 client.16917 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.086+0000 7fa191620740 10 client.16917 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.087+0000 7fa191620740 20 client.16917 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:26.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.087+0000 7fa191620740 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ecce50340 0x560ecce52730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.087+0000 7fa191620740 4 client.16917 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.087+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eccdacdf0 con 0x560ecce50340 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.087+0000 7fa187fff640 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ecce50340 0x560ecce52730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.087+0000 7fa187fff640 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ecce50340 0x560ecce52730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e71864404864e98 server_cookie=b9fbb2b50f371348 in_seq=0 out_seq=0 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.087+0000 7fa167fff640 10 client.16917 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa167fff640 1 -- 172.21.15.33:0/201821272 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560eccdacdf0 con 0x560ecce50340 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa167fff640 10 client.16917 handle_command_reply: tid=0 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 client.16917 shutdown 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce340f0 msgr2=0x560ecce40300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce340f0 0x560ecce40300 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa17c0e9ec0 tx=0x7fa17c0e9470 comp rx=0 tx=0).stop 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 shutdown_connections 2024-09-10T21:12:26.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560ecce50340 0x560ecce52730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce409a0 0x560ecce3ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce3d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa167fff640 0 client.16917 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 --2- 172.21.15.33:0/201821272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce340f0 0x560ecce40300 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 >> 172.21.15.33:0/201821272 conn(0x560ecce31f20 msgr2=0x560ecce37ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.088+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 shutdown_connections 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.089+0000 7fa191620740 1 -- 172.21.15.33:0/201821272 wait complete. 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.089+0000 7fa191620740 20 client.16917 trim_cache size 0 max 16384 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 Processor -- start 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 -- start start 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce41d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce3a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881ae2b0 con 0x560ecce409a0 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881afdf0 con 0x560ecce34a10 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa191620740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881ae980 con 0x560ecce340f0 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce3a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce3a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42458/0 (socket says 172.21.15.33:42458) 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 -- 172.21.15.33:0/1598073141 learned_addr learned my addr 172.21.15.33:0/1598073141 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce41d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa187fff640 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 -- 172.21.15.33:0/1598073141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 msgr2=0x560ecce41d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce41d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 -- 172.21.15.33:0/1598073141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 msgr2=0x560ecce422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.102+0000 7fa18d7e8640 1 -- 172.21.15.33:0/1598073141 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ecce2d7e0 con 0x560ecce409a0 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce3a960 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa180004af0 tx=0x7fa180040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c285896d0fefd887 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa1727fc640 1 -- 172.21.15.33:0/1598073141 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180003970 con 0x560ecce409a0 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa1727fc640 1 -- 172.21.15.33:0/1598073141 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa180036040 con 0x560ecce409a0 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa1727fc640 1 -- 172.21.15.33:0/1598073141 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18002f610 con 0x560ecce409a0 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 -- 172.21.15.33:0/1598073141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 msgr2=0x560ecce3a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce3a960 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa180004af0 tx=0x7fa180040000 comp rx=0 tx=0).stop 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 -- 172.21.15.33:0/1598073141 shutdown_connections 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce3a960 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 --2- 172.21.15.33:0/1598073141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce41d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 -- 172.21.15.33:0/1598073141 >> 172.21.15.33:0/1598073141 conn(0x560ecce31f20 msgr2=0x560ecce3bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 -- 172.21.15.33:0/1598073141 shutdown_connections 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.103+0000 7fa191620740 1 -- 172.21.15.33:0/1598073141 wait complete. 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 -- start start 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 Processor -- start 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce52d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce53260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881af720 con 0x560ecce409a0 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881af050 con 0x560ecce34a10 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1881ac270 con 0x560ecce340f0 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa191620740 10 client.0 _ll_register_callbacks cb 0x7fa17c0e9dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:26.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa18d7e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce53260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa18d7e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce53260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42474/0 (socket says 172.21.15.33:42474) 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa18d7e8640 1 -- 172.21.15.33:0/135080658 learned_addr learned my addr 172.21.15.33:0/135080658 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.104+0000 7fa187fff640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce52d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa18d7e8640 1 -- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 msgr2=0x560ecce527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa18d7e8640 1 -- 172.21.15.33:0/135080658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 msgr2=0x560ecce52d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce52d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa18d7e8640 1 -- 172.21.15.33:0/135080658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ecce329a0 con 0x560ecce409a0 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa187fff640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce52d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa18d7e8640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce53260 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa18002aeb0 tx=0x7fa18002faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755b2b8993a3b2de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:26.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa167fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa167fff640 1 -- 172.21.15.33:0/135080658 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa180060060 con 0x560ecce409a0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa167fff640 1 -- 172.21.15.33:0/135080658 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa18004e050 con 0x560ecce409a0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa180036040 con 0x560ecce409a0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa167fff640 1 -- 172.21.15.33:0/135080658 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa18002fb60 con 0x560ecce409a0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa191620740 10 client.16923 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.105+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x560eccebe0e0 con 0x560ecce409a0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa167fff640 1 -- 172.21.15.33:0/135080658 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa180036040 con 0x560ecce409a0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa167fff640 1 -- 172.21.15.33:0/135080658 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fa180044040 con 0x560ecce409a0 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa191620740 10 client.16923 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa191620740 10 client.16923 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa191620740 20 client.16923 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa191620740 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560eccebe750 0x560eccec0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:26.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa191620740 4 client.16923 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560eccda88a0 con 0x560eccebe750 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.106+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560eccebe750 0x560eccec0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa18cfe7640 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560eccebe750 0x560eccec0b40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a7eda02923a8004b server_cookie=f13571d492b66e7b in_seq=0 out_seq=0 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa167fff640 10 client.16923 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa167fff640 1 -- 172.21.15.33:0/135080658 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560eccda88a0 con 0x560eccebe750 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa167fff640 10 client.16923 handle_command_reply: tid=0 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 client.16923 shutdown 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 msgr2=0x560ecce53260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce53260 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa18002aeb0 tx=0x7fa18002faf0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 shutdown_connections 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x560eccebe750 0x560eccec0b40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560ecce409a0 0x560ecce53260 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560ecce34a10 0x560ecce52d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa167fff640 0 client.16923 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:12:26.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 --2- 172.21.15.33:0/135080658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ecce340f0 0x560ecce527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 >> 172.21.15.33:0/135080658 conn(0x560ecce31f20 msgr2=0x560ecce3f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 shutdown_connections 2024-09-10T21:12:26.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 1 -- 172.21.15.33:0/135080658 wait complete. 2024-09-10T21:12:26.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.107+0000 7fa191620740 20 client.16923 trim_cache size 0 max 16384 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1780b52e0 msgr2=0x7fa1780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1780b52e0 0x7fa1780b77a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa180009430 tx=0x7fa180009360 comp rx=0 tx=0).stop 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa188103ce0 msgr2=0x7fa188199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa188103ce0 0x7fa188199130 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00c920 tx=0x7fa17c00cde0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 shutdown_connections 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1780b52e0 0x7fa1780b77a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa18810d140 0x7fa18819ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa188104600 0x7fa188199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 --2- 172.21.15.33:0/608314050 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa188103ce0 0x7fa188199130 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:26.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 >> 172.21.15.33:0/608314050 conn(0x7fa1880fd700 msgr2=0x7fa188104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:26.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 shutdown_connections 2024-09-10T21:12:26.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:26.108+0000 7fa18f24b640 1 -- 172.21.15.33:0/608314050 wait complete. 2024-09-10T21:12:26.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 571974 5.50 MB/sec execute 327 sec latency 3030.261 ms 2024-09-10T21:12:26.992 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:12:27.256 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:27 smithi174 ceph-mon[35635]: pgmap v600: 97 pgs: 97 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-09-10T21:12:27.650 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:27 smithi033 ceph-mon[30860]: pgmap v600: 97 pgs: 97 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-09-10T21:12:27.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572044 5.48 MB/sec execute 328 sec latency 3391.305 ms 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 -- 172.21.15.33:0/533310110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 msgr2=0x7fbe0810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 --2- 172.21.15.33:0/533310110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0810f950 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009900 tx=0x7fbe0002a410 comp rx=0 tx=0).stop 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 -- 172.21.15.33:0/533310110 shutdown_connections 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 --2- 172.21.15.33:0/533310110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0810f950 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 --2- 172.21.15.33:0/533310110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe081087d0 0x7fbe08108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 --2- 172.21.15.33:0/533310110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe08102dc0 0x7fbe081031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 -- 172.21.15.33:0/533310110 >> 172.21.15.33:0/533310110 conn(0x7fbe080fe830 msgr2=0x7fbe08100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 -- 172.21.15.33:0/533310110 shutdown_connections 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.732+0000 7fbe103c9640 1 -- 172.21.15.33:0/533310110 wait complete. 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 Processor -- start 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 -- start start 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe08102dc0 0x7fbe08199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe081087d0 0x7fbe08199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08112dd0 con 0x7fbe08109150 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08112c50 con 0x7fbe081087d0 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe103c9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe08112f50 con 0x7fbe08102dc0 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42492/0 (socket says 172.21.15.33:42492) 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 -- 172.21.15.33:0/4215052307 learned_addr learned my addr 172.21.15.33:0/4215052307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:27.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e165640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe08102dc0 0x7fbe08199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0d964640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe081087d0 0x7fbe08199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 -- 172.21.15.33:0/4215052307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe08102dc0 msgr2=0x7fbe08199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe08102dc0 0x7fbe08199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 -- 172.21.15.33:0/4215052307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe081087d0 msgr2=0x7fbe08199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe081087d0 0x7fbe08199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0e966640 1 -- 172.21.15.33:0/4215052307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe0819f2f0 con 0x7fbe08109150 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.733+0000 7fbe0d964640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe081087d0 0x7fbe08199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.734+0000 7fbe0e966640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0819eb20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbe00004770 tx=0x7fbe00004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.734+0000 7fbdff7fe640 1 -- 172.21.15.33:0/4215052307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0003f030 con 0x7fbe08109150 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.734+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe0819f580 con 0x7fbe08109150 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.734+0000 7fbdff7fe640 1 -- 172.21.15.33:0/4215052307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe00003aa0 con 0x7fbe08109150 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.734+0000 7fbdff7fe640 1 -- 172.21.15.33:0/4215052307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe00035070 con 0x7fbe08109150 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.734+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe081a5ca0 con 0x7fbe08109150 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.736+0000 7fbdff7fe640 1 -- 172.21.15.33:0/4215052307 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbe000035f0 con 0x7fbe08109150 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.736+0000 7fbe1279e740 1 Processor -- start 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.736+0000 7fbe1279e740 1 -- start start 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd874d0 0x55e39bd878b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd87df0 0x55e39bd93840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd93d80 0x55e39bd96170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe1279e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e39bcfdea0 con 0x55e39bd93d80 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe1279e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e39bced2d0 con 0x55e39bd87df0 2024-09-10T21:12:27.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe1279e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e39bcf10e0 con 0x55e39bd874d0 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd93d80 0x55e39bd96170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd93d80 0x55e39bd96170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42498/0 (socket says 172.21.15.33:42498) 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 -- 172.21.15.33:0/525152874 learned_addr learned my addr 172.21.15.33:0/525152874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0e165640 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd87df0 0x55e39bd93840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd874d0 0x55e39bd878b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbdff7fe640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdf40b52e0 0x7fbdf40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 -- 172.21.15.33:0/525152874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd874d0 msgr2=0x55e39bd878b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd874d0 0x55e39bd878b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 -- 172.21.15.33:0/525152874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd87df0 msgr2=0x55e39bd93840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd87df0 0x55e39bd93840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0d964640 1 -- 172.21.15.33:0/525152874 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39ba4b650 con 0x55e39bd93d80 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.737+0000 7fbe0e966640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdf40b52e0 0x7fbdf40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.738+0000 7fbe0d964640 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd93d80 0x55e39bd96170 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8013610 tx=0x7fbdf800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a948126cd10837 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.738+0000 7fbdff7fe640 1 -- 172.21.15.33:0/4215052307 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbe00106620 con 0x7fbe08109150 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.738+0000 7fbdfcde6640 1 -- 172.21.15.33:0/525152874 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8019070 con 0x55e39bd93d80 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.738+0000 7fbdfcde6640 1 -- 172.21.15.33:0/525152874 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf8004230 con 0x55e39bd93d80 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.738+0000 7fbdfcde6640 1 -- 172.21.15.33:0/525152874 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8011880 con 0x55e39bd93d80 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- 172.21.15.33:0/525152874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd93d80 msgr2=0x55e39bd96170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe0e966640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdf40b52e0 0x7fbdf40b77a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe000c93b0 tx=0x7fbe00091550 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd93d80 0x55e39bd96170 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8013610 tx=0x7fbdf800c750 comp rx=0 tx=0).stop 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- 172.21.15.33:0/525152874 shutdown_connections 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd93d80 0x55e39bd96170 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd87df0 0x55e39bd93840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 --2- 172.21.15.33:0/525152874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd874d0 0x55e39bd878b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- 172.21.15.33:0/525152874 >> 172.21.15.33:0/525152874 conn(0x55e39bd85300 msgr2=0x55e39bd864a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- 172.21.15.33:0/525152874 shutdown_connections 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- 172.21.15.33:0/525152874 wait complete. 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- start start 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 Processor -- start 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd874d0 0x55e39bd936e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd87df0 0x55e39bd90b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd93d80 0x55e39bd911b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e39bb4e700 con 0x55e39bd87df0 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e39bc88160 con 0x55e39bd874d0 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.739+0000 7fbe1279e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e39bd96a00 con 0x55e39bd93d80 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe1279e740 10 client.0 _ll_register_callbacks cb 0x55e39bca19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd87df0 0x55e39bd90b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd87df0 0x55e39bd90b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42512/0 (socket says 172.21.15.33:42512) 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 -- 172.21.15.33:0/3376635902 learned_addr learned my addr 172.21.15.33:0/3376635902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0d964640 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd874d0 0x55e39bd936e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0d964640 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd93d80 0x55e39bd911b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 -- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd93d80 msgr2=0x55e39bd911b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd93d80 0x55e39bd911b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 -- 172.21.15.33:0/3376635902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd874d0 msgr2=0x55e39bd936e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd874d0 0x55e39bd936e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 -- 172.21.15.33:0/3376635902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39bcb5c90 con 0x55e39bd87df0 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe0e165640 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd87df0 0x55e39bd90b30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbe04004070 tx=0x7fbe04004530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af89ab03b050fac2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbdf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/3376635902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe04013bc0 con 0x55e39bd87df0 2024-09-10T21:12:27.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/3376635902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbe04013d60 con 0x55e39bd87df0 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.740+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdf8004230 con 0x55e39bd87df0 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/3376635902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbe0401c440 con 0x55e39bd87df0 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 10 client.16935 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55e39bda2410 con 0x55e39bd87df0 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/3376635902 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf8004230 con 0x55e39bd87df0 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/3376635902 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fbe0401a070 con 0x55e39bd87df0 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 10 client.16935 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 10 client.16935 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 20 client.16935 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39bda3980 0x55e39bda5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 4 client.16935 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e39bcffdf0 con 0x55e39bda3980 2024-09-10T21:12:27.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.741+0000 7fbe0d964640 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39bda3980 0x55e39bda5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.742+0000 7fbe0d964640 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39bda3980 0x55e39bda5e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf2d878f8f8ab0eb server_cookie=1b7921906d31031a in_seq=0 out_seq=0 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.742+0000 7fbdf1ffb640 10 client.16935 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.742+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/3376635902 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e39bcffdf0 con 0x55e39bda3980 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.742+0000 7fbdf1ffb640 10 client.16935 handle_command_reply: tid=0 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.742+0000 7fbe1279e740 1 client.16935 shutdown 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd87df0 msgr2=0x55e39bd90b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd87df0 0x55e39bd90b30 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fbe04004070 tx=0x7fbe04004530 comp rx=0 tx=0).stop 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 shutdown_connections 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39bda3980 0x55e39bda5e00 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd93d80 0x55e39bd911b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd87df0 0x55e39bd90b30 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbdf1ffb640 0 client.16935 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:12:27.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 --2- 172.21.15.33:0/3376635902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd874d0 0x55e39bd936e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 >> 172.21.15.33:0/3376635902 conn(0x55e39bd85300 msgr2=0x55e39bd8aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 shutdown_connections 2024-09-10T21:12:27.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 1 -- 172.21.15.33:0/3376635902 wait complete. 2024-09-10T21:12:27.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.743+0000 7fbe1279e740 20 client.16935 trim_cache size 0 max 16384 2024-09-10T21:12:27.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 Processor -- start 2024-09-10T21:12:27.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 -- start start 2024-09-10T21:12:27.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd8f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 0x55e39bd8f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081ae3b0 con 0x55e39bd874d0 2024-09-10T21:12:27.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081afef0 con 0x55e39bd93d80 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe1279e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081aea80 con 0x55e39bd87df0 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe0d964640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe0d964640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42520/0 (socket says 172.21.15.33:42520) 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe0d964640 1 -- 172.21.15.33:0/712850356 learned_addr learned my addr 172.21.15.33:0/712850356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.757+0000 7fbe0e165640 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd8f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbe0d964640 1 -- 172.21.15.33:0/712850356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 msgr2=0x55e39bd8f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbe0d964640 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd8f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbe0d964640 1 -- 172.21.15.33:0/712850356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 msgr2=0x55e39bd8f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbe0d964640 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 0x55e39bd8f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbe0d964640 1 -- 172.21.15.33:0/712850356 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39bd80bb0 con 0x55e39bd874d0 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbe0d964640 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9fc30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8003d90 tx=0x7fbdf8013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505164505ddd89c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:27.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbdf3fff640 1 -- 172.21.15.33:0/712850356 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8019040 con 0x55e39bd874d0 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbdf3fff640 1 -- 172.21.15.33:0/712850356 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf80054d0 con 0x55e39bd874d0 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.758+0000 7fbdf3fff640 1 -- 172.21.15.33:0/712850356 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80038e0 con 0x55e39bd874d0 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 -- 172.21.15.33:0/712850356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 msgr2=0x55e39bd9fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9fc30 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8003d90 tx=0x7fbdf8013460 comp rx=0 tx=0).stop 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 -- 172.21.15.33:0/712850356 shutdown_connections 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 0x55e39bd8f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd8f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 --2- 172.21.15.33:0/712850356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9fc30 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 -- 172.21.15.33:0/712850356 >> 172.21.15.33:0/712850356 conn(0x55e39bd85300 msgr2=0x55e39bd8bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 -- 172.21.15.33:0/712850356 shutdown_connections 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 -- 172.21.15.33:0/712850356 wait complete. 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 -- start start 2024-09-10T21:12:27.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.759+0000 7fbe1279e740 1 Processor -- start 2024-09-10T21:12:27.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd9c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 0x55e39bd96c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081af820 con 0x55e39bd874d0 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081af150 con 0x55e39bd93d80 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe081ac370 con 0x55e39bd87df0 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 10 client.0 _ll_register_callbacks cb 0x7fbe04009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42528/0 (socket says 172.21.15.33:42528) 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 -- 172.21.15.33:0/738196709 learned_addr learned my addr 172.21.15.33:0/738196709 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0e165640 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd9c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 -- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 msgr2=0x55e39bd9c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd9c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 -- 172.21.15.33:0/738196709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 msgr2=0x55e39bd96c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 0x55e39bd96c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 -- 172.21.15.33:0/738196709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39bd9c970 con 0x55e39bd874d0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe0d964640 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9bca0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8015010 tx=0x7fbdf8003dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bc7ea3457a45 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbdf1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/738196709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8019040 con 0x55e39bd874d0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/738196709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbdf80035f0 con 0x55e39bd874d0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.760+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdf80054d0 con 0x55e39bd874d0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/738196709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbdf80037f0 con 0x55e39bd874d0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 10 client.16941 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55e39be10e60 con 0x55e39bd874d0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/738196709 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdf80054d0 con 0x55e39bd874d0 2024-09-10T21:12:27.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/738196709 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fbdf802f070 con 0x55e39bd874d0 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 10 client.16941 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 10 client.16941 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 20 client.16941 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39be11320 0x55e39be13710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 4 client.16941 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e39bcfb8a0 con 0x55e39be11320 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.761+0000 7fbe0e165640 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39be11320 0x55e39be13710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe0e165640 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39be11320 0x55e39be13710 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3b7b0dd929a90daf server_cookie=dfa4255624918465 in_seq=0 out_seq=0 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbdf1ffb640 10 client.16941 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbdf1ffb640 1 -- 172.21.15.33:0/738196709 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e39bcfb8a0 con 0x55e39be11320 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbdf1ffb640 10 client.16941 handle_command_reply: tid=0 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 client.16941 shutdown 2024-09-10T21:12:27.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 msgr2=0x55e39bd9bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9bca0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8015010 tx=0x7fbdf8003dc0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 shutdown_connections 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e39bd93d80 0x55e39bd96c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e39be11320 0x55e39be13710 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbdf1ffb640 0 client.16941 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e39bd87df0 0x55e39bd9c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 --2- 172.21.15.33:0/738196709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e39bd874d0 0x55e39bd9bca0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 >> 172.21.15.33:0/738196709 conn(0x55e39bd85300 msgr2=0x55e39bd92160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 shutdown_connections 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 1 -- 172.21.15.33:0/738196709 wait complete. 2024-09-10T21:12:27.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.762+0000 7fbe1279e740 20 client.16941 trim_cache size 0 max 16384 2024-09-10T21:12:27.773 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:12:27.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdf40b52e0 msgr2=0x7fbdf40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdf40b52e0 0x7fbdf40b77a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbe000c93b0 tx=0x7fbe00091550 comp rx=0 tx=0).stop 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 msgr2=0x7fbe0819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0819eb20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbe00004770 tx=0x7fbe00004270 comp rx=0 tx=0).stop 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 shutdown_connections 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdf40b52e0 0x7fbdf40b77a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe08109150 0x7fbe0819eb20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe081087d0 0x7fbe08199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 --2- 172.21.15.33:0/4215052307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe08102dc0 0x7fbe08199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 >> 172.21.15.33:0/4215052307 conn(0x7fbe080fe830 msgr2=0x7fbe080ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 shutdown_connections 2024-09-10T21:12:27.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:27.763+0000 7fbe103c9640 1 -- 172.21.15.33:0/4215052307 wait complete. 2024-09-10T21:12:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:27 smithi104 ceph-mon[35325]: pgmap v600: 97 pgs: 97 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-09-10T21:12:28.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:12:28.739 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:28.741 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572044 5.47 MB/sec execute 329 sec latency 1570.743 ms 2024-09-10T21:12:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/2125835986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88109150 msgr2=0x7f4d8810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/2125835986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88109150 0x7f4d8810f950 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009ca0 tx=0x7f4d8002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/2125835986 shutdown_connections 2024-09-10T21:12:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/2125835986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88109150 0x7f4d8810f950 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/2125835986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d881087d0 0x7f4d88108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/2125835986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d88102dc0 0x7f4d881031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/2125835986 >> 172.21.15.33:0/2125835986 conn(0x7f4d880fe830 msgr2=0x7f4d88100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/2125835986 shutdown_connections 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.208+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/2125835986 wait complete. 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.209+0000 7f4d8e4c3640 1 Processor -- start 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.209+0000 7f4d8e4c3640 1 -- start start 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8e4c3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88102dc0 0x7f4d88199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8e4c3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d881087d0 0x7f4d881997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8e4c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88109150 0x7f4d8819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8e4c3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88112e10 con 0x7f4d88102dc0 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8e4c3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88112c90 con 0x7f4d881087d0 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8e4c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88112f90 con 0x7f4d88109150 2024-09-10T21:12:29.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88102dc0 0x7f4d88199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88102dc0 0x7f4d88199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42544/0 (socket says 172.21.15.33:42544) 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 -- 172.21.15.33:0/3794606595 learned_addr learned my addr 172.21.15.33:0/3794606595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d877fe640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d881087d0 0x7f4d881997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88109150 0x7f4d8819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 -- 172.21.15.33:0/3794606595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88109150 msgr2=0x7f4d8819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88109150 0x7f4d8819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 -- 172.21.15.33:0/3794606595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d881087d0 msgr2=0x7f4d881997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d881087d0 0x7f4d881997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 -- 172.21.15.33:0/3794606595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8819f310 con 0x7f4d88102dc0 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88109150 0x7f4d8819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d87fff640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88102dc0 0x7f4d88199260 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c920 tx=0x7f4d7800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d857fa640 1 -- 172.21.15.33:0/3794606595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78007c70 con 0x7f4d88102dc0 2024-09-10T21:12:29.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.210+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d8819f600 con 0x7f4d88102dc0 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.211+0000 7f4d857fa640 1 -- 172.21.15.33:0/3794606595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d78007e10 con 0x7f4d88102dc0 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.211+0000 7f4d857fa640 1 -- 172.21.15.33:0/3794606595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d78018820 con 0x7f4d88102dc0 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.211+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881a5e90 con 0x7f4d88102dc0 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 Processor -- start 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 -- start start 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d857fa640 1 -- 172.21.15.33:0/3794606595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4d7800f450 con 0x7f4d88102dc0 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4df94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e05460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5560c4d6fea0 con 0x5560c4df90f0 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560c4d5f2d0 con 0x5560c4df9a10 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d90898740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560c4d630e0 con 0x5560c4e059a0 2024-09-10T21:12:29.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.213+0000 7f4d8ca60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4df94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d87fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e05460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4df94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42560/0 (socket says 172.21.15.33:42560) 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 -- 172.21.15.33:0/2460446154 learned_addr learned my addr 172.21.15.33:0/2460446154 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d877fe640 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 -- 172.21.15.33:0/2460446154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 msgr2=0x5560c4e07d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e07d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 -- 172.21.15.33:0/2460446154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 msgr2=0x5560c4e05460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e05460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 -- 172.21.15.33:0/2460446154 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c4abd650 con 0x5560c4df90f0 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4df94d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4d800098e0 tx=0x7f4d80009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8408150c4d5a532 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d6e7fc640 1 -- 172.21.15.33:0/2460446154 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80031df0 con 0x5560c4df90f0 2024-09-10T21:12:29.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d6e7fc640 1 -- 172.21.15.33:0/2460446154 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80036070 con 0x5560c4df90f0 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d857fa640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d700b52e0 0x7f4d700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.214+0000 7f4d6e7fc640 1 -- 172.21.15.33:0/2460446154 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80004330 con 0x5560c4df90f0 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d87fff640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d700b52e0 0x7f4d700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d857fa640 1 -- 172.21.15.33:0/3794606595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4d780e5ce0 con 0x7f4d88102dc0 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d87fff640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d700b52e0 0x7f4d700b77a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007590 tx=0x7f4d780e9760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 -- 172.21.15.33:0/2460446154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 msgr2=0x5560c4df94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4df94d0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4d800098e0 tx=0x7f4d80009910 comp rx=0 tx=0).stop 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 -- 172.21.15.33:0/2460446154 shutdown_connections 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e07d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e05460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 --2- 172.21.15.33:0/2460446154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4df94d0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 -- 172.21.15.33:0/2460446154 >> 172.21.15.33:0/2460446154 conn(0x5560c4df6f20 msgr2=0x5560c4df80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 -- 172.21.15.33:0/2460446154 shutdown_connections 2024-09-10T21:12:29.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.215+0000 7f4d90898740 1 -- 172.21.15.33:0/2460446154 wait complete. 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 -- start start 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 Processor -- start 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4df90f0 0x5560c4e05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e026b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4e059a0 0x5560c4e02c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5560c4bc0700 con 0x5560c4e059a0 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560c4cfa160 con 0x5560c4df9a10 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560c4e0c9d0 con 0x5560c4df90f0 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d90898740 10 client.0 _ll_register_callbacks cb 0x5560c4d139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.216+0000 7f4d8ca60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4e059a0 0x5560c4e02c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d877fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e026b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4e059a0 0x5560c4e02c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42576/0 (socket says 172.21.15.33:42576) 2024-09-10T21:12:29.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3061215828 learned_addr learned my addr 172.21.15.33:0/3061215828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4df90f0 0x5560c4e05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3061215828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4df90f0 msgr2=0x5560c4e05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4df90f0 0x5560c4e05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3061215828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 msgr2=0x5560c4e026b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e026b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3061215828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c4d27c90 con 0x5560c4e059a0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4e059a0 0x5560c4e02c90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4d80031fd0 tx=0x7f4d80003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342130698d5eb3c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d677fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d677fe640 1 -- 172.21.15.33:0/3061215828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80049040 con 0x5560c4e059a0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d677fe640 1 -- 172.21.15.33:0/3061215828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4d80003b70 con 0x5560c4e059a0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.217+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d80036070 con 0x5560c4e059a0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d677fe640 1 -- 172.21.15.33:0/3061215828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4d80004b40 con 0x5560c4e059a0 2024-09-10T21:12:29.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 10 client.16953 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5560c4e18300 con 0x5560c4e059a0 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d677fe640 1 -- 172.21.15.33:0/3061215828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d80036070 con 0x5560c4e059a0 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d677fe640 1 -- 172.21.15.33:0/3061215828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4d8004f070 con 0x5560c4e059a0 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 10 client.16953 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 10 client.16953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 20 client.16953 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e19650 0x5560c4e1ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 4 client.16953 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.218+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c4d71df0 con 0x5560c4e19650 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.219+0000 7f4d877fe640 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e19650 0x5560c4e1ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.219+0000 7f4d877fe640 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e19650 0x5560c4e1ba40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bba62cca46f75fb5 server_cookie=f0cfc8cdefd53eb2 in_seq=0 out_seq=0 2024-09-10T21:12:29.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.219+0000 7f4d677fe640 10 client.16953 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.220+0000 7f4d677fe640 1 -- 172.21.15.33:0/3061215828 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560c4d71df0 con 0x5560c4e19650 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.220+0000 7f4d677fe640 10 client.16953 handle_command_reply: tid=0 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.220+0000 7f4d90898740 1 client.16953 shutdown 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4e059a0 msgr2=0x5560c4e02c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4e059a0 0x5560c4e02c90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4d80031fd0 tx=0x7f4d80003f60 comp rx=0 tx=0).stop 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 shutdown_connections 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e19650 0x5560c4e1ba40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4e059a0 0x5560c4e02c90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 --2- 172.21.15.33:0/3061215828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4df90f0 0x5560c4e05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 >> 172.21.15.33:0/3061215828 conn(0x5560c4df6f20 msgr2=0x5560c4dfca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d677fe640 0 client.16953 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 shutdown_connections 2024-09-10T21:12:29.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 1 -- 172.21.15.33:0/3061215828 wait complete. 2024-09-10T21:12:29.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.221+0000 7f4d90898740 20 client.16953 trim_cache size 0 max 16384 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 Processor -- start 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 -- start start 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df90f0 0x5560c4e15c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df9a10 0x5560c4e06d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e072d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881ac1b0 con 0x5560c4df9a10 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881adcf0 con 0x5560c4df90f0 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d90898740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881ac880 con 0x5560c4e059a0 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df9a10 0x5560c4e06d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d8ca60640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df90f0 0x5560c4e15c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df9a10 0x5560c4e06d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42588/0 (socket says 172.21.15.33:42588) 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 -- 172.21.15.33:0/957779656 learned_addr learned my addr 172.21.15.33:0/957779656 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e072d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 -- 172.21.15.33:0/957779656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 msgr2=0x5560c4e072d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e072d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 -- 172.21.15.33:0/957779656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df90f0 msgr2=0x5560c4e15c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df90f0 0x5560c4e15c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.235+0000 7f4d877fe640 1 -- 172.21.15.33:0/957779656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c4e055d0 con 0x5560c4df9a10 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.236+0000 7f4d877fe640 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df9a10 0x5560c4e06d90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c005f50 tx=0x7f4d7c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e840a262388fe5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.236+0000 7f4d6dffb640 1 -- 172.21.15.33:0/957779656 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c003d50 con 0x5560c4df9a10 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.236+0000 7f4d6dffb640 1 -- 172.21.15.33:0/957779656 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c0054d0 con 0x5560c4df9a10 2024-09-10T21:12:29.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.236+0000 7f4d6dffb640 1 -- 172.21.15.33:0/957779656 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0246d0 con 0x5560c4df9a10 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- 172.21.15.33:0/957779656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df9a10 msgr2=0x5560c4e06d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df9a10 0x5560c4e06d90 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c005f50 tx=0x7f4d7c00b390 comp rx=0 tx=0).stop 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- 172.21.15.33:0/957779656 shutdown_connections 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df9a10 0x5560c4e06d90 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 --2- 172.21.15.33:0/957779656 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df90f0 0x5560c4e15c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- 172.21.15.33:0/957779656 >> 172.21.15.33:0/957779656 conn(0x5560c4df6f20 msgr2=0x5560c4e04500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- 172.21.15.33:0/957779656 shutdown_connections 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- 172.21.15.33:0/957779656 wait complete. 2024-09-10T21:12:29.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- start start 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 Processor -- start 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4e11fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e12500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e0ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881ad620 con 0x5560c4df90f0 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881acf50 con 0x5560c4df9a10 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.237+0000 7f4d90898740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881aa170 con 0x5560c4e059a0 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d90898740 10 client.0 _ll_register_callbacks cb 0x7f4d780e9630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4e11fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4e11fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42592/0 (socket says 172.21.15.33:42592) 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3959810673 learned_addr learned my addr 172.21.15.33:0/3959810673 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d877fe640 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e12500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3959810673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 msgr2=0x5560c4e0ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e0ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 msgr2=0x5560c4e12500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e12500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 -- 172.21.15.33:0/3959810673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c4e0d500 con 0x5560c4df90f0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d8ca60640 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4e11fa0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4d80003fd0 tx=0x7f4d8002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c141023e211820c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d677fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d677fe640 1 -- 172.21.15.33:0/3959810673 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d80061020 con 0x5560c4df90f0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d677fe640 1 -- 172.21.15.33:0/3959810673 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4d8002f660 con 0x5560c4df90f0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.238+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c0054d0 con 0x5560c4df90f0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d677fe640 1 -- 172.21.15.33:0/3959810673 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4d80004160 con 0x5560c4df90f0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 10 client.16959 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5560c4e86d80 con 0x5560c4df90f0 2024-09-10T21:12:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d677fe640 1 -- 172.21.15.33:0/3959810673 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7c0054d0 con 0x5560c4df90f0 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d677fe640 1 -- 172.21.15.33:0/3959810673 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4d8004f070 con 0x5560c4df90f0 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 10 client.16959 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 10 client.16959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 20 client.16959 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e87420 0x5560c4e89840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 4 client.16959 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5560c4d6d8a0 con 0x5560c4e87420 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.239+0000 7f4d877fe640 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e87420 0x5560c4e89840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d877fe640 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e87420 0x5560c4e89840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f8a2fdbb46f81be0 server_cookie=70da975d7aa65588 in_seq=0 out_seq=0 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d677fe640 10 client.16959 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d677fe640 1 -- 172.21.15.33:0/3959810673 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5560c4d6d8a0 con 0x5560c4e87420 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d677fe640 10 client.16959 handle_command_reply: tid=0 2024-09-10T21:12:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 client.16959 shutdown 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 msgr2=0x5560c4e11fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4e11fa0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f4d80003fd0 tx=0x7f4d8002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 shutdown_connections 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560c4e059a0 0x5560c4e0ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560c4df9a10 0x5560c4e12500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5560c4e87420 0x5560c4e89840 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 --2- 172.21.15.33:0/3959810673 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560c4df90f0 0x5560c4e11fa0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 >> 172.21.15.33:0/3959810673 conn(0x5560c4df6f20 msgr2=0x5560c4df5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d677fe640 0 client.16959 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 shutdown_connections 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.240+0000 7f4d90898740 1 -- 172.21.15.33:0/3959810673 wait complete. 2024-09-10T21:12:29.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d90898740 20 client.16959 trim_cache size 0 max 16384 2024-09-10T21:12:29.250 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:12:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d700b52e0 msgr2=0x7f4d700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d700b52e0 0x7f4d700b77a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007590 tx=0x7f4d780e9760 comp rx=0 tx=0).stop 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88102dc0 msgr2=0x7f4d88199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88102dc0 0x7f4d88199260 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c920 tx=0x7f4d7800cde0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 shutdown_connections 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4d700b52e0 0x7f4d700b77a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88109150 0x7f4d8819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d881087d0 0x7f4d881997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 --2- 172.21.15.33:0/3794606595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d88102dc0 0x7f4d88199260 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 >> 172.21.15.33:0/3794606595 conn(0x7f4d880fe830 msgr2=0x7f4d88100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 shutdown_connections 2024-09-10T21:12:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:29.241+0000 7f4d8e4c3640 1 -- 172.21.15.33:0/3794606595 wait complete. 2024-09-10T21:12:29.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:29 smithi033 ceph-mon[30860]: pgmap v601: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 975 KiB/s wr, 137 op/s 2024-09-10T21:12:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:29 smithi174 ceph-mon[35635]: pgmap v601: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 975 KiB/s wr, 137 op/s 2024-09-10T21:12:29.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572044 5.45 MB/sec execute 330 sec latency 2570.790 ms 2024-09-10T21:12:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:29 smithi104 ceph-mon[35325]: pgmap v601: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 975 KiB/s wr, 137 op/s 2024-09-10T21:12:29.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:12:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 -- 172.21.15.33:0/2983191137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d8109150 msgr2=0x7f84d810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 --2- 172.21.15.33:0/2983191137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d8109150 0x7f84d810f950 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009ca0 tx=0x7f84d002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 -- 172.21.15.33:0/2983191137 shutdown_connections 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 --2- 172.21.15.33:0/2983191137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d8109150 0x7f84d810f950 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 --2- 172.21.15.33:0/2983191137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d81087d0 0x7f84d8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 --2- 172.21.15.33:0/2983191137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8102dc0 0x7f84d81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 -- 172.21.15.33:0/2983191137 >> 172.21.15.33:0/2983191137 conn(0x7f84d80fe830 msgr2=0x7f84d8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 -- 172.21.15.33:0/2983191137 shutdown_connections 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 -- 172.21.15.33:0/2983191137 wait complete. 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 Processor -- start 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.599+0000 7f84dee5a640 1 -- start start 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dee5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d8102dc0 0x7f84d8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dee5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d81087d0 0x7f84d81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dee5a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8109150 0x7f84d819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dee5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112dd0 con 0x7f84d81087d0 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dee5a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112c50 con 0x7f84d8109150 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dee5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112f50 con 0x7f84d8102dc0 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d81087d0 0x7f84d81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d81087d0 0x7f84d81997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53626/0 (socket says 172.21.15.33:53626) 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 -- 172.21.15.33:0/3717384107 learned_addr learned my addr 172.21.15.33:0/3717384107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8109150 0x7f84d819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d8102dc0 0x7f84d8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 -- 172.21.15.33:0/3717384107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d8102dc0 msgr2=0x7f84d8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d8102dc0 0x7f84d8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 -- 172.21.15.33:0/3717384107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8109150 msgr2=0x7f84d819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8109150 0x7f84d819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 -- 172.21.15.33:0/3717384107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d819f310 con 0x7f84d81087d0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d8102dc0 0x7f84d8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d7fff640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d81087d0 0x7f84d81997a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00c890 tx=0x7f84cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d5ffb640 1 -- 172.21.15.33:0/3717384107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc004220 con 0x7f84d81087d0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d819f600 con 0x7f84d81087d0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d5ffb640 1 -- 172.21.15.33:0/3717384107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84cc0043c0 con 0x7f84d81087d0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.600+0000 7f84d5ffb640 1 -- 172.21.15.33:0/3717384107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc004a00 con 0x7f84d81087d0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.601+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a5cc0 con 0x7f84d81087d0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 Processor -- start 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 -- start start 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84d5ffb640 1 -- 172.21.15.33:0/3717384107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f84cc0079b0 con 0x7f84d81087d0 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fabdf0 0x565402fb7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fb7d80 0x565402fba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565402f21ea0 con 0x565402fb7d80 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565402f112d0 con 0x565402fab4d0 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.603+0000 7f84e122f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565402f150e0 con 0x565402fabdf0 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fb7d80 0x565402fba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fb7d80 0x565402fba170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53632/0 (socket says 172.21.15.33:53632) 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 -- 172.21.15.33:0/2636973521 learned_addr learned my addr 172.21.15.33:0/2636973521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fabdf0 0x565402fb7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 -- 172.21.15.33:0/2636973521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fabdf0 msgr2=0x565402fb7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fabdf0 0x565402fb7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 -- 172.21.15.33:0/2636973521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 msgr2=0x565402fab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 -- 172.21.15.33:0/2636973521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565402c6f650 con 0x565402fb7d80 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d7fff640 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fb7d80 0x565402fba170 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0ebc20 tx=0x7f84cc0ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09837ca509036e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84d5ffb640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f84c00b52e0 0x7f84c00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f84c00b52e0 0x7f84c00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.604+0000 7f84beffd640 1 -- 172.21.15.33:0/2636973521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0f4070 con 0x565402fb7d80 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84d5ffb640 1 -- 172.21.15.33:0/3717384107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f84cc0e5970 con 0x7f84d81087d0 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84beffd640 1 -- 172.21.15.33:0/2636973521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84cc005290 con 0x565402fb7d80 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84beffd640 1 -- 172.21.15.33:0/2636973521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0a8780 con 0x565402fb7d80 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f84c00b52e0 0x7f84c00b77a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d0005fa0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 -- 172.21.15.33:0/2636973521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fb7d80 msgr2=0x565402fba170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fb7d80 0x565402fba170 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0ebc20 tx=0x7f84cc0ed670 comp rx=0 tx=0).stop 2024-09-10T21:12:30.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 -- 172.21.15.33:0/2636973521 shutdown_connections 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fb7d80 0x565402fba170 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fabdf0 0x565402fb7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 --2- 172.21.15.33:0/2636973521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 -- 172.21.15.33:0/2636973521 >> 172.21.15.33:0/2636973521 conn(0x565402fa9300 msgr2=0x565402faa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 -- 172.21.15.33:0/2636973521 shutdown_connections 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.605+0000 7f84e122f740 1 -- 172.21.15.33:0/2636973521 wait complete. 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 -- start start 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 Processor -- start 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fab4d0 0x565402fb76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fabdf0 0x565402fb4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fb5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x565402d72700 con 0x565402fab4d0 2024-09-10T21:12:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x565402eac160 con 0x565402fabdf0 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565402fbaa00 con 0x565402fb7d80 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84e122f740 10 client.0 _ll_register_callbacks cb 0x565402ec59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84dcbf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fab4d0 0x565402fb76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84dcbf6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fab4d0 0x565402fb76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53648/0 (socket says 172.21.15.33:53648) 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84dcbf6640 1 -- 172.21.15.33:0/3625296535 learned_addr learned my addr 172.21.15.33:0/3625296535 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.606+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fb5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84d7fff640 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fabdf0 0x565402fb4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84dcbf6640 1 -- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 msgr2=0x565402fb5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fb5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84dcbf6640 1 -- 172.21.15.33:0/3625296535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fabdf0 msgr2=0x565402fb4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fabdf0 0x565402fb4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84dcbf6640 1 -- 172.21.15.33:0/3625296535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565402ed9c90 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fab4d0 0x565402fb76d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f84c800bf30 tx=0x7f84c8012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2402676da40a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84b7fff640 1 -- 172.21.15.33:0/3625296535 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c80035f0 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84b7fff640 1 -- 172.21.15.33:0/3625296535 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f84c8003790 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x565402f23df0 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84b7fff640 1 -- 172.21.15.33:0/3625296535 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f84c8003990 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84e122f740 10 client.16971 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.607+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x565402fc63f0 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84b7fff640 1 -- 172.21.15.33:0/3625296535 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x565402f23df0 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84b7fff640 1 -- 172.21.15.33:0/3625296535 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f84c8019070 con 0x565402fab4d0 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84e122f740 10 client.16971 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84e122f740 10 client.16971 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:12:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84e122f740 20 client.16971 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84e122f740 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565402fc78f0 0x565402fc9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84e122f740 4 client.16971 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565402f1f8a0 con 0x565402fc78f0 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.608+0000 7f84d7fff640 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565402fc78f0 0x565402fc9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84b7fff640 10 client.16971 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84d7fff640 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565402fc78f0 0x565402fc9d40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c58faba3205f6141 server_cookie=3aaee26b746b3511 in_seq=0 out_seq=0 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84b7fff640 1 -- 172.21.15.33:0/3625296535 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565402f1f8a0 con 0x565402fc78f0 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84b7fff640 10 client.16971 handle_command_reply: tid=0 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 client.16971 shutdown 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fab4d0 msgr2=0x565402fb76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fab4d0 0x565402fb76d0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f84c800bf30 tx=0x7f84c8012820 comp rx=0 tx=0).stop 2024-09-10T21:12:30.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 shutdown_connections 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fb5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fabdf0 0x565402fb4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565402fc78f0 0x565402fc9d40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 --2- 172.21.15.33:0/3625296535 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fab4d0 0x565402fb76d0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84b7fff640 0 client.16971 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.609+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 >> 172.21.15.33:0/3625296535 conn(0x565402fa9300 msgr2=0x565402faee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.610+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 shutdown_connections 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.610+0000 7f84e122f740 1 -- 172.21.15.33:0/3625296535 wait complete. 2024-09-10T21:12:30.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.610+0000 7f84e122f740 20 client.16971 trim_cache size 0 max 16384 2024-09-10T21:12:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.623+0000 7f84e122f740 1 Processor -- start 2024-09-10T21:12:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.623+0000 7f84e122f740 1 -- start start 2024-09-10T21:12:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84e122f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fa8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84e122f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fc0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84e122f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fc05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84e122f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81ae4a0 con 0x565402fabdf0 2024-09-10T21:12:30.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84e122f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81affe0 con 0x565402fab4d0 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84e122f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81aeb70 con 0x565402fb7d80 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fc0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fc0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53650/0 (socket says 172.21.15.33:53650) 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 -- 172.21.15.33:0/4015798525 learned_addr learned my addr 172.21.15.33:0/4015798525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fa8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fc05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 -- 172.21.15.33:0/4015798525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 msgr2=0x565402fc05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fc05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 -- 172.21.15.33:0/4015798525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 msgr2=0x565402fa8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fa8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 -- 172.21.15.33:0/4015798525 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565402fabc20 con 0x565402fabdf0 2024-09-10T21:12:30.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fc05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84d7fff640 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fc0070 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0a8700 tx=0x7f84cc0eb530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19d2d306591f6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.624+0000 7f84be7fc640 1 -- 172.21.15.33:0/4015798525 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc0f4040 con 0x565402fabdf0 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84be7fc640 1 -- 172.21.15.33:0/4015798525 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84cc0abc40 con 0x565402fabdf0 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84be7fc640 1 -- 172.21.15.33:0/4015798525 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc01d040 con 0x565402fabdf0 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 -- 172.21.15.33:0/4015798525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 msgr2=0x565402fc0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fc0070 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0a8700 tx=0x7f84cc0eb530 comp rx=0 tx=0).stop 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 -- 172.21.15.33:0/4015798525 shutdown_connections 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fb7d80 0x565402fc05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fc0070 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 --2- 172.21.15.33:0/4015798525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fab4d0 0x565402fa8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 -- 172.21.15.33:0/4015798525 >> 172.21.15.33:0/4015798525 conn(0x565402fa9300 msgr2=0x565402fb6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 -- 172.21.15.33:0/4015798525 shutdown_connections 2024-09-10T21:12:30.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 -- 172.21.15.33:0/4015798525 wait complete. 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.625+0000 7f84e122f740 1 -- start start 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 1 Processor -- start 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fab4d0 0x565402fcfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fb1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fb7d80 0x565402fb2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81af910 con 0x565402fabdf0 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81af240 con 0x565402fb7d80 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81ac460 con 0x565402fab4d0 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84e122f740 10 client.0 _ll_register_callbacks cb 0x7f84c800b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fb1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84d7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fb1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53662/0 (socket says 172.21.15.33:53662) 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84d7fff640 1 -- 172.21.15.33:0/2284336214 learned_addr learned my addr 172.21.15.33:0/2284336214 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:30.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fb7d80 0x565402fb2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.626+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fab4d0 0x565402fcfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84d7fff640 1 -- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fab4d0 msgr2=0x565402fcfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84d7fff640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fab4d0 0x565402fcfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84d7fff640 1 -- 172.21.15.33:0/2284336214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fb7d80 msgr2=0x565402fb2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84d7fff640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fb7d80 0x565402fb2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84d7fff640 1 -- 172.21.15.33:0/2284336214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565402fb2980 con 0x565402fabdf0 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84dd3f7640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fb7d80 0x565402fb2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84d7fff640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fb1d50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0edb70 tx=0x7f84cc0076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ed339565f7d3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84b7fff640 1 -- 172.21.15.33:0/2284336214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84cc01d030 con 0x565402fabdf0 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84b7fff640 1 -- 172.21.15.33:0/2284336214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f84cc0eb6d0 con 0x565402fabdf0 2024-09-10T21:12:30.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84cc0abc40 con 0x565402fabdf0 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84b7fff640 1 -- 172.21.15.33:0/2284336214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f84cc0ab8a0 con 0x565402fabdf0 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84e122f740 10 client.16977 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x565403039890 con 0x565402fabdf0 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84b7fff640 1 -- 172.21.15.33:0/2284336214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84cc0abc40 con 0x565402fabdf0 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84b7fff640 1 -- 172.21.15.33:0/2284336214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f84cc0e34f0 con 0x565402fabdf0 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84e122f740 10 client.16977 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84e122f740 10 client.16977 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.627+0000 7f84e122f740 20 client.16977 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84e122f740 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565403039890 0x56540303bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84e122f740 4 client.16977 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x565402f1ccd0 con 0x565403039890 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565403039890 0x56540303bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:30.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84dcbf6640 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565403039890 0x56540303bc80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d99bab058fa0130 server_cookie=98b2bdf29ae067c5 in_seq=0 out_seq=0 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84b7fff640 10 client.16977 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84b7fff640 1 -- 172.21.15.33:0/2284336214 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x565402f1ccd0 con 0x565403039890 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84b7fff640 10 client.16977 handle_command_reply: tid=0 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.628+0000 7f84e122f740 1 client.16977 shutdown 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 msgr2=0x565402fb1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fb1d50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f84cc0edb70 tx=0x7f84cc0076c0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 shutdown_connections 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x565402fb7d80 0x565402fb2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x565403039890 0x56540303bc80 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84b7fff640 0 client.16977 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x565402fabdf0 0x565402fb1d50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 --2- 172.21.15.33:0/2284336214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565402fab4d0 0x565402fcfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 >> 172.21.15.33:0/2284336214 conn(0x565402fa9300 msgr2=0x565402fae540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 shutdown_connections 2024-09-10T21:12:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 1 -- 172.21.15.33:0/2284336214 wait complete. 2024-09-10T21:12:30.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84e122f740 20 client.16977 trim_cache size 0 max 16384 2024-09-10T21:12:30.642 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:12:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f84c00b52e0 msgr2=0x7f84c00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84dee5a640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f84c00b52e0 0x7f84c00b77a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f84d0006010 tx=0x7f84d0005fa0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d81087d0 msgr2=0x7f84d81997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.629+0000 7f84dee5a640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d81087d0 0x7f84d81997a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00c890 tx=0x7f84cc00cd50 comp rx=0 tx=0).stop 2024-09-10T21:12:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 shutdown_connections 2024-09-10T21:12:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f84c00b52e0 0x7f84c00b77a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f84d8109150 0x7f84d819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84d81087d0 0x7f84d81997a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 --2- 172.21.15.33:0/3717384107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84d8102dc0 0x7f84d8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 >> 172.21.15.33:0/3717384107 conn(0x7f84d80fe830 msgr2=0x7f84d80ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 shutdown_connections 2024-09-10T21:12:30.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:30.630+0000 7f84dee5a640 1 -- 172.21.15.33:0/3717384107 wait complete. 2024-09-10T21:12:30.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572044 5.43 MB/sec execute 331 sec latency 3571.146 ms 2024-09-10T21:12:31.330 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:12:31.590 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:31.616 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:31 smithi104 ceph-mon[35325]: pgmap v602: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 841 KiB/s wr, 107 op/s 2024-09-10T21:12:31.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:31 smithi174 ceph-mon[35635]: pgmap v602: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 841 KiB/s wr, 107 op/s 2024-09-10T21:12:31.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:31 smithi033 ceph-mon[30860]: pgmap v602: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 841 KiB/s wr, 107 op/s 2024-09-10T21:12:31.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572044 5.42 MB/sec execute 332 sec latency 4571.192 ms 2024-09-10T21:12:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.075+0000 7fc48f1aa640 1 -- 172.21.15.33:0/1848195896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc488104590 msgr2=0x7fc48810cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.075+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/1848195896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc488104590 0x7fc48810cb70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009900 tx=0x7fc47c02a410 comp rx=0 tx=0).stop 2024-09-10T21:12:32.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.075+0000 7fc48f1aa640 1 -- 172.21.15.33:0/1848195896 shutdown_connections 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.075+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/1848195896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc48810d140 0x7fc48810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.075+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/1848195896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc488104590 0x7fc48810cb70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.075+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/1848195896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc488103b90 0x7fc488103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 -- 172.21.15.33:0/1848195896 >> 172.21.15.33:0/1848195896 conn(0x7fc4880fd700 msgr2=0x7fc4880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 -- 172.21.15.33:0/1848195896 shutdown_connections 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 -- 172.21.15.33:0/1848195896 wait complete. 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 Processor -- start 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 -- start start 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc488103b90 0x7fc488101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc488104590 0x7fc488101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48810d140 0x7fc488102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc488112d60 con 0x7fc48810d140 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc488112be0 con 0x7fc488104590 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48f1aa640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc488112ee0 con 0x7fc488103b90 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48d747640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48810d140 0x7fc488102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.076+0000 7fc48d747640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48810d140 0x7fc488102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53694/0 (socket says 172.21.15.33:53694) 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48d747640 1 -- 172.21.15.33:0/4076232376 learned_addr learned my addr 172.21.15.33:0/4076232376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48cf46640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc488103b90 0x7fc488101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48d747640 1 -- 172.21.15.33:0/4076232376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc488103b90 msgr2=0x7fc488101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48d747640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc488103b90 0x7fc488101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48d747640 1 -- 172.21.15.33:0/4076232376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc488104590 msgr2=0x7fc488101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc487fff640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc488104590 0x7fc488101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48d747640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc488104590 0x7fc488101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48d747640 1 -- 172.21.15.33:0/4076232376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4881a7670 con 0x7fc48810d140 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc487fff640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc488104590 0x7fc488101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48d747640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48810d140 0x7fc488102490 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e9d0 tx=0x7fc48000edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc485ffb640 1 -- 172.21.15.33:0/4076232376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480004d20 con 0x7fc48810d140 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc485ffb640 1 -- 172.21.15.33:0/4076232376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc480005020 con 0x7fc48810d140 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4881a7960 con 0x7fc48810d140 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc485ffb640 1 -- 172.21.15.33:0/4076232376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4800116b0 con 0x7fc48810d140 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.077+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4881a7ea0 con 0x7fc48810d140 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc485ffb640 1 -- 172.21.15.33:0/4076232376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc48000f040 con 0x7fc48810d140 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 Processor -- start 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 -- start start 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eb0f0 0x5578474eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557847461ea0 con 0x5578474eb0f0 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5578474512d0 con 0x5578474f79a0 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc49157f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578474550e0 con 0x5578474eba10 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.080+0000 7fc487fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eb0f0 0x5578474eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc487fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eb0f0 0x5578474eb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53696/0 (socket says 172.21.15.33:53696) 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc487fff640 1 -- 172.21.15.33:0/1621749941 learned_addr learned my addr 172.21.15.33:0/1621749941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48cf46640 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc485ffb640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4700b52e0 0x7fc4700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc487fff640 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48cf46640 1 -- 172.21.15.33:0/1621749941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 msgr2=0x5578474f9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48cf46640 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48cf46640 1 -- 172.21.15.33:0/1621749941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eb0f0 msgr2=0x5578474eb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48cf46640 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eb0f0 0x5578474eb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48d747640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4700b52e0 0x7fc4700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48cf46640 1 -- 172.21.15.33:0/1621749941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578471af650 con 0x5578474eba10 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc487fff640 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eb0f0 0x5578474eb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc485ffb640 1 -- 172.21.15.33:0/4076232376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc4800e5e50 con 0x7fc48810d140 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48cf46640 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f7460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc47800abf0 tx=0x7fc47800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20d0581c1950710d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc48d747640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4700b52e0 0x7fc4700b77a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc48004f260 tx=0x7fc480079aa0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc46effd640 1 -- 172.21.15.33:0/1621749941 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478005220 con 0x5578474eba10 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc46effd640 1 -- 172.21.15.33:0/1621749941 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4780047a0 con 0x5578474eba10 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.081+0000 7fc46effd640 1 -- 172.21.15.33:0/1621749941 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4780186e0 con 0x5578474eba10 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 -- 172.21.15.33:0/1621749941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 msgr2=0x5578474f7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f7460 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc47800abf0 tx=0x7fc47800f790 comp rx=0 tx=0).stop 2024-09-10T21:12:32.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 -- 172.21.15.33:0/1621749941 shutdown_connections 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f7460 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 --2- 172.21.15.33:0/1621749941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eb0f0 0x5578474eb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 -- 172.21.15.33:0/1621749941 >> 172.21.15.33:0/1621749941 conn(0x5578474e8f20 msgr2=0x5578474ea0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 -- 172.21.15.33:0/1621749941 shutdown_connections 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 -- 172.21.15.33:0/1621749941 wait complete. 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.082+0000 7fc49157f740 1 -- start start 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 1 Processor -- start 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474f79a0 0x5578474f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5578472b2700 con 0x5578474f79a0 2024-09-10T21:12:32.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5578473ec160 con 0x5578474eb0f0 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578474fa620 con 0x5578474eba10 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc49157f740 10 client.0 _ll_register_callbacks cb 0x5578474059e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc487fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474f79a0 0x5578474f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc48cf46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc48cf46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59204/0 (socket says 172.21.15.33:59204) 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc48cf46640 1 -- 172.21.15.33:0/285675256 learned_addr learned my addr 172.21.15.33:0/285675256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.083+0000 7fc487fff640 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc48cf46640 1 -- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 msgr2=0x5578474f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc48cf46640 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474f71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc48cf46640 1 -- 172.21.15.33:0/285675256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474f79a0 msgr2=0x5578474f4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc48cf46640 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474f79a0 0x5578474f4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc48cf46640 1 -- 172.21.15.33:0/285675256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557847419c90 con 0x5578474eba10 2024-09-10T21:12:32.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc487fff640 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474f79a0 0x5578474f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc48cf46640 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f4640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc478010010 tx=0x7fc4780051d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c80824407cead9d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc467fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc467fff640 1 -- 172.21.15.33:0/285675256 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478027070 con 0x5578474eba10 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4780047a0 con 0x5578474eba10 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc467fff640 1 -- 172.21.15.33:0/285675256 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc4780055d0 con 0x5578474eba10 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.084+0000 7fc467fff640 1 -- 172.21.15.33:0/285675256 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc478031050 con 0x5578474eba10 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 10 client.26395 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557847505f10 con 0x5578474eba10 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc467fff640 1 -- 172.21.15.33:0/285675256 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4780047a0 con 0x5578474eba10 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc467fff640 1 -- 172.21.15.33:0/285675256 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fc478034020 con 0x5578474eba10 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 10 client.26395 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 10 client.26395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 20 client.26395 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:32.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5578475073d0 0x5578475097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 4 client.26395 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.085+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557847463df0 con 0x5578475073d0 2024-09-10T21:12:32.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.086+0000 7fc487fff640 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5578475073d0 0x5578475097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.086+0000 7fc467fff640 10 client.26395 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.086+0000 7fc487fff640 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5578475073d0 0x5578475097c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a43b015faac4f9c1 server_cookie=eef834cd96c83329 in_seq=0 out_seq=0 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc467fff640 1 -- 172.21.15.33:0/285675256 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557847463df0 con 0x5578475073d0 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc467fff640 10 client.26395 handle_command_reply: tid=0 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 client.26395 shutdown 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 msgr2=0x5578474f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f4640 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc478010010 tx=0x7fc4780051d0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 shutdown_connections 2024-09-10T21:12:32.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5578475073d0 0x5578475097c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474f79a0 0x5578474f4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc467fff640 0 client.26395 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eba10 0x5578474f4640 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 --2- 172.21.15.33:0/285675256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 >> 172.21.15.33:0/285675256 conn(0x5578474e8f20 msgr2=0x5578474eea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 shutdown_connections 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 1 -- 172.21.15.33:0/285675256 wait complete. 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.087+0000 7fc49157f740 20 client.26395 trim_cache size 0 max 16384 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 Processor -- start 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 -- start start 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eb0f0 0x5578474e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474e8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f6380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4881b0630 con 0x5578474eba10 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4881b2170 con 0x5578474f79a0 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc49157f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4881b0d00 con 0x5578474eb0f0 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474e8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474e8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53728/0 (socket says 172.21.15.33:53728) 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 -- 172.21.15.33:0/2833382775 learned_addr learned my addr 172.21.15.33:0/2833382775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc487fff640 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc487fff640 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eb0f0 0x5578474e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 -- 172.21.15.33:0/2833382775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eb0f0 msgr2=0x5578474e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eb0f0 0x5578474e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 -- 172.21.15.33:0/2833382775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 msgr2=0x5578474f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 -- 172.21.15.33:0/2833382775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578474fa670 con 0x5578474eba10 2024-09-10T21:12:32.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.101+0000 7fc48cf46640 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474e8760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc478000c00 tx=0x7fc478005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45154fc7e0330c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc46e7fc640 1 -- 172.21.15.33:0/2833382775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478027070 con 0x5578474eba10 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc46e7fc640 1 -- 172.21.15.33:0/2833382775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc47800aa40 con 0x5578474eba10 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc46e7fc640 1 -- 172.21.15.33:0/2833382775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478023550 con 0x5578474eba10 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc49157f740 1 -- 172.21.15.33:0/2833382775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 msgr2=0x5578474e8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc49157f740 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474e8760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fc478000c00 tx=0x7fc478005240 comp rx=0 tx=0).stop 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc49157f740 1 -- 172.21.15.33:0/2833382775 shutdown_connections 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc49157f740 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474f79a0 0x5578474f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc49157f740 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474e8760 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.102+0000 7fc49157f740 1 --2- 172.21.15.33:0/2833382775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474eb0f0 0x5578474e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 -- 172.21.15.33:0/2833382775 >> 172.21.15.33:0/2833382775 conn(0x5578474e8f20 msgr2=0x5578474f5c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 -- 172.21.15.33:0/2833382775 shutdown_connections 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 -- 172.21.15.33:0/2833382775 wait complete. 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 -- start start 2024-09-10T21:12:32.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 Processor -- start 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474f79a0 0x557847573020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc4881b1aa0 con 0x5578474eba10 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc4881b13d0 con 0x5578474eb0f0 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4881ae5f0 con 0x5578474f79a0 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.103+0000 7fc49157f740 10 client.0 _ll_register_callbacks cb 0x7fc47c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc48cf46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56632/0 (socket says 172.21.15.33:56632) 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 -- 172.21.15.33:0/631518670 learned_addr learned my addr 172.21.15.33:0/631518670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc48cf46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474fc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53738/0 (socket says 172.21.15.33:53738) 2024-09-10T21:12:32.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474f79a0 0x557847573020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 -- 172.21.15.33:0/631518670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474f79a0 msgr2=0x557847573020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474f79a0 0x557847573020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 -- 172.21.15.33:0/631518670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 msgr2=0x5578474fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 -- 172.21.15.33:0/631518670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578474e8ca0 con 0x5578474eb0f0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc48cf46640 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc487fff640 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474fbbc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006010 tx=0x7fc47800a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfeedc4375c5ac33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc467fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc467fff640 1 -- 172.21.15.33:0/631518670 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c058060 con 0x5578474eb0f0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc47800aa40 con 0x5578474eb0f0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.104+0000 7fc467fff640 1 -- 172.21.15.33:0/631518670 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc47c004250 con 0x5578474eb0f0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc467fff640 1 -- 172.21.15.33:0/631518670 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc47c04d040 con 0x5578474eb0f0 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 10 client.26459 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:32.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557847574590 con 0x5578474eb0f0 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc467fff640 1 -- 172.21.15.33:0/631518670 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc47800aa40 con 0x5578474eb0f0 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc467fff640 1 -- 172.21.15.33:0/631518670 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fc47c05d070 con 0x5578474eb0f0 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 10 client.26459 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 10 client.26459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 20 client.26459 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557847574ca0 0x5578475770c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 4 client.26459 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55784745f8a0 con 0x557847574ca0 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.105+0000 7fc48cf46640 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557847574ca0 0x5578475770c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.106+0000 7fc48cf46640 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557847574ca0 0x5578475770c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7873d97799c6c6d7 server_cookie=eb489c01aad5a069 in_seq=0 out_seq=0 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.106+0000 7fc467fff640 10 client.26459 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:32.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.106+0000 7fc467fff640 1 -- 172.21.15.33:0/631518670 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55784745f8a0 con 0x557847574ca0 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.106+0000 7fc467fff640 10 client.26459 handle_command_reply: tid=0 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.106+0000 7fc49157f740 1 client.26459 shutdown 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 msgr2=0x5578474fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474fbbc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fc47c006010 tx=0x7fc47800a940 comp rx=0 tx=0).stop 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 shutdown_connections 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x557847574ca0 0x5578475770c0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc467fff640 0 client.26459 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578474f79a0 0x557847573020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578474eba10 0x5578474fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 --2- 172.21.15.33:0/631518670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578474eb0f0 0x5578474fbbc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 >> 172.21.15.33:0/631518670 conn(0x5578474e8f20 msgr2=0x5578474f6490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 shutdown_connections 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 1 -- 172.21.15.33:0/631518670 wait complete. 2024-09-10T21:12:32.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.107+0000 7fc49157f740 20 client.26459 trim_cache size 0 max 16384 2024-09-10T21:12:32.120 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:12:32.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4700b52e0 msgr2=0x7fc4700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4700b52e0 0x7fc4700b77a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc48004f260 tx=0x7fc480079aa0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48810d140 msgr2=0x7fc488102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48810d140 0x7fc488102490 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fc48000e9d0 tx=0x7fc48000edd0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 shutdown_connections 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4700b52e0 0x7fc4700b77a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc48810d140 0x7fc488102490 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc488104590 0x7fc488101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 --2- 172.21.15.33:0/4076232376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc488103b90 0x7fc488101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 >> 172.21.15.33:0/4076232376 conn(0x7fc4880fd700 msgr2=0x7fc4880fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 shutdown_connections 2024-09-10T21:12:32.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:32.108+0000 7fc48f1aa640 1 -- 172.21.15.33:0/4076232376 wait complete. 2024-09-10T21:12:32.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 572866 5.41 MB/sec execute 333 sec latency 4929.495 ms 2024-09-10T21:12:33.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:33 smithi174 ceph-mon[35635]: pgmap v603: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 808 KiB/s wr, 101 op/s 2024-09-10T21:12:33.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:33 smithi033 ceph-mon[30860]: pgmap v603: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 808 KiB/s wr, 101 op/s 2024-09-10T21:12:33.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 575549 5.44 MB/sec execute 334 sec latency 51.981 ms 2024-09-10T21:12:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:33 smithi104 ceph-mon[35325]: pgmap v603: 97 pgs: 97 active+clean; 221 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 808 KiB/s wr, 101 op/s 2024-09-10T21:12:33.925 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:12:33.925 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:12:34.186 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:34.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 -- 172.21.15.33:0/3992742693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3958102dc0 msgr2=0x7f39581031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 --2- 172.21.15.33:0/3992742693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3958102dc0 0x7f39581031a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3948009900 tx=0x7f394802a410 comp rx=0 tx=0).stop 2024-09-10T21:12:34.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 -- 172.21.15.33:0/3992742693 shutdown_connections 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 --2- 172.21.15.33:0/3992742693 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3958109150 0x7f395810f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 --2- 172.21.15.33:0/3992742693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39581087d0 0x7f3958108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 --2- 172.21.15.33:0/3992742693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3958102dc0 0x7f39581031a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 -- 172.21.15.33:0/3992742693 >> 172.21.15.33:0/3992742693 conn(0x7f39580fe830 msgr2=0x7f3958100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 -- 172.21.15.33:0/3992742693 shutdown_connections 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.657+0000 7f395d611640 1 -- 172.21.15.33:0/3992742693 wait complete. 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 Processor -- start 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 -- start start 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3958102dc0 0x7f39581992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39581087d0 0x7f3958199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3958109150 0x7f395819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3958112dd0 con 0x7f39581087d0 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3958112c50 con 0x7f3958109150 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f395d611640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3958112f50 con 0x7f3958102dc0 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f394ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39581087d0 0x7f3958199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f394ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39581087d0 0x7f3958199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53752/0 (socket says 172.21.15.33:53752) 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f394ffff640 1 -- 172.21.15.33:0/2978120173 learned_addr learned my addr 172.21.15.33:0/2978120173 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f39577fe640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3958109150 0x7f395819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.658+0000 7f3956ffd640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3958102dc0 0x7f39581992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f394ffff640 1 -- 172.21.15.33:0/2978120173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3958102dc0 msgr2=0x7f39581992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f394ffff640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3958102dc0 0x7f39581992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f394ffff640 1 -- 172.21.15.33:0/2978120173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3958109150 msgr2=0x7f395819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f394ffff640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3958109150 0x7f395819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f394ffff640 1 -- 172.21.15.33:0/2978120173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395819f370 con 0x7f39581087d0 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f3956ffd640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3958102dc0 0x7f39581992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f394ffff640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39581087d0 0x7f3958199800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f394000c920 tx=0x7f394000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f3954ff9640 1 -- 172.21.15.33:0/2978120173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940007c70 con 0x7f39581087d0 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f3954ff9640 1 -- 172.21.15.33:0/2978120173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3940007e10 con 0x7f39581087d0 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f3954ff9640 1 -- 172.21.15.33:0/2978120173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940018820 con 0x7f39581087d0 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395819f660 con 0x7f39581087d0 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.659+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39581a5ef0 con 0x7f39581087d0 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 Processor -- start 2024-09-10T21:12:34.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f3954ff9640 1 -- 172.21.15.33:0/2978120173 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f394000f450 con 0x7f39581087d0 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 -- start start 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e81e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e820b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c61e788e60 con 0x55c61e81e790 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c61e778290 con 0x55c61e811ee0 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.662+0000 7f395f9e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c61e77c0a0 con 0x55c61e812800 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f394ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e820b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e81e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e81e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59246/0 (socket says 172.21.15.33:59246) 2024-09-10T21:12:34.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f394ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e820b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53754/0 (socket says 172.21.15.33:53754) 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 -- 172.21.15.33:0/398971510 learned_addr learned my addr 172.21.15.33:0/398971510 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f39577fe640 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 -- 172.21.15.33:0/398971510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 msgr2=0x55c61e8122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 -- 172.21.15.33:0/398971510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 msgr2=0x55c61e820b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e820b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 -- 172.21.15.33:0/398971510 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61e4d6930 con 0x55c61e812800 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f39577fe640 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3954ff9640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39440b5360 0x7f39440b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f39577fe640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39440b5360 0x7f39440b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3956ffd640 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e81e250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3948009900 tx=0x7f3948031e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbba06ad1533d678 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f3954ff9640 1 -- 172.21.15.33:0/2978120173 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f39400e5c90 con 0x7f39581087d0 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.663+0000 7f394d7fa640 1 -- 172.21.15.33:0/398971510 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394803f070 con 0x55c61e812800 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f394d7fa640 1 -- 172.21.15.33:0/398971510 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39480316a0 con 0x55c61e812800 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f394d7fa640 1 -- 172.21.15.33:0/398971510 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948035070 con 0x55c61e812800 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 -- 172.21.15.33:0/398971510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 msgr2=0x55c61e81e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f39577fe640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39440b5360 0x7f39440b7820 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3950010540 tx=0x7f395000a4d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e81e250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3948009900 tx=0x7f3948031e60 comp rx=0 tx=0).stop 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 -- 172.21.15.33:0/398971510 shutdown_connections 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e820b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e81e250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 --2- 172.21.15.33:0/398971510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 -- 172.21.15.33:0/398971510 >> 172.21.15.33:0/398971510 conn(0x55c61e80fd10 msgr2=0x55c61e810eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 -- 172.21.15.33:0/398971510 shutdown_connections 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.664+0000 7f395f9e6740 1 -- 172.21.15.33:0/398971510 wait complete. 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 -- start start 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 Processor -- start 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e81b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e81bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e81c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c61e5d9890 con 0x55c61e81e790 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c61e712d40 con 0x55c61e812800 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c61e8213e0 con 0x55c61e811ee0 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f395f9e6740 10 client.0 _ll_register_callbacks cb 0x55c61e776c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f39577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e81c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f39577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e81c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53766/0 (socket says 172.21.15.33:53766) 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f3956ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e81b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f39577fe640 1 -- 172.21.15.33:0/2348000554 learned_addr learned my addr 172.21.15.33:0/2348000554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.665+0000 7f394ffff640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e81bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f39577fe640 1 -- 172.21.15.33:0/2348000554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 msgr2=0x55c61e81b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f39577fe640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e81b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f39577fe640 1 -- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 msgr2=0x55c61e81bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f39577fe640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e81bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f39577fe640 1 -- 172.21.15.33:0/2348000554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61e82a870 con 0x55c61e81e790 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f394ffff640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e81bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f3956ffd640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e81b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f39577fe640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e81c240 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3950005e50 tx=0x7f3950004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed367964f1e18ba7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f393e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f393e7fc640 1 -- 172.21.15.33:0/2348000554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395000fd80 con 0x55c61e81e790 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f393e7fc640 1 -- 172.21.15.33:0/2348000554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3950016030 con 0x55c61e81e790 2024-09-10T21:12:34.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39480316a0 con 0x55c61e81e790 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f393e7fc640 1 -- 172.21.15.33:0/2348000554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3950003970 con 0x55c61e81e790 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f395f9e6740 10 client.17007 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c61e82d260 con 0x55c61e81e790 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.666+0000 7f393e7fc640 1 -- 172.21.15.33:0/2348000554 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39480316a0 con 0x55c61e81e790 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f393e7fc640 1 -- 172.21.15.33:0/2348000554 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f3950027980 con 0x55c61e81e790 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f395f9e6740 10 client.17007 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f395f9e6740 10 client.17007 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f395f9e6740 20 client.17007 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f395f9e6740 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e82e7c0 0x55c61e830c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f395f9e6740 4 client.17007 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:34.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c61e78adb0 con 0x55c61e82e7c0 2024-09-10T21:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f3956ffd640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e82e7c0 0x55c61e830c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f393e7fc640 10 client.17007 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.667+0000 7f3956ffd640 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e82e7c0 0x55c61e830c20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd0696fd8364071 server_cookie=d42a0a76578513df in_seq=0 out_seq=0 2024-09-10T21:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f393e7fc640 1 -- 172.21.15.33:0/2348000554 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c61e78adb0 con 0x55c61e82e7c0 2024-09-10T21:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f393e7fc640 10 client.17007 handle_command_reply: tid=0 2024-09-10T21:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 client.17007 shutdown 2024-09-10T21:12:34.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 msgr2=0x55c61e81c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e81c240 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f3950005e50 tx=0x7f3950004ab0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 shutdown_connections 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e82e7c0 0x55c61e830c20 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e81c240 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e81bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 --2- 172.21.15.33:0/2348000554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e81b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f393e7fc640 0 client.17007 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 >> 172.21.15.33:0/2348000554 conn(0x55c61e80fd10 msgr2=0x55c61e80ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 shutdown_connections 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.668+0000 7f395f9e6740 1 -- 172.21.15.33:0/2348000554 wait complete. 2024-09-10T21:12:34.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.669+0000 7f395f9e6740 20 client.17007 trim_cache size 0 max 16384 2024-09-10T21:12:34.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 Processor -- start 2024-09-10T21:12:34.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 -- start start 2024-09-10T21:12:34.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8157a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e8189d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e818f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ae420 con 0x55c61e81e790 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39581aff60 con 0x55c61e811ee0 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f395f9e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39581aeaf0 con 0x55c61e812800 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f39577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e818f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f39577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e818f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53768/0 (socket says 172.21.15.33:53768) 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f39577fe640 1 -- 172.21.15.33:0/3308868582 learned_addr learned my addr 172.21.15.33:0/3308868582 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f3956ffd640 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8157a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.687+0000 7f394ffff640 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e8189d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f39577fe640 1 -- 172.21.15.33:0/3308868582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 msgr2=0x55c61e8189d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f39577fe640 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e8189d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f39577fe640 1 -- 172.21.15.33:0/3308868582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 msgr2=0x55c61e8157a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f39577fe640 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8157a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f39577fe640 1 -- 172.21.15.33:0/3308868582 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61e810790 con 0x55c61e81e790 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f39577fe640 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e818f10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3950017040 tx=0x7f395000a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8632abd6b8d13c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f394cff9640 1 -- 172.21.15.33:0/3308868582 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950036030 con 0x55c61e81e790 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f394cff9640 1 -- 172.21.15.33:0/3308868582 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3950038070 con 0x55c61e81e790 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.688+0000 7f394cff9640 1 -- 172.21.15.33:0/3308868582 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395001cb00 con 0x55c61e81e790 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 -- 172.21.15.33:0/3308868582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 msgr2=0x55c61e818f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e818f10 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f3950017040 tx=0x7f395000a8a0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 -- 172.21.15.33:0/3308868582 shutdown_connections 2024-09-10T21:12:34.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e818f10 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e812800 0x55c61e8189d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 --2- 172.21.15.33:0/3308868582 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e811ee0 0x55c61e8157a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 -- 172.21.15.33:0/3308868582 >> 172.21.15.33:0/3308868582 conn(0x55c61e80fd10 msgr2=0x55c61e815090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 -- 172.21.15.33:0/3308868582 shutdown_connections 2024-09-10T21:12:34.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 -- 172.21.15.33:0/3308868582 wait complete. 2024-09-10T21:12:34.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 -- start start 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.689+0000 7f395f9e6740 1 Processor -- start 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e826cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e8231a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f395f9e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e8236e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f395f9e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39581af890 con 0x55c61e81e790 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f395f9e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39581af1c0 con 0x55c61e812800 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f395f9e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39581ac3e0 con 0x55c61e811ee0 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f395f9e6740 10 client.0 _ll_register_callbacks cb 0x7f395000ad70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e8236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e8236e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53770/0 (socket says 172.21.15.33:53770) 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 -- 172.21.15.33:0/865923545 learned_addr learned my addr 172.21.15.33:0/865923545 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f3956ffd640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e826cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f394ffff640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e8231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 -- 172.21.15.33:0/865923545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 msgr2=0x55c61e826cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e826cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 -- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 msgr2=0x55c61e8231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e8231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 -- 172.21.15.33:0/865923545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c61e823d80 con 0x55c61e81e790 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f394ffff640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e8231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f39577fe640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e8236e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3950010dd0 tx=0x7f3950015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8757998872e59154 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.690+0000 7f393e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:12:34.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f393e7fc640 1 -- 172.21.15.33:0/865923545 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3950036070 con 0x55c61e81e790 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3950038070 con 0x55c61e81e790 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f393e7fc640 1 -- 172.21.15.33:0/865923545 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3950027420 con 0x55c61e81e790 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f393e7fc640 1 -- 172.21.15.33:0/865923545 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3950027620 con 0x55c61e81e790 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 10 client.17013 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55c61e89c610 con 0x55c61e81e790 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f393e7fc640 1 -- 172.21.15.33:0/865923545 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3950038070 con 0x55c61e81e790 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f393e7fc640 1 -- 172.21.15.33:0/865923545 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f3950005150 con 0x55c61e81e790 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 10 client.17013 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 10 client.17013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 20 client.17013 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e89cc70 0x55c61e89f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 4 client.17013 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:12:34.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c61e786860 con 0x55c61e89cc70 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.691+0000 7f3956ffd640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e89cc70 0x55c61e89f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.692+0000 7f3956ffd640 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e89cc70 0x55c61e89f060 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=970053c1d0f6e283 server_cookie=4546572604eef0a4 in_seq=0 out_seq=0 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.692+0000 7f393e7fc640 10 client.17013 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f393e7fc640 1 -- 172.21.15.33:0/865923545 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c61e786860 con 0x55c61e89cc70 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f393e7fc640 10 client.17013 handle_command_reply: tid=0 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f395f9e6740 1 client.17013 shutdown 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 msgr2=0x55c61e8236e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f395f9e6740 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e8236e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3950010dd0 tx=0x7f3950015000 comp rx=0 tx=0).stop 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 shutdown_connections 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f395f9e6740 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c61e89cc70 0x55c61e89f060 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.693+0000 7f395f9e6740 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c61e81e790 0x55c61e8236e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395f9e6740 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c61e812800 0x55c61e8231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395f9e6740 1 --2- 172.21.15.33:0/865923545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c61e811ee0 0x55c61e826cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 >> 172.21.15.33:0/865923545 conn(0x55c61e80fd10 msgr2=0x55c61e8158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f393e7fc640 0 client.17013 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 shutdown_connections 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395f9e6740 1 -- 172.21.15.33:0/865923545 wait complete. 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395f9e6740 20 client.17013 trim_cache size 0 max 16384 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "e69efbf4-61ef-4ef6-be19-b7823dae502f", 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:12:34.711 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:12:34.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39440b5360 msgr2=0x7f39440b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395d611640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39440b5360 0x7f39440b7820 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3950010540 tx=0x7f395000a4d0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39581087d0 msgr2=0x7f3958199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:34.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395d611640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39581087d0 0x7f3958199800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f394000c920 tx=0x7f394000cde0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.694+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 shutdown_connections 2024-09-10T21:12:34.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.695+0000 7f395d611640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39440b5360 0x7f39440b7820 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.695+0000 7f395d611640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3958109150 0x7f395819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.695+0000 7f395d611640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39581087d0 0x7f3958199800 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.695+0000 7f395d611640 1 --2- 172.21.15.33:0/2978120173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3958102dc0 0x7f39581992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:34.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.695+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 >> 172.21.15.33:0/2978120173 conn(0x7f39580fe830 msgr2=0x7f3958100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:34.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.695+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 shutdown_connections 2024-09-10T21:12:34.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:34.695+0000 7f395d611640 1 -- 172.21.15.33:0/2978120173 wait complete. 2024-09-10T21:12:34.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 575549 5.42 MB/sec execute 335 sec latency 1052.022 ms 2024-09-10T21:12:35.412 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: pgmap v604: 97 pgs: 97 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 821 KiB/s wr, 92 op/s 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: pgmap v604: 97 pgs: 97 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 821 KiB/s wr, 92 op/s 2024-09-10T21:12:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:12:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.690 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:12:35.742 INFO:tasks.workunit.client.0.smithi033.stdout: 1 577102 5.43 MB/sec execute 336 sec latency 1053.123 ms 2024-09-10T21:12:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: pgmap v604: 97 pgs: 97 active+clean; 222 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 579 KiB/s rd, 821 KiB/s wr, 92 op/s 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:35.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:35.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:35.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 -- 172.21.15.33:0/2147763961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5788108020 msgr2=0x7f578810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 --2- 172.21.15.33:0/2147763961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5788108020 0x7f578810e820 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5780009be0 tx=0x7f578002a6c0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 -- 172.21.15.33:0/2147763961 shutdown_connections 2024-09-10T21:12:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 --2- 172.21.15.33:0/2147763961 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5788108020 0x7f578810e820 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 --2- 172.21.15.33:0/2147763961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57881076a0 0x7f5788107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 --2- 172.21.15.33:0/2147763961 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788101c90 0x7f5788102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 -- 172.21.15.33:0/2147763961 >> 172.21.15.33:0/2147763961 conn(0x7f57880fd700 msgr2=0x7f57880ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 -- 172.21.15.33:0/2147763961 shutdown_connections 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 -- 172.21.15.33:0/2147763961 wait complete. 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 Processor -- start 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.254+0000 7f578e4f1640 1 -- start start 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578e4f1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788101c90 0x7f5788199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578e4f1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57881076a0 0x7f57881997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578e4f1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788108020 0x7f578819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578e4f1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5788111ca0 con 0x7f5788108020 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578e4f1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5788111b20 con 0x7f57881076a0 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578e4f1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5788111e20 con 0x7f5788101c90 2024-09-10T21:12:36.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578ca8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788108020 0x7f578819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f57877fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57881076a0 0x7f57881997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578ca8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788108020 0x7f578819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53784/0 (socket says 172.21.15.33:53784) 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f57877fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57881076a0 0x7f57881997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56716/0 (socket says 172.21.15.33:56716) 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578ca8e640 1 -- 172.21.15.33:0/417111241 learned_addr learned my addr 172.21.15.33:0/417111241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f5787fff640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788101c90 0x7f5788199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f57877fe640 1 -- 172.21.15.33:0/417111241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788101c90 msgr2=0x7f5788199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f57877fe640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788101c90 0x7f5788199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f57877fe640 1 -- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788108020 msgr2=0x7f578819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f57877fe640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788108020 0x7f578819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f57877fe640 1 -- 172.21.15.33:0/417111241 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f578819f320 con 0x7f57881076a0 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.255+0000 7f578ca8e640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788108020 0x7f578819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.256+0000 7f57877fe640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57881076a0 0x7f57881997b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f577c00e9f0 tx=0x7f577c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.256+0000 7f57857fa640 1 -- 172.21.15.33:0/417111241 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c00ccc0 con 0x7f57881076a0 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.256+0000 7f57857fa640 1 -- 172.21.15.33:0/417111241 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c005020 con 0x7f57881076a0 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.256+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f578819f610 con 0x7f57881076a0 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.256+0000 7f57857fa640 1 -- 172.21.15.33:0/417111241 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c00ce60 con 0x7f57881076a0 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.256+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57881a5cd0 con 0x7f57881076a0 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.258+0000 7f57908c6740 1 Processor -- start 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.258+0000 7f57908c6740 1 -- start start 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57908c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bc480 0x5555f25bc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57908c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bcda0 0x5555f25ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57908c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25cd140 0x5555f25cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57908c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5555f2532e50 con 0x5555f25bc480 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57908c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5555f2522280 con 0x5555f25bcda0 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57908c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555f2526090 con 0x5555f25cd140 2024-09-10T21:12:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57857fa640 1 -- 172.21.15.33:0/417111241 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f577c004a60 con 0x7f57881076a0 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bc480 0x5555f25bc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bc480 0x5555f25bc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53786/0 (socket says 172.21.15.33:53786) 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f5787fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bcda0 0x5555f25ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 -- 172.21.15.33:0/1803896436 learned_addr learned my addr 172.21.15.33:0/1803896436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f57877fe640 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25cd140 0x5555f25cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 -- 172.21.15.33:0/1803896436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25cd140 msgr2=0x5555f25cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25cd140 0x5555f25cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 -- 172.21.15.33:0/1803896436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bcda0 msgr2=0x5555f25ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bcda0 0x5555f25ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 -- 172.21.15.33:0/1803896436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f2280650 con 0x5555f25bc480 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.259+0000 7f578ca8e640 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bc480 0x5555f25bc860 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5780030830 tx=0x7f5780030860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55e7e4e47f67c04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f576e7fc640 1 -- 172.21.15.33:0/1803896436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780041070 con 0x5555f25bc480 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f576e7fc640 1 -- 172.21.15.33:0/1803896436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5780030e20 con 0x5555f25bc480 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f576e7fc640 1 -- 172.21.15.33:0/1803896436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5780036040 con 0x5555f25bc480 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57857fa640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57700b52e0 0x7f57700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f5787fff640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57700b52e0 0x7f57700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57857fa640 1 -- 172.21.15.33:0/417111241 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f577c0e5b60 con 0x7f57881076a0 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57908c6740 1 -- 172.21.15.33:0/1803896436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bc480 msgr2=0x5555f25bc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f5787fff640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57700b52e0 0x7f57700b77a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5555f25bf600 tx=0x7f5778009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57908c6740 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bc480 0x5555f25bc860 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f5780030830 tx=0x7f5780030860 comp rx=0 tx=0).stop 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57908c6740 1 -- 172.21.15.33:0/1803896436 shutdown_connections 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57908c6740 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25cd140 0x5555f25cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57908c6740 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bcda0 0x5555f25ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57908c6740 1 --2- 172.21.15.33:0/1803896436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bc480 0x5555f25bc860 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.260+0000 7f57908c6740 1 -- 172.21.15.33:0/1803896436 >> 172.21.15.33:0/1803896436 conn(0x5555f25ba2b0 msgr2=0x5555f25bb450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.261+0000 7f57908c6740 1 -- 172.21.15.33:0/1803896436 shutdown_connections 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.261+0000 7f57908c6740 1 -- 172.21.15.33:0/1803896436 wait complete. 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.261+0000 7f57908c6740 1 -- start start 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.261+0000 7f57908c6740 1 Processor -- start 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f57908c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f57908c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f57908c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f57908c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5555f2383700 con 0x5555f25bcda0 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f57908c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5555f24bd110 con 0x5555f25cd140 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f57908c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555f25cfd90 con 0x5555f25bc480 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f57908c6740 10 client.0 _ll_register_callbacks cb 0x5555f24d6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59318/0 (socket says 172.21.15.33:59318) 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 -- 172.21.15.33:0/982915188 learned_addr learned my addr 172.21.15.33:0/982915188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f578ca8e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 -- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 msgr2=0x5555f25ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 -- 172.21.15.33:0/982915188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 msgr2=0x5555f25c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f5787fff640 1 -- 172.21.15.33:0/982915188 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f24eac40 con 0x5555f25bc480 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.262+0000 7f578ca8e640 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f5787fff640 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25cc9e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5778004560 tx=0x7f5778004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ec2e267fe320173 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57677fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57677fe640 1 -- 172.21.15.33:0/982915188 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577802c060 con 0x5555f25bc480 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57677fe640 1 -- 172.21.15.33:0/982915188 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f577801d710 con 0x5555f25bc480 2024-09-10T21:12:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5780030e20 con 0x5555f25bc480 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57677fe640 1 -- 172.21.15.33:0/982915188 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f577801d910 con 0x5555f25bc480 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57908c6740 10 client.26428 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5555f25db710 con 0x5555f25bc480 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57677fe640 1 -- 172.21.15.33:0/982915188 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5780030e20 con 0x5555f25bc480 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57677fe640 1 -- 172.21.15.33:0/982915188 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f5778031070 con 0x5555f25bc480 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57908c6740 10 client.26428 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57908c6740 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.263+0000 7f57908c6740 20 client.26428 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.264+0000 7f57908c6740 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f25dcc70 0x5555f25df0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.264+0000 7f57908c6740 4 client.26428 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.264+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555f2534da0 con 0x5555f25dcc70 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.264+0000 7f57877fe640 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f25dcc70 0x5555f25df0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.264+0000 7f57877fe640 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f25dcc70 0x5555f25df0f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cc81e61199093b server_cookie=3d1eea617ce066ad in_seq=0 out_seq=0 2024-09-10T21:12:36.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.264+0000 7f57677fe640 10 client.26428 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57677fe640 1 -- 172.21.15.33:0/982915188 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5555f2534da0 con 0x5555f25dcc70 2024-09-10T21:12:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57677fe640 10 client.26428 handle_command_reply: tid=0 2024-09-10T21:12:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 client.26428 shutdown 2024-09-10T21:12:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 msgr2=0x5555f25cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25cc9e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5778004560 tx=0x7f5778004730 comp rx=0 tx=0).stop 2024-09-10T21:12:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 shutdown_connections 2024-09-10T21:12:36.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f25dcc70 0x5555f25df0f0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 --2- 172.21.15.33:0/982915188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25cc9e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 >> 172.21.15.33:0/982915188 conn(0x5555f25ba2b0 msgr2=0x5555f25bfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.265+0000 7f57677fe640 0 client.26428 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.266+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 shutdown_connections 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.266+0000 7f57908c6740 1 -- 172.21.15.33:0/982915188 wait complete. 2024-09-10T21:12:36.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.266+0000 7f57908c6740 20 client.26428 trim_cache size 0 max 16384 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 Processor -- start 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 -- start start 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25d8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57881ac170 con 0x5555f25bcda0 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57881adcb0 con 0x5555f25cd140 2024-09-10T21:12:36.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57908c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57881ac840 con 0x5555f25bc480 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57877fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53806/0 (socket says 172.21.15.33:53806) 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f57877fe640 1 -- 172.21.15.33:0/384978252 learned_addr learned my addr 172.21.15.33:0/384978252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.284+0000 7f5787fff640 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25d8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f578ca8e640 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f57877fe640 1 -- 172.21.15.33:0/384978252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 msgr2=0x5555f25d8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f57877fe640 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25d8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f57877fe640 1 -- 172.21.15.33:0/384978252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 msgr2=0x5555f25c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f57877fe640 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f57877fe640 1 -- 172.21.15.33:0/384978252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555f25ca760 con 0x5555f25bcda0 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f57877fe640 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c86b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f577c0ac020 tx=0x7f577c0e3530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25964a80c5c78d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f576dffb640 1 -- 172.21.15.33:0/384978252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c01c040 con 0x5555f25bcda0 2024-09-10T21:12:36.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f576dffb640 1 -- 172.21.15.33:0/384978252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c0eb020 con 0x5555f25bcda0 2024-09-10T21:12:36.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.285+0000 7f576dffb640 1 -- 172.21.15.33:0/384978252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0b58b0 con 0x5555f25bcda0 2024-09-10T21:12:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 -- 172.21.15.33:0/384978252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 msgr2=0x5555f25c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c86b0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f577c0ac020 tx=0x7f577c0e3530 comp rx=0 tx=0).stop 2024-09-10T21:12:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 -- 172.21.15.33:0/384978252 shutdown_connections 2024-09-10T21:12:36.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25cd140 0x5555f25c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25bcda0 0x5555f25c86b0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 --2- 172.21.15.33:0/384978252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bc480 0x5555f25d8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 -- 172.21.15.33:0/384978252 >> 172.21.15.33:0/384978252 conn(0x5555f25ba2b0 msgr2=0x5555f25ceee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 -- 172.21.15.33:0/384978252 shutdown_connections 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 -- 172.21.15.33:0/384978252 wait complete. 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.286+0000 7f57908c6740 1 -- start start 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 1 Processor -- start 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bc480 0x5555f25d5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bcda0 0x5555f25d5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25cd140 0x5555f25d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57881ad5e0 con 0x5555f25cd140 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57881acf10 con 0x5555f25bc480 2024-09-10T21:12:36.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57881aa130 con 0x5555f25bcda0 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57908c6740 10 client.0 _ll_register_callbacks cb 0x7f5778009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f578ca8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25cd140 0x5555f25d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bc480 0x5555f25d5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f57877fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bcda0 0x5555f25d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bc480 0x5555f25d5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56740/0 (socket says 172.21.15.33:56740) 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 -- 172.21.15.33:0/534806480 learned_addr learned my addr 172.21.15.33:0/534806480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f578ca8e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25cd140 0x5555f25d2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53816/0 (socket says 172.21.15.33:53816) 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 -- 172.21.15.33:0/534806480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bcda0 msgr2=0x5555f25d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bcda0 0x5555f25d5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 -- 172.21.15.33:0/534806480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25cd140 msgr2=0x5555f25d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25cd140 0x5555f25d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.287+0000 7f5787fff640 1 -- 172.21.15.33:0/534806480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555f25d27b0 con 0x5555f25bc480 2024-09-10T21:12:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f5787fff640 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bc480 0x5555f25d5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f577801ef20 tx=0x7f577801dfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd0008ed66979055 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57677fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57677fe640 1 -- 172.21.15.33:0/534806480 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778042060 con 0x5555f25bc480 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57677fe640 1 -- 172.21.15.33:0/534806480 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f577801e9e0 con 0x5555f25bc480 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577c0eb020 con 0x5555f25bc480 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57677fe640 1 -- 172.21.15.33:0/534806480 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f5778037070 con 0x5555f25bc480 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57908c6740 10 client.26492 fetch_fsmap learned FSMap version 58 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5555f264a6a0 con 0x5555f25bc480 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.288+0000 7f57677fe640 1 -- 172.21.15.33:0/534806480 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c0eb020 con 0x5555f25bc480 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57677fe640 1 -- 172.21.15.33:0/534806480 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f5778031040 con 0x5555f25bc480 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57908c6740 10 client.26492 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57908c6740 10 client.26492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57908c6740 20 client.26492 populate_metadata read hostname 'smithi033' 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57908c6740 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f264ae00 0x5555f264d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:12:36.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57908c6740 4 client.26492 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:12:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5555f2530850 con 0x5555f264ae00 2024-09-10T21:12:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57877fe640 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f264ae00 0x5555f264d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:12:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57877fe640 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f264ae00 0x5555f264d220 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abaecae2925b4a44 server_cookie=ff473ffd405b0fc4 in_seq=0 out_seq=0 2024-09-10T21:12:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57677fe640 10 client.26492 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57677fe640 1 -- 172.21.15.33:0/534806480 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x5555f2530850 con 0x5555f264ae00 2024-09-10T21:12:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57677fe640 10 client.26492 handle_command_reply: tid=0 2024-09-10T21:12:36.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.289+0000 7f57908c6740 1 client.26492 shutdown 2024-09-10T21:12:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bc480 msgr2=0x5555f25d5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bc480 0x5555f25d5230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f577801ef20 tx=0x7f577801dfd0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 shutdown_connections 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5555f264ae00 0x5555f264d220 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5555f25cd140 0x5555f25d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555f25bcda0 0x5555f25d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57677fe640 0 client.26492 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 --2- 172.21.15.33:0/534806480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5555f25bc480 0x5555f25d5230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 >> 172.21.15.33:0/534806480 conn(0x5555f25ba2b0 msgr2=0x5555f25c82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 shutdown_connections 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 1 -- 172.21.15.33:0/534806480 wait complete. 2024-09-10T21:12:36.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.290+0000 7f57908c6740 20 client.26492 trim_cache size 0 max 16384 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "e69efbf4-61ef-4ef6-be19-b7823dae502f", 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:12:36.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:12:36.303 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:12:36.307 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:12:36.309 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57700b52e0 msgr2=0x7f57700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57700b52e0 0x7f57700b77a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x5555f25bf600 tx=0x7f5778009260 comp rx=0 tx=0).stop 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57881076a0 msgr2=0x7f57881997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57881076a0 0x7f57881997b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f577c00e9f0 tx=0x7f577c00edf0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 shutdown_connections 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57700b52e0 0x7f57700b77a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5788108020 0x7f578819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57881076a0 0x7f57881997b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 --2- 172.21.15.33:0/417111241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788101c90 0x7f5788199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:12:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 >> 172.21.15.33:0/417111241 conn(0x7f57880fd700 msgr2=0x7f578806ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:12:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 shutdown_connections 2024-09-10T21:12:36.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:12:36.291+0000 7f578e4f1640 1 -- 172.21.15.33:0/417111241 wait complete. 2024-09-10T21:12:36.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:36.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:36.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:12:36.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:36.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:36.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:36.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:12:36.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:36.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 579118 5.45 MB/sec execute 337 sec latency 267.886 ms 2024-09-10T21:12:36.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:36.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:36.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:12:36.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:12:37.010 INFO:tasks.cephfs.filesystem:scrub status for tag:e69efbf4-61ef-4ef6-be19-b7823dae502f - {'path': '/', 'tag': 'e69efbf4-61ef-4ef6-be19-b7823dae502f', 'options': 'recursive,force'} 2024-09-10T21:12:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:37 smithi174 ceph-mon[35635]: pgmap v605: 97 pgs: 97 active+clean; 223 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 681 KiB/s wr, 83 op/s 2024-09-10T21:12:37.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:37 smithi033 ceph-mon[30860]: pgmap v605: 97 pgs: 97 active+clean; 223 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 681 KiB/s wr, 83 op/s 2024-09-10T21:12:37.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 579861 5.44 MB/sec execute 338 sec latency 628.284 ms 2024-09-10T21:12:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:37 smithi104 ceph-mon[35325]: pgmap v605: 97 pgs: 97 active+clean; 223 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 681 KiB/s wr, 83 op/s 2024-09-10T21:12:38.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582610 5.47 MB/sec execute 339 sec latency 13.260 ms 2024-09-10T21:12:39.641 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:39 smithi033 ceph-mon[30860]: pgmap v606: 97 pgs: 97 active+clean; 224 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 717 KiB/s wr, 128 op/s 2024-09-10T21:12:39.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:39 smithi174 ceph-mon[35635]: pgmap v606: 97 pgs: 97 active+clean; 224 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 717 KiB/s wr, 128 op/s 2024-09-10T21:12:39.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582610 5.46 MB/sec execute 340 sec latency 1010.234 ms 2024-09-10T21:12:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:39 smithi104 ceph-mon[35325]: pgmap v606: 97 pgs: 97 active+clean; 224 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 787 KiB/s rd, 717 KiB/s wr, 128 op/s 2024-09-10T21:12:40.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582610 5.44 MB/sec execute 341 sec latency 2010.355 ms 2024-09-10T21:12:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:41 smithi174 ceph-mon[35635]: pgmap v607: 97 pgs: 97 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 695 KiB/s wr, 116 op/s 2024-09-10T21:12:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:12:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:12:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:12:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:41 smithi033 ceph-mon[30860]: pgmap v607: 97 pgs: 97 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 695 KiB/s wr, 116 op/s 2024-09-10T21:12:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:12:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:12:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:12:41.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582610 5.43 MB/sec execute 342 sec latency 3010.480 ms 2024-09-10T21:12:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:41 smithi104 ceph-mon[35325]: pgmap v607: 97 pgs: 97 active+clean; 225 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 695 KiB/s wr, 116 op/s 2024-09-10T21:12:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:12:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:12:41.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:12:42.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:12:42.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:12:42.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:12:42.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:12:42.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582661 5.41 MB/sec execute 343 sec latency 3373.122 ms 2024-09-10T21:12:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:12:42.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:12:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:43 smithi174 ceph-mon[35635]: pgmap v608: 97 pgs: 97 active+clean; 226 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 787 KiB/s wr, 135 op/s 2024-09-10T21:12:43.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:43 smithi033 ceph-mon[30860]: pgmap v608: 97 pgs: 97 active+clean; 226 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 787 KiB/s wr, 135 op/s 2024-09-10T21:12:43.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582661 5.39 MB/sec execute 344 sec latency 1594.833 ms 2024-09-10T21:12:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:43 smithi104 ceph-mon[35325]: pgmap v608: 97 pgs: 97 active+clean; 226 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 787 KiB/s wr, 135 op/s 2024-09-10T21:12:44.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582661 5.38 MB/sec execute 345 sec latency 2594.960 ms 2024-09-10T21:12:45.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:45 smithi174 ceph-mon[35635]: pgmap v609: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-09-10T21:12:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:45 smithi033 ceph-mon[30860]: pgmap v609: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-09-10T21:12:45.743 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582661 5.36 MB/sec execute 346 sec latency 3595.103 ms 2024-09-10T21:12:45.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:45 smithi104 ceph-mon[35325]: pgmap v609: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 146 op/s 2024-09-10T21:12:46.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 582661 5.35 MB/sec execute 347 sec latency 4595.140 ms 2024-09-10T21:12:47.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:47 smithi174 ceph-mon[35635]: pgmap v610: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 947 KiB/s wr, 137 op/s 2024-09-10T21:12:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:47 smithi033 ceph-mon[30860]: pgmap v610: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 947 KiB/s wr, 137 op/s 2024-09-10T21:12:47.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 583429 5.34 MB/sec execute 348 sec latency 4957.858 ms 2024-09-10T21:12:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:47 smithi104 ceph-mon[35325]: pgmap v610: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1011 KiB/s rd, 947 KiB/s wr, 137 op/s 2024-09-10T21:12:48.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 586135 5.37 MB/sec execute 349 sec latency 10.565 ms 2024-09-10T21:12:49.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:49 smithi033 ceph-mon[30860]: pgmap v611: 97 pgs: 97 active+clean; 229 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-09-10T21:12:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:49 smithi174 ceph-mon[35635]: pgmap v611: 97 pgs: 97 active+clean; 229 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-09-10T21:12:49.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 586212 5.36 MB/sec execute 350 sec latency 921.952 ms 2024-09-10T21:12:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:49 smithi104 ceph-mon[35325]: pgmap v611: 97 pgs: 97 active+clean; 229 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.0 MiB/s wr, 135 op/s 2024-09-10T21:12:50.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 586212 5.34 MB/sec execute 351 sec latency 1922.071 ms 2024-09-10T21:12:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:51 smithi174 ceph-mon[35635]: pgmap v612: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 874 KiB/s wr, 91 op/s 2024-09-10T21:12:51.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:51 smithi033 ceph-mon[30860]: pgmap v612: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 874 KiB/s wr, 91 op/s 2024-09-10T21:12:51.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 586212 5.33 MB/sec execute 352 sec latency 2922.169 ms 2024-09-10T21:12:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:51 smithi104 ceph-mon[35325]: pgmap v612: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 874 KiB/s wr, 91 op/s 2024-09-10T21:12:52.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 586984 5.32 MB/sec execute 353 sec latency 3281.977 ms 2024-09-10T21:12:53.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:53 smithi174 ceph-mon[35635]: pgmap v613: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 859 KiB/s wr, 93 op/s 2024-09-10T21:12:53.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:53 smithi033 ceph-mon[30860]: pgmap v613: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 859 KiB/s wr, 93 op/s 2024-09-10T21:12:53.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589735 5.35 MB/sec execute 354 sec latency 30.477 ms 2024-09-10T21:12:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:53 smithi104 ceph-mon[35325]: pgmap v613: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 859 KiB/s wr, 93 op/s 2024-09-10T21:12:54.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589735 5.33 MB/sec execute 355 sec latency 1030.576 ms 2024-09-10T21:12:55.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:55 smithi174 ceph-mon[35635]: pgmap v614: 97 pgs: 97 active+clean; 227 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1018 KiB/s wr, 101 op/s 2024-09-10T21:12:55.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:55 smithi033 ceph-mon[30860]: pgmap v614: 97 pgs: 97 active+clean; 227 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1018 KiB/s wr, 101 op/s 2024-09-10T21:12:55.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589735 5.32 MB/sec execute 356 sec latency 2030.673 ms 2024-09-10T21:12:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:55 smithi104 ceph-mon[35325]: pgmap v614: 97 pgs: 97 active+clean; 227 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 626 KiB/s rd, 1018 KiB/s wr, 101 op/s 2024-09-10T21:12:56.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 589735 5.30 MB/sec execute 357 sec latency 3030.790 ms 2024-09-10T21:12:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:57 smithi174 ceph-mon[35635]: pgmap v615: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 741 KiB/s wr, 96 op/s 2024-09-10T21:12:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:57 smithi033 ceph-mon[30860]: pgmap v615: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 741 KiB/s wr, 96 op/s 2024-09-10T21:12:57.744 INFO:tasks.workunit.client.0.smithi033.stdout: 1 590428 5.29 MB/sec execute 358 sec latency 3389.002 ms 2024-09-10T21:12:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:57 smithi104 ceph-mon[35325]: pgmap v615: 97 pgs: 97 active+clean; 228 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 589 KiB/s rd, 741 KiB/s wr, 96 op/s 2024-09-10T21:12:58.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593252 5.33 MB/sec execute 359 sec latency 19.670 ms 2024-09-10T21:12:59.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:12:59 smithi174 ceph-mon[35635]: pgmap v616: 97 pgs: 97 active+clean; 230 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-09-10T21:12:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:12:59 smithi033 ceph-mon[30860]: pgmap v616: 97 pgs: 97 active+clean; 230 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-09-10T21:12:59.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593252 5.31 MB/sec execute 360 sec latency 1019.790 ms 2024-09-10T21:12:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:12:59 smithi104 ceph-mon[35325]: pgmap v616: 97 pgs: 97 active+clean; 230 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-09-10T21:13:00.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593252 5.30 MB/sec execute 361 sec latency 2019.902 ms 2024-09-10T21:13:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:01 smithi174 ceph-mon[35635]: pgmap v617: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 916 KiB/s wr, 131 op/s 2024-09-10T21:13:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:01 smithi033 ceph-mon[30860]: pgmap v617: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 916 KiB/s wr, 131 op/s 2024-09-10T21:13:01.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593252 5.28 MB/sec execute 362 sec latency 3020.013 ms 2024-09-10T21:13:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:01 smithi104 ceph-mon[35325]: pgmap v617: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 909 KiB/s rd, 916 KiB/s wr, 131 op/s 2024-09-10T21:13:02.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593303 5.27 MB/sec execute 363 sec latency 3378.718 ms 2024-09-10T21:13:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:03 smithi174 ceph-mon[35635]: pgmap v618: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 915 KiB/s wr, 121 op/s 2024-09-10T21:13:03.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:03 smithi033 ceph-mon[30860]: pgmap v618: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 915 KiB/s wr, 121 op/s 2024-09-10T21:13:03.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593303 5.26 MB/sec execute 364 sec latency 1595.840 ms 2024-09-10T21:13:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:03 smithi104 ceph-mon[35325]: pgmap v618: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 827 KiB/s rd, 915 KiB/s wr, 121 op/s 2024-09-10T21:13:04.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593303 5.24 MB/sec execute 365 sec latency 2595.916 ms 2024-09-10T21:13:05.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:05 smithi174 ceph-mon[35635]: pgmap v619: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 912 KiB/s wr, 127 op/s 2024-09-10T21:13:05.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:05 smithi033 ceph-mon[30860]: pgmap v619: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 912 KiB/s wr, 127 op/s 2024-09-10T21:13:05.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593303 5.23 MB/sec execute 366 sec latency 3596.011 ms 2024-09-10T21:13:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:05 smithi104 ceph-mon[35325]: pgmap v619: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 912 KiB/s wr, 127 op/s 2024-09-10T21:13:06.745 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593303 5.21 MB/sec execute 367 sec latency 4596.106 ms 2024-09-10T21:13:07.012 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:13:07.300 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:07.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:07 smithi174 ceph-mon[35635]: pgmap v620: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T21:13:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:07 smithi033 ceph-mon[30860]: pgmap v620: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T21:13:07.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 593955 5.20 MB/sec execute 368 sec latency 4952.623 ms 2024-09-10T21:13:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:07 smithi104 ceph-mon[35325]: pgmap v620: 97 pgs: 97 active+clean; 230 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 690 KiB/s rd, 661 KiB/s wr, 99 op/s 2024-09-10T21:13:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 -- 172.21.15.33:0/341931259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc0075de0 msgr2=0x7f4cc010eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 --2- 172.21.15.33:0/341931259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc0075de0 0x7f4cc010eeb0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009900 tx=0x7f4cb402a410 comp rx=0 tx=0).stop 2024-09-10T21:13:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 -- 172.21.15.33:0/341931259 shutdown_connections 2024-09-10T21:13:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 --2- 172.21.15.33:0/341931259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc010f3f0 0x7f4cc01117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 --2- 172.21.15.33:0/341931259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc0075de0 0x7f4cc010eeb0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 --2- 172.21.15.33:0/341931259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc00754c0 0x7f4cc00758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 -- 172.21.15.33:0/341931259 >> 172.21.15.33:0/341931259 conn(0x7f4cc00fe830 msgr2=0x7f4cc0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 -- 172.21.15.33:0/341931259 shutdown_connections 2024-09-10T21:13:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.894+0000 7f4cc526f640 1 -- 172.21.15.33:0/341931259 wait complete. 2024-09-10T21:13:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.895+0000 7f4cc526f640 1 Processor -- start 2024-09-10T21:13:07.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.895+0000 7f4cc526f640 1 -- start start 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cc526f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc00754c0 0x7f4cc019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cc526f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc0075de0 0x7f4cc019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cc526f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc010f3f0 0x7f4cc01a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cc526f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0114fe0 con 0x7f4cc00754c0 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cc526f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0114e60 con 0x7f4cc0075de0 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cc526f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc0115160 con 0x7f4cc010f3f0 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc0075de0 0x7f4cc019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc010f3f0 0x7f4cc01a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc0075de0 0x7f4cc019ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:48266/0 (socket says 172.21.15.33:48266) 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc010f3f0 0x7f4cc01a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47612/0 (socket says 172.21.15.33:47612) 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 -- 172.21.15.33:0/649354328 learned_addr learned my addr 172.21.15.33:0/649354328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc00754c0 0x7f4cc019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 -- 172.21.15.33:0/649354328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc010f3f0 msgr2=0x7f4cc01a0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc010f3f0 0x7f4cc01a0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 -- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc00754c0 msgr2=0x7f4cc019b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc00754c0 0x7f4cc019b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbe7fc640 1 -- 172.21.15.33:0/649354328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc01a15a0 con 0x7f4cc0075de0 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc00754c0 0x7f4cc019b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.896+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc010f3f0 0x7f4cc01a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.897+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc0075de0 0x7f4cc019ba30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4005940 tx=0x7f4cb40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.897+0000 7f4ca7fff640 1 -- 172.21.15.33:0/649354328 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb403f030 con 0x7f4cc0075de0 2024-09-10T21:13:07.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.897+0000 7f4ca7fff640 1 -- 172.21.15.33:0/649354328 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb40044c0 con 0x7f4cc0075de0 2024-09-10T21:13:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.897+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc01a1830 con 0x7f4cc0075de0 2024-09-10T21:13:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.897+0000 7f4ca7fff640 1 -- 172.21.15.33:0/649354328 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4035070 con 0x7f4cc0075de0 2024-09-10T21:13:07.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.897+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc01a7f50 con 0x7f4cc0075de0 2024-09-10T21:13:07.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 Processor -- start 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 -- start start 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b9c0 0x55d40dd17410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 0x55d40dd19d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d40dc81e50 con 0x55d40dd0b0a0 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d40dc71280 con 0x55d40dd0b9c0 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.900+0000 7f4cc7644740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d40dc75090 con 0x55d40dd17950 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52368/0 (socket says 172.21.15.33:52368) 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 -- 172.21.15.33:0/1174498741 learned_addr learned my addr 172.21.15.33:0/1174498741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:07.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b9c0 0x55d40dd17410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 -- 172.21.15.33:0/1174498741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 msgr2=0x55d40dd19d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 0x55d40dd19d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 -- 172.21.15.33:0/1174498741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b9c0 msgr2=0x55d40dd17410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b9c0 0x55d40dd17410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 -- 172.21.15.33:0/1174498741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40d9cf650 con 0x55d40dd0b0a0 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b9c0 0x55d40dd17410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0b480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80117b0 tx=0x7f4cb8011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747d21ce347a30ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4ca7fff640 1 -- 172.21.15.33:0/649354328 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4cb403b9d0 con 0x7f4cc0075de0 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.901+0000 7f4ca57fa640 1 -- 172.21.15.33:0/1174498741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8004d20 con 0x55d40dd0b0a0 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.902+0000 7f4ca57fa640 1 -- 172.21.15.33:0/1174498741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb8005020 con 0x55d40dd0b0a0 2024-09-10T21:13:07.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.902+0000 7f4ca57fa640 1 -- 172.21.15.33:0/1174498741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8019690 con 0x55d40dd0b0a0 2024-09-10T21:13:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.902+0000 7f4ca7fff640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ca80b5360 0x7f4ca80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.902+0000 7f4cc7644740 1 -- 172.21.15.33:0/1174498741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 msgr2=0x55d40dd0b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.902+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0b480 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80117b0 tx=0x7f4cb8011c70 comp rx=0 tx=0).stop 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.902+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ca80b5360 0x7f4ca80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4ca7fff640 1 -- 172.21.15.33:0/649354328 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cb4106270 con 0x7f4cc0075de0 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cc7644740 1 -- 172.21.15.33:0/1174498741 shutdown_connections 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 0x55d40dd19d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b9c0 0x55d40dd17410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1174498741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0b480 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cc7644740 1 -- 172.21.15.33:0/1174498741 >> 172.21.15.33:0/1174498741 conn(0x55d40dd08ed0 msgr2=0x55d40dd0a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ca80b5360 0x7f4ca80b7820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55d40dd09db0 tx=0x7f4cb0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cc7644740 1 -- 172.21.15.33:0/1174498741 shutdown_connections 2024-09-10T21:13:07.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.903+0000 7f4cc7644740 1 -- 172.21.15.33:0/1174498741 wait complete. 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 -- start start 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 Processor -- start 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd172e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd14730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd14e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d40dad2700 con 0x55d40dd0b0a0 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d40dc0c110 con 0x55d40dd17950 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d40dd1a5d0 con 0x55d40dd0b9c0 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cc7644740 10 client.0 _ll_register_callbacks cb 0x55d40dc25990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cbeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd172e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cbeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd172e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52382/0 (socket says 172.21.15.33:52382) 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1692424768 learned_addr learned my addr 172.21.15.33:0/1692424768 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:07.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd14730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.904+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd14e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1692424768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 msgr2=0x55d40dd14730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd14730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 msgr2=0x55d40dd14e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd14e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1692424768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40dc39c40 con 0x55d40dd0b0a0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd172e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0011a20 tx=0x7f4cb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5465bf22763944a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4ca2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/1692424768 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0035060 con 0x55d40dd0b0a0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.905+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/1692424768 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cb0018030 con 0x55d40dd0b0a0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.906+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb8005020 con 0x55d40dd0b0a0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.906+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/1692424768 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4cb000f950 con 0x55d40dd0b0a0 2024-09-10T21:13:07.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.906+0000 7f4cc7644740 10 client.17040 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.906+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d40dd26060 con 0x55d40dd0b0a0 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.906+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/1692424768 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb8005020 con 0x55d40dd0b0a0 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.906+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/1692424768 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4cb001d3f0 con 0x55d40dd0b0a0 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cc7644740 10 client.17040 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cc7644740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cc7644740 20 client.17040 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd275b0 0x55d40dd299a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cc7644740 4 client.17040 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d40dc83da0 con 0x55d40dd275b0 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd275b0 0x55d40dd299a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4ca2ff5640 10 client.17040 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:07.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.907+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd275b0 0x55d40dd299a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d85937d23169d2f server_cookie=25700723a609e476 in_seq=0 out_seq=0 2024-09-10T21:13:07.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.908+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/1692424768 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d40dc83da0 con 0x55d40dd275b0 2024-09-10T21:13:07.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.908+0000 7f4ca2ff5640 10 client.17040 handle_command_reply: tid=0 2024-09-10T21:13:07.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.908+0000 7f4cc7644740 1 client.17040 shutdown 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.908+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 msgr2=0x55d40dd172e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.908+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd172e0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0011a20 tx=0x7f4cb0004730 comp rx=0 tx=0).stop 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 shutdown_connections 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd275b0 0x55d40dd299a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4ca2ff5640 0 client.17040 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd14e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd14730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1692424768 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd172e0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 >> 172.21.15.33:0/1692424768 conn(0x55d40dd08ed0 msgr2=0x55d40dd0ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 shutdown_connections 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 1 -- 172.21.15.33:0/1692424768 wait complete. 2024-09-10T21:13:07.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.909+0000 7f4cc7644740 20 client.17040 trim_cache size 0 max 16384 2024-09-10T21:13:07.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.928+0000 7f4cc7644740 1 Processor -- start 2024-09-10T21:13:07.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.928+0000 7f4cc7644740 1 -- start start 2024-09-10T21:13:07.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.928+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b9c0 0x55d40dd23860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 0x55d40dd03e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cc7644740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01ac260 con 0x55d40dd0b9c0 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cc7644740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01adda0 con 0x55d40dd0b0a0 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cc7644740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01ac930 con 0x55d40dd17950 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:48298/0 (socket says 172.21.15.33:48298) 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1774373632 learned_addr learned my addr 172.21.15.33:0/1774373632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b9c0 0x55d40dd23860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 0x55d40dd03e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1774373632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 msgr2=0x55d40dd03e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:07.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 0x55d40dd03e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1774373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b9c0 msgr2=0x55d40dd23860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b9c0 0x55d40dd23860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 -- 172.21.15.33:0/1774373632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d40dd13690 con 0x55d40dd0b0a0 2024-09-10T21:13:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0a6a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4cb001d7c0 tx=0x7f4cb00117e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f98305902bc286f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.929+0000 7f4ca4ff9640 1 -- 172.21.15.33:0/1774373632 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb003f080 con 0x55d40dd0b0a0 2024-09-10T21:13:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4ca4ff9640 1 -- 172.21.15.33:0/1774373632 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb000fd80 con 0x55d40dd0b0a0 2024-09-10T21:13:07.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4ca4ff9640 1 -- 172.21.15.33:0/1774373632 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb003aaf0 con 0x55d40dd0b0a0 2024-09-10T21:13:07.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 -- 172.21.15.33:0/1774373632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b0a0 msgr2=0x55d40dd0a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0a6a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4cb001d7c0 tx=0x7f4cb00117e0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 -- 172.21.15.33:0/1774373632 shutdown_connections 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd17950 0x55d40dd03e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b9c0 0x55d40dd23860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 --2- 172.21.15.33:0/1774373632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd0b0a0 0x55d40dd0a6a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 -- 172.21.15.33:0/1774373632 >> 172.21.15.33:0/1774373632 conn(0x55d40dd08ed0 msgr2=0x55d40dd09f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 -- 172.21.15.33:0/1774373632 shutdown_connections 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.930+0000 7f4cc7644740 1 -- 172.21.15.33:0/1774373632 wait complete. 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 -- start start 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 Processor -- start 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd20000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd1b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd1bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01ad6d0 con 0x55d40dd0b0a0 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01ad000 con 0x55d40dd17950 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc01aa220 con 0x55d40dd0b9c0 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cc7644740 10 client.0 _ll_register_callbacks cb 0x7f4cb0009bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cbeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd20000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cbeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd20000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52400/0 (socket says 172.21.15.33:52400) 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cbeffd640 1 -- 172.21.15.33:0/45211104 learned_addr learned my addr 172.21.15.33:0/45211104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd1b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.931+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd1bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cbeffd640 1 -- 172.21.15.33:0/45211104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 msgr2=0x55d40dd1b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd1b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cbeffd640 1 -- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 msgr2=0x55d40dd1bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd1bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cbeffd640 1 -- 172.21.15.33:0/45211104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d40dd15170 con 0x55d40dd0b0a0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cbf7fe640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd1bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cbeffd640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd20000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0011970 tx=0x7f4cb001dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49196604208cdabc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4ca2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/45211104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb0048020 con 0x55d40dd0b0a0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/45211104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cb0005450 con 0x55d40dd0b0a0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb000fd80 con 0x55d40dd0b0a0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.932+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/45211104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4cb0035070 con 0x55d40dd0b0a0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 10 client.17046 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d40dd95170 con 0x55d40dd0b0a0 2024-09-10T21:13:07.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/45211104 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb000f7a0 con 0x55d40dd0b0a0 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/45211104 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4cb00446d0 con 0x55d40dd0b0a0 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 10 client.17046 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 20 client.17046 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd95830 0x55d40dd97c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 4 client.17046 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f4cb000fd80 con 0x55d40dd95830 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.933+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd95830 0x55d40dd97c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cbe7fc640 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd95830 0x55d40dd97c50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49e7f4ff8f6ce2c3 server_cookie=a823d1feeed2ea5f in_seq=0 out_seq=0 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4ca2ff5640 10 client.17046 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4ca2ff5640 1 -- 172.21.15.33:0/45211104 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x7f4cb000fd80 con 0x55d40dd95830 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4ca2ff5640 10 client.17046 handle_command_reply: tid=0 2024-09-10T21:13:07.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 client.17046 shutdown 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 msgr2=0x55d40dd20000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd20000 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0011970 tx=0x7f4cb001dc40 comp rx=0 tx=0).stop 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 shutdown_connections 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d40dd95830 0x55d40dd97c50 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d40dd17950 0x55d40dd1bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d40dd0b9c0 0x55d40dd1b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4ca2ff5640 0 client.17046 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 --2- 172.21.15.33:0/45211104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d40dd0b0a0 0x55d40dd20000 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.934+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 >> 172.21.15.33:0/45211104 conn(0x55d40dd08ed0 msgr2=0x55d40dd15d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 shutdown_connections 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc7644740 1 -- 172.21.15.33:0/45211104 wait complete. 2024-09-10T21:13:07.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc7644740 20 client.17046 trim_cache size 0 max 16384 2024-09-10T21:13:07.944 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:13:07.944 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:13:07.944 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:13:07.944 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:13:07.944 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:13:07.944 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:13:07.944 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.945 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.946 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.947 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.948 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:13:07.949 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:07.951 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:07.952 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:07.952 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:07.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:13:07.952 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:13:07.952 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:13:07.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ca80b5360 msgr2=0x7f4ca80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ca80b5360 0x7f4ca80b7820 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55d40dd09db0 tx=0x7f4cb0005f90 comp rx=0 tx=0).stop 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc0075de0 msgr2=0x7f4cc019ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc0075de0 0x7f4cc019ba30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4005940 tx=0x7f4cb40040c0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 shutdown_connections 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ca80b5360 0x7f4ca80b7820 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4cc010f3f0 0x7f4cc01a0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4cc0075de0 0x7f4cc019ba30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 --2- 172.21.15.33:0/649354328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cc00754c0 0x7f4cc019b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 >> 172.21.15.33:0/649354328 conn(0x7f4cc00fe830 msgr2=0x7f4cc00ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 shutdown_connections 2024-09-10T21:13:07.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:07.935+0000 7f4cc526f640 1 -- 172.21.15.33:0/649354328 wait complete. 2024-09-10T21:13:08.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 596798 5.24 MB/sec execute 369 sec latency 14.521 ms 2024-09-10T21:13:08.878 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:13:08.878 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:13:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:09.522 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:09 smithi033 ceph-mon[30860]: pgmap v621: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 780 KiB/s wr, 102 op/s 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 -- 172.21.15.33:0/185487084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 msgr2=0x7efddc107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 --2- 172.21.15.33:0/185487084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc107ae0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7efdc4009900 tx=0x7efdc402a410 comp rx=0 tx=0).stop 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 -- 172.21.15.33:0/185487084 shutdown_connections 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 --2- 172.21.15.33:0/185487084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efddc108020 0x7efddc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 --2- 172.21.15.33:0/185487084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc107ae0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 --2- 172.21.15.33:0/185487084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efddc101c90 0x7efddc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 -- 172.21.15.33:0/185487084 >> 172.21.15.33:0/185487084 conn(0x7efddc0fd700 msgr2=0x7efddc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 -- 172.21.15.33:0/185487084 shutdown_connections 2024-09-10T21:13:09.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.602+0000 7efde0b98640 1 -- 172.21.15.33:0/185487084 wait complete. 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 Processor -- start 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 -- start start 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efddc101c90 0x7efddc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efddc108020 0x7efddc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efddc111ce0 con 0x7efddc1076a0 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efddc111b60 con 0x7efddc108020 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efde0b98640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efddc111e60 con 0x7efddc101c90 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52814/0 (socket says 172.21.15.33:52814) 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 -- 172.21.15.33:0/4274881038 learned_addr learned my addr 172.21.15.33:0/4274881038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:09.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efddad9d640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efddc108020 0x7efddc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdda59c640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efddc101c90 0x7efddc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 -- 172.21.15.33:0/4274881038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efddc101c90 msgr2=0x7efddc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efddc101c90 0x7efddc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 -- 172.21.15.33:0/4274881038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efddc108020 msgr2=0x7efddc19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efddc108020 0x7efddc19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.603+0000 7efdd3fff640 1 -- 172.21.15.33:0/4274881038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc19f400 con 0x7efddc1076a0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.604+0000 7efddad9d640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efddc108020 0x7efddc19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.604+0000 7efdd3fff640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc199890 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7efdc4005940 tx=0x7efdc40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.604+0000 7efdd37fe640 1 -- 172.21.15.33:0/4274881038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc403f030 con 0x7efddc1076a0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.604+0000 7efdd37fe640 1 -- 172.21.15.33:0/4274881038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdc4003cb0 con 0x7efddc1076a0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.604+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc19f690 con 0x7efddc1076a0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.604+0000 7efdd37fe640 1 -- 172.21.15.33:0/4274881038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc4035070 con 0x7efddc1076a0 2024-09-10T21:13:09.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.604+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc1a5db0 con 0x7efddc1076a0 2024-09-10T21:13:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.605+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efddc102070 con 0x7efddc1076a0 2024-09-10T21:13:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.607+0000 7efdd37fe640 1 -- 172.21.15.33:0/4274881038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efdc403bbf0 con 0x7efddc1076a0 2024-09-10T21:13:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.607+0000 7efdd37fe640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efdc80b52e0 0x7efdc80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.607+0000 7efdd37fe640 1 -- 172.21.15.33:0/4274881038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efdc4106a80 con 0x7efddc1076a0 2024-09-10T21:13:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.607+0000 7efdda59c640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efdc80b52e0 0x7efdc80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.608+0000 7efdda59c640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efdc80b52e0 0x7efdc80b77a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efdcc005fd0 tx=0x7efdcc005e20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:09.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.610+0000 7efdd37fe640 1 -- 172.21.15.33:0/4274881038 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7efdc40cd940 con 0x7efddc1076a0 2024-09-10T21:13:09.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 596868 5.22 MB/sec execute 370 sec latency 945.180 ms 2024-09-10T21:13:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:09 smithi104 ceph-mon[35325]: pgmap v621: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 780 KiB/s wr, 102 op/s 2024-09-10T21:13:09.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.834+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efddc19a480 con 0x7efddc1076a0 2024-09-10T21:13:09.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.835+0000 7efdd37fe640 1 -- 172.21.15.33:0/4274881038 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11973 (secure 0 0 0) 0x7efddc19a480 con 0x7efddc1076a0 2024-09-10T21:13:09.839 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:13:09.839 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"btime":"2024-09-10T21:11:31:688744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:11:30.989029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:13:09.840 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.837+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efdc80b52e0 msgr2=0x7efdc80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.837+0000 7efde0b98640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efdc80b52e0 0x7efdc80b77a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7efdcc005fd0 tx=0x7efdcc005e20 comp rx=0 tx=0).stop 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.837+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 msgr2=0x7efddc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.837+0000 7efde0b98640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc199890 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7efdc4005940 tx=0x7efdc40042f0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 shutdown_connections 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efdc80b52e0 0x7efdc80b77a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efddc108020 0x7efddc19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efddc1076a0 0x7efddc199890 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 --2- 172.21.15.33:0/4274881038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efddc101c90 0x7efddc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 >> 172.21.15.33:0/4274881038 conn(0x7efddc0fd700 msgr2=0x7efddc06adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 shutdown_connections 2024-09-10T21:13:09.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:09.838+0000 7efde0b98640 1 -- 172.21.15.33:0/4274881038 wait complete. 2024-09-10T21:13:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:09 smithi174 ceph-mon[35635]: pgmap v621: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 597 KiB/s rd, 780 KiB/s wr, 102 op/s 2024-09-10T21:13:10.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:13:10.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:10 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4274881038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:13:10.739 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:10.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 596868 5.21 MB/sec execute 371 sec latency 1945.210 ms 2024-09-10T21:13:10.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:10 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4274881038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:13:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:10 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4274881038' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:13:11.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 -- 172.21.15.33:0/3208839201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 msgr2=0x7efc2c10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 --2- 172.21.15.33:0/3208839201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c10e860 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7efc24009ca0 tx=0x7efc2402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 -- 172.21.15.33:0/3208839201 shutdown_connections 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 --2- 172.21.15.33:0/3208839201 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c10e860 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 --2- 172.21.15.33:0/3208839201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc2c1076e0 0x7efc2c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 --2- 172.21.15.33:0/3208839201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc2c101cd0 0x7efc2c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 -- 172.21.15.33:0/3208839201 >> 172.21.15.33:0/3208839201 conn(0x7efc2c0fd720 msgr2=0x7efc2c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 -- 172.21.15.33:0/3208839201 shutdown_connections 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.227+0000 7efc321d8640 1 -- 172.21.15.33:0/3208839201 wait complete. 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.228+0000 7efc321d8640 1 Processor -- start 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.228+0000 7efc321d8640 1 -- start start 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.228+0000 7efc321d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc2c101cd0 0x7efc2c0fdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.228+0000 7efc321d8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc2c1076e0 0x7efc2c0fe160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.228+0000 7efc321d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c0fe6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.228+0000 7efc321d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c111d20 con 0x7efc2c108060 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc321d8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c111ba0 con 0x7efc2c1076e0 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc321d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c111ea0 con 0x7efc2c101cd0 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c0fe6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c0fe6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52822/0 (socket says 172.21.15.33:52822) 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 -- 172.21.15.33:0/68718293 learned_addr learned my addr 172.21.15.33:0/68718293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc2c101cd0 0x7efc2c0fdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2affd640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc2c1076e0 0x7efc2c0fe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 -- 172.21.15.33:0/68718293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc2c101cd0 msgr2=0x7efc2c0fdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc2c101cd0 0x7efc2c0fdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 -- 172.21.15.33:0/68718293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc2c1076e0 msgr2=0x7efc2c0fe160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc2c1076e0 0x7efc2c0fe160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 -- 172.21.15.33:0/68718293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2c1a7660 con 0x7efc2c108060 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc2bfff640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c0fe6a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efc24005cc0 tx=0x7efc2402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc28ff9640 1 -- 172.21.15.33:0/68718293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24003bf0 con 0x7efc2c108060 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2c1a78f0 con 0x7efc2c108060 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc28ff9640 1 -- 172.21.15.33:0/68718293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc24003d90 con 0x7efc2c108060 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc28ff9640 1 -- 172.21.15.33:0/68718293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2402fbc0 con 0x7efc2c108060 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.229+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2c1a7e30 con 0x7efc2c108060 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 Processor -- start 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc28ff9640 1 -- 172.21.15.33:0/68718293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efc24004610 con 0x7efc2c108060 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 -- start start 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0ee74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0ef3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0ef5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5602a0e5dea0 con 0x5602a0ee70f0 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5602a0e4d2d0 con 0x5602a0ef39a0 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc345ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602a0e510e0 con 0x5602a0ee7a10 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0ee74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0ee74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52838/0 (socket says 172.21.15.33:52838) 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc2affd640 1 -- 172.21.15.33:0/3871446841 learned_addr learned my addr 172.21.15.33:0/3871446841 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.232+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0ef3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2affd640 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0ef5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2b7fe640 1 -- 172.21.15.33:0/3871446841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 msgr2=0x5602a0ef5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0ef5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2b7fe640 1 -- 172.21.15.33:0/3871446841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 msgr2=0x5602a0ee74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0ee74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc28ff9640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc140b52e0 0x7efc140b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2b7fe640 1 -- 172.21.15.33:0/3871446841 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a0bab650 con 0x5602a0ee7a10 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2affd640 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0ee74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2affd640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc140b52e0 0x7efc140b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc28ff9640 1 -- 172.21.15.33:0/68718293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efc24107020 con 0x7efc2c108060 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0ef3460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00bbf0 tx=0x7efc1c0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6afb40f97ee22a9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc11ffb640 1 -- 172.21.15.33:0/3871446841 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c004030 con 0x5602a0ee7a10 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc11ffb640 1 -- 172.21.15.33:0/3871446841 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1c0035f0 con 0x5602a0ee7a10 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc11ffb640 1 -- 172.21.15.33:0/3871446841 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c011680 con 0x5602a0ee7a10 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.233+0000 7efc2affd640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc140b52e0 0x7efc140b77a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efc2000afd0 tx=0x7efc20015040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 -- 172.21.15.33:0/3871446841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 msgr2=0x5602a0ef3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0ef3460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00bbf0 tx=0x7efc1c0107a0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 -- 172.21.15.33:0/3871446841 shutdown_connections 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0ef5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0ef3460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 --2- 172.21.15.33:0/3871446841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0ee74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 -- 172.21.15.33:0/3871446841 >> 172.21.15.33:0/3871446841 conn(0x5602a0ee4f20 msgr2=0x5602a0ee60c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 -- 172.21.15.33:0/3871446841 shutdown_connections 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 -- 172.21.15.33:0/3871446841 wait complete. 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.234+0000 7efc345ad740 1 -- start start 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 1 Processor -- start 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ee70f0 0x5602a0ef31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ef39a0 0x5602a0ef0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5602a0cae700 con 0x5602a0ee7a10 2024-09-10T21:13:11.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5602a0de8160 con 0x5602a0ee70f0 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602a0ef6620 con 0x5602a0ef39a0 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc345ad740 10 client.0 _ll_register_callbacks cb 0x5602a0e019e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52842/0 (socket says 172.21.15.33:52842) 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 -- 172.21.15.33:0/2980562591 learned_addr learned my addr 172.21.15.33:0/2980562591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2bfff640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ef39a0 0x5602a0ef0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ee70f0 0x5602a0ef31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 -- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ef39a0 msgr2=0x5602a0ef0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ef39a0 0x5602a0ef0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 -- 172.21.15.33:0/2980562591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ee70f0 msgr2=0x5602a0ef31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ee70f0 0x5602a0ef31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2affd640 1 -- 172.21.15.33:0/2980562591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602a0e15c90 con 0x5602a0ee7a10 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.235+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ee70f0 0x5602a0ef31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:11.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc2affd640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef0630 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efc2000eee0 tx=0x7efc2000a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffcdac8530c0079e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc0affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc0affd640 1 -- 172.21.15.33:0/2980562591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2001f810 con 0x5602a0ee7a10 2024-09-10T21:13:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc0affd640 1 -- 172.21.15.33:0/2980562591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efc2001f9b0 con 0x5602a0ee7a10 2024-09-10T21:13:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c0035f0 con 0x5602a0ee7a10 2024-09-10T21:13:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc0affd640 1 -- 172.21.15.33:0/2980562591 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efc2000fe30 con 0x5602a0ee7a10 2024-09-10T21:13:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc345ad740 10 client.17064 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:11.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.236+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5602a0f01f40 con 0x5602a0ee7a10 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc0affd640 1 -- 172.21.15.33:0/2980562591 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1c0035f0 con 0x5602a0ee7a10 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc0affd640 1 -- 172.21.15.33:0/2980562591 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efc20003770 con 0x5602a0ee7a10 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc345ad740 10 client.17064 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc345ad740 10 client.17064 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc345ad740 20 client.17064 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc345ad740 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f03400 0x5602a0f05880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc345ad740 4 client.17064 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602a0e5fdf0 con 0x5602a0f03400 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.237+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f03400 0x5602a0f05880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.238+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f03400 0x5602a0f05880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=39d5a07108de062 server_cookie=b7e42f4aec2bd67f in_seq=0 out_seq=0 2024-09-10T21:13:11.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.238+0000 7efc0affd640 10 client.17064 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc0affd640 1 -- 172.21.15.33:0/2980562591 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5602a0e5fdf0 con 0x5602a0f03400 2024-09-10T21:13:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc0affd640 10 client.17064 handle_command_reply: tid=0 2024-09-10T21:13:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 client.17064 shutdown 2024-09-10T21:13:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 msgr2=0x5602a0ef0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef0630 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7efc2000eee0 tx=0x7efc2000a760 comp rx=0 tx=0).stop 2024-09-10T21:13:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 shutdown_connections 2024-09-10T21:13:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f03400 0x5602a0f05880 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ef39a0 0x5602a0ef0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef0630 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc0affd640 0 client.17064 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 --2- 172.21.15.33:0/2980562591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ee70f0 0x5602a0ef31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 >> 172.21.15.33:0/2980562591 conn(0x5602a0ee4f20 msgr2=0x5602a0eeaa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 shutdown_connections 2024-09-10T21:13:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 1 -- 172.21.15.33:0/2980562591 wait complete. 2024-09-10T21:13:11.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.239+0000 7efc345ad740 20 client.17064 trim_cache size 0 max 16384 2024-09-10T21:13:11.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 Processor -- start 2024-09-10T21:13:11.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 -- start start 2024-09-10T21:13:11.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0eeeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0eef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0eed840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b05e0 con 0x5602a0ee70f0 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b2120 con 0x5602a0ef39a0 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc345ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b0cb0 con 0x5602a0ee7a10 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc2b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0eeeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.252+0000 7efc2b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0eeeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52856/0 (socket says 172.21.15.33:52856) 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2b7fe640 1 -- 172.21.15.33:0/3352089284 learned_addr learned my addr 172.21.15.33:0/3352089284 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2affd640 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0eef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2bfff640 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0eed840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2b7fe640 1 -- 172.21.15.33:0/3352089284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 msgr2=0x5602a0eef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0eef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2b7fe640 1 -- 172.21.15.33:0/3352089284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 msgr2=0x5602a0eed840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0eed840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2b7fe640 1 -- 172.21.15.33:0/3352089284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a0ee59a0 con 0x5602a0ee70f0 2024-09-10T21:13:11.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2bfff640 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0eed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:11.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0eeeeb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005d00 tx=0x7efc1c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90561b05c8f709 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:11.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc117fa640 1 -- 172.21.15.33:0/3352089284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c005240 con 0x5602a0ee70f0 2024-09-10T21:13:11.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc117fa640 1 -- 172.21.15.33:0/3352089284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1c0053e0 con 0x5602a0ee70f0 2024-09-10T21:13:11.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.253+0000 7efc117fa640 1 -- 172.21.15.33:0/3352089284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c0266e0 con 0x5602a0ee70f0 2024-09-10T21:13:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 -- 172.21.15.33:0/3352089284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 msgr2=0x5602a0eeeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0eeeeb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7efc1c005d00 tx=0x7efc1c005c70 comp rx=0 tx=0).stop 2024-09-10T21:13:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 -- 172.21.15.33:0/3352089284 shutdown_connections 2024-09-10T21:13:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0eed840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee7a10 0x5602a0eef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 --2- 172.21.15.33:0/3352089284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee70f0 0x5602a0eeeeb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 -- 172.21.15.33:0/3352089284 >> 172.21.15.33:0/3352089284 conn(0x5602a0ee4f20 msgr2=0x5602a0ee5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 -- 172.21.15.33:0/3352089284 shutdown_connections 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 -- 172.21.15.33:0/3352089284 wait complete. 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.254+0000 7efc345ad740 1 -- start start 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 1 Processor -- start 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee70f0 0x5602a0ef8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0f69440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b1a50 con 0x5602a0ee7a10 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1b1380 con 0x5602a0ef39a0 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1ae5a0 con 0x5602a0ee70f0 2024-09-10T21:13:11.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc345ad740 10 client.0 _ll_register_callbacks cb 0x7efc2000eb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52870/0 (socket says 172.21.15.33:52870) 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 -- 172.21.15.33:0/3094345593 learned_addr learned my addr 172.21.15.33:0/3094345593 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee70f0 0x5602a0ef8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2bfff640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0f69440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 -- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee70f0 msgr2=0x5602a0ef8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee70f0 0x5602a0ef8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 -- 172.21.15.33:0/3094345593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 msgr2=0x5602a0f69440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0f69440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 -- 172.21.15.33:0/3094345593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602a0f69b50 con 0x5602a0ee7a10 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2bfff640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0f69440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc2affd640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef88d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efc20015010 tx=0x7efc200124d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4120fec535e9bd59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:11.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.255+0000 7efc0affd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc0affd640 1 -- 172.21.15.33:0/3094345593 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2003a070 con 0x5602a0ee7a10 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc0affd640 1 -- 172.21.15.33:0/3094345593 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efc20005450 con 0x5602a0ee7a10 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c0053e0 con 0x5602a0ee7a10 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc0affd640 1 -- 172.21.15.33:0/3094345593 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efc20005650 con 0x5602a0ee7a10 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 10 client.17070 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5602a0f70f30 con 0x5602a0ee7a10 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc0affd640 1 -- 172.21.15.33:0/3094345593 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc1c0053e0 con 0x5602a0ee7a10 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc0affd640 1 -- 172.21.15.33:0/3094345593 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efc2003d020 con 0x5602a0ee7a10 2024-09-10T21:13:11.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 10 client.17070 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 10 client.17070 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 20 client.17070 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f716d0 0x5602a0f73af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 4 client.17070 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.256+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5602a0e5b8a0 con 0x5602a0f716d0 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f716d0 0x5602a0f73af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc0affd640 10 client.17070 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc2b7fe640 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f716d0 0x5602a0f73af0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20db4b0bb6cb38d3 server_cookie=aca131fbe32f251c in_seq=0 out_seq=0 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc0affd640 1 -- 172.21.15.33:0/3094345593 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5602a0e5b8a0 con 0x5602a0f716d0 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc0affd640 10 client.17070 handle_command_reply: tid=0 2024-09-10T21:13:11.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 client.17070 shutdown 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 msgr2=0x5602a0ef88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef88d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7efc20015010 tx=0x7efc200124d0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 shutdown_connections 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5602a0f716d0 0x5602a0f73af0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5602a0ef39a0 0x5602a0f69440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc0affd640 0 client.17070 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5602a0ee7a10 0x5602a0ef88d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 --2- 172.21.15.33:0/3094345593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602a0ee70f0 0x5602a0ef8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.257+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 >> 172.21.15.33:0/3094345593 conn(0x5602a0ee4f20 msgr2=0x5602a0ef1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:11.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 shutdown_connections 2024-09-10T21:13:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc345ad740 1 -- 172.21.15.33:0/3094345593 wait complete. 2024-09-10T21:13:11.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc345ad740 20 client.17070 trim_cache size 0 max 16384 2024-09-10T21:13:11.269 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc140b52e0 msgr2=0x7efc140b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc140b52e0 0x7efc140b77a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7efc2000afd0 tx=0x7efc20015040 comp rx=0 tx=0).stop 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 msgr2=0x7efc2c0fe6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c0fe6a0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7efc24005cc0 tx=0x7efc2402f380 comp rx=0 tx=0).stop 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 shutdown_connections 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efc140b52e0 0x7efc140b77a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc2c108060 0x7efc2c0fe6a0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc2c1076e0 0x7efc2c0fe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 --2- 172.21.15.33:0/68718293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc2c101cd0 0x7efc2c0fdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 >> 172.21.15.33:0/68718293 conn(0x7efc2c0fd720 msgr2=0x7efc2c06b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:11.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 shutdown_connections 2024-09-10T21:13:11.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:11.258+0000 7efc321d8640 1 -- 172.21.15.33:0/68718293 wait complete. 2024-09-10T21:13:11.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:11 smithi033 ceph-mon[30860]: pgmap v622: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 679 KiB/s wr, 86 op/s 2024-09-10T21:13:11.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 596868 5.19 MB/sec execute 372 sec latency 2945.276 ms 2024-09-10T21:13:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:11 smithi104 ceph-mon[35325]: pgmap v622: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 679 KiB/s wr, 86 op/s 2024-09-10T21:13:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:11 smithi174 ceph-mon[35635]: pgmap v622: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 679 KiB/s wr, 86 op/s 2024-09-10T21:13:11.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:13:12.221 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.699+0000 7f2d71791640 1 -- 172.21.15.33:0/9821223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c101cd0 msgr2=0x7f2d6c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.699+0000 7f2d71791640 1 --2- 172.21.15.33:0/9821223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c101cd0 0x7f2d6c1020b0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009900 tx=0x7f2d5c02a410 comp rx=0 tx=0).stop 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.699+0000 7f2d71791640 1 -- 172.21.15.33:0/9821223 shutdown_connections 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.699+0000 7f2d71791640 1 --2- 172.21.15.33:0/9821223 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c108060 0x7f2d6c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.699+0000 7f2d71791640 1 --2- 172.21.15.33:0/9821223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c1076e0 0x7f2d6c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.699+0000 7f2d71791640 1 --2- 172.21.15.33:0/9821223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c101cd0 0x7f2d6c1020b0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.699+0000 7f2d71791640 1 -- 172.21.15.33:0/9821223 >> 172.21.15.33:0/9821223 conn(0x7f2d6c0fd740 msgr2=0x7f2d6c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.700+0000 7f2d71791640 1 -- 172.21.15.33:0/9821223 shutdown_connections 2024-09-10T21:13:12.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.700+0000 7f2d71791640 1 -- 172.21.15.33:0/9821223 wait complete. 2024-09-10T21:13:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.700+0000 7f2d71791640 1 Processor -- start 2024-09-10T21:13:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d71791640 1 -- start start 2024-09-10T21:13:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d71791640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c101cd0 0x7f2d6c0fdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d71791640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c1076e0 0x7f2d6c0fe180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d71791640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c108060 0x7f2d6c0fe6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d71791640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c111d20 con 0x7f2d6c108060 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d71791640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c111ba0 con 0x7f2d6c1076e0 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d71791640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c111ea0 con 0x7f2d6c101cd0 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c108060 0x7f2d6c0fe6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d6b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c108060 0x7f2d6c0fe6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52882/0 (socket says 172.21.15.33:52882) 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d6b7fe640 1 -- 172.21.15.33:0/1698208088 learned_addr learned my addr 172.21.15.33:0/1698208088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c1076e0 0x7f2d6c0fe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.701+0000 7f2d6affd640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c101cd0 0x7f2d6c0fdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/1698208088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c101cd0 msgr2=0x7f2d6c0fdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c101cd0 0x7f2d6c0fdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c108060 msgr2=0x7f2d6c0fe6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c108060 0x7f2d6c0fe6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/1698208088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6c1abc70 con 0x7f2d6c1076e0 2024-09-10T21:13:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6affd640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c101cd0 0x7f2d6c0fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c108060 0x7f2d6c0fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c1076e0 0x7f2d6c0fe180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9f0 tx=0x7f2d6000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d53fff640 1 -- 172.21.15.33:0/1698208088 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000ccc0 con 0x7f2d6c1076e0 2024-09-10T21:13:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d53fff640 1 -- 172.21.15.33:0/1698208088 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d60005020 con 0x7f2d6c1076e0 2024-09-10T21:13:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d6c1abe40 con 0x7f2d6c1076e0 2024-09-10T21:13:12.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d53fff640 1 -- 172.21.15.33:0/1698208088 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60004a00 con 0x7f2d6c1076e0 2024-09-10T21:13:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.702+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6c1ac2a0 con 0x7f2d6c1076e0 2024-09-10T21:13:12.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.705+0000 7f2d53fff640 1 -- 172.21.15.33:0/1698208088 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2d60004ba0 con 0x7f2d6c1076e0 2024-09-10T21:13:12.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.705+0000 7f2d73b66740 1 Processor -- start 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.705+0000 7f2d73b66740 1 -- start start 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430fe34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fef460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d73b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561430f59ea0 con 0x561430fe3a10 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d73b66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561430f492d0 con 0x561430fe30f0 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d73b66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561430f4d0e0 con 0x561430fef9a0 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fef460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fef460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52890/0 (socket says 172.21.15.33:52890) 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 -- 172.21.15.33:0/3021524382 learned_addr learned my addr 172.21.15.33:0/3021524382 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:12.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430fe34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d53fff640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d540b52e0 0x7f2d540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d540b52e0 0x7f2d540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 -- 172.21.15.33:0/3021524382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 msgr2=0x561430ff1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 -- 172.21.15.33:0/3021524382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 msgr2=0x561430fe34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430fe34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 -- 172.21.15.33:0/3021524382 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561430ca7650 con 0x561430fe3a10 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430fe34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d53fff640 1 -- 172.21.15.33:0/1698208088 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2d600e5bc0 con 0x7f2d6c1076e0 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.706+0000 7f2d6affd640 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fef460 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009900 tx=0x7f2d5c005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccf17fe6d25e2ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d517fa640 1 -- 172.21.15.33:0/3021524382 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c040070 con 0x561430fe3a10 2024-09-10T21:13:12.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d517fa640 1 -- 172.21.15.33:0/3021524382 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d5c0317a0 con 0x561430fe3a10 2024-09-10T21:13:12.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d517fa640 1 -- 172.21.15.33:0/3021524382 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c035040 con 0x561430fe3a10 2024-09-10T21:13:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d73b66740 1 -- 172.21.15.33:0/3021524382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 msgr2=0x561430fef460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d540b52e0 0x7f2d540b77a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009aa0 tx=0x7f2d64004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fef460 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009900 tx=0x7f2d5c005d90 comp rx=0 tx=0).stop 2024-09-10T21:13:12.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d73b66740 1 -- 172.21.15.33:0/3021524382 shutdown_connections 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fef460 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3021524382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430fe34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.707+0000 7f2d73b66740 1 -- 172.21.15.33:0/3021524382 >> 172.21.15.33:0/3021524382 conn(0x561430fe0f20 msgr2=0x561430fe20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 -- 172.21.15.33:0/3021524382 shutdown_connections 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 -- 172.21.15.33:0/3021524382 wait complete. 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 -- start start 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 Processor -- start 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fe30f0 0x561430fef290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fec6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fef9a0 0x561430fecd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561430daa700 con 0x561430fe3a10 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561430ee4160 con 0x561430fef9a0 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561430ff25c0 con 0x561430fe30f0 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fec6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fe30f0 0x561430fef290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fec6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52904/0 (socket says 172.21.15.33:52904) 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/3966786317 learned_addr learned my addr 172.21.15.33:0/3966786317 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.708+0000 7f2d73b66740 10 client.0 _ll_register_callbacks cb 0x561430efd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fef9a0 0x561430fecd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/3966786317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fe30f0 msgr2=0x561430fef290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fe30f0 0x561430fef290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/3966786317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fef9a0 msgr2=0x561430fecd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fef9a0 0x561430fecd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/3966786317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561430f11c90 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fec6e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2d600a8910 tx=0x7f2d600e39a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5c4ba6036523f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.709+0000 7f2d4eff5640 1 -- 172.21.15.33:0/3966786317 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60074e60 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d4eff5640 1 -- 172.21.15.33:0/3966786317 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2d600b5640 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d5c0317a0 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d4eff5640 1 -- 172.21.15.33:0/3966786317 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2d6001c070 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 10 client.17082 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561430ffdf90 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d4eff5640 1 -- 172.21.15.33:0/3966786317 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d5c0317a0 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d4eff5640 1 -- 172.21.15.33:0/3966786317 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f2d600cf0d0 con 0x561430fe3a10 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 10 client.17082 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 10 client.17082 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:13:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 20 client.17082 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x561430fff860 0x561431001cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.710+0000 7f2d73b66740 4 client.17082 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.711+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561430f5bdf0 con 0x561430fff860 2024-09-10T21:13:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.711+0000 7f2d6affd640 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x561430fff860 0x561431001cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.711+0000 7f2d6affd640 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x561430fff860 0x561431001cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a014bbb3620d8c22 server_cookie=8a1ab42395e9195 in_seq=0 out_seq=0 2024-09-10T21:13:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.711+0000 7f2d4eff5640 10 client.17082 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d4eff5640 1 -- 172.21.15.33:0/3966786317 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561430f5bdf0 con 0x561430fff860 2024-09-10T21:13:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d4eff5640 10 client.17082 handle_command_reply: tid=0 2024-09-10T21:13:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 client.17082 shutdown 2024-09-10T21:13:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 msgr2=0x561430fec6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fec6e0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f2d600a8910 tx=0x7f2d600e39a0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 shutdown_connections 2024-09-10T21:13:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fef9a0 0x561430fecd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430fec6e0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x561430fff860 0x561431001cf0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 --2- 172.21.15.33:0/3966786317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fe30f0 0x561430fef290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d4eff5640 0 client.17082 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 >> 172.21.15.33:0/3966786317 conn(0x561430fe0f20 msgr2=0x561430fe6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 shutdown_connections 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 1 -- 172.21.15.33:0/3966786317 wait complete. 2024-09-10T21:13:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.712+0000 7f2d73b66740 20 client.17082 trim_cache size 0 max 16384 2024-09-10T21:13:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 Processor -- start 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 -- start start 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe30f0 0x561430feaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe3a10 0x561430feb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430fe98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1b49f0 con 0x561430fe30f0 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1b6530 con 0x561430fe3a10 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.727+0000 7f2d73b66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1b50c0 con 0x561430fef9a0 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe3a10 0x561430feb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe3a10 0x561430feb4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:47372/0 (socket says 172.21.15.33:47372) 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/4013256634 learned_addr learned my addr 172.21.15.33:0/4013256634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430fe98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6affd640 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe30f0 0x561430feaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/4013256634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 msgr2=0x561430fe98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430fe98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/4013256634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe30f0 msgr2=0x561430feaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe30f0 0x561430feaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/4013256634 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561430fff620 con 0x561430fe3a10 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6affd640 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe30f0 0x561430feaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430fe98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.728+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe3a10 0x561430feb4a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9c0 tx=0x7f2d60004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=354acd46aeab657d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.729+0000 7f2d50ff9640 1 -- 172.21.15.33:0/4013256634 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600e5e40 con 0x561430fe3a10 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.729+0000 7f2d50ff9640 1 -- 172.21.15.33:0/4013256634 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d600abe50 con 0x561430fe3a10 2024-09-10T21:13:12.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.729+0000 7f2d50ff9640 1 -- 172.21.15.33:0/4013256634 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600ac340 con 0x561430fe3a10 2024-09-10T21:13:12.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 -- 172.21.15.33:0/4013256634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe3a10 msgr2=0x561430feb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe3a10 0x561430feb4a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9c0 tx=0x7f2d60004ec0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 -- 172.21.15.33:0/4013256634 shutdown_connections 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430fe98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe3a10 0x561430feb4a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 --2- 172.21.15.33:0/4013256634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe30f0 0x561430feaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 -- 172.21.15.33:0/4013256634 >> 172.21.15.33:0/4013256634 conn(0x561430fe0f20 msgr2=0x561430fe1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 -- 172.21.15.33:0/4013256634 shutdown_connections 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 -- 172.21.15.33:0/4013256634 wait complete. 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.730+0000 7f2d73b66740 1 -- start start 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d73b66740 1 Processor -- start 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430ff7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430ff7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d73b66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d6affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430ff7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d6affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430ff7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:47388/0 (socket says 172.21.15.33:47388) 2024-09-10T21:13:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430ff7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.731+0000 7f2d6a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430ff7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52924/0 (socket says 172.21.15.33:52924) 2024-09-10T21:13:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d73b66740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1b5e60 con 0x561430fe3a10 2024-09-10T21:13:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d73b66740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1b5790 con 0x561430fe30f0 2024-09-10T21:13:12.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d73b66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6c1b29b0 con 0x561430fef9a0 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d73b66740 10 client.0 _ll_register_callbacks cb 0x7f2d600ac1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6affd640 1 -- 172.21.15.33:0/1998732949 learned_addr learned my addr 172.21.15.33:0/1998732949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6b7fe640 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 msgr2=0x561430ff2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/1998732949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 msgr2=0x561430ff7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430ff7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6a7fc640 1 -- 172.21.15.33:0/1998732949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561430ff2f10 con 0x561430fe3a10 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.732+0000 7f2d6a7fc640 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430ff7fe0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2d600ac2d0 tx=0x7f2d600ab970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f704d3515f8b9993 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.733+0000 7f2d4eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.733+0000 7f2d4eff5640 1 -- 172.21.15.33:0/1998732949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d600b5640 con 0x561430fe3a10 2024-09-10T21:13:12.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.733+0000 7f2d4eff5640 1 -- 172.21.15.33:0/1998732949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2d600e3b40 con 0x561430fe3a10 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.733+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d600abe50 con 0x561430fe3a10 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.733+0000 7f2d4eff5640 1 -- 172.21.15.33:0/1998732949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2d600aca40 con 0x561430fe3a10 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.733+0000 7f2d73b66740 10 client.17088 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.733+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561430ff2f10 con 0x561430fe3a10 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d4eff5640 1 -- 172.21.15.33:0/1998732949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d600abe50 con 0x561430fe3a10 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d4eff5640 1 -- 172.21.15.33:0/1998732949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f2d60093ad0 con 0x561430fe3a10 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d73b66740 10 client.17088 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d73b66740 10 client.17088 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d73b66740 20 client.17088 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d73b66740 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56143106d3a0 0x56143106f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d73b66740 4 client.17088 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.734+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561430f578a0 con 0x56143106d3a0 2024-09-10T21:13:12.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.735+0000 7f2d6affd640 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56143106d3a0 0x56143106f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.736+0000 7f2d6affd640 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56143106d3a0 0x56143106f790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=608b0e4590815bdd server_cookie=5329fc988666697c in_seq=0 out_seq=0 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.736+0000 7f2d4eff5640 10 client.17088 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.736+0000 7f2d4eff5640 1 -- 172.21.15.33:0/1998732949 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561430f578a0 con 0x56143106d3a0 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.736+0000 7f2d4eff5640 10 client.17088 handle_command_reply: tid=0 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.736+0000 7f2d73b66740 1 client.17088 shutdown 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 msgr2=0x561430ff7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430ff7fe0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2d600ac2d0 tx=0x7f2d600ab970 comp rx=0 tx=0).stop 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 shutdown_connections 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56143106d3a0 0x56143106f790 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561430fef9a0 0x561430ff2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d4eff5640 0 client.17088 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:12.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561430fe3a10 0x561430ff7fe0 secure :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f2d600ac2d0 tx=0x7f2d600ab970 comp rx=0 tx=0).stop 2024-09-10T21:13:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 --2- 172.21.15.33:0/1998732949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561430fe30f0 0x561430ff7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 >> 172.21.15.33:0/1998732949 conn(0x561430fe0f20 msgr2=0x561430fe1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 shutdown_connections 2024-09-10T21:13:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.737+0000 7f2d73b66740 1 -- 172.21.15.33:0/1998732949 wait complete. 2024-09-10T21:13:12.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.738+0000 7f2d73b66740 20 client.17088 trim_cache size 0 max 16384 2024-09-10T21:13:12.746 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:13:12.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.738+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d540b52e0 msgr2=0x7f2d540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.738+0000 7f2d71791640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d540b52e0 0x7f2d540b77a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009aa0 tx=0x7f2d64004ab0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c1076e0 msgr2=0x7f2d6c0fe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:12.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c1076e0 0x7f2d6c0fe180 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000e9f0 tx=0x7f2d6000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 shutdown_connections 2024-09-10T21:13:12.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2d540b52e0 0x7f2d540b77a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c108060 0x7f2d6c0fe6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2d6c1076e0 0x7f2d6c0fe180 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 --2- 172.21.15.33:0/1698208088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d6c101cd0 0x7f2d6c0fdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 >> 172.21.15.33:0/1698208088 conn(0x7f2d6c0fd740 msgr2=0x7f2d6c06b560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 shutdown_connections 2024-09-10T21:13:12.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:12.739+0000 7f2d71791640 1 -- 172.21.15.33:0/1698208088 wait complete. 2024-09-10T21:13:12.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 597538 5.18 MB/sec execute 373 sec latency 3301.312 ms 2024-09-10T21:13:13.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:13:13.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:13 smithi033 ceph-mon[30860]: pgmap v623: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 720 KiB/s wr, 75 op/s 2024-09-10T21:13:13.726 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:13.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 600339 5.22 MB/sec execute 374 sec latency 15.185 ms 2024-09-10T21:13:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:13 smithi104 ceph-mon[35325]: pgmap v623: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 720 KiB/s wr, 75 op/s 2024-09-10T21:13:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:13 smithi174 ceph-mon[35635]: pgmap v623: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 720 KiB/s wr, 75 op/s 2024-09-10T21:13:14.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 -- 172.21.15.33:0/3759743919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 msgr2=0x7efc101040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 --2- 172.21.15.33:0/3759743919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc101040c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7efc0c009900 tx=0x7efc0c02a410 comp rx=0 tx=0).stop 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 -- 172.21.15.33:0/3759743919 shutdown_connections 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 --2- 172.21.15.33:0/3759743919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1010d140 0x7efc1010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 --2- 172.21.15.33:0/3759743919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc10104600 0x7efc1010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 --2- 172.21.15.33:0/3759743919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc101040c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 -- 172.21.15.33:0/3759743919 >> 172.21.15.33:0/3759743919 conn(0x7efc100fd700 msgr2=0x7efc100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 -- 172.21.15.33:0/3759743919 shutdown_connections 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.183+0000 7efc17313640 1 -- 172.21.15.33:0/3759743919 wait complete. 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 Processor -- start 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 -- start start 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc10199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc10104600 0x7efc10199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1010d140 0x7efc1019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc10112e90 con 0x7efc10103ce0 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc10112d10 con 0x7efc1010d140 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc17313640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc10113010 con 0x7efc10104600 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc150af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc10199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc150af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc10199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52942/0 (socket says 172.21.15.33:52942) 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.184+0000 7efc150af640 1 -- 172.21.15.33:0/2094125905 learned_addr learned my addr 172.21.15.33:0/2094125905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc148ae640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc10104600 0x7efc10199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc158b0640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1010d140 0x7efc1019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc150af640 1 -- 172.21.15.33:0/2094125905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc10104600 msgr2=0x7efc10199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc150af640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc10104600 0x7efc10199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc150af640 1 -- 172.21.15.33:0/2094125905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1010d140 msgr2=0x7efc1019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc150af640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1010d140 0x7efc1019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc150af640 1 -- 172.21.15.33:0/2094125905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc1019f4b0 con 0x7efc10103ce0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc150af640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc10199400 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efc0c006010 tx=0x7efc0c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc027fc640 1 -- 172.21.15.33:0/2094125905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c03f030 con 0x7efc10103ce0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc027fc640 1 -- 172.21.15.33:0/2094125905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc0c003800 con 0x7efc10103ce0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc1019f740 con 0x7efc10103ce0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc027fc640 1 -- 172.21.15.33:0/2094125905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c035070 con 0x7efc10103ce0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.185+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc101a5e60 con 0x7efc10103ce0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.188+0000 7efc196e8740 1 Processor -- start 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc196e8740 1 -- start start 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc027fc640 1 -- 172.21.15.33:0/2094125905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efc0c031b10 con 0x7efc10103ce0 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc196e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca433a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc196e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca433adf0 0x564ca4346840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc196e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca4346d80 0x564ca4349170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc196e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ca42b0ea0 con 0x564ca4346d80 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc196e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ca42a02d0 con 0x564ca433a4d0 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc196e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ca42a40e0 con 0x564ca433adf0 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca4346d80 0x564ca4349170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca4346d80 0x564ca4349170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52952/0 (socket says 172.21.15.33:52952) 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 -- 172.21.15.33:0/2075244241 learned_addr learned my addr 172.21.15.33:0/2075244241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc158b0640 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca433a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc150af640 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca433adf0 0x564ca4346840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 -- 172.21.15.33:0/2075244241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca433adf0 msgr2=0x564ca4346840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca433adf0 0x564ca4346840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 -- 172.21.15.33:0/2075244241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 msgr2=0x564ca433a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca433a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.189+0000 7efc148ae640 1 -- 172.21.15.33:0/2075244241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca3ffe650 con 0x564ca4346d80 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.190+0000 7efc148ae640 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca4346d80 0x564ca4349170 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efc040099e0 tx=0x7efc04009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd225aac806dbad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.190+0000 7efc027fc640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efbfc0b52e0 0x7efbfc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.190+0000 7efc158b0640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efbfc0b52e0 0x7efbfc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.190+0000 7efbfb7fe640 1 -- 172.21.15.33:0/2075244241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04017070 con 0x564ca4346d80 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.190+0000 7efc027fc640 1 -- 172.21.15.33:0/2094125905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efc0c106560 con 0x7efc10103ce0 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.190+0000 7efbfb7fe640 1 -- 172.21.15.33:0/2075244241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc04013470 con 0x564ca4346d80 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.190+0000 7efbfb7fe640 1 -- 172.21.15.33:0/2075244241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc040125d0 con 0x564ca4346d80 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 -- 172.21.15.33:0/2075244241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca4346d80 msgr2=0x564ca4349170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc158b0640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efbfc0b52e0 0x7efbfc0b77a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efc0800b4d0 tx=0x7efc0800e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca4346d80 0x564ca4349170 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7efc040099e0 tx=0x7efc04009ea0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 -- 172.21.15.33:0/2075244241 shutdown_connections 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca4346d80 0x564ca4349170 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca433adf0 0x564ca4346840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 --2- 172.21.15.33:0/2075244241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca433a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 -- 172.21.15.33:0/2075244241 >> 172.21.15.33:0/2075244241 conn(0x564ca4338300 msgr2=0x564ca43394a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 -- 172.21.15.33:0/2075244241 shutdown_connections 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 -- 172.21.15.33:0/2075244241 wait complete. 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 -- start start 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.191+0000 7efc196e8740 1 Processor -- start 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc196e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4346630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc196e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca4343a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc196e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca4344040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc196e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564ca4101700 con 0x564ca433adf0 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc196e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564ca423b160 con 0x564ca433a4d0 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc196e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564ca4349a00 con 0x564ca4346d80 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc196e8740 10 client.0 _ll_register_callbacks cb 0x564ca42549e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca4343a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc150af640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4346630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc150af640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4346630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:47420/0 (socket says 172.21.15.33:47420) 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca4343a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52958/0 (socket says 172.21.15.33:52958) 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc150af640 1 -- 172.21.15.33:0/43190958 learned_addr learned my addr 172.21.15.33:0/43190958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc158b0640 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca4344040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 -- 172.21.15.33:0/43190958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 msgr2=0x564ca4344040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca4344040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 -- 172.21.15.33:0/43190958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 msgr2=0x564ca4346630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4346630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 -- 172.21.15.33:0/43190958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca4268c90 con 0x564ca433adf0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efc148ae640 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca4343a80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efc04013fd0 tx=0x7efc0400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65c9aeffa105136 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efbf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efbf8ff9640 1 -- 172.21.15.33:0/43190958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04017050 con 0x564ca433adf0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.192+0000 7efbf8ff9640 1 -- 172.21.15.33:0/43190958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efc0401c450 con 0x564ca433adf0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc04013470 con 0x564ca433adf0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efbf8ff9640 1 -- 172.21.15.33:0/43190958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efc04006d60 con 0x564ca433adf0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 10 client.17100 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564ca4355320 con 0x564ca433adf0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efbf8ff9640 1 -- 172.21.15.33:0/43190958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc04013470 con 0x564ca433adf0 2024-09-10T21:13:14.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efbf8ff9640 1 -- 172.21.15.33:0/43190958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efc04026530 con 0x564ca433adf0 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 10 client.17100 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 10 client.17100 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 20 client.17100 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca4356670 0x564ca4358a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 4 client.17100 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.193+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ca42b2df0 con 0x564ca4356670 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.194+0000 7efc150af640 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca4356670 0x564ca4358a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.194+0000 7efbf8ff9640 10 client.17100 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.194+0000 7efc150af640 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca4356670 0x564ca4358a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c575cd145c01ded0 server_cookie=91e4b2cdb39d3918 in_seq=0 out_seq=0 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efbf8ff9640 1 -- 172.21.15.33:0/43190958 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ca42b2df0 con 0x564ca4356670 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efbf8ff9640 10 client.17100 handle_command_reply: tid=0 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 client.17100 shutdown 2024-09-10T21:13:14.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 msgr2=0x564ca4343a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca4343a80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efc04013fd0 tx=0x7efc0400ee20 comp rx=0 tx=0).stop 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 shutdown_connections 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca4344040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca4356670 0x564ca4358a60 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca4343a80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 --2- 172.21.15.33:0/43190958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4346630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efbf8ff9640 0 client.17100 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 >> 172.21.15.33:0/43190958 conn(0x564ca4338300 msgr2=0x564ca433de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 shutdown_connections 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 1 -- 172.21.15.33:0/43190958 wait complete. 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.195+0000 7efc196e8740 20 client.17100 trim_cache size 0 max 16384 2024-09-10T21:13:14.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.208+0000 7efc196e8740 1 Processor -- start 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.208+0000 7efc196e8740 1 -- start start 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.208+0000 7efc196e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4352c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc196e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca434f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc196e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca434f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc196e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc101ae510 con 0x564ca433adf0 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc196e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc101b0050 con 0x564ca433a4d0 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc196e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc101aebe0 con 0x564ca4346d80 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca434f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca434f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52966/0 (socket says 172.21.15.33:52966) 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 -- 172.21.15.33:0/337214297 learned_addr learned my addr 172.21.15.33:0/337214297 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:14.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc150af640 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4352c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 -- 172.21.15.33:0/337214297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 msgr2=0x564ca434f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc158b0640 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca434f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca434f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 -- 172.21.15.33:0/337214297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 msgr2=0x564ca4352c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4352c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 -- 172.21.15.33:0/337214297 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ca43443b0 con 0x564ca433adf0 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efc148ae640 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca434f0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efc04026f80 tx=0x7efc040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168838ddb87a62f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efbfaffd640 1 -- 172.21.15.33:0/337214297 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04035030 con 0x564ca433adf0 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efbfaffd640 1 -- 172.21.15.33:0/337214297 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc04013470 con 0x564ca433adf0 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.209+0000 7efbfaffd640 1 -- 172.21.15.33:0/337214297 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc04017070 con 0x564ca433adf0 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 -- 172.21.15.33:0/337214297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 msgr2=0x564ca434f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca434f0a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efc04026f80 tx=0x7efc040046c0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 -- 172.21.15.33:0/337214297 shutdown_connections 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca434f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433adf0 0x564ca434f0a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 --2- 172.21.15.33:0/337214297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433a4d0 0x564ca4352c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 -- 172.21.15.33:0/337214297 >> 172.21.15.33:0/337214297 conn(0x564ca4338300 msgr2=0x564ca43458d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 -- 172.21.15.33:0/337214297 shutdown_connections 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 -- 172.21.15.33:0/337214297 wait complete. 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 -- start start 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.210+0000 7efc196e8740 1 Processor -- start 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433a4d0 0x564ca4348c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433adf0 0x564ca4347190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca43477c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efc101af980 con 0x564ca433a4d0 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efc101af2b0 con 0x564ca433adf0 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc101ac4d0 con 0x564ca4346d80 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 10 client.0 _ll_register_callbacks cb 0x7efc08009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433a4d0 0x564ca4348c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433a4d0 0x564ca4348c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52980/0 (socket says 172.21.15.33:52980) 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 -- 172.21.15.33:0/2050808479 learned_addr learned my addr 172.21.15.33:0/2050808479 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc148ae640 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433adf0 0x564ca4347190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc158b0640 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca43477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 -- 172.21.15.33:0/2050808479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 msgr2=0x564ca43477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca43477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 -- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433adf0 msgr2=0x564ca4347190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433adf0 0x564ca4347190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 -- 172.21.15.33:0/2050808479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ca43c7300 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc150af640 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433a4d0 0x564ca4348c80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efc0c031f90 tx=0x7efc0c009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95990ddc908fa05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efbf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efbf8ff9640 1 -- 172.21.15.33:0/2050808479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc0c049040 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efbf8ff9640 1 -- 172.21.15.33:0/2050808479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efc0c003cb0 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.211+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc04013470 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efbf8ff9640 1 -- 172.21.15.33:0/2050808479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7efc0c0f3020 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 10 client.17106 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564ca43c8450 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efbf8ff9640 1 -- 172.21.15.33:0/2050808479 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc04013470 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efbf8ff9640 1 -- 172.21.15.33:0/2050808479 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7efc0c0a6ce0 con 0x564ca433a4d0 2024-09-10T21:13:14.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 10 client.17106 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 10 client.17106 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 20 client.17106 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca43c8bf0 0x564ca43cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 4 client.17106 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ca42ae8a0 con 0x564ca43c8bf0 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.212+0000 7efc148ae640 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca43c8bf0 0x564ca43cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efbf8ff9640 10 client.17106 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc148ae640 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca43c8bf0 0x564ca43cb010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8affe82621c6e71b server_cookie=34a3df3f929cc026 in_seq=0 out_seq=0 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efbf8ff9640 1 -- 172.21.15.33:0/2050808479 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ca42ae8a0 con 0x564ca43c8bf0 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efbf8ff9640 10 client.17106 handle_command_reply: tid=0 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 client.17106 shutdown 2024-09-10T21:13:14.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433a4d0 msgr2=0x564ca4348c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433a4d0 0x564ca4348c80 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efc0c031f90 tx=0x7efc0c009620 comp rx=0 tx=0).stop 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 shutdown_connections 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564ca4346d80 0x564ca43477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564ca433adf0 0x564ca4347190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x564ca43c8bf0 0x564ca43cb010 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 --2- 172.21.15.33:0/2050808479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564ca433a4d0 0x564ca4348c80 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efbf8ff9640 0 client.17106 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 >> 172.21.15.33:0/2050808479 conn(0x564ca4338300 msgr2=0x564ca4336c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 shutdown_connections 2024-09-10T21:13:14.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 1 -- 172.21.15.33:0/2050808479 wait complete. 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.213+0000 7efc196e8740 20 client.17106 trim_cache size 0 max 16384 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efbfc0b52e0 msgr2=0x7efbfc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efbfc0b52e0 0x7efbfc0b77a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efc0800b4d0 tx=0x7efc0800e040 comp rx=0 tx=0).stop 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 msgr2=0x7efc10199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc10199400 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7efc0c006010 tx=0x7efc0c0042f0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 shutdown_connections 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7efbfc0b52e0 0x7efbfc0b77a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efc1010d140 0x7efc1019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc10104600 0x7efc10199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 --2- 172.21.15.33:0/2094125905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc10103ce0 0x7efc10199400 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 >> 172.21.15.33:0/2094125905 conn(0x7efc100fd700 msgr2=0x7efc100fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 shutdown_connections 2024-09-10T21:13:14.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:14.214+0000 7efc17313640 1 -- 172.21.15.33:0/2094125905 wait complete. 2024-09-10T21:13:14.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 600344 5.20 MB/sec execute 375 sec latency 999.108 ms 2024-09-10T21:13:15.023 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:13:15.283 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:15.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:15 smithi033 ceph-mon[30860]: pgmap v624: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 909 KiB/s wr, 107 op/s 2024-09-10T21:13:15.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 600344 5.19 MB/sec execute 376 sec latency 1999.136 ms 2024-09-10T21:13:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 -- 172.21.15.33:0/889628947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 msgr2=0x7f9e5810cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 --2- 172.21.15.33:0/889628947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e5810cb30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009900 tx=0x7f9e4c02a410 comp rx=0 tx=0).stop 2024-09-10T21:13:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 -- 172.21.15.33:0/889628947 shutdown_connections 2024-09-10T21:13:15.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 --2- 172.21.15.33:0/889628947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e5810d100 0x7f9e5810f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 --2- 172.21.15.33:0/889628947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e5810cb30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 --2- 172.21.15.33:0/889628947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e58103b50 0x7f9e58103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 -- 172.21.15.33:0/889628947 >> 172.21.15.33:0/889628947 conn(0x7f9e580fd700 msgr2=0x7f9e580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 -- 172.21.15.33:0/889628947 shutdown_connections 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.781+0000 7f9e5e233640 1 -- 172.21.15.33:0/889628947 wait complete. 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 Processor -- start 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 -- start start 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e58103b50 0x7f9e58103480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e581019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e5810d100 0x7f9e58102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58112d80 con 0x7f9e58104550 2024-09-10T21:13:15.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58112c00 con 0x7f9e5810d100 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.782+0000 7f9e5e233640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e58112f00 con 0x7f9e58103b50 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e581019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e581019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53002/0 (socket says 172.21.15.33:53002) 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 -- 172.21.15.33:0/1333055831 learned_addr learned my addr 172.21.15.33:0/1333055831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e577fe640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e58103b50 0x7f9e58103480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e57fff640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e5810d100 0x7f9e58102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 -- 172.21.15.33:0/1333055831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e58103b50 msgr2=0x7f9e58103480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e58103b50 0x7f9e58103480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 -- 172.21.15.33:0/1333055831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e5810d100 msgr2=0x7f9e58102000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e5810d100 0x7f9e58102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 -- 172.21.15.33:0/1333055831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e581abbd0 con 0x7f9e58104550 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e577fe640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e58103b50 0x7f9e58103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:15.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e581019d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005940 tx=0x7f9e4c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e54ff9640 1 -- 172.21.15.33:0/1333055831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c03f030 con 0x7f9e58104550 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e581abda0 con 0x7f9e58104550 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e54ff9640 1 -- 172.21.15.33:0/1333055831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e4c003cb0 con 0x7f9e58104550 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e54ff9640 1 -- 172.21.15.33:0/1333055831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c035070 con 0x7f9e58104550 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.783+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581ac1a0 con 0x7f9e58104550 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e54ff9640 1 -- 172.21.15.33:0/1333055831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9e4c03bbf0 con 0x7f9e58104550 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 Processor -- start 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 -- start start 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c21a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c226460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c228d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c190ea0 con 0x55bb6c21a0f0 2024-09-10T21:13:15.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c1802d0 con 0x55bb6c2269a0 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e60608740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c1840e0 con 0x55bb6c21aa10 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c21a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c21a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53004/0 (socket says 172.21.15.33:53004) 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e57fff640 1 -- 172.21.15.33:0/1957660828 learned_addr learned my addr 172.21.15.33:0/1957660828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.786+0000 7f9e577fe640 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c226460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c228d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e57fff640 1 -- 172.21.15.33:0/1957660828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 msgr2=0x55bb6c226460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e57fff640 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c226460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e57fff640 1 -- 172.21.15.33:0/1957660828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 msgr2=0x55bb6c228d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e57fff640 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c228d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e57fff640 1 -- 172.21.15.33:0/1957660828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb6bede650 con 0x55bb6c21a0f0 2024-09-10T21:13:15.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c228d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e54ff9640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e400b52e0 0x7f9e400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e57fff640 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c21a4d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009df0 tx=0x7f9e5000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43493439e8e61fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e577fe640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e400b52e0 0x7f9e400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e54ff9640 1 -- 172.21.15.33:0/1333055831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9e4c1067b0 con 0x7f9e58104550 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e3dffb640 1 -- 172.21.15.33:0/1957660828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50003970 con 0x55bb6c21a0f0 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e3dffb640 1 -- 172.21.15.33:0/1957660828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e50004030 con 0x55bb6c21a0f0 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.787+0000 7f9e3dffb640 1 -- 172.21.15.33:0/1957660828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500116e0 con 0x55bb6c21a0f0 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 -- 172.21.15.33:0/1957660828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 msgr2=0x55bb6c21a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c21a4d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009df0 tx=0x7f9e5000f730 comp rx=0 tx=0).stop 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e577fe640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e400b52e0 0x7f9e400b77a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55bb6c218e00 tx=0x7f9e4800c330 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 -- 172.21.15.33:0/1957660828 shutdown_connections 2024-09-10T21:13:15.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c228d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c226460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 --2- 172.21.15.33:0/1957660828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c21a4d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 -- 172.21.15.33:0/1957660828 >> 172.21.15.33:0/1957660828 conn(0x55bb6c217f20 msgr2=0x55bb6c2190c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 -- 172.21.15.33:0/1957660828 shutdown_connections 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 -- 172.21.15.33:0/1957660828 wait complete. 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 -- start start 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.788+0000 7f9e60608740 1 Processor -- start 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e60608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c21a0f0 0x55bb6c226270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e60608740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c2236c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e60608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c2269a0 0x55bb6c223ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e60608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bb6bfe1700 con 0x55bb6c2269a0 2024-09-10T21:13:15.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e60608740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c11b160 con 0x55bb6c21a0f0 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e60608740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb6c229620 con 0x55bb6c21aa10 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e60608740 10 client.0 _ll_register_callbacks cb 0x55bb6c1349e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c2269a0 0x55bb6c223ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c2269a0 0x55bb6c223ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53006/0 (socket says 172.21.15.33:53006) 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 -- 172.21.15.33:0/2385985365 learned_addr learned my addr 172.21.15.33:0/2385985365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c2236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e577fe640 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c21a0f0 0x55bb6c226270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 -- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 msgr2=0x55bb6c2236c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c2236c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 -- 172.21.15.33:0/2385985365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c21a0f0 msgr2=0x55bb6c226270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c21a0f0 0x55bb6c226270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 -- 172.21.15.33:0/2385985365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb6c148c90 con 0x55bb6c2269a0 2024-09-10T21:13:15.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e57fff640 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c2269a0 0x55bb6c223ca0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006010 tx=0x7f9e50004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc065ac65a4d430f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.789+0000 7f9e36ffd640 1 -- 172.21.15.33:0/2385985365 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50004a40 con 0x55bb6c2269a0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e36ffd640 1 -- 172.21.15.33:0/2385985365 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9e50004be0 con 0x55bb6c2269a0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e50004030 con 0x55bb6c2269a0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e36ffd640 1 -- 172.21.15.33:0/2385985365 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9e50004de0 con 0x55bb6c2269a0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 10 client.17118 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bb6c234ed0 con 0x55bb6c2269a0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e36ffd640 1 -- 172.21.15.33:0/2385985365 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e50004030 con 0x55bb6c2269a0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e36ffd640 1 -- 172.21.15.33:0/2385985365 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f9e500056c0 con 0x55bb6c2269a0 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 10 client.17118 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 10 client.17118 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:13:15.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 20 client.17118 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2362d0 0x55bb6c2386c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 4 client.17118 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb6c192df0 con 0x55bb6c2362d0 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.790+0000 7f9e577fe640 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2362d0 0x55bb6c2386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.791+0000 7f9e36ffd640 10 client.17118 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.791+0000 7f9e577fe640 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2362d0 0x55bb6c2386c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2bd6622ae03511b2 server_cookie=23e414b916895097 in_seq=0 out_seq=0 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e36ffd640 1 -- 172.21.15.33:0/2385985365 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb6c192df0 con 0x55bb6c2362d0 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e36ffd640 10 client.17118 handle_command_reply: tid=0 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 client.17118 shutdown 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c2269a0 msgr2=0x55bb6c223ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c2269a0 0x55bb6c223ca0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9e50006010 tx=0x7f9e50004410 comp rx=0 tx=0).stop 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 shutdown_connections 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2362d0 0x55bb6c2386c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c2269a0 0x55bb6c223ca0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c2236c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e36ffd640 0 client.17118 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 --2- 172.21.15.33:0/2385985365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c21a0f0 0x55bb6c226270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 >> 172.21.15.33:0/2385985365 conn(0x55bb6c217f20 msgr2=0x55bb6c21da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 shutdown_connections 2024-09-10T21:13:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 1 -- 172.21.15.33:0/2385985365 wait complete. 2024-09-10T21:13:15.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.792+0000 7f9e60608740 20 client.17118 trim_cache size 0 max 16384 2024-09-10T21:13:15.819 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:15 smithi104 ceph-mon[35325]: pgmap v624: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 909 KiB/s wr, 107 op/s 2024-09-10T21:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 Processor -- start 2024-09-10T21:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 -- start start 2024-09-10T21:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c232880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c212e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c225400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581b4a10 con 0x55bb6c21a0f0 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581b6550 con 0x55bb6c2269a0 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e60608740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581b50e0 con 0x55bb6c21aa10 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c232880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c232880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53014/0 (socket says 172.21.15.33:53014) 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.806+0000 7f9e577fe640 1 -- 172.21.15.33:0/4165746811 learned_addr learned my addr 172.21.15.33:0/4165746811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c212e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e57fff640 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c225400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e577fe640 1 -- 172.21.15.33:0/4165746811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 msgr2=0x55bb6c212e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e577fe640 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c212e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e577fe640 1 -- 172.21.15.33:0/4165746811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 msgr2=0x55bb6c225400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e577fe640 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c225400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e577fe640 1 -- 172.21.15.33:0/4165746811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb6c218c00 con 0x55bb6c21a0f0 2024-09-10T21:13:15.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e57fff640 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c225400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e577fe640 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c232880 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55bb6c238cc0 tx=0x7f9e48013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b768d061a8bec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e3d7fa640 1 -- 172.21.15.33:0/4165746811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4801f890 con 0x55bb6c21a0f0 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e3d7fa640 1 -- 172.21.15.33:0/4165746811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e4802b070 con 0x55bb6c21a0f0 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.807+0000 7f9e3d7fa640 1 -- 172.21.15.33:0/4165746811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48030e60 con 0x55bb6c21a0f0 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 -- 172.21.15.33:0/4165746811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 msgr2=0x55bb6c232880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c232880 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x55bb6c238cc0 tx=0x7f9e48013000 comp rx=0 tx=0).stop 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 -- 172.21.15.33:0/4165746811 shutdown_connections 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c225400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c212e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 --2- 172.21.15.33:0/4165746811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c232880 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 -- 172.21.15.33:0/4165746811 >> 172.21.15.33:0/4165746811 conn(0x55bb6c217f20 msgr2=0x55bb6c224cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 -- 172.21.15.33:0/4165746811 shutdown_connections 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 -- 172.21.15.33:0/4165746811 wait complete. 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.808+0000 7f9e60608740 1 -- start start 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 1 Processor -- start 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c22eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c22f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c2298c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581b5e80 con 0x55bb6c21a0f0 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581b57b0 con 0x55bb6c2269a0 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e581b29d0 con 0x55bb6c21aa10 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e60608740 10 client.0 _ll_register_callbacks cb 0x7f9e50009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c22eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c22eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53028/0 (socket says 172.21.15.33:53028) 2024-09-10T21:13:15.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 -- 172.21.15.33:0/3859836742 learned_addr learned my addr 172.21.15.33:0/3859836742 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c22f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e57fff640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c2298c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 -- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 msgr2=0x55bb6c22f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c22f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 -- 172.21.15.33:0/3859836742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 msgr2=0x55bb6c2298c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c2298c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 -- 172.21.15.33:0/3859836742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb6c22a0f0 con 0x55bb6c21a0f0 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c22f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e57fff640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c2298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e577fe640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c22eb60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9e48030e10 tx=0x7f9e48030e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75524a1705b4f03c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.809+0000 7f9e36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e36ffd640 1 -- 172.21.15.33:0/3859836742 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4803e060 con 0x55bb6c21a0f0 2024-09-10T21:13:15.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e36ffd640 1 -- 172.21.15.33:0/3859836742 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9e48030680 con 0x55bb6c21a0f0 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4802b070 con 0x55bb6c21a0f0 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e36ffd640 1 -- 172.21.15.33:0/3859836742 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f9e4801f690 con 0x55bb6c21a0f0 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 10 client.17124 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bb6c2a39d0 con 0x55bb6c21a0f0 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e36ffd640 1 -- 172.21.15.33:0/3859836742 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e4802b070 con 0x55bb6c21a0f0 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e36ffd640 1 -- 172.21.15.33:0/3859836742 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f9e48043070 con 0x55bb6c21a0f0 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 10 client.17124 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 10 client.17124 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 20 client.17124 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2a4090 0x55bb6c2a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 4 client.17124 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:13:15.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.810+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb6c18e8a0 con 0x55bb6c2a4090 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.811+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2a4090 0x55bb6c2a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.811+0000 7f9e56ffd640 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2a4090 0x55bb6c2a64b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7c886d20cf493686 server_cookie=d65719bc075c10b1 in_seq=0 out_seq=0 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.811+0000 7f9e36ffd640 10 client.17124 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.811+0000 7f9e36ffd640 1 -- 172.21.15.33:0/3859836742 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb6c18e8a0 con 0x55bb6c2a4090 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.811+0000 7f9e36ffd640 10 client.17124 handle_command_reply: tid=0 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.811+0000 7f9e60608740 1 client.17124 shutdown 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 msgr2=0x55bb6c22eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c22eb60 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9e48030e10 tx=0x7f9e48030e40 comp rx=0 tx=0).stop 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 shutdown_connections 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bb6c2269a0 0x55bb6c2298c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb6c21aa10 0x55bb6c22f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55bb6c2a4090 0x55bb6c2a64b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 --2- 172.21.15.33:0/3859836742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bb6c21a0f0 0x55bb6c22eb60 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 >> 172.21.15.33:0/3859836742 conn(0x55bb6c217f20 msgr2=0x55bb6c225510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e36ffd640 0 client.17124 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:13:15.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 shutdown_connections 2024-09-10T21:13:15.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 1 -- 172.21.15.33:0/3859836742 wait complete. 2024-09-10T21:13:15.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e60608740 20 client.17124 trim_cache size 0 max 16384 2024-09-10T21:13:15.830 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e400b52e0 msgr2=0x7f9e400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e400b52e0 0x7f9e400b77a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55bb6c218e00 tx=0x7f9e4800c330 comp rx=0 tx=0).stop 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 msgr2=0x7f9e581019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e581019d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c005940 tx=0x7f9e4c0042f0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 shutdown_connections 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e400b52e0 0x7f9e400b77a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e5810d100 0x7f9e58102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e58104550 0x7f9e581019d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 --2- 172.21.15.33:0/1333055831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e58103b50 0x7f9e58103480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:15.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.812+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 >> 172.21.15.33:0/1333055831 conn(0x7f9e580fd700 msgr2=0x7f9e580fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:15.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.813+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 shutdown_connections 2024-09-10T21:13:15.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:15.813+0000 7f9e5e233640 1 -- 172.21.15.33:0/1333055831 wait complete. 2024-09-10T21:13:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:15 smithi174 ceph-mon[35635]: pgmap v624: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 909 KiB/s wr, 107 op/s 2024-09-10T21:13:16.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:13:16.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:16 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:13:16.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 600344 5.18 MB/sec execute 377 sec latency 2999.170 ms 2024-09-10T21:13:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:16 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:13:16.847 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:16 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:13:17.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 -- 172.21.15.33:0/42416486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8644108060 msgr2=0x7f864410e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 --2- 172.21.15.33:0/42416486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8644108060 0x7f864410e860 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f863c009900 tx=0x7f863c02a410 comp rx=0 tx=0).stop 2024-09-10T21:13:17.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 -- 172.21.15.33:0/42416486 shutdown_connections 2024-09-10T21:13:17.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 --2- 172.21.15.33:0/42416486 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8644108060 0x7f864410e860 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 --2- 172.21.15.33:0/42416486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86441076e0 0x7f8644107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 --2- 172.21.15.33:0/42416486 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8644101cd0 0x7f86441020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 -- 172.21.15.33:0/42416486 >> 172.21.15.33:0/42416486 conn(0x7f86440fd700 msgr2=0x7f86440ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 -- 172.21.15.33:0/42416486 shutdown_connections 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.331+0000 7f864aaa6640 1 -- 172.21.15.33:0/42416486 wait complete. 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 Processor -- start 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 -- start start 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8644101cd0 0x7f8644199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86441076e0 0x7f86441998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8644108060 0x7f864419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8644111ce0 con 0x7f86441076e0 2024-09-10T21:13:17.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8644111b60 con 0x7f8644108060 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f864aaa6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8644111e60 con 0x7f8644101cd0 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86441076e0 0x7f86441998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86441076e0 0x7f86441998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53060/0 (socket says 172.21.15.33:53060) 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 -- 172.21.15.33:0/1749782121 learned_addr learned my addr 172.21.15.33:0/1749782121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8649043640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8644108060 0x7f864419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8648842640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8644101cd0 0x7f8644199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 -- 172.21.15.33:0/1749782121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8644101cd0 msgr2=0x7f8644199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8644101cd0 0x7f8644199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 -- 172.21.15.33:0/1749782121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8644108060 msgr2=0x7f864419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8644108060 0x7f864419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.332+0000 7f8643fff640 1 -- 172.21.15.33:0/1749782121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864419f410 con 0x7f86441076e0 2024-09-10T21:13:17.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.333+0000 7f8643fff640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86441076e0 0x7f86441998a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f863400eb90 tx=0x7f863400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.333+0000 7f8641ffb640 1 -- 172.21.15.33:0/1749782121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634004d20 con 0x7f86441076e0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.333+0000 7f8641ffb640 1 -- 172.21.15.33:0/1749782121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f863400ee90 con 0x7f86441076e0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.333+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864419f700 con 0x7f86441076e0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.333+0000 7f8641ffb640 1 -- 172.21.15.33:0/1749782121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634010670 con 0x7f86441076e0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.333+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86441a5dc0 con 0x7f86441076e0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f8641ffb640 1 -- 172.21.15.33:0/1749782121 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8634005020 con 0x7f86441076e0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 Processor -- start 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 -- start start 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115dddb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115ddebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115ddee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56115dd51ea0 con 0x56115dddbdf0 2024-09-10T21:13:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56115dd412d0 con 0x56115ddec190 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f864ce7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56115dd450e0 con 0x56115dddb4d0 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f8648842640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115ddebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f8648842640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115ddebc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53066/0 (socket says 172.21.15.33:53066) 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f8643fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115ddee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.336+0000 7f8648842640 1 -- 172.21.15.33:0/3301443696 learned_addr learned my addr 172.21.15.33:0/3301443696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8649043640 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115dddb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8648842640 1 -- 172.21.15.33:0/3301443696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 msgr2=0x56115dddb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8648842640 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115dddb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8648842640 1 -- 172.21.15.33:0/3301443696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 msgr2=0x56115ddee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8648842640 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115ddee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8648842640 1 -- 172.21.15.33:0/3301443696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115da9f650 con 0x56115dddbdf0 2024-09-10T21:13:17.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8648842640 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115ddebc50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f862c009d20 tx=0x7f862c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68103754e102308f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8641ffb640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86300b52e0 0x7f86300b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f863affd640 1 -- 172.21.15.33:0/3301443696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c004230 con 0x56115dddbdf0 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8649043640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86300b52e0 0x7f86300b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f863affd640 1 -- 172.21.15.33:0/3301443696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f862c0043d0 con 0x56115dddbdf0 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f863affd640 1 -- 172.21.15.33:0/3301443696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862c0197c0 con 0x56115dddbdf0 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.337+0000 7f8641ffb640 1 -- 172.21.15.33:0/1749782121 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86340e61c0 con 0x7f86441076e0 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 -- 172.21.15.33:0/3301443696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 msgr2=0x56115ddebc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115ddebc50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f862c009d20 tx=0x7f862c011810 comp rx=0 tx=0).stop 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f8649043640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86300b52e0 0x7f86300b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f863c006010 tx=0x7f863c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 -- 172.21.15.33:0/3301443696 shutdown_connections 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115ddee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115ddebc50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3301443696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115dddb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 -- 172.21.15.33:0/3301443696 >> 172.21.15.33:0/3301443696 conn(0x56115ddd9300 msgr2=0x56115ddda4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 -- 172.21.15.33:0/3301443696 shutdown_connections 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 -- 172.21.15.33:0/3301443696 wait complete. 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.338+0000 7f864ce7b740 1 -- start start 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 1 Processor -- start 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115ddeba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115dde8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115dde93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56115dba2700 con 0x56115dddbdf0 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56115dcdc160 con 0x56115ddec190 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56115ddeee40 con 0x56115dddb4d0 2024-09-10T21:13:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f864ce7b740 10 client.0 _ll_register_callbacks cb 0x56115dcf59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8643fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115dde8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8648842640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115ddeba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8649043640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115dde93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8648842640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115ddeba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:48662/0 (socket says 172.21.15.33:48662) 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8649043640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115dde93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:47510/0 (socket says 172.21.15.33:47510) 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8648842640 1 -- 172.21.15.33:0/650134132 learned_addr learned my addr 172.21.15.33:0/650134132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8643fff640 1 -- 172.21.15.33:0/650134132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 msgr2=0x56115ddeba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8643fff640 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115ddeba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8643fff640 1 -- 172.21.15.33:0/650134132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 msgr2=0x56115dde93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8643fff640 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115dde93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.339+0000 7f8643fff640 1 -- 172.21.15.33:0/650134132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115dd09c90 con 0x56115dddbdf0 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f8643fff640 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115dde8e50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f86340e3cc0 tx=0x7f86340e9770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2f826c77796b50f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f8623fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f8623fff640 1 -- 172.21.15.33:0/650134132 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340ac750 con 0x56115dddbdf0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f8623fff640 1 -- 172.21.15.33:0/650134132 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86340b5a80 con 0x56115dddbdf0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f862c0043d0 con 0x56115dddbdf0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f8623fff640 1 -- 172.21.15.33:0/650134132 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f86340a80f0 con 0x56115dddbdf0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f864ce7b740 10 client.17136 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56115ddfa720 con 0x56115dddbdf0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f8623fff640 1 -- 172.21.15.33:0/650134132 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f862c0043d0 con 0x56115dddbdf0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f8623fff640 1 -- 172.21.15.33:0/650134132 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8634005340 con 0x56115dddbdf0 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f864ce7b740 10 client.17136 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f864ce7b740 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f864ce7b740 20 client.17136 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.340+0000 7f864ce7b740 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115ddfbae0 0x56115ddfdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.341+0000 7f864ce7b740 4 client.17136 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:17.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.341+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56115dd53df0 con 0x56115ddfbae0 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.341+0000 7f8648842640 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115ddfbae0 0x56115ddfdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.341+0000 7f8648842640 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115ddfbae0 0x56115ddfdf10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef3969b587ef31a server_cookie=30aed7688ac572f1 in_seq=0 out_seq=0 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.341+0000 7f8623fff640 10 client.17136 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f8623fff640 1 -- 172.21.15.33:0/650134132 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56115dd53df0 con 0x56115ddfbae0 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f8623fff640 10 client.17136 handle_command_reply: tid=0 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 client.17136 shutdown 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 msgr2=0x56115dde8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115dde8e50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f86340e3cc0 tx=0x7f86340e9770 comp rx=0 tx=0).stop 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 shutdown_connections 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115ddfbae0 0x56115ddfdf10 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115ddec190 0x56115dde93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddbdf0 0x56115dde8e50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f8623fff640 0 client.17136 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 --2- 172.21.15.33:0/650134132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115dddb4d0 0x56115ddeba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 >> 172.21.15.33:0/650134132 conn(0x56115ddd9300 msgr2=0x56115dddeea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 shutdown_connections 2024-09-10T21:13:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 1 -- 172.21.15.33:0/650134132 wait complete. 2024-09-10T21:13:17.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.342+0000 7f864ce7b740 20 client.17136 trim_cache size 0 max 16384 2024-09-10T21:13:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 Processor -- start 2024-09-10T21:13:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 -- start start 2024-09-10T21:13:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115de03830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115de03d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86441ac270 con 0x56115dddb4d0 2024-09-10T21:13:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86441addb0 con 0x56115dddbdf0 2024-09-10T21:13:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f864ce7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86441ac940 con 0x56115ddec190 2024-09-10T21:13:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f8643fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115de03830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f8648842640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f8643fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115de03830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:47524/0 (socket says 172.21.15.33:47524) 2024-09-10T21:13:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f8643fff640 1 -- 172.21.15.33:0/981355098 learned_addr learned my addr 172.21.15.33:0/981355098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:17.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.356+0000 7f8649043640 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115de03d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f8643fff640 1 -- 172.21.15.33:0/981355098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 msgr2=0x56115de03d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f8643fff640 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115de03d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f8643fff640 1 -- 172.21.15.33:0/981355098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 msgr2=0x56115ddf80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f8643fff640 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f8643fff640 1 -- 172.21.15.33:0/981355098 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115dde8c00 con 0x56115dddbdf0 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f8643fff640 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115de03830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86340a9bf0 tx=0x7f8634009200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84f1e69a36efca6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f863a7fc640 1 -- 172.21.15.33:0/981355098 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340ac0f0 con 0x56115dddbdf0 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f863a7fc640 1 -- 172.21.15.33:0/981355098 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86340e5070 con 0x56115dddbdf0 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.357+0000 7f863a7fc640 1 -- 172.21.15.33:0/981355098 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340ac2d0 con 0x56115dddbdf0 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 -- 172.21.15.33:0/981355098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 msgr2=0x56115de03830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115de03830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f86340a9bf0 tx=0x7f8634009200 comp rx=0 tx=0).stop 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 -- 172.21.15.33:0/981355098 shutdown_connections 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115de03d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115de03830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 --2- 172.21.15.33:0/981355098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 -- 172.21.15.33:0/981355098 >> 172.21.15.33:0/981355098 conn(0x56115ddd9300 msgr2=0x56115ddedf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 -- 172.21.15.33:0/981355098 shutdown_connections 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 -- 172.21.15.33:0/981355098 wait complete. 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 -- start start 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.358+0000 7f864ce7b740 1 Processor -- start 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115ddf10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115ddf1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86441ad6e0 con 0x56115dddb4d0 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86441ad010 con 0x56115dddbdf0 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86441aa230 con 0x56115ddec190 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 10 client.0 _ll_register_callbacks cb 0x7f86340a8040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115ddf10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115ddf10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:47526/0 (socket says 172.21.15.33:47526) 2024-09-10T21:13:17.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 -- 172.21.15.33:0/3651516490 learned_addr learned my addr 172.21.15.33:0/3651516490 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8648842640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8649043640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115ddf1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 -- 172.21.15.33:0/3651516490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 msgr2=0x56115ddf1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115ddf1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 -- 172.21.15.33:0/3651516490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 msgr2=0x56115ddf4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 -- 172.21.15.33:0/3651516490 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56115ddf1d80 con 0x56115dddbdf0 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8648842640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8643fff640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115ddf10d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f86340a8f50 tx=0x7f86340ad080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2b530a3c07d0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8623fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8623fff640 1 -- 172.21.15.33:0/3651516490 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86340ac0f0 con 0x56115dddbdf0 2024-09-10T21:13:17.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f8623fff640 1 -- 172.21.15.33:0/3651516490 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86340b11b0 con 0x56115dddbdf0 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.359+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86340e5070 con 0x56115dddbdf0 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f8623fff640 1 -- 172.21.15.33:0/3651516490 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f86340e9c40 con 0x56115dddbdf0 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 10 client.26582 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56115de699e0 con 0x56115dddbdf0 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f8623fff640 1 -- 172.21.15.33:0/3651516490 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86340e5070 con 0x56115dddbdf0 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f8623fff640 1 -- 172.21.15.33:0/3651516490 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f86340ab650 con 0x56115dddbdf0 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 10 client.26582 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 10 client.26582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 20 client.26582 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115de69e70 0x56115de6c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 4 client.26582 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56115dd4f8a0 con 0x56115de69e70 2024-09-10T21:13:17.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.360+0000 7f8648842640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115de69e70 0x56115de6c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f8623fff640 10 client.26582 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f8648842640 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115de69e70 0x56115de6c290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e7f66ee02963b3 server_cookie=ae22f7ad3adbded1 in_seq=0 out_seq=0 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f8623fff640 1 -- 172.21.15.33:0/3651516490 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56115dd4f8a0 con 0x56115de69e70 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f8623fff640 10 client.26582 handle_command_reply: tid=0 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 client.26582 shutdown 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 msgr2=0x56115ddf10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115ddf10d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f86340a8f50 tx=0x7f86340ad080 comp rx=0 tx=0).stop 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 shutdown_connections 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56115de69e70 0x56115de6c290 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56115ddec190 0x56115ddf1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56115dddbdf0 0x56115ddf10d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f8623fff640 0 client.26582 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:17.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 --2- 172.21.15.33:0/3651516490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56115dddb4d0 0x56115ddf4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 >> 172.21.15.33:0/3651516490 conn(0x56115ddd9300 msgr2=0x56115ddea480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 shutdown_connections 2024-09-10T21:13:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 1 -- 172.21.15.33:0/3651516490 wait complete. 2024-09-10T21:13:17.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.361+0000 7f864ce7b740 20 client.26582 trim_cache size 0 max 16384 2024-09-10T21:13:17.371 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:13:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86300b52e0 msgr2=0x7f86300b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86300b52e0 0x7f86300b77a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f863c006010 tx=0x7f863c0058d0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86441076e0 msgr2=0x7f86441998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86441076e0 0x7f86441998a0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f863400eb90 tx=0x7f863400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 shutdown_connections 2024-09-10T21:13:17.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86300b52e0 0x7f86300b77a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8644108060 0x7f864419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86441076e0 0x7f86441998a0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 --2- 172.21.15.33:0/1749782121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8644101cd0 0x7f8644199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 >> 172.21.15.33:0/1749782121 conn(0x7f86440fd700 msgr2=0x7f864406b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 shutdown_connections 2024-09-10T21:13:17.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:17.362+0000 7f864aaa6640 1 -- 172.21.15.33:0/1749782121 wait complete. 2024-09-10T21:13:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:17 smithi033 ceph-mon[30860]: pgmap v625: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 901 KiB/s wr, 96 op/s 2024-09-10T21:13:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:13:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:13:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:13:17.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 600965 5.17 MB/sec execute 378 sec latency 3353.531 ms 2024-09-10T21:13:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:17 smithi104 ceph-mon[35325]: pgmap v625: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 901 KiB/s wr, 96 op/s 2024-09-10T21:13:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:13:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:13:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:13:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:17 smithi174 ceph-mon[35635]: pgmap v625: 97 pgs: 97 active+clean; 231 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 901 KiB/s wr, 96 op/s 2024-09-10T21:13:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:13:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:13:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:13:18.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 603615 5.19 MB/sec execute 379 sec latency 12.258 ms 2024-09-10T21:13:19.153 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:13:19.153 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:13:19.421 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:19.547 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:19 smithi033 ceph-mon[30860]: pgmap v626: 97 pgs: 97 active+clean; 232 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-10T21:13:19.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 605054 5.19 MB/sec execute 380 sec latency 17.201 ms 2024-09-10T21:13:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:19 smithi104 ceph-mon[35325]: pgmap v626: 97 pgs: 97 active+clean; 232 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-10T21:13:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:19 smithi174 ceph-mon[35635]: pgmap v626: 97 pgs: 97 active+clean; 232 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-09-10T21:13:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 -- 172.21.15.33:0/2992506316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b8101cb0 msgr2=0x7f70b8102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/2992506316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b8101cb0 0x7f70b8102090 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009900 tx=0x7f70b402a410 comp rx=0 tx=0).stop 2024-09-10T21:13:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 -- 172.21.15.33:0/2992506316 shutdown_connections 2024-09-10T21:13:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/2992506316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70b8108040 0x7f70b810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/2992506316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70b81076c0 0x7f70b8107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/2992506316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b8101cb0 0x7f70b8102090 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 -- 172.21.15.33:0/2992506316 >> 172.21.15.33:0/2992506316 conn(0x7f70b80fd740 msgr2=0x7f70b80ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 -- 172.21.15.33:0/2992506316 shutdown_connections 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 -- 172.21.15.33:0/2992506316 wait complete. 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 Processor -- start 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.913+0000 7f70bf5f9640 1 -- start start 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bf5f9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70b8101cb0 0x7f70b8199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bf5f9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b81076c0 0x7f70b81997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bf5f9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70b8108040 0x7f70b819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bf5f9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8111d00 con 0x7f70b81076c0 2024-09-10T21:13:19.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bf5f9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8111b80 con 0x7f70b8108040 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bf5f9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70b8111e80 con 0x7f70b8101cb0 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b81076c0 0x7f70b81997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b81076c0 0x7f70b81997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56622/0 (socket says 172.21.15.33:56622) 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 -- 172.21.15.33:0/3828135845 learned_addr learned my addr 172.21.15.33:0/3828135845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bdb96640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70b8108040 0x7f70b819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bd395640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70b8101cb0 0x7f70b8199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 -- 172.21.15.33:0/3828135845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70b8101cb0 msgr2=0x7f70b8199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70b8101cb0 0x7f70b8199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 -- 172.21.15.33:0/3828135845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70b8108040 msgr2=0x7f70b819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70b8108040 0x7f70b819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 -- 172.21.15.33:0/3828135845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70b819f330 con 0x7f70b81076c0 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bd395640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70b8101cb0 0x7f70b8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:19.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bcb94640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b81076c0 0x7f70b81997c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f70a800e9f0 tx=0x7f70a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70ae7fc640 1 -- 172.21.15.33:0/3828135845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a800cd80 con 0x7f70b81076c0 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70ae7fc640 1 -- 172.21.15.33:0/3828135845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70a80108e0 con 0x7f70b81076c0 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70b819f620 con 0x7f70b81076c0 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.914+0000 7f70ae7fc640 1 -- 172.21.15.33:0/3828135845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a8018980 con 0x7f70b81076c0 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.915+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70b81a5ce0 con 0x7f70b81076c0 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.917+0000 7f70c19ce740 1 Processor -- start 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.917+0000 7f70ae7fc640 1 -- 172.21.15.33:0/3828135845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f70a8010430 con 0x7f70b81076c0 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.917+0000 7f70c19ce740 1 -- start start 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff66800 0x55bc5ff72250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff74b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70c19ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bc5fedce60 con 0x55bc5ff72790 2024-09-10T21:13:19.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70c19ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc5fecc290 con 0x55bc5ff65ee0 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70c19ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc5fed00a0 con 0x55bc5ff66800 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bcb94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff74b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bd395640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff66800 0x55bc5ff72250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff662c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58112/0 (socket says 172.21.15.33:58112) 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 -- 172.21.15.33:0/1821127527 learned_addr learned my addr 172.21.15.33:0/1821127527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bd395640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff66800 0x55bc5ff72250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59350/0 (socket says 172.21.15.33:59350) 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 -- 172.21.15.33:0/1821127527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff66800 msgr2=0x55bc5ff72250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 --2- 172.21.15.33:0/1821127527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff66800 0x55bc5ff72250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 -- 172.21.15.33:0/1821127527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 msgr2=0x55bc5ff74b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 --2- 172.21.15.33:0/1821127527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff74b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bdb96640 1 -- 172.21.15.33:0/1821127527 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc5fc2a930 con 0x55bc5ff65ee0 2024-09-10T21:13:19.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70ae7fc640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f70a40b52e0 0x7f70a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.918+0000 7f70bd395640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f70a40b52e0 0x7f70a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70bdb96640 1 --2- 172.21.15.33:0/1821127527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff662c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009fd0 tx=0x7f70b00118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6efb047207d75909 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70ae7fc640 1 -- 172.21.15.33:0/3828135845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f70a80e5f70 con 0x7f70b81076c0 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70a37fe640 1 -- 172.21.15.33:0/1821127527 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0004d20 con 0x55bc5ff65ee0 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70a37fe640 1 -- 172.21.15.33:0/1821127527 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b0004050 con 0x55bc5ff65ee0 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70a37fe640 1 -- 172.21.15.33:0/1821127527 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0005220 con 0x55bc5ff65ee0 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70c19ce740 1 -- 172.21.15.33:0/1821127527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 msgr2=0x55bc5ff662c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70bd395640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f70a40b52e0 0x7f70a40b77a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f70b4031440 tx=0x7f70b4005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.919+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1821127527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff662c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009fd0 tx=0x7f70b00118d0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- 172.21.15.33:0/1821127527 shutdown_connections 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1821127527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff74b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1821127527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff66800 0x55bc5ff72250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1821127527 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff662c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- 172.21.15.33:0/1821127527 >> 172.21.15.33:0/1821127527 conn(0x55bc5ff63d10 msgr2=0x55bc5ff64eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- 172.21.15.33:0/1821127527 shutdown_connections 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- 172.21.15.33:0/1821127527 wait complete. 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- start start 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 Processor -- start 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ff701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bc5fd2d890 con 0x55bc5ff66800 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bc5fe66d40 con 0x55bc5ff65ee0 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.920+0000 7f70c19ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc5ff75440 con 0x55bc5ff72790 2024-09-10T21:13:19.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70c19ce740 10 client.0 _ll_register_callbacks cb 0x55bc5fecac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bcb94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58128/0 (socket says 172.21.15.33:58128) 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bcb94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56648/0 (socket says 172.21.15.33:56648) 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 -- 172.21.15.33:0/1090582004 learned_addr learned my addr 172.21.15.33:0/1090582004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bdb96640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ff701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 -- 172.21.15.33:0/1090582004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 msgr2=0x55bc5ff701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ff701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 -- 172.21.15.33:0/1090582004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 msgr2=0x55bc5ff6fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 -- 172.21.15.33:0/1090582004 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5ff7e820 con 0x55bc5ff65ee0 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bcb94640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bdb96640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ff701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:19.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70bd395640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6f740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009900 tx=0x7f70b4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a98c0a12aba51bf3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.921+0000 7f70a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70a0ff9640 1 -- 172.21.15.33:0/1090582004 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b404c070 con 0x55bc5ff65ee0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b0004050 con 0x55bc5ff65ee0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70a0ff9640 1 -- 172.21.15.33:0/1090582004 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f70b4005260 con 0x55bc5ff65ee0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70a0ff9640 1 -- 172.21.15.33:0/1090582004 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f70b4058050 con 0x55bc5ff65ee0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 10 client.26594 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bc5ff811b0 con 0x55bc5ff65ee0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70a0ff9640 1 -- 172.21.15.33:0/1090582004 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b0004050 con 0x55bc5ff65ee0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70a0ff9640 1 -- 172.21.15.33:0/1090582004 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f70b40035f0 con 0x55bc5ff65ee0 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 10 client.26594 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 10 client.26594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:19.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 20 client.26594 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5ff84ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 4 client.26594 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc5fededb0 con 0x55bc5ff82680 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.922+0000 7f70bcb94640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5ff84ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.923+0000 7f70a0ff9640 10 client.26594 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.923+0000 7f70bcb94640 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5ff84ab0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d89652058773cd8 server_cookie=4d29fbf0ca103ac4 in_seq=0 out_seq=0 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70a0ff9640 1 -- 172.21.15.33:0/1090582004 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bc5fededb0 con 0x55bc5ff82680 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70a0ff9640 10 client.26594 handle_command_reply: tid=0 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 client.26594 shutdown 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 msgr2=0x55bc5ff6f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6f740 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009900 tx=0x7f70b4009260 comp rx=0 tx=0).stop 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 shutdown_connections 2024-09-10T21:13:19.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5ff84ab0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ff701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70a0ff9640 0 client.26594 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 --2- 172.21.15.33:0/1090582004 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6f740 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 >> 172.21.15.33:0/1090582004 conn(0x55bc5ff63d10 msgr2=0x55bc5ff62f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 shutdown_connections 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 1 -- 172.21.15.33:0/1090582004 wait complete. 2024-09-10T21:13:19.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.924+0000 7f70c19ce740 20 client.26594 trim_cache size 0 max 16384 2024-09-10T21:13:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 Processor -- start 2024-09-10T21:13:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 -- start start 2024-09-10T21:13:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff6c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81ae4e0 con 0x55bc5ff72790 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81b0020 con 0x55bc5ff66800 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70c19ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81aebb0 con 0x55bc5ff65ee0 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70bdb96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff6c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70bd395640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70bdb96640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff6c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56652/0 (socket says 172.21.15.33:56652) 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70bdb96640 1 -- 172.21.15.33:0/3320734449 learned_addr learned my addr 172.21.15.33:0/3320734449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:19.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.945+0000 7f70bcb94640 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70bdb96640 1 -- 172.21.15.33:0/3320734449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff65ee0 msgr2=0x55bc5ff6eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70bdb96640 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70bdb96640 1 -- 172.21.15.33:0/3320734449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff66800 msgr2=0x55bc5ff6fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70bdb96640 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70bdb96640 1 -- 172.21.15.33:0/3320734449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc5ff63b40 con 0x55bc5ff72790 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70bdb96640 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff6c990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009fd0 tx=0x7f70b0003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ee8a6fd1ccc48e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70a2ffd640 1 -- 172.21.15.33:0/3320734449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0003e20 con 0x55bc5ff72790 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70a2ffd640 1 -- 172.21.15.33:0/3320734449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b0004050 con 0x55bc5ff72790 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.946+0000 7f70a2ffd640 1 -- 172.21.15.33:0/3320734449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b00053b0 con 0x55bc5ff72790 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 -- 172.21.15.33:0/3320734449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 msgr2=0x55bc5ff6c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff6c990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009fd0 tx=0x7f70b0003a30 comp rx=0 tx=0).stop 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 -- 172.21.15.33:0/3320734449 shutdown_connections 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff72790 0x55bc5ff6c990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff66800 0x55bc5ff6fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 --2- 172.21.15.33:0/3320734449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff6eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 -- 172.21.15.33:0/3320734449 >> 172.21.15.33:0/3320734449 conn(0x55bc5ff63d10 msgr2=0x55bc5ff6f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 -- 172.21.15.33:0/3320734449 shutdown_connections 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 -- 172.21.15.33:0/3320734449 wait complete. 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.947+0000 7f70c19ce740 1 -- start start 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 1 Processor -- start 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff76ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff77570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ffec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81af950 con 0x55bc5ff66800 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81af280 con 0x55bc5ff65ee0 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70b81ac4a0 con 0x55bc5ff72790 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70c19ce740 10 client.0 _ll_register_callbacks cb 0x7f70b00093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70bcb94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff77570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70bd395640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff76ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70bdb96640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ffec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70bd395640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff76ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58142/0 (socket says 172.21.15.33:58142) 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70bd395640 1 -- 172.21.15.33:0/4233879730 learned_addr learned my addr 172.21.15.33:0/4233879730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70bcb94640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff77570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56666/0 (socket says 172.21.15.33:56666) 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.948+0000 7f70bdb96640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ffec9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59372/0 (socket says 172.21.15.33:59372) 2024-09-10T21:13:19.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bcb94640 1 -- 172.21.15.33:0/4233879730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 msgr2=0x55bc5ffec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bcb94640 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ffec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bcb94640 1 -- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 msgr2=0x55bc5ff76ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bcb94640 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff76ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bcb94640 1 -- 172.21.15.33:0/4233879730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc5ffed0e0 con 0x55bc5ff66800 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bdb96640 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ffec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bd395640 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff76ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70bcb94640 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff77570 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x55bc5ff65e90 tx=0x7f70a8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce9f45213a736a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70a0ff9640 1 -- 172.21.15.33:0/4233879730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70a80ac080 con 0x55bc5ff66800 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70a0ff9640 1 -- 172.21.15.33:0/4233879730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f70a80b5a30 con 0x55bc5ff66800 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b0004050 con 0x55bc5ff66800 2024-09-10T21:13:19.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70a0ff9640 1 -- 172.21.15.33:0/4233879730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f70a80abea0 con 0x55bc5ff66800 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70c19ce740 10 client.17160 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55bc5fff4560 con 0x55bc5ff66800 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.949+0000 7f70a0ff9640 1 -- 172.21.15.33:0/4233879730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70b0004050 con 0x55bc5ff66800 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70a0ff9640 1 -- 172.21.15.33:0/4233879730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f70a80e9020 con 0x55bc5ff66800 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70c19ce740 10 client.17160 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70c19ce740 10 client.17160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70c19ce740 20 client.17160 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70c19ce740 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5fff6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70c19ce740 4 client.17160 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc5feda860 con 0x55bc5ff82680 2024-09-10T21:13:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70bd395640 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5fff6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70bd395640 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5fff6c80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=995ad9f551ce0a50 server_cookie=7ef3112d140f0a1e in_seq=0 out_seq=0 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.950+0000 7f70a0ff9640 10 client.17160 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70a0ff9640 1 -- 172.21.15.33:0/4233879730 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc5feda860 con 0x55bc5ff82680 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70a0ff9640 10 client.17160 handle_command_reply: tid=0 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 client.17160 shutdown 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 msgr2=0x55bc5ff77570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff77570 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x55bc5ff65e90 tx=0x7f70a8000bc0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 shutdown_connections 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bc5ff82680 0x55bc5fff6c80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc5ff72790 0x55bc5ffec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bc5ff66800 0x55bc5ff77570 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 --2- 172.21.15.33:0/4233879730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bc5ff65ee0 0x55bc5ff76ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70a0ff9640 0 client.17160 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 >> 172.21.15.33:0/4233879730 conn(0x55bc5ff63d10 msgr2=0x55bc5ff6fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 shutdown_connections 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 1 -- 172.21.15.33:0/4233879730 wait complete. 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.953+0000 7f70c19ce740 20 client.17160 trim_cache size 0 max 16384 2024-09-10T21:13:19.961 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:13:19.962 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:13:19.962 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "1ad13a76-9513-4068-b340-0c8aa514c166", 2024-09-10T21:13:19.962 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:13:19.962 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:13:19.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f70a40b52e0 msgr2=0x7f70a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f70a40b52e0 0x7f70a40b77a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f70b4031440 tx=0x7f70b4005b50 comp rx=0 tx=0).stop 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b81076c0 msgr2=0x7f70b81997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b81076c0 0x7f70b81997c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f70a800e9f0 tx=0x7f70a800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 shutdown_connections 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f70a40b52e0 0x7f70a40b77a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f70b8108040 0x7f70b819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b81076c0 0x7f70b81997c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 --2- 172.21.15.33:0/3828135845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70b8101cb0 0x7f70b8199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 >> 172.21.15.33:0/3828135845 conn(0x7f70b80fd740 msgr2=0x7f70b806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:19.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 shutdown_connections 2024-09-10T21:13:19.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:19.954+0000 7f70bf5f9640 1 -- 172.21.15.33:0/3828135845 wait complete. 2024-09-10T21:13:20.627 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:13:20.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:13:20.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:20.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:20.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:20.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:20.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 607568 5.22 MB/sec execute 381 sec latency 74.950 ms 2024-09-10T21:13:20.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:13:20.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.893 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:20.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 -- 172.21.15.33:0/3402774730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 msgr2=0x7f4408104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 --2- 172.21.15.33:0/3402774730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408104080 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f43f8009900 tx=0x7f43f802a410 comp rx=0 tx=0).stop 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 -- 172.21.15.33:0/3402774730 shutdown_connections 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 --2- 172.21.15.33:0/3402774730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f440810d100 0x7f440810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 --2- 172.21.15.33:0/3402774730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44081045c0 0x7f440810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 --2- 172.21.15.33:0/3402774730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408104080 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 -- 172.21.15.33:0/3402774730 >> 172.21.15.33:0/3402774730 conn(0x7f44080fd700 msgr2=0x7f44080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 -- 172.21.15.33:0/3402774730 shutdown_connections 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.386+0000 7f440e1a0640 1 -- 172.21.15.33:0/3402774730 wait complete. 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 Processor -- start 2024-09-10T21:13:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 -- start start 2024-09-10T21:13:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44081045c0 0x7f44081997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f440810d100 0x7f440819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4408112d70 con 0x7f4408103ca0 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4408112bf0 con 0x7f44081045c0 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f440e1a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4408112ef0 con 0x7f440810d100 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f44077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f44077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56680/0 (socket says 172.21.15.33:56680) 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f44077fe640 1 -- 172.21.15.33:0/4143633998 learned_addr learned my addr 172.21.15.33:0/4143633998 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f4406ffd640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44081045c0 0x7f44081997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f4407fff640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f440810d100 0x7f440819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f44077fe640 1 -- 172.21.15.33:0/4143633998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f440810d100 msgr2=0x7f440819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f44077fe640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f440810d100 0x7f440819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f44077fe640 1 -- 172.21.15.33:0/4143633998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44081045c0 msgr2=0x7f44081997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.387+0000 7f44077fe640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44081045c0 0x7f44081997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.388+0000 7f44077fe640 1 -- 172.21.15.33:0/4143633998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f440819f330 con 0x7f4408103ca0 2024-09-10T21:13:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.388+0000 7f44077fe640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408199280 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f43f8005940 tx=0x7f43f80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.388+0000 7f4404ff9640 1 -- 172.21.15.33:0/4143633998 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f803f030 con 0x7f4408103ca0 2024-09-10T21:13:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.388+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f440819f5c0 con 0x7f4408103ca0 2024-09-10T21:13:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.388+0000 7f4404ff9640 1 -- 172.21.15.33:0/4143633998 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43f80044c0 con 0x7f4408103ca0 2024-09-10T21:13:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.388+0000 7f4404ff9640 1 -- 172.21.15.33:0/4143633998 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43f8035070 con 0x7f4408103ca0 2024-09-10T21:13:21.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.388+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44081a5ce0 con 0x7f4408103ca0 2024-09-10T21:13:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 Processor -- start 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 -- start start 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd20a0 0x563968bd2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd29c0 0x563968bde410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bde950 0x563968be0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4404ff9640 1 -- 172.21.15.33:0/4143633998 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f43f803b960 con 0x7f4408103ca0 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563968b48e50 con 0x563968bd29c0 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563968b38280 con 0x563968bde950 2024-09-10T21:13:21.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4410575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563968b3c090 con 0x563968bd20a0 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bde950 0x563968be0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f44077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd29c0 0x563968bde410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bde950 0x563968be0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58174/0 (socket says 172.21.15.33:58174) 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4406ffd640 1 -- 172.21.15.33:0/1318722117 learned_addr learned my addr 172.21.15.33:0/1318722117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.391+0000 7f4407fff640 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd20a0 0x563968bd2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f4406ffd640 1 -- 172.21.15.33:0/1318722117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd20a0 msgr2=0x563968bd2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f4406ffd640 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd20a0 0x563968bd2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f4406ffd640 1 -- 172.21.15.33:0/1318722117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd29c0 msgr2=0x563968bde410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f4406ffd640 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd29c0 0x563968bde410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f4406ffd640 1 -- 172.21.15.33:0/1318722117 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563968896650 con 0x563968bde950 2024-09-10T21:13:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f4406ffd640 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bde950 0x563968be0d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009c50 tx=0x7f43fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32cabddcedc3b048 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f43edffb640 1 -- 172.21.15.33:0/1318722117 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc004220 con 0x563968bde950 2024-09-10T21:13:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f43edffb640 1 -- 172.21.15.33:0/1318722117 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43fc0043c0 con 0x563968bde950 2024-09-10T21:13:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f43edffb640 1 -- 172.21.15.33:0/1318722117 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc004d20 con 0x563968bde950 2024-09-10T21:13:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.392+0000 7f4404ff9640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f43f00b52e0 0x7f43f00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4407fff640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f43f00b52e0 0x7f43f00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4404ff9640 1 -- 172.21.15.33:0/4143633998 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f43f81061c0 con 0x7f4408103ca0 2024-09-10T21:13:21.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 -- 172.21.15.33:0/1318722117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bde950 msgr2=0x563968be0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4407fff640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f43f00b52e0 0x7f43f00b77a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4400005fd0 tx=0x7f4400009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bde950 0x563968be0d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43fc009c50 tx=0x7f43fc011810 comp rx=0 tx=0).stop 2024-09-10T21:13:21.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 -- 172.21.15.33:0/1318722117 shutdown_connections 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bde950 0x563968be0d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd29c0 0x563968bde410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 --2- 172.21.15.33:0/1318722117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd20a0 0x563968bd2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 -- 172.21.15.33:0/1318722117 >> 172.21.15.33:0/1318722117 conn(0x563968bcfed0 msgr2=0x563968bd1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 -- 172.21.15.33:0/1318722117 shutdown_connections 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.393+0000 7f4410575740 1 -- 172.21.15.33:0/1318722117 wait complete. 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 -- start start 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 Processor -- start 2024-09-10T21:13:21.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bde200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968bdbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563968999700 con 0x563968bde950 2024-09-10T21:13:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563968ad3110 con 0x563968bd20a0 2024-09-10T21:13:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563968be15d0 con 0x563968bd29c0 2024-09-10T21:13:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4410575740 10 client.0 _ll_register_callbacks cb 0x563968aec990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:21.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59412/0 (socket says 172.21.15.33:59412) 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4406ffd640 1 -- 172.21.15.33:0/1696174425 learned_addr learned my addr 172.21.15.33:0/1696174425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.394+0000 7f4407fff640 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968bdbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.395+0000 7f44077fe640 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bde200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.395+0000 7f4406ffd640 1 -- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 msgr2=0x563968bde200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.395+0000 7f4406ffd640 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bde200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.395+0000 7f4406ffd640 1 -- 172.21.15.33:0/1696174425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 msgr2=0x563968bdbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.395+0000 7f4406ffd640 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968bdbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.395+0000 7f4406ffd640 1 -- 172.21.15.33:0/1696174425 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563968b00c40 con 0x563968bd29c0 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.395+0000 7f4406ffd640 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdb650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43fc01b040 tx=0x7f43fc019a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a6673458e9c4c82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f43e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f43e6ffd640 1 -- 172.21.15.33:0/1696174425 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc027030 con 0x563968bd29c0 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f43e6ffd640 1 -- 172.21.15.33:0/1696174425 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f43fc0046c0 con 0x563968bd29c0 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43fc0043c0 con 0x563968bd29c0 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f43e6ffd640 1 -- 172.21.15.33:0/1696174425 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f43fc01d040 con 0x563968bd29c0 2024-09-10T21:13:21.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 10 client.26566 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563968becf20 con 0x563968bd29c0 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f43e6ffd640 1 -- 172.21.15.33:0/1696174425 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43fc0043c0 con 0x563968bd29c0 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f43e6ffd640 1 -- 172.21.15.33:0/1696174425 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f43fc023a30 con 0x563968bd29c0 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 10 client.26566 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 10 client.26566 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 20 client.26566 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968bee320 0x563968bf0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 4 client.26566 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.396+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563968b4ada0 con 0x563968bee320 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.397+0000 7f44077fe640 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968bee320 0x563968bf0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.397+0000 7f44077fe640 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968bee320 0x563968bf0710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f622e02101c049 server_cookie=14d24984a06864cf in_seq=0 out_seq=0 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.397+0000 7f43e6ffd640 10 client.26566 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f43e6ffd640 1 -- 172.21.15.33:0/1696174425 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563968b4ada0 con 0x563968bee320 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f43e6ffd640 10 client.26566 handle_command_reply: tid=0 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 client.26566 shutdown 2024-09-10T21:13:21.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 msgr2=0x563968bdb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdb650 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f43fc01b040 tx=0x7f43fc019a20 comp rx=0 tx=0).stop 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 shutdown_connections 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968bee320 0x563968bf0710 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968bdbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdb650 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f43e6ffd640 0 client.26566 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 --2- 172.21.15.33:0/1696174425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bde200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 >> 172.21.15.33:0/1696174425 conn(0x563968bcfed0 msgr2=0x563968bd5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 shutdown_connections 2024-09-10T21:13:21.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 1 -- 172.21.15.33:0/1696174425 wait complete. 2024-09-10T21:13:21.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.398+0000 7f4410575740 20 client.26566 trim_cache size 0 max 16384 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 Processor -- start 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 -- start start 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bcf710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968be0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44081ac180 con 0x563968bde950 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44081adcc0 con 0x563968bd20a0 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4410575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44081ac850 con 0x563968bd29c0 2024-09-10T21:13:21.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968be0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f44077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bcf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f44077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bcf710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58184/0 (socket says 172.21.15.33:58184) 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f44077fe640 1 -- 172.21.15.33:0/692833449 learned_addr learned my addr 172.21.15.33:0/692833449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.419+0000 7f4407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968be0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56706/0 (socket says 172.21.15.33:56706) 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.420+0000 7f4406ffd640 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.420+0000 7f4407fff640 1 -- 172.21.15.33:0/692833449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 msgr2=0x563968bdfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.420+0000 7f4407fff640 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdfd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.420+0000 7f4407fff640 1 -- 172.21.15.33:0/692833449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 msgr2=0x563968bcf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.420+0000 7f4407fff640 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bcf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.420+0000 7f4407fff640 1 -- 172.21.15.33:0/692833449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563968bf0a80 con 0x563968bde950 2024-09-10T21:13:21.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.420+0000 7f4407fff640 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968be0280 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4400003e40 tx=0x7f440001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c6956859dbafa20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.422+0000 7f43ed7fa640 1 -- 172.21.15.33:0/692833449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440000f540 con 0x563968bde950 2024-09-10T21:13:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.422+0000 7f43ed7fa640 1 -- 172.21.15.33:0/692833449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440000fb90 con 0x563968bde950 2024-09-10T21:13:21.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.422+0000 7f43ed7fa640 1 -- 172.21.15.33:0/692833449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440001ebd0 con 0x563968bde950 2024-09-10T21:13:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 -- 172.21.15.33:0/692833449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 msgr2=0x563968be0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968be0280 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4400003e40 tx=0x7f440001b000 comp rx=0 tx=0).stop 2024-09-10T21:13:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 -- 172.21.15.33:0/692833449 shutdown_connections 2024-09-10T21:13:21.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bde950 0x563968be0280 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bd29c0 0x563968bdfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 --2- 172.21.15.33:0/692833449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd20a0 0x563968bcf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 -- 172.21.15.33:0/692833449 >> 172.21.15.33:0/692833449 conn(0x563968bcfed0 msgr2=0x563968bdd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 -- 172.21.15.33:0/692833449 shutdown_connections 2024-09-10T21:13:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 -- 172.21.15.33:0/692833449 wait complete. 2024-09-10T21:13:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.423+0000 7f4410575740 1 -- start start 2024-09-10T21:13:21.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 1 Processor -- start 2024-09-10T21:13:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd20a0 0x563968be6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd29c0 0x563968be7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bde950 0x563968bd7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44081ad5f0 con 0x563968bd20a0 2024-09-10T21:13:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44081acf20 con 0x563968bd29c0 2024-09-10T21:13:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44081aa160 con 0x563968bde950 2024-09-10T21:13:21.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4410575740 10 client.0 _ll_register_callbacks cb 0x7f43f8109b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f44077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd20a0 0x563968be6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4406ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd29c0 0x563968be7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f44077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd20a0 0x563968be6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56712/0 (socket says 172.21.15.33:56712) 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f44077fe640 1 -- 172.21.15.33:0/607314100 learned_addr learned my addr 172.21.15.33:0/607314100 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4407fff640 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bde950 0x563968bd7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4406ffd640 1 -- 172.21.15.33:0/607314100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bde950 msgr2=0x563968bd7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4406ffd640 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bde950 0x563968bd7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4406ffd640 1 -- 172.21.15.33:0/607314100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd20a0 msgr2=0x563968be6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4406ffd640 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd20a0 0x563968be6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.424+0000 7f4406ffd640 1 -- 172.21.15.33:0/607314100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563968bd8250 con 0x563968bd29c0 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4406ffd640 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd29c0 0x563968be7260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43fc023e00 tx=0x7f43fc009640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efbae8edf13ce6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:21.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f43e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f43e6ffd640 1 -- 172.21.15.33:0/607314100 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43fc027040 con 0x563968bd29c0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440000fb90 con 0x563968bd29c0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f43e6ffd640 1 -- 172.21.15.33:0/607314100 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f43fc019a90 con 0x563968bd29c0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f43e6ffd640 1 -- 172.21.15.33:0/607314100 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f43fc040050 con 0x563968bd29c0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 10 client.26618 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563968bd9560 con 0x563968bd29c0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f43e6ffd640 1 -- 172.21.15.33:0/607314100 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f440000fb90 con 0x563968bd29c0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f43e6ffd640 1 -- 172.21.15.33:0/607314100 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f43fc02e120 con 0x563968bd29c0 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 10 client.26618 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 10 client.26618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 20 client.26618 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968c64d30 0x563968c67150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:21.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 4 client.26618 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.425+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563968b46850 con 0x563968c64d30 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f44077fe640 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968c64d30 0x563968c67150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f44077fe640 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968c64d30 0x563968c67150 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28bcda866a6aab77 server_cookie=2d7445a2a33efa65 in_seq=0 out_seq=0 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f43e6ffd640 10 client.26618 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f43e6ffd640 1 -- 172.21.15.33:0/607314100 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x563968b46850 con 0x563968c64d30 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f43e6ffd640 10 client.26618 handle_command_reply: tid=0 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f4410575740 1 client.26618 shutdown 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd29c0 msgr2=0x563968be7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.426+0000 7f4410575740 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd29c0 0x563968be7260 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43fc023e00 tx=0x7f43fc009640 comp rx=0 tx=0).stop 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 shutdown_connections 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563968c64d30 0x563968c67150 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563968bde950 0x563968bd7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563968bd29c0 0x563968be7260 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f43e6ffd640 0 client.26618 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 --2- 172.21.15.33:0/607314100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563968bd20a0 0x563968be6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 >> 172.21.15.33:0/607314100 conn(0x563968bcfed0 msgr2=0x563968bd5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 shutdown_connections 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 1 -- 172.21.15.33:0/607314100 wait complete. 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f4410575740 20 client.26618 trim_cache size 0 max 16384 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:13:21.435 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "1ad13a76-9513-4068-b340-0c8aa514c166", 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.436 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:13:21.437 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.439 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:13:21.440 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:13:21.442 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:13:21.444 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:13:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f43f00b52e0 msgr2=0x7f43f00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f43f00b52e0 0x7f43f00b77a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4400005fd0 tx=0x7f4400009500 comp rx=0 tx=0).stop 2024-09-10T21:13:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 msgr2=0x7f4408199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408199280 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f43f8005940 tx=0x7f43f80040c0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 shutdown_connections 2024-09-10T21:13:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f43f00b52e0 0x7f43f00b77a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f440810d100 0x7f440819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44081045c0 0x7f44081997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 --2- 172.21.15.33:0/4143633998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4408103ca0 0x7f4408199280 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.427+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 >> 172.21.15.33:0/4143633998 conn(0x7f44080fd700 msgr2=0x7f44080fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.428+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 shutdown_connections 2024-09-10T21:13:21.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:21.428+0000 7f440e1a0640 1 -- 172.21.15.33:0/4143633998 wait complete. 2024-09-10T21:13:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:21.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:21.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:13:21.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:21 smithi033 ceph-mon[30860]: pgmap v627: 97 pgs: 97 active+clean; 232 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-09-10T21:13:21.746 INFO:tasks.workunit.client.0.smithi033.stdout: 1 609264 5.23 MB/sec execute 382 sec latency 40.030 ms 2024-09-10T21:13:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:13:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:21 smithi104 ceph-mon[35325]: pgmap v627: 97 pgs: 97 active+clean; 232 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:13:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:13:21.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:21 smithi174 ceph-mon[35635]: pgmap v627: 97 pgs: 97 active+clean; 232 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 895 KiB/s rd, 1.1 MiB/s wr, 126 op/s 2024-09-10T21:13:22.265 INFO:tasks.cephfs.filesystem:scrub status for tag:1ad13a76-9513-4068-b340-0c8aa514c166 - {'path': '/', 'tag': '1ad13a76-9513-4068-b340-0c8aa514c166', 'options': 'recursive,force'} 2024-09-10T21:13:22.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 611334 5.25 MB/sec execute 383 sec latency 11.033 ms 2024-09-10T21:13:23.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:23 smithi033 ceph-mon[30860]: pgmap v628: 97 pgs: 97 active+clean; 233 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1015 KiB/s wr, 123 op/s 2024-09-10T21:13:23.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 613667 5.26 MB/sec execute 384 sec latency 23.071 ms 2024-09-10T21:13:23.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:23 smithi104 ceph-mon[35325]: pgmap v628: 97 pgs: 97 active+clean; 233 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1015 KiB/s wr, 123 op/s 2024-09-10T21:13:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:23 smithi174 ceph-mon[35635]: pgmap v628: 97 pgs: 97 active+clean; 233 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 938 KiB/s rd, 1015 KiB/s wr, 123 op/s 2024-09-10T21:13:24.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 615415 5.27 MB/sec execute 385 sec latency 10.355 ms 2024-09-10T21:13:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:25 smithi033 ceph-mon[30860]: pgmap v629: 97 pgs: 97 active+clean; 236 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-10T21:13:25.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 618210 5.30 MB/sec execute 386 sec latency 11.541 ms 2024-09-10T21:13:25.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:25 smithi104 ceph-mon[35325]: pgmap v629: 97 pgs: 97 active+clean; 236 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-10T21:13:25.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:25 smithi174 ceph-mon[35635]: pgmap v629: 97 pgs: 97 active+clean; 236 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-09-10T21:13:26.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 620319 5.31 MB/sec execute 387 sec latency 10.179 ms 2024-09-10T21:13:26.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:13:26.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:13:26.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:26 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:13:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:13:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:13:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:26 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:13:26.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:13:26.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:13:26.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:26 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:13:27.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 622011 5.33 MB/sec execute 388 sec latency 11.116 ms 2024-09-10T21:13:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:27 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:13:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:27 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:13:27.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:27 smithi104 ceph-mon[35325]: pgmap v630: 97 pgs: 97 active+clean; 239 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-10T21:13:27.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:27 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:13:27.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:27 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:13:27.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:27 smithi174 ceph-mon[35635]: pgmap v630: 97 pgs: 97 active+clean; 239 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-10T21:13:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:27 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:13:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:27 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:13:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:27 smithi033 ceph-mon[30860]: pgmap v630: 97 pgs: 97 active+clean; 239 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 212 op/s 2024-09-10T21:13:28.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 624425 5.34 MB/sec execute 389 sec latency 17.883 ms 2024-09-10T21:13:29.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:29 smithi174 ceph-mon[35635]: pgmap v631: 97 pgs: 97 active+clean; 239 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-09-10T21:13:29.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 626003 5.35 MB/sec execute 390 sec latency 10.143 ms 2024-09-10T21:13:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:29 smithi104 ceph-mon[35325]: pgmap v631: 97 pgs: 97 active+clean; 239 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-09-10T21:13:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:29 smithi033 ceph-mon[30860]: pgmap v631: 97 pgs: 97 active+clean; 239 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.4 MiB/s wr, 220 op/s 2024-09-10T21:13:30.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 628764 5.38 MB/sec execute 391 sec latency 9.682 ms 2024-09-10T21:13:31.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 630645 5.39 MB/sec execute 392 sec latency 13.406 ms 2024-09-10T21:13:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:31 smithi104 ceph-mon[35325]: pgmap v632: 97 pgs: 97 active+clean; 243 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2024-09-10T21:13:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:31 smithi174 ceph-mon[35635]: pgmap v632: 97 pgs: 97 active+clean; 243 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2024-09-10T21:13:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:31 smithi033 ceph-mon[30860]: pgmap v632: 97 pgs: 97 active+clean; 243 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 284 op/s 2024-09-10T21:13:32.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 632582 5.41 MB/sec execute 393 sec latency 55.828 ms 2024-09-10T21:13:33.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 635045 5.42 MB/sec execute 394 sec latency 17.610 ms 2024-09-10T21:13:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:33 smithi104 ceph-mon[35325]: pgmap v633: 97 pgs: 97 active+clean; 244 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2024-09-10T21:13:33.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:33 smithi174 ceph-mon[35635]: pgmap v633: 97 pgs: 97 active+clean; 244 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2024-09-10T21:13:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:33 smithi033 ceph-mon[30860]: pgmap v633: 97 pgs: 97 active+clean; 244 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.6 MiB/s wr, 294 op/s 2024-09-10T21:13:34.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 636689 5.43 MB/sec execute 395 sec latency 10.257 ms 2024-09-10T21:13:35.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 639471 5.46 MB/sec execute 396 sec latency 19.542 ms 2024-09-10T21:13:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:35 smithi104 ceph-mon[35325]: pgmap v634: 97 pgs: 97 active+clean; 245 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-09-10T21:13:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:35 smithi174 ceph-mon[35635]: pgmap v634: 97 pgs: 97 active+clean; 245 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-09-10T21:13:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:35 smithi033 ceph-mon[30860]: pgmap v634: 97 pgs: 97 active+clean; 245 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.0 MiB/s wr, 313 op/s 2024-09-10T21:13:36.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 641592 5.46 MB/sec execute 397 sec latency 10.726 ms 2024-09-10T21:13:37.729 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:37 smithi033 ceph-mon[30860]: pgmap v635: 97 pgs: 97 active+clean; 248 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-09-10T21:13:37.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 643343 5.48 MB/sec execute 398 sec latency 22.816 ms 2024-09-10T21:13:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:37 smithi104 ceph-mon[35325]: pgmap v635: 97 pgs: 97 active+clean; 248 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-09-10T21:13:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:37 smithi174 ceph-mon[35635]: pgmap v635: 97 pgs: 97 active+clean; 248 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 302 op/s 2024-09-10T21:13:38.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 645820 5.50 MB/sec execute 399 sec latency 11.887 ms 2024-09-10T21:13:39.747 INFO:tasks.workunit.client.0.smithi033.stdout: 1 647567 5.51 MB/sec execute 400 sec latency 16.971 ms 2024-09-10T21:13:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:39 smithi104 ceph-mon[35325]: pgmap v636: 97 pgs: 97 active+clean; 249 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-09-10T21:13:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:39 smithi174 ceph-mon[35635]: pgmap v636: 97 pgs: 97 active+clean; 249 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-09-10T21:13:39.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:39 smithi033 ceph-mon[30860]: pgmap v636: 97 pgs: 97 active+clean; 249 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 279 op/s 2024-09-10T21:13:40.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 650203 5.54 MB/sec execute 401 sec latency 16.952 ms 2024-09-10T21:13:41.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 652346 5.54 MB/sec execute 402 sec latency 17.514 ms 2024-09-10T21:13:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:41 smithi104 ceph-mon[35325]: pgmap v637: 97 pgs: 97 active+clean; 250 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2024-09-10T21:13:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:41 smithi174 ceph-mon[35635]: pgmap v637: 97 pgs: 97 active+clean; 250 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2024-09-10T21:13:41.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:41 smithi033 ceph-mon[30860]: pgmap v637: 97 pgs: 97 active+clean; 250 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.7 MiB/s wr, 298 op/s 2024-09-10T21:13:42.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 654126 5.56 MB/sec execute 403 sec latency 20.464 ms 2024-09-10T21:13:43.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 656813 5.58 MB/sec execute 404 sec latency 16.418 ms 2024-09-10T21:13:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:43 smithi104 ceph-mon[35325]: pgmap v638: 97 pgs: 97 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-09-10T21:13:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:43 smithi174 ceph-mon[35635]: pgmap v638: 97 pgs: 97 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-09-10T21:13:43.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:43 smithi033 ceph-mon[30860]: pgmap v638: 97 pgs: 97 active+clean; 254 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 1.7 MiB/s wr, 294 op/s 2024-09-10T21:13:44.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 658476 5.59 MB/sec execute 405 sec latency 17.800 ms 2024-09-10T21:13:45.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 660902 5.61 MB/sec execute 406 sec latency 11.963 ms 2024-09-10T21:13:45.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:45 smithi104 ceph-mon[35325]: pgmap v639: 97 pgs: 97 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-09-10T21:13:45.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:45 smithi174 ceph-mon[35635]: pgmap v639: 97 pgs: 97 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-09-10T21:13:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:45 smithi033 ceph-mon[30860]: pgmap v639: 97 pgs: 97 active+clean; 255 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.7 MiB/s wr, 297 op/s 2024-09-10T21:13:46.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 663300 5.62 MB/sec execute 407 sec latency 10.262 ms 2024-09-10T21:13:47.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664190 5.63 MB/sec execute 408 sec latency 654.390 ms 2024-09-10T21:13:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:47 smithi104 ceph-mon[35325]: pgmap v640: 97 pgs: 97 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 323 op/s 2024-09-10T21:13:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:47 smithi174 ceph-mon[35635]: pgmap v640: 97 pgs: 97 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 323 op/s 2024-09-10T21:13:47.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:47 smithi033 ceph-mon[30860]: pgmap v640: 97 pgs: 97 active+clean; 258 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.5 MiB/s wr, 323 op/s 2024-09-10T21:13:48.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664190 5.61 MB/sec execute 409 sec latency 1654.541 ms 2024-09-10T21:13:49.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664190 5.60 MB/sec execute 410 sec latency 2654.657 ms 2024-09-10T21:13:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:49 smithi104 ceph-mon[35325]: pgmap v641: 97 pgs: 97 active+clean; 259 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 300 op/s 2024-09-10T21:13:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:49 smithi174 ceph-mon[35635]: pgmap v641: 97 pgs: 97 active+clean; 259 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 300 op/s 2024-09-10T21:13:49.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:49 smithi033 ceph-mon[30860]: pgmap v641: 97 pgs: 97 active+clean; 259 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 300 op/s 2024-09-10T21:13:50.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664190 5.58 MB/sec execute 411 sec latency 3654.727 ms 2024-09-10T21:13:51.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664190 5.57 MB/sec execute 412 sec latency 4654.840 ms 2024-09-10T21:13:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:51 smithi104 ceph-mon[35325]: pgmap v642: 97 pgs: 97 active+clean; 261 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-09-10T21:13:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:51 smithi174 ceph-mon[35635]: pgmap v642: 97 pgs: 97 active+clean; 261 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-09-10T21:13:51.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:51 smithi033 ceph-mon[30860]: pgmap v642: 97 pgs: 97 active+clean; 261 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 291 op/s 2024-09-10T21:13:52.266 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:13:52.551 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:52.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 664997 5.56 MB/sec execute 413 sec latency 5007.478 ms 2024-09-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 -- 172.21.15.33:0/277522188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a41076a0 msgr2=0x7fd7a4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/277522188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a41076a0 0x7fd7a4107ae0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fd798009900 tx=0x7fd79802a410 comp rx=0 tx=0).stop 2024-09-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 -- 172.21.15.33:0/277522188 shutdown_connections 2024-09-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/277522188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7a4108020 0x7fd7a410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/277522188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a41076a0 0x7fd7a4107ae0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/277522188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7a4101c90 0x7fd7a4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 -- 172.21.15.33:0/277522188 >> 172.21.15.33:0/277522188 conn(0x7fd7a40fd700 msgr2=0x7fd7a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 -- 172.21.15.33:0/277522188 shutdown_connections 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.136+0000 7fd7abdb1640 1 -- 172.21.15.33:0/277522188 wait complete. 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 Processor -- start 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 -- start start 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7a4101c90 0x7fd7a419b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7a41076a0 0x7fd7a419b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a4108020 0x7fd7a41a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4111ce0 con 0x7fd7a4108020 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4111b60 con 0x7fd7a4101c90 2024-09-10T21:13:53.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7abdb1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4111e60 con 0x7fd7a41076a0 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7aa34e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a4108020 0x7fd7a41a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7aa34e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a4108020 0x7fd7a41a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39562/0 (socket says 172.21.15.33:39562) 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7aa34e640 1 -- 172.21.15.33:0/3177336680 learned_addr learned my addr 172.21.15.33:0/3177336680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.137+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7a4101c90 0x7fd7a419b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7a934c640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7a41076a0 0x7fd7a419b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7aa34e640 1 -- 172.21.15.33:0/3177336680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7a41076a0 msgr2=0x7fd7a419b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7aa34e640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7a41076a0 0x7fd7a419b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7aa34e640 1 -- 172.21.15.33:0/3177336680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7a4101c90 msgr2=0x7fd7a419b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7aa34e640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7a4101c90 0x7fd7a419b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7aa34e640 1 -- 172.21.15.33:0/3177336680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a41a1540 con 0x7fd7a4108020 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7a4101c90 0x7fd7a419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:53.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7aa34e640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a4108020 0x7fd7a41a0d70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00eb10 tx=0x7fd79c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd796ffd640 1 -- 172.21.15.33:0/3177336680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c004b20 con 0x7fd7a4108020 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd796ffd640 1 -- 172.21.15.33:0/3177336680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd79c004cc0 con 0x7fd7a4108020 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a41a1830 con 0x7fd7a4108020 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd796ffd640 1 -- 172.21.15.33:0/3177336680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c010670 con 0x7fd7a4108020 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.138+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a41a7ef0 con 0x7fd7a4108020 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.141+0000 7fd796ffd640 1 -- 172.21.15.33:0/3177336680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd79c005020 con 0x7fd7a4108020 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.141+0000 7fd7ae186740 1 Processor -- start 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.141+0000 7fd7ae186740 1 -- start start 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035480 0x558105035860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x5581050417f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105041d30 0x558105044120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7ae186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558104fabe50 con 0x558105035480 2024-09-10T21:13:53.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7ae186740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558104f9b280 con 0x558105041d30 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7ae186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558104f9f090 con 0x558105035da0 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a934c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035480 0x558105035860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x5581050417f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x5581050417f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:41634/0 (socket says 172.21.15.33:41634) 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a934c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035480 0x558105035860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39570/0 (socket says 172.21.15.33:39570) 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/5211162 learned_addr learned my addr 172.21.15.33:0/5211162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a934c640 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105041d30 0x558105044120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd796ffd640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7900b52e0 0x7fd7900b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/5211162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105041d30 msgr2=0x558105044120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105041d30 0x558105044120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/5211162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035480 msgr2=0x558105035860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035480 0x558105035860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/5211162 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558104cf9650 con 0x558105035da0 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd796ffd640 1 -- 172.21.15.33:0/3177336680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd79c0e6130 con 0x7fd7a4108020 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.142+0000 7fd7aa34e640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7900b52e0 0x7fd7900b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.143+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x5581050417f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00115f0 tx=0x7fd7a0011ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7856535cf5cad0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.143+0000 7fd78ffff640 1 -- 172.21.15.33:0/5211162 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a001e070 con 0x558105035da0 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.143+0000 7fd78ffff640 1 -- 172.21.15.33:0/5211162 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a000faf0 con 0x558105035da0 2024-09-10T21:13:53.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.143+0000 7fd78ffff640 1 -- 172.21.15.33:0/5211162 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0019af0 con 0x558105035da0 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 -- 172.21.15.33:0/5211162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 msgr2=0x5581050417f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7aa34e640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7900b52e0 0x7fd7900b77a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0049d0 tx=0x7fd79c000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x5581050417f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd7a00115f0 tx=0x7fd7a0011ab0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 -- 172.21.15.33:0/5211162 shutdown_connections 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105041d30 0x558105044120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x5581050417f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 --2- 172.21.15.33:0/5211162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035480 0x558105035860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 -- 172.21.15.33:0/5211162 >> 172.21.15.33:0/5211162 conn(0x5581050332b0 msgr2=0x558105034450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 -- 172.21.15.33:0/5211162 shutdown_connections 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 -- 172.21.15.33:0/5211162 wait complete. 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 -- start start 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.144+0000 7fd7ae186740 1 Processor -- start 2024-09-10T21:13:53.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105041600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x55810503ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x55810503f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7ae186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558104dfc700 con 0x558105041d30 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7ae186740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558104f36110 con 0x558105035480 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7ae186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581050449b0 con 0x558105035da0 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7ae186740 10 client.0 _ll_register_callbacks cb 0x558104f4f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105041600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105041600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59834/0 (socket says 172.21.15.33:59834) 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 -- 172.21.15.33:0/447511079 learned_addr learned my addr 172.21.15.33:0/447511079 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x55810503f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x55810503ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 -- 172.21.15.33:0/447511079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 msgr2=0x55810503ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x55810503ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 -- 172.21.15.33:0/447511079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 msgr2=0x55810503f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x55810503f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 -- 172.21.15.33:0/447511079 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558104f63c40 con 0x558105035480 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd7a934c640 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105041600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd798009900 tx=0x7fd798005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d04e1c7c2f2b1600 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.145+0000 7fd78d7fa640 1 -- 172.21.15.33:0/447511079 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798035070 con 0x558105035480 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd78d7fa640 1 -- 172.21.15.33:0/447511079 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd798004230 con 0x558105035480 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7a000faf0 con 0x558105035480 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd78d7fa640 1 -- 172.21.15.33:0/447511079 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd798053050 con 0x558105035480 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 10 client.26630 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5581050502f0 con 0x558105035480 2024-09-10T21:13:53.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd78d7fa640 1 -- 172.21.15.33:0/447511079 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a000faf0 con 0x558105035480 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd78d7fa640 1 -- 172.21.15.33:0/447511079 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd79803d450 con 0x558105035480 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 10 client.26630 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 10 client.26630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 20 client.26630 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558105051750 0x558105053b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 4 client.26630 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558104fadda0 con 0x558105051750 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.146+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558105051750 0x558105053b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.147+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558105051750 0x558105053b60 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436587a16fc1e344 server_cookie=6091cf6c074be485 in_seq=0 out_seq=0 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.147+0000 7fd78d7fa640 10 client.26630 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.147+0000 7fd78d7fa640 1 -- 172.21.15.33:0/447511079 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558104fadda0 con 0x558105051750 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.147+0000 7fd78d7fa640 10 client.26630 handle_command_reply: tid=0 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 client.26630 shutdown 2024-09-10T21:13:53.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 msgr2=0x558105041600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105041600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd798009900 tx=0x7fd798005b50 comp rx=0 tx=0).stop 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 shutdown_connections 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558105051750 0x558105053b60 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x55810503f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035da0 0x55810503ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd78d7fa640 0 client.26630 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 --2- 172.21.15.33:0/447511079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105041600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 >> 172.21.15.33:0/447511079 conn(0x5581050332b0 msgr2=0x558105038e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 shutdown_connections 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 1 -- 172.21.15.33:0/447511079 wait complete. 2024-09-10T21:13:53.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.148+0000 7fd7ae186740 20 client.26630 trim_cache size 0 max 16384 2024-09-10T21:13:53.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.167+0000 7fd7ae186740 1 Processor -- start 2024-09-10T21:13:53.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.167+0000 7fd7ae186740 1 -- start start 2024-09-10T21:13:53.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105032bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035da0 0x558105043120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105041d30 0x558105043660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7ae186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b06c0 con 0x558105035da0 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7ae186740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b2200 con 0x558105035480 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7ae186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b0d90 con 0x558105041d30 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035da0 0x558105043120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035da0 0x558105043120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39588/0 (socket says 172.21.15.33:39588) 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/2023996136 learned_addr learned my addr 172.21.15.33:0/2023996136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a934c640 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105032bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a934c640 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105041d30 0x558105043660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/2023996136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105041d30 msgr2=0x558105043660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105041d30 0x558105043660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/2023996136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 msgr2=0x558105032bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105032bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/2023996136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55810503e800 con 0x558105035da0 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a934c640 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105032bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.168+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035da0 0x558105043120 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000c00 tx=0x7fd7a0019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5111ebc19b0e3db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd78f7fe640 1 -- 172.21.15.33:0/2023996136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a001e040 con 0x558105035da0 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd78f7fe640 1 -- 172.21.15.33:0/2023996136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a000faf0 con 0x558105035da0 2024-09-10T21:13:53.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd78f7fe640 1 -- 172.21.15.33:0/2023996136 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a0019840 con 0x558105035da0 2024-09-10T21:13:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd7ae186740 1 -- 172.21.15.33:0/2023996136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035da0 msgr2=0x558105043120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd7ae186740 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035da0 0x558105043120 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0000c00 tx=0x7fd7a0019690 comp rx=0 tx=0).stop 2024-09-10T21:13:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd7ae186740 1 -- 172.21.15.33:0/2023996136 shutdown_connections 2024-09-10T21:13:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd7ae186740 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105041d30 0x558105043660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd7ae186740 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105035da0 0x558105043120 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd7ae186740 1 --2- 172.21.15.33:0/2023996136 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035480 0x558105032bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.169+0000 7fd7ae186740 1 -- 172.21.15.33:0/2023996136 >> 172.21.15.33:0/2023996136 conn(0x5581050332b0 msgr2=0x558105040080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.170+0000 7fd7ae186740 1 -- 172.21.15.33:0/2023996136 shutdown_connections 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.170+0000 7fd7ae186740 1 -- 172.21.15.33:0/2023996136 wait complete. 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.170+0000 7fd7ae186740 1 -- start start 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.170+0000 7fd7ae186740 1 Processor -- start 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035480 0x55810504a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035da0 0x558105047ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7ae186740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x558105048410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7ae186740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b1b30 con 0x558105041d30 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7ae186740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41b1460 con 0x558105035da0 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7ae186740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a41ae680 con 0x558105035480 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7ae186740 10 client.0 _ll_register_callbacks cb 0x7fd798005bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a9b4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035da0 0x558105047ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a934c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035480 0x55810504a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a9b4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035da0 0x558105047ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59848/0 (socket says 172.21.15.33:59848) 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a9b4d640 1 -- 172.21.15.33:0/105625132 learned_addr learned my addr 172.21.15.33:0/105625132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a934c640 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x558105048410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a934c640 1 -- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035da0 msgr2=0x558105047ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a934c640 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035da0 0x558105047ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a934c640 1 -- 172.21.15.33:0/105625132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 msgr2=0x558105048410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a934c640 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x558105048410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a934c640 1 -- 172.21.15.33:0/105625132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558105048c20 con 0x558105035480 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.171+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035da0 0x558105047ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd7a934c640 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035480 0x55810504a420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd7980098d0 tx=0x7fd798005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17b2a8e0c6104e00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd78d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:13:53.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd78d7fa640 1 -- 172.21.15.33:0/105625132 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798035030 con 0x558105035480 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd78d7fa640 1 -- 172.21.15.33:0/105625132 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd798005460 con 0x558105035480 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7a000faf0 con 0x558105035480 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd78d7fa640 1 -- 172.21.15.33:0/105625132 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd79803d610 con 0x558105035480 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd7ae186740 10 client.26590 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5581050bf6e0 con 0x558105035480 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.172+0000 7fd78d7fa640 1 -- 172.21.15.33:0/105625132 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a000faf0 con 0x558105035480 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd78d7fa640 1 -- 172.21.15.33:0/105625132 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd79804c460 con 0x558105035480 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd7ae186740 10 client.26590 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd7ae186740 10 client.26590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:13:53.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd7ae186740 20 client.26590 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd7ae186740 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5581050bfe00 0x5581050c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd7ae186740 4 client.26590 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558104fa9850 con 0x5581050bfe00 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.173+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5581050bfe00 0x5581050c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd78d7fa640 10 client.26590 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd7a9b4d640 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5581050bfe00 0x5581050c2220 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862736f4587713ae server_cookie=53ec8119486eba8e in_seq=0 out_seq=0 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd78d7fa640 1 -- 172.21.15.33:0/105625132 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x558104fa9850 con 0x5581050bfe00 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd78d7fa640 10 client.26590 handle_command_reply: tid=0 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd7ae186740 1 client.26590 shutdown 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035480 msgr2=0x55810504a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd7ae186740 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035480 0x55810504a420 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd7980098d0 tx=0x7fd798005db0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 shutdown_connections 2024-09-10T21:13:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.174+0000 7fd7ae186740 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5581050bfe00 0x5581050c2220 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd7ae186740 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558105041d30 0x558105048410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd7ae186740 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558105035da0 0x558105047ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd7ae186740 1 --2- 172.21.15.33:0/105625132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558105035480 0x55810504a420 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 >> 172.21.15.33:0/105625132 conn(0x5581050332b0 msgr2=0x55810503d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd78d7fa640 0 client.26590 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 shutdown_connections 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd7ae186740 1 -- 172.21.15.33:0/105625132 wait complete. 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.175+0000 7fd7ae186740 20 client.26590 trim_cache size 0 max 16384 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:13:53.184 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.187 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:13:53.189 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.190 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.191 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:13:53.192 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:13:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:13:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:13:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:13:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:13:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:13:53.193 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:13:53.193 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:13:53.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7900b52e0 msgr2=0x7fd7900b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7900b52e0 0x7fd7900b77a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0049d0 tx=0x7fd79c000f40 comp rx=0 tx=0).stop 2024-09-10T21:13:53.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a4108020 msgr2=0x7fd7a41a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a4108020 0x7fd7a41a0d70 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fd79c00eb10 tx=0x7fd79c00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 shutdown_connections 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7900b52e0 0x7fd7900b77a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7a4108020 0x7fd7a41a0d70 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7a41076a0 0x7fd7a419b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 --2- 172.21.15.33:0/3177336680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7a4101c90 0x7fd7a419b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.176+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 >> 172.21.15.33:0/3177336680 conn(0x7fd7a40fd700 msgr2=0x7fd7a406b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.179+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 shutdown_connections 2024-09-10T21:13:53.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:53.180+0000 7fd7abdb1640 1 -- 172.21.15.33:0/3177336680 wait complete. 2024-09-10T21:13:53.748 INFO:tasks.workunit.client.0.smithi033.stdout: 1 667625 5.59 MB/sec execute 414 sec latency 37.339 ms 2024-09-10T21:13:53.771 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:53 smithi033 ceph-mon[30860]: pgmap v643: 97 pgs: 97 active+clean; 261 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-09-10T21:13:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:53 smithi104 ceph-mon[35325]: pgmap v643: 97 pgs: 97 active+clean; 261 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-09-10T21:13:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:53 smithi174 ceph-mon[35635]: pgmap v643: 97 pgs: 97 active+clean; 261 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 271 op/s 2024-09-10T21:13:54.136 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:13:54.136 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:13:54.449 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:54.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 669153 5.59 MB/sec execute 415 sec latency 13.404 ms 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 -- 172.21.15.33:0/1497232353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff058108020 msgr2=0x7ff05810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 --2- 172.21.15.33:0/1497232353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff058108020 0x7ff05810e820 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7ff050009900 tx=0x7ff05002a410 comp rx=0 tx=0).stop 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 -- 172.21.15.33:0/1497232353 shutdown_connections 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 --2- 172.21.15.33:0/1497232353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff058108020 0x7ff05810e820 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 --2- 172.21.15.33:0/1497232353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0581076a0 0x7ff058107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 --2- 172.21.15.33:0/1497232353 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff058101c90 0x7ff058102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 -- 172.21.15.33:0/1497232353 >> 172.21.15.33:0/1497232353 conn(0x7ff0580fd700 msgr2=0x7ff0580ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 -- 172.21.15.33:0/1497232353 shutdown_connections 2024-09-10T21:13:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.953+0000 7ff05db70640 1 -- 172.21.15.33:0/1497232353 wait complete. 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.954+0000 7ff05db70640 1 Processor -- start 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.954+0000 7ff05db70640 1 -- start start 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.954+0000 7ff05db70640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff058101c90 0x7ff058198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff05db70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0581076a0 0x7ff0581986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff05db70640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff058108020 0x7ff05819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff05db70640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff058111ca0 con 0x7ff0581076a0 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff05db70640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff058111b20 con 0x7ff058101c90 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff05db70640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff058111e20 con 0x7ff058108020 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0581076a0 0x7ff0581986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff0577fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff058101c90 0x7ff058198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0581076a0 0x7ff0581986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39614/0 (socket says 172.21.15.33:39614) 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 -- 172.21.15.33:0/429830596 learned_addr learned my addr 172.21.15.33:0/429830596 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff057fff640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff058108020 0x7ff05819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 -- 172.21.15.33:0/429830596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff058108020 msgr2=0x7ff05819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff058108020 0x7ff05819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 -- 172.21.15.33:0/429830596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff058101c90 msgr2=0x7ff058198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff058101c90 0x7ff058198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 -- 172.21.15.33:0/429830596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05819e220 con 0x7ff0581076a0 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff0577fe640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff058101c90 0x7ff058198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff056ffd640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0581076a0 0x7ff0581986b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009860 tx=0x7ff04c009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff054ff9640 1 -- 172.21.15.33:0/429830596 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c005020 con 0x7ff0581076a0 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff054ff9640 1 -- 172.21.15.33:0/429830596 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04c0047a0 con 0x7ff0581076a0 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05819e510 con 0x7ff0581076a0 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff054ff9640 1 -- 172.21.15.33:0/429830596 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c007b00 con 0x7ff0581076a0 2024-09-10T21:13:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.955+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05806aab0 con 0x7ff0581076a0 2024-09-10T21:13:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.956+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff058102070 con 0x7ff0581076a0 2024-09-10T21:13:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.958+0000 7ff054ff9640 1 -- 172.21.15.33:0/429830596 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff04c029080 con 0x7ff0581076a0 2024-09-10T21:13:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.959+0000 7ff054ff9640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0400b52e0 0x7ff0400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.959+0000 7ff0577fe640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0400b52e0 0x7ff0400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.959+0000 7ff054ff9640 1 -- 172.21.15.33:0/429830596 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff04c0e7590 con 0x7ff0581076a0 2024-09-10T21:13:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.959+0000 7ff0577fe640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0400b52e0 0x7ff0400b77a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff048009760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:54.962+0000 7ff054ff9640 1 -- 172.21.15.33:0/429830596 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff04c0ae410 con 0x7ff0581076a0 2024-09-10T21:13:55.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.202+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff05806b4d0 con 0x7ff0581076a0 2024-09-10T21:13:55.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.203+0000 7ff054ff9640 1 -- 172.21.15.33:0/429830596 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11973 (secure 0 0 0) 0x7ff04c0add10 con 0x7ff0581076a0 2024-09-10T21:13:55.207 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:13:55.208 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"btime":"2024-09-10T21:11:31:688744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:11:30.989029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:13:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0400b52e0 msgr2=0x7ff0400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0400b52e0 0x7ff0400b77a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff048005fd0 tx=0x7ff048009760 comp rx=0 tx=0).stop 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0581076a0 msgr2=0x7ff0581986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0581076a0 0x7ff0581986b0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009860 tx=0x7ff04c009d20 comp rx=0 tx=0).stop 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 shutdown_connections 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff0400b52e0 0x7ff0400b77a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff058108020 0x7ff05819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0581076a0 0x7ff0581986b0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 --2- 172.21.15.33:0/429830596 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff058101c90 0x7ff058198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 >> 172.21.15.33:0/429830596 conn(0x7ff0580fd700 msgr2=0x7ff0580ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 shutdown_connections 2024-09-10T21:13:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:55.206+0000 7ff05db70640 1 -- 172.21.15.33:0/429830596 wait complete. 2024-09-10T21:13:55.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 671475 5.61 MB/sec execute 416 sec latency 39.570 ms 2024-09-10T21:13:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:55 smithi104 ceph-mon[35325]: pgmap v644: 97 pgs: 97 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-10T21:13:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:55 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/429830596' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:13:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:55 smithi174 ceph-mon[35635]: pgmap v644: 97 pgs: 97 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-10T21:13:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:55 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/429830596' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:13:55.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:55 smithi033 ceph-mon[30860]: pgmap v644: 97 pgs: 97 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-09-10T21:13:55.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:55 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/429830596' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:13:56.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:13:56.458 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:56.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 673789 5.62 MB/sec execute 417 sec latency 16.877 ms 2024-09-10T21:13:57.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 -- 172.21.15.33:0/536106271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 msgr2=0x7f09a410cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/536106271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a410cb20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0998009900 tx=0x7f099802a410 comp rx=0 tx=0).stop 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 -- 172.21.15.33:0/536106271 shutdown_connections 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/536106271 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09a410d0f0 0x7f09a410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/536106271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a410cb20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/536106271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4103b40 0x7f09a4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 -- 172.21.15.33:0/536106271 >> 172.21.15.33:0/536106271 conn(0x7f09a40fd730 msgr2=0x7f09a40ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 -- 172.21.15.33:0/536106271 shutdown_connections 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.978+0000 7f09a9d5b640 1 -- 172.21.15.33:0/536106271 wait complete. 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 Processor -- start 2024-09-10T21:13:57.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 -- start start 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4103b40 0x7f09a41019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a4101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09a410d0f0 0x7f09a4102440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4112d70 con 0x7f09a4104540 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4112bf0 con 0x7f09a410d0f0 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.979+0000 7f09a9d5b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4112ef0 con 0x7f09a4103b40 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a4101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a4101f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39632/0 (socket says 172.21.15.33:39632) 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4103b40 0x7f09a41019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 -- 172.21.15.33:0/2402008037 learned_addr learned my addr 172.21.15.33:0/2402008037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a3fff640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09a410d0f0 0x7f09a4102440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 -- 172.21.15.33:0/2402008037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4103b40 msgr2=0x7f09a41019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4103b40 0x7f09a41019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 -- 172.21.15.33:0/2402008037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09a410d0f0 msgr2=0x7f09a4102440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09a410d0f0 0x7f09a4102440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 -- 172.21.15.33:0/2402008037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a41a7680 con 0x7f09a4104540 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a37fe640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4103b40 0x7f09a41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a3fff640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09a410d0f0 0x7f09a4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a4101f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0998005940 tx=0x7f09980040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.980+0000 7f09a0ff9640 1 -- 172.21.15.33:0/2402008037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099803f030 con 0x7f09a4104540 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.981+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a41a7910 con 0x7f09a4104540 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.981+0000 7f09a0ff9640 1 -- 172.21.15.33:0/2402008037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09980044c0 con 0x7f09a4104540 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.981+0000 7f09a0ff9640 1 -- 172.21.15.33:0/2402008037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998035070 con 0x7f09a4104540 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.981+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41a7e40 con 0x7f09a4104540 2024-09-10T21:13:57.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.983+0000 7f09a0ff9640 1 -- 172.21.15.33:0/2402008037 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f099803ba20 con 0x7f09a4104540 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.983+0000 7f09ac130740 1 Processor -- start 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.983+0000 7f09ac130740 1 -- start start 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09ac130740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9284d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09ac130740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee928a10 0x5640ee934460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09ac130740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9349a0 0x5640ee936d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09ac130740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5640ee89eea0 con 0x5640ee928a10 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09ac130740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640ee88e2d0 con 0x5640ee9280f0 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09ac130740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5640ee8920e0 con 0x5640ee9349a0 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee928a10 0x5640ee934460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee928a10 0x5640ee934460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39640/0 (socket says 172.21.15.33:39640) 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 -- 172.21.15.33:0/1055058041 learned_addr learned my addr 172.21.15.33:0/1055058041 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a3fff640 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9284d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9349a0 0x5640ee936d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 -- 172.21.15.33:0/1055058041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9349a0 msgr2=0x5640ee936d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9349a0 0x5640ee936d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 -- 172.21.15.33:0/1055058041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 msgr2=0x5640ee9284d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9284d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 -- 172.21.15.33:0/1055058041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ee5ec650 con 0x5640ee928a10 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9349a0 0x5640ee936d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a3fff640 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9284d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:57.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a0ff9640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f098c0b52e0 0x7f098c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a3fff640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f098c0b52e0 0x7f098c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a37fe640 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee928a10 0x5640ee934460 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0994011540 tx=0x7f0994011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9d0fa1c26a4329 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.984+0000 7f09a0ff9640 1 -- 172.21.15.33:0/2402008037 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f09981061a0 con 0x7f09a4104540 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f0989ffb640 1 -- 172.21.15.33:0/1055058041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099400f640 con 0x5640ee928a10 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f0989ffb640 1 -- 172.21.15.33:0/1055058041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099400fc90 con 0x5640ee928a10 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f0989ffb640 1 -- 172.21.15.33:0/1055058041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0994019c60 con 0x5640ee928a10 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 -- 172.21.15.33:0/1055058041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee928a10 msgr2=0x5640ee934460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee928a10 0x5640ee934460 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0994011540 tx=0x7f0994011a00 comp rx=0 tx=0).stop 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09a3fff640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f098c0b52e0 0x7f098c0b77a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f099c009fd0 tx=0x7f099c009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 -- 172.21.15.33:0/1055058041 shutdown_connections 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9349a0 0x5640ee936d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee928a10 0x5640ee934460 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 --2- 172.21.15.33:0/1055058041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9284d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 -- 172.21.15.33:0/1055058041 >> 172.21.15.33:0/1055058041 conn(0x5640ee925f20 msgr2=0x5640ee9270c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 -- 172.21.15.33:0/1055058041 shutdown_connections 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.985+0000 7f09ac130740 1 -- 172.21.15.33:0/1055058041 wait complete. 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 -- start start 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 Processor -- start 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9280f0 0x5640ee934290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee928a10 0x5640ee9316e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee931d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5640ee6ef700 con 0x5640ee9349a0 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5640ee829160 con 0x5640ee928a10 2024-09-10T21:13:57.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5640ee937620 con 0x5640ee9280f0 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09ac130740 10 client.0 _ll_register_callbacks cb 0x5640ee8429e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9280f0 0x5640ee934290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee928a10 0x5640ee9316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.986+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee931d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9280f0 0x5640ee934290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:41714/0 (socket says 172.21.15.33:41714) 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee928a10 0x5640ee9316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59904/0 (socket says 172.21.15.33:59904) 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a37fe640 1 -- 172.21.15.33:0/1058827410 learned_addr learned my addr 172.21.15.33:0/1058827410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a2ffd640 1 -- 172.21.15.33:0/1058827410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9280f0 msgr2=0x5640ee934290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9280f0 0x5640ee934290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a2ffd640 1 -- 172.21.15.33:0/1058827410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 msgr2=0x5640ee931d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee931d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a2ffd640 1 -- 172.21.15.33:0/1058827410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ee856c90 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a3fff640 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee931d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee928a10 0x5640ee9316e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09980c9c80 tx=0x7f09980c9120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e2f4b722d51a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f0982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f0982ffd640 1 -- 172.21.15.33:0/1058827410 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998049040 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099400fc90 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f0982ffd640 1 -- 172.21.15.33:0/1058827410 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f09980ccc70 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f0982ffd640 1 -- 172.21.15.33:0/1058827410 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f09980cc360 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09ac130740 10 client.26654 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.987+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5640ee942fb0 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f0982ffd640 1 -- 172.21.15.33:0/1058827410 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099400fc90 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f0982ffd640 1 -- 172.21.15.33:0/1058827410 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f09980d1020 con 0x5640ee928a10 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09ac130740 10 client.26654 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:57.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09ac130740 10 client.26654 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09ac130740 20 client.26654 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09ac130740 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09ac130740 4 client.26654 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640ee8a0df0 con 0x5640ee944460 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09a37fe640 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f09a37fe640 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9468a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=59ede989bd4882bc server_cookie=e2abff0c893ed9c2 in_seq=0 out_seq=0 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.988+0000 7f0982ffd640 10 client.26654 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f0982ffd640 1 -- 172.21.15.33:0/1058827410 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5640ee8a0df0 con 0x5640ee944460 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f0982ffd640 10 client.26654 handle_command_reply: tid=0 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 client.26654 shutdown 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee928a10 msgr2=0x5640ee9316e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee928a10 0x5640ee9316e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f09980c9c80 tx=0x7f09980c9120 comp rx=0 tx=0).stop 2024-09-10T21:13:57.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 shutdown_connections 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9468a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee931d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee928a10 0x5640ee9316e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f0982ffd640 0 client.26654 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 --2- 172.21.15.33:0/1058827410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee9280f0 0x5640ee934290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.989+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 >> 172.21.15.33:0/1058827410 conn(0x5640ee925f20 msgr2=0x5640ee92bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.990+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 shutdown_connections 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.990+0000 7f09ac130740 1 -- 172.21.15.33:0/1058827410 wait complete. 2024-09-10T21:13:57.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:56.990+0000 7f09ac130740 20 client.26654 trim_cache size 0 max 16384 2024-09-10T21:13:57.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.004+0000 7f09ac130740 1 Processor -- start 2024-09-10T21:13:57.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.004+0000 7f09ac130740 1 -- start start 2024-09-10T21:13:57.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09ac130740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9408b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09ac130740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee926f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09ac130740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9274b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09ac130740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41b0610 con 0x5640ee9349a0 2024-09-10T21:13:57.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09ac130740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41b2150 con 0x5640ee9280f0 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09ac130740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41b0ce0 con 0x5640ee928a10 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9274b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9274b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39652/0 (socket says 172.21.15.33:39652) 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 -- 172.21.15.33:0/1607070706 learned_addr learned my addr 172.21.15.33:0/1607070706 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee926f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 -- 172.21.15.33:0/1607070706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 msgr2=0x5640ee926f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee926f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 -- 172.21.15.33:0/1607070706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 msgr2=0x5640ee9408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9408b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 -- 172.21.15.33:0/1607070706 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640ee92fbe0 con 0x5640ee9349a0 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a2ffd640 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee926f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:13:57.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a37fe640 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09a3fff640 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9274b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f099c011960 tx=0x7f099c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98220d6d66b951bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.005+0000 7f09897fa640 1 -- 172.21.15.33:0/1607070706 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c01e9b0 con 0x5640ee9349a0 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09897fa640 1 -- 172.21.15.33:0/1607070706 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c010430 con 0x5640ee9349a0 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09897fa640 1 -- 172.21.15.33:0/1607070706 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c01c660 con 0x5640ee9349a0 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 -- 172.21.15.33:0/1607070706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 msgr2=0x5640ee9274b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9274b0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f099c011960 tx=0x7f099c01a000 comp rx=0 tx=0).stop 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 -- 172.21.15.33:0/1607070706 shutdown_connections 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9274b0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee926f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 --2- 172.21.15.33:0/1607070706 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee9408b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 -- 172.21.15.33:0/1607070706 >> 172.21.15.33:0/1607070706 conn(0x5640ee925f20 msgr2=0x5640ee920f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 -- 172.21.15.33:0/1607070706 shutdown_connections 2024-09-10T21:13:57.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.006+0000 7f09ac130740 1 -- 172.21.15.33:0/1607070706 wait complete. 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 -- start start 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 Processor -- start 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee938f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee9394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9b0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41b1a80 con 0x5640ee9349a0 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41b13b0 con 0x5640ee9280f0 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41ae5d0 con 0x5640ee928a10 2024-09-10T21:13:57.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09ac130740 10 client.0 _ll_register_callbacks cb 0x7f0998005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee938f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09a3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee938f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59916/0 (socket says 172.21.15.33:59916) 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09a2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee9394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.007+0000 7f09a37fe640 1 -- 172.21.15.33:0/3955072600 learned_addr learned my addr 172.21.15.33:0/3955072600 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09a3fff640 1 -- 172.21.15.33:0/3955072600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 msgr2=0x5640ee9394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09a3fff640 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee9394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09a3fff640 1 -- 172.21.15.33:0/3955072600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 msgr2=0x5640ee938f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09a3fff640 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee938f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09a3fff640 1 -- 172.21.15.33:0/3955072600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640ee9b08e0 con 0x5640ee9349a0 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09a37fe640 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee938f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09a3fff640 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9b0370 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f099c010e10 tx=0x7f099c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280252d8411126c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:57.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f0982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f0982ffd640 1 -- 172.21.15.33:0/3955072600 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c004a40 con 0x5640ee9349a0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f0982ffd640 1 -- 172.21.15.33:0/3955072600 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f099c004be0 con 0x5640ee9349a0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f099c010430 con 0x5640ee9349a0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f0982ffd640 1 -- 172.21.15.33:0/3955072600 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f099c004de0 con 0x5640ee9349a0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09ac130740 10 client.17220 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.008+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5640ee9b1b50 con 0x5640ee9349a0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f0982ffd640 1 -- 172.21.15.33:0/3955072600 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f099c010430 con 0x5640ee9349a0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f0982ffd640 1 -- 172.21.15.33:0/3955072600 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f099c005430 con 0x5640ee9349a0 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09ac130740 10 client.17220 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09ac130740 10 client.17220 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09ac130740 20 client.17220 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09ac130740 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9b4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:57.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09ac130740 4 client.17220 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640ee89c8a0 con 0x5640ee944460 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09a37fe640 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9b4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f0982ffd640 10 client.17220 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.009+0000 7f09a37fe640 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9b4250 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c1f876ae369967d server_cookie=744f918b567c3460 in_seq=0 out_seq=0 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f0982ffd640 1 -- 172.21.15.33:0/3955072600 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640ee89c8a0 con 0x5640ee944460 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f0982ffd640 10 client.17220 handle_command_reply: tid=0 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 client.17220 shutdown 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 msgr2=0x5640ee9b0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9b0370 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f099c010e10 tx=0x7f099c010c20 comp rx=0 tx=0).stop 2024-09-10T21:13:57.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 shutdown_connections 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5640ee944460 0x5640ee9b4250 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5640ee9349a0 0x5640ee9b0370 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f0982ffd640 0 client.17220 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5640ee928a10 0x5640ee9394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 --2- 172.21.15.33:0/3955072600 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5640ee9280f0 0x5640ee938f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 >> 172.21.15.33:0/3955072600 conn(0x5640ee925f20 msgr2=0x5640ee926f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 shutdown_connections 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 1 -- 172.21.15.33:0/3955072600 wait complete. 2024-09-10T21:13:57.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.010+0000 7f09ac130740 20 client.17220 trim_cache size 0 max 16384 2024-09-10T21:13:57.028 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f098c0b52e0 msgr2=0x7f098c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f098c0b52e0 0x7f098c0b77a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f099c009fd0 tx=0x7f099c009500 comp rx=0 tx=0).stop 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 msgr2=0x7f09a4101f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a4101f00 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0998005940 tx=0x7f09980040c0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 shutdown_connections 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f098c0b52e0 0x7f098c0b77a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09a410d0f0 0x7f09a4102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09a4104540 0x7f09a4101f00 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 --2- 172.21.15.33:0/2402008037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09a4103b40 0x7f09a41019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 >> 172.21.15.33:0/2402008037 conn(0x7f09a40fd730 msgr2=0x7f09a40fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 shutdown_connections 2024-09-10T21:13:57.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:57.011+0000 7f09a9d5b640 1 -- 172.21.15.33:0/2402008037 wait complete. 2024-09-10T21:13:57.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 675488 5.63 MB/sec execute 418 sec latency 20.616 ms 2024-09-10T21:13:57.791 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:57 smithi033 ceph-mon[30860]: pgmap v645: 97 pgs: 97 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-10T21:13:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:57 smithi104 ceph-mon[35325]: pgmap v645: 97 pgs: 97 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-10T21:13:57.821 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:13:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:57 smithi174 ceph-mon[35635]: pgmap v645: 97 pgs: 97 active+clean; 263 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 196 op/s 2024-09-10T21:13:58.095 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 -- 172.21.15.33:0/3086143337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 msgr2=0x7fcf1410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 --2- 172.21.15.33:0/3086143337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 0x7fcf1410e820 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c009ca0 tx=0x7fcf0c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 -- 172.21.15.33:0/3086143337 shutdown_connections 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 --2- 172.21.15.33:0/3086143337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 0x7fcf1410e820 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 --2- 172.21.15.33:0/3086143337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 0x7fcf14107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 --2- 172.21.15.33:0/3086143337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf14101c90 0x7fcf14102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 -- 172.21.15.33:0/3086143337 >> 172.21.15.33:0/3086143337 conn(0x7fcf140fd700 msgr2=0x7fcf140ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 -- 172.21.15.33:0/3086143337 shutdown_connections 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.565+0000 7fcf1c527640 1 -- 172.21.15.33:0/3086143337 wait complete. 2024-09-10T21:13:58.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 Processor -- start 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 -- start start 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf14101c90 0x7fcf140fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 0x7fcf140fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 0x7fcf140fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14111ce0 con 0x7fcf141076a0 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14111b60 con 0x7fcf14108020 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.566+0000 7fcf1c527640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf14111e60 con 0x7fcf14101c90 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 0x7fcf140fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 0x7fcf140fe140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39684/0 (socket says 172.21.15.33:39684) 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf1a2c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf14101c90 0x7fcf140fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 0x7fcf140fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 -- 172.21.15.33:0/2110395249 learned_addr learned my addr 172.21.15.33:0/2110395249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 -- 172.21.15.33:0/2110395249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf14101c90 msgr2=0x7fcf140fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf14101c90 0x7fcf140fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 -- 172.21.15.33:0/2110395249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 msgr2=0x7fcf140fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 0x7fcf140fe680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 -- 172.21.15.33:0/2110395249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf141a7620 con 0x7fcf141076a0 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf14101c90 0x7fcf140fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 0x7fcf140fe140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400e9f0 tx=0x7fcf0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2110395249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0400cd80 con 0x7fcf141076a0 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf141a7910 con 0x7fcf141076a0 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2110395249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf040108e0 con 0x7fcf141076a0 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2110395249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04018980 con 0x7fcf141076a0 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.567+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf141a7ea0 con 0x7fcf141076a0 2024-09-10T21:13:58.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.570+0000 7fcf1e8fc740 1 Processor -- start 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.570+0000 7fcf1e8fc740 1 -- start start 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2110395249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fcf04018b20 con 0x7fcf141076a0 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x5635625864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562592460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x563562594d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5635624fcea0 con 0x5635625929a0 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5635624ec2d0 con 0x5635625860f0 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5635624f00e0 con 0x563562586a10 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x5635625864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x5635625864d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59954/0 (socket says 172.21.15.33:59954) 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 -- 172.21.15.33:0/4124010697 learned_addr learned my addr 172.21.15.33:0/4124010697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:58.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x563562594d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562592460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 -- 172.21.15.33:0/4124010697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 msgr2=0x563562592460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562592460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 -- 172.21.15.33:0/4124010697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 msgr2=0x563562594d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x563562594d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.571+0000 7fcf1aac4640 1 -- 172.21.15.33:0/4124010697 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56356224a650 con 0x5635625860f0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.572+0000 7fcf1aac4640 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x5635625864d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c035000 tx=0x7fcf0c005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c339a794ef89b44 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.572+0000 7fcf0b7fe640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcf000b52e0 0x7fcf000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.572+0000 7fcf08ff9640 1 -- 172.21.15.33:0/4124010697 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c031cf0 con 0x5635625860f0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.572+0000 7fcf08ff9640 1 -- 172.21.15.33:0/4124010697 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c036030 con 0x5635625860f0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.572+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcf000b52e0 0x7fcf000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.572+0000 7fcf08ff9640 1 -- 172.21.15.33:0/4124010697 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0042a0 con 0x5635625860f0 2024-09-10T21:13:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.572+0000 7fcf0b7fe640 1 -- 172.21.15.33:0/2110395249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcf040e5f70 con 0x7fcf141076a0 2024-09-10T21:13:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/4124010697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 msgr2=0x5635625864d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x5635625864d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c035000 tx=0x7fcf0c005ef0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcf000b52e0 0x7fcf000b77a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x563562584e00 tx=0x7fcf10005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/4124010697 shutdown_connections 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x563562594d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562592460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/4124010697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x5635625864d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/4124010697 >> 172.21.15.33:0/4124010697 conn(0x563562583f20 msgr2=0x5635625850c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/4124010697 shutdown_connections 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/4124010697 wait complete. 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 -- start start 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.573+0000 7fcf1e8fc740 1 Processor -- start 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x5635625922c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563562586a10 0x56356258f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625929a0 0x56356258fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56356234d700 con 0x563562586a10 2024-09-10T21:13:58.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563562487160 con 0x5635625929a0 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563562595440 con 0x5635625860f0 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 10 client.0 _ll_register_callbacks cb 0x5635624a09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1a2c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x5635625922c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625929a0 0x56356258fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf19ac2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563562586a10 0x56356258f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1a2c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x5635625922c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:41760/0 (socket says 172.21.15.33:41760) 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625929a0 0x56356258fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59958/0 (socket says 172.21.15.33:59958) 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1a2c3640 1 -- 172.21.15.33:0/3663000691 learned_addr learned my addr 172.21.15.33:0/3663000691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf19ac2640 1 -- 172.21.15.33:0/3663000691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 msgr2=0x5635625922c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x5635625922c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf19ac2640 1 -- 172.21.15.33:0/3663000691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625929a0 msgr2=0x56356258fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625929a0 0x56356258fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf19ac2640 1 -- 172.21.15.33:0/3663000691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635624b4c90 con 0x563562586a10 2024-09-10T21:13:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563562586a10 0x56356258f710 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcf040e3a70 tx=0x7fcf040e9520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee19bc43feeb0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcefdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:13:58.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcefdffb640 1 -- 172.21.15.33:0/3663000691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040ac5b0 con 0x563562586a10 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcefdffb640 1 -- 172.21.15.33:0/3663000691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcf04010430 con 0x563562586a10 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.574+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf0c036030 con 0x563562586a10 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcefdffb640 1 -- 172.21.15.33:0/3663000691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fcf040abf50 con 0x563562586a10 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 10 client.17232 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5635625a0dd0 con 0x563562586a10 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcefdffb640 1 -- 172.21.15.33:0/3663000691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf0c036030 con 0x563562586a10 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcefdffb640 1 -- 172.21.15.33:0/3663000691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fcf040a8130 con 0x563562586a10 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 10 client.17232 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 10 client.17232 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 20 client.17232 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:58.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5635625a2230 0x5635625a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 4 client.17232 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635624fedf0 con 0x5635625a2230 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.575+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5635625a2230 0x5635625a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.576+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5635625a2230 0x5635625a4620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb2b3f925c0bb8bd server_cookie=f59740b7e3daab3d in_seq=0 out_seq=0 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.576+0000 7fcefdffb640 10 client.17232 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.576+0000 7fcefdffb640 1 -- 172.21.15.33:0/3663000691 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5635624fedf0 con 0x5635625a2230 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.576+0000 7fcefdffb640 10 client.17232 handle_command_reply: tid=0 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 client.17232 shutdown 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563562586a10 msgr2=0x56356258f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563562586a10 0x56356258f710 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fcf040e3a70 tx=0x7fcf040e9520 comp rx=0 tx=0).stop 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 shutdown_connections 2024-09-10T21:13:58.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625929a0 0x56356258fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563562586a10 0x56356258f710 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5635625a2230 0x5635625a4620 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/3663000691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x5635625922c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcefdffb640 0 client.17232 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 >> 172.21.15.33:0/3663000691 conn(0x563562583f20 msgr2=0x563562589a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 shutdown_connections 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/3663000691 wait complete. 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.577+0000 7fcf1e8fc740 20 client.17232 trim_cache size 0 max 16384 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 Processor -- start 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 -- start start 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x563562583760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562593d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x5635625942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b0570 con 0x5635625929a0 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b20b0 con 0x5635625860f0 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.591+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b0c40 con 0x563562586a10 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x5635625942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf1a2c3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x563562583760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x5635625942d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39718/0 (socket says 172.21.15.33:39718) 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf1aac4640 1 -- 172.21.15.33:0/73974581 learned_addr learned my addr 172.21.15.33:0/73974581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562593d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf19ac2640 1 -- 172.21.15.33:0/73974581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 msgr2=0x563562583760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x563562583760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf19ac2640 1 -- 172.21.15.33:0/73974581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 msgr2=0x5635625942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x5635625942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf19ac2640 1 -- 172.21.15.33:0/73974581 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563562586840 con 0x563562586a10 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x563562583760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf1aac4640 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x5635625942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562593d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf040a8d60 tx=0x7fcf04018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df1c75270e308944 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcefffff640 1 -- 172.21.15.33:0/73974581 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040ac130 con 0x563562586a10 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcefffff640 1 -- 172.21.15.33:0/73974581 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf040ac2f0 con 0x563562586a10 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.592+0000 7fcefffff640 1 -- 172.21.15.33:0/73974581 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf04010670 con 0x563562586a10 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/73974581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 msgr2=0x563562593d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562593d90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf040a8d60 tx=0x7fcf04018420 comp rx=0 tx=0).stop 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/73974581 shutdown_connections 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x5635625942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563562586a10 0x563562593d90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/73974581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5635625860f0 0x563562583760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/73974581 >> 172.21.15.33:0/73974581 conn(0x563562583f20 msgr2=0x563562591560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/73974581 shutdown_connections 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.593+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/73974581 wait complete. 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 -- start start 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 Processor -- start 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x56356259a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563562586a10 0x56356259aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x56356258b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b19e0 con 0x5635625929a0 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141b1310 con 0x563562586a10 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf141ae530 con 0x5635625860f0 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1e8fc740 10 client.0 _ll_register_callbacks cb 0x7fcf040e96e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:13:58.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x56356258b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1a2c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x56356259a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf19ac2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563562586a10 0x56356259aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1aac4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x56356258b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39720/0 (socket says 172.21.15.33:39720) 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.594+0000 7fcf1aac4640 1 -- 172.21.15.33:0/2645954668 learned_addr learned my addr 172.21.15.33:0/2645954668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf19ac2640 1 -- 172.21.15.33:0/2645954668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 msgr2=0x56356259a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x56356259a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf19ac2640 1 -- 172.21.15.33:0/2645954668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 msgr2=0x56356258b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x56356258b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf19ac2640 1 -- 172.21.15.33:0/2645954668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56356258c160 con 0x563562586a10 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf19ac2640 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563562586a10 0x56356259aef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcf040a8d60 tx=0x7fcf040abcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97e89c0ff11bb3fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcefdffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcefdffb640 1 -- 172.21.15.33:0/2645954668 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf040ac130 con 0x563562586a10 2024-09-10T21:13:58.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcefdffb640 1 -- 172.21.15.33:0/2645954668 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcf040e9320 con 0x563562586a10 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf040ac2f0 con 0x563562586a10 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcefdffb640 1 -- 172.21.15.33:0/2645954668 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fcf040ac7e0 con 0x563562586a10 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf1e8fc740 10 client.26678 fetch_fsmap learned FSMap version 58 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56356258d460 con 0x563562586a10 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.595+0000 7fcefdffb640 1 -- 172.21.15.33:0/2645954668 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf040ac2f0 con 0x563562586a10 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcefdffb640 1 -- 172.21.15.33:0/2645954668 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fcf040b2580 con 0x563562586a10 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1e8fc740 10 client.26678 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1e8fc740 10 client.26678 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1e8fc740 20 client.26678 populate_metadata read hostname 'smithi033' 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x563562610130 0x563562612520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1e8fc740 4 client.26678 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:13:58.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5635624fa8a0 con 0x563562610130 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x563562610130 0x563562612520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcf1a2c3640 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x563562610130 0x563562612520 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e8d002f93daa07eb server_cookie=5dbd8d1017fb9f06 in_seq=0 out_seq=0 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.596+0000 7fcefdffb640 10 client.26678 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcefdffb640 1 -- 172.21.15.33:0/2645954668 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5635624fa8a0 con 0x563562610130 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcefdffb640 10 client.26678 handle_command_reply: tid=0 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 client.26678 shutdown 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563562586a10 msgr2=0x56356259aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563562586a10 0x56356259aef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcf040a8d60 tx=0x7fcf040abcf0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 shutdown_connections 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5635625929a0 0x56356258b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563562586a10 0x56356259aef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x563562610130 0x563562612520 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 --2- 172.21.15.33:0/2645954668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635625860f0 0x56356259a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcefdffb640 0 client.26678 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:13:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 >> 172.21.15.33:0/2645954668 conn(0x563562583f20 msgr2=0x56356258de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 shutdown_connections 2024-09-10T21:13:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 1 -- 172.21.15.33:0/2645954668 wait complete. 2024-09-10T21:13:58.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1e8fc740 20 client.26678 trim_cache size 0 max 16384 2024-09-10T21:13:58.619 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:13:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.597+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcf000b52e0 msgr2=0x7fcf000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcf000b52e0 0x7fcf000b77a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x563562584e00 tx=0x7fcf10005f90 comp rx=0 tx=0).stop 2024-09-10T21:13:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 msgr2=0x7fcf140fe140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:13:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 0x7fcf140fe140 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fcf0400e9f0 tx=0x7fcf0400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 shutdown_connections 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcf000b52e0 0x7fcf000b77a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcf14108020 0x7fcf140fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf141076a0 0x7fcf140fe140 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 --2- 172.21.15.33:0/2110395249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf14101c90 0x7fcf140fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 >> 172.21.15.33:0/2110395249 conn(0x7fcf140fd700 msgr2=0x7fcf1406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 shutdown_connections 2024-09-10T21:13:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:13:58.598+0000 7fcf1c527640 1 -- 172.21.15.33:0/2110395249 wait complete. 2024-09-10T21:13:58.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 678122 5.65 MB/sec execute 419 sec latency 45.958 ms 2024-09-10T21:13:59.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:13:59.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:13:59 smithi033 ceph-mon[30860]: pgmap v646: 97 pgs: 97 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-10T21:13:59.684 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:13:59.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 679744 5.66 MB/sec execute 420 sec latency 20.472 ms 2024-09-10T21:13:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:13:59 smithi104 ceph-mon[35325]: pgmap v646: 97 pgs: 97 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-10T21:13:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:13:59 smithi174 ceph-mon[35635]: pgmap v646: 97 pgs: 97 active+clean; 265 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 219 op/s 2024-09-10T21:14:00.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.191+0000 7f4ceef77640 1 -- 172.21.15.33:0/959830583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce810d160 msgr2=0x7f4ce810f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.191+0000 7f4ceef77640 1 --2- 172.21.15.33:0/959830583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce810d160 0x7f4ce810f570 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0009ca0 tx=0x7f4ce002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.191+0000 7f4ceef77640 1 -- 172.21.15.33:0/959830583 shutdown_connections 2024-09-10T21:14:00.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.191+0000 7f4ceef77640 1 --2- 172.21.15.33:0/959830583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce810d160 0x7f4ce810f570 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 --2- 172.21.15.33:0/959830583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ce81045b0 0x7f4ce810cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 --2- 172.21.15.33:0/959830583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce8103bb0 0x7f4ce8103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 -- 172.21.15.33:0/959830583 >> 172.21.15.33:0/959830583 conn(0x7f4ce80fd700 msgr2=0x7f4ce80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 -- 172.21.15.33:0/959830583 shutdown_connections 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 -- 172.21.15.33:0/959830583 wait complete. 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 Processor -- start 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 -- start start 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce8103bb0 0x7f4ce8199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce81045b0 0x7f4ce81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ce810d160 0x7f4ce819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8112d80 con 0x7f4ce81045b0 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8112c00 con 0x7f4ce810d160 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.192+0000 7f4ceef77640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce8112f00 con 0x7f4ce8103bb0 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce81045b0 0x7f4ce81995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce81045b0 0x7f4ce81995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48622/0 (socket says 172.21.15.33:48622) 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 -- 172.21.15.33:0/830495606 learned_addr learned my addr 172.21.15.33:0/830495606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ced514640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ce810d160 0x7f4ce819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4cecd13640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce8103bb0 0x7f4ce8199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 -- 172.21.15.33:0/830495606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce8103bb0 msgr2=0x7f4ce8199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce8103bb0 0x7f4ce8199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 -- 172.21.15.33:0/830495606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ce810d160 msgr2=0x7f4ce819e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ce810d160 0x7f4ce819e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 -- 172.21.15.33:0/830495606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce819f140 con 0x7f4ce81045b0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce81045b0 0x7f4ce81995d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9f0 tx=0x7f4cd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce5ffb640 1 -- 172.21.15.33:0/830495606 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd400cd80 con 0x7f4ce81045b0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce819f430 con 0x7f4ce81045b0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce5ffb640 1 -- 172.21.15.33:0/830495606 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40108e0 con 0x7f4ce81045b0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.193+0000 7f4ce5ffb640 1 -- 172.21.15.33:0/830495606 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4018980 con 0x7f4ce81045b0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.194+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce81a5cc0 con 0x7f4ce81045b0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 Processor -- start 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 -- start start 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4ce5ffb640 1 -- 172.21.15.33:0/830495606 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4cd4010430 con 0x7f4ce81045b0 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a3d4d0 0x557cc8a3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a49840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a49d80 0x557cc8a4c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557cc89b3ea0 con 0x557cc8a3ddf0 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557cc89a32d0 con 0x557cc8a49d80 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.196+0000 7f4cf134c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557cc89a70e0 con 0x557cc8a3d4d0 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a49840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a49840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48636/0 (socket says 172.21.15.33:48636) 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 -- 172.21.15.33:0/4129455991 learned_addr learned my addr 172.21.15.33:0/4129455991 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a49d80 0x557cc8a4c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4ced514640 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a3d4d0 0x557cc8a3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 -- 172.21.15.33:0/4129455991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a3d4d0 msgr2=0x557cc8a3d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a3d4d0 0x557cc8a3d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 -- 172.21.15.33:0/4129455991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a49d80 msgr2=0x557cc8a4c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a49d80 0x557cc8a4c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 -- 172.21.15.33:0/4129455991 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc8701650 con 0x557cc8a3ddf0 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4ced514640 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a3d4d0 0x557cc8a3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cecd13640 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a49840 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009cf0 tx=0x7f4cdc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d85ce22a15e817 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4ce5ffb640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cd80b52e0 0x7f4cd80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/4129455991 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc004230 con 0x557cc8a3ddf0 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4ced514640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cd80b52e0 0x7f4cd80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4ce5ffb640 1 -- 172.21.15.33:0/830495606 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cd40e5ec0 con 0x7f4ce81045b0 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/4129455991 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cdc0043d0 con 0x557cc8a3ddf0 2024-09-10T21:14:00.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.197+0000 7f4cd2ffd640 1 -- 172.21.15.33:0/4129455991 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc019880 con 0x557cc8a3ddf0 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 -- 172.21.15.33:0/4129455991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 msgr2=0x557cc8a49840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a49840 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009cf0 tx=0x7f4cdc011810 comp rx=0 tx=0).stop 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4ced514640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cd80b52e0 0x7f4cd80b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0004af0 tx=0x7f4ce0005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 -- 172.21.15.33:0/4129455991 shutdown_connections 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a49d80 0x557cc8a4c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a49840 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 --2- 172.21.15.33:0/4129455991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a3d4d0 0x557cc8a3d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 -- 172.21.15.33:0/4129455991 >> 172.21.15.33:0/4129455991 conn(0x557cc8a3b300 msgr2=0x557cc8a3c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 -- 172.21.15.33:0/4129455991 shutdown_connections 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.198+0000 7f4cf134c740 1 -- 172.21.15.33:0/4129455991 wait complete. 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 -- start start 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 Processor -- start 2024-09-10T21:14:00.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a46a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a47040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557cc8804700 con 0x557cc8a3d4d0 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557cc893e160 con 0x557cc8a3ddf0 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557cc8a4ca00 con 0x557cc8a49d80 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cf134c740 10 client.0 _ll_register_callbacks cb 0x557cc89579e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cecd13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cecd13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a49630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48648/0 (socket says 172.21.15.33:48648) 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4cecd13640 1 -- 172.21.15.33:0/3226597113 learned_addr learned my addr 172.21.15.33:0/3226597113 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a46a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.199+0000 7f4ced514640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a47040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cecd13640 1 -- 172.21.15.33:0/3226597113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 msgr2=0x557cc8a47040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cecd13640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a47040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cecd13640 1 -- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 msgr2=0x557cc8a46a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cecd13640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a46a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cecd13640 1 -- 172.21.15.33:0/3226597113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc896bc90 con 0x557cc8a3d4d0 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4ced514640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a47040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a46a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cecd13640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a49630 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009560 tx=0x7f4cdc0096f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b5f2233983905d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cc7fff640 1 -- 172.21.15.33:0/3226597113 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc027070 con 0x557cc8a3d4d0 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cc7fff640 1 -- 172.21.15.33:0/3226597113 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cdc005240 con 0x557cc8a3d4d0 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cdc0043d0 con 0x557cc8a3d4d0 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cc7fff640 1 -- 172.21.15.33:0/3226597113 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4cdc033050 con 0x557cc8a3d4d0 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cf134c740 10 client.17250 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:00.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557cc8a582f0 con 0x557cc8a3d4d0 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.200+0000 7f4cc7fff640 1 -- 172.21.15.33:0/3226597113 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cdc0043d0 con 0x557cc8a3d4d0 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cc7fff640 1 -- 172.21.15.33:0/3226597113 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4cdc004b70 con 0x557cc8a3d4d0 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cf134c740 10 client.17250 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cf134c740 10 client.17250 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cf134c740 20 client.17250 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cf134c740 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8a598d0 0x557cc8a5bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cf134c740 4 client.17250 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557cc89b5df0 con 0x557cc8a598d0 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8a598d0 0x557cc8a5bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8a598d0 0x557cc8a5bcc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32a3a229df491562 server_cookie=a7c984769f0ba70d in_seq=0 out_seq=0 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.201+0000 7f4cc7fff640 10 client.17250 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cc7fff640 1 -- 172.21.15.33:0/3226597113 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557cc89b5df0 con 0x557cc8a598d0 2024-09-10T21:14:00.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cc7fff640 10 client.17250 handle_command_reply: tid=0 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 client.17250 shutdown 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 msgr2=0x557cc8a49630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a49630 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc009560 tx=0x7f4cdc0096f0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 shutdown_connections 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a47040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a46a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8a598d0 0x557cc8a5bcc0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 --2- 172.21.15.33:0/3226597113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a49630 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 >> 172.21.15.33:0/3226597113 conn(0x557cc8a3b300 msgr2=0x557cc8a40ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.202+0000 7f4cc7fff640 0 client.17250 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.203+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 shutdown_connections 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.203+0000 7f4cf134c740 1 -- 172.21.15.33:0/3226597113 wait complete. 2024-09-10T21:14:00.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.203+0000 7f4cf134c740 20 client.17250 trim_cache size 0 max 16384 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 Processor -- start 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 -- start start 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a55c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a45300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a45840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81ae4d0 con 0x557cc8a3d4d0 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81b0010 con 0x557cc8a3ddf0 2024-09-10T21:14:00.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cf134c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81aeba0 con 0x557cc8a49d80 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cecd13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a55c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cecd13640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a55c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48662/0 (socket says 172.21.15.33:48662) 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a45300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.217+0000 7f4cecd13640 1 -- 172.21.15.33:0/2447074263 learned_addr learned my addr 172.21.15.33:0/2447074263 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ced514640 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a45840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ce7fff640 1 -- 172.21.15.33:0/2447074263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 msgr2=0x557cc8a45840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a45840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ce7fff640 1 -- 172.21.15.33:0/2447074263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 msgr2=0x557cc8a55c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a55c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ce7fff640 1 -- 172.21.15.33:0/2447074263 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557cc8a46830 con 0x557cc8a3ddf0 2024-09-10T21:14:00.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4cecd13640 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a55c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ced514640 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a45840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:00.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a45300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9c0 tx=0x7f4cd40e9940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3407d1498a5bd46a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4cd27fc640 1 -- 172.21.15.33:0/2447074263 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40b5e60 con 0x557cc8a3ddf0 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4cd27fc640 1 -- 172.21.15.33:0/2447074263 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40ac240 con 0x557cc8a3ddf0 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.218+0000 7f4cd27fc640 1 -- 172.21.15.33:0/2447074263 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40b5c10 con 0x557cc8a3ddf0 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 -- 172.21.15.33:0/2447074263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 msgr2=0x557cc8a45300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a45300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9c0 tx=0x7f4cd40e9940 comp rx=0 tx=0).stop 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 -- 172.21.15.33:0/2447074263 shutdown_connections 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a45840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3ddf0 0x557cc8a45300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 --2- 172.21.15.33:0/2447074263 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3d4d0 0x557cc8a55c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 -- 172.21.15.33:0/2447074263 >> 172.21.15.33:0/2447074263 conn(0x557cc8a3b300 msgr2=0x557cc8a3c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 -- 172.21.15.33:0/2447074263 shutdown_connections 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 -- 172.21.15.33:0/2447074263 wait complete. 2024-09-10T21:14:00.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.219+0000 7f4cf134c740 1 -- start start 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 1 Processor -- start 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3d4d0 0x557cc8a51ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81af940 con 0x557cc8a3ddf0 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81af270 con 0x557cc8a3d4d0 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce81ac490 con 0x557cc8a49d80 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cf134c740 10 client.0 _ll_register_callbacks cb 0x7f4cd40e9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4cecd13640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3d4d0 0x557cc8a51ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4ce7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48672/0 (socket says 172.21.15.33:48672) 2024-09-10T21:14:00.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4ce7fff640 1 -- 172.21.15.33:0/1278200831 learned_addr learned my addr 172.21.15.33:0/1278200831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.220+0000 7f4ced514640 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4ce7fff640 1 -- 172.21.15.33:0/1278200831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 msgr2=0x557cc8a4cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a4cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4ce7fff640 1 -- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3d4d0 msgr2=0x557cc8a51ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3d4d0 0x557cc8a51ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4ce7fff640 1 -- 172.21.15.33:0/1278200831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557cc8a527a0 con 0x557cc8a3ddf0 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cecd13640 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3d4d0 0x557cc8a51ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4ce7fff640 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a521e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40ac430 tx=0x7f4cd40b1160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4814211322bf9906 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cc7fff640 1 -- 172.21.15.33:0/1278200831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40abea0 con 0x557cc8a3ddf0 2024-09-10T21:14:00.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cc7fff640 1 -- 172.21.15.33:0/1278200831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cd40abb90 con 0x557cc8a3ddf0 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd40ac240 con 0x557cc8a3ddf0 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cc7fff640 1 -- 172.21.15.33:0/1278200831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4cd40ac730 con 0x557cc8a3ddf0 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cf134c740 10 client.17256 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557cc8acac90 con 0x557cc8a3ddf0 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.221+0000 7f4cc7fff640 1 -- 172.21.15.33:0/1278200831 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40ac240 con 0x557cc8a3ddf0 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cc7fff640 1 -- 172.21.15.33:0/1278200831 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4cd40e7040 con 0x557cc8a3ddf0 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cf134c740 10 client.17256 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cf134c740 10 client.17256 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cf134c740 20 client.17256 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cf134c740 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8acb3a0 0x557cc8acd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cf134c740 4 client.17256 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557cc89b18a0 con 0x557cc8acb3a0 2024-09-10T21:14:00.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cecd13640 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8acb3a0 0x557cc8acd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cc7fff640 10 client.17256 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.222+0000 7f4cecd13640 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8acb3a0 0x557cc8acd7c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ac378001ae8e82c6 server_cookie=412d01dfdef9328e in_seq=0 out_seq=0 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cc7fff640 1 -- 172.21.15.33:0/1278200831 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557cc89b18a0 con 0x557cc8acb3a0 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cc7fff640 10 client.17256 handle_command_reply: tid=0 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 client.17256 shutdown 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 msgr2=0x557cc8a521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a521e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40ac430 tx=0x7f4cd40b1160 comp rx=0 tx=0).stop 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 shutdown_connections 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557cc8a49d80 0x557cc8a4cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557cc8acb3a0 0x557cc8acd7c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557cc8a3ddf0 0x557cc8a521e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 --2- 172.21.15.33:0/1278200831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557cc8a3d4d0 0x557cc8a51ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 >> 172.21.15.33:0/1278200831 conn(0x557cc8a3b300 msgr2=0x557cc8a44ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cc7fff640 0 client.17256 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:00.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 shutdown_connections 2024-09-10T21:14:00.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 1 -- 172.21.15.33:0/1278200831 wait complete. 2024-09-10T21:14:00.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.223+0000 7f4cf134c740 20 client.17256 trim_cache size 0 max 16384 2024-09-10T21:14:00.233 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cd80b52e0 msgr2=0x7f4cd80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cd80b52e0 0x7f4cd80b77a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0004af0 tx=0x7f4ce0005c50 comp rx=0 tx=0).stop 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce81045b0 msgr2=0x7f4ce81995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce81045b0 0x7f4ce81995d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400e9f0 tx=0x7f4cd400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 shutdown_connections 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cd80b52e0 0x7f4cd80b77a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ce810d160 0x7f4ce819e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ce81045b0 0x7f4ce81995d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 --2- 172.21.15.33:0/830495606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ce8103bb0 0x7f4ce8199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 >> 172.21.15.33:0/830495606 conn(0x7f4ce80fd700 msgr2=0x7f4ce8104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:00.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 shutdown_connections 2024-09-10T21:14:00.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:00.224+0000 7f4ceef77640 1 -- 172.21.15.33:0/830495606 wait complete. 2024-09-10T21:14:00.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 682176 5.68 MB/sec execute 421 sec latency 39.399 ms 2024-09-10T21:14:01.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:14:01.666 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:01.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:01 smithi033 ceph-mon[30860]: pgmap v647: 97 pgs: 97 active+clean; 267 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-10T21:14:01.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 684428 5.69 MB/sec execute 422 sec latency 12.459 ms 2024-09-10T21:14:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:01 smithi104 ceph-mon[35325]: pgmap v647: 97 pgs: 97 active+clean; 267 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-10T21:14:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:01 smithi174 ceph-mon[35635]: pgmap v647: 97 pgs: 97 active+clean; 267 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 209 op/s 2024-09-10T21:14:02.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 -- 172.21.15.33:0/132316253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 msgr2=0x7fd5fc10cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 --2- 172.21.15.33:0/132316253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 0x7fd5fc10cb70 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec009900 tx=0x7fd5ec02a410 comp rx=0 tx=0).stop 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 -- 172.21.15.33:0/132316253 shutdown_connections 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 --2- 172.21.15.33:0/132316253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5fc10d140 0x7fd5fc10f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 --2- 172.21.15.33:0/132316253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 0x7fd5fc10cb70 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 --2- 172.21.15.33:0/132316253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 0x7fd5fc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 -- 172.21.15.33:0/132316253 >> 172.21.15.33:0/132316253 conn(0x7fd5fc0fd700 msgr2=0x7fd5fc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 -- 172.21.15.33:0/132316253 shutdown_connections 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.170+0000 7fd60439a640 1 -- 172.21.15.33:0/132316253 wait complete. 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 Processor -- start 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 -- start start 2024-09-10T21:14:02.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 0x7fd5fc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 0x7fd5fc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5fc10d140 0x7fd5fc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc112da0 con 0x7fd5fc104590 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc112c20 con 0x7fd5fc103b90 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.171+0000 7fd60439a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc112f20 con 0x7fd5fc10d140 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd601935640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 0x7fd5fc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 0x7fd5fc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 0x7fd5fc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43612/0 (socket says 172.21.15.33:43612) 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 -- 172.21.15.33:0/1386468493 learned_addr learned my addr 172.21.15.33:0/1386468493 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602937640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5fc10d140 0x7fd5fc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 -- 172.21.15.33:0/1386468493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5fc10d140 msgr2=0x7fd5fc19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5fc10d140 0x7fd5fc19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 -- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 msgr2=0x7fd5fc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 0x7fd5fc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 -- 172.21.15.33:0/1386468493 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc19f340 con 0x7fd5fc103b90 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd602136640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 0x7fd5fc199290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800e9f0 tx=0x7fd5f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd5f37fe640 1 -- 172.21.15.33:0/1386468493 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800cd80 con 0x7fd5fc103b90 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc19f630 con 0x7fd5fc103b90 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd5f37fe640 1 -- 172.21.15.33:0/1386468493 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f80108e0 con 0x7fd5fc103b90 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd5f37fe640 1 -- 172.21.15.33:0/1386468493 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8018980 con 0x7fd5fc103b90 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.172+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1a5cf0 con 0x7fd5fc103b90 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 Processor -- start 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 -- start start 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b26684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2678870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636b25deea0 con 0x5636b2668a10 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636b25ce2d0 con 0x5636b2678db0 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd60676f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5636b25d20e0 con 0x5636b26680f0 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd602937640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b26684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd602937640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b26684d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:33216/0 (socket says 172.21.15.33:33216) 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd601935640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.175+0000 7fd602937640 1 -- 172.21.15.33:0/3753147303 learned_addr learned my addr 172.21.15.33:0/3753147303 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd602937640 1 -- 172.21.15.33:0/3753147303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 msgr2=0x5636b267b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd602937640 1 --2- 172.21.15.33:0/3753147303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd602937640 1 -- 172.21.15.33:0/3753147303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 msgr2=0x5636b2678870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd602937640 1 --2- 172.21.15.33:0/3753147303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2678870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd602937640 1 -- 172.21.15.33:0/3753147303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b232c650 con 0x5636b26680f0 2024-09-10T21:14:02.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd602937640 1 --2- 172.21.15.33:0/3753147303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b26684d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4011440 tx=0x7fd5f4011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df91261ee67a57c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd5f37fe640 1 -- 172.21.15.33:0/1386468493 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd5f8010430 con 0x7fd5fc103b90 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd5f0de6640 1 -- 172.21.15.33:0/3753147303 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4004d20 con 0x5636b26680f0 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd5f0de6640 1 -- 172.21.15.33:0/3753147303 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f4004050 con 0x5636b26680f0 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.176+0000 7fd5f0de6640 1 -- 172.21.15.33:0/3753147303 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400f7a0 con 0x5636b26680f0 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd5f37fe640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5e80b52e0 0x7fd5e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd60676f740 1 -- 172.21.15.33:0/3753147303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 msgr2=0x5636b26684d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd60676f740 1 --2- 172.21.15.33:0/3753147303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b26684d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4011440 tx=0x7fd5f4011900 comp rx=0 tx=0).stop 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd60676f740 1 -- 172.21.15.33:0/3753147303 shutdown_connections 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd60676f740 1 --2- 172.21.15.33:0/3753147303 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd60676f740 1 --2- 172.21.15.33:0/3753147303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2678870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd60676f740 1 --2- 172.21.15.33:0/3753147303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b26684d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd60676f740 1 -- 172.21.15.33:0/3753147303 >> 172.21.15.33:0/3753147303 conn(0x5636b2665f20 msgr2=0x5636b26670c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.177+0000 7fd5f37fe640 1 -- 172.21.15.33:0/1386468493 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd5f80e5f20 con 0x7fd5fc103b90 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.178+0000 7fd602136640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5e80b52e0 0x7fd5e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.178+0000 7fd60676f740 1 -- 172.21.15.33:0/3753147303 shutdown_connections 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.178+0000 7fd60676f740 1 -- 172.21.15.33:0/3753147303 wait complete. 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.178+0000 7fd602136640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5e80b52e0 0x7fd5e80b77a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80e3a20 tx=0x7fd5f8018420 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.178+0000 7fd60676f740 1 -- start start 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.178+0000 7fd60676f740 1 Processor -- start 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd60676f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b26680f0 0x5636b2678760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd60676f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2675c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd60676f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2678db0 0x5636b26762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd60676f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5636b242f700 con 0x5636b2678db0 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd60676f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636b2569160 con 0x5636b26680f0 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd60676f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5636b267ba60 con 0x5636b2668a10 2024-09-10T21:14:02.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd60676f740 10 client.0 _ll_register_callbacks cb 0x5636b25829e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd602937640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b26680f0 0x5636b2678760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2675c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2675c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:33226/0 (socket says 172.21.15.33:33226) 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 -- 172.21.15.33:0/1326123454 learned_addr learned my addr 172.21.15.33:0/1326123454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd602937640 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2678db0 0x5636b26762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 -- 172.21.15.33:0/1326123454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b26680f0 msgr2=0x5636b2678760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b26680f0 0x5636b2678760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 -- 172.21.15.33:0/1326123454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2678db0 msgr2=0x5636b26762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2678db0 0x5636b26762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 -- 172.21.15.33:0/1326123454 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b2596c90 con 0x5636b2668a10 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd601935640 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2675c00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec02a930 tx=0x7fd5ec032f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a72a284564cded40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd5e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:14:02.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.179+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/1326123454 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec035040 con 0x5636b2668a10 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5f4004050 con 0x5636b2668a10 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/1326123454 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd5ec031720 con 0x5636b2668a10 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/1326123454 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec031920 con 0x5636b2668a10 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 10 client.26659 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5636b26874f0 con 0x5636b2668a10 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/1326123454 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f4004050 con 0x5636b2668a10 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/1326123454 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd5ec004750 con 0x5636b2668a10 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 10 client.26659 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 10 client.26659 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 20 client.26659 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b2688950 0x5636b268ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 4 client.26659 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636b25e0df0 con 0x5636b2688950 2024-09-10T21:14:02.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.180+0000 7fd602937640 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b2688950 0x5636b268ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.181+0000 7fd602937640 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b2688950 0x5636b268ad70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82346f587824ca2f server_cookie=7fc6914f8bc3bc70 in_seq=0 out_seq=0 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.181+0000 7fd5e5ffb640 10 client.26659 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/1326123454 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636b25e0df0 con 0x5636b2688950 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd5e5ffb640 10 client.26659 handle_command_reply: tid=0 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 client.26659 shutdown 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 msgr2=0x5636b2675c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2675c00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec02a930 tx=0x7fd5ec032f00 comp rx=0 tx=0).stop 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 shutdown_connections 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b2688950 0x5636b268ad70 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2678db0 0x5636b26762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2675c00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd5e5ffb640 0 client.26659 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:02.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 --2- 172.21.15.33:0/1326123454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b26680f0 0x5636b2678760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 >> 172.21.15.33:0/1326123454 conn(0x5636b2665f20 msgr2=0x5636b266bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 shutdown_connections 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 1 -- 172.21.15.33:0/1326123454 wait complete. 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.182+0000 7fd60676f740 20 client.26659 trim_cache size 0 max 16384 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.195+0000 7fd60676f740 1 Processor -- start 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.195+0000 7fd60676f740 1 -- start start 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd60676f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b2684cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd60676f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2675130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd60676f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b2690620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd60676f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ac150 con 0x5636b2668a10 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd60676f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1adc90 con 0x5636b2678db0 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd60676f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ac820 con 0x5636b26680f0 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2675130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2675130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48698/0 (socket says 172.21.15.33:48698) 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 -- 172.21.15.33:0/2569310409 learned_addr learned my addr 172.21.15.33:0/2569310409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd602937640 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b2684cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd602937640 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b2690620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 -- 172.21.15.33:0/2569310409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 msgr2=0x5636b2684cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b2684cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 -- 172.21.15.33:0/2569310409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 msgr2=0x5636b2690620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:02.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b2690620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 -- 172.21.15.33:0/2569310409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636b26617e0 con 0x5636b2668a10 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd602937640 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b2690620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.196+0000 7fd601935640 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2675130 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec004d10 tx=0x7fd5ec0048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f913cea722a0a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd5e7fff640 1 -- 172.21.15.33:0/2569310409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec035070 con 0x5636b2668a10 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd5e7fff640 1 -- 172.21.15.33:0/2569310409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec0316a0 con 0x5636b2668a10 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd5e7fff640 1 -- 172.21.15.33:0/2569310409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec046470 con 0x5636b2668a10 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 -- 172.21.15.33:0/2569310409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 msgr2=0x5636b2675130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2675130 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec004d10 tx=0x7fd5ec0048d0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 -- 172.21.15.33:0/2569310409 shutdown_connections 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b2690620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b2668a10 0x5636b2675130 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 --2- 172.21.15.33:0/2569310409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b26680f0 0x5636b2684cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 -- 172.21.15.33:0/2569310409 >> 172.21.15.33:0/2569310409 conn(0x5636b2665f20 msgr2=0x5636b2666c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 -- 172.21.15.33:0/2569310409 shutdown_connections 2024-09-10T21:14:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.197+0000 7fd60676f740 1 -- 172.21.15.33:0/2569310409 wait complete. 2024-09-10T21:14:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 -- start start 2024-09-10T21:14:02.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 Processor -- start 2024-09-10T21:14:02.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b26680f0 0x5636b2680f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2681440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1ad5c0 con 0x5636b26680f0 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1acef0 con 0x5636b2678db0 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc1aa110 con 0x5636b2668a10 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd60676f740 10 client.0 _ll_register_callbacks cb 0x7fd5ec009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd602937640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b26680f0 0x5636b2680f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd602937640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b26680f0 0x5636b2680f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48704/0 (socket says 172.21.15.33:48704) 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd602937640 1 -- 172.21.15.33:0/2274096909 learned_addr learned my addr 172.21.15.33:0/2274096909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd601935640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2681440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.198+0000 7fd602937640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd602937640 1 -- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 msgr2=0x5636b2681440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd602937640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2681440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd602937640 1 -- 172.21.15.33:0/2274096909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 msgr2=0x5636b267dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd602937640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd602937640 1 -- 172.21.15.33:0/2274096909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636b267e3e0 con 0x5636b26680f0 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd601935640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2681440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd602937640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b26680f0 0x5636b2680f00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4003c40 tx=0x7fd5ec032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f1a37ffc867d7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd5e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/2274096909 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f401d040 con 0x5636b26680f0 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/2274096909 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd5f4004ce0 con 0x5636b26680f0 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ec0316a0 con 0x5636b26680f0 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/2274096909 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fd5f4004050 con 0x5636b26680f0 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd60676f740 10 client.17271 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:02.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.199+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5636b26f6210 con 0x5636b26680f0 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/2274096909 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5ec0316a0 con 0x5636b26680f0 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/2274096909 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fd5f4039030 con 0x5636b26680f0 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd60676f740 10 client.17271 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd60676f740 10 client.17271 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd60676f740 20 client.17271 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd60676f740 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b26f6770 0x5636b26f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd60676f740 4 client.17271 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636b25dc8a0 con 0x5636b26f6770 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd601935640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b26f6770 0x5636b26f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd601935640 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b26f6770 0x5636b26f8b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fde96ccbd83f79b server_cookie=d39563c61b4092c1 in_seq=0 out_seq=0 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.200+0000 7fd5e5ffb640 10 client.17271 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd5e5ffb640 1 -- 172.21.15.33:0/2274096909 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636b25dc8a0 con 0x5636b26f6770 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd5e5ffb640 10 client.17271 handle_command_reply: tid=0 2024-09-10T21:14:02.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 client.17271 shutdown 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b26680f0 msgr2=0x5636b2680f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b26680f0 0x5636b2680f00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4003c40 tx=0x7fd5ec032b20 comp rx=0 tx=0).stop 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 shutdown_connections 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636b2678db0 0x5636b267dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636b2668a10 0x5636b2681440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5636b26f6770 0x5636b26f8b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 --2- 172.21.15.33:0/2274096909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636b26680f0 0x5636b2680f00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 >> 172.21.15.33:0/2274096909 conn(0x5636b2665f20 msgr2=0x5636b2664750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd5e5ffb640 0 client.17271 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 shutdown_connections 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 1 -- 172.21.15.33:0/2274096909 wait complete. 2024-09-10T21:14:02.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.201+0000 7fd60676f740 20 client.17271 trim_cache size 0 max 16384 2024-09-10T21:14:02.221 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:02.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5e80b52e0 msgr2=0x7fd5e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5e80b52e0 0x7fd5e80b77a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd5f80e3a20 tx=0x7fd5f8018420 comp rx=0 tx=0).stop 2024-09-10T21:14:02.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 msgr2=0x7fd5fc199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 0x7fd5fc199290 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800e9f0 tx=0x7fd5f800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 shutdown_connections 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd5e80b52e0 0x7fd5e80b77a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5fc10d140 0x7fd5fc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5fc104590 0x7fd5fc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 --2- 172.21.15.33:0/1386468493 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd5fc103b90 0x7fd5fc199290 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd5f800e9f0 tx=0x7fd5f800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 >> 172.21.15.33:0/1386468493 conn(0x7fd5fc0fd700 msgr2=0x7fd5fc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 shutdown_connections 2024-09-10T21:14:02.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:02.202+0000 7fd60439a640 1 -- 172.21.15.33:0/1386468493 wait complete. 2024-09-10T21:14:02.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 686058 5.70 MB/sec execute 423 sec latency 45.877 ms 2024-09-10T21:14:03.046 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:14:03.330 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:03.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 688758 5.72 MB/sec execute 424 sec latency 27.450 ms 2024-09-10T21:14:03.761 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:03 smithi033 ceph-mon[30860]: pgmap v648: 97 pgs: 97 active+clean; 271 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-09-10T21:14:03.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:03 smithi104 ceph-mon[35325]: pgmap v648: 97 pgs: 97 active+clean; 271 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-09-10T21:14:03.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.865+0000 7f57b8ff1640 1 -- 172.21.15.33:0/3774746094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4102dc0 msgr2=0x7f57b41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.865+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/3774746094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4102dc0 0x7f57b41031a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f57a4009900 tx=0x7f57a402a410 comp rx=0 tx=0).stop 2024-09-10T21:14:03.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 -- 172.21.15.33:0/3774746094 shutdown_connections 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/3774746094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57b4109150 0x7f57b410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/3774746094 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b41087d0 0x7f57b4108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/3774746094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b4102dc0 0x7f57b41031a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 -- 172.21.15.33:0/3774746094 >> 172.21.15.33:0/3774746094 conn(0x7f57b40fe830 msgr2=0x7f57b4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 -- 172.21.15.33:0/3774746094 shutdown_connections 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 -- 172.21.15.33:0/3774746094 wait complete. 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 Processor -- start 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.866+0000 7f57b8ff1640 1 -- start start 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b8ff1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57b4102dc0 0x7f57b40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b8ff1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41087d0 0x7f57b40759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b8ff1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b4109150 0x7f57b4075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b8ff1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4112e10 con 0x7f57b41087d0 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b8ff1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4112c90 con 0x7f57b4109150 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b8ff1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57b4112f90 con 0x7f57b4102dc0 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41087d0 0x7f57b40759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57b4102dc0 0x7f57b40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41087d0 0x7f57b40759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48724/0 (socket says 172.21.15.33:48724) 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b1d9b640 1 -- 172.21.15.33:0/1229041017 learned_addr learned my addr 172.21.15.33:0/1229041017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b4109150 0x7f57b4075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b259c640 1 -- 172.21.15.33:0/1229041017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b4109150 msgr2=0x7f57b4075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b259c640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b4109150 0x7f57b4075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b259c640 1 -- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41087d0 msgr2=0x7f57b40759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b259c640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41087d0 0x7f57b40759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b259c640 1 -- 172.21.15.33:0/1229041017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b41a76c0 con 0x7f57b4102dc0 2024-09-10T21:14:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41087d0 0x7f57b40759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:03.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.867+0000 7f57b259c640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57b4102dc0 0x7f57b40754a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57a402a930 tx=0x7f57a4031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.868+0000 7f57a37fe640 1 -- 172.21.15.33:0/1229041017 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a403f030 con 0x7f57b4102dc0 2024-09-10T21:14:03.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.868+0000 7f57a37fe640 1 -- 172.21.15.33:0/1229041017 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a4004490 con 0x7f57b4102dc0 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.868+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b41a7950 con 0x7f57b4102dc0 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.868+0000 7f57a37fe640 1 -- 172.21.15.33:0/1229041017 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a4035070 con 0x7f57b4102dc0 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.868+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b41a7e30 con 0x7f57b4102dc0 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.870+0000 7f57bb3c6740 1 Processor -- start 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.870+0000 7f57a37fe640 1 -- 172.21.15.33:0/1229041017 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f57a403ba30 con 0x7f57b4102dc0 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57bb3c6740 1 -- start start 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c875508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8755c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c8755f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558c874c6ea0 con 0x558c8755cd80 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c874b62d0 con 0x558c875504d0 2024-09-10T21:14:03.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c874ba0e0 con 0x558c87550df0 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57b259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8755c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57b2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c875508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57b2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c875508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43680/0 (socket says 172.21.15.33:43680) 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57b259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8755c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:33262/0 (socket says 172.21.15.33:33262) 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.871+0000 7f57b2d9d640 1 -- 172.21.15.33:0/81775155 learned_addr learned my addr 172.21.15.33:0/81775155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c8755f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b2d9d640 1 -- 172.21.15.33:0/81775155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 msgr2=0x558c8755c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8755c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b2d9d640 1 -- 172.21.15.33:0/81775155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 msgr2=0x558c8755f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c8755f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b2d9d640 1 -- 172.21.15.33:0/81775155 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c87214650 con 0x558c875504d0 2024-09-10T21:14:03.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c8755f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b259c640 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8755c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c875508b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009f90 tx=0x7f57ac0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e52f312e65d27e84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57a37fe640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f579c0b5360 0x7f579c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57a0ff9640 1 -- 172.21.15.33:0/81775155 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac004d20 con 0x558c875504d0 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57a0ff9640 1 -- 172.21.15.33:0/81775155 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ac004050 con 0x558c875504d0 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57a0ff9640 1 -- 172.21.15.33:0/81775155 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac005280 con 0x558c875504d0 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.872+0000 7f57b259c640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f579c0b5360 0x7f579c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57a37fe640 1 -- 172.21.15.33:0/1229041017 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f57a4106160 con 0x7f57b4102dc0 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 -- 172.21.15.33:0/81775155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 msgr2=0x558c875508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c875508b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57ac009f90 tx=0x7f57ac0118d0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57b259c640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f579c0b5360 0x7f579c0b7820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f57a40c8f60 tx=0x7f57a410b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 -- 172.21.15.33:0/81775155 shutdown_connections 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c8755f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8755c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/81775155 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c875508b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 -- 172.21.15.33:0/81775155 >> 172.21.15.33:0/81775155 conn(0x558c8754e300 msgr2=0x558c8754f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 -- 172.21.15.33:0/81775155 shutdown_connections 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.873+0000 7f57bb3c6740 1 -- 172.21.15.33:0/81775155 wait complete. 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 -- start start 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 Processor -- start 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c8755c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c87550df0 0x558c87559a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8755a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558c87317700 con 0x558c87550df0 2024-09-10T21:14:03.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558c87451160 con 0x558c875504d0 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558c8755fa00 con 0x558c8755cd80 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57bb3c6740 10 client.0 _ll_register_callbacks cb 0x558c8746a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c87550df0 0x558c87559a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c87550df0 0x558c87559a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48742/0 (socket says 172.21.15.33:48742) 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57b1d9b640 1 -- 172.21.15.33:0/3881988695 learned_addr learned my addr 172.21.15.33:0/3881988695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8755a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c8755c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.874+0000 7f57b1d9b640 1 -- 172.21.15.33:0/3881988695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 msgr2=0x558c8755a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8755a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57b1d9b640 1 -- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 msgr2=0x558c8755c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c8755c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57b1d9b640 1 -- 172.21.15.33:0/3881988695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c8747ec90 con 0x558c87550df0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c87550df0 0x558c87559a60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f57a80129f0 tx=0x7f57a8012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0eb9dfcb5f6a327 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f5799ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f5799ffb640 1 -- 172.21.15.33:0/3881988695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a800fac0 con 0x558c87550df0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f5799ffb640 1 -- 172.21.15.33:0/3881988695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f57a800fc60 con 0x558c87550df0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57ac004050 con 0x558c87550df0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f5799ffb640 1 -- 172.21.15.33:0/3881988695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f57a800fe60 con 0x558c87550df0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57bb3c6740 10 client.17283 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x558c87573b70 con 0x558c87550df0 2024-09-10T21:14:03.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f5799ffb640 1 -- 172.21.15.33:0/3881988695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57ac004050 con 0x558c87550df0 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.875+0000 7f5799ffb640 1 -- 172.21.15.33:0/3881988695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f57a80104c0 con 0x558c87550df0 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57bb3c6740 10 client.17283 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57bb3c6740 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57bb3c6740 20 client.17283 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c87575080 0x558c875774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57bb3c6740 4 client.17283 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c874c8df0 con 0x558c87575080 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c87575080 0x558c875774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f5799ffb640 10 client.17283 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.876+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c87575080 0x558c875774c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283f356afa9bde83 server_cookie=244a3897f86acc76 in_seq=0 out_seq=0 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f5799ffb640 1 -- 172.21.15.33:0/3881988695 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c874c8df0 con 0x558c87575080 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f5799ffb640 10 client.17283 handle_command_reply: tid=0 2024-09-10T21:14:03.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 client.17283 shutdown 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c87550df0 msgr2=0x558c87559a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c87550df0 0x558c87559a60 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f57a80129f0 tx=0x7f57a8012eb0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 shutdown_connections 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c87575080 0x558c875774c0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8755a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c87550df0 0x558c87559a60 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/3881988695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c8755c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 >> 172.21.15.33:0/3881988695 conn(0x558c8754e300 msgr2=0x558c87553e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f5799ffb640 0 client.17283 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 shutdown_connections 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.877+0000 7f57bb3c6740 1 -- 172.21.15.33:0/3881988695 wait complete. 2024-09-10T21:14:03.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.878+0000 7f57bb3c6740 20 client.17283 trim_cache size 0 max 16384 2024-09-10T21:14:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:03 smithi174 ceph-mon[35635]: pgmap v648: 97 pgs: 97 active+clean; 271 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 287 op/s 2024-09-10T21:14:03.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.891+0000 7f57bb3c6740 1 Processor -- start 2024-09-10T21:14:03.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.891+0000 7f57bb3c6740 1 -- start start 2024-09-10T21:14:03.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c875504d0 0x558c8754dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c87550df0 0x558c8754f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8754f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ac160 con 0x558c875504d0 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41adca0 con 0x558c87550df0 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ac830 con 0x558c8755cd80 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c875504d0 0x558c8754dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c87550df0 0x558c8754f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c87550df0 0x558c8754f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43704/0 (socket says 172.21.15.33:43704) 2024-09-10T21:14:03.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 -- 172.21.15.33:0/2334096450 learned_addr learned my addr 172.21.15.33:0/2334096450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8754f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 -- 172.21.15.33:0/2334096450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 msgr2=0x558c8754f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8754f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 -- 172.21.15.33:0/2334096450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c875504d0 msgr2=0x558c8754dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c875504d0 0x558c8754dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 -- 172.21.15.33:0/2334096450 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c8755a8d0 con 0x558c87550df0 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8754f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c87550df0 0x558c8754f3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009280 tx=0x7f57a80092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55fe1651e0e1d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.892+0000 7f579bfff640 1 -- 172.21.15.33:0/2334096450 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a800fac0 con 0x558c87550df0 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f579bfff640 1 -- 172.21.15.33:0/2334096450 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a800fc60 con 0x558c87550df0 2024-09-10T21:14:03.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f579bfff640 1 -- 172.21.15.33:0/2334096450 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a8005460 con 0x558c87550df0 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 -- 172.21.15.33:0/2334096450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c87550df0 msgr2=0x558c8754f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c87550df0 0x558c8754f3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009280 tx=0x7f57a80092b0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 -- 172.21.15.33:0/2334096450 shutdown_connections 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c8755cd80 0x558c8754f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c87550df0 0x558c8754f3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/2334096450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c875504d0 0x558c8754dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 -- 172.21.15.33:0/2334096450 >> 172.21.15.33:0/2334096450 conn(0x558c8754e300 msgr2=0x558c87554c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 -- 172.21.15.33:0/2334096450 shutdown_connections 2024-09-10T21:14:03.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.893+0000 7f57bb3c6740 1 -- 172.21.15.33:0/2334096450 wait complete. 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 -- start start 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 Processor -- start 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c87569c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8756a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c875e0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41ad5d0 con 0x558c8755cd80 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41acf00 con 0x558c875504d0 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57b41aa120 con 0x558c87550df0 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57bb3c6740 10 client.0 _ll_register_callbacks cb 0x7f57a8009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57b2d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c87569c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8756a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57b2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c875e0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57b1d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8756a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:33290/0 (socket says 172.21.15.33:33290) 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.894+0000 7f57b1d9b640 1 -- 172.21.15.33:0/4278163808 learned_addr learned my addr 172.21.15.33:0/4278163808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57b1d9b640 1 -- 172.21.15.33:0/4278163808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 msgr2=0x558c87569c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c87569c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57b1d9b640 1 -- 172.21.15.33:0/4278163808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 msgr2=0x558c875e0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c875e0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57b1d9b640 1 -- 172.21.15.33:0/4278163808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c875e1550 con 0x558c87550df0 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c875e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57b1d9b640 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8756a150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f57a800fe30 tx=0x7f57a80106c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc66c9428eb644d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f5799ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f5799ffb640 1 -- 172.21.15.33:0/4278163808 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a800fac0 con 0x558c87550df0 2024-09-10T21:14:03.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f5799ffb640 1 -- 172.21.15.33:0/4278163808 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f57a80052f0 con 0x558c87550df0 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57a800fc60 con 0x558c87550df0 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f5799ffb640 1 -- 172.21.15.33:0/4278163808 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f57a800f790 con 0x558c87550df0 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57bb3c6740 10 client.26680 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x558c875e27c0 con 0x558c87550df0 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.895+0000 7f5799ffb640 1 -- 172.21.15.33:0/4278163808 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57a800fc60 con 0x558c87550df0 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f5799ffb640 1 -- 172.21.15.33:0/4278163808 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f57a801c470 con 0x558c87550df0 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57bb3c6740 10 client.26680 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57bb3c6740 10 client.26680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57bb3c6740 20 client.26680 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c875e2ec0 0x558c875e52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57bb3c6740 4 client.26680 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:03.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558c874c48a0 con 0x558c875e2ec0 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c875e2ec0 0x558c875e52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f57b2d9d640 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c875e2ec0 0x558c875e52e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ecfa182a7ad2750 server_cookie=91bb833d945ec94e in_seq=0 out_seq=0 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.896+0000 7f5799ffb640 10 client.26680 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f5799ffb640 1 -- 172.21.15.33:0/4278163808 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558c874c48a0 con 0x558c875e2ec0 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f5799ffb640 10 client.26680 handle_command_reply: tid=0 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 client.26680 shutdown 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 msgr2=0x558c8756a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8756a150 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f57a800fe30 tx=0x7f57a80106c0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 shutdown_connections 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558c875e2ec0 0x558c875e52e0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558c8755cd80 0x558c875e0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558c87550df0 0x558c8756a150 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f5799ffb640 0 client.26680 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 --2- 172.21.15.33:0/4278163808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558c875504d0 0x558c87569c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 >> 172.21.15.33:0/4278163808 conn(0x558c8754e300 msgr2=0x558c8754f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 shutdown_connections 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 1 -- 172.21.15.33:0/4278163808 wait complete. 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57bb3c6740 20 client.26680 trim_cache size 0 max 16384 2024-09-10T21:14:03.908 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f579c0b5360 msgr2=0x7f579c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f579c0b5360 0x7f579c0b7820 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f57a40c8f60 tx=0x7f57a410b040 comp rx=0 tx=0).stop 2024-09-10T21:14:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57b4102dc0 msgr2=0x7f57b40754a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57b4102dc0 0x7f57b40754a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f57a402a930 tx=0x7f57a4031a20 comp rx=0 tx=0).stop 2024-09-10T21:14:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 shutdown_connections 2024-09-10T21:14:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f579c0b5360 0x7f579c0b7820 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57b4109150 0x7f57b4075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57b41087d0 0x7f57b40759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 --2- 172.21.15.33:0/1229041017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57b4102dc0 0x7f57b40754a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.897+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 >> 172.21.15.33:0/1229041017 conn(0x7f57b40fe830 msgr2=0x7f57b4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.898+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 shutdown_connections 2024-09-10T21:14:03.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:03.898+0000 7f57b8ff1640 1 -- 172.21.15.33:0/1229041017 wait complete. 2024-09-10T21:14:04.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 690112 5.72 MB/sec execute 425 sec latency 32.940 ms 2024-09-10T21:14:05.648 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:14:05.649 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:14:05.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 692740 5.75 MB/sec execute 426 sec latency 13.929 ms 2024-09-10T21:14:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:05 smithi104 ceph-mon[35325]: pgmap v649: 97 pgs: 97 active+clean; 268 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-09-10T21:14:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:05 smithi174 ceph-mon[35635]: pgmap v649: 97 pgs: 97 active+clean; 268 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-09-10T21:14:05.920 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:05.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:05 smithi033 ceph-mon[30860]: pgmap v649: 97 pgs: 97 active+clean; 268 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 -- 172.21.15.33:0/3887658559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c109160 msgr2=0x7f337c10f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 --2- 172.21.15.33:0/3887658559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c109160 0x7f337c10f960 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3374009ca0 tx=0x7f337402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 -- 172.21.15.33:0/3887658559 shutdown_connections 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 --2- 172.21.15.33:0/3887658559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c109160 0x7f337c10f960 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 --2- 172.21.15.33:0/3887658559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f337c1087e0 0x7f337c108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 --2- 172.21.15.33:0/3887658559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c102dd0 0x7f337c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 -- 172.21.15.33:0/3887658559 >> 172.21.15.33:0/3887658559 conn(0x7f337c0fe800 msgr2=0x7f337c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 -- 172.21.15.33:0/3887658559 shutdown_connections 2024-09-10T21:14:06.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.405+0000 7f338213c640 1 -- 172.21.15.33:0/3887658559 wait complete. 2024-09-10T21:14:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.406+0000 7f338213c640 1 Processor -- start 2024-09-10T21:14:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.406+0000 7f338213c640 1 -- start start 2024-09-10T21:14:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f338213c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c102dd0 0x7f337c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f338213c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1087e0 0x7f337c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f338213c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f337c109160 0x7f337c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f338213c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112e70 con 0x7f337c102dd0 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f338213c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112cf0 con 0x7f337c1087e0 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f338213c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f337c112ff0 con 0x7f337c109160 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c102dd0 0x7f337c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c102dd0 0x7f337c0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48764/0 (socket says 172.21.15.33:48764) 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337b7fe640 1 -- 172.21.15.33:0/1697924326 learned_addr learned my addr 172.21.15.33:0/1697924326 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337affd640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1087e0 0x7f337c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337bfff640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f337c109160 0x7f337c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337affd640 1 -- 172.21.15.33:0/1697924326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f337c109160 msgr2=0x7f337c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337affd640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f337c109160 0x7f337c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337affd640 1 -- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c102dd0 msgr2=0x7f337c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337affd640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c102dd0 0x7f337c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.407+0000 7f337affd640 1 -- 172.21.15.33:0/1697924326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f337c1a7640 con 0x7f337c1087e0 2024-09-10T21:14:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.408+0000 7f337affd640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1087e0 0x7f337c0759e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f336400d6d0 tx=0x7f336400db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.408+0000 7f3378ff9640 1 -- 172.21.15.33:0/1697924326 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364017070 con 0x7f337c1087e0 2024-09-10T21:14:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.408+0000 7f3378ff9640 1 -- 172.21.15.33:0/1697924326 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364009820 con 0x7f337c1087e0 2024-09-10T21:14:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.408+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f337c1a7930 con 0x7f337c1087e0 2024-09-10T21:14:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.408+0000 7f3378ff9640 1 -- 172.21.15.33:0/1697924326 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364004c40 con 0x7f337c1087e0 2024-09-10T21:14:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.408+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f337c1a7e70 con 0x7f337c1087e0 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.410+0000 7f3384511740 1 Processor -- start 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3378ff9640 1 -- 172.21.15.33:0/1697924326 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3364005220 con 0x7f337c1087e0 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3384511740 1 -- start start 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3384511740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3384511740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b99c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3384511740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3384511740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55718b906e60 con 0x55718b990800 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3384511740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55718b8f6290 con 0x55718b98fee0 2024-09-10T21:14:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f3384511740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55718b8fa0a0 con 0x55718b99c790 2024-09-10T21:14:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b99c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43754/0 (socket says 172.21.15.33:43754) 2024-09-10T21:14:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 -- 172.21.15.33:0/1894820756 learned_addr learned my addr 172.21.15.33:0/1894820756 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 -- 172.21.15.33:0/1894820756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 msgr2=0x55718b99eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 --2- 172.21.15.33:0/1894820756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 -- 172.21.15.33:0/1894820756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 msgr2=0x55718b99c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 --2- 172.21.15.33:0/1894820756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b99c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.411+0000 7f337bfff640 1 -- 172.21.15.33:0/1894820756 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718b654930 con 0x55718b98fee0 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.412+0000 7f3378ff9640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33680b52e0 0x7f33680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.412+0000 7f337bfff640 1 --2- 172.21.15.33:0/1894820756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9902c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3374009ca0 tx=0x7f337402f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7af13db47c50fa07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.412+0000 7f337b7fe640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33680b52e0 0x7f33680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.412+0000 7f3378ff9640 1 -- 172.21.15.33:0/1697924326 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f33640e5620 con 0x7f337c1087e0 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.412+0000 7f3371ffb640 1 -- 172.21.15.33:0/1894820756 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3374031cf0 con 0x55718b98fee0 2024-09-10T21:14:06.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.412+0000 7f3371ffb640 1 -- 172.21.15.33:0/1894820756 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374036030 con 0x55718b98fee0 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.412+0000 7f3371ffb640 1 -- 172.21.15.33:0/1894820756 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33740042a0 con 0x55718b98fee0 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 -- 172.21.15.33:0/1894820756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 msgr2=0x55718b9902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 --2- 172.21.15.33:0/1894820756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9902c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3374009ca0 tx=0x7f337402f310 comp rx=0 tx=0).stop 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f337b7fe640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33680b52e0 0x7f33680b77a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55718b993060 tx=0x7f336c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 -- 172.21.15.33:0/1894820756 shutdown_connections 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 --2- 172.21.15.33:0/1894820756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 --2- 172.21.15.33:0/1894820756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b99c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 --2- 172.21.15.33:0/1894820756 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9902c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 -- 172.21.15.33:0/1894820756 >> 172.21.15.33:0/1894820756 conn(0x55718b98dd10 msgr2=0x55718b98eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 -- 172.21.15.33:0/1894820756 shutdown_connections 2024-09-10T21:14:06.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 -- 172.21.15.33:0/1894820756 wait complete. 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 -- start start 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.413+0000 7f3384511740 1 Processor -- start 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f3384511740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9996f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f3384511740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f3384511740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f3384511740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55718b757890 con 0x55718b990800 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f3384511740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55718b890d40 con 0x55718b98fee0 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f3384511740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55718b99f260 con 0x55718b99c790 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f3384511740 10 client.0 _ll_register_callbacks cb 0x55718b8f4c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43764/0 (socket says 172.21.15.33:43764) 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 -- 172.21.15.33:0/2707422677 learned_addr learned my addr 172.21.15.33:0/2707422677 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337bfff640 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 -- 172.21.15.33:0/2707422677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 msgr2=0x55718b99a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 -- 172.21.15.33:0/2707422677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 msgr2=0x55718b999c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337b7fe640 1 -- 172.21.15.33:0/2707422677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718b9a8610 con 0x55718b98fee0 2024-09-10T21:14:06.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.414+0000 7f337affd640 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f337b7fe640 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9996f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f336c01ea00 tx=0x7f336c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf3f1d7d3458e234 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f335affd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f335affd640 1 -- 172.21.15.33:0/2707422677 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c004020 con 0x55718b98fee0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3374036030 con 0x55718b98fee0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f335affd640 1 -- 172.21.15.33:0/2707422677 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f336c018030 con 0x55718b98fee0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f335affd640 1 -- 172.21.15.33:0/2707422677 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f336c0041c0 con 0x55718b98fee0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f3384511740 10 client.26744 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55718b9ab070 con 0x55718b98fee0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.415+0000 7f335affd640 1 -- 172.21.15.33:0/2707422677 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374036030 con 0x55718b98fee0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f335affd640 1 -- 172.21.15.33:0/2707422677 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f336c00f480 con 0x55718b98fee0 2024-09-10T21:14:06.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f3384511740 10 client.26744 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f3384511740 10 client.26744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f3384511740 20 client.26744 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f3384511740 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718b9ac4d0 0x55718b9ae900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f3384511740 4 client.26744 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55718b908db0 con 0x55718b9ac4d0 2024-09-10T21:14:06.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f337affd640 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718b9ac4d0 0x55718b9ae900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f337affd640 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718b9ac4d0 0x55718b9ae900 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0012070b408313 server_cookie=25b6af6a62929c9f in_seq=0 out_seq=0 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.416+0000 7f335affd640 10 client.26744 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f335affd640 1 -- 172.21.15.33:0/2707422677 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f3364085e50 con 0x55718b9ac4d0 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f335affd640 10 client.26744 handle_command_reply: tid=0 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 client.26744 shutdown 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 msgr2=0x55718b9996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9996f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f336c01ea00 tx=0x7f336c004730 comp rx=0 tx=0).stop 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 shutdown_connections 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718b9ac4d0 0x55718b9ae900 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b99a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 --2- 172.21.15.33:0/2707422677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b9996f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 >> 172.21.15.33:0/2707422677 conn(0x55718b98dd10 msgr2=0x55718b98cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f335affd640 0 client.26744 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 shutdown_connections 2024-09-10T21:14:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.417+0000 7f3384511740 1 -- 172.21.15.33:0/2707422677 wait complete. 2024-09-10T21:14:06.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.418+0000 7f3384511740 20 client.26744 trim_cache size 0 max 16384 2024-09-10T21:14:06.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 Processor -- start 2024-09-10T21:14:06.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 -- start start 2024-09-10T21:14:06.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b998f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b996900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1b0510 con 0x55718b990800 2024-09-10T21:14:06.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1b2050 con 0x55718b98fee0 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f3384511740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1b0be0 con 0x55718b99c790 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f337affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f337affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48800/0 (socket says 172.21.15.33:48800) 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f337affd640 1 -- 172.21.15.33:0/4092907153 learned_addr learned my addr 172.21.15.33:0/4092907153 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f337bfff640 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b996900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.438+0000 7f337b7fe640 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b998f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f337bfff640 1 -- 172.21.15.33:0/4092907153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 msgr2=0x55718b998f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f337bfff640 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b998f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f337bfff640 1 -- 172.21.15.33:0/4092907153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 msgr2=0x55718b999d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f337bfff640 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f337bfff640 1 -- 172.21.15.33:0/4092907153 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718b98e790 con 0x55718b99c790 2024-09-10T21:14:06.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f337bfff640 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b996900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3374009bf0 tx=0x7f3374003cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc15a4ddf1ac93e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f33717fa640 1 -- 172.21.15.33:0/4092907153 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337404a040 con 0x55718b99c790 2024-09-10T21:14:06.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f33717fa640 1 -- 172.21.15.33:0/4092907153 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374036040 con 0x55718b99c790 2024-09-10T21:14:06.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.439+0000 7f33717fa640 1 -- 172.21.15.33:0/4092907153 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337402f870 con 0x55718b99c790 2024-09-10T21:14:06.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 -- 172.21.15.33:0/4092907153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 msgr2=0x55718b996900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b996900 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3374009bf0 tx=0x7f3374003cc0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 -- 172.21.15.33:0/4092907153 shutdown_connections 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b99c790 0x55718b996900 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b990800 0x55718b999d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 --2- 172.21.15.33:0/4092907153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b98fee0 0x55718b998f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 -- 172.21.15.33:0/4092907153 >> 172.21.15.33:0/4092907153 conn(0x55718b98dd10 msgr2=0x55718b999660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 -- 172.21.15.33:0/4092907153 shutdown_connections 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 -- 172.21.15.33:0/4092907153 wait complete. 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.440+0000 7f3384511740 1 -- start start 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 1 Processor -- start 2024-09-10T21:14:06.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b98fee0 0x55718b9a0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b990800 0x55718b9a0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b99c790 0x55718ba183d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1b1980 con 0x55718b99c790 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1b12b0 con 0x55718b990800 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f337c1ae4d0 con 0x55718b98fee0 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f3384511740 10 client.0 _ll_register_callbacks cb 0x7f336c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b99c790 0x55718ba183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b98fee0 0x55718b9a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b99c790 0x55718ba183d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48808/0 (socket says 172.21.15.33:48808) 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337bfff640 1 -- 172.21.15.33:0/2207349065 learned_addr learned my addr 172.21.15.33:0/2207349065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337affd640 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b990800 0x55718b9a0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337b7fe640 1 -- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b990800 msgr2=0x55718b9a0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337b7fe640 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b990800 0x55718b9a0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337b7fe640 1 -- 172.21.15.33:0/2207349065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b99c790 msgr2=0x55718ba183d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337b7fe640 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b99c790 0x55718ba183d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337b7fe640 1 -- 172.21.15.33:0/2207349065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718ba18910 con 0x55718b98fee0 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.441+0000 7f337bfff640 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b99c790 0x55718ba183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f337b7fe640 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b98fee0 0x55718b9a0950 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f336c000c00 tx=0x7f336c0095e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1a5746024641adf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f335affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f335affd640 1 -- 172.21.15.33:0/2207349065 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f336c038040 con 0x55718b98fee0 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3374036040 con 0x55718b98fee0 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f335affd640 1 -- 172.21.15.33:0/2207349065 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f336c024040 con 0x55718b98fee0 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f335affd640 1 -- 172.21.15.33:0/2207349065 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f336c005bf0 con 0x55718b98fee0 2024-09-10T21:14:06.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f3384511740 10 client.26698 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55718ba19c20 con 0x55718b98fee0 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.442+0000 7f335affd640 1 -- 172.21.15.33:0/2207349065 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3374036040 con 0x55718b98fee0 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f335affd640 1 -- 172.21.15.33:0/2207349065 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f336c047020 con 0x55718b98fee0 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f3384511740 10 client.26698 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f3384511740 10 client.26698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f3384511740 20 client.26698 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f3384511740 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718ba1a2d0 0x55718ba1c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f3384511740 4 client.26698 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55718b904860 con 0x55718ba1a2d0 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f337affd640 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718ba1a2d0 0x55718ba1c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f337affd640 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718ba1a2d0 0x55718ba1c6c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2464197bbdfa38d3 server_cookie=51bd4484fac737e3 in_seq=0 out_seq=0 2024-09-10T21:14:06.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.443+0000 7f335affd640 10 client.26698 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.444+0000 7f335affd640 1 -- 172.21.15.33:0/2207349065 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55718b904860 con 0x55718ba1a2d0 2024-09-10T21:14:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.444+0000 7f335affd640 10 client.26698 handle_command_reply: tid=0 2024-09-10T21:14:06.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.444+0000 7f3384511740 1 client.26698 shutdown 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b98fee0 msgr2=0x55718b9a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b98fee0 0x55718b9a0950 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f336c000c00 tx=0x7f336c0095e0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 shutdown_connections 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55718ba1a2d0 0x55718ba1c6c0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55718b99c790 0x55718ba183d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55718b990800 0x55718b9a0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f335affd640 0 client.26698 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 --2- 172.21.15.33:0/2207349065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55718b98fee0 0x55718b9a0950 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 >> 172.21.15.33:0/2207349065 conn(0x55718b98dd10 msgr2=0x55718b9a7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 shutdown_connections 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 1 -- 172.21.15.33:0/2207349065 wait complete. 2024-09-10T21:14:06.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.445+0000 7f3384511740 20 client.26698 trim_cache size 0 max 16384 2024-09-10T21:14:06.456 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:14:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:14:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "279c3483-bbc8-45ec-93b3-a95fbd081ae3", 2024-09-10T21:14:06.456 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:14:06.456 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33680b52e0 msgr2=0x7f33680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33680b52e0 0x7f33680b77a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x55718b993060 tx=0x7f336c005f90 comp rx=0 tx=0).stop 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1087e0 msgr2=0x7f337c0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1087e0 0x7f337c0759e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f336400d6d0 tx=0x7f336400db90 comp rx=0 tx=0).stop 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 shutdown_connections 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f33680b52e0 0x7f33680b77a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f337c109160 0x7f337c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f337c1087e0 0x7f337c0759e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 --2- 172.21.15.33:0/1697924326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f337c102dd0 0x7f337c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 >> 172.21.15.33:0/1697924326 conn(0x7f337c0fe800 msgr2=0x7f337c100bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 shutdown_connections 2024-09-10T21:14:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:06.446+0000 7f338213c640 1 -- 172.21.15.33:0/1697924326 wait complete. 2024-09-10T21:14:06.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 694582 5.75 MB/sec execute 427 sec latency 37.075 ms 2024-09-10T21:14:07.349 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:14:07.623 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: pgmap v650: 97 pgs: 97 active+clean; 269 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-09-10T21:14:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:14:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.678 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:07 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 696509 5.77 MB/sec execute 428 sec latency 11.567 ms 2024-09-10T21:14:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: pgmap v650: 97 pgs: 97 active+clean; 269 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-09-10T21:14:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:07.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.814 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:07 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: pgmap v650: 97 pgs: 97 active+clean; 269 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 291 op/s 2024-09-10T21:14:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:14:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:07 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:08.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.391+0000 7fecc8a67640 1 -- 172.21.15.33:0/2987552851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 msgr2=0x7fecc410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.391+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2987552851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc410e820 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fecbc009900 tx=0x7fecbc02a410 comp rx=0 tx=0).stop 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 -- 172.21.15.33:0/2987552851 shutdown_connections 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2987552851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc410e820 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2987552851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc41076a0 0x7fecc4107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2987552851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc4101c90 0x7fecc4102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 -- 172.21.15.33:0/2987552851 >> 172.21.15.33:0/2987552851 conn(0x7fecc40fd700 msgr2=0x7fecc40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 -- 172.21.15.33:0/2987552851 shutdown_connections 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 -- 172.21.15.33:0/2987552851 wait complete. 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 Processor -- start 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.392+0000 7fecc8a67640 1 -- start start 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc8a67640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc4101c90 0x7fecc4199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc8a67640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc41076a0 0x7fecc41997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc8a67640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc8a67640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4111d30 con 0x7fecc4108020 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc8a67640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4111bb0 con 0x7fecc4101c90 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc8a67640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4111eb0 con 0x7fecc41076a0 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48822/0 (socket says 172.21.15.33:48822) 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 -- 172.21.15.33:0/2669078560 learned_addr learned my addr 172.21.15.33:0/2669078560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc4101c90 0x7fecc4199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc41076a0 0x7fecc41997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 -- 172.21.15.33:0/2669078560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc41076a0 msgr2=0x7fecc41997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc41076a0 0x7fecc41997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 -- 172.21.15.33:0/2669078560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc4101c90 msgr2=0x7fecc4199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc4101c90 0x7fecc4199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 -- 172.21.15.33:0/2669078560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc419f310 con 0x7fecc4108020 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc259c640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc4101c90 0x7fecc4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecc2d9d640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc419eb40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fecbc02a930 tx=0x7fecbc003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.393+0000 7fecb37fe640 1 -- 172.21.15.33:0/2669078560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc03f070 con 0x7fecc4108020 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.394+0000 7fecb37fe640 1 -- 172.21.15.33:0/2669078560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecbc003e20 con 0x7fecc4108020 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.394+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc419f5a0 con 0x7fecc4108020 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.394+0000 7fecb37fe640 1 -- 172.21.15.33:0/2669078560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecbc035030 con 0x7fecc4108020 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.394+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc41a5cc0 con 0x7fecc4108020 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.396+0000 7feccae3c740 1 Processor -- start 2024-09-10T21:14:08.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7fecb37fe640 1 -- 172.21.15.33:0/2669078560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fecbc03b460 con 0x7fecc4108020 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7feccae3c740 1 -- start start 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7feccae3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf2b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7feccae3c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf377f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7feccae3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf3a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7feccae3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607bbea1e50 con 0x5607bbf37d30 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7feccae3c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5607bbe91280 con 0x5607bbf2bda0 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7feccae3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607bbe95090 con 0x5607bbf2b480 2024-09-10T21:14:08.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7fecc259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf377f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7fecc259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf377f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43820/0 (socket says 172.21.15.33:43820) 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7fecc259c640 1 -- 172.21.15.33:0/4004946632 learned_addr learned my addr 172.21.15.33:0/4004946632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf3a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.397+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf2b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecc259c640 1 -- 172.21.15.33:0/4004946632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 msgr2=0x5607bbf2b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecc259c640 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf2b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecc259c640 1 -- 172.21.15.33:0/4004946632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 msgr2=0x5607bbf3a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecc259c640 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf3a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecc259c640 1 -- 172.21.15.33:0/4004946632 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607bbbef650 con 0x5607bbf2bda0 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecc259c640 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf377f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fecb40115f0 tx=0x7fecb4011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=110cc42cf6140f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecb37fe640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fecac0b53e0 0x7fecac0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecb0ff9640 1 -- 172.21.15.33:0/4004946632 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb401e070 con 0x5607bbf2bda0 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fecac0b53e0 0x7fecac0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecb0ff9640 1 -- 172.21.15.33:0/4004946632 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb400fa70 con 0x5607bbf2bda0 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecb37fe640 1 -- 172.21.15.33:0/2669078560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fecbc106560 con 0x7fecc4108020 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.398+0000 7fecb0ff9640 1 -- 172.21.15.33:0/4004946632 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4019b50 con 0x5607bbf2bda0 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- 172.21.15.33:0/4004946632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 msgr2=0x5607bbf377f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf377f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fecb40115f0 tx=0x7fecb4011ab0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fecac0b53e0 0x7fecac0b78a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009fd0 tx=0x7fecb8004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- 172.21.15.33:0/4004946632 shutdown_connections 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf3a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf377f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 --2- 172.21.15.33:0/4004946632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf2b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- 172.21.15.33:0/4004946632 >> 172.21.15.33:0/4004946632 conn(0x5607bbf292b0 msgr2=0x5607bbf2a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- 172.21.15.33:0/4004946632 shutdown_connections 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- 172.21.15.33:0/4004946632 wait complete. 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- start start 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 Processor -- start 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf375a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf349f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf34f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607bbcf2700 con 0x5607bbf37d30 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5607bbe2c110 con 0x5607bbf2bda0 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.400+0000 7feccae3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607bbf3a9b0 con 0x5607bbf2b480 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7feccae3c740 10 client.0 _ll_register_callbacks cb 0x5607bbe45990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf375a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf375a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:33372/0 (socket says 172.21.15.33:33372) 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf34f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 -- 172.21.15.33:0/4106403926 learned_addr learned my addr 172.21.15.33:0/4106403926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 -- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 msgr2=0x5607bbf349f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf349f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 -- 172.21.15.33:0/4106403926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 msgr2=0x5607bbf34f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf34f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 -- 172.21.15.33:0/4106403926 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607bbe59c40 con 0x5607bbf2b480 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7fecc259c640 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf375a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fecb400ffe0 tx=0x7fecb4009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed6d499f8e7030c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.401+0000 7feca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:14:08.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feca9ffb640 1 -- 172.21.15.33:0/4106403926 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4027060 con 0x5607bbf2b480 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feca9ffb640 1 -- 172.21.15.33:0/4106403926 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fecb400f5c0 con 0x5607bbf2b480 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb400fa70 con 0x5607bbf2b480 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feca9ffb640 1 -- 172.21.15.33:0/4106403926 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fecb401e040 con 0x5607bbf2b480 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 10 client.26710 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5607bbf462f0 con 0x5607bbf2b480 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feca9ffb640 1 -- 172.21.15.33:0/4106403926 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb400fa70 con 0x5607bbf2b480 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feca9ffb640 1 -- 172.21.15.33:0/4106403926 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fecb402c070 con 0x5607bbf2b480 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 10 client.26710 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 10 client.26710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 20 client.26710 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbf476f0 0x5607bbf49ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 4 client.26710 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607bbea3da0 con 0x5607bbf476f0 2024-09-10T21:14:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.402+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbf476f0 0x5607bbf49ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.403+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbf476f0 0x5607bbf49ae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcba1b8e4a263e server_cookie=4dbe08cb376fc205 in_seq=0 out_seq=0 2024-09-10T21:14:08.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.403+0000 7feca9ffb640 10 client.26710 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feca9ffb640 1 -- 172.21.15.33:0/4106403926 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607bbea3da0 con 0x5607bbf476f0 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feca9ffb640 10 client.26710 handle_command_reply: tid=0 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 client.26710 shutdown 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 msgr2=0x5607bbf375a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf375a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fecb400ffe0 tx=0x7fecb4009da0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 shutdown_connections 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbf476f0 0x5607bbf49ae0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbf34f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf349f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feca9ffb640 0 client.26710 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 --2- 172.21.15.33:0/4106403926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf375a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 >> 172.21.15.33:0/4106403926 conn(0x5607bbf292b0 msgr2=0x5607bbf2ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 shutdown_connections 2024-09-10T21:14:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 1 -- 172.21.15.33:0/4106403926 wait complete. 2024-09-10T21:14:08.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.404+0000 7feccae3c740 20 client.26710 trim_cache size 0 max 16384 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 Processor -- start 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 -- start start 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf2b480 0x5607bbf2aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2bda0 0x5607bbf43c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf37d30 0x5607bbf31c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41ae3e0 con 0x5607bbf2b480 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41aff20 con 0x5607bbf37d30 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7feccae3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41aeab0 con 0x5607bbf2bda0 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf2b480 0x5607bbf2aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf2b480 0x5607bbf2aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48848/0 (socket says 172.21.15.33:48848) 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc259c640 1 -- 172.21.15.33:0/1172006587 learned_addr learned my addr 172.21.15.33:0/1172006587 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc2d9d640 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf37d30 0x5607bbf31c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2bda0 0x5607bbf43c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc259c640 1 -- 172.21.15.33:0/1172006587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2bda0 msgr2=0x5607bbf43c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc259c640 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2bda0 0x5607bbf43c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc259c640 1 -- 172.21.15.33:0/1172006587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf37d30 msgr2=0x5607bbf31c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.423+0000 7fecc259c640 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf37d30 0x5607bbf31c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.424+0000 7fecc259c640 1 -- 172.21.15.33:0/1172006587 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607bbf347a0 con 0x5607bbf2b480 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.424+0000 7fecc259c640 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf2b480 0x5607bbf2aaf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fecb4000c00 tx=0x7fecb4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a7b39b156386c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.424+0000 7fecabfff640 1 -- 172.21.15.33:0/1172006587 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4032020 con 0x5607bbf2b480 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.424+0000 7fecabfff640 1 -- 172.21.15.33:0/1172006587 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb40055e0 con 0x5607bbf2b480 2024-09-10T21:14:08.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.424+0000 7fecabfff640 1 -- 172.21.15.33:0/1172006587 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4027040 con 0x5607bbf2b480 2024-09-10T21:14:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 -- 172.21.15.33:0/1172006587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf2b480 msgr2=0x5607bbf2aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf2b480 0x5607bbf2aaf0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fecb4000c00 tx=0x7fecb4011ab0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 -- 172.21.15.33:0/1172006587 shutdown_connections 2024-09-10T21:14:08.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf37d30 0x5607bbf31c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2bda0 0x5607bbf43c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 --2- 172.21.15.33:0/1172006587 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf2b480 0x5607bbf2aaf0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 -- 172.21.15.33:0/1172006587 >> 172.21.15.33:0/1172006587 conn(0x5607bbf292b0 msgr2=0x5607bbf2a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 -- 172.21.15.33:0/1172006587 shutdown_connections 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 -- 172.21.15.33:0/1172006587 wait complete. 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 -- start start 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 Processor -- start 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf3b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.425+0000 7feccae3c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf3bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feccae3c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbfac520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feccae3c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41af850 con 0x5607bbf37d30 2024-09-10T21:14:08.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feccae3c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41af180 con 0x5607bbf2bda0 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feccae3c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41ac3a0 con 0x5607bbf2b480 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feccae3c740 10 client.0 _ll_register_callbacks cb 0x7fecb401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbfac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf3bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf3b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf3bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43854/0 (socket says 172.21.15.33:43854) 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbfac520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48850/0 (socket says 172.21.15.33:48850) 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 -- 172.21.15.33:0/1064290731 learned_addr learned my addr 172.21.15.33:0/1064290731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 -- 172.21.15.33:0/1064290731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 msgr2=0x5607bbf3b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf3b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 -- 172.21.15.33:0/1064290731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 msgr2=0x5607bbfac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbfac520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 -- 172.21.15.33:0/1064290731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607bbfacc30 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc2d9d640 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbfac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7fecc1d9b640 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf3bb20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fecb8003bc0 tx=0x7fecb8009680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3942ee704d5d3c0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feca9ffb640 1 -- 172.21.15.33:0/1064290731 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8020c60 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feca9ffb640 1 -- 172.21.15.33:0/1064290731 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fecb8020e00 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.426+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb40055e0 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feca9ffb640 1 -- 172.21.15.33:0/1064290731 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fecb8004b20 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 10 client.26768 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5607bbfb4070 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feca9ffb640 1 -- 172.21.15.33:0/1064290731 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb40055e0 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feca9ffb640 1 -- 172.21.15.33:0/1064290731 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fecb803a050 con 0x5607bbf2bda0 2024-09-10T21:14:08.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 10 client.26768 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 10 client.26768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 20 client.26768 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbfb47d0 0x5607bbfb6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 4 client.26768 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5607bbe9f850 con 0x5607bbfb47d0 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.427+0000 7fecc259c640 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbfb47d0 0x5607bbfb6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.428+0000 7fecc259c640 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbfb47d0 0x5607bbfb6bf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b520cc4ba408774e server_cookie=3ee26b8e0a7dc67f in_seq=0 out_seq=0 2024-09-10T21:14:08.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.428+0000 7feca9ffb640 10 client.26768 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feca9ffb640 1 -- 172.21.15.33:0/1064290731 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4262 (crc 0 0 0) 0x5607bbe9f850 con 0x5607bbfb47d0 2024-09-10T21:14:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feca9ffb640 10 client.26768 handle_command_reply: tid=0 2024-09-10T21:14:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 client.26768 shutdown 2024-09-10T21:14:08.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 msgr2=0x5607bbf3bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf3bb20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fecb8003bc0 tx=0x7fecb8009680 comp rx=0 tx=0).stop 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 shutdown_connections 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607bbfb47d0 0x5607bbfb6bf0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607bbf37d30 0x5607bbfac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607bbf2bda0 0x5607bbf3bb20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 --2- 172.21.15.33:0/1064290731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607bbf2b480 0x5607bbf3b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 >> 172.21.15.33:0/1064290731 conn(0x5607bbf292b0 msgr2=0x5607bbf27bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feca9ffb640 0 client.26768 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 shutdown_connections 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.429+0000 7feccae3c740 1 -- 172.21.15.33:0/1064290731 wait complete. 2024-09-10T21:14:08.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7feccae3c740 20 client.26768 trim_cache size 0 max 16384 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "279c3483-bbc8-45ec-93b3-a95fbd081ae3", 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:14:08.439 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.440 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:14:08.441 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.443 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.444 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.445 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:14:08.446 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:08.448 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:14:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fecac0b53e0 msgr2=0x7fecac0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fecac0b53e0 0x7fecac0b78a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009fd0 tx=0x7fecb8004ab0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 msgr2=0x7fecc419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc419eb40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fecbc02a930 tx=0x7fecbc003c00 comp rx=0 tx=0).stop 2024-09-10T21:14:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 shutdown_connections 2024-09-10T21:14:08.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fecac0b53e0 0x7fecac0b78a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecc4108020 0x7fecc419eb40 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecc41076a0 0x7fecc41997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 --2- 172.21.15.33:0/2669078560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fecc4101c90 0x7fecc4199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 >> 172.21.15.33:0/2669078560 conn(0x7fecc40fd700 msgr2=0x7fecc406acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 shutdown_connections 2024-09-10T21:14:08.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:08.430+0000 7fecc8a67640 1 -- 172.21.15.33:0/2669078560 wait complete. 2024-09-10T21:14:08.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 698648 5.77 MB/sec execute 429 sec latency 30.250 ms 2024-09-10T21:14:09.169 INFO:tasks.cephfs.filesystem:scrub status for tag:279c3483-bbc8-45ec-93b3-a95fbd081ae3 - {'path': '/', 'tag': '279c3483-bbc8-45ec-93b3-a95fbd081ae3', 'options': 'recursive,force'} 2024-09-10T21:14:09.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 700268 5.79 MB/sec execute 430 sec latency 29.009 ms 2024-09-10T21:14:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:09 smithi104 ceph-mon[35325]: pgmap v651: 97 pgs: 97 active+clean; 273 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 378 op/s 2024-09-10T21:14:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:09 smithi174 ceph-mon[35635]: pgmap v651: 97 pgs: 97 active+clean; 273 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 378 op/s 2024-09-10T21:14:09.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:09 smithi033 ceph-mon[30860]: pgmap v651: 97 pgs: 97 active+clean; 273 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 378 op/s 2024-09-10T21:14:10.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 702970 5.81 MB/sec execute 431 sec latency 9.615 ms 2024-09-10T21:14:11.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 704586 5.81 MB/sec execute 432 sec latency 13.843 ms 2024-09-10T21:14:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:11 smithi104 ceph-mon[35325]: pgmap v652: 97 pgs: 97 active+clean; 272 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-09-10T21:14:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:11 smithi174 ceph-mon[35635]: pgmap v652: 97 pgs: 97 active+clean; 272 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-09-10T21:14:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:11 smithi033 ceph-mon[30860]: pgmap v652: 97 pgs: 97 active+clean; 272 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 319 op/s 2024-09-10T21:14:12.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 707001 5.83 MB/sec execute 433 sec latency 11.139 ms 2024-09-10T21:14:13.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 709121 5.84 MB/sec execute 434 sec latency 12.169 ms 2024-09-10T21:14:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:13 smithi104 ceph-mon[35325]: pgmap v653: 97 pgs: 97 active+clean; 274 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 329 op/s 2024-09-10T21:14:13.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:13 smithi174 ceph-mon[35635]: pgmap v653: 97 pgs: 97 active+clean; 274 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 329 op/s 2024-09-10T21:14:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:13 smithi033 ceph-mon[30860]: pgmap v653: 97 pgs: 97 active+clean; 274 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.8 MiB/s wr, 329 op/s 2024-09-10T21:14:14.749 INFO:tasks.workunit.client.0.smithi033.stdout: 1 710662 5.85 MB/sec execute 435 sec latency 62.618 ms 2024-09-10T21:14:15.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 713198 5.87 MB/sec execute 436 sec latency 10.301 ms 2024-09-10T21:14:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:15 smithi104 ceph-mon[35325]: pgmap v654: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 321 op/s 2024-09-10T21:14:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:15 smithi174 ceph-mon[35635]: pgmap v654: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 321 op/s 2024-09-10T21:14:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:15 smithi033 ceph-mon[30860]: pgmap v654: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 321 op/s 2024-09-10T21:14:16.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 714890 5.87 MB/sec execute 437 sec latency 11.731 ms 2024-09-10T21:14:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:16 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:14:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:16 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:14:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:16 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:14:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:16 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:14:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:16 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:14:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:16 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:14:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:16 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:14:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:16 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:14:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:16 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:14:17.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 717275 5.89 MB/sec execute 438 sec latency 88.746 ms 2024-09-10T21:14:17.769 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:17 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:14:17.769 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:17 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:14:17.770 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:17 smithi033 ceph-mon[30860]: pgmap v655: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-09-10T21:14:17.770 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:17 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:14:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:17 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:14:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:17 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:14:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:17 smithi104 ceph-mon[35325]: pgmap v655: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-09-10T21:14:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:17 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:14:17.881 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:17 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:14:17.881 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:17 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:14:17.881 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:17 smithi174 ceph-mon[35635]: pgmap v655: 97 pgs: 97 active+clean; 275 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 297 op/s 2024-09-10T21:14:17.882 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:17 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:14:18.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 718439 5.89 MB/sec execute 439 sec latency 62.591 ms 2024-09-10T21:14:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:19 smithi033 ceph-mon[30860]: pgmap v656: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-09-10T21:14:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:19 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:19 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:19 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:19 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 720864 5.91 MB/sec execute 440 sec latency 37.533 ms 2024-09-10T21:14:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:19 smithi104 ceph-mon[35325]: pgmap v656: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-09-10T21:14:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:19 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:19 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:19 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:19 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:19 smithi174 ceph-mon[35635]: pgmap v656: 97 pgs: 97 active+clean; 278 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 394 op/s 2024-09-10T21:14:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:19 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:20.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 722478 5.91 MB/sec execute 441 sec latency 17.752 ms 2024-09-10T21:14:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:14:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:14:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:14:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:14:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:14:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:14:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:14:21.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 724803 5.93 MB/sec execute 442 sec latency 10.727 ms 2024-09-10T21:14:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:21 smithi104 ceph-mon[35325]: pgmap v657: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-09-10T21:14:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:21 smithi174 ceph-mon[35635]: pgmap v657: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-09-10T21:14:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:21 smithi033 ceph-mon[30860]: pgmap v657: 97 pgs: 97 active+clean; 279 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 303 op/s 2024-09-10T21:14:22.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 727173 5.94 MB/sec execute 443 sec latency 13.651 ms 2024-09-10T21:14:23.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 728860 5.95 MB/sec execute 444 sec latency 11.412 ms 2024-09-10T21:14:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:23 smithi104 ceph-mon[35325]: pgmap v658: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-09-10T21:14:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:23 smithi174 ceph-mon[35635]: pgmap v658: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-09-10T21:14:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:23 smithi033 ceph-mon[30860]: pgmap v658: 97 pgs: 97 active+clean; 280 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 311 op/s 2024-09-10T21:14:24.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 731207 5.97 MB/sec execute 445 sec latency 48.984 ms 2024-09-10T21:14:25.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 732651 5.97 MB/sec execute 446 sec latency 54.830 ms 2024-09-10T21:14:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:25 smithi104 ceph-mon[35325]: pgmap v659: 97 pgs: 97 active+clean; 282 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-09-10T21:14:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:25 smithi174 ceph-mon[35635]: pgmap v659: 97 pgs: 97 active+clean; 282 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-09-10T21:14:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:25 smithi033 ceph-mon[30860]: pgmap v659: 97 pgs: 97 active+clean; 282 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 353 op/s 2024-09-10T21:14:26.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 735276 5.99 MB/sec execute 447 sec latency 11.286 ms 2024-09-10T21:14:27.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 737240 5.99 MB/sec execute 448 sec latency 9.799 ms 2024-09-10T21:14:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:27 smithi104 ceph-mon[35325]: pgmap v660: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-09-10T21:14:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:27 smithi174 ceph-mon[35635]: pgmap v660: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-09-10T21:14:28.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:27 smithi033 ceph-mon[30860]: pgmap v660: 97 pgs: 97 active+clean; 283 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 280 op/s 2024-09-10T21:14:28.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 739009 6.01 MB/sec execute 449 sec latency 13.812 ms 2024-09-10T21:14:29.750 INFO:tasks.workunit.client.0.smithi033.stdout: 1 741077 6.01 MB/sec execute 450 sec latency 49.204 ms 2024-09-10T21:14:29.946 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:29 smithi174 ceph-mon[35635]: pgmap v661: 97 pgs: 97 active+clean; 286 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-10T21:14:30.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:29 smithi104 ceph-mon[35325]: pgmap v661: 97 pgs: 97 active+clean; 286 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-10T21:14:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:29 smithi033 ceph-mon[30860]: pgmap v661: 97 pgs: 97 active+clean; 286 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.5 MiB/s wr, 358 op/s 2024-09-10T21:14:30.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 742666 6.02 MB/sec execute 451 sec latency 33.523 ms 2024-09-10T21:14:31.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 745060 6.04 MB/sec execute 452 sec latency 13.893 ms 2024-09-10T21:14:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:31 smithi104 ceph-mon[35325]: pgmap v662: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-10T21:14:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:31 smithi174 ceph-mon[35635]: pgmap v662: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-10T21:14:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:31 smithi033 ceph-mon[30860]: pgmap v662: 97 pgs: 97 active+clean; 288 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 268 op/s 2024-09-10T21:14:32.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 746563 6.04 MB/sec execute 453 sec latency 14.279 ms 2024-09-10T21:14:33.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 749174 6.07 MB/sec execute 454 sec latency 11.127 ms 2024-09-10T21:14:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:33 smithi104 ceph-mon[35325]: pgmap v663: 97 pgs: 97 active+clean; 289 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-09-10T21:14:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:33 smithi174 ceph-mon[35635]: pgmap v663: 97 pgs: 97 active+clean; 289 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-09-10T21:14:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:33 smithi033 ceph-mon[30860]: pgmap v663: 97 pgs: 97 active+clean; 289 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-09-10T21:14:34.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 750470 6.06 MB/sec execute 455 sec latency 49.966 ms 2024-09-10T21:14:35.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 752980 6.08 MB/sec execute 456 sec latency 24.708 ms 2024-09-10T21:14:36.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:35 smithi104 ceph-mon[35325]: pgmap v664: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-09-10T21:14:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:35 smithi174 ceph-mon[35635]: pgmap v664: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-09-10T21:14:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:35 smithi033 ceph-mon[30860]: pgmap v664: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.7 MiB/s wr, 340 op/s 2024-09-10T21:14:36.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 754860 6.08 MB/sec execute 457 sec latency 11.203 ms 2024-09-10T21:14:37.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 756744 6.10 MB/sec execute 458 sec latency 10.933 ms 2024-09-10T21:14:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:37 smithi104 ceph-mon[35325]: pgmap v665: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 278 op/s 2024-09-10T21:14:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:37 smithi174 ceph-mon[35635]: pgmap v665: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 278 op/s 2024-09-10T21:14:38.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:37 smithi033 ceph-mon[30860]: pgmap v665: 97 pgs: 97 active+clean; 291 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 278 op/s 2024-09-10T21:14:38.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 758926 6.10 MB/sec execute 459 sec latency 11.503 ms 2024-09-10T21:14:39.170 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:14:39.440 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:39.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 760466 6.11 MB/sec execute 460 sec latency 39.862 ms 2024-09-10T21:14:39.840 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:39 smithi033 ceph-mon[30860]: pgmap v666: 97 pgs: 97 active+clean; 293 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 341 op/s 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 -- 172.21.15.33:0/3857397712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc103ca0 msgr2=0x7f31fc104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 --2- 172.21.15.33:0/3857397712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc103ca0 0x7f31fc104080 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009900 tx=0x7f31ec02a410 comp rx=0 tx=0).stop 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 -- 172.21.15.33:0/3857397712 shutdown_connections 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 --2- 172.21.15.33:0/3857397712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31fc10d100 0x7f31fc10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 --2- 172.21.15.33:0/3857397712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31fc1045c0 0x7f31fc10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 --2- 172.21.15.33:0/3857397712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc103ca0 0x7f31fc104080 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 -- 172.21.15.33:0/3857397712 >> 172.21.15.33:0/3857397712 conn(0x7f31fc0fd700 msgr2=0x7f31fc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 -- 172.21.15.33:0/3857397712 shutdown_connections 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.925+0000 7f3202837640 1 -- 172.21.15.33:0/3857397712 wait complete. 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 Processor -- start 2024-09-10T21:14:39.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 -- start start 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31fc103ca0 0x7f31fc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31fc1045c0 0x7f31fc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc10d100 0x7f31fc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc112dc0 con 0x7f31fc10d100 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc112c40 con 0x7f31fc1045c0 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3202837640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc112f40 con 0x7f31fc103ca0 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3200dd4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc10d100 0x7f31fc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3200dd4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc10d100 0x7f31fc19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44628/0 (socket says 172.21.15.33:44628) 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f3200dd4640 1 -- 172.21.15.33:0/3599320377 learned_addr learned my addr 172.21.15.33:0/3599320377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31fc1045c0 0x7f31fc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.926+0000 7f31fbfff640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31fc103ca0 0x7f31fc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3200dd4640 1 -- 172.21.15.33:0/3599320377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31fc103ca0 msgr2=0x7f31fc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3200dd4640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31fc103ca0 0x7f31fc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3200dd4640 1 -- 172.21.15.33:0/3599320377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31fc1045c0 msgr2=0x7f31fc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3200dd4640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31fc1045c0 0x7f31fc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3200dd4640 1 -- 172.21.15.33:0/3599320377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31fc19f340 con 0x7f31fc10d100 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f31fbfff640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31fc103ca0 0x7f31fc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3200dd4640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc10d100 0x7f31fc19eb70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f31f400eb10 tx=0x7f31f400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f31f97fa640 1 -- 172.21.15.33:0/3599320377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4004b20 con 0x7f31fc10d100 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f31f97fa640 1 -- 172.21.15.33:0/3599320377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f4004cc0 con 0x7f31fc10d100 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f31f97fa640 1 -- 172.21.15.33:0/3599320377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f4010670 con 0x7f31fc10d100 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31fc19f630 con 0x7f31fc10d100 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.927+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31fc1a5cf0 con 0x7f31fc10d100 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.929+0000 7f31f97fa640 1 -- 172.21.15.33:0/3599320377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f31f4005020 con 0x7f31fc10d100 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.929+0000 7f3204c0c740 1 Processor -- start 2024-09-10T21:14:39.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f3204c0c740 1 -- start start 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0550a0 0x56371c055480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 0x56371c065820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c065d60 0x56371c068150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f3204c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56371bfcbe50 con 0x56371c065d60 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f3204c0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56371bfbb280 con 0x56371c0559c0 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f3204c0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56371bfbf090 con 0x56371c0550a0 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0550a0 0x56371c055480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 0x56371c065820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0550a0 0x56371c055480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56712/0 (socket says 172.21.15.33:56712) 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 -- 172.21.15.33:0/1601534087 learned_addr learned my addr 172.21.15.33:0/1601534087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c065d60 0x56371c068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 -- 172.21.15.33:0/1601534087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 msgr2=0x56371c065820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 0x56371c065820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 -- 172.21.15.33:0/1601534087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c065d60 msgr2=0x56371c068150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c065d60 0x56371c068150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.930+0000 7f31fb7fe640 1 -- 172.21.15.33:0/1601534087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371bd19650 con 0x56371c0550a0 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.931+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0550a0 0x56371c055480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31f00117e0 tx=0x7f31f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cae3de4f40a41070 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.931+0000 7f31f97fa640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f31e40b5360 0x7f31e40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.931+0000 7f31e27fc640 1 -- 172.21.15.33:0/1601534087 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0004050 con 0x56371c0550a0 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.931+0000 7f31fbfff640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f31e40b5360 0x7f31e40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.931+0000 7f31e27fc640 1 -- 172.21.15.33:0/1601534087 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f00054d0 con 0x56371c0550a0 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.931+0000 7f31e27fc640 1 -- 172.21.15.33:0/1601534087 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f0004bc0 con 0x56371c0550a0 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.931+0000 7f31f97fa640 1 -- 172.21.15.33:0/3599320377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f31f40e61c0 con 0x7f31fc10d100 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f31fbfff640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f31e40b5360 0x7f31e40b7820 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f31ec000c00 tx=0x7f31ec005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:39.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 -- 172.21.15.33:0/1601534087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0550a0 msgr2=0x56371c055480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0550a0 0x56371c055480 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31f00117e0 tx=0x7f31f00046c0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 -- 172.21.15.33:0/1601534087 shutdown_connections 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c065d60 0x56371c068150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 0x56371c065820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1601534087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0550a0 0x56371c055480 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 -- 172.21.15.33:0/1601534087 >> 172.21.15.33:0/1601534087 conn(0x56371c052ed0 msgr2=0x56371c054070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 -- 172.21.15.33:0/1601534087 shutdown_connections 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 -- 172.21.15.33:0/1601534087 wait complete. 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.932+0000 7f3204c0c740 1 -- start start 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 1 Processor -- start 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c0655d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0559c0 0x56371c062a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c065d60 0x56371c062fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56371be1c700 con 0x56371c0550a0 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56371bf56110 con 0x56371c065d60 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56371c068a10 con 0x56371c0559c0 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3204c0c740 10 client.0 _ll_register_callbacks cb 0x56371bf6f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c0655d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c0655d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44650/0 (socket says 172.21.15.33:44650) 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 -- 172.21.15.33:0/1126840777 learned_addr learned my addr 172.21.15.33:0/1126840777 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f3200dd4640 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c065d60 0x56371c062fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0559c0 0x56371c062a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 -- 172.21.15.33:0/1126840777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0559c0 msgr2=0x56371c062a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0559c0 0x56371c062a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 -- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c065d60 msgr2=0x56371c062fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:39.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c065d60 0x56371c062fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.933+0000 7f31fbfff640 1 -- 172.21.15.33:0/1126840777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371bf83c40 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f31fbfff640 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c0655d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f31ec033000 tx=0x7f31ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f0210258514e92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f31db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f31db7fe640 1 -- 172.21.15.33:0/1126840777 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec04d070 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31f00054d0 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f31db7fe640 1 -- 172.21.15.33:0/1126840777 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f31ec004c50 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f31db7fe640 1 -- 172.21.15.33:0/1126840777 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f31ec059050 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f3204c0c740 10 client.17337 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56371c0742f0 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f31db7fe640 1 -- 172.21.15.33:0/1126840777 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31f00054d0 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f31db7fe640 1 -- 172.21.15.33:0/1126840777 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f31ec0324d0 con 0x56371c0550a0 2024-09-10T21:14:39.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f3204c0c740 10 client.17337 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f3204c0c740 10 client.17337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.934+0000 7f3204c0c740 20 client.17337 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.935+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0757c0 0x56371c077bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.935+0000 7f3204c0c740 4 client.17337 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.935+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56371bfcdda0 con 0x56371c0757c0 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.935+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0757c0 0x56371c077bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.935+0000 7f31db7fe640 10 client.17337 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.935+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0757c0 0x56371c077bf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c7b034a6e798de server_cookie=d14a23e96fd2452a in_seq=0 out_seq=0 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f31db7fe640 1 -- 172.21.15.33:0/1126840777 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56371bfcdda0 con 0x56371c0757c0 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f31db7fe640 10 client.17337 handle_command_reply: tid=0 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 client.17337 shutdown 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 msgr2=0x56371c0655d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c0655d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f31ec033000 tx=0x7f31ec009260 comp rx=0 tx=0).stop 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 shutdown_connections 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0757c0 0x56371c077bf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c065d60 0x56371c062fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c0559c0 0x56371c062a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1126840777 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c0655d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 >> 172.21.15.33:0/1126840777 conn(0x56371c052ed0 msgr2=0x56371c058a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f31db7fe640 0 client.17337 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 shutdown_connections 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 1 -- 172.21.15.33:0/1126840777 wait complete. 2024-09-10T21:14:39.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.936+0000 7f3204c0c740 20 client.17337 trim_cache size 0 max 16384 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 Processor -- start 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 -- start start 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0550a0 0x56371c071c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0559c0 0x56371c061f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c053ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc1ac1f0 con 0x56371c0559c0 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc1add30 con 0x56371c0550a0 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3204c0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc1ac8c0 con 0x56371c065d60 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0559c0 0x56371c061f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0559c0 0x56371c061f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44654/0 (socket says 172.21.15.33:44654) 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fb7fe640 1 -- 172.21.15.33:0/71955221 learned_addr learned my addr 172.21.15.33:0/71955221 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fbfff640 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0550a0 0x56371c071c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f3200dd4640 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c053ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fbfff640 1 -- 172.21.15.33:0/71955221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 msgr2=0x56371c053ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fbfff640 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c053ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fbfff640 1 -- 172.21.15.33:0/71955221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0559c0 msgr2=0x56371c061f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fbfff640 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0559c0 0x56371c061f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fbfff640 1 -- 172.21.15.33:0/71955221 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371c0627d0 con 0x56371c0550a0 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.955+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0559c0 0x56371c061f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.956+0000 7f31fbfff640 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0550a0 0x56371c071c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009900 tx=0x7f31ec005e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=423850af75922a1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.956+0000 7f31e1ffb640 1 -- 172.21.15.33:0/71955221 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec04d050 con 0x56371c0550a0 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.956+0000 7f31e1ffb640 1 -- 172.21.15.33:0/71955221 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ec009520 con 0x56371c0550a0 2024-09-10T21:14:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.956+0000 7f31e1ffb640 1 -- 172.21.15.33:0/71955221 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec035040 con 0x56371c0550a0 2024-09-10T21:14:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 -- 172.21.15.33:0/71955221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0550a0 msgr2=0x56371c071c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0550a0 0x56371c071c70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009900 tx=0x7f31ec005e10 comp rx=0 tx=0).stop 2024-09-10T21:14:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 -- 172.21.15.33:0/71955221 shutdown_connections 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c053ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0559c0 0x56371c061f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 --2- 172.21.15.33:0/71955221 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0550a0 0x56371c071c70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 -- 172.21.15.33:0/71955221 >> 172.21.15.33:0/71955221 conn(0x56371c052ed0 msgr2=0x56371c066b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 -- 172.21.15.33:0/71955221 shutdown_connections 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 -- 172.21.15.33:0/71955221 wait complete. 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.957+0000 7f3204c0c740 1 -- start start 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 1 Processor -- start 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c06e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 0x56371c07d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c07dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc1ad660 con 0x56371c0550a0 2024-09-10T21:14:39.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc1acf90 con 0x56371c0559c0 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31fc1aa1b0 con 0x56371c065d60 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3204c0c740 10 client.0 _ll_register_callbacks cb 0x7f31f40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c06e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f31fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c06e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44664/0 (socket says 172.21.15.33:44664) 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f31fbfff640 1 -- 172.21.15.33:0/1827102245 learned_addr learned my addr 172.21.15.33:0/1827102245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f3200dd4640 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c07dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f31fbfff640 1 -- 172.21.15.33:0/1827102245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 msgr2=0x56371c07dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f31fbfff640 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c07dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.958+0000 7f31fbfff640 1 -- 172.21.15.33:0/1827102245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 msgr2=0x56371c07d330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.959+0000 7f31fbfff640 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 0x56371c07d330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.959+0000 7f31fbfff640 1 -- 172.21.15.33:0/1827102245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371c0e16c0 con 0x56371c0550a0 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.959+0000 7f31fbfff640 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c06e700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f31ec032c90 tx=0x7f31ec032cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fff9cbd4d1a12d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:39.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.959+0000 7f31db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.959+0000 7f31db7fe640 1 -- 172.21.15.33:0/1827102245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec035070 con 0x56371c0550a0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.959+0000 7f31db7fe640 1 -- 172.21.15.33:0/1827102245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f31ec004a50 con 0x56371c0550a0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.959+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31ec009520 con 0x56371c0550a0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f31db7fe640 1 -- 172.21.15.33:0/1827102245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f31ec04d040 con 0x56371c0550a0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 10 client.17343 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x56371c0e2970 con 0x56371c0550a0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f31db7fe640 1 -- 172.21.15.33:0/1827102245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31ec009520 con 0x56371c0550a0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f31db7fe640 1 -- 172.21.15.33:0/1827102245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f31ec06e470 con 0x56371c0550a0 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 10 client.17343 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 10 client.17343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 20 client.17343 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0e2fb0 0x56371c0e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:39.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 4 client.17343 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:14:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56371bfc9850 con 0x56371c0e2fb0 2024-09-10T21:14:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.960+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0e2fb0 0x56371c0e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f31db7fe640 10 client.17343 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:39.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f31fb7fe640 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0e2fb0 0x56371c0e53d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eecfcddede6e668f server_cookie=b59e20b6ea75e162 in_seq=0 out_seq=0 2024-09-10T21:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f31db7fe640 1 -- 172.21.15.33:0/1827102245 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x56371bfc9850 con 0x56371c0e2fb0 2024-09-10T21:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f31db7fe640 10 client.17343 handle_command_reply: tid=0 2024-09-10T21:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 client.17343 shutdown 2024-09-10T21:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 msgr2=0x56371c06e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c06e700 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f31ec032c90 tx=0x7f31ec032cc0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 shutdown_connections 2024-09-10T21:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56371c0e2fb0 0x56371c0e53d0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56371c065d60 0x56371c07dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56371c0559c0 0x56371c07d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f31db7fe640 0 client.17343 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 --2- 172.21.15.33:0/1827102245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56371c0550a0 0x56371c06e700 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 >> 172.21.15.33:0/1827102245 conn(0x56371c052ed0 msgr2=0x56371c067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 shutdown_connections 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 1 -- 172.21.15.33:0/1827102245 wait complete. 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.961+0000 7f3204c0c740 20 client.17343 trim_cache size 0 max 16384 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:14:39.971 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:14:39.972 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:14:39.975 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.976 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.977 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:14:39.979 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:39.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:39.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:39.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:39.981 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:39.981 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:14:39.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f31e40b5360 msgr2=0x7f31e40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f31e40b5360 0x7f31e40b7820 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f31ec000c00 tx=0x7f31ec005b50 comp rx=0 tx=0).stop 2024-09-10T21:14:39.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc10d100 msgr2=0x7f31fc19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc10d100 0x7f31fc19eb70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f31f400eb10 tx=0x7f31f400c2d0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 shutdown_connections 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f31e40b5360 0x7f31e40b7820 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f31fc10d100 0x7f31fc19eb70 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f31fc1045c0 0x7f31fc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 --2- 172.21.15.33:0/3599320377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31fc103ca0 0x7f31fc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 >> 172.21.15.33:0/3599320377 conn(0x7f31fc0fd700 msgr2=0x7f31fc0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 shutdown_connections 2024-09-10T21:14:39.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:39.962+0000 7f3202837640 1 -- 172.21.15.33:0/3599320377 wait complete. 2024-09-10T21:14:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:39 smithi104 ceph-mon[35325]: pgmap v666: 97 pgs: 97 active+clean; 293 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 341 op/s 2024-09-10T21:14:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:39 smithi174 ceph-mon[35635]: pgmap v666: 97 pgs: 97 active+clean; 293 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.9 MiB/s wr, 341 op/s 2024-09-10T21:14:40.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 762785 6.13 MB/sec execute 461 sec latency 40.083 ms 2024-09-10T21:14:40.778 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:14:40.778 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:14:41.054 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 -- 172.21.15.33:0/4202502407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d41087d0 msgr2=0x7fe6d4108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/4202502407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d41087d0 0x7fe6d4108c10 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009900 tx=0x7fe6c402a410 comp rx=0 tx=0).stop 2024-09-10T21:14:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 -- 172.21.15.33:0/4202502407 shutdown_connections 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/4202502407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6d4109150 0x7fe6d410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/4202502407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d41087d0 0x7fe6d4108c10 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/4202502407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d4102dc0 0x7fe6d41031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 -- 172.21.15.33:0/4202502407 >> 172.21.15.33:0/4202502407 conn(0x7fe6d40fe830 msgr2=0x7fe6d4100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 -- 172.21.15.33:0/4202502407 shutdown_connections 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.533+0000 7fe6db4e2640 1 -- 172.21.15.33:0/4202502407 wait complete. 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 Processor -- start 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 -- start start 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d4102dc0 0x7fe6d4076f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6d41087d0 0x7fe6d40754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d4109150 0x7fe6d4075ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4112e10 con 0x7fe6d4109150 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4112c90 con 0x7fe6d41087d0 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6db4e2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d4112f90 con 0x7fe6d4102dc0 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d4109150 0x7fe6d4075ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d927e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d4102dc0 0x7fe6d4076f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d4109150 0x7fe6d4075ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44680/0 (socket says 172.21.15.33:44680) 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 -- 172.21.15.33:0/2623784968 learned_addr learned my addr 172.21.15.33:0/2623784968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d8a7d640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6d41087d0 0x7fe6d40754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 -- 172.21.15.33:0/2623784968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d4102dc0 msgr2=0x7fe6d4076f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d4102dc0 0x7fe6d4076f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 -- 172.21.15.33:0/2623784968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6d41087d0 msgr2=0x7fe6d40754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6d41087d0 0x7fe6d40754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d9a7f640 1 -- 172.21.15.33:0/2623784968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d41abc40 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.534+0000 7fe6d927e640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d4102dc0 0x7fe6d4076f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.535+0000 7fe6d9a7f640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d4109150 0x7fe6d4075ad0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009fd0 tx=0x7fe6cc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.535+0000 7fe6ca7fc640 1 -- 172.21.15.33:0/2623784968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc017070 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.535+0000 7fe6ca7fc640 1 -- 172.21.15.33:0/2623784968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6cc003e20 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.535+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d41abe10 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.535+0000 7fe6ca7fc640 1 -- 172.21.15.33:0/2623784968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc010460 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.535+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d41ac270 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.536+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d41031a0 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.537+0000 7fe6ca7fc640 1 -- 172.21.15.33:0/2623784968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe6cc003970 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.538+0000 7fe6ca7fc640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe6c00b52e0 0x7fe6c00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.538+0000 7fe6ca7fc640 1 -- 172.21.15.33:0/2623784968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe6cc0e5880 con 0x7fe6d4109150 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.538+0000 7fe6d927e640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe6c00b52e0 0x7fe6c00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.539+0000 7fe6d927e640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe6c00b52e0 0x7fe6c00b77a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fc0 tx=0x7fe6d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.541+0000 7fe6ca7fc640 1 -- 172.21.15.33:0/2623784968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe6cc0ac0f0 con 0x7fe6d4109150 2024-09-10T21:14:41.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 764131 6.13 MB/sec execute 462 sec latency 36.775 ms 2024-09-10T21:14:41.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.781+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d402e5c0 con 0x7fe6d4109150 2024-09-10T21:14:41.787 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:41 smithi033 ceph-mon[30860]: pgmap v667: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-10T21:14:41.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.782+0000 7fe6ca7fc640 1 -- 172.21.15.33:0/2623784968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11973 (secure 0 0 0) 0x7fe6cc0b0b20 con 0x7fe6d4109150 2024-09-10T21:14:41.787 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:14:41.787 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"btime":"2024-09-10T21:11:31:688744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:11:30.989029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:14:41.788 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe6c00b52e0 msgr2=0x7fe6c00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe6c00b52e0 0x7fe6c00b77a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0009fc0 tx=0x7fe6d0005f90 comp rx=0 tx=0).stop 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d4109150 msgr2=0x7fe6d4075ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d4109150 0x7fe6d4075ad0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009fd0 tx=0x7fe6cc00b260 comp rx=0 tx=0).stop 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 shutdown_connections 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe6c00b52e0 0x7fe6c00b77a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6d4109150 0x7fe6d4075ad0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe6d41087d0 0x7fe6d40754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 --2- 172.21.15.33:0/2623784968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6d4102dc0 0x7fe6d4076f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 >> 172.21.15.33:0/2623784968 conn(0x7fe6d40fe830 msgr2=0x7fe6d4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.784+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 shutdown_connections 2024-09-10T21:14:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:41.785+0000 7fe6db4e2640 1 -- 172.21.15.33:0/2623784968 wait complete. 2024-09-10T21:14:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:41 smithi104 ceph-mon[35325]: pgmap v667: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-10T21:14:42.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:41 smithi174 ceph-mon[35635]: pgmap v667: 97 pgs: 97 active+clean; 294 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 279 op/s 2024-09-10T21:14:42.454 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:14:42.735 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:42.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 766703 6.15 MB/sec execute 463 sec latency 37.166 ms 2024-09-10T21:14:43.003 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:42 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2623784968' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:14:43.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:42 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2623784968' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:14:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:42 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2623784968' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:14:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2673843884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 msgr2=0x7f8c34107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2673843884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34107ae0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009ca0 tx=0x7f8c2802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2673843884 shutdown_connections 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2673843884 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c34108020 0x7f8c3410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2673843884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34107ae0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2673843884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c34101c90 0x7f8c34102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2673843884 >> 172.21.15.33:0/2673843884 conn(0x7f8c340fd700 msgr2=0x7f8c340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2673843884 shutdown_connections 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.219+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2673843884 wait complete. 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 Processor -- start 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 -- start start 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c34101c90 0x7f8c34198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c34108020 0x7f8c3419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34111ca0 con 0x7f8c341076a0 2024-09-10T21:14:43.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34111b20 con 0x7f8c34108020 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c39a6f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c34111e20 con 0x7f8c34101c90 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44700/0 (socket says 172.21.15.33:44700) 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c32ffd640 1 -- 172.21.15.33:0/2533605075 learned_addr learned my addr 172.21.15.33:0/2533605075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c337fe640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c34101c90 0x7f8c34198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.220+0000 7f8c33fff640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c34108020 0x7f8c3419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c32ffd640 1 -- 172.21.15.33:0/2533605075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c34101c90 msgr2=0x7f8c34198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c34101c90 0x7f8c34198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c32ffd640 1 -- 172.21.15.33:0/2533605075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c34108020 msgr2=0x7f8c3419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c34108020 0x7f8c3419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c32ffd640 1 -- 172.21.15.33:0/2533605075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c3419e2b0 con 0x7f8c341076a0 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c337fe640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c34101c90 0x7f8c34198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34198740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8c28033040 tx=0x7f8c280316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:43.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c30ff9640 1 -- 172.21.15.33:0/2533605075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28003b70 con 0x7f8c341076a0 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c3419e540 con 0x7f8c341076a0 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c30ff9640 1 -- 172.21.15.33:0/2533605075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28003d10 con 0x7f8c341076a0 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c30ff9640 1 -- 172.21.15.33:0/2533605075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2802f9e0 con 0x7f8c341076a0 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.221+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c341a4c60 con 0x7f8c341076a0 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 Processor -- start 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 -- start start 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06db08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dbea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06dbef90 0x563e06dc1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c30ff9640 1 -- 172.21.15.33:0/2533605075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8c2802fba0 con 0x7f8c341076a0 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e06d26ea0 con 0x563e06db04d0 2024-09-10T21:14:43.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e06d162d0 con 0x563e06db0df0 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c3be44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e06d1a0e0 con 0x563e06dbef90 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06db08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dbea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06db08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44716/0 (socket says 172.21.15.33:44716) 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dbea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52444/0 (socket says 172.21.15.33:52444) 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c33fff640 1 -- 172.21.15.33:0/2170433216 learned_addr learned my addr 172.21.15.33:0/2170433216 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06dbef90 0x563e06dc1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c337fe640 1 -- 172.21.15.33:0/2170433216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06dbef90 msgr2=0x563e06dc1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c337fe640 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06dbef90 0x563e06dc1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c337fe640 1 -- 172.21.15.33:0/2170433216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 msgr2=0x563e06db08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.224+0000 7f8c337fe640 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06db08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c337fe640 1 -- 172.21.15.33:0/2170433216 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e06a74650 con 0x563e06db0df0 2024-09-10T21:14:43.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c337fe640 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dbea50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009fd0 tx=0x7f8c240118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d9c426a652a0f16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c19ffb640 1 -- 172.21.15.33:0/2170433216 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24004310 con 0x563e06db0df0 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c19ffb640 1 -- 172.21.15.33:0/2170433216 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c240044b0 con 0x563e06db0df0 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c19ffb640 1 -- 172.21.15.33:0/2170433216 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c240035f0 con 0x563e06db0df0 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c30ff9640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c1c0b52e0 0x7f8c1c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c33fff640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c1c0b52e0 0x7f8c1c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.225+0000 7f8c30ff9640 1 -- 172.21.15.33:0/2533605075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8c28106da0 con 0x7f8c341076a0 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 -- 172.21.15.33:0/2170433216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 msgr2=0x563e06dbea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c33fff640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c1c0b52e0 0x7f8c1c0b77a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009ee0 tx=0x7f8c2c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dbea50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8c24009fd0 tx=0x7f8c240118d0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 -- 172.21.15.33:0/2170433216 shutdown_connections 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06dbef90 0x563e06dc1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dbea50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 --2- 172.21.15.33:0/2170433216 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06db08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 -- 172.21.15.33:0/2170433216 >> 172.21.15.33:0/2170433216 conn(0x563e06dae300 msgr2=0x563e06daf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 -- 172.21.15.33:0/2170433216 shutdown_connections 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 -- 172.21.15.33:0/2170433216 wait complete. 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.226+0000 7f8c3be44740 1 -- start start 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 1 Processor -- start 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06dbe800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db0df0 0x563e06dbbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dbc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563e06b77700 con 0x563e06db04d0 2024-09-10T21:14:43.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563e06cb1160 con 0x563e06dbef90 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563e06dc1c40 con 0x563e06db0df0 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c3be44740 10 client.0 _ll_register_callbacks cb 0x563e06cca9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06dbe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06dbe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44730/0 (socket says 172.21.15.33:44730) 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 -- 172.21.15.33:0/3469239105 learned_addr learned my addr 172.21.15.33:0/3469239105 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db0df0 0x563e06dbbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c33fff640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dbc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 -- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db0df0 msgr2=0x563e06dbbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db0df0 0x563e06dbbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 -- 172.21.15.33:0/3469239105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 msgr2=0x563e06dbc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dbc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c337fe640 1 -- 172.21.15.33:0/3469239105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e06cdec90 con 0x563e06db04d0 2024-09-10T21:14:43.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.227+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db0df0 0x563e06dbbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c337fe640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06dbe800 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f8c24004ab0 tx=0x7f8c24004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683b55d37e13bccc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3469239105 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c24029070 con 0x563e06db04d0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c240044b0 con 0x563e06db04d0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3469239105 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8c24003a00 con 0x563e06db04d0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3469239105 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8c24035050 con 0x563e06db04d0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 10 client.17361 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563e06dcd520 con 0x563e06db04d0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3469239105 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c240044b0 con 0x563e06db04d0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3469239105 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8c2400f4e0 con 0x563e06db04d0 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 10 client.17361 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 10 client.17361 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 20 client.17361 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:43.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06dce8d0 0x563e06dd0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 4 client.17361 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.228+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e06d28df0 con 0x563e06dce8d0 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.229+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06dce8d0 0x563e06dd0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.229+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06dce8d0 0x563e06dd0cc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95ed6ae1d454f0a9 server_cookie=b1a25d55eeb4c042 in_seq=0 out_seq=0 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.229+0000 7f8c12ffd640 10 client.17361 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3469239105 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563e06d28df0 con 0x563e06dce8d0 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c12ffd640 10 client.17361 handle_command_reply: tid=0 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 client.17361 shutdown 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 msgr2=0x563e06dbe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06dbe800 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f8c24004ab0 tx=0x7f8c24004340 comp rx=0 tx=0).stop 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 shutdown_connections 2024-09-10T21:14:43.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06dce8d0 0x563e06dd0cc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dbc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db0df0 0x563e06dbbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3469239105 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db04d0 0x563e06dbe800 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 >> 172.21.15.33:0/3469239105 conn(0x563e06dae300 msgr2=0x563e06db3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c12ffd640 0 client.17361 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 shutdown_connections 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 1 -- 172.21.15.33:0/3469239105 wait complete. 2024-09-10T21:14:43.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.230+0000 7f8c3be44740 20 client.17361 trim_cache size 0 max 16384 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.244+0000 7f8c3be44740 1 Processor -- start 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.244+0000 7f8c3be44740 1 -- start start 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.244+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06dcaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.244+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db0df0 0x563e06dbb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dd64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c3be44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341ac250 con 0x563e06db0df0 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c3be44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341add90 con 0x563e06dbef90 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c3be44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341ac920 con 0x563e06db04d0 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db0df0 0x563e06dbb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c33fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dd64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06dcaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06dcaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56794/0 (socket says 172.21.15.33:56794) 2024-09-10T21:14:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db0df0 0x563e06dbb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44736/0 (socket says 172.21.15.33:44736) 2024-09-10T21:14:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c33fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dd64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52462/0 (socket says 172.21.15.33:52462) 2024-09-10T21:14:43.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c337fe640 1 -- 172.21.15.33:0/3924702441 learned_addr learned my addr 172.21.15.33:0/3924702441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:43.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 -- 172.21.15.33:0/3924702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 msgr2=0x563e06dcaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06dcaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 -- 172.21.15.33:0/3924702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 msgr2=0x563e06dd64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dd64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 -- 172.21.15.33:0/3924702441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e06dbeb70 con 0x563e06db0df0 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c33fff640 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dd64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c337fe640 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06dcaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db0df0 0x563e06dbb160 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x563e06db0480 tx=0x7f8c28109230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8656b75b352430a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.245+0000 7f8c197fa640 1 -- 172.21.15.33:0/3924702441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280c9400 con 0x563e06db0df0 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c197fa640 1 -- 172.21.15.33:0/3924702441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28036040 con 0x563e06db0df0 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c197fa640 1 -- 172.21.15.33:0/3924702441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280cccd0 con 0x563e06db0df0 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 -- 172.21.15.33:0/3924702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db0df0 msgr2=0x563e06dbb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db0df0 0x563e06dbb160 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x563e06db0480 tx=0x7f8c28109230 comp rx=0 tx=0).stop 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 -- 172.21.15.33:0/3924702441 shutdown_connections 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06dbef90 0x563e06dd64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06db0df0 0x563e06dbb160 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3924702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06dcaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 -- 172.21.15.33:0/3924702441 >> 172.21.15.33:0/3924702441 conn(0x563e06dae300 msgr2=0x563e06dc0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 -- 172.21.15.33:0/3924702441 shutdown_connections 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.246+0000 7f8c3be44740 1 -- 172.21.15.33:0/3924702441 wait complete. 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 -- start start 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 Processor -- start 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06e36e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dc6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06dbef90 0x563e06dc7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341ad6c0 con 0x563e06dbef90 2024-09-10T21:14:43.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341acff0 con 0x563e06db0df0 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c341aa230 con 0x563e06db04d0 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.247+0000 7f8c3be44740 10 client.0 _ll_register_callbacks cb 0x7f8c2c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06e36e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dc6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c33fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06dbef90 0x563e06dc7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dc6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52472/0 (socket says 172.21.15.33:52472) 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c337fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06e36e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56808/0 (socket says 172.21.15.33:56808) 2024-09-10T21:14:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 -- 172.21.15.33:0/3020641509 learned_addr learned my addr 172.21.15.33:0/3020641509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 -- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 msgr2=0x563e06e36e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06e36e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 -- 172.21.15.33:0/3020641509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06dbef90 msgr2=0x563e06dc7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06dbef90 0x563e06dc7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 -- 172.21.15.33:0/3020641509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e06dc7ba0 con 0x563e06db0df0 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c337fe640 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06e36e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c33fff640 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06dbef90 0x563e06dc7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c32ffd640 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dc6ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c280cd4f0 tx=0x7f8c28009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a0ba0501a65f58d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:43.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3020641509 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c280cceb0 con 0x563e06db0df0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3020641509 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8c280cd7c0 con 0x563e06db0df0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.248+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c28036040 con 0x563e06db0df0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3020641509 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8c280cd330 con 0x563e06db0df0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 10 client.26807 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563e06e3daf0 con 0x563e06db0df0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3020641509 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c28036040 con 0x563e06db0df0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3020641509 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f8c280d6660 con 0x563e06db0df0 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 10 client.26807 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 10 client.26807 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 20 client.26807 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06e3e1b0 0x563e06e405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:43.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 4 client.26807 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563e06d248a0 con 0x563e06e3e1b0 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.249+0000 7f8c337fe640 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06e3e1b0 0x563e06e405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c12ffd640 10 client.26807 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c337fe640 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06e3e1b0 0x563e06e405a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b155dd3e330b7103 server_cookie=2ae8454e429ba1db in_seq=0 out_seq=0 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c12ffd640 1 -- 172.21.15.33:0/3020641509 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563e06d248a0 con 0x563e06e3e1b0 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c12ffd640 10 client.26807 handle_command_reply: tid=0 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 client.26807 shutdown 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 msgr2=0x563e06dc6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dc6ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8c280cd4f0 tx=0x7f8c28009950 comp rx=0 tx=0).stop 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 shutdown_connections 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563e06e3e1b0 0x563e06e405a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563e06dbef90 0x563e06dc7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563e06db0df0 0x563e06dc6ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 --2- 172.21.15.33:0/3020641509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563e06db04d0 0x563e06e36e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 >> 172.21.15.33:0/3020641509 conn(0x563e06dae300 msgr2=0x563e06dbd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.250+0000 7f8c12ffd640 0 client.26807 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:14:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 shutdown_connections 2024-09-10T21:14:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c3be44740 1 -- 172.21.15.33:0/3020641509 wait complete. 2024-09-10T21:14:43.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c3be44740 20 client.26807 trim_cache size 0 max 16384 2024-09-10T21:14:43.261 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:43.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c1c0b52e0 msgr2=0x7f8c1c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c1c0b52e0 0x7f8c1c0b77a0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c009ee0 tx=0x7f8c2c005f90 comp rx=0 tx=0).stop 2024-09-10T21:14:43.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 msgr2=0x7f8c34198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:43.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34198740 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f8c28033040 tx=0x7f8c280316f0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 shutdown_connections 2024-09-10T21:14:43.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c1c0b52e0 0x7f8c1c0b77a0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c34108020 0x7f8c3419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c341076a0 0x7f8c34198740 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 --2- 172.21.15.33:0/2533605075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c34101c90 0x7f8c34198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.251+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 >> 172.21.15.33:0/2533605075 conn(0x7f8c340fd700 msgr2=0x7f8c340ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.252+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 shutdown_connections 2024-09-10T21:14:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:43.252+0000 7f8c39a6f640 1 -- 172.21.15.33:0/2533605075 wait complete. 2024-09-10T21:14:43.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 767959 6.15 MB/sec execute 464 sec latency 48.486 ms 2024-09-10T21:14:43.917 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:43 smithi033 ceph-mon[30860]: pgmap v668: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-09-10T21:14:43.949 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:14:44.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:43 smithi104 ceph-mon[35325]: pgmap v668: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-09-10T21:14:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:43 smithi174 ceph-mon[35635]: pgmap v668: 97 pgs: 97 active+clean; 297 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 337 op/s 2024-09-10T21:14:44.229 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:44.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 770513 6.17 MB/sec execute 465 sec latency 28.355 ms 2024-09-10T21:14:44.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 -- 172.21.15.33:0/1803569431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd4103b30 msgr2=0x7f6fd4103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/1803569431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd4103b30 0x7f6fd4103f10 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009900 tx=0x7f6fc402a410 comp rx=0 tx=0).stop 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 -- 172.21.15.33:0/1803569431 shutdown_connections 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/1803569431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fd410d0e0 0x7f6fd410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/1803569431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4104530 0x7f6fd410cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/1803569431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd4103b30 0x7f6fd4103f10 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 -- 172.21.15.33:0/1803569431 >> 172.21.15.33:0/1803569431 conn(0x7f6fd40fd700 msgr2=0x7f6fd40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.723+0000 7f6fda0a9640 1 -- 172.21.15.33:0/1803569431 shutdown_connections 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.724+0000 7f6fda0a9640 1 -- 172.21.15.33:0/1803569431 wait complete. 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.724+0000 7f6fda0a9640 1 Processor -- start 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.724+0000 7f6fda0a9640 1 -- start start 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fda0a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4103b30 0x7f6fd41019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fda0a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fd4104530 0x7f6fd4101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fda0a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd410d0e0 0x7f6fd4102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fda0a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4112d60 con 0x7f6fd410d0e0 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fda0a9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4112be0 con 0x7f6fd4103b30 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fda0a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd4112ee0 con 0x7f6fd4104530 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd410d0e0 0x7f6fd4102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4103b30 0x7f6fd41019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fd4104530 0x7f6fd4101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4103b30 0x7f6fd41019b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52480/0 (socket says 172.21.15.33:52480) 2024-09-10T21:14:44.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 -- 172.21.15.33:0/804164643 learned_addr learned my addr 172.21.15.33:0/804164643 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd410d0e0 0x7f6fd4102430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44764/0 (socket says 172.21.15.33:44764) 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 -- 172.21.15.33:0/804164643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fd4104530 msgr2=0x7f6fd4101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fd4104530 0x7f6fd4101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 -- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd410d0e0 msgr2=0x7f6fd4102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd410d0e0 0x7f6fd4102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 -- 172.21.15.33:0/804164643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd41a76b0 con 0x7f6fd4103b30 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd410d0e0 0x7f6fd4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd37fe640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4103b30 0x7f6fd41019b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005940 tx=0x7f6fc4004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.725+0000 7f6fd0ff9640 1 -- 172.21.15.33:0/804164643 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc403f030 con 0x7f6fd4103b30 2024-09-10T21:14:44.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.726+0000 7f6fd0ff9640 1 -- 172.21.15.33:0/804164643 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fc4003aa0 con 0x7f6fd4103b30 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.726+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd41a7940 con 0x7f6fd4103b30 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.726+0000 7f6fd0ff9640 1 -- 172.21.15.33:0/804164643 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4035070 con 0x7f6fd4103b30 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.726+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd41a7e70 con 0x7f6fd4103b30 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 Processor -- start 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 -- start start 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fd0ff9640 1 -- 172.21.15.33:0/804164643 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6fc4003c50 con 0x7f6fd4103b30 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b74d0 0x55aae56b78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56c3d80 0x55aae56c6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aae562dea0 con 0x55aae56b74d0 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55aae561d2d0 con 0x55aae56c3d80 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aae56210e0 con 0x55aae56b7df0 2024-09-10T21:14:44.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b74d0 0x55aae56b78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b74d0 0x55aae56b78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44772/0 (socket says 172.21.15.33:44772) 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fd3fff640 1 -- 172.21.15.33:0/3095355808 learned_addr learned my addr 172.21.15.33:0/3095355808 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fd37fe640 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.728+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56c3d80 0x55aae56c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd3fff640 1 -- 172.21.15.33:0/3095355808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 msgr2=0x55aae56c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd3fff640 1 -- 172.21.15.33:0/3095355808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56c3d80 msgr2=0x55aae56c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56c3d80 0x55aae56c6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd3fff640 1 -- 172.21.15.33:0/3095355808 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae537b650 con 0x55aae56b74d0 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b74d0 0x55aae56b78b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc011760 tx=0x7f6fcc011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65992deac0c7f5be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fb9ffb640 1 -- 172.21.15.33:0/3095355808 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc004d20 con 0x55aae56b74d0 2024-09-10T21:14:44.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fb9ffb640 1 -- 172.21.15.33:0/3095355808 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fcc005020 con 0x55aae56b74d0 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fb9ffb640 1 -- 172.21.15.33:0/3095355808 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc019670 con 0x55aae56b74d0 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd0ff9640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6fbc0b5360 0x7f6fbc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.729+0000 7f6fd37fe640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6fbc0b5360 0x7f6fbc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fd0ff9640 1 -- 172.21.15.33:0/804164643 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6fc41064a0 con 0x7f6fd4103b30 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3095355808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b74d0 msgr2=0x55aae56b78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b74d0 0x55aae56b78b0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc011760 tx=0x7f6fcc011c20 comp rx=0 tx=0).stop 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fd37fe640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6fbc0b5360 0x7f6fbc0b7820 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6fc403f000 tx=0x7f6fc4109ee0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:44.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3095355808 shutdown_connections 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56c3d80 0x55aae56c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3095355808 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b74d0 0x55aae56b78b0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3095355808 >> 172.21.15.33:0/3095355808 conn(0x55aae56b5300 msgr2=0x55aae56b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3095355808 shutdown_connections 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3095355808 wait complete. 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.730+0000 7f6fdc47e740 1 -- start start 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 1 Processor -- start 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b74d0 0x55aae56c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b7df0 0x55aae56c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aae547e700 con 0x55aae56c3d80 2024-09-10T21:14:44.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55aae55b8160 con 0x55aae56b7df0 2024-09-10T21:14:44.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aae56c6a00 con 0x55aae56b74d0 2024-09-10T21:14:44.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fdc47e740 10 client.0 _ll_register_callbacks cb 0x55aae55d19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:44.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b7df0 0x55aae56c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b7df0 0x55aae56c0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52502/0 (socket says 172.21.15.33:52502) 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/3269376904 learned_addr learned my addr 172.21.15.33:0/3269376904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b74d0 0x55aae56c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b74d0 msgr2=0x55aae56c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b74d0 0x55aae56c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/3269376904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 msgr2=0x55aae56c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/3269376904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae55e5c90 con 0x55aae56b7df0 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.731+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:44.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b7df0 0x55aae56c0aa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800bfc0 tx=0x7f6fc80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ecc9e88bc466042 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/3269376904 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8004020 con 0x55aae56b7df0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/3269376904 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6fc80041c0 con 0x55aae56b7df0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fcc005020 con 0x55aae56b7df0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/3269376904 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6fc80043c0 con 0x55aae56b7df0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 10 client.26819 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55aae56d2370 con 0x55aae56b7df0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/3269376904 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fcc005020 con 0x55aae56b7df0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/3269376904 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f6fc8003a30 con 0x55aae56b7df0 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 10 client.26819 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 10 client.26819 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:14:44.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 20 client.26819 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae56d3920 0x55aae56d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 4 client.26819 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.732+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aae562fdf0 con 0x55aae56d3920 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.733+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae56d3920 0x55aae56d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.733+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae56d3920 0x55aae56d5d40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f180792c0d285493 server_cookie=5f83300f3d853b8d in_seq=0 out_seq=0 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.733+0000 7f6fb2ffd640 10 client.26819 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/3269376904 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aae562fdf0 con 0x55aae56d3920 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fb2ffd640 10 client.26819 handle_command_reply: tid=0 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 client.26819 shutdown 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b7df0 msgr2=0x55aae56c0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b7df0 0x55aae56c0aa0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800bfc0 tx=0x7f6fc80118d0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 shutdown_connections 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b7df0 0x55aae56c0aa0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae56d3920 0x55aae56d5d40 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/3269376904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b74d0 0x55aae56c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fb2ffd640 0 client.26819 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 >> 172.21.15.33:0/3269376904 conn(0x55aae56b5300 msgr2=0x55aae56bae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 shutdown_connections 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 1 -- 172.21.15.33:0/3269376904 wait complete. 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.734+0000 7f6fdc47e740 20 client.26819 trim_cache size 0 max 16384 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 Processor -- start 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 -- start start 2024-09-10T21:14:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b7df0 0x55aae56b6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56c3d80 0x55aae56b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41b0610 con 0x55aae56b7df0 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41b2150 con 0x55aae56b74d0 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fdc47e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41b0ce0 con 0x55aae56c3d80 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b7df0 0x55aae56b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b7df0 0x55aae56b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44784/0 (socket says 172.21.15.33:44784) 2024-09-10T21:14:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/1314821938 learned_addr learned my addr 172.21.15.33:0/1314821938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56c3d80 0x55aae56b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/1314821938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56c3d80 msgr2=0x55aae56b6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56c3d80 0x55aae56b6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/1314821938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 msgr2=0x55aae56cfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.748+0000 7f6fd2ffd640 1 -- 172.21.15.33:0/1314821938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aae56bb7f0 con 0x55aae56b7df0 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.749+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b7df0 0x55aae56b6430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000c00 tx=0x7f6fc8004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5536ae379d769b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.749+0000 7f6fb97fa640 1 -- 172.21.15.33:0/1314821938 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8029040 con 0x55aae56b7df0 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.749+0000 7f6fb97fa640 1 -- 172.21.15.33:0/1314821938 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fc801ad60 con 0x55aae56b7df0 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.749+0000 7f6fb97fa640 1 -- 172.21.15.33:0/1314821938 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc8003850 con 0x55aae56b7df0 2024-09-10T21:14:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 -- 172.21.15.33:0/1314821938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b7df0 msgr2=0x55aae56b6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b7df0 0x55aae56b6430 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8000c00 tx=0x7f6fc8004250 comp rx=0 tx=0).stop 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 -- 172.21.15.33:0/1314821938 shutdown_connections 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56c3d80 0x55aae56b6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56b7df0 0x55aae56b6430 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/1314821938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 -- 172.21.15.33:0/1314821938 >> 172.21.15.33:0/1314821938 conn(0x55aae56b5300 msgr2=0x55aae56bbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 -- 172.21.15.33:0/1314821938 shutdown_connections 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 -- 172.21.15.33:0/1314821938 wait complete. 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 -- start start 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.750+0000 7f6fdc47e740 1 Processor -- start 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56db0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fdc47e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56db610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52520/0 (socket says 172.21.15.33:52520) 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56db0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd3fff640 1 -- 172.21.15.33:0/625218618 learned_addr learned my addr 172.21.15.33:0/625218618 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41b1a80 con 0x55aae56c3d80 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41b13b0 con 0x55aae56b74d0 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd41ae5d0 con 0x55aae56b7df0 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56db610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fdc47e740 10 client.0 _ll_register_callbacks cb 0x7f6fc800bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd3fff640 1 -- 172.21.15.33:0/625218618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 msgr2=0x55aae56db0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56db0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.751+0000 7f6fd3fff640 1 -- 172.21.15.33:0/625218618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 msgr2=0x55aae56cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fd3fff640 1 -- 172.21.15.33:0/625218618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aae56dbd00 con 0x55aae56c3d80 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fd3fff640 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56db610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005970 tx=0x7f6fcc009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e779fa65d4681169 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fb2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/625218618 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc031040 con 0x55aae56c3d80 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/625218618 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6fcc004ce0 con 0x55aae56c3d80 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fc801ad60 con 0x55aae56c3d80 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/625218618 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f6fcc00fe80 con 0x55aae56c3d80 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fdc47e740 10 client.17385 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55aae573a7e0 con 0x55aae56c3d80 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.752+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/625218618 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fc801ad60 con 0x55aae56c3d80 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/625218618 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f6fcc037040 con 0x55aae56c3d80 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fdc47e740 10 client.17385 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fdc47e740 10 client.17385 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fdc47e740 20 client.17385 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae573ac60 0x55aae573d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fdc47e740 4 client.17385 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aae562b8a0 con 0x55aae573ac60 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae573ac60 0x55aae573d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fb2ffd640 10 client.17385 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:14:44.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.753+0000 7f6fd2ffd640 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae573ac60 0x55aae573d080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1df7a0bce0d58e06 server_cookie=d437360d18a1b0ed in_seq=0 out_seq=0 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fb2ffd640 1 -- 172.21.15.33:0/625218618 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aae562b8a0 con 0x55aae573ac60 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fb2ffd640 10 client.17385 handle_command_reply: tid=0 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 client.17385 shutdown 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 msgr2=0x55aae56db610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56db610 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc005970 tx=0x7f6fcc009500 comp rx=0 tx=0).stop 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 shutdown_connections 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aae56c3d80 0x55aae56db610 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55aae573ac60 0x55aae573d080 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fb2ffd640 0 client.17385 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aae56b7df0 0x55aae56db0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 --2- 172.21.15.33:0/625218618 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aae56b74d0 0x55aae56cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 >> 172.21.15.33:0/625218618 conn(0x55aae56b5300 msgr2=0x55aae56b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:44.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 shutdown_connections 2024-09-10T21:14:44.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 1 -- 172.21.15.33:0/625218618 wait complete. 2024-09-10T21:14:44.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.754+0000 7f6fdc47e740 20 client.17385 trim_cache size 0 max 16384 2024-09-10T21:14:44.782 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:44.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.755+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6fbc0b5360 msgr2=0x7f6fbc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.755+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6fbc0b5360 0x7f6fbc0b7820 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6fc403f000 tx=0x7f6fc4109ee0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.755+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4103b30 msgr2=0x7f6fd41019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.755+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4103b30 0x7f6fd41019b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005940 tx=0x7f6fc4004270 comp rx=0 tx=0).stop 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.756+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 shutdown_connections 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.756+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6fbc0b5360 0x7f6fbc0b7820 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.756+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fd410d0e0 0x7f6fd4102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.756+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6fd4104530 0x7f6fd4101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.756+0000 7f6fda0a9640 1 --2- 172.21.15.33:0/804164643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6fd4103b30 0x7f6fd41019b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.756+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 >> 172.21.15.33:0/804164643 conn(0x7f6fd40fd700 msgr2=0x7f6fd40fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.761+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 shutdown_connections 2024-09-10T21:14:44.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:44.762+0000 7f6fda0a9640 1 -- 172.21.15.33:0/804164643 wait complete. 2024-09-10T21:14:45.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 771641 6.16 MB/sec execute 466 sec latency 37.401 ms 2024-09-10T21:14:45.811 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:14:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:45 smithi104 ceph-mon[35325]: pgmap v669: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-09-10T21:14:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:45 smithi174 ceph-mon[35635]: pgmap v669: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-09-10T21:14:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:45 smithi033 ceph-mon[30860]: pgmap v669: 97 pgs: 97 active+clean; 298 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.2 MiB/s wr, 347 op/s 2024-09-10T21:14:46.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.578+0000 7f684353a640 1 -- 172.21.15.33:0/3147554376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c108020 msgr2=0x7f683c10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.578+0000 7f684353a640 1 --2- 172.21.15.33:0/3147554376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c108020 0x7f683c10e820 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6834009ca0 tx=0x7f683402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.578+0000 7f684353a640 1 -- 172.21.15.33:0/3147554376 shutdown_connections 2024-09-10T21:14:46.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.578+0000 7f684353a640 1 --2- 172.21.15.33:0/3147554376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c108020 0x7f683c10e820 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.578+0000 7f684353a640 1 --2- 172.21.15.33:0/3147554376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c1076a0 0x7f683c107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.578+0000 7f684353a640 1 --2- 172.21.15.33:0/3147554376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c101c90 0x7f683c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.578+0000 7f684353a640 1 -- 172.21.15.33:0/3147554376 >> 172.21.15.33:0/3147554376 conn(0x7f683c0fd700 msgr2=0x7f683c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 -- 172.21.15.33:0/3147554376 shutdown_connections 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 -- 172.21.15.33:0/3147554376 wait complete. 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 Processor -- start 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 -- start start 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c101c90 0x7f683c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c1076a0 0x7f683c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c108020 0x7f683c1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f683c111ce0 con 0x7f683c101c90 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f683c111b60 con 0x7f683c108020 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.579+0000 7f684353a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f683c111e60 con 0x7f683c1076a0 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c101c90 0x7f683c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c101c90 0x7f683c19b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44800/0 (socket says 172.21.15.33:44800) 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 -- 172.21.15.33:0/294314338 learned_addr learned my addr 172.21.15.33:0/294314338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f6841ad7640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c108020 0x7f683c1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f6840ad5640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c1076a0 0x7f683c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 -- 172.21.15.33:0/294314338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c1076a0 msgr2=0x7f683c19b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c1076a0 0x7f683c19b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 -- 172.21.15.33:0/294314338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c108020 msgr2=0x7f683c1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c108020 0x7f683c1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 -- 172.21.15.33:0/294314338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c1a14b0 con 0x7f683c101c90 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68412d6640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c101c90 0x7f683c19b400 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f683800e9f0 tx=0x7f683800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68327fc640 1 -- 172.21.15.33:0/294314338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683800cd80 con 0x7f683c101c90 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68327fc640 1 -- 172.21.15.33:0/294314338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68380108e0 con 0x7f683c101c90 2024-09-10T21:14:46.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c1a17a0 con 0x7f683c101c90 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f68327fc640 1 -- 172.21.15.33:0/294314338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6838018980 con 0x7f683c101c90 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.580+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c1a8030 con 0x7f683c101c90 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f68327fc640 1 -- 172.21.15.33:0/294314338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6838010430 con 0x7f683c101c90 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 Processor -- start 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 -- start start 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfb8f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfb9b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acdfb05ea0 con 0x55acdfb8f4d0 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55acdfaf52d0 con 0x55acdfb9bd80 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.583+0000 7f684590f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55acdfaf90e0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfb8f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6840ad5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfb8f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44812/0 (socket says 172.21.15.33:44812) 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 -- 172.21.15.33:0/2482580164 learned_addr learned my addr 172.21.15.33:0/2482580164 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f68412d6640 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfb9b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 -- 172.21.15.33:0/2482580164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 msgr2=0x55acdfb9b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfb9b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 -- 172.21.15.33:0/2482580164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 msgr2=0x55acdfb9e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 -- 172.21.15.33:0/2482580164 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdf853650 con 0x55acdfb8f4d0 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6840ad5640 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f68327fc640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f68280b5360 0x7f68280b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f68412d6640 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfb9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f68412d6640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f68280b5360 0x7f68280b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f68327fc640 1 -- 172.21.15.33:0/294314338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f68380e5ea0 con 0x7f683c101c90 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f6841ad7640 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfb8f8b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6834009410 tx=0x7f6834005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb0698066fcf75a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.584+0000 7f68277fe640 1 -- 172.21.15.33:0/2482580164 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834031cf0 con 0x55acdfb8f4d0 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f68277fe640 1 -- 172.21.15.33:0/2482580164 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6834036030 con 0x55acdfb8f4d0 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f68277fe640 1 -- 172.21.15.33:0/2482580164 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6834004040 con 0x55acdfb8f4d0 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f68412d6640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f68280b5360 0x7f68280b7820 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f68380a8ce0 tx=0x7f6838018420 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 -- 172.21.15.33:0/2482580164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 msgr2=0x55acdfb8f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfb8f8b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f6834009410 tx=0x7f6834005ef0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 -- 172.21.15.33:0/2482580164 shutdown_connections 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfb9b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 --2- 172.21.15.33:0/2482580164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfb8f8b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 -- 172.21.15.33:0/2482580164 >> 172.21.15.33:0/2482580164 conn(0x55acdfb8d300 msgr2=0x55acdfb8e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 -- 172.21.15.33:0/2482580164 shutdown_connections 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.585+0000 7f684590f740 1 -- 172.21.15.33:0/2482580164 wait complete. 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 -- start start 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 Processor -- start 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8f4d0 0x55acdfb9b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8fdf0 0x55acdfb98ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55acdf956700 con 0x55acdfb8fdf0 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55acdfa90160 con 0x55acdfb8f4d0 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.586+0000 7f684590f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55acdfb9e820 con 0x55acdfb9bd80 2024-09-10T21:14:46.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.587+0000 7f684590f740 10 client.0 _ll_register_callbacks cb 0x55acdfaa99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.587+0000 7f6840ad5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8fdf0 0x55acdfb98ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.587+0000 7f6840ad5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8fdf0 0x55acdfb98ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44816/0 (socket says 172.21.15.33:44816) 2024-09-10T21:14:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.587+0000 7f6840ad5640 1 -- 172.21.15.33:0/3515476775 learned_addr learned my addr 172.21.15.33:0/3515476775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:46.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.587+0000 7f6841ad7640 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8f4d0 0x55acdfb9b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6841ad7640 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6840ad5640 1 -- 172.21.15.33:0/3515476775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 msgr2=0x55acdfb990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6840ad5640 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6840ad5640 1 -- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8f4d0 msgr2=0x55acdfb9b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6840ad5640 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8f4d0 0x55acdfb9b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6840ad5640 1 -- 172.21.15.33:0/3515476775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdfabdc90 con 0x55acdfb8fdf0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6840ad5640 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8fdf0 0x55acdfb98ac0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f682c012740 tx=0x7f682c012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1acc80ac6533e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6824ff9640 1 -- 172.21.15.33:0/3515476775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c004030 con 0x55acdfb8fdf0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f6824ff9640 1 -- 172.21.15.33:0/3515476775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f682c0041d0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.588+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6834036030 con 0x55acdfb8fdf0 2024-09-10T21:14:46.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f6824ff9640 1 -- 172.21.15.33:0/3515476775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f682c0043d0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 10 client.17397 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55acdfbaa1e0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f6824ff9640 1 -- 172.21.15.33:0/3515476775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c012e50 con 0x55acdfb8fdf0 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f6824ff9640 1 -- 172.21.15.33:0/3515476775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f682c003b40 con 0x55acdfb8fdf0 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 10 client.17397 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 10 client.17397 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 20 client.17397 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfbab6a0 0x55acdfbadab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 4 client.17397 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.589+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6834036030 con 0x55acdfbab6a0 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.590+0000 7f6841ad7640 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfbab6a0 0x55acdfbadab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.590+0000 7f6824ff9640 10 client.17397 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:46.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.590+0000 7f6841ad7640 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfbab6a0 0x55acdfbadab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3cf2f8de73968da2 server_cookie=15dbe1e9d62a63b7 in_seq=0 out_seq=0 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f6824ff9640 1 -- 172.21.15.33:0/3515476775 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f6834036030 con 0x55acdfbab6a0 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f6824ff9640 10 client.17397 handle_command_reply: tid=0 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 client.17397 shutdown 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8fdf0 msgr2=0x55acdfb98ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8fdf0 0x55acdfb98ac0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f682c012740 tx=0x7f682c012c00 comp rx=0 tx=0).stop 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 shutdown_connections 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfbab6a0 0x55acdfbadab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8fdf0 0x55acdfb98ac0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 --2- 172.21.15.33:0/3515476775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8f4d0 0x55acdfb9b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 >> 172.21.15.33:0/3515476775 conn(0x55acdfb8d300 msgr2=0x55acdfb92e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f6824ff9640 0 client.17397 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:46.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 shutdown_connections 2024-09-10T21:14:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 1 -- 172.21.15.33:0/3515476775 wait complete. 2024-09-10T21:14:46.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.591+0000 7f684590f740 20 client.17397 trim_cache size 0 max 16384 2024-09-10T21:14:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 Processor -- start 2024-09-10T21:14:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 -- start start 2024-09-10T21:14:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8fdf0 0x55acdfb97fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb97340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b0590 con 0x55acdfb8f4d0 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b20d0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f684590f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b0c60 con 0x55acdfb9bd80 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f6841ad7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f6841ad7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44832/0 (socket says 172.21.15.33:44832) 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f6841ad7640 1 -- 172.21.15.33:0/2048257194 learned_addr learned my addr 172.21.15.33:0/2048257194 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.605+0000 7f6840ad5640 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8fdf0 0x55acdfb97fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6841ad7640 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb97340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6840ad5640 1 -- 172.21.15.33:0/2048257194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 msgr2=0x55acdfb97340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6840ad5640 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb97340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6840ad5640 1 -- 172.21.15.33:0/2048257194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 msgr2=0x55acdfba7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6840ad5640 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6840ad5640 1 -- 172.21.15.33:0/2048257194 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acdfb96fc0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6840ad5640 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8fdf0 0x55acdfb97fb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f682c003f60 tx=0x7f682c005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4a816ce1c45be3a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6826ffd640 1 -- 172.21.15.33:0/2048257194 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c000e60 con 0x55acdfb8fdf0 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6826ffd640 1 -- 172.21.15.33:0/2048257194 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c00fdd0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.606+0000 7f6826ffd640 1 -- 172.21.15.33:0/2048257194 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c013e60 con 0x55acdfb8fdf0 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 -- 172.21.15.33:0/2048257194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8fdf0 msgr2=0x55acdfb97fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8fdf0 0x55acdfb97fb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f682c003f60 tx=0x7f682c005790 comp rx=0 tx=0).stop 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 -- 172.21.15.33:0/2048257194 shutdown_connections 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb9bd80 0x55acdfb97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb8fdf0 0x55acdfb97fb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 --2- 172.21.15.33:0/2048257194 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 -- 172.21.15.33:0/2048257194 >> 172.21.15.33:0/2048257194 conn(0x55acdfb8d300 msgr2=0x55acdfb8e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:46.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 -- 172.21.15.33:0/2048257194 shutdown_connections 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.607+0000 7f684590f740 1 -- 172.21.15.33:0/2048257194 wait complete. 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 -- start start 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 Processor -- start 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfba4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b1a00 con 0x55acdfb8f4d0 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1b1330 con 0x55acdfb9bd80 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f683c1ae550 con 0x55acdfb8fdf0 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f684590f740 10 client.0 _ll_register_callbacks cb 0x7f682c009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.608+0000 7f6841ad7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfba4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfba4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56902/0 (socket says 172.21.15.33:56902) 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 -- 172.21.15.33:0/1452954898 learned_addr learned my addr 172.21.15.33:0/1452954898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6841ad7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44844/0 (socket says 172.21.15.33:44844) 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6841ad7640 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 -- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 msgr2=0x55acdfb9eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 -- 172.21.15.33:0/1452954898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 msgr2=0x55acdfba3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 -- 172.21.15.33:0/1452954898 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acdfb9f230 con 0x55acdfb8fdf0 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6841ad7640 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6840ad5640 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfba4230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f682c005fd0 tx=0x7f682c00fbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23c7f3ee299ff4cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6824ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:14:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6824ff9640 1 -- 172.21.15.33:0/1452954898 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f682c035020 con 0x55acdfb8fdf0 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f6824ff9640 1 -- 172.21.15.33:0/1452954898 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f682c005260 con 0x55acdfb8fdf0 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.609+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f682c00fdd0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f6824ff9640 1 -- 172.21.15.33:0/1452954898 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f682c000e40 con 0x55acdfb8fdf0 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 10 client.26794 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55acdfc19040 con 0x55acdfb8fdf0 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f6824ff9640 1 -- 172.21.15.33:0/1452954898 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f682c00fdd0 con 0x55acdfb8fdf0 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f6824ff9640 1 -- 172.21.15.33:0/1452954898 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f682c010630 con 0x55acdfb8fdf0 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 10 client.26794 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 10 client.26794 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 20 client.26794 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfc19700 0x55acdfc1bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 4 client.26794 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55acdfb07df0 con 0x55acdfc19700 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.610+0000 7f6841ad7640 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfc19700 0x55acdfc1bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f6841ad7640 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfc19700 0x55acdfc1bb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c320c6a02d295af server_cookie=c251d5118bb2e35f in_seq=0 out_seq=0 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f6824ff9640 10 client.26794 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f6824ff9640 1 -- 172.21.15.33:0/1452954898 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55acdfb07df0 con 0x55acdfc19700 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f6824ff9640 10 client.26794 handle_command_reply: tid=0 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 client.26794 shutdown 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 msgr2=0x55acdfba4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfba4230 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f682c005fd0 tx=0x7f682c00fbd0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 shutdown_connections 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55acdfb9bd80 0x55acdfb9eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55acdfb8fdf0 0x55acdfba4230 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55acdfc19700 0x55acdfc1bb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 --2- 172.21.15.33:0/1452954898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55acdfb8f4d0 0x55acdfba3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f6824ff9640 0 client.26794 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:14:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 >> 172.21.15.33:0/1452954898 conn(0x55acdfb8d300 msgr2=0x55acdfb97300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 shutdown_connections 2024-09-10T21:14:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 1 -- 172.21.15.33:0/1452954898 wait complete. 2024-09-10T21:14:46.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.611+0000 7f684590f740 20 client.26794 trim_cache size 0 max 16384 2024-09-10T21:14:46.625 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f68280b5360 msgr2=0x7f68280b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f68280b5360 0x7f68280b7820 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f68380a8ce0 tx=0x7f6838018420 comp rx=0 tx=0).stop 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c101c90 msgr2=0x7f683c19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c101c90 0x7f683c19b400 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f683800e9f0 tx=0x7f683800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 shutdown_connections 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f68280b5360 0x7f68280b7820 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f683c108020 0x7f683c1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f683c1076a0 0x7f683c19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 --2- 172.21.15.33:0/294314338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f683c101c90 0x7f683c19b400 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 >> 172.21.15.33:0/294314338 conn(0x7f683c0fd700 msgr2=0x7f683c06b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 shutdown_connections 2024-09-10T21:14:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:46.612+0000 7f684353a640 1 -- 172.21.15.33:0/294314338 wait complete. 2024-09-10T21:14:46.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 774184 6.18 MB/sec execute 467 sec latency 24.882 ms 2024-09-10T21:14:47.422 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:14:47.702 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:47.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 776140 6.19 MB/sec execute 468 sec latency 12.514 ms 2024-09-10T21:14:47.975 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:47 smithi033 ceph-mon[30860]: pgmap v670: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 285 op/s 2024-09-10T21:14:48.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:47 smithi104 ceph-mon[35325]: pgmap v670: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 285 op/s 2024-09-10T21:14:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:47 smithi174 ceph-mon[35635]: pgmap v670: 97 pgs: 97 active+clean; 299 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 285 op/s 2024-09-10T21:14:48.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 -- 172.21.15.33:0/4059516710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a8103b50 msgr2=0x7f91a8103f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 --2- 172.21.15.33:0/4059516710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a8103b50 0x7f91a8103f30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009900 tx=0x7f91a402a410 comp rx=0 tx=0).stop 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 -- 172.21.15.33:0/4059516710 shutdown_connections 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 --2- 172.21.15.33:0/4059516710 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a810d100 0x7f91a810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 --2- 172.21.15.33:0/4059516710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8104550 0x7f91a810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 --2- 172.21.15.33:0/4059516710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a8103b50 0x7f91a8103f30 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 -- 172.21.15.33:0/4059516710 >> 172.21.15.33:0/4059516710 conn(0x7f91a80fd700 msgr2=0x7f91a80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 -- 172.21.15.33:0/4059516710 shutdown_connections 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 -- 172.21.15.33:0/4059516710 wait complete. 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.199+0000 7f91afd27640 1 Processor -- start 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91afd27640 1 -- start start 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91afd27640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8103b50 0x7f91a8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91afd27640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a8104550 0x7f91a81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91afd27640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a810d100 0x7f91a819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91afd27640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8112d60 con 0x7f91a810d100 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91afd27640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8112be0 con 0x7f91a8104550 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91afd27640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91a8112ee0 con 0x7f91a8103b50 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a810d100 0x7f91a819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91adac3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8103b50 0x7f91a8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a810d100 0x7f91a819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44854/0 (socket says 172.21.15.33:44854) 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 -- 172.21.15.33:0/1071120691 learned_addr learned my addr 172.21.15.33:0/1071120691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a8104550 0x7f91a81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 -- 172.21.15.33:0/1071120691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8103b50 msgr2=0x7f91a8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8103b50 0x7f91a8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 -- 172.21.15.33:0/1071120691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a8104550 msgr2=0x7f91a81997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a8104550 0x7f91a81997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.200+0000 7f91ae2c4640 1 -- 172.21.15.33:0/1071120691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a819f340 con 0x7f91a810d100 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.201+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a810d100 0x7f91a819eb70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f91a000efd0 tx=0x7f91a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.201+0000 7f919effd640 1 -- 172.21.15.33:0/1071120691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0003b70 con 0x7f91a810d100 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.201+0000 7f919effd640 1 -- 172.21.15.33:0/1071120691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a0003d10 con 0x7f91a810d100 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.201+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a819f630 con 0x7f91a810d100 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.201+0000 7f919effd640 1 -- 172.21.15.33:0/1071120691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00108a0 con 0x7f91a810d100 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.201+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a81a5cf0 con 0x7f91a810d100 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.203+0000 7f91b20fc740 1 Processor -- start 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f919effd640 1 -- 172.21.15.33:0/1071120691 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f91a001f020 con 0x7f91a810d100 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91b20fc740 1 -- start start 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b0004d0 0x557d4b0008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b000df0 0x557d4b010c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b011190 0x557d4b013580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91b20fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557d4af76ea0 con 0x557d4b0004d0 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91b20fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557d4af662d0 con 0x557d4b000df0 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91b20fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d4af6a0e0 con 0x557d4b011190 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.204+0000 7f91adac3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b000df0 0x557d4b010c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91ad2c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b0004d0 0x557d4b0008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b000df0 0x557d4b010c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52590/0 (socket says 172.21.15.33:52590) 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 -- 172.21.15.33:0/2666435455 learned_addr learned my addr 172.21.15.33:0/2666435455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b011190 0x557d4b013580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 -- 172.21.15.33:0/2666435455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b011190 msgr2=0x557d4b013580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b011190 0x557d4b013580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 -- 172.21.15.33:0/2666435455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b0004d0 msgr2=0x557d4b0008b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b0004d0 0x557d4b0008b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 -- 172.21.15.33:0/2666435455 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4acc4650 con 0x557d4b000df0 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f919effd640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f91940b5360 0x7f91940b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91adac3640 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b000df0 0x557d4b010c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009900 tx=0x7f91a4032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6b8351cd4709e1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f91940b5360 0x7f91940b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f919effd640 1 -- 172.21.15.33:0/1071120691 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f91a00e5c30 con 0x7f91a810d100 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f9193fff640 1 -- 172.21.15.33:0/2666435455 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4040070 con 0x557d4b000df0 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f9193fff640 1 -- 172.21.15.33:0/2666435455 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a4004030 con 0x557d4b000df0 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.205+0000 7f9193fff640 1 -- 172.21.15.33:0/2666435455 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4035040 con 0x557d4b000df0 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 -- 172.21.15.33:0/2666435455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b000df0 msgr2=0x557d4b010c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b000df0 0x557d4b010c50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009900 tx=0x7f91a4032b20 comp rx=0 tx=0).stop 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f91940b5360 0x7f91940b7820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x557d4b000480 tx=0x7f9198015040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 -- 172.21.15.33:0/2666435455 shutdown_connections 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b011190 0x557d4b013580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b000df0 0x557d4b010c50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2666435455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b0004d0 0x557d4b0008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 -- 172.21.15.33:0/2666435455 >> 172.21.15.33:0/2666435455 conn(0x557d4affe300 msgr2=0x557d4afff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:48.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 -- 172.21.15.33:0/2666435455 shutdown_connections 2024-09-10T21:14:48.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.206+0000 7f91b20fc740 1 -- 172.21.15.33:0/2666435455 wait complete. 2024-09-10T21:14:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 -- start start 2024-09-10T21:14:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 Processor -- start 2024-09-10T21:14:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b010aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b00def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b00e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557d4adc7700 con 0x557d4b000df0 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557d4af01160 con 0x557d4b011190 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d4b013de0 con 0x557d4b0004d0 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91b20fc740 10 client.0 _ll_register_callbacks cb 0x557d4af1a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.207+0000 7f91ad2c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b00def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91adac3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b010aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91adac3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b010aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56928/0 (socket says 172.21.15.33:56928) 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ad2c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b00def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44870/0 (socket says 172.21.15.33:44870) 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91adac3640 1 -- 172.21.15.33:0/1189497827 learned_addr learned my addr 172.21.15.33:0/1189497827 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ae2c4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b00e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ad2c2640 1 -- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 msgr2=0x557d4b010aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b010aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ad2c2640 1 -- 172.21.15.33:0/1189497827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 msgr2=0x557d4b00e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b00e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ad2c2640 1 -- 172.21.15.33:0/1189497827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4af2ec90 con 0x557d4b000df0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b00e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91adac3640 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b010aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b00def0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9198011fd0 tx=0x7f9198005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bb067e1063bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91917fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91917fa640 1 -- 172.21.15.33:0/1189497827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919802b670 con 0x557d4b000df0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91917fa640 1 -- 172.21.15.33:0/1189497827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f919802b810 con 0x557d4b000df0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.208+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a4004030 con 0x557d4b000df0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91917fa640 1 -- 172.21.15.33:0/1189497827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f919802ba10 con 0x557d4b000df0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 10 client.17415 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557d4b01f7a0 con 0x557d4b000df0 2024-09-10T21:14:48.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91917fa640 1 -- 172.21.15.33:0/1189497827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a4004030 con 0x557d4b000df0 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91917fa640 1 -- 172.21.15.33:0/1189497827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f919801f070 con 0x557d4b000df0 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 10 client.17415 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 10 client.17415 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 20 client.17415 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b020c60 0x557d4b023090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 4 client.17415 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d4af78df0 con 0x557d4b020c60 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.209+0000 7f91adac3640 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b020c60 0x557d4b023090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.210+0000 7f91adac3640 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b020c60 0x557d4b023090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=82f841cf657244b8 server_cookie=b2c7559c2e7057aa in_seq=0 out_seq=0 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.210+0000 7f91917fa640 10 client.17415 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91917fa640 1 -- 172.21.15.33:0/1189497827 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557d4af78df0 con 0x557d4b020c60 2024-09-10T21:14:48.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91917fa640 10 client.17415 handle_command_reply: tid=0 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 client.17415 shutdown 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 msgr2=0x557d4b00def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b00def0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f9198011fd0 tx=0x7f9198005f90 comp rx=0 tx=0).stop 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 shutdown_connections 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b00e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b020c60 0x557d4b023090 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b00def0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 --2- 172.21.15.33:0/1189497827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b010aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91917fa640 0 client.17415 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 >> 172.21.15.33:0/1189497827 conn(0x557d4affe300 msgr2=0x557d4b003e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 shutdown_connections 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 1 -- 172.21.15.33:0/1189497827 wait complete. 2024-09-10T21:14:48.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.211+0000 7f91b20fc740 20 client.17415 trim_cache size 0 max 16384 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 Processor -- start 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 -- start start 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b01d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b012060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b0125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81ac1a0 con 0x557d4b000df0 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81adce0 con 0x557d4b011190 2024-09-10T21:14:48.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.226+0000 7f91b20fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81ac870 con 0x557d4b0004d0 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ad2c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b012060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ae2c4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b0125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91adac3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b01d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91adac3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b01d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56930/0 (socket says 172.21.15.33:56930) 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ad2c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b012060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44874/0 (socket says 172.21.15.33:44874) 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91adac3640 1 -- 172.21.15.33:0/2811949995 learned_addr learned my addr 172.21.15.33:0/2811949995 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ae2c4640 1 -- 172.21.15.33:0/2811949995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 msgr2=0x557d4b01d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b01d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ae2c4640 1 -- 172.21.15.33:0/2811949995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 msgr2=0x557d4b012060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b012060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ae2c4640 1 -- 172.21.15.33:0/2811949995 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d4b00e890 con 0x557d4b011190 2024-09-10T21:14:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ad2c2640 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b012060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:14:48.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b0125a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f91a00a9660 tx=0x7f91a00abe00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db6c72ad6c0b6037 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:48.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91937fe640 1 -- 172.21.15.33:0/2811949995 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00acaf0 con 0x557d4b011190 2024-09-10T21:14:48.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91937fe640 1 -- 172.21.15.33:0/2811949995 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a00ac3f0 con 0x557d4b011190 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.227+0000 7f91937fe640 1 -- 172.21.15.33:0/2811949995 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00ab8b0 con 0x557d4b011190 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.228+0000 7f91b20fc740 1 -- 172.21.15.33:0/2811949995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 msgr2=0x557d4b0125a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.228+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b0125a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f91a00a9660 tx=0x7f91a00abe00 comp rx=0 tx=0).stop 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 -- 172.21.15.33:0/2811949995 shutdown_connections 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b0125a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b012060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 --2- 172.21.15.33:0/2811949995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b01d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 -- 172.21.15.33:0/2811949995 >> 172.21.15.33:0/2811949995 conn(0x557d4affe300 msgr2=0x557d4aff9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 -- 172.21.15.33:0/2811949995 shutdown_connections 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 -- 172.21.15.33:0/2811949995 wait complete. 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 -- start start 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.229+0000 7f91b20fc740 1 Processor -- start 2024-09-10T21:14:48.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b019810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b028560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91b20fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b028aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91b20fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81ad610 con 0x557d4b000df0 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91b20fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81acf40 con 0x557d4b011190 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91b20fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91a81aa160 con 0x557d4b0004d0 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91b20fc740 10 client.0 _ll_register_callbacks cb 0x7f9198005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ae2c4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b028aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91adac3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b019810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ad2c2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b028560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91adac3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b019810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56934/0 (socket says 172.21.15.33:56934) 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91adac3640 1 -- 172.21.15.33:0/4082437737 learned_addr learned my addr 172.21.15.33:0/4082437737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ae2c4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b028aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52632/0 (socket says 172.21.15.33:52632) 2024-09-10T21:14:48.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ae2c4640 1 -- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 msgr2=0x557d4b019810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b019810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ae2c4640 1 -- 172.21.15.33:0/4082437737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 msgr2=0x557d4b028560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b028560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91ae2c4640 1 -- 172.21.15.33:0/4082437737 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d4b0292d0 con 0x557d4b011190 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.230+0000 7f91adac3640 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b019810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91ae2c4640 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b028aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91a00a9660 tx=0x7f91a00d3460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca015aad15d7581 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91917fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91917fa640 1 -- 172.21.15.33:0/4082437737 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a00c5020 con 0x557d4b011190 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91917fa640 1 -- 172.21.15.33:0/4082437737 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f91a0004d40 con 0x557d4b011190 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a00ac3f0 con 0x557d4b011190 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91917fa640 1 -- 172.21.15.33:0/4082437737 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f91a00acaf0 con 0x557d4b011190 2024-09-10T21:14:48.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 10 client.26861 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557d4b08db30 con 0x557d4b011190 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91917fa640 1 -- 172.21.15.33:0/4082437737 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a00ac3f0 con 0x557d4b011190 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91917fa640 1 -- 172.21.15.33:0/4082437737 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f91a00a7dd0 con 0x557d4b011190 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 10 client.26861 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 10 client.26861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 20 client.26861 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b08e1d0 0x557d4b0905f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 4 client.26861 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.231+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d4af748a0 con 0x557d4b08e1d0 2024-09-10T21:14:48.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91adac3640 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b08e1d0 0x557d4b0905f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91adac3640 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b08e1d0 0x557d4b0905f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a5d5ff74f1976e1e server_cookie=b4f51f3abdd8fedd in_seq=0 out_seq=0 2024-09-10T21:14:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91917fa640 10 client.26861 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:48.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91917fa640 1 -- 172.21.15.33:0/4082437737 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d4af748a0 con 0x557d4b08e1d0 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91917fa640 10 client.26861 handle_command_reply: tid=0 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91b20fc740 1 client.26861 shutdown 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 msgr2=0x557d4b028aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91b20fc740 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b028aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f91a00a9660 tx=0x7f91a00d3460 comp rx=0 tx=0).stop 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 shutdown_connections 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.232+0000 7f91b20fc740 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557d4b011190 0x557d4b028aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91b20fc740 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557d4b08e1d0 0x557d4b0905f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91b20fc740 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557d4b000df0 0x557d4b028560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91b20fc740 1 --2- 172.21.15.33:0/4082437737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d4b0004d0 0x557d4b019810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 >> 172.21.15.33:0/4082437737 conn(0x557d4affe300 msgr2=0x557d4b012060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91917fa640 0 client.26861 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 shutdown_connections 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91b20fc740 1 -- 172.21.15.33:0/4082437737 wait complete. 2024-09-10T21:14:48.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91b20fc740 20 client.26861 trim_cache size 0 max 16384 2024-09-10T21:14:48.243 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f91940b5360 msgr2=0x7f91940b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f91940b5360 0x7f91940b7820 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x557d4b000480 tx=0x7f9198015040 comp rx=0 tx=0).stop 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a810d100 msgr2=0x7f91a819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a810d100 0x7f91a819eb70 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f91a000efd0 tx=0x7f91a000c6a0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 shutdown_connections 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f91940b5360 0x7f91940b7820 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91a810d100 0x7f91a819eb70 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91a8104550 0x7f91a81997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 --2- 172.21.15.33:0/1071120691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91a8103b50 0x7f91a8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.233+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 >> 172.21.15.33:0/1071120691 conn(0x7f91a80fd700 msgr2=0x7f91a80fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.234+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 shutdown_connections 2024-09-10T21:14:48.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:48.234+0000 7f91afd27640 1 -- 172.21.15.33:0/1071120691 wait complete. 2024-09-10T21:14:48.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 778062 6.20 MB/sec execute 469 sec latency 15.551 ms 2024-09-10T21:14:49.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:14:49.403 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:49.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 780505 6.21 MB/sec execute 470 sec latency 59.813 ms 2024-09-10T21:14:49.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:49 smithi033 ceph-mon[30860]: pgmap v671: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-10T21:14:49.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 -- 172.21.15.33:0/2020169259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948105f40 msgr2=0x7f494810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2020169259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948105f40 0x7f494810f960 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4940009ca0 tx=0x7f494002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 -- 172.21.15.33:0/2020169259 shutdown_connections 2024-09-10T21:14:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2020169259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948105f40 0x7f494810f960 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2020169259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 0x7f4948105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2020169259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948101250 0x7f4948101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 -- 172.21.15.33:0/2020169259 >> 172.21.15.33:0/2020169259 conn(0x7f4948078140 msgr2=0x7f49480ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 -- 172.21.15.33:0/2020169259 shutdown_connections 2024-09-10T21:14:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.918+0000 7f494d0e6640 1 -- 172.21.15.33:0/2020169259 wait complete. 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.919+0000 7f494d0e6640 1 Processor -- start 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.919+0000 7f494d0e6640 1 -- start start 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.919+0000 7f494d0e6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948101250 0x7f4948199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.919+0000 7f494d0e6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 0x7f49481997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.919+0000 7f494d0e6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948105f40 0x7f494819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.919+0000 7f494d0e6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4948112d80 con 0x7f4948101250 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.919+0000 7f494d0e6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4948112c00 con 0x7f4948105f40 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494d0e6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4948112f00 con 0x7f49481055a0 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 0x7f49481997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948101250 0x7f4948199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494759e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948105f40 0x7f494819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 0x7f49481997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46590/0 (socket says 172.21.15.33:46590) 2024-09-10T21:14:49.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 -- 172.21.15.33:0/2746939338 learned_addr learned my addr 172.21.15.33:0/2746939338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:49.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494759e640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948105f40 0x7f494819eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39240/0 (socket says 172.21.15.33:39240) 2024-09-10T21:14:49.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948101250 0x7f4948199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33014/0 (socket says 172.21.15.33:33014) 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 -- 172.21.15.33:0/2746939338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948105f40 msgr2=0x7f494819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948105f40 0x7f494819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 -- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948101250 msgr2=0x7f4948199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948101250 0x7f4948199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 -- 172.21.15.33:0/2746939338 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494819f320 con 0x7f49481055a0 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f4946d9d640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948101250 0x7f4948199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494759e640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948105f40 0x7f494819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.920+0000 7f494659c640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 0x7f49481997b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e9f0 tx=0x7f493c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.921+0000 7f492ffff640 1 -- 172.21.15.33:0/2746939338 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c00cd80 con 0x7f49481055a0 2024-09-10T21:14:49.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.921+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494819f610 con 0x7f49481055a0 2024-09-10T21:14:49.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.921+0000 7f492ffff640 1 -- 172.21.15.33:0/2746939338 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493c0108e0 con 0x7f49481055a0 2024-09-10T21:14:49.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.921+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49481a5ea0 con 0x7f49481055a0 2024-09-10T21:14:49.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.921+0000 7f492ffff640 1 -- 172.21.15.33:0/2746939338 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c018980 con 0x7f49481055a0 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 Processor -- start 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 -- start start 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174a78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174b3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174b3d80 0x55fa174b6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fa1741dea0 con 0x55fa174a74d0 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa1740d2d0 con 0x55fa174a7df0 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494f4bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa174110e0 con 0x55fa174b3d80 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f492ffff640 1 -- 172.21.15.33:0/2746939338 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f493c018b20 con 0x7f49481055a0 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174a78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174b3d80 0x55fa174b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174b3d80 0x55fa174b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46596/0 (socket says 172.21.15.33:46596) 2024-09-10T21:14:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174a78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33018/0 (socket says 172.21.15.33:33018) 2024-09-10T21:14:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.924+0000 7f494659c640 1 -- 172.21.15.33:0/3607079837 learned_addr learned my addr 172.21.15.33:0/3607079837 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.925+0000 7f4946d9d640 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.925+0000 7f494659c640 1 -- 172.21.15.33:0/3607079837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 msgr2=0x55fa174b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.925+0000 7f494659c640 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174b3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.925+0000 7f494659c640 1 -- 172.21.15.33:0/3607079837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 msgr2=0x55fa174a78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.925+0000 7f494659c640 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174a78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.925+0000 7f494659c640 1 -- 172.21.15.33:0/3607079837 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa1716b650 con 0x55fa174b3d80 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f4946d9d640 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f494759e640 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f494659c640 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174b3d80 0x55fa174b6170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f493c0e9f90 tx=0x7f493c0ee570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1802c8765d8f8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f492ffff640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49300b5360 0x7f49300b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f492d7fa640 1 -- 172.21.15.33:0/3607079837 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0a8de0 con 0x55fa174b3d80 2024-09-10T21:14:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f494759e640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49300b5360 0x7f49300b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f492d7fa640 1 -- 172.21.15.33:0/3607079837 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493c0a8280 con 0x55fa174b3d80 2024-09-10T21:14:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f492d7fa640 1 -- 172.21.15.33:0/3607079837 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0abf00 con 0x55fa174b3d80 2024-09-10T21:14:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.926+0000 7f492ffff640 1 -- 172.21.15.33:0/2746939338 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f493c0e5f60 con 0x7f49481055a0 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494759e640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49300b5360 0x7f49300b7820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4940005fe0 tx=0x7f4940005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494f4bb740 1 -- 172.21.15.33:0/3607079837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174b3d80 msgr2=0x55fa174b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174b3d80 0x55fa174b6170 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f493c0e9f90 tx=0x7f493c0ee570 comp rx=0 tx=0).stop 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494f4bb740 1 -- 172.21.15.33:0/3607079837 shutdown_connections 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174b3d80 0x55fa174b6170 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3607079837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174a78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.927+0000 7f494f4bb740 1 -- 172.21.15.33:0/3607079837 >> 172.21.15.33:0/3607079837 conn(0x55fa174a5300 msgr2=0x55fa174a64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 -- 172.21.15.33:0/3607079837 shutdown_connections 2024-09-10T21:14:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 -- 172.21.15.33:0/3607079837 wait complete. 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 -- start start 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 Processor -- start 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a7df0 0x55fa174b0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174b3d80 0x55fa174b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fa1726e700 con 0x55fa174a74d0 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fa173a8160 con 0x55fa174b3d80 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa174bae10 con 0x55fa174a7df0 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.928+0000 7f494f4bb740 10 client.0 _ll_register_callbacks cb 0x55fa173c19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f494659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a7df0 0x55fa174b0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f494659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a7df0 0x55fa174b0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46602/0 (socket says 172.21.15.33:46602) 2024-09-10T21:14:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f494659c640 1 -- 172.21.15.33:0/4050070426 learned_addr learned my addr 172.21.15.33:0/4050070426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33020/0 (socket says 172.21.15.33:33020) 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f494759e640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174b3d80 0x55fa174b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 -- 172.21.15.33:0/4050070426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a7df0 msgr2=0x55fa174b0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a7df0 0x55fa174b0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 -- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174b3d80 msgr2=0x55fa174b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174b3d80 0x55fa174b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 -- 172.21.15.33:0/4050070426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa173d5c90 con 0x55fa174a74d0 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f494659c640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a7df0 0x55fa174b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f4946d9d640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174b3600 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x55fa174a61e0 tx=0x7f4938012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310ae7b541b99092 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f492aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f492aff5640 1 -- 172.21.15.33:0/4050070426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800f9a0 con 0x55fa174a74d0 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f493c0a8280 con 0x55fa174a74d0 2024-09-10T21:14:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.929+0000 7f492aff5640 1 -- 172.21.15.33:0/4050070426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f493800fb40 con 0x55fa174a74d0 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f492aff5640 1 -- 172.21.15.33:0/4050070426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f493800fd40 con 0x55fa174a74d0 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 10 client.17430 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fa174c6770 con 0x55fa174a74d0 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f492aff5640 1 -- 172.21.15.33:0/4050070426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493c0a8280 con 0x55fa174a74d0 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f492aff5640 1 -- 172.21.15.33:0/4050070426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f493801c420 con 0x55fa174a74d0 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 10 client.17430 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 10 client.17430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 20 client.17430 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa174c7c20 0x55fa174ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 4 client.17430 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa1741fdf0 con 0x55fa174c7c20 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.930+0000 7f494659c640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa174c7c20 0x55fa174ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.931+0000 7f494659c640 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa174c7c20 0x55fa174ca0c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0655020154e797 server_cookie=ffef985b963cf742 in_seq=0 out_seq=0 2024-09-10T21:14:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.931+0000 7f492aff5640 10 client.17430 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.931+0000 7f492aff5640 1 -- 172.21.15.33:0/4050070426 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa1741fdf0 con 0x55fa174c7c20 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.931+0000 7f492aff5640 10 client.17430 handle_command_reply: tid=0 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.931+0000 7f494f4bb740 1 client.17430 shutdown 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 msgr2=0x55fa174b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174b3600 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x55fa174a61e0 tx=0x7f4938012ca0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 shutdown_connections 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa174c7c20 0x55fa174ca0c0 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174b3d80 0x55fa174b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a7df0 0x55fa174b0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f492aff5640 0 client.17430 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 --2- 172.21.15.33:0/4050070426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174a74d0 0x55fa174b3600 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 >> 172.21.15.33:0/4050070426 conn(0x55fa174a5300 msgr2=0x55fa174aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 shutdown_connections 2024-09-10T21:14:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 1 -- 172.21.15.33:0/4050070426 wait complete. 2024-09-10T21:14:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.932+0000 7f494f4bb740 20 client.17430 trim_cache size 0 max 16384 2024-09-10T21:14:49.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 Processor -- start 2024-09-10T21:14:49.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 -- start start 2024-09-10T21:14:49.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174a4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49481ac2a0 con 0x55fa174b3d80 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49481adde0 con 0x55fa174a7df0 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.945+0000 7f494f4bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49481ac970 con 0x55fa174a74d0 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174a4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174ab180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33034/0 (socket says 172.21.15.33:33034) 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494759e640 1 -- 172.21.15.33:0/414977104 learned_addr learned my addr 172.21.15.33:0/414977104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:49.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174aac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39272/0 (socket says 172.21.15.33:39272) 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 -- 172.21.15.33:0/414977104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 msgr2=0x55fa174a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174a4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 -- 172.21.15.33:0/414977104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 msgr2=0x55fa174ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 -- 172.21.15.33:0/414977104 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa174b1360 con 0x55fa174a7df0 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494759e640 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f494659c640 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174aac40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f493c0a8430 tx=0x7f493c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=983fa0171152735c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:49.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f492cff9640 1 -- 172.21.15.33:0/414977104 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0ac0e0 con 0x55fa174a7df0 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f492cff9640 1 -- 172.21.15.33:0/414977104 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493c0a8280 con 0x55fa174a7df0 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.946+0000 7f492cff9640 1 -- 172.21.15.33:0/414977104 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0ac2d0 con 0x55fa174a7df0 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 -- 172.21.15.33:0/414977104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 msgr2=0x55fa174aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174aac40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f493c0a8430 tx=0x7f493c014000 comp rx=0 tx=0).stop 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 -- 172.21.15.33:0/414977104 shutdown_connections 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174aac40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 --2- 172.21.15.33:0/414977104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174a4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 -- 172.21.15.33:0/414977104 >> 172.21.15.33:0/414977104 conn(0x55fa174a5300 msgr2=0x55fa174aa530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 -- 172.21.15.33:0/414977104 shutdown_connections 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.947+0000 7f494f4bb740 1 -- 172.21.15.33:0/414977104 wait complete. 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.948+0000 7f494f4bb740 1 -- start start 2024-09-10T21:14:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.948+0000 7f494f4bb740 1 Processor -- start 2024-09-10T21:14:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.948+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174bc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.948+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174bc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.948+0000 7f494f4bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494f4bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49481ad710 con 0x55fa174b3d80 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494f4bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49481ad040 con 0x55fa174a7df0 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494f4bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49481aa260 con 0x55fa174a74d0 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494f4bb740 10 client.0 _ll_register_callbacks cb 0x7f4938009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174bc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174bc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46620/0 (socket says 172.21.15.33:46620) 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 -- 172.21.15.33:0/3994901919 learned_addr learned my addr 172.21.15.33:0/3994901919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174bcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33046/0 (socket says 172.21.15.33:33046) 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494659c640 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 -- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 msgr2=0x55fa174bc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174bc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 -- 172.21.15.33:0/3994901919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 msgr2=0x55fa174bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174bcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 -- 172.21.15.33:0/3994901919 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa17533db0 con 0x55fa174a74d0 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f494759e640 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f4946d9d640 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174bc200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49380058c0 tx=0x7f493800fbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f126c05c8a72da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f492aff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.949+0000 7f492aff5640 1 -- 172.21.15.33:0/3994901919 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493800f790 con 0x55fa174a74d0 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f492aff5640 1 -- 172.21.15.33:0/3994901919 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f493801cab0 con 0x55fa174a74d0 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f493c0a8280 con 0x55fa174a74d0 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f492aff5640 1 -- 172.21.15.33:0/3994901919 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4938037050 con 0x55fa174a74d0 2024-09-10T21:14:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 10 client.26830 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55fa175350b0 con 0x55fa174a74d0 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f492aff5640 1 -- 172.21.15.33:0/3994901919 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f493c0a8280 con 0x55fa174a74d0 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f492aff5640 1 -- 172.21.15.33:0/3994901919 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f493801ccb0 con 0x55fa174a74d0 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 10 client.26830 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 10 client.26830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 20 client.26830 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa17535860 0x55fa17537c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 4 client.26830 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa1741b8a0 con 0x55fa17535860 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.950+0000 7f494659c640 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa17535860 0x55fa17537c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.951+0000 7f494659c640 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa17535860 0x55fa17537c80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2800b7ee6e54e2c6 server_cookie=6b8f2114554ec400 in_seq=0 out_seq=0 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.951+0000 7f492aff5640 10 client.26830 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f492aff5640 1 -- 172.21.15.33:0/3994901919 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa1741b8a0 con 0x55fa17535860 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f492aff5640 10 client.26830 handle_command_reply: tid=0 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 client.26830 shutdown 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 msgr2=0x55fa174bc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174bc200 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49380058c0 tx=0x7f493800fbd0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 shutdown_connections 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fa17535860 0x55fa17537c80 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fa174b3d80 0x55fa174bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fa174a7df0 0x55fa174bc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f492aff5640 0 client.26830 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 --2- 172.21.15.33:0/3994901919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa174a74d0 0x55fa174bc200 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 >> 172.21.15.33:0/3994901919 conn(0x55fa174a5300 msgr2=0x55fa174aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:49.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 shutdown_connections 2024-09-10T21:14:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 1 -- 172.21.15.33:0/3994901919 wait complete. 2024-09-10T21:14:49.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.952+0000 7f494f4bb740 20 client.26830 trim_cache size 0 max 16384 2024-09-10T21:14:49.961 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:14:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49300b5360 msgr2=0x7f49300b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49300b5360 0x7f49300b7820 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4940005fe0 tx=0x7f4940005ef0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 msgr2=0x7f49481997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 0x7f49481997b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e9f0 tx=0x7f493c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 shutdown_connections 2024-09-10T21:14:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49300b5360 0x7f49300b7820 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4948105f40 0x7f494819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49481055a0 0x7f49481997b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 --2- 172.21.15.33:0/2746939338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4948101250 0x7f4948199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 >> 172.21.15.33:0/2746939338 conn(0x7f4948078140 msgr2=0x7f4948108f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:49.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 shutdown_connections 2024-09-10T21:14:49.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:49.953+0000 7f494d0e6640 1 -- 172.21.15.33:0/2746939338 wait complete. 2024-09-10T21:14:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:49 smithi104 ceph-mon[35325]: pgmap v671: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-10T21:14:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:49 smithi174 ceph-mon[35635]: pgmap v671: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 350 op/s 2024-09-10T21:14:50.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 781919 6.22 MB/sec execute 471 sec latency 23.988 ms 2024-09-10T21:14:51.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 784665 6.24 MB/sec execute 472 sec latency 10.188 ms 2024-09-10T21:14:52.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:51 smithi104 ceph-mon[35325]: pgmap v672: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2024-09-10T21:14:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:51 smithi174 ceph-mon[35635]: pgmap v672: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2024-09-10T21:14:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:51 smithi033 ceph-mon[30860]: pgmap v672: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 309 op/s 2024-09-10T21:14:52.200 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:14:52.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:14:52.477 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:52.751 INFO:tasks.workunit.client.0.smithi033.stdout: 1 786374 6.24 MB/sec execute 473 sec latency 10.515 ms 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 -- 172.21.15.33:0/21278425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee810d140 msgr2=0x7f4ee810f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 --2- 172.21.15.33:0/21278425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee810d140 0x7f4ee810f5a0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009ca0 tx=0x7f4ee002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 -- 172.21.15.33:0/21278425 shutdown_connections 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 --2- 172.21.15.33:0/21278425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee810d140 0x7f4ee810f5a0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 --2- 172.21.15.33:0/21278425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee8104600 0x7f4ee810caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 --2- 172.21.15.33:0/21278425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee8103ce0 0x7f4ee81040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 -- 172.21.15.33:0/21278425 >> 172.21.15.33:0/21278425 conn(0x7f4ee80fd720 msgr2=0x7f4ee80ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 -- 172.21.15.33:0/21278425 shutdown_connections 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.047+0000 7f4ef0794640 1 -- 172.21.15.33:0/21278425 wait complete. 2024-09-10T21:14:53.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 Processor -- start 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 -- start start 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee8103ce0 0x7f4ee81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee8104600 0x7f4ee8199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee810d140 0x7f4ee819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8112db0 con 0x7f4ee8103ce0 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8112c30 con 0x7f4ee8104600 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.048+0000 7f4ef0794640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee8112f30 con 0x7f4ee810d140 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee8103ce0 0x7f4ee81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee8103ce0 0x7f4ee81992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33050/0 (socket says 172.21.15.33:33050) 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 -- 172.21.15.33:0/3434799905 learned_addr learned my addr 172.21.15.33:0/3434799905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee8104600 0x7f4ee8199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eeed31640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee810d140 0x7f4ee819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 -- 172.21.15.33:0/3434799905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee810d140 msgr2=0x7f4ee819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee810d140 0x7f4ee819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 -- 172.21.15.33:0/3434799905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee8104600 msgr2=0x7f4ee8199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee8104600 0x7f4ee8199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 -- 172.21.15.33:0/3434799905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee819f390 con 0x7f4ee8103ce0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4eee530640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee8103ce0 0x7f4ee81992e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e9f0 tx=0x7f4ee400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4edb7fe640 1 -- 172.21.15.33:0/3434799905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400cd80 con 0x7f4ee8103ce0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4edb7fe640 1 -- 172.21.15.33:0/3434799905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee40108e0 con 0x7f4ee8103ce0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee819f680 con 0x7f4ee8103ce0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4edb7fe640 1 -- 172.21.15.33:0/3434799905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4018980 con 0x7f4ee8103ce0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.049+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee81a5d40 con 0x7f4ee8103ce0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.052+0000 7f4edb7fe640 1 -- 172.21.15.33:0/3434799905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4ee4010430 con 0x7f4ee8103ce0 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.052+0000 7f4ef2b69740 1 Processor -- start 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.052+0000 7f4ef2b69740 1 -- start start 2024-09-10T21:14:53.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.052+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fbee0 0x5616d75fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7608250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d7608790 0x5616d760ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616d7572e60 con 0x5616d75fbee0 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616d7562290 con 0x5616d7608790 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5616d75660a0 con 0x5616d75fc800 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eee530640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7608250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fbee0 0x5616d75fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fbee0 0x5616d75fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33058/0 (socket says 172.21.15.33:33058) 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 -- 172.21.15.33:0/3369411598 learned_addr learned my addr 172.21.15.33:0/3369411598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 -- 172.21.15.33:0/3369411598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 msgr2=0x5616d7608250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7608250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 -- 172.21.15.33:0/3369411598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d7608790 msgr2=0x5616d760ab80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d7608790 0x5616d760ab80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 -- 172.21.15.33:0/3369411598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d72c0930 con 0x5616d75fbee0 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eee530640 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7608250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4edb7fe640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ed40b52e0 0x7f4ed40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.053+0000 7f4eeed31640 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fbee0 0x5616d75fc2c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005fd0 tx=0x7f4ee0009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751b1b2f38c0ea99 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.054+0000 7f4eee530640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ed40b52e0 0x7f4ed40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.054+0000 7f4ed8ff9640 1 -- 172.21.15.33:0/3369411598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0031cf0 con 0x5616d75fbee0 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.054+0000 7f4ed8ff9640 1 -- 172.21.15.33:0/3369411598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee0036070 con 0x5616d75fbee0 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.054+0000 7f4edb7fe640 1 -- 172.21.15.33:0/3434799905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4ee40e5ec0 con 0x7f4ee8103ce0 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.054+0000 7f4ed8ff9640 1 -- 172.21.15.33:0/3369411598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00042a0 con 0x5616d75fbee0 2024-09-10T21:14:53.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 -- 172.21.15.33:0/3369411598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fbee0 msgr2=0x5616d75fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fbee0 0x5616d75fc2c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0005fd0 tx=0x7f4ee0009800 comp rx=0 tx=0).stop 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4eee530640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ed40b52e0 0x7f4ed40b77a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e9c0 tx=0x7f4ee4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 -- 172.21.15.33:0/3369411598 shutdown_connections 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d7608790 0x5616d760ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7608250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/3369411598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fbee0 0x5616d75fc2c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 -- 172.21.15.33:0/3369411598 >> 172.21.15.33:0/3369411598 conn(0x5616d75f9d10 msgr2=0x5616d75faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 -- 172.21.15.33:0/3369411598 shutdown_connections 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 -- 172.21.15.33:0/3369411598 wait complete. 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 -- start start 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.055+0000 7f4ef2b69740 1 Processor -- start 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d76057c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7605d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d7608790 0x5616d7606240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616d73c3890 con 0x5616d7608790 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616d74fcd40 con 0x5616d75fbee0 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5616d760b260 con 0x5616d75fc800 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ef2b69740 10 client.0 _ll_register_callbacks cb 0x5616d7560c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d7608790 0x5616d7606240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eedd2f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7605d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d7608790 0x5616d7606240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33064/0 (socket says 172.21.15.33:33064) 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eeed31640 1 -- 172.21.15.33:0/2248861859 learned_addr learned my addr 172.21.15.33:0/2248861859 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d76057c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eedd2f640 1 -- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 msgr2=0x5616d76057c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d76057c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eedd2f640 1 -- 172.21.15.33:0/2248861859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d7608790 msgr2=0x5616d7606240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d7608790 0x5616d7606240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eedd2f640 1 -- 172.21.15.33:0/2248861859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d76146f0 con 0x5616d75fc800 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7605d00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80ffe20 tx=0x7f4edc0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65adeb0563bcad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.056+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2248861859 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4edc004220 con 0x5616d75fc800 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2248861859 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4edc0043c0 con 0x5616d75fc800 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee0036070 con 0x5616d75fc800 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2248861859 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4edc0035f0 con 0x5616d75fc800 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 10 client.26839 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5616d7617050 con 0x5616d75fc800 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2248861859 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee0036070 con 0x5616d75fc800 2024-09-10T21:14:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2248861859 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4edc011820 con 0x5616d75fc800 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 10 client.26839 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 10 client.26839 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 20 client.26839 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7618560 0x5616d761a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 4 client.26839 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616d7574db0 con 0x5616d7618560 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.057+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7618560 0x5616d761a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.058+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7618560 0x5616d761a980 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c18ff0cc9be73c4 server_cookie=6f45312d922c3a08 in_seq=0 out_seq=0 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.058+0000 7f4ed1ffb640 10 client.26839 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.058+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2248861859 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616d7574db0 con 0x5616d7618560 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.058+0000 7f4ed1ffb640 10 client.26839 handle_command_reply: tid=0 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 client.26839 shutdown 2024-09-10T21:14:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 msgr2=0x5616d7605d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7605d00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80ffe20 tx=0x7f4edc0107a0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 shutdown_connections 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7618560 0x5616d761a980 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d7608790 0x5616d7606240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d75fc800 0x5616d7605d00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ed1ffb640 0 client.26839 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2248861859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d76057c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 >> 172.21.15.33:0/2248861859 conn(0x5616d75f9d10 msgr2=0x5616d75f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 shutdown_connections 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2248861859 wait complete. 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.059+0000 7f4ef2b69740 20 client.26839 trim_cache size 0 max 16384 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 Processor -- start 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 -- start start 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d75fb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d7603250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d75ff7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81ae450 con 0x5616d75fc800 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81aff90 con 0x5616d75fbee0 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81aeb20 con 0x5616d7608790 2024-09-10T21:14:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4eedd2f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d7603250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d75fb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d75fb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39332/0 (socket says 172.21.15.33:39332) 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4eeed31640 1 -- 172.21.15.33:0/2738042557 learned_addr learned my addr 172.21.15.33:0/2738042557 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.078+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d75ff7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4eeed31640 1 -- 172.21.15.33:0/2738042557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 msgr2=0x5616d75ff7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d75ff7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4eeed31640 1 -- 172.21.15.33:0/2738042557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 msgr2=0x5616d7603250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d7603250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4eeed31640 1 -- 172.21.15.33:0/2738042557 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616d75f9b40 con 0x5616d75fbee0 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d7603250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d75fb440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0003d40 tx=0x7f4ee0003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5360d3cca6975d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4ed3fff640 1 -- 172.21.15.33:0/2738042557 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0031af0 con 0x5616d75fbee0 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4ed3fff640 1 -- 172.21.15.33:0/2738042557 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee0036040 con 0x5616d75fbee0 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.079+0000 7f4ed3fff640 1 -- 172.21.15.33:0/2738042557 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0031640 con 0x5616d75fbee0 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2738042557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 msgr2=0x5616d75fb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d75fb440 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0003d40 tx=0x7f4ee0003d70 comp rx=0 tx=0).stop 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2738042557 shutdown_connections 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d75ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d7603250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2738042557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d75fb440 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2738042557 >> 172.21.15.33:0/2738042557 conn(0x5616d75f9d10 msgr2=0x5616d75ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2738042557 shutdown_connections 2024-09-10T21:14:53.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.080+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2738042557 wait complete. 2024-09-10T21:14:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 -- start start 2024-09-10T21:14:53.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 Processor -- start 2024-09-10T21:14:53.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d760c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d760cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d7684430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81af8c0 con 0x5616d75fc800 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81af1f0 con 0x5616d75fbee0 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee81ac410 con 0x5616d7608790 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4ef2b69740 10 client.0 _ll_register_callbacks cb 0x7f4edc00b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d760c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4eedd2f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d760cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4eeed31640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d760c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39342/0 (socket says 172.21.15.33:39342) 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4eeed31640 1 -- 172.21.15.33:0/2478280076 learned_addr learned my addr 172.21.15.33:0/2478280076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.081+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d7684430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4eeed31640 1 -- 172.21.15.33:0/2478280076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 msgr2=0x5616d7684430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d7684430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4eeed31640 1 -- 172.21.15.33:0/2478280076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 msgr2=0x5616d760cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:53.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d760cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4eeed31640 1 -- 172.21.15.33:0/2478280076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616d7684970 con 0x5616d75fbee0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d760cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4eeed31640 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d760c940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00317c0 tx=0x7f4ee0005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0ffa85a9d1091d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4ed1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2478280076 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0031cf0 con 0x5616d75fbee0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2478280076 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4ee0004470 con 0x5616d75fbee0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ee0036040 con 0x5616d75fbee0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2478280076 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f4ee0004670 con 0x5616d75fbee0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4ef2b69740 10 client.26894 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.082+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5616d7685bf0 con 0x5616d75fbee0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2478280076 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ee0036040 con 0x5616d75fbee0 2024-09-10T21:14:53.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2478280076 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f4ee0043020 con 0x5616d75fbee0 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ef2b69740 10 client.26894 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ef2b69740 10 client.26894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ef2b69740 20 client.26894 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7685fe0 0x5616d7688400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ef2b69740 4 client.26894 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5616d7570860 con 0x5616d7685fe0 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.083+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7685fe0 0x5616d7688400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.084+0000 7f4ed1ffb640 10 client.26894 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.084+0000 7f4eedd2f640 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7685fe0 0x5616d7688400 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48509f68a8e1de7b server_cookie=8c1681be02e81629 in_seq=0 out_seq=0 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ed1ffb640 1 -- 172.21.15.33:0/2478280076 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5616d7570860 con 0x5616d7685fe0 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ed1ffb640 10 client.26894 handle_command_reply: tid=0 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 client.26894 shutdown 2024-09-10T21:14:53.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 msgr2=0x5616d760c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d760c940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00317c0 tx=0x7f4ee0005d80 comp rx=0 tx=0).stop 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 shutdown_connections 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616d7685fe0 0x5616d7688400 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616d7608790 0x5616d7684430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616d75fc800 0x5616d760cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 --2- 172.21.15.33:0/2478280076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616d75fbee0 0x5616d760c940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ed1ffb640 0 client.26894 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:53.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.085+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 >> 172.21.15.33:0/2478280076 conn(0x5616d75f9d10 msgr2=0x5616d75f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 shutdown_connections 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef2b69740 1 -- 172.21.15.33:0/2478280076 wait complete. 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef2b69740 20 client.26894 trim_cache size 0 max 16384 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "bac5b008-4545-4044-8f53-cc00a8b4c23e", 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:14:53.106 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:14:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ed40b52e0 msgr2=0x7f4ed40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ed40b52e0 0x7f4ed40b77a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e9c0 tx=0x7f4ee4000bc0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee8103ce0 msgr2=0x7f4ee81992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee8103ce0 0x7f4ee81992e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e9f0 tx=0x7f4ee400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 shutdown_connections 2024-09-10T21:14:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4ed40b52e0 0x7f4ed40b77a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ee810d140 0x7f4ee819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4ee8104600 0x7f4ee8199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 --2- 172.21.15.33:0/3434799905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4ee8103ce0 0x7f4ee81992e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:53.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 >> 172.21.15.33:0/3434799905 conn(0x7f4ee80fd720 msgr2=0x7f4ee810f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:53.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 shutdown_connections 2024-09-10T21:14:53.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:53.086+0000 7f4ef0794640 1 -- 172.21.15.33:0/3434799905 wait complete. 2024-09-10T21:14:53.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 788481 6.26 MB/sec execute 474 sec latency 93.240 ms 2024-09-10T21:14:53.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:14:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: pgmap v673: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-09-10T21:14:54.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:53 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.077 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: pgmap v673: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:53 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: pgmap v673: 97 pgs: 97 active+clean; 301 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-09-10T21:14:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:54.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:54.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:53 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1017275267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc101c90 msgr2=0x7f7cdc102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1017275267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc101c90 0x7f7cdc102070 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009ca0 tx=0x7f7ccc02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1017275267 shutdown_connections 2024-09-10T21:14:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1017275267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cdc108020 0x7f7cdc10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1017275267 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cdc1076a0 0x7f7cdc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1017275267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc101c90 0x7f7cdc102070 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1017275267 >> 172.21.15.33:0/1017275267 conn(0x7f7cdc0fd700 msgr2=0x7f7cdc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1017275267 shutdown_connections 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.599+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1017275267 wait complete. 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 Processor -- start 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 -- start start 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cdc101c90 0x7f7cdc1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc1076a0 0x7f7cdc1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cdc108020 0x7f7cdc19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc111ce0 con 0x7f7cdc1076a0 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc111b60 con 0x7f7cdc108020 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7ce1e5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc111e60 con 0x7f7cdc101c90 2024-09-10T21:14:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc1076a0 0x7f7cdc1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cdc101c90 0x7f7cdc1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc1076a0 0x7f7cdc1997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33104/0 (socket says 172.21.15.33:33104) 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7cdaffd640 1 -- 172.21.15.33:0/1322329140 learned_addr learned my addr 172.21.15.33:0/1322329140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.600+0000 7f7cdbfff640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cdc108020 0x7f7cdc19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cdaffd640 1 -- 172.21.15.33:0/1322329140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cdc101c90 msgr2=0x7f7cdc1992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cdc101c90 0x7f7cdc1992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cdaffd640 1 -- 172.21.15.33:0/1322329140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cdc108020 msgr2=0x7f7cdc19eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cdc108020 0x7f7cdc19eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cdaffd640 1 -- 172.21.15.33:0/1322329140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cdc19f350 con 0x7f7cdc1076a0 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc1076a0 0x7f7cdc1997e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000d950 tx=0x7f7cd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cd8ff9640 1 -- 172.21.15.33:0/1322329140 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0017070 con 0x7f7cdc1076a0 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cdc19f640 con 0x7f7cdc1076a0 2024-09-10T21:14:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cd8ff9640 1 -- 172.21.15.33:0/1322329140 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd0009c30 con 0x7f7cdc1076a0 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7cd8ff9640 1 -- 172.21.15.33:0/1322329140 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd000f620 con 0x7f7cdc1076a0 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.601+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cdc1a5ed0 con 0x7f7cdc1076a0 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 Processor -- start 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 -- start start 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7cd8ff9640 1 -- 172.21.15.33:0/1322329140 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7cd0010460 con 0x7f7cdc1076a0 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f530a0 0x561626f53480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626f5f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f61d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561626ec9e50 con 0x561626f539c0 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561626eb9280 con 0x561626f530a0 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.604+0000 7f7ce422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561626ebd090 con 0x561626f5f950 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f530a0 0x561626f53480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626f5f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f530a0 0x561626f53480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39376/0 (socket says 172.21.15.33:39376) 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 -- 172.21.15.33:0/1163770398 learned_addr learned my addr 172.21.15.33:0/1163770398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f61d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 -- 172.21.15.33:0/1163770398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 msgr2=0x561626f61d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f61d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 -- 172.21.15.33:0/1163770398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 msgr2=0x561626f5f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626f5f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 -- 172.21.15.33:0/1163770398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561626c17650 con 0x561626f530a0 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cd8ff9640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b52e0 0x7f7cc40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdbfff640 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f530a0 0x561626f53480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400b700 tx=0x7f7cd400bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aca0e231ba437512 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cdb7fe640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b52e0 0x7f7cc40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cd8ff9640 1 -- 172.21.15.33:0/1322329140 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7cd00e57c0 con 0x7f7cdc1076a0 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cc1ffb640 1 -- 172.21.15.33:0/1163770398 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd4003dd0 con 0x561626f530a0 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cc1ffb640 1 -- 172.21.15.33:0/1163770398 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd4015070 con 0x561626f530a0 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.605+0000 7f7cc1ffb640 1 -- 172.21.15.33:0/1163770398 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40106a0 con 0x561626f530a0 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 -- 172.21.15.33:0/1163770398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f530a0 msgr2=0x561626f53480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7cdb7fe640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b52e0 0x7f7cc40b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009430 tx=0x7f7ccc005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f530a0 0x561626f53480 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7cd400b700 tx=0x7f7cd400bbc0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 -- 172.21.15.33:0/1163770398 shutdown_connections 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f61d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626f5f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 --2- 172.21.15.33:0/1163770398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f530a0 0x561626f53480 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 -- 172.21.15.33:0/1163770398 >> 172.21.15.33:0/1163770398 conn(0x561626f50ed0 msgr2=0x561626f52070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 -- 172.21.15.33:0/1163770398 shutdown_connections 2024-09-10T21:14:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.606+0000 7f7ce422f740 1 -- 172.21.15.33:0/1163770398 wait complete. 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 -- start start 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 Processor -- start 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f530a0 0x561626f5f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f5cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561626d1a700 con 0x561626f530a0 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561626e54110 con 0x561626f539c0 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561626f625d0 con 0x561626f5f950 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.607+0000 7f7ce422f740 10 client.0 _ll_register_callbacks cb 0x561626e6d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f530a0 0x561626f5f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f530a0 0x561626f5f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33122/0 (socket says 172.21.15.33:33122) 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 -- 172.21.15.33:0/2593628144 learned_addr learned my addr 172.21.15.33:0/2593628144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdbfff640 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f5cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 -- 172.21.15.33:0/2593628144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 msgr2=0x561626f5cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f5cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 -- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 msgr2=0x561626f5c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 -- 172.21.15.33:0/2593628144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561626e81c40 con 0x561626f530a0 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cdb7fe640 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f530a0 0x561626f5f260 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc035000 tx=0x7f7ccc02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82837d9b0372b0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cbaffd640 1 -- 172.21.15.33:0/2593628144 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc003e00 con 0x561626f530a0 2024-09-10T21:14:54.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7cbaffd640 1 -- 172.21.15.33:0/2593628144 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7ccc004040 con 0x561626f530a0 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.608+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd4015070 con 0x561626f530a0 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7cbaffd640 1 -- 172.21.15.33:0/2593628144 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7ccc004240 con 0x561626f530a0 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 10 client.17463 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561626f6df30 con 0x561626f530a0 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7cbaffd640 1 -- 172.21.15.33:0/2593628144 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd4015070 con 0x561626f530a0 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7cbaffd640 1 -- 172.21.15.33:0/2593628144 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f7ccc051070 con 0x561626f530a0 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 10 client.17463 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 20 client.17463 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f6f340 0x561626f717c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 4 client.17463 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561626ecbda0 con 0x561626f6f340 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.609+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f6f340 0x561626f717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.610+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f6f340 0x561626f717c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6489f7d029b2c17f server_cookie=996e65abd7f2d0a5 in_seq=0 out_seq=0 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.610+0000 7f7cbaffd640 10 client.17463 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7cbaffd640 1 -- 172.21.15.33:0/2593628144 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561626ecbda0 con 0x561626f6f340 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7cbaffd640 10 client.17463 handle_command_reply: tid=0 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 client.17463 shutdown 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f530a0 msgr2=0x561626f5f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f530a0 0x561626f5f260 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc035000 tx=0x7f7ccc02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 shutdown_connections 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f6f340 0x561626f717c0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f5f950 0x561626f5cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 --2- 172.21.15.33:0/2593628144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f530a0 0x561626f5f260 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 >> 172.21.15.33:0/2593628144 conn(0x561626f50ed0 msgr2=0x561626f56a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:54.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7cbaffd640 0 client.17463 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:54.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 shutdown_connections 2024-09-10T21:14:54.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 1 -- 172.21.15.33:0/2593628144 wait complete. 2024-09-10T21:14:54.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.611+0000 7f7ce422f740 20 client.17463 trim_cache size 0 max 16384 2024-09-10T21:14:54.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 Processor -- start 2024-09-10T21:14:54.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 -- start start 2024-09-10T21:14:54.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626f6b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f5f950 0x561626f5af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1ae410 con 0x561626f5f950 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1aff50 con 0x561626f539c0 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.630+0000 7f7ce422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1aeae0 con 0x561626f530a0 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f5f950 0x561626f5af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f5f950 0x561626f5af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33134/0 (socket says 172.21.15.33:33134) 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626f6b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdbfff640 1 -- 172.21.15.33:0/288643437 learned_addr learned my addr 172.21.15.33:0/288643437 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdaffd640 1 -- 172.21.15.33:0/288643437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 msgr2=0x561626f6b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626f6b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdaffd640 1 -- 172.21.15.33:0/288643437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f5f950 msgr2=0x561626f5af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f5f950 0x561626f5af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdaffd640 1 -- 172.21.15.33:0/288643437 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561626f537f0 con 0x561626f539c0 2024-09-10T21:14:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdbfff640 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f5f950 0x561626f5af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:14:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5bba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00abfc0 tx=0x7f7cd00e3190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c527628524cfe42a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.631+0000 7f7cc17fa640 1 -- 172.21.15.33:0/288643437 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0020040 con 0x561626f539c0 2024-09-10T21:14:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7cc17fa640 1 -- 172.21.15.33:0/288643437 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd0091e50 con 0x561626f539c0 2024-09-10T21:14:54.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7cc17fa640 1 -- 172.21.15.33:0/288643437 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd00b0a60 con 0x561626f539c0 2024-09-10T21:14:54.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 -- 172.21.15.33:0/288643437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 msgr2=0x561626f5bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5bba0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00abfc0 tx=0x7f7cd00e3190 comp rx=0 tx=0).stop 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 -- 172.21.15.33:0/288643437 shutdown_connections 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f5f950 0x561626f5af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f539c0 0x561626f5bba0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 --2- 172.21.15.33:0/288643437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626f6b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 -- 172.21.15.33:0/288643437 >> 172.21.15.33:0/288643437 conn(0x561626f50ed0 msgr2=0x561626f52000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 -- 172.21.15.33:0/288643437 shutdown_connections 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.632+0000 7f7ce422f740 1 -- 172.21.15.33:0/288643437 wait complete. 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 -- start start 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 Processor -- start 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626fd9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626fd99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f5f950 0x561626fd6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1af880 con 0x561626f539c0 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1af1b0 con 0x561626f5f950 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cdc1ac3d0 con 0x561626f530a0 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7ce422f740 10 client.0 _ll_register_callbacks cb 0x7f7ccc031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626fd99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7cdaffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626fd99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33144/0 (socket says 172.21.15.33:33144) 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7cdaffd640 1 -- 172.21.15.33:0/839660936 learned_addr learned my addr 172.21.15.33:0/839660936 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.633+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626fd9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cdbfff640 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f5f950 0x561626fd6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cdaffd640 1 -- 172.21.15.33:0/839660936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 msgr2=0x561626fd9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626fd9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cdaffd640 1 -- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f5f950 msgr2=0x561626fd6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:14:54.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f5f950 0x561626fd6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cdaffd640 1 -- 172.21.15.33:0/839660936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561626fd6850 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cdaffd640 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626fd99b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00e3290 tx=0x7f7cd00eb530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e315ac2b3b61807f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cbaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cbaffd640 1 -- 172.21.15.33:0/839660936 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0020070 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cbaffd640 1 -- 172.21.15.33:0/839660936 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7cd00ab490 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cd0091e50 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cbaffd640 1 -- 172.21.15.33:0/839660936 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f7cd0009de0 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7ce422f740 10 client.17469 fetch_fsmap learned FSMap version 58 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x561626f67140 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.634+0000 7f7cbaffd640 1 -- 172.21.15.33:0/839660936 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cd0091e50 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7cbaffd640 1 -- 172.21.15.33:0/839660936 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f7cd00108a0 con 0x561626f539c0 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7ce422f740 10 client.17469 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:14:54.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7ce422f740 10 client.17469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:14:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7ce422f740 20 client.17469 populate_metadata read hostname 'smithi033' 2024-09-10T21:14:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7ce422f740 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f67830 0x561626f67c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:14:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7ce422f740 4 client.17469 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:14:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561626ec7850 con 0x561626f67830 2024-09-10T21:14:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7cdb7fe640 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f67830 0x561626f67c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:14:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7cbaffd640 10 client.17469 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:54.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.635+0000 7f7cdb7fe640 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f67830 0x561626f67c40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4743497088091bb7 server_cookie=2f49930c51849aeb in_seq=0 out_seq=0 2024-09-10T21:14:54.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.636+0000 7f7cbaffd640 1 -- 172.21.15.33:0/839660936 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x561626ec7850 con 0x561626f67830 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.636+0000 7f7cbaffd640 10 client.17469 handle_command_reply: tid=0 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 client.17469 shutdown 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 msgr2=0x561626fd99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626fd99b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00e3290 tx=0x7f7cd00eb530 comp rx=0 tx=0).stop 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 shutdown_connections 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561626f67830 0x561626f67c40 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561626f5f950 0x561626fd6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561626f539c0 0x561626fd99b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7cbaffd640 0 client.17469 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 --2- 172.21.15.33:0/839660936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561626f530a0 0x561626fd9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 >> 172.21.15.33:0/839660936 conn(0x561626f50ed0 msgr2=0x561626f5dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 shutdown_connections 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 1 -- 172.21.15.33:0/839660936 wait complete. 2024-09-10T21:14:54.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.637+0000 7f7ce422f740 20 client.17469 trim_cache size 0 max 16384 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "bac5b008-4545-4044-8f53-cc00a8b4c23e", 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:14:54.646 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.648 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.653 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:14:54.654 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.655 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.656 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:14:54.657 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:14:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b52e0 msgr2=0x7f7cc40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b52e0 0x7f7cc40b77a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc009430 tx=0x7f7ccc005ef0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc1076a0 msgr2=0x7f7cdc1997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:14:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc1076a0 0x7f7cdc1997e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7cd000d950 tx=0x7f7cd000de10 comp rx=0 tx=0).stop 2024-09-10T21:14:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 shutdown_connections 2024-09-10T21:14:54.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7cc40b52e0 0x7f7cc40b77a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7cdc108020 0x7f7cdc19eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cdc1076a0 0x7f7cdc1997e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 --2- 172.21.15.33:0/1322329140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cdc101c90 0x7f7cdc1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:14:54.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 >> 172.21.15.33:0/1322329140 conn(0x7f7cdc0fd700 msgr2=0x7f7cdc06ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:14:54.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 shutdown_connections 2024-09-10T21:14:54.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:14:54.638+0000 7f7ce1e5a640 1 -- 172.21.15.33:0/1322329140 wait complete. 2024-09-10T21:14:54.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 790659 6.26 MB/sec execute 475 sec latency 19.214 ms 2024-09-10T21:14:54.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:14:55.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:55.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:55.006 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:14:55.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.007 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:54 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:14:55.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:55.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:55.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:14:55.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:54 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:54 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:14:55.434 INFO:tasks.cephfs.filesystem:scrub status for tag:bac5b008-4545-4044-8f53-cc00a8b4c23e - {'path': '/', 'tag': 'bac5b008-4545-4044-8f53-cc00a8b4c23e', 'options': 'recursive,force'} 2024-09-10T21:14:55.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 792255 6.27 MB/sec execute 476 sec latency 51.790 ms 2024-09-10T21:14:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:55 smithi104 ceph-mon[35325]: pgmap v674: 97 pgs: 97 active+clean; 303 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-09-10T21:14:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:55 smithi174 ceph-mon[35635]: pgmap v674: 97 pgs: 97 active+clean; 303 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-09-10T21:14:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:55 smithi033 ceph-mon[30860]: pgmap v674: 97 pgs: 97 active+clean; 303 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 330 op/s 2024-09-10T21:14:56.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 794958 6.29 MB/sec execute 477 sec latency 10.561 ms 2024-09-10T21:14:57.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 796572 6.29 MB/sec execute 478 sec latency 11.173 ms 2024-09-10T21:14:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:57 smithi104 ceph-mon[35325]: pgmap v675: 97 pgs: 97 active+clean; 304 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 308 op/s 2024-09-10T21:14:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:57 smithi174 ceph-mon[35635]: pgmap v675: 97 pgs: 97 active+clean; 304 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 308 op/s 2024-09-10T21:14:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:57 smithi033 ceph-mon[30860]: pgmap v675: 97 pgs: 97 active+clean; 304 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 308 op/s 2024-09-10T21:14:58.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 799146 6.31 MB/sec execute 479 sec latency 15.202 ms 2024-09-10T21:14:59.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 801304 6.31 MB/sec execute 480 sec latency 10.211 ms 2024-09-10T21:15:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:14:59 smithi104 ceph-mon[35325]: pgmap v676: 97 pgs: 97 active+clean; 307 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 379 op/s 2024-09-10T21:15:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:14:59 smithi174 ceph-mon[35635]: pgmap v676: 97 pgs: 97 active+clean; 307 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 379 op/s 2024-09-10T21:15:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:14:59 smithi033 ceph-mon[30860]: pgmap v676: 97 pgs: 97 active+clean; 307 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.0 MiB/s wr, 379 op/s 2024-09-10T21:15:00.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 802874 6.32 MB/sec execute 481 sec latency 52.588 ms 2024-09-10T21:15:01.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 805288 6.34 MB/sec execute 482 sec latency 15.072 ms 2024-09-10T21:15:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:01 smithi104 ceph-mon[35325]: pgmap v677: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 329 op/s 2024-09-10T21:15:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:01 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:01 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:01 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:02.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:01 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:01 smithi174 ceph-mon[35635]: pgmap v677: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 329 op/s 2024-09-10T21:15:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:01 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:01 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:01 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:01 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:01 smithi033 ceph-mon[30860]: pgmap v677: 97 pgs: 97 active+clean; 308 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.0 MiB/s wr, 329 op/s 2024-09-10T21:15:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:01 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:01 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:01 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:01 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:02.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 806711 6.34 MB/sec execute 483 sec latency 13.871 ms 2024-09-10T21:15:03.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:02 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:02 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:03.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:02 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:03.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 809303 6.35 MB/sec execute 484 sec latency 16.327 ms 2024-09-10T21:15:04.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:03 smithi104 ceph-mon[35325]: pgmap v678: 97 pgs: 97 active+clean; 310 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 320 op/s 2024-09-10T21:15:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:03 smithi174 ceph-mon[35635]: pgmap v678: 97 pgs: 97 active+clean; 310 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 320 op/s 2024-09-10T21:15:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:03 smithi033 ceph-mon[30860]: pgmap v678: 97 pgs: 97 active+clean; 310 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 320 op/s 2024-09-10T21:15:04.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 810680 6.36 MB/sec execute 485 sec latency 16.227 ms 2024-09-10T21:15:05.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 813013 6.37 MB/sec execute 486 sec latency 32.339 ms 2024-09-10T21:15:06.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:05 smithi104 ceph-mon[35325]: pgmap v679: 97 pgs: 97 active+clean; 312 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 362 op/s 2024-09-10T21:15:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:05 smithi174 ceph-mon[35635]: pgmap v679: 97 pgs: 97 active+clean; 312 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 362 op/s 2024-09-10T21:15:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:05 smithi033 ceph-mon[30860]: pgmap v679: 97 pgs: 97 active+clean; 312 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 362 op/s 2024-09-10T21:15:06.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 814263 6.37 MB/sec execute 487 sec latency 17.342 ms 2024-09-10T21:15:07.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 816813 6.39 MB/sec execute 488 sec latency 12.350 ms 2024-09-10T21:15:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:07 smithi104 ceph-mon[35325]: pgmap v680: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-10T21:15:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:07 smithi174 ceph-mon[35635]: pgmap v680: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-10T21:15:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:07 smithi033 ceph-mon[30860]: pgmap v680: 97 pgs: 97 active+clean; 311 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 297 op/s 2024-09-10T21:15:08.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 818369 6.39 MB/sec execute 489 sec latency 12.010 ms 2024-09-10T21:15:09.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 820431 6.40 MB/sec execute 490 sec latency 19.184 ms 2024-09-10T21:15:10.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:09 smithi104 ceph-mon[35325]: pgmap v681: 97 pgs: 97 active+clean; 313 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-09-10T21:15:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:09 smithi174 ceph-mon[35635]: pgmap v681: 97 pgs: 97 active+clean; 313 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-09-10T21:15:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:09 smithi033 ceph-mon[30860]: pgmap v681: 97 pgs: 97 active+clean; 313 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 363 op/s 2024-09-10T21:15:10.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 822471 6.40 MB/sec execute 491 sec latency 47.910 ms 2024-09-10T21:15:11.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 824056 6.42 MB/sec execute 492 sec latency 18.260 ms 2024-09-10T21:15:12.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:11 smithi104 ceph-mon[35325]: pgmap v682: 97 pgs: 97 active+clean; 314 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 296 op/s 2024-09-10T21:15:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:11 smithi174 ceph-mon[35635]: pgmap v682: 97 pgs: 97 active+clean; 314 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 296 op/s 2024-09-10T21:15:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:11 smithi033 ceph-mon[30860]: pgmap v682: 97 pgs: 97 active+clean; 314 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 296 op/s 2024-09-10T21:15:12.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 826113 6.42 MB/sec execute 493 sec latency 13.242 ms 2024-09-10T21:15:13.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 827698 6.43 MB/sec execute 494 sec latency 12.032 ms 2024-09-10T21:15:14.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:13 smithi104 ceph-mon[35325]: pgmap v683: 97 pgs: 97 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 317 op/s 2024-09-10T21:15:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:13 smithi174 ceph-mon[35635]: pgmap v683: 97 pgs: 97 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 317 op/s 2024-09-10T21:15:14.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:13 smithi033 ceph-mon[30860]: pgmap v683: 97 pgs: 97 active+clean; 316 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 317 op/s 2024-09-10T21:15:14.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 830142 6.44 MB/sec execute 495 sec latency 12.372 ms 2024-09-10T21:15:15.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 831702 6.44 MB/sec execute 496 sec latency 34.915 ms 2024-09-10T21:15:16.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:15 smithi104 ceph-mon[35325]: pgmap v684: 97 pgs: 97 active+clean; 317 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-09-10T21:15:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:15 smithi174 ceph-mon[35635]: pgmap v684: 97 pgs: 97 active+clean; 317 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-09-10T21:15:16.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:15 smithi033 ceph-mon[30860]: pgmap v684: 97 pgs: 97 active+clean; 317 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 332 op/s 2024-09-10T21:15:16.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 834441 6.46 MB/sec execute 497 sec latency 16.220 ms 2024-09-10T21:15:17.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 836539 6.47 MB/sec execute 498 sec latency 13.213 ms 2024-09-10T21:15:18.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:17 smithi104 ceph-mon[35325]: pgmap v685: 97 pgs: 97 active+clean; 318 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-10T21:15:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:17 smithi174 ceph-mon[35635]: pgmap v685: 97 pgs: 97 active+clean; 318 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-10T21:15:18.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:17 smithi033 ceph-mon[30860]: pgmap v685: 97 pgs: 97 active+clean; 318 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 283 op/s 2024-09-10T21:15:18.752 INFO:tasks.workunit.client.0.smithi033.stdout: 1 838436 6.48 MB/sec execute 499 sec latency 14.321 ms 2024-09-10T21:15:19.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 841073 6.49 MB/sec execute 500 sec latency 10.339 ms 2024-09-10T21:15:20.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:19 smithi104 ceph-mon[35325]: pgmap v686: 97 pgs: 97 active+clean; 321 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2024-09-10T21:15:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:19 smithi174 ceph-mon[35635]: pgmap v686: 97 pgs: 97 active+clean; 321 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2024-09-10T21:15:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:19 smithi033 ceph-mon[30860]: pgmap v686: 97 pgs: 97 active+clean; 321 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 358 op/s 2024-09-10T21:15:20.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 842417 6.49 MB/sec execute 501 sec latency 52.711 ms 2024-09-10T21:15:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:15:21.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:20 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:15:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:20 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:15:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:20 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:21.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 845106 6.51 MB/sec execute 502 sec latency 17.987 ms 2024-09-10T21:15:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:21 smithi104 ceph-mon[35325]: pgmap v687: 97 pgs: 97 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 292 op/s 2024-09-10T21:15:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:15:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:15:22.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:21 smithi174 ceph-mon[35635]: pgmap v687: 97 pgs: 97 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 292 op/s 2024-09-10T21:15:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:15:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:15:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:21 smithi033 ceph-mon[30860]: pgmap v687: 97 pgs: 97 active+clean; 322 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 292 op/s 2024-09-10T21:15:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:15:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:15:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:15:22.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 846712 6.52 MB/sec execute 503 sec latency 34.809 ms 2024-09-10T21:15:23.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 848996 6.53 MB/sec execute 504 sec latency 13.521 ms 2024-09-10T21:15:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:23 smithi104 ceph-mon[35325]: pgmap v688: 97 pgs: 97 active+clean; 325 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-09-10T21:15:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:23 smithi174 ceph-mon[35635]: pgmap v688: 97 pgs: 97 active+clean; 325 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-09-10T21:15:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:23 smithi033 ceph-mon[30860]: pgmap v688: 97 pgs: 97 active+clean; 325 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.8 MiB/s wr, 306 op/s 2024-09-10T21:15:24.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 851482 6.54 MB/sec execute 505 sec latency 13.774 ms 2024-09-10T21:15:25.435 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:15:25.716 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:25.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 852923 6.54 MB/sec execute 506 sec latency 24.279 ms 2024-09-10T21:15:26.039 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:25 smithi033 ceph-mon[30860]: pgmap v689: 97 pgs: 97 active+clean; 328 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-09-10T21:15:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:25 smithi104 ceph-mon[35325]: pgmap v689: 97 pgs: 97 active+clean; 328 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-09-10T21:15:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:25 smithi174 ceph-mon[35635]: pgmap v689: 97 pgs: 97 active+clean; 328 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.2 MiB/s wr, 335 op/s 2024-09-10T21:15:26.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 -- 172.21.15.33:0/4156133528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4101830 msgr2=0x7fa6b4101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/4156133528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4101830 0x7fa6b4101c10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009ca0 tx=0x7fa6a402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 -- 172.21.15.33:0/4156133528 shutdown_connections 2024-09-10T21:15:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/4156133528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b410d0f0 0x7fa6b410f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/4156133528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b4102230 0x7fa6b410c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/4156133528 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4101830 0x7fa6b4101c10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 -- 172.21.15.33:0/4156133528 >> 172.21.15.33:0/4156133528 conn(0x7fa6b4076e20 msgr2=0x7fa6b4077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 -- 172.21.15.33:0/4156133528 shutdown_connections 2024-09-10T21:15:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.245+0000 7fa6b9a58640 1 -- 172.21.15.33:0/4156133528 wait complete. 2024-09-10T21:15:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.246+0000 7fa6b9a58640 1 Processor -- start 2024-09-10T21:15:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.246+0000 7fa6b9a58640 1 -- start start 2024-09-10T21:15:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b9a58640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b4101830 0x7fa6b41992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b9a58640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4102230 0x7fa6b4199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b9a58640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b410d0f0 0x7fa6b419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b9a58640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4112df0 con 0x7fa6b410d0f0 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b9a58640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4112c70 con 0x7fa6b4102230 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b9a58640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b4112f70 con 0x7fa6b4101830 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b4101830 0x7fa6b41992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b410d0f0 0x7fa6b419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b4101830 0x7fa6b41992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:43028/0 (socket says 172.21.15.33:43028) 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b410d0f0 0x7fa6b419eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33066/0 (socket says 172.21.15.33:33066) 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/2525312613 learned_addr learned my addr 172.21.15.33:0/2525312613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4102230 0x7fa6b4199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b27fc640 1 -- 172.21.15.33:0/2525312613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b4101830 msgr2=0x7fa6b41992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b4101830 0x7fa6b41992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b27fc640 1 -- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b410d0f0 msgr2=0x7fa6b419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b410d0f0 0x7fa6b419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b27fc640 1 -- 172.21.15.33:0/2525312613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b419f370 con 0x7fa6b4102230 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b4101830 0x7fa6b41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.247+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b410d0f0 0x7fa6b419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.248+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4102230 0x7fa6b4199800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800e9c0 tx=0x7fa6a800edc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.248+0000 7fa6b8a56640 1 -- 172.21.15.33:0/2525312613 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800ccc0 con 0x7fa6b4102230 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.248+0000 7fa6b8a56640 1 -- 172.21.15.33:0/2525312613 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a80035f0 con 0x7fa6b4102230 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.248+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b419f660 con 0x7fa6b4102230 2024-09-10T21:15:26.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.248+0000 7fa6b8a56640 1 -- 172.21.15.33:0/2525312613 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a800ce60 con 0x7fa6b4102230 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.248+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b41a5ef0 con 0x7fa6b4102230 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.250+0000 7fa6b8a56640 1 -- 172.21.15.33:0/2525312613 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa6a8004000 con 0x7fa6b4102230 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 Processor -- start 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 -- start start 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b769480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b779820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55912b6dfe50 con 0x55912b779d60 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55912b6cf280 con 0x55912b7690a0 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55912b6d3090 con 0x55912b7699c0 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b769480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6b27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b769480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:35804/0 (socket says 172.21.15.33:35804) 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6b37fe640 1 -- 172.21.15.33:0/3527376452 learned_addr learned my addr 172.21.15.33:0/3527376452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b779820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.251+0000 7fa6b8a56640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa69c0b5360 0x7fa69c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b27fc640 1 -- 172.21.15.33:0/3527376452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 msgr2=0x55912b779820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b779820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b27fc640 1 -- 172.21.15.33:0/3527376452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 msgr2=0x55912b769480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b769480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b8a56640 1 -- 172.21.15.33:0/2525312613 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa6a80e5e50 con 0x7fa6b4102230 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b27fc640 1 -- 172.21.15.33:0/3527376452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912b42d650 con 0x55912b779d60 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b769480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa69c0b5360 0x7fa69c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77c150 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80b0bf0 tx=0x7fa6a8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fec2e95161d60b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa699ffb640 1 -- 172.21.15.33:0/3527376452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a80ac0e0 con 0x55912b779d60 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa699ffb640 1 -- 172.21.15.33:0/3527376452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a80accd0 con 0x55912b779d60 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.252+0000 7fa699ffb640 1 -- 172.21.15.33:0/3527376452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8009430 con 0x55912b779d60 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/3527376452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 msgr2=0x55912b77c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77c150 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa6a80b0bf0 tx=0x7fa6a8005bb0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa69c0b5360 0x7fa69c0b7820 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009fd0 tx=0x7fa6ac0046c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/3527376452 shutdown_connections 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77c150 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b779820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/3527376452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b769480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/3527376452 >> 172.21.15.33:0/3527376452 conn(0x55912b766ed0 msgr2=0x55912b768070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/3527376452 shutdown_connections 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/3527376452 wait complete. 2024-09-10T21:15:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.253+0000 7fa6bbe2d740 1 -- start start 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 1 Processor -- start 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b779610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7699c0 0x55912b776ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b779d60 0x55912b777070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55912b530700 con 0x55912b7690a0 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55912b66a110 con 0x55912b7699c0 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55912b77c9b0 con 0x55912b779d60 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6bbe2d740 10 client.0 _ll_register_callbacks cb 0x55912b683990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b779610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b779610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33070/0 (socket says 172.21.15.33:33070) 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/440179639 learned_addr learned my addr 172.21.15.33:0/440179639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:26.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b779d60 0x55912b777070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.254+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7699c0 0x55912b776ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/440179639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b779d60 msgr2=0x55912b777070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b779d60 0x55912b777070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7699c0 msgr2=0x55912b776ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7699c0 0x55912b776ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/440179639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912b697c40 con 0x55912b7690a0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7699c0 0x55912b776ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b779610 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402aeb0 tx=0x7fa6a4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88060e226afe148b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa692ffd640 1 -- 172.21.15.33:0/440179639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a404b060 con 0x55912b7690a0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a80accd0 con 0x55912b7690a0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa692ffd640 1 -- 172.21.15.33:0/440179639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa6a4003b80 con 0x55912b7690a0 2024-09-10T21:15:26.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa692ffd640 1 -- 172.21.15.33:0/440179639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa6a40098c0 con 0x55912b7690a0 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6bbe2d740 10 client.17481 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.255+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55912b7882a0 con 0x55912b7690a0 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa692ffd640 1 -- 172.21.15.33:0/440179639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a80accd0 con 0x55912b7690a0 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa692ffd640 1 -- 172.21.15.33:0/440179639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fa6a4050070 con 0x55912b7690a0 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa6bbe2d740 10 client.17481 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa6bbe2d740 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa6bbe2d740 20 client.17481 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7897b0 0x55912b78bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa6bbe2d740 4 client.17481 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55912b6e1da0 con 0x55912b7897b0 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.256+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7897b0 0x55912b78bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.257+0000 7fa692ffd640 10 client.17481 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:26.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.257+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7897b0 0x55912b78bc10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cebf368d1c335051 server_cookie=7042476c5db6e642 in_seq=0 out_seq=0 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.257+0000 7fa692ffd640 1 -- 172.21.15.33:0/440179639 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55912b6e1da0 con 0x55912b7897b0 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.257+0000 7fa692ffd640 10 client.17481 handle_command_reply: tid=0 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.257+0000 7fa6bbe2d740 1 client.17481 shutdown 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 msgr2=0x55912b779610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b779610 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fa6a402aeb0 tx=0x7fa6a4005f90 comp rx=0 tx=0).stop 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 shutdown_connections 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7897b0 0x55912b78bc10 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b779d60 0x55912b777070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7699c0 0x55912b776ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa692ffd640 0 client.17481 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/440179639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b779610 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 >> 172.21.15.33:0/440179639 conn(0x55912b766ed0 msgr2=0x55912b76ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 shutdown_connections 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/440179639 wait complete. 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.258+0000 7fa6bbe2d740 20 client.17481 trim_cache size 0 max 16384 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 Processor -- start 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 -- start start 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b785be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b77ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b779d60 0x55912b77b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41ac1a0 con 0x55912b7690a0 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41adce0 con 0x55912b779d60 2024-09-10T21:15:26.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41ac870 con 0x55912b7699c0 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b785be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b785be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33082/0 (socket says 172.21.15.33:33082) 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/1948720236 learned_addr learned my addr 172.21.15.33:0/1948720236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b77ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b779d60 0x55912b77b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/1948720236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 msgr2=0x55912b77ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b77ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/1948720236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b779d60 msgr2=0x55912b77b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b779d60 0x55912b77b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b2ffd640 1 -- 172.21.15.33:0/1948720236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55912b7773e0 con 0x55912b7690a0 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.278+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b77ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:26.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.279+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b785be0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009bf0 tx=0x7fa6a4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d6d6a635f2ffb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.279+0000 7fa6997fa640 1 -- 172.21.15.33:0/1948720236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4056070 con 0x55912b7690a0 2024-09-10T21:15:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.279+0000 7fa6997fa640 1 -- 172.21.15.33:0/1948720236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a4036040 con 0x55912b7690a0 2024-09-10T21:15:26.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.279+0000 7fa6997fa640 1 -- 172.21.15.33:0/1948720236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4040020 con 0x55912b7690a0 2024-09-10T21:15:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/1948720236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 msgr2=0x55912b785be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b785be0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009bf0 tx=0x7fa6a4005ef0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/1948720236 shutdown_connections 2024-09-10T21:15:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b779d60 0x55912b77b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b77ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/1948720236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b7690a0 0x55912b785be0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/1948720236 >> 172.21.15.33:0/1948720236 conn(0x55912b766ed0 msgr2=0x55912b77a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.280+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/1948720236 shutdown_connections 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/1948720236 wait complete. 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 -- start start 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 Processor -- start 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b781e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b782370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41ad610 con 0x55912b779d60 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41acf40 con 0x55912b7690a0 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.281+0000 7fa6bbe2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b41aa160 con 0x55912b7699c0 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6bbe2d740 10 client.0 _ll_register_callbacks cb 0x7fa6a4009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33090/0 (socket says 172.21.15.33:33090) 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 -- 172.21.15.33:0/455882175 learned_addr learned my addr 172.21.15.33:0/455882175 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b781e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b782370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 -- 172.21.15.33:0/455882175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 msgr2=0x55912b782370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b782370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 -- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 msgr2=0x55912b781e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b781e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 -- 172.21.15.33:0/455882175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55912b77d2a0 con 0x55912b779d60 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b27fc640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b782370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b781e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:26.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa6b37fe640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77cc00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac01e710 tx=0x7fa6ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d737855e9f4f5610 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa692ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa692ffd640 1 -- 172.21.15.33:0/455882175 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac02a650 con 0x55912b779d60 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.282+0000 7fa692ffd640 1 -- 172.21.15.33:0/455882175 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa6ac02a7f0 con 0x55912b779d60 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a4036040 con 0x55912b779d60 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa692ffd640 1 -- 172.21.15.33:0/455882175 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fa6ac02a9f0 con 0x55912b779d60 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 10 client.17487 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55912b7f70b0 con 0x55912b779d60 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa692ffd640 1 -- 172.21.15.33:0/455882175 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a4036040 con 0x55912b779d60 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa692ffd640 1 -- 172.21.15.33:0/455882175 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fa6ac01cbc0 con 0x55912b779d60 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 10 client.17487 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 20 client.17487 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:26.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7f78e0 0x55912b7f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 4 client.17487 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55912b6dd850 con 0x55912b7f78e0 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.283+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7f78e0 0x55912b7f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6b2ffd640 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7f78e0 0x55912b7f9d00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d533565c660bb76b server_cookie=f558ecfcecfdc5ef in_seq=0 out_seq=0 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa692ffd640 10 client.17487 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa692ffd640 1 -- 172.21.15.33:0/455882175 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55912b6dd850 con 0x55912b7f78e0 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa692ffd640 10 client.17487 handle_command_reply: tid=0 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 client.17487 shutdown 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 msgr2=0x55912b77cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77cc00 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac01e710 tx=0x7fa6ac004730 comp rx=0 tx=0).stop 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 shutdown_connections 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55912b7f78e0 0x55912b7f9d00 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa692ffd640 0 client.17487 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55912b779d60 0x55912b77cc00 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55912b7699c0 0x55912b782370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 --2- 172.21.15.33:0/455882175 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55912b7690a0 0x55912b781e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 >> 172.21.15.33:0/455882175 conn(0x55912b766ed0 msgr2=0x55912b767fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 shutdown_connections 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.284+0000 7fa6bbe2d740 1 -- 172.21.15.33:0/455882175 wait complete. 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.285+0000 7fa6bbe2d740 20 client.17487 trim_cache size 0 max 16384 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:15:26.306 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:15:26.308 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:15:26.311 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:15:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.313 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:15:26.313 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:15:26.313 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.313 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.313 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.314 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:15:26.314 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:15:26.314 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:15:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:15:26.315 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:15:26.316 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:15:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.317 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:15:26.318 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:15:26.318 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.318 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.318 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.318 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.318 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.319 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:15:26.319 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:15:26.319 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.319 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.319 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:26.320 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:26.321 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:15:26.321 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:15:26.321 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:15:26.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.285+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa69c0b5360 msgr2=0x7fa69c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.285+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa69c0b5360 0x7fa69c0b7820 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009fd0 tx=0x7fa6ac0046c0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.285+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4102230 msgr2=0x7fa6b4199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.285+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4102230 0x7fa6b4199800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa6a800e9c0 tx=0x7fa6a800edc0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 shutdown_connections 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa69c0b5360 0x7fa69c0b7820 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6b410d0f0 0x7fa6b419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa6b4102230 0x7fa6b4199800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 --2- 172.21.15.33:0/2525312613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6b4101830 0x7fa6b41992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 >> 172.21.15.33:0/2525312613 conn(0x7fa6b4076e20 msgr2=0x7fa6b40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 shutdown_connections 2024-09-10T21:15:26.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:26.286+0000 7fa6b9a58640 1 -- 172.21.15.33:0/2525312613 wait complete. 2024-09-10T21:15:26.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 855474 6.56 MB/sec execute 507 sec latency 27.381 ms 2024-09-10T21:15:27.052 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:15:27.053 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:15:27.328 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:27.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 856837 6.56 MB/sec execute 508 sec latency 65.921 ms 2024-09-10T21:15:27.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 -- 172.21.15.33:0/3377440395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 msgr2=0x7f6dfc10e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:27.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3377440395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc10e820 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f6df4009ca0 tx=0x7f6df402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:15:27.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 -- 172.21.15.33:0/3377440395 shutdown_connections 2024-09-10T21:15:27.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3377440395 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc10e820 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:27.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3377440395 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3377440395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dfc101c90 0x7f6dfc102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 -- 172.21.15.33:0/3377440395 >> 172.21.15.33:0/3377440395 conn(0x7f6dfc0fd700 msgr2=0x7f6dfc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 -- 172.21.15.33:0/3377440395 shutdown_connections 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.784+0000 7f6e014dc640 1 -- 172.21.15.33:0/3377440395 wait complete. 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 Processor -- start 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 -- start start 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dfc101c90 0x7f6dfc19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111ca0 con 0x7f6dfc108020 2024-09-10T21:15:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111b20 con 0x7f6dfc1076a0 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6e014dc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6dfc111e20 con 0x7f6dfc101c90 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6dfaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dfc101c90 0x7f6dfc19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6dfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33098/0 (socket says 172.21.15.33:33098) 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.785+0000 7f6dfb7fe640 1 -- 172.21.15.33:0/3273657275 learned_addr learned my addr 172.21.15.33:0/3273657275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6dfa7fc640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6dfb7fe640 1 -- 172.21.15.33:0/3273657275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dfc101c90 msgr2=0x7f6dfc19a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6dfb7fe640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dfc101c90 0x7f6dfc19a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6dfb7fe640 1 -- 172.21.15.33:0/3273657275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dfc1076a0 msgr2=0x7f6dfc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6dfb7fe640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:27.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6dfb7fe640 1 -- 172.21.15.33:0/3273657275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dfc1a54b0 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6dfb7fe640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ab70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098d0 tx=0x7f6df40317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6de3fff640 1 -- 172.21.15.33:0/3273657275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df4003b70 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6de3fff640 1 -- 172.21.15.33:0/3273657275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6df4003d10 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dfc1a5740 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6de3fff640 1 -- 172.21.15.33:0/3273657275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6df402fa20 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.786+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dfc1a5c80 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.787+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dfc102070 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.789+0000 7f6de3fff640 1 -- 172.21.15.33:0/3273657275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6df402fbe0 con 0x7f6dfc108020 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.790+0000 7f6de3fff640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6de40b52e0 0x7f6de40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.790+0000 7f6dfaffd640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6de40b52e0 0x7f6de40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:27.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.790+0000 7f6de3fff640 1 -- 172.21.15.33:0/3273657275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6df4106fc0 con 0x7f6dfc108020 2024-09-10T21:15:27.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.791+0000 7f6dfaffd640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6de40b52e0 0x7f6de40b77a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005d00 tx=0x7f6dec00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:27.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:27.793+0000 7f6de3fff640 1 -- 172.21.15.33:0/3273657275 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6df40cde80 con 0x7f6dfc108020 2024-09-10T21:15:28.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.034+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6dfc02e5c0 con 0x7f6dfc108020 2024-09-10T21:15:28.038 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:27 smithi033 ceph-mon[30860]: pgmap v690: 97 pgs: 97 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-09-10T21:15:28.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.035+0000 7f6de3fff640 1 -- 172.21.15.33:0/3273657275 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+11973 (secure 0 0 0) 0x7f6df40cd780 con 0x7f6dfc108020 2024-09-10T21:15:28.039 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:15:28.039 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":58,"btime":"2024-09-10T21:11:31:688744+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":58,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:11:30.989029+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:15:28.039 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 58 2024-09-10T21:15:28.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6de40b52e0 msgr2=0x7f6de40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:28.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6de40b52e0 0x7f6de40b77a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6dec005d00 tx=0x7f6dec00b410 comp rx=0 tx=0).stop 2024-09-10T21:15:28.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 msgr2=0x7f6dfc19ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:28.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ab70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f6df40098d0 tx=0x7f6df40317b0 comp rx=0 tx=0).stop 2024-09-10T21:15:28.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 shutdown_connections 2024-09-10T21:15:28.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6de40b52e0 0x7f6de40b77a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:28.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6dfc108020 0x7f6dfc19ab70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:28.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6dfc1076a0 0x7f6dfc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:28.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 --2- 172.21.15.33:0/3273657275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dfc101c90 0x7f6dfc19a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:28.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 >> 172.21.15.33:0/3273657275 conn(0x7f6dfc0fd700 msgr2=0x7f6dfc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:28.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 shutdown_connections 2024-09-10T21:15:28.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:28.037+0000 7f6e014dc640 1 -- 172.21.15.33:0/3273657275 wait complete. 2024-09-10T21:15:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:27 smithi104 ceph-mon[35325]: pgmap v690: 97 pgs: 97 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-09-10T21:15:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:27 smithi174 ceph-mon[35635]: pgmap v690: 97 pgs: 97 active+clean; 327 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.8 MiB/s wr, 298 op/s 2024-09-10T21:15:28.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 859470 6.58 MB/sec execute 509 sec latency 10.821 ms 2024-09-10T21:15:28.754 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:15:29.029 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:29.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:28 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3273657275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:15:29.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:28 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3273657275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:15:29.152 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:28 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3273657275' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:15:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.742+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/3177037873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8101c90 msgr2=0x7fb4e8102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.742+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/3177037873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8101c90 0x7fb4e8102070 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc009900 tx=0x7fb4dc02a410 comp rx=0 tx=0).stop 2024-09-10T21:15:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.742+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/3177037873 shutdown_connections 2024-09-10T21:15:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.742+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/3177037873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8108020 0x7fb4e810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.742+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/3177037873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4e81076a0 0x7fb4e8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.742+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/3177037873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8101c90 0x7fb4e8102070 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.742+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/3177037873 >> 172.21.15.33:0/3177037873 conn(0x7fb4e80fd700 msgr2=0x7fb4e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:29.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.743+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/3177037873 shutdown_connections 2024-09-10T21:15:29.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.743+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/3177037873 wait complete. 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.743+0000 7fb4ef1ee640 1 Processor -- start 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.743+0000 7fb4ef1ee640 1 -- start start 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ef1ee640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8101c90 0x7fb4e8197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ef1ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4e81076a0 0x7fb4e8197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ef1ee640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8108020 0x7fb4e819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8101c90 0x7fb4e8197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ef1ee640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8110be0 con 0x7fb4e8108020 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8101c90 0x7fb4e8197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37580/0 (socket says 172.21.15.33:37580) 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2405946465 learned_addr learned my addr 172.21.15.33:0/2405946465 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ed78b640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8108020 0x7fb4e819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8110a60 con 0x7fb4e8101c90 2024-09-10T21:15:29.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e8110d60 con 0x7fb4e81076a0 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4e81076a0 0x7fb4e8197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2405946465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4e81076a0 msgr2=0x7fb4e8197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4e81076a0 0x7fb4e8197690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8108020 msgr2=0x7fb4e819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8108020 0x7fb4e819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.744+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2405946465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4e819d200 con 0x7fb4e8101c90 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.745+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8101c90 0x7fb4e8197150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005940 tx=0x7fb4dc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.745+0000 7fb4e5ffb640 1 -- 172.21.15.33:0/2405946465 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc03f030 con 0x7fb4e8101c90 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.745+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4e819d490 con 0x7fb4e8101c90 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.745+0000 7fb4e5ffb640 1 -- 172.21.15.33:0/2405946465 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc003aa0 con 0x7fb4e8101c90 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.745+0000 7fb4e5ffb640 1 -- 172.21.15.33:0/2405946465 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc035070 con 0x7fb4e8101c90 2024-09-10T21:15:29.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.745+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4e81a3bb0 con 0x7fb4e8101c90 2024-09-10T21:15:29.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.748+0000 7fb4e5ffb640 1 -- 172.21.15.33:0/2405946465 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb4dc04c060 con 0x7fb4e8101c90 2024-09-10T21:15:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.748+0000 7fb4f15c3740 1 Processor -- start 2024-09-10T21:15:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4f15c3740 1 -- start start 2024-09-10T21:15:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749e0f0 0x55ac3749e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374ae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4e5ffb640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb4d80b5360 0x7fb4d80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ac37414ea0 con 0x55ac374aedb0 2024-09-10T21:15:29.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 861620 6.58 MB/sec execute 510 sec latency 40.861 ms 2024-09-10T21:15:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ac374042d0 con 0x55ac3749e0f0 2024-09-10T21:15:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac374080e0 con 0x55ac3749ea10 2024-09-10T21:15:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:38084/0 (socket says 172.21.15.33:38084) 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2433209975 learned_addr learned my addr 172.21.15.33:0/2433209975 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb4d80b5360 0x7fb4d80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ed78b640 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749e0f0 0x55ac3749e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4e5ffb640 1 -- 172.21.15.33:0/2405946465 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb4dc105c40 con 0x7fb4e8101c90 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ed78b640 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2433209975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749e0f0 msgr2=0x55ac3749e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749e0f0 0x55ac3749e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2433209975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 msgr2=0x55ac374b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/2433209975 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac37162650 con 0x55ac3749ea10 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.749+0000 7fb4ed78b640 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749e0f0 0x55ac3749e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:15:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.750+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374ae870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0c8bc0 tx=0x7fb4dc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceeef58c98bd44dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.750+0000 7fb4d2ffd640 1 -- 172.21.15.33:0/2433209975 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc049070 con 0x55ac3749ea10 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.750+0000 7fb4d2ffd640 1 -- 172.21.15.33:0/2433209975 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc0cbfc0 con 0x55ac3749ea10 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.750+0000 7fb4d2ffd640 1 -- 172.21.15.33:0/2433209975 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc0d5990 con 0x55ac3749ea10 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.750+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb4d80b5360 0x7fb4d80b7820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb4e81010e0 tx=0x7fb4d4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 -- 172.21.15.33:0/2433209975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 msgr2=0x55ac374ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374ae870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0c8bc0 tx=0x7fb4dc02a410 comp rx=0 tx=0).stop 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 -- 172.21.15.33:0/2433209975 shutdown_connections 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374ae870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/2433209975 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749e0f0 0x55ac3749e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 -- 172.21.15.33:0/2433209975 >> 172.21.15.33:0/2433209975 conn(0x55ac3749bf20 msgr2=0x55ac3749d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 -- 172.21.15.33:0/2433209975 shutdown_connections 2024-09-10T21:15:29.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.751+0000 7fb4f15c3740 1 -- 172.21.15.33:0/2433209975 wait complete. 2024-09-10T21:15:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 -- start start 2024-09-10T21:15:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 Processor -- start 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749e0f0 0x55ac374ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374ac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ac37265700 con 0x55ac3749e0f0 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ac3739f160 con 0x55ac374aedb0 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ac374b1820 con 0x55ac3749ea10 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4f15c3740 10 client.0 _ll_register_callbacks cb 0x55ac373b89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749e0f0 0x55ac374ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4e7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749e0f0 0x55ac374ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48346/0 (socket says 172.21.15.33:48346) 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4ed78b640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374ac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.752+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/1324638995 learned_addr learned my addr 172.21.15.33:0/1324638995 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4e7fff640 1 -- 172.21.15.33:0/1324638995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 msgr2=0x55ac374ac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374ac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4e7fff640 1 -- 172.21.15.33:0/1324638995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749e0f0 msgr2=0x55ac374ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749e0f0 0x55ac374ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4e7fff640 1 -- 172.21.15.33:0/1324638995 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac373ccc90 con 0x55ac3749ea10 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749e0f0 0x55ac374ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4ed78b640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374abb10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40118d0 tx=0x7fb4d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2575f2442f4f8c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4c7fff640 1 -- 172.21.15.33:0/1324638995 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4004220 con 0x55ac3749ea10 2024-09-10T21:15:29.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4c7fff640 1 -- 172.21.15.33:0/1324638995 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb4d40043c0 con 0x55ac3749ea10 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4dc0cbfc0 con 0x55ac3749ea10 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4c7fff640 1 -- 172.21.15.33:0/1324638995 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb4d401c8a0 con 0x55ac3749ea10 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4f15c3740 10 client.26899 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.753+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ac374bd120 con 0x55ac3749ea10 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4c7fff640 1 -- 172.21.15.33:0/1324638995 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4dc0cbfc0 con 0x55ac3749ea10 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4c7fff640 1 -- 172.21.15.33:0/1324638995 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fb4d4003ce0 con 0x55ac3749ea10 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4f15c3740 10 client.26899 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4f15c3740 10 client.26899 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4f15c3740 20 client.26899 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac374be5e0 0x55ac374c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4f15c3740 4 client.26899 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac37416df0 con 0x55ac374be5e0 2024-09-10T21:15:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac374be5e0 0x55ac374c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac374be5e0 0x55ac374c0a10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=905c1881db2bfb33 server_cookie=f95899eeea659dfd in_seq=0 out_seq=0 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.754+0000 7fb4c7fff640 10 client.26899 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4c7fff640 1 -- 172.21.15.33:0/1324638995 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac37416df0 con 0x55ac374be5e0 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4c7fff640 10 client.26899 handle_command_reply: tid=0 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 client.26899 shutdown 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 msgr2=0x55ac374abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374abb10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40118d0 tx=0x7fb4d4004730 comp rx=0 tx=0).stop 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 shutdown_connections 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac374be5e0 0x55ac374c0a10 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374ac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749ea10 0x55ac374abb10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4c7fff640 0 client.26899 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:15:29.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1324638995 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749e0f0 0x55ac374ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 >> 172.21.15.33:0/1324638995 conn(0x55ac3749bf20 msgr2=0x55ac374a1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 shutdown_connections 2024-09-10T21:15:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.755+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1324638995 wait complete. 2024-09-10T21:15:29.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.756+0000 7fb4f15c3740 20 client.26899 trim_cache size 0 max 16384 2024-09-10T21:15:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 Processor -- start 2024-09-10T21:15:29.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 -- start start 2024-09-10T21:15:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749ea10 0x55ac374b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e81ac2c0 con 0x55ac374aedb0 2024-09-10T21:15:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e81ade00 con 0x55ac3749ea10 2024-09-10T21:15:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e81ac990 con 0x55ac3749e0f0 2024-09-10T21:15:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4ed78b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4e7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749ea10 0x55ac374b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374baa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:38094/0 (socket says 172.21.15.33:38094) 2024-09-10T21:15:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4ecf8a640 1 -- 172.21.15.33:0/1832822941 learned_addr learned my addr 172.21.15.33:0/1832822941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4e7fff640 1 -- 172.21.15.33:0/1832822941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 msgr2=0x55ac374baa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374baa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4e7fff640 1 -- 172.21.15.33:0/1832822941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 msgr2=0x55ac374b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.771+0000 7fb4e7fff640 1 -- 172.21.15.33:0/1832822941 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac374ac4b0 con 0x55ac3749ea10 2024-09-10T21:15:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.772+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:29.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.772+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749ea10 0x55ac374b0bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4027580 tx=0x7fb4d4004680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661f3cd5812a8736 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.772+0000 7fb4d27fc640 1 -- 172.21.15.33:0/1832822941 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400e790 con 0x55ac3749ea10 2024-09-10T21:15:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.772+0000 7fb4d27fc640 1 -- 172.21.15.33:0/1832822941 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d40276f0 con 0x55ac3749ea10 2024-09-10T21:15:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.772+0000 7fb4d27fc640 1 -- 172.21.15.33:0/1832822941 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d401d670 con 0x55ac3749ea10 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1832822941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749ea10 msgr2=0x55ac374b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749ea10 0x55ac374b0bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4027580 tx=0x7fb4d4004680 comp rx=0 tx=0).stop 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1832822941 shutdown_connections 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac374aedb0 0x55ac374b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac3749ea10 0x55ac374b0bc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/1832822941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374baa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1832822941 >> 172.21.15.33:0/1832822941 conn(0x55ac3749bf20 msgr2=0x55ac374aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1832822941 shutdown_connections 2024-09-10T21:15:29.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 -- 172.21.15.33:0/1832822941 wait complete. 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 -- start start 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.773+0000 7fb4f15c3740 1 Processor -- start 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374b6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749ea10 0x55ac374b6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4f15c3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e81ad730 con 0x55ac3749ea10 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e81ad060 con 0x55ac374aedb0 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4f15c3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4e81aa280 con 0x55ac3749e0f0 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4f15c3740 10 client.0 _ll_register_callbacks cb 0x7fb4d40099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749ea10 0x55ac374b6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4ecf8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374b6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749ea10 0x55ac374b6fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48362/0 (socket says 172.21.15.33:48362) 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3467208657 learned_addr learned my addr 172.21.15.33:0/3467208657 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4ed78b640 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3467208657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 msgr2=0x55ac374b6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374b6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3467208657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 msgr2=0x55ac374b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.774+0000 7fb4e7fff640 1 -- 172.21.15.33:0/3467208657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac374b7660 con 0x55ac3749ea10 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4e7fff640 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749ea10 0x55ac374b6fc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4027500 tx=0x7fb4d401cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a3c60c6fb30055 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4c7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4c7fff640 1 -- 172.21.15.33:0/3467208657 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d40278a0 con 0x55ac3749ea10 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4c7fff640 1 -- 172.21.15.33:0/3467208657 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb4d4027a40 con 0x55ac3749ea10 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4d40276f0 con 0x55ac3749ea10 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4c7fff640 1 -- 172.21.15.33:0/3467208657 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fb4d4027c40 con 0x55ac3749ea10 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 10 client.17511 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55ac3752b9c0 con 0x55ac3749ea10 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4c7fff640 1 -- 172.21.15.33:0/3467208657 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4d40276f0 con 0x55ac3749ea10 2024-09-10T21:15:29.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4c7fff640 1 -- 172.21.15.33:0/3467208657 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fb4d4043030 con 0x55ac3749ea10 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 10 client.17511 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 10 client.17511 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 20 client.17511 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac3752c020 0x55ac3752e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 4 client.17511 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.775+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ac374128a0 con 0x55ac3752c020 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac3752c020 0x55ac3752e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4c7fff640 10 client.17511 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:15:29.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4ecf8a640 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac3752c020 0x55ac3752e440 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b26392098485d76b server_cookie=ddba87140b0dfd5b in_seq=0 out_seq=0 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4c7fff640 1 -- 172.21.15.33:0/3467208657 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ac374128a0 con 0x55ac3752c020 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4c7fff640 10 client.17511 handle_command_reply: tid=0 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 client.17511 shutdown 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749ea10 msgr2=0x55ac374b6fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749ea10 0x55ac374b6fc0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4027500 tx=0x7fb4d401cc70 comp rx=0 tx=0).stop 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 shutdown_connections 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55ac3752c020 0x55ac3752e440 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ac374aedb0 0x55ac374b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ac3749ea10 0x55ac374b6fc0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 --2- 172.21.15.33:0/3467208657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ac3749e0f0 0x55ac374b6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 >> 172.21.15.33:0/3467208657 conn(0x55ac3749bf20 msgr2=0x55ac374b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.776+0000 7fb4c7fff640 0 client.17511 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:15:29.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.777+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 shutdown_connections 2024-09-10T21:15:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.777+0000 7fb4f15c3740 1 -- 172.21.15.33:0/3467208657 wait complete. 2024-09-10T21:15:29.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.777+0000 7fb4f15c3740 20 client.17511 trim_cache size 0 max 16384 2024-09-10T21:15:29.790 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.777+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb4d80b5360 msgr2=0x7fb4d80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.777+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb4d80b5360 0x7fb4d80b7820 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb4e81010e0 tx=0x7fb4d4005f90 comp rx=0 tx=0).stop 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.777+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8101c90 msgr2=0x7fb4e8197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.777+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8101c90 0x7fb4e8197150 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc005940 tx=0x7fb4dc004270 comp rx=0 tx=0).stop 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 shutdown_connections 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb4d80b5360 0x7fb4d80b7820 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4e8108020 0x7fb4e819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb4e81076a0 0x7fb4e8197690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 --2- 172.21.15.33:0/2405946465 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4e8101c90 0x7fb4e8197150 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 >> 172.21.15.33:0/2405946465 conn(0x7fb4e80fd700 msgr2=0x7fb4e8105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:29.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 shutdown_connections 2024-09-10T21:15:29.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:29.778+0000 7fb4ef1ee640 1 -- 172.21.15.33:0/2405946465 wait complete. 2024-09-10T21:15:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:29 smithi033 ceph-mon[30860]: pgmap v691: 97 pgs: 97 active+clean; 330 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-09-10T21:15:30.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:29 smithi104 ceph-mon[35325]: pgmap v691: 97 pgs: 97 active+clean; 330 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-09-10T21:15:30.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:29 smithi174 ceph-mon[35635]: pgmap v691: 97 pgs: 97 active+clean; 330 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 387 op/s 2024-09-10T21:15:30.508 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:15:30.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 863346 6.59 MB/sec execute 511 sec latency 10.872 ms 2024-09-10T21:15:30.771 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.387+0000 7fbb2d59a640 1 -- 172.21.15.33:0/4149913403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb200318a0 con 0x7fbb30110f30 2024-09-10T21:15:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 -- 172.21.15.33:0/4149913403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30110f30 msgr2=0x7fbb30113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 --2- 172.21.15.33:0/4149913403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30110f30 0x7fbb30113320 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009900 tx=0x7fbb2002a410 comp rx=0 tx=0).stop 2024-09-10T21:15:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 -- 172.21.15.33:0/4149913403 shutdown_connections 2024-09-10T21:15:31.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 --2- 172.21.15.33:0/4149913403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb30113860 0x7fbb30115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 --2- 172.21.15.33:0/4149913403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30110f30 0x7fbb30113320 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 --2- 172.21.15.33:0/4149913403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb30101250 0x7fbb30101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 -- 172.21.15.33:0/4149913403 >> 172.21.15.33:0/4149913403 conn(0x7fbb30078140 msgr2=0x7fbb300ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 -- 172.21.15.33:0/4149913403 shutdown_connections 2024-09-10T21:15:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.388+0000 7fbb34e87640 1 -- 172.21.15.33:0/4149913403 wait complete. 2024-09-10T21:15:31.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.389+0000 7fbb34e87640 1 Processor -- start 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.389+0000 7fbb34e87640 1 -- start start 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb34e87640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30101250 0x7fbb3019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb34e87640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb30110f30 0x7fbb3019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb34e87640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb30113860 0x7fbb301a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb34e87640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb301193b0 con 0x7fbb30101250 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb34e87640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30119230 con 0x7fbb30110f30 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb34e87640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb30119530 con 0x7fbb30113860 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30101250 0x7fbb3019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30101250 0x7fbb3019f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48372/0 (socket says 172.21.15.33:48372) 2024-09-10T21:15:31.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2911138450 learned_addr learned my addr 172.21.15.33:0/2911138450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2911138450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb30113860 msgr2=0x7fbb301a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.390+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb30113860 0x7fbb301a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2911138450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb30110f30 msgr2=0x7fbb3019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb30110f30 0x7fbb3019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2911138450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb301a5950 con 0x7fbb30101250 2024-09-10T21:15:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30101250 0x7fbb3019f8a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400e9f0 tx=0x7fbb2400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb1f7fe640 1 -- 172.21.15.33:0/2911138450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2400cd80 con 0x7fbb30101250 2024-09-10T21:15:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb1f7fe640 1 -- 172.21.15.33:0/2911138450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb240108e0 con 0x7fbb30101250 2024-09-10T21:15:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb301a5c40 con 0x7fbb30101250 2024-09-10T21:15:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.391+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb3006cd40 con 0x7fbb30101250 2024-09-10T21:15:31.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.392+0000 7fbb1f7fe640 1 -- 172.21.15.33:0/2911138450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb24018980 con 0x7fbb30101250 2024-09-10T21:15:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.398+0000 7fbb1f7fe640 1 -- 172.21.15.33:0/2911138450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbb24010430 con 0x7fbb30101250 2024-09-10T21:15:31.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.399+0000 7fbb1f7fe640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbb180b5420 0x7fbb180b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.399+0000 7fbb3725c740 1 Processor -- start 2024-09-10T21:15:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.400+0000 7fbb1f7fe640 1 -- 172.21.15.33:0/2911138450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbb240e6020 con 0x7fbb30101250 2024-09-10T21:15:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.400+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbb180b5420 0x7fbb180b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.400+0000 7fbb3725c740 1 -- start start 2024-09-10T21:15:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c668b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 0x557b02c72840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 0x557b02c75170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbb180b5420 0x7fbb180b78e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005c20 tx=0x7fbb20031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c668b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c668b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37646/0 (socket says 172.21.15.33:37646) 2024-09-10T21:15:31.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2232513876 learned_addr learned my addr 172.21.15.33:0/2232513876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b02bdcea0 con 0x557b02c66df0 2024-09-10T21:15:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557b02bcc2d0 con 0x557b02c664d0 2024-09-10T21:15:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.401+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b02bd00e0 con 0x557b02c72d80 2024-09-10T21:15:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.402+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2232513876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 msgr2=0x557b02c75170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:15:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.402+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2232513876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 0x557b02c75170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.402+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2232513876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 msgr2=0x557b02c72840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:31.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.402+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2232513876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 0x557b02c72840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.402+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2232513876 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b0292a650 con 0x557b02c664d0 2024-09-10T21:15:31.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.403+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2232513876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c668b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb240ac730 tx=0x7fbb240ac670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5c08b5010f60357 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:31.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.404+0000 7fbb1cff9640 1 -- 172.21.15.33:0/2232513876 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240acea0 con 0x557b02c664d0 2024-09-10T21:15:31.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.404+0000 7fbb1cff9640 1 -- 172.21.15.33:0/2232513876 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb240cf070 con 0x557b02c664d0 2024-09-10T21:15:31.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.404+0000 7fbb1cff9640 1 -- 172.21.15.33:0/2232513876 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240a80f0 con 0x557b02c664d0 2024-09-10T21:15:31.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.405+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 msgr2=0x557b02c668b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.406+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2232513876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c668b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbb240ac730 tx=0x7fbb240ac670 comp rx=0 tx=0).stop 2024-09-10T21:15:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.407+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 shutdown_connections 2024-09-10T21:15:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.407+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2232513876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 0x557b02c75170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.407+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2232513876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 0x557b02c72840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.407+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2232513876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c668b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.407+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 >> 172.21.15.33:0/2232513876 conn(0x557b02c64300 msgr2=0x557b02c654a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.410+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 shutdown_connections 2024-09-10T21:15:31.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.410+0000 7fbb3725c740 1 -- 172.21.15.33:0/2232513876 wait complete. 2024-09-10T21:15:31.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.411+0000 7fbb3725c740 1 -- start start 2024-09-10T21:15:31.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.411+0000 7fbb3725c740 1 Processor -- start 2024-09-10T21:15:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.412+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c664d0 0x557b02c72690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.412+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 0x557b02c6fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.412+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c664d0 0x557b02c72690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.412+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c664d0 0x557b02c72690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:38134/0 (socket says 172.21.15.33:38134) 2024-09-10T21:15:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.412+0000 7fbb2e59c640 1 -- 172.21.15.33:0/147428648 learned_addr learned my addr 172.21.15.33:0/147428648 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.412+0000 7fbb3725c740 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c72d80 0x557b02c70110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.413+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b02a2d700 con 0x557b02c66df0 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.413+0000 7fbb2ed9d640 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c72d80 0x557b02c70110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.413+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557b02b67160 con 0x557b02c72d80 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.413+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b02c759a0 con 0x557b02c664d0 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.413+0000 7fbb3725c740 10 client.0 _ll_register_callbacks cb 0x557b02b809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.414+0000 7fbb2ed9d640 1 -- 172.21.15.33:0/147428648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c664d0 msgr2=0x557b02c72690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.414+0000 7fbb2ed9d640 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c664d0 0x557b02c72690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.414+0000 7fbb2ed9d640 1 -- 172.21.15.33:0/147428648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 msgr2=0x557b02c6fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.414+0000 7fbb2ed9d640 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 0x557b02c6fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.414+0000 7fbb2ed9d640 1 -- 172.21.15.33:0/147428648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b02b94c90 con 0x557b02c72d80 2024-09-10T21:15:31.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.414+0000 7fbb2ed9d640 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c72d80 0x557b02c70110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e6a0 tx=0x7fbb2800eb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e010f39d71937a5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.415+0000 7fbb15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:15:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.415+0000 7fbb15ffb640 1 -- 172.21.15.33:0/147428648 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28004d20 con 0x557b02c72d80 2024-09-10T21:15:31.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.415+0000 7fbb15ffb640 1 -- 172.21.15.33:0/147428648 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbb2800c830 con 0x557b02c72d80 2024-09-10T21:15:31.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.415+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb240cf070 con 0x557b02c72d80 2024-09-10T21:15:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.416+0000 7fbb15ffb640 1 -- 172.21.15.33:0/147428648 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbb2800ca30 con 0x557b02c72d80 2024-09-10T21:15:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.416+0000 7fbb3725c740 10 client.26963 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:31.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.416+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557b02c813a0 con 0x557b02c72d80 2024-09-10T21:15:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.418+0000 7fbb15ffb640 1 -- 172.21.15.33:0/147428648 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb2800cbd0 con 0x557b02c72d80 2024-09-10T21:15:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.418+0000 7fbb15ffb640 1 -- 172.21.15.33:0/147428648 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fbb28018430 con 0x557b02c72d80 2024-09-10T21:15:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.418+0000 7fbb3725c740 10 client.26963 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.418+0000 7fbb3725c740 10 client.26963 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:15:31.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.418+0000 7fbb3725c740 20 client.26963 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:31.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.418+0000 7fbb3725c740 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02c828a0 0x557b02c84ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.419+0000 7fbb3725c740 4 client.26963 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:31.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.419+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fbb240cf070 con 0x557b02c828a0 2024-09-10T21:15:31.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.419+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02c828a0 0x557b02c84ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.419+0000 7fbb15ffb640 10 client.26963 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:15:31.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.419+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02c828a0 0x557b02c84ce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48e3f15736649983 server_cookie=4e32153d3b59b170 in_seq=0 out_seq=0 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb15ffb640 1 -- 172.21.15.33:0/147428648 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fbb240cf070 con 0x557b02c828a0 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb15ffb640 10 client.26963 handle_command_reply: tid=0 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 client.26963 shutdown 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c72d80 msgr2=0x557b02c70110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c72d80 0x557b02c70110 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e6a0 tx=0x7fbb2800eb60 comp rx=0 tx=0).stop 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 shutdown_connections 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c72d80 0x557b02c70110 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c66df0 0x557b02c6fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02c828a0 0x557b02c84ce0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 --2- 172.21.15.33:0/147428648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c664d0 0x557b02c72690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 >> 172.21.15.33:0/147428648 conn(0x557b02c64300 msgr2=0x557b02c69e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:31.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.422+0000 7fbb15ffb640 0 client.26963 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:15:31.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.423+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 shutdown_connections 2024-09-10T21:15:31.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.423+0000 7fbb3725c740 1 -- 172.21.15.33:0/147428648 wait complete. 2024-09-10T21:15:31.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.423+0000 7fbb3725c740 20 client.26963 trim_cache size 0 max 16384 2024-09-10T21:15:31.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 Processor -- start 2024-09-10T21:15:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 -- start start 2024-09-10T21:15:31.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c63c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c66df0 0x557b02c74170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c72d80 0x557b02c746b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c63c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb2ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c72d80 0x557b02c746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c63c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37666/0 (socket says 172.21.15.33:37666) 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb2ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c72d80 0x557b02c746b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48404/0 (socket says 172.21.15.33:48404) 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb2e59c640 1 -- 172.21.15.33:0/681209352 learned_addr learned my addr 172.21.15.33:0/681209352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1000e590 con 0x557b02c72d80 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb100100d0 con 0x557b02c664d0 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.445+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1000ec60 con 0x557b02c66df0 2024-09-10T21:15:31.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.446+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c66df0 0x557b02c74170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.446+0000 7fbb2dd9b640 1 -- 172.21.15.33:0/681209352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 msgr2=0x557b02c63c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.446+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c63c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.446+0000 7fbb2dd9b640 1 -- 172.21.15.33:0/681209352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c72d80 msgr2=0x557b02c746b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.446+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c72d80 0x557b02c746b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.446+0000 7fbb2dd9b640 1 -- 172.21.15.33:0/681209352 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b02c66c20 con 0x557b02c66df0 2024-09-10T21:15:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.447+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c66df0 0x557b02c74170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009230 tx=0x7fbb20009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9040b2064dcf28e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.447+0000 7fbb17fff640 1 -- 172.21.15.33:0/681209352 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2004c070 con 0x557b02c66df0 2024-09-10T21:15:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.447+0000 7fbb17fff640 1 -- 172.21.15.33:0/681209352 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb20003d60 con 0x557b02c66df0 2024-09-10T21:15:31.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.447+0000 7fbb17fff640 1 -- 172.21.15.33:0/681209352 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20043040 con 0x557b02c66df0 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c66df0 msgr2=0x557b02c74170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c66df0 0x557b02c74170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbb20009230 tx=0x7fbb20009260 comp rx=0 tx=0).stop 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 shutdown_connections 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c72d80 0x557b02c746b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c66df0 0x557b02c74170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 --2- 172.21.15.33:0/681209352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c664d0 0x557b02c63c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 >> 172.21.15.33:0/681209352 conn(0x557b02c64300 msgr2=0x557b02c6fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 shutdown_connections 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 -- 172.21.15.33:0/681209352 wait complete. 2024-09-10T21:15:31.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.448+0000 7fbb3725c740 1 -- start start 2024-09-10T21:15:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 1 Processor -- start 2024-09-10T21:15:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c664d0 0x557b02c7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c66df0 0x557b02c7b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 0x557b02c75c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1000fa00 con 0x557b02c664d0 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1000f330 con 0x557b02c66df0 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1000c550 con 0x557b02c72d80 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb3725c740 10 client.0 _ll_register_callbacks cb 0x7fbb30078550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c664d0 0x557b02c7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb2e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c664d0 0x557b02c7aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48418/0 (socket says 172.21.15.33:48418) 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2166957061 learned_addr learned my addr 172.21.15.33:0/2166957061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c66df0 0x557b02c7b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.449+0000 7fbb2ed9d640 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 0x557b02c75c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 msgr2=0x557b02c75c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 0x557b02c75c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2166957061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c66df0 msgr2=0x557b02c7b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c66df0 0x557b02c7b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb2e59c640 1 -- 172.21.15.33:0/2166957061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b02c7ba10 con 0x557b02c664d0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb2e59c640 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c664d0 0x557b02c7aca0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbb240ac730 tx=0x7fbb240a8c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a61b927a2346f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb15ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb15ffb640 1 -- 172.21.15.33:0/2166957061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb240ac7a0 con 0x557b02c664d0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb20003d60 con 0x557b02c664d0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb15ffb640 1 -- 172.21.15.33:0/2166957061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbb24018b20 con 0x557b02c664d0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb15ffb640 1 -- 172.21.15.33:0/2166957061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fbb24018d20 con 0x557b02c664d0 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb3725c740 10 client.17523 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:31.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.450+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x557b02cefd70 con 0x557b02c664d0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb15ffb640 1 -- 172.21.15.33:0/2166957061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb20003d60 con 0x557b02c664d0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb15ffb640 1 -- 172.21.15.33:0/2166957061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fbb240ab470 con 0x557b02c664d0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb3725c740 10 client.17523 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb3725c740 10 client.17523 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb3725c740 20 client.17523 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02cf00b0 0x557b02cf24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb3725c740 4 client.17523 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b02bdedf0 con 0x557b02cf00b0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.451+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02cf00b0 0x557b02cf24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb15ffb640 10 client.17523 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb2dd9b640 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02cf00b0 0x557b02cf24a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=67529689a0e99279 server_cookie=f9d91b8151d90a5a in_seq=0 out_seq=0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb15ffb640 1 -- 172.21.15.33:0/2166957061 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b02bdedf0 con 0x557b02cf00b0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb15ffb640 10 client.17523 handle_command_reply: tid=0 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 client.17523 shutdown 2024-09-10T21:15:31.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c664d0 msgr2=0x557b02c7aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c664d0 0x557b02c7aca0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbb240ac730 tx=0x7fbb240a8c40 comp rx=0 tx=0).stop 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 shutdown_connections 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x557b02cf00b0 0x557b02cf24a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b02c72d80 0x557b02c75c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb15ffb640 0 client.17523 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b02c66df0 0x557b02c7b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 --2- 172.21.15.33:0/2166957061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b02c664d0 0x557b02c7aca0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 >> 172.21.15.33:0/2166957061 conn(0x557b02c64300 msgr2=0x557b02c71110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 shutdown_connections 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.452+0000 7fbb3725c740 1 -- 172.21.15.33:0/2166957061 wait complete. 2024-09-10T21:15:31.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb3725c740 20 client.17523 trim_cache size 0 max 16384 2024-09-10T21:15:31.461 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:15:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbb180b5420 msgr2=0x7fbb180b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbb180b5420 0x7fbb180b78e0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fbb20005c20 tx=0x7fbb20031400 comp rx=0 tx=0).stop 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30101250 msgr2=0x7fbb3019f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30101250 0x7fbb3019f8a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400e9f0 tx=0x7fbb2400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 shutdown_connections 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbb180b5420 0x7fbb180b78e0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb30113860 0x7fbb301a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb30110f30 0x7fbb3019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 --2- 172.21.15.33:0/2911138450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb30101250 0x7fbb3019f8a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.453+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 >> 172.21.15.33:0/2911138450 conn(0x7fbb30078140 msgr2=0x7fbb301140b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.454+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 shutdown_connections 2024-09-10T21:15:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:31.454+0000 7fbb34e87640 1 -- 172.21.15.33:0/2911138450 wait complete. 2024-09-10T21:15:31.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 865660 6.60 MB/sec execute 512 sec latency 40.774 ms 2024-09-10T21:15:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:31 smithi104 ceph-mon[35325]: pgmap v692: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T21:15:32.114 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:31 smithi033 ceph-mon[30860]: pgmap v692: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T21:15:32.142 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:15:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:31 smithi174 ceph-mon[35635]: pgmap v692: 97 pgs: 97 active+clean; 330 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.2 MiB/s wr, 319 op/s 2024-09-10T21:15:32.415 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:32.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 867195 6.60 MB/sec execute 513 sec latency 21.202 ms 2024-09-10T21:15:32.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.904+0000 7f87ece2b640 1 -- 172.21.15.33:0/2907052102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 msgr2=0x7f87e81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.904+0000 7f87ece2b640 1 --2- 172.21.15.33:0/2907052102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e81020b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009900 tx=0x7f87d802a410 comp rx=0 tx=0).stop 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 -- 172.21.15.33:0/2907052102 shutdown_connections 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 --2- 172.21.15.33:0/2907052102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8108060 0x7f87e810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 --2- 172.21.15.33:0/2907052102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87e81076e0 0x7f87e8107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 --2- 172.21.15.33:0/2907052102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e81020b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 -- 172.21.15.33:0/2907052102 >> 172.21.15.33:0/2907052102 conn(0x7f87e80fd700 msgr2=0x7f87e80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 -- 172.21.15.33:0/2907052102 shutdown_connections 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 -- 172.21.15.33:0/2907052102 wait complete. 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.905+0000 7f87ece2b640 1 Processor -- start 2024-09-10T21:15:32.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87ece2b640 1 -- start start 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87ece2b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87ece2b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87e81076e0 0x7f87e819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87ece2b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8108060 0x7f87e819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87ece2b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111d30 con 0x7f87e8101cd0 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87ece2b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111bb0 con 0x7f87e8108060 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87ece2b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87e8111eb0 con 0x7f87e81076e0 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48434/0 (socket says 172.21.15.33:48434) 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 -- 172.21.15.33:0/3989595240 learned_addr learned my addr 172.21.15.33:0/3989595240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:32.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87e81076e0 0x7f87e819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8108060 0x7f87e819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 -- 172.21.15.33:0/3989595240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87e81076e0 msgr2=0x7f87e819a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87e81076e0 0x7f87e819a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 -- 172.21.15.33:0/3989595240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8108060 msgr2=0x7f87e819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8108060 0x7f87e819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e659c640 1 -- 172.21.15.33:0/3989595240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e81a54a0 con 0x7f87e8101cd0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.906+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8108060 0x7f87e819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.907+0000 7f87e659c640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e819a0c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005ab0 tx=0x7f87d8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.907+0000 7f87d77fe640 1 -- 172.21.15.33:0/3989595240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d803f070 con 0x7f87e8101cd0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.907+0000 7f87d77fe640 1 -- 172.21.15.33:0/3989595240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87d8031d60 con 0x7f87e8101cd0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.907+0000 7f87d77fe640 1 -- 172.21.15.33:0/3989595240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d8035030 con 0x7f87e8101cd0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.907+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e81a5730 con 0x7f87e8101cd0 2024-09-10T21:15:32.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.907+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e81a5bc0 con 0x7f87e8101cd0 2024-09-10T21:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 Processor -- start 2024-09-10T21:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 -- start start 2024-09-10T21:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87d77fe640 1 -- 172.21.15.33:0/3989595240 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f87d804c060 con 0x7f87e8101cd0 2024-09-10T21:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48984d0 0x55d4e48988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48a4d80 0x55d4e48a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d4e480eea0 con 0x55d4e48984d0 2024-09-10T21:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d4e47fe2d0 con 0x55d4e48a4d80 2024-09-10T21:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87ef200740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4e48020e0 con 0x55d4e4898df0 2024-09-10T21:15:32.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.910+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48984d0 0x55d4e48988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48984d0 0x55d4e48988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48448/0 (socket says 172.21.15.33:48448) 2024-09-10T21:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 -- 172.21.15.33:0/3837377825 learned_addr learned my addr 172.21.15.33:0/3837377825 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:32.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48a4d80 0x55d4e48a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 -- 172.21.15.33:0/3837377825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 msgr2=0x55d4e48a4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 -- 172.21.15.33:0/3837377825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48a4d80 msgr2=0x55d4e48a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48a4d80 0x55d4e48a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 -- 172.21.15.33:0/3837377825 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e455c650 con 0x55d4e48984d0 2024-09-10T21:15:32.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48a4d80 0x55d4e48a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:32.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48984d0 0x55d4e48988b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f87e0011760 tx=0x7f87e0011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5282e184533d5909 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:32.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87d4ff9640 1 -- 172.21.15.33:0/3837377825 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0004d20 con 0x55d4e48984d0 2024-09-10T21:15:32.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87d4ff9640 1 -- 172.21.15.33:0/3837377825 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e0005020 con 0x55d4e48984d0 2024-09-10T21:15:32.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.911+0000 7f87d4ff9640 1 -- 172.21.15.33:0/3837377825 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e0019630 con 0x55d4e48984d0 2024-09-10T21:15:32.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87d77fe640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87d00b52e0 0x7f87d00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87e659c640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87d00b52e0 0x7f87d00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87d77fe640 1 -- 172.21.15.33:0/3989595240 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87d8105fa0 con 0x7f87e8101cd0 2024-09-10T21:15:32.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87ef200740 1 -- 172.21.15.33:0/3837377825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48984d0 msgr2=0x55d4e48988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87ef200740 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48984d0 0x55d4e48988b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f87e0011760 tx=0x7f87e0011c20 comp rx=0 tx=0).stop 2024-09-10T21:15:32.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87ef200740 1 -- 172.21.15.33:0/3837377825 shutdown_connections 2024-09-10T21:15:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87ef200740 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48a4d80 0x55d4e48a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87ef200740 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87ef200740 1 --2- 172.21.15.33:0/3837377825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48984d0 0x55d4e48988b0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.912+0000 7f87ef200740 1 -- 172.21.15.33:0/3837377825 >> 172.21.15.33:0/3837377825 conn(0x55d4e4896300 msgr2=0x55d4e48974a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87e659c640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87d00b52e0 0x7f87d00b77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cc7a0 tx=0x7f87d8109a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:32.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 -- 172.21.15.33:0/3837377825 shutdown_connections 2024-09-10T21:15:32.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 -- 172.21.15.33:0/3837377825 wait complete. 2024-09-10T21:15:32.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 -- start start 2024-09-10T21:15:32.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 Processor -- start 2024-09-10T21:15:32.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e48a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e48a2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d4e465f700 con 0x55d4e48a4d80 2024-09-10T21:15:32.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d4e4799160 con 0x55d4e48984d0 2024-09-10T21:15:32.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.913+0000 7f87ef200740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d4e48abe10 con 0x55d4e4898df0 2024-09-10T21:15:32.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87ef200740 10 client.0 _ll_register_callbacks cb 0x55d4e47b29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:32.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e48a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e48a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e5d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e48a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37698/0 (socket says 172.21.15.33:37698) 2024-09-10T21:15:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e6d9d640 1 -- 172.21.15.33:0/3813795594 learned_addr learned my addr 172.21.15.33:0/3813795594 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e5d9b640 1 -- 172.21.15.33:0/3813795594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 msgr2=0x55d4e48a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e48a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e5d9b640 1 -- 172.21.15.33:0/3813795594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 msgr2=0x55d4e48a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e48a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e5d9b640 1 -- 172.21.15.33:0/3813795594 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e47c6c90 con 0x55d4e4898df0 2024-09-10T21:15:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e48a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a1a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87dc005d00 tx=0x7f87dc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e06fa5b502153dac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:15:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87cdffb640 1 -- 172.21.15.33:0/3813795594 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc004020 con 0x55d4e4898df0 2024-09-10T21:15:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87cdffb640 1 -- 172.21.15.33:0/3813795594 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87dc0041c0 con 0x55d4e4898df0 2024-09-10T21:15:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.914+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87e0005020 con 0x55d4e4898df0 2024-09-10T21:15:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87cdffb640 1 -- 172.21.15.33:0/3813795594 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f87dc0043c0 con 0x55d4e4898df0 2024-09-10T21:15:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 10 client.26929 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d4e48b7710 con 0x55d4e4898df0 2024-09-10T21:15:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87cdffb640 1 -- 172.21.15.33:0/3813795594 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87e0005020 con 0x55d4e4898df0 2024-09-10T21:15:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87cdffb640 1 -- 172.21.15.33:0/3813795594 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f87dc023020 con 0x55d4e4898df0 2024-09-10T21:15:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 10 client.26929 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 10 client.26929 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:15:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 20 client.26929 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e48b8be0 0x55d4e48bafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 4 client.26929 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e4810df0 con 0x55d4e48b8be0 2024-09-10T21:15:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.915+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e48b8be0 0x55d4e48bafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.916+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e48b8be0 0x55d4e48bafd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9960ea24edd70691 server_cookie=88727546031b0fbd in_seq=0 out_seq=0 2024-09-10T21:15:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.916+0000 7f87cdffb640 10 client.26929 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:15:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.916+0000 7f87cdffb640 1 -- 172.21.15.33:0/3813795594 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4e4810df0 con 0x55d4e48b8be0 2024-09-10T21:15:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.916+0000 7f87cdffb640 10 client.26929 handle_command_reply: tid=0 2024-09-10T21:15:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.916+0000 7f87ef200740 1 client.26929 shutdown 2024-09-10T21:15:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.916+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 msgr2=0x55d4e48a1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.916+0000 7f87ef200740 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a1a90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f87dc005d00 tx=0x7f87dc0118d0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 shutdown_connections 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e48b8be0 0x55d4e48bafd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e48a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e4898df0 0x55d4e48a1a90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 --2- 172.21.15.33:0/3813795594 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e48a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87cdffb640 0 client.26929 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 >> 172.21.15.33:0/3813795594 conn(0x55d4e4896300 msgr2=0x55d4e489be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 shutdown_connections 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 1 -- 172.21.15.33:0/3813795594 wait complete. 2024-09-10T21:15:32.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.917+0000 7f87ef200740 20 client.26929 trim_cache size 0 max 16384 2024-09-10T21:15:32.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 Processor -- start 2024-09-10T21:15:32.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 -- start start 2024-09-10T21:15:32.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48984d0 0x55d4e48b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e4898df0 0x55d4e4897350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e4897890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81ac240 con 0x55d4e48a4d80 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81add80 con 0x55d4e4898df0 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87ef200740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81ac910 con 0x55d4e48984d0 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e4898df0 0x55d4e4897350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e4897890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e4898df0 0x55d4e4897350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37714/0 (socket says 172.21.15.33:37714) 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e4897890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48462/0 (socket says 172.21.15.33:48462) 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 -- 172.21.15.33:0/1157523722 learned_addr learned my addr 172.21.15.33:0/1157523722 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48984d0 0x55d4e48b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 -- 172.21.15.33:0/1157523722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48984d0 msgr2=0x55d4e48b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48984d0 0x55d4e48b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 -- 172.21.15.33:0/1157523722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 msgr2=0x55d4e4897890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e4897890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e5d9b640 1 -- 172.21.15.33:0/1157523722 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e4896fe0 con 0x55d4e4898df0 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.931+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e4897890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e4898df0 0x55d4e4897350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f87dc004e80 tx=0x7f87dc011dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ddbbccd53296b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87cffff640 1 -- 172.21.15.33:0/1157523722 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc00b480 con 0x55d4e4898df0 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87cffff640 1 -- 172.21.15.33:0/1157523722 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87dc0107c0 con 0x55d4e4898df0 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87cffff640 1 -- 172.21.15.33:0/1157523722 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc005160 con 0x55d4e4898df0 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87ef200740 1 -- 172.21.15.33:0/1157523722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e4898df0 msgr2=0x55d4e4897350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87ef200740 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e4898df0 0x55d4e4897350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f87dc004e80 tx=0x7f87dc011dd0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87ef200740 1 -- 172.21.15.33:0/1157523722 shutdown_connections 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87ef200740 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e48a4d80 0x55d4e4897890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87ef200740 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e4898df0 0x55d4e4897350 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87ef200740 1 --2- 172.21.15.33:0/1157523722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48984d0 0x55d4e48b5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.932+0000 7f87ef200740 1 -- 172.21.15.33:0/1157523722 >> 172.21.15.33:0/1157523722 conn(0x55d4e4896300 msgr2=0x55d4e48a1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 -- 172.21.15.33:0/1157523722 shutdown_connections 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 -- 172.21.15.33:0/1157523722 wait complete. 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 -- start start 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 Processor -- start 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e489f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e4898df0 0x55d4e48b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48a4d80 0x55d4e48b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81ad6b0 con 0x55d4e4898df0 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81acfe0 con 0x55d4e48984d0 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87e81aa200 con 0x55d4e48a4d80 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87ef200740 10 client.0 _ll_register_callbacks cb 0x7f87dc00bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:32.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e489f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87e5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e4898df0 0x55d4e48b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87e6d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e489f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37730/0 (socket says 172.21.15.33:37730) 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87e5d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e4898df0 0x55d4e48b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48470/0 (socket says 172.21.15.33:48470) 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87e6d9d640 1 -- 172.21.15.33:0/856831862 learned_addr learned my addr 172.21.15.33:0/856831862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.933+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48a4d80 0x55d4e48b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87e6d9d640 1 -- 172.21.15.33:0/856831862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48a4d80 msgr2=0x55d4e48b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48a4d80 0x55d4e48b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87e6d9d640 1 -- 172.21.15.33:0/856831862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e4898df0 msgr2=0x55d4e48b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e4898df0 0x55d4e48b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87e6d9d640 1 -- 172.21.15.33:0/856831862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e48c0730 con 0x55d4e48984d0 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e4898df0 0x55d4e48b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87e6d9d640 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e489f6c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87e0005fd0 tx=0x7f87e002ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44bc823955221c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:15:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87cdffb640 1 -- 172.21.15.33:0/856831862 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e000fc70 con 0x55d4e48984d0 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87cdffb640 1 -- 172.21.15.33:0/856831862 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87e000fe10 con 0x55d4e48984d0 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f87dc0107c0 con 0x55d4e48984d0 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87cdffb640 1 -- 172.21.15.33:0/856831862 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f87e0003970 con 0x55d4e48984d0 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87ef200740 10 client.26987 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55d4e4925da0 con 0x55d4e48984d0 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.934+0000 7f87cdffb640 1 -- 172.21.15.33:0/856831862 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87dc0107c0 con 0x55d4e48984d0 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87cdffb640 1 -- 172.21.15.33:0/856831862 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f87e0027470 con 0x55d4e48984d0 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87ef200740 10 client.26987 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87ef200740 10 client.26987 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87ef200740 20 client.26987 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87ef200740 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e4926270 0x55d4e4928690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87ef200740 4 client.26987 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:15:32.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d4e480c8a0 con 0x55d4e4926270 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e4926270 0x55d4e4928690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87e5d9b640 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e4926270 0x55d4e4928690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fd3244ffe9f3a7c server_cookie=5e76f1272bca536a in_seq=0 out_seq=0 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87cdffb640 10 client.26987 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87cdffb640 1 -- 172.21.15.33:0/856831862 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d4e480c8a0 con 0x55d4e4926270 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87cdffb640 10 client.26987 handle_command_reply: tid=0 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.935+0000 7f87ef200740 1 client.26987 shutdown 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 msgr2=0x55d4e489f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e489f6c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f87e0005fd0 tx=0x7f87e002ff90 comp rx=0 tx=0).stop 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 shutdown_connections 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d4e48a4d80 0x55d4e48b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55d4e4926270 0x55d4e4928690 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d4e4898df0 0x55d4e48b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 --2- 172.21.15.33:0/856831862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d4e48984d0 0x55d4e489f6c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87cdffb640 0 client.26987 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:15:32.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 >> 172.21.15.33:0/856831862 conn(0x55d4e4896300 msgr2=0x55d4e4897350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:32.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 shutdown_connections 2024-09-10T21:15:32.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 1 -- 172.21.15.33:0/856831862 wait complete. 2024-09-10T21:15:32.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.936+0000 7f87ef200740 20 client.26987 trim_cache size 0 max 16384 2024-09-10T21:15:32.953 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87d00b52e0 msgr2=0x7f87d00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87d00b52e0 0x7f87d00b77a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f87d80cc7a0 tx=0x7f87d8109a20 comp rx=0 tx=0).stop 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 msgr2=0x7f87e819a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e819a0c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f87d8005ab0 tx=0x7f87d8003c80 comp rx=0 tx=0).stop 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 shutdown_connections 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87d00b52e0 0x7f87d00b77a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87e8108060 0x7f87e819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87e81076e0 0x7f87e819a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 --2- 172.21.15.33:0/3989595240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87e8101cd0 0x7f87e819a0c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:32.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 >> 172.21.15.33:0/3989595240 conn(0x7f87e80fd700 msgr2=0x7f87e80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:32.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 shutdown_connections 2024-09-10T21:15:32.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:32.937+0000 7f87ece2b640 1 -- 172.21.15.33:0/3989595240 wait complete. 2024-09-10T21:15:33.665 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:15:33.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 869925 6.62 MB/sec execute 514 sec latency 12.391 ms 2024-09-10T21:15:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:33 smithi033 ceph-mon[30860]: pgmap v693: 97 pgs: 97 active+clean; 332 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2024-09-10T21:15:33.935 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:34.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:33 smithi104 ceph-mon[35325]: pgmap v693: 97 pgs: 97 active+clean; 332 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2024-09-10T21:15:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:33 smithi174 ceph-mon[35635]: pgmap v693: 97 pgs: 97 active+clean; 332 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.9 MiB/s wr, 336 op/s 2024-09-10T21:15:34.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 871494 6.62 MB/sec execute 515 sec latency 16.488 ms 2024-09-10T21:15:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 -- 172.21.15.33:0/3764499002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c10d0e0 msgr2=0x7f256c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 --2- 172.21.15.33:0/3764499002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c10d0e0 0x7f256c10f550 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2564009900 tx=0x7f256402a410 comp rx=0 tx=0).stop 2024-09-10T21:15:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 -- 172.21.15.33:0/3764499002 shutdown_connections 2024-09-10T21:15:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 --2- 172.21.15.33:0/3764499002 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c10d0e0 0x7f256c10f550 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 --2- 172.21.15.33:0/3764499002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f256c104530 0x7f256c10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 --2- 172.21.15.33:0/3764499002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 0x7f256c103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 -- 172.21.15.33:0/3764499002 >> 172.21.15.33:0/3764499002 conn(0x7f256c0fd700 msgr2=0x7f256c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 -- 172.21.15.33:0/3764499002 shutdown_connections 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 -- 172.21.15.33:0/3764499002 wait complete. 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.439+0000 7f2571490640 1 Processor -- start 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f2571490640 1 -- start start 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f2571490640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 0x7f256c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f2571490640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c104530 0x7f256c199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f2571490640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f256c10d0e0 0x7f256c19eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f2571490640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f256c112d60 con 0x7f256c104530 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f2571490640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f256c112be0 con 0x7f256c10d0e0 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f2571490640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f256c112ee0 con 0x7f256c103b30 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c104530 0x7f256c199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 0x7f256c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 0x7f256c199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:38214/0 (socket says 172.21.15.33:38214) 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f256c10d0e0 0x7f256c19eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 -- 172.21.15.33:0/519319901 learned_addr learned my addr 172.21.15.33:0/519319901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 -- 172.21.15.33:0/519319901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f256c10d0e0 msgr2=0x7f256c19eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f256c10d0e0 0x7f256c19eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 -- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c104530 msgr2=0x7f256c199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c104530 0x7f256c199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256affd640 1 -- 172.21.15.33:0/519319901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256c19f2c0 con 0x7f256c103b30 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256a7fc640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c104530 0x7f256c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.440+0000 7f256b7fe640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f256c10d0e0 0x7f256c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.441+0000 7f256affd640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 0x7f256c199210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f255c00eb90 tx=0x7f255c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.441+0000 7f2553fff640 1 -- 172.21.15.33:0/519319901 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c004d20 con 0x7f256c103b30 2024-09-10T21:15:34.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.441+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256c19f5b0 con 0x7f256c103b30 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.441+0000 7f2553fff640 1 -- 172.21.15.33:0/519319901 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f255c00ee90 con 0x7f256c103b30 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.441+0000 7f2553fff640 1 -- 172.21.15.33:0/519319901 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255c010670 con 0x7f256c103b30 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.441+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f256c1a5c70 con 0x7f256c103b30 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.443+0000 7f2573865740 1 Processor -- start 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2573865740 1 -- start start 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2553fff640 1 -- 172.21.15.33:0/519319901 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f255c005020 con 0x7f256c103b30 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2573865740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf3104d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2573865740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf31c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2573865740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2573865740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638cf286ea0 con 0x5638cf3100f0 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2573865740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5638cf2762d0 con 0x5638cf310a10 2024-09-10T21:15:34.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f2573865740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638cf27a0e0 con 0x5638cf31c9a0 2024-09-10T21:15:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf3104d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf3104d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48488/0 (socket says 172.21.15.33:48488) 2024-09-10T21:15:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 -- 172.21.15.33:0/3163964831 learned_addr learned my addr 172.21.15.33:0/3163964831 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256affd640 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf31c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256a7fc640 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 -- 172.21.15.33:0/3163964831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 msgr2=0x5638cf31ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 -- 172.21.15.33:0/3163964831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 msgr2=0x5638cf31c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf31c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.444+0000 7f256b7fe640 1 -- 172.21.15.33:0/3163964831 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cefd4650 con 0x5638cf3100f0 2024-09-10T21:15:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.445+0000 7f256b7fe640 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf3104d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2564009900 tx=0x7f2564005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4379794baa2ffe22 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.445+0000 7f2553fff640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f25540b52e0 0x7f25540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.445+0000 7f25517fa640 1 -- 172.21.15.33:0/3163964831 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564040070 con 0x5638cf3100f0 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.445+0000 7f25517fa640 1 -- 172.21.15.33:0/3163964831 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25640317a0 con 0x5638cf3100f0 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.445+0000 7f25517fa640 1 -- 172.21.15.33:0/3163964831 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564035040 con 0x5638cf3100f0 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.445+0000 7f256affd640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f25540b52e0 0x7f25540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.445+0000 7f2553fff640 1 -- 172.21.15.33:0/519319901 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f255c0e61f0 con 0x7f256c103b30 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f256affd640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f25540b52e0 0x7f25540b77a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f255c0a9c20 tx=0x7f255c0e8040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 -- 172.21.15.33:0/3163964831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 msgr2=0x5638cf3104d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf3104d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2564009900 tx=0x7f2564005df0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 -- 172.21.15.33:0/3163964831 shutdown_connections 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf31c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 --2- 172.21.15.33:0/3163964831 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf3104d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 -- 172.21.15.33:0/3163964831 >> 172.21.15.33:0/3163964831 conn(0x5638cf30df20 msgr2=0x5638cf30f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 -- 172.21.15.33:0/3163964831 shutdown_connections 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 -- 172.21.15.33:0/3163964831 wait complete. 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.446+0000 7f2573865740 1 -- start start 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 1 Processor -- start 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf31c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf319710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf319d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5638cf0d7700 con 0x5638cf310a10 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5638cf211160 con 0x5638cf31c9a0 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638cf31f5c0 con 0x5638cf3100f0 2024-09-10T21:15:34.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f2573865740 10 client.0 _ll_register_callbacks cb 0x5638cf22a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf319710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf319710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48498/0 (socket says 172.21.15.33:48498) 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256a7fc640 1 -- 172.21.15.33:0/2641083436 learned_addr learned my addr 172.21.15.33:0/2641083436 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256b7fe640 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf31c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256b7fe640 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf319d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256a7fc640 1 -- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 msgr2=0x5638cf31c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256a7fc640 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf31c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256a7fc640 1 -- 172.21.15.33:0/2641083436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 msgr2=0x5638cf319d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.447+0000 7f256a7fc640 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf319d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f256a7fc640 1 -- 172.21.15.33:0/2641083436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cf23ec90 con 0x5638cf310a10 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f256a7fc640 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf319710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2560012d20 tx=0x7f256000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db241857d0e782b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f254eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:34.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f254eff5640 1 -- 172.21.15.33:0/2641083436 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560012110 con 0x5638cf310a10 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f254eff5640 1 -- 172.21.15.33:0/2641083436 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2560004050 con 0x5638cf310a10 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25640317a0 con 0x5638cf310a10 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f254eff5640 1 -- 172.21.15.33:0/2641083436 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f25600122b0 con 0x5638cf310a10 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f2573865740 10 client.17553 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5638cf32af80 con 0x5638cf310a10 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.448+0000 7f254eff5640 1 -- 172.21.15.33:0/2641083436 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25640317a0 con 0x5638cf310a10 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f254eff5640 1 -- 172.21.15.33:0/2641083436 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f2560010430 con 0x5638cf310a10 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f2573865740 10 client.17553 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f2573865740 10 client.17553 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f2573865740 20 client.17553 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f2573865740 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf32c490 0x5638cf32e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f2573865740 4 client.17553 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638cf288df0 con 0x5638cf32c490 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f256b7fe640 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf32c490 0x5638cf32e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f256b7fe640 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf32c490 0x5638cf32e8b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d17648c893730acd server_cookie=138a52a56e46ea09 in_seq=0 out_seq=0 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.449+0000 7f254eff5640 10 client.17553 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f254eff5640 1 -- 172.21.15.33:0/2641083436 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638cf288df0 con 0x5638cf32c490 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f254eff5640 10 client.17553 handle_command_reply: tid=0 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 client.17553 shutdown 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 msgr2=0x5638cf319710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf319710 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f2560012d20 tx=0x7f256000f790 comp rx=0 tx=0).stop 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 shutdown_connections 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf319d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf32c490 0x5638cf32e8b0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf319710 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 --2- 172.21.15.33:0/2641083436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf31c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f254eff5640 0 client.17553 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 >> 172.21.15.33:0/2641083436 conn(0x5638cf30df20 msgr2=0x5638cf313ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 shutdown_connections 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 1 -- 172.21.15.33:0/2641083436 wait complete. 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.450+0000 7f2573865740 20 client.17553 trim_cache size 0 max 16384 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 Processor -- start 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 -- start start 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf3287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf3090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf31dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1ae340 con 0x5638cf310a10 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1afe80 con 0x5638cf31c9a0 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f2573865740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1aea10 con 0x5638cf3100f0 2024-09-10T21:15:34.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f256a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf3090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f256b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf3287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f256a7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf3090b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48510/0 (socket says 172.21.15.33:48510) 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f256a7fc640 1 -- 172.21.15.33:0/1198087231 learned_addr learned my addr 172.21.15.33:0/1198087231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.465+0000 7f256b7fe640 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf31dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f256a7fc640 1 -- 172.21.15.33:0/1198087231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 msgr2=0x5638cf3287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f256a7fc640 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf3287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f256a7fc640 1 -- 172.21.15.33:0/1198087231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 msgr2=0x5638cf31dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f256a7fc640 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf31dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f256a7fc640 1 -- 172.21.15.33:0/1198087231 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638cf30ec00 con 0x5638cf310a10 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f256a7fc640 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf3090b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2560012cf0 tx=0x7f2560005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1bdc36bfc1423e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f2550ff9640 1 -- 172.21.15.33:0/1198087231 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560010ac0 con 0x5638cf310a10 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f2550ff9640 1 -- 172.21.15.33:0/1198087231 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2560010c60 con 0x5638cf310a10 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.466+0000 7f2550ff9640 1 -- 172.21.15.33:0/1198087231 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2560003a80 con 0x5638cf310a10 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 -- 172.21.15.33:0/1198087231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 msgr2=0x5638cf3090b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf3090b0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2560012cf0 tx=0x7f2560005870 comp rx=0 tx=0).stop 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 -- 172.21.15.33:0/1198087231 shutdown_connections 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf31c9a0 0x5638cf31dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf310a10 0x5638cf3090b0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 --2- 172.21.15.33:0/1198087231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf3100f0 0x5638cf3287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 -- 172.21.15.33:0/1198087231 >> 172.21.15.33:0/1198087231 conn(0x5638cf30df20 msgr2=0x5638cf3194c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 -- 172.21.15.33:0/1198087231 shutdown_connections 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 -- 172.21.15.33:0/1198087231 wait complete. 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 -- start start 2024-09-10T21:15:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.467+0000 7f2573865740 1 Processor -- start 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f2573865740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf324a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f2573865740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf324f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f2573865740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f2573865740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1af7b0 con 0x5638cf3100f0 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f2573865740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1af0e0 con 0x5638cf310a10 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f2573865740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f256c1ac300 con 0x5638cf31c9a0 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f2573865740 10 client.0 _ll_register_callbacks cb 0x7f2560009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf324a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256a7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf324f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf324a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48520/0 (socket says 172.21.15.33:48520) 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256a7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf324f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37786/0 (socket says 172.21.15.33:37786) 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 -- 172.21.15.33:0/900834163 learned_addr learned my addr 172.21.15.33:0/900834163 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 -- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 msgr2=0x5638cf31f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 -- 172.21.15.33:0/900834163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 msgr2=0x5638cf324f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf324f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 -- 172.21.15.33:0/900834163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638cf31ff50 con 0x5638cf3100f0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256a7fc640 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf324f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f256b7fe640 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf324a40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2564031440 tx=0x7f25600128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30aca115e8558796 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.468+0000 7f254eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f254eff5640 1 -- 172.21.15.33:0/900834163 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2564040040 con 0x5638cf3100f0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f254eff5640 1 -- 172.21.15.33:0/900834163 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f256404c040 con 0x5638cf3100f0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2560010c60 con 0x5638cf3100f0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f254eff5640 1 -- 172.21.15.33:0/900834163 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2564035040 con 0x5638cf3100f0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 10 client.17559 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x5638cf39e170 con 0x5638cf3100f0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f254eff5640 1 -- 172.21.15.33:0/900834163 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2560010c60 con 0x5638cf3100f0 2024-09-10T21:15:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f254eff5640 1 -- 172.21.15.33:0/900834163 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f2564066030 con 0x5638cf3100f0 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 10 client.17559 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 10 client.17559 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 20 client.17559 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf39e3a0 0x5638cf3a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 4 client.17559 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638cf2848a0 con 0x5638cf39e3a0 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.469+0000 7f256a7fc640 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf39e3a0 0x5638cf3a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f256a7fc640 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf39e3a0 0x5638cf3a0790 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=319131b2a0676b3d server_cookie=181b52b4f33a2a0 in_seq=0 out_seq=0 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f254eff5640 10 client.17559 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f254eff5640 1 -- 172.21.15.33:0/900834163 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638cf2848a0 con 0x5638cf39e3a0 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f254eff5640 10 client.17559 handle_command_reply: tid=0 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 client.17559 shutdown 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 msgr2=0x5638cf324a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf324a40 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2564031440 tx=0x7f25600128a0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.785 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:15:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 shutdown_connections 2024-09-10T21:15:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638cf31c9a0 0x5638cf31f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5638cf310a10 0x5638cf324f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5638cf39e3a0 0x5638cf3a0790 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 --2- 172.21.15.33:0/900834163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5638cf3100f0 0x5638cf324a40 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f254eff5640 0 client.17559 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 >> 172.21.15.33:0/900834163 conn(0x5638cf30df20 msgr2=0x5638cf30c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 shutdown_connections 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 1 -- 172.21.15.33:0/900834163 wait complete. 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.470+0000 7f2573865740 20 client.17559 trim_cache size 0 max 16384 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f25540b52e0 msgr2=0x7f25540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f25540b52e0 0x7f25540b77a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f255c0a9c20 tx=0x7f255c0e8040 comp rx=0 tx=0).stop 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 msgr2=0x7f256c199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 0x7f256c199210 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f255c00eb90 tx=0x7f255c00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 shutdown_connections 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f25540b52e0 0x7f25540b77a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f256c10d0e0 0x7f256c19eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f256c104530 0x7f256c199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 --2- 172.21.15.33:0/519319901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f256c103b30 0x7f256c199210 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:34.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 >> 172.21.15.33:0/519319901 conn(0x7f256c0fd700 msgr2=0x7f256c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:34.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 shutdown_connections 2024-09-10T21:15:34.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:34.471+0000 7f2571490640 1 -- 172.21.15.33:0/519319901 wait complete. 2024-09-10T21:15:35.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 873744 6.64 MB/sec execute 516 sec latency 97.939 ms 2024-09-10T21:15:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:35 smithi033 ceph-mon[30860]: pgmap v694: 97 pgs: 97 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-09-10T21:15:36.018 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:15:36.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:35 smithi104 ceph-mon[35325]: pgmap v694: 97 pgs: 97 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-09-10T21:15:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:35 smithi174 ceph-mon[35635]: pgmap v694: 97 pgs: 97 active+clean; 334 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 380 op/s 2024-09-10T21:15:36.301 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:36.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 875720 6.64 MB/sec execute 517 sec latency 56.895 ms 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.892+0000 7f08dac68640 1 -- 172.21.15.33:0/1247893824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 msgr2=0x7f08d410cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.892+0000 7f08dac68640 1 --2- 172.21.15.33:0/1247893824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d410cb30 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009860 tx=0x7f08c002a340 comp rx=0 tx=0).stop 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.892+0000 7f08dac68640 1 -- 172.21.15.33:0/1247893824 shutdown_connections 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.892+0000 7f08dac68640 1 --2- 172.21.15.33:0/1247893824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d410d100 0x7f08d410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.892+0000 7f08dac68640 1 --2- 172.21.15.33:0/1247893824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d410cb30 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.893+0000 7f08dac68640 1 --2- 172.21.15.33:0/1247893824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08d4103b50 0x7f08d4103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.893+0000 7f08dac68640 1 -- 172.21.15.33:0/1247893824 >> 172.21.15.33:0/1247893824 conn(0x7f08d40fd700 msgr2=0x7f08d40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.893+0000 7f08dac68640 1 -- 172.21.15.33:0/1247893824 shutdown_connections 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.893+0000 7f08dac68640 1 -- 172.21.15.33:0/1247893824 wait complete. 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.893+0000 7f08dac68640 1 Processor -- start 2024-09-10T21:15:36.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.893+0000 7f08dac68640 1 -- start start 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08dac68640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4103b50 0x7f08d4199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08dac68640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d41997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08dac68640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08d410d100 0x7f08d419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08dac68640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4112d60 con 0x7f08d4103b50 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08dac68640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4112be0 con 0x7f08d410d100 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08dac68640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08d4112ee0 con 0x7f08d4104550 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d8a04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4103b50 0x7f08d4199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d41997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d41997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:38276/0 (socket says 172.21.15.33:38276) 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 -- 172.21.15.33:0/1424684624 learned_addr learned my addr 172.21.15.33:0/1424684624 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:36.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d8a04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4103b50 0x7f08d4199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48536/0 (socket says 172.21.15.33:48536) 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d9205640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08d410d100 0x7f08d419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 -- 172.21.15.33:0/1424684624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08d410d100 msgr2=0x7f08d419eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08d410d100 0x7f08d419eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 -- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4103b50 msgr2=0x7f08d4199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4103b50 0x7f08d4199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 -- 172.21.15.33:0/1424684624 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d419f340 con 0x7f08d4104550 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d9205640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08d410d100 0x7f08d419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d8a04640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4103b50 0x7f08d4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.894+0000 7f08d3fff640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d41997d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08c00303f0 tx=0x7f08c0032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.895+0000 7f08d1ffb640 1 -- 172.21.15.33:0/1424684624 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0040020 con 0x7f08d4104550 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.895+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d419f5d0 con 0x7f08d4104550 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.895+0000 7f08d1ffb640 1 -- 172.21.15.33:0/1424684624 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08c0030dd0 con 0x7f08d4104550 2024-09-10T21:15:36.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.895+0000 7f08d1ffb640 1 -- 172.21.15.33:0/1424684624 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0037040 con 0x7f08d4104550 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.895+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d41a5cf0 con 0x7f08d4104550 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 Processor -- start 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 -- start start 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08d1ffb640 1 -- 172.21.15.33:0/1424684624 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f08c0030450 con 0x7f08d4104550 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d691538b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69163c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69164190 0x563d69166580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d690c9ea0 con 0x563d691534d0 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563d690b92d0 con 0x563d69153df0 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08dd03d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d690bd0e0 con 0x563d69164190 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08d9205640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d691538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08d9205640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d691538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48546/0 (socket says 172.21.15.33:48546) 2024-09-10T21:15:36.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08d9205640 1 -- 172.21.15.33:0/3751220624 learned_addr learned my addr 172.21.15.33:0/3751220624 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.898+0000 7f08d3fff640 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69164190 0x563d69166580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d9205640 1 -- 172.21.15.33:0/3751220624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69164190 msgr2=0x563d69166580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d9205640 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69164190 0x563d69166580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d9205640 1 -- 172.21.15.33:0/3751220624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 msgr2=0x563d69163c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d8a04640 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69163c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d9205640 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69163c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d9205640 1 -- 172.21.15.33:0/3751220624 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d68e17650 con 0x563d691534d0 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d9205640 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d691538b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00e920 tx=0x7f08cc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed42ce3dc4df6c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d1ffb640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08c40b52e0 0x7f08c40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08beffd640 1 -- 172.21.15.33:0/3751220624 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc00de90 con 0x563d691534d0 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08beffd640 1 -- 172.21.15.33:0/3751220624 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc01c070 con 0x563d691534d0 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08beffd640 1 -- 172.21.15.33:0/3751220624 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0165e0 con 0x563d691534d0 2024-09-10T21:15:36.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d1ffb640 1 -- 172.21.15.33:0/1424684624 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f08c0105dc0 con 0x7f08d4104550 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.899+0000 7f08d8a04640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08c40b52e0 0x7f08c40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 -- 172.21.15.33:0/3751220624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 msgr2=0x563d691538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08d8a04640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08c40b52e0 0x7f08c40b77a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x563d69156650 tx=0x7f08c8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d691538b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00e920 tx=0x7f08cc0095c0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 -- 172.21.15.33:0/3751220624 shutdown_connections 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69164190 0x563d69166580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69163c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3751220624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d691538b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 -- 172.21.15.33:0/3751220624 >> 172.21.15.33:0/3751220624 conn(0x563d69151300 msgr2=0x563d691524a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 -- 172.21.15.33:0/3751220624 shutdown_connections 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 -- 172.21.15.33:0/3751220624 wait complete. 2024-09-10T21:15:36.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.900+0000 7f08dd03d740 1 -- start start 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 1 Processor -- start 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69163a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d691614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563d68f1a700 con 0x563d69164190 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563d69054160 con 0x563d69153df0 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d69166e40 con 0x563d691534d0 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08dd03d740 10 client.0 _ll_register_callbacks cb 0x563d6906d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08d9205640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d691614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08d9205640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d691614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48556/0 (socket says 172.21.15.33:48556) 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08d8a04640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08d9205640 1 -- 172.21.15.33:0/284516766 learned_addr learned my addr 172.21.15.33:0/284516766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:36.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.901+0000 7f08d3fff640 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08d9205640 1 -- 172.21.15.33:0/284516766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 msgr2=0x563d69163a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08d9205640 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69163a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08d9205640 1 -- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 msgr2=0x563d69160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08d9205640 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08d9205640 1 -- 172.21.15.33:0/284516766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d69081c90 con 0x563d69164190 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08d3fff640 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08d9205640 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d691614f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00df40 tx=0x7f08cc00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1297dca015cbfaf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08b3fff640 1 -- 172.21.15.33:0/284516766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc003970 con 0x563d69164190 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08b3fff640 1 -- 172.21.15.33:0/284516766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f08cc003b10 con 0x563d69164190 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08cc01c070 con 0x563d69164190 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08b3fff640 1 -- 172.21.15.33:0/284516766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f08cc003d10 con 0x563d69164190 2024-09-10T21:15:36.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08dd03d740 10 client.17571 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563d69172790 con 0x563d69164190 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.902+0000 7f08b3fff640 1 -- 172.21.15.33:0/284516766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08cc01c070 con 0x563d69164190 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08b3fff640 1 -- 172.21.15.33:0/284516766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f08cc00b040 con 0x563d69164190 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08dd03d740 10 client.17571 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08dd03d740 10 client.17571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08dd03d740 20 client.17571 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08dd03d740 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d69173bf0 0x563d69175fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08dd03d740 4 client.17571 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d690cbdf0 con 0x563d69173bf0 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08d8a04640 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d69173bf0 0x563d69175fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08d8a04640 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d69173bf0 0x563d69175fe0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=139513ec724a1d9a server_cookie=dcad753c8d2486f in_seq=0 out_seq=0 2024-09-10T21:15:36.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.903+0000 7f08b3fff640 10 client.17571 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08b3fff640 1 -- 172.21.15.33:0/284516766 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563d690cbdf0 con 0x563d69173bf0 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08b3fff640 10 client.17571 handle_command_reply: tid=0 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 client.17571 shutdown 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 msgr2=0x563d691614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d691614f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f08cc00df40 tx=0x7f08cc00df70 comp rx=0 tx=0).stop 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 shutdown_connections 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d69173bf0 0x563d69175fe0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d691614f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 --2- 172.21.15.33:0/284516766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69163a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08b3fff640 0 client.17571 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:36.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.904+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 >> 172.21.15.33:0/284516766 conn(0x563d69151300 msgr2=0x563d69156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:36.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.905+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 shutdown_connections 2024-09-10T21:15:36.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.905+0000 7f08dd03d740 1 -- 172.21.15.33:0/284516766 wait complete. 2024-09-10T21:15:36.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.905+0000 7f08dd03d740 20 client.17571 trim_cache size 0 max 16384 2024-09-10T21:15:36.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 Processor -- start 2024-09-10T21:15:36.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 -- start start 2024-09-10T21:15:36.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69150c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69170070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d69175c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41ac1d0 con 0x563d69164190 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41add10 con 0x563d69153df0 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08dd03d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41ac8a0 con 0x563d691534d0 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69170070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d9205640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d69175c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69170070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37846/0 (socket says 172.21.15.33:37846) 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 -- 172.21.15.33:0/2286735874 learned_addr learned my addr 172.21.15.33:0/2286735874 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d9205640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d69175c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48564/0 (socket says 172.21.15.33:48564) 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d8a04640 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 -- 172.21.15.33:0/2286735874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 msgr2=0x563d69150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 -- 172.21.15.33:0/2286735874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 msgr2=0x563d69175c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d69175c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.919+0000 7f08d3fff640 1 -- 172.21.15.33:0/2286735874 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d69160c90 con 0x563d69153df0 2024-09-10T21:15:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.920+0000 7f08d3fff640 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69170070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cc510 tx=0x7f08c002ab40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c82fffd26ded1ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.920+0000 7f08be7fc640 1 -- 172.21.15.33:0/2286735874 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00c8420 con 0x563d69153df0 2024-09-10T21:15:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.920+0000 7f08be7fc640 1 -- 172.21.15.33:0/2286735874 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08c00d5450 con 0x563d69153df0 2024-09-10T21:15:36.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.920+0000 7f08be7fc640 1 -- 172.21.15.33:0/2286735874 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c00cbcf0 con 0x563d69153df0 2024-09-10T21:15:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 -- 172.21.15.33:0/2286735874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 msgr2=0x563d69170070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69170070 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f08c00cc510 tx=0x7f08c002ab40 comp rx=0 tx=0).stop 2024-09-10T21:15:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 -- 172.21.15.33:0/2286735874 shutdown_connections 2024-09-10T21:15:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d69164190 0x563d69175c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69153df0 0x563d69170070 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 --2- 172.21.15.33:0/2286735874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d691534d0 0x563d69150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 -- 172.21.15.33:0/2286735874 >> 172.21.15.33:0/2286735874 conn(0x563d69151300 msgr2=0x563d69165f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 -- 172.21.15.33:0/2286735874 shutdown_connections 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 -- 172.21.15.33:0/2286735874 wait complete. 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 -- start start 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.921+0000 7f08dd03d740 1 Processor -- start 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d6916c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69153df0 0x563d6917b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08dd03d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69164190 0x563d6917ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08dd03d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41ad640 con 0x563d691534d0 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08dd03d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41acf70 con 0x563d69164190 2024-09-10T21:15:36.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08dd03d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08d41aa190 con 0x563d69153df0 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08dd03d740 10 client.0 _ll_register_callbacks cb 0x7f08c0109710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d6916c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d6916c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48576/0 (socket says 172.21.15.33:48576) 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 -- 172.21.15.33:0/3252645474 learned_addr learned my addr 172.21.15.33:0/3252645474 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d3fff640 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69153df0 0x563d6917b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d9205640 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69164190 0x563d6917ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 -- 172.21.15.33:0/3252645474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69153df0 msgr2=0x563d6917b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69153df0 0x563d6917b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 -- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69164190 msgr2=0x563d6917ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69164190 0x563d6917ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 -- 172.21.15.33:0/3252645474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d691df810 con 0x563d691534d0 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08d8a04640 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d6916c9f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f08c8013cc0 tx=0x7f08c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f117a2a2cc665c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:36.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.922+0000 7f08b3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08b3fff640 1 -- 172.21.15.33:0/3252645474 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c8036060 con 0x563d691534d0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08b3fff640 1 -- 172.21.15.33:0/3252645474 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f08c801d980 con 0x563d691534d0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d690c78a0 con 0x563d691534d0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08b3fff640 1 -- 172.21.15.33:0/3252645474 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f08c801db80 con 0x563d691534d0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 10 client.17577 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x563d691e0a40 con 0x563d691534d0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08b3fff640 1 -- 172.21.15.33:0/3252645474 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d690c78a0 con 0x563d691534d0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08b3fff640 1 -- 172.21.15.33:0/3252645474 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7f08c803b070 con 0x563d691534d0 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 10 client.17577 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 10 client.17577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 20 client.17577 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d691e1220 0x563d691e3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 4 client.17577 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.923+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d690c4cd0 con 0x563d691e1220 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08d3fff640 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d691e1220 0x563d691e3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08d3fff640 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d691e1220 0x563d691e3640 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c11f9bac7fcf9bf3 server_cookie=f114e24b5fd9d5c0 in_seq=0 out_seq=0 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08b3fff640 10 client.17577 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08b3fff640 1 -- 172.21.15.33:0/3252645474 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d690c4cd0 con 0x563d691e1220 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08b3fff640 10 client.17577 handle_command_reply: tid=0 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 client.17577 shutdown 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 msgr2=0x563d6916c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d6916c9f0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f08c8013cc0 tx=0x7f08c80046c0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 shutdown_connections 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x563d691e1220 0x563d691e3640 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563d69164190 0x563d6917ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d69153df0 0x563d6917b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08b3fff640 0 client.17577 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 --2- 172.21.15.33:0/3252645474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563d691534d0 0x563d6916c9f0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.924+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 >> 172.21.15.33:0/3252645474 conn(0x563d69151300 msgr2=0x563d69162510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.925+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 shutdown_connections 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.925+0000 7f08dd03d740 1 -- 172.21.15.33:0/3252645474 wait complete. 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.925+0000 7f08dd03d740 20 client.17577 trim_cache size 0 max 16384 2024-09-10T21:15:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:15:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.925+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08c40b52e0 msgr2=0x7f08c40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.925+0000 7f08dac68640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08c40b52e0 0x7f08c40b77a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x563d69156650 tx=0x7f08c8005f90 comp rx=0 tx=0).stop 2024-09-10T21:15:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.925+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 msgr2=0x7f08d41997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.925+0000 7f08dac68640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d41997d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f08c00303f0 tx=0x7f08c0032da0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 shutdown_connections 2024-09-10T21:15:36.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f08c40b52e0 0x7f08c40b77a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08d410d100 0x7f08d419eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08d4104550 0x7f08d41997d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 --2- 172.21.15.33:0/1424684624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08d4103b50 0x7f08d4199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 >> 172.21.15.33:0/1424684624 conn(0x7f08d40fd700 msgr2=0x7f08d40fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 shutdown_connections 2024-09-10T21:15:36.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:36.926+0000 7f08dac68640 1 -- 172.21.15.33:0/1424684624 wait complete. 2024-09-10T21:15:37.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 877385 6.65 MB/sec execute 518 sec latency 12.744 ms 2024-09-10T21:15:37.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:37 smithi033 ceph-mon[30860]: pgmap v695: 97 pgs: 97 active+clean; 333 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-09-10T21:15:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:37 smithi104 ceph-mon[35325]: pgmap v695: 97 pgs: 97 active+clean; 333 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-09-10T21:15:38.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:37 smithi174 ceph-mon[35635]: pgmap v695: 97 pgs: 97 active+clean; 333 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.7 MiB/s wr, 300 op/s 2024-09-10T21:15:38.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 879906 6.66 MB/sec execute 519 sec latency 10.902 ms 2024-09-10T21:15:38.764 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:15:38.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:15:39.041 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 -- 172.21.15.33:0/1243633363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108fa0 msgr2=0x7fcec410f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/1243633363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108fa0 0x7fcec410f7a0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc02a410 comp rx=0 tx=0).stop 2024-09-10T21:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 -- 172.21.15.33:0/1243633363 shutdown_connections 2024-09-10T21:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/1243633363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108fa0 0x7fcec410f7a0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/1243633363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4108620 0x7fcec4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/1243633363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcec4102c10 0x7fcec4102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 -- 172.21.15.33:0/1243633363 >> 172.21.15.33:0/1243633363 conn(0x7fcec40fe6c0 msgr2=0x7fcec4100ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 -- 172.21.15.33:0/1243633363 shutdown_connections 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.507+0000 7fcec8d9e640 1 -- 172.21.15.33:0/1243633363 wait complete. 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 Processor -- start 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 -- start start 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4102c10 0x7fcec4199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108620 0x7fcec4199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcec4108fa0 0x7fcec419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4112c30 con 0x7fcec4108620 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4112ab0 con 0x7fcec4108fa0 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec8d9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcec4112db0 con 0x7fcec4102c10 2024-09-10T21:15:39.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108620 0x7fcec4199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108620 0x7fcec4199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57416/0 (socket says 172.21.15.33:57416) 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec2ffd640 1 -- 172.21.15.33:0/3453213581 learned_addr learned my addr 172.21.15.33:0/3453213581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.508+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4102c10 0x7fcec4199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec3fff640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcec4108fa0 0x7fcec419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec2ffd640 1 -- 172.21.15.33:0/3453213581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4102c10 msgr2=0x7fcec4199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4102c10 0x7fcec4199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec2ffd640 1 -- 172.21.15.33:0/3453213581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcec4108fa0 msgr2=0x7fcec419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcec4108fa0 0x7fcec419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec2ffd640 1 -- 172.21.15.33:0/3453213581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec419f0f0 con 0x7fcec4108620 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108620 0x7fcec4199580 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fceb800e9f0 tx=0x7fceb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec0ff9640 1 -- 172.21.15.33:0/3453213581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb800cd80 con 0x7fcec4108620 2024-09-10T21:15:39.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec0ff9640 1 -- 172.21.15.33:0/3453213581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb80054d0 con 0x7fcec4108620 2024-09-10T21:15:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcec419f3e0 con 0x7fcec4108620 2024-09-10T21:15:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec0ff9640 1 -- 172.21.15.33:0/3453213581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb8010690 con 0x7fcec4108620 2024-09-10T21:15:39.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.509+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcec41a5aa0 con 0x7fcec4108620 2024-09-10T21:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.512+0000 7fcecc3d5740 1 Processor -- start 2024-09-10T21:15:39.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.512+0000 7fcecc3d5740 1 -- start start 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a00f0 0x564c4e0a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a0a10 0x564c4e0ac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec0ff9640 1 -- 172.21.15.33:0/3453213581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fceb8004e90 con 0x7fcec4108620 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c4e016e60 con 0x564c4e0a0a10 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564c4e006290 con 0x564c4e0ac9a0 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c4e00a0a0 con 0x564c4e0a00f0 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a00f0 0x564c4e0a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a0a10 0x564c4e0ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a00f0 0x564c4e0a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55412/0 (socket says 172.21.15.33:55412) 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 -- 172.21.15.33:0/4213460145 learned_addr learned my addr 172.21.15.33:0/4213460145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 -- 172.21.15.33:0/4213460145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 msgr2=0x564c4e0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 -- 172.21.15.33:0/4213460145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a0a10 msgr2=0x564c4e0ac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a0a10 0x564c4e0ac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.513+0000 7fcec3fff640 1 -- 172.21.15.33:0/4213460145 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4dd64930 con 0x564c4e0a00f0 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcec3fff640 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a00f0 0x564c4e0a04d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc032ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ccc4628daf4a8a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcec0ff9640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fceac0b5390 0x7fceac0b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcea9ffb640 1 -- 172.21.15.33:0/4213460145 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc035040 con 0x564c4e0a00f0 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcec37fe640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fceac0b5390 0x7fceac0b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcea9ffb640 1 -- 172.21.15.33:0/4213460145 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcebc004030 con 0x564c4e0a00f0 2024-09-10T21:15:39.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcec0ff9640 1 -- 172.21.15.33:0/3453213581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fceb80e5f90 con 0x7fcec4108620 2024-09-10T21:15:39.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.514+0000 7fcea9ffb640 1 -- 172.21.15.33:0/4213460145 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc005620 con 0x564c4e0a00f0 2024-09-10T21:15:39.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 -- 172.21.15.33:0/4213460145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a00f0 msgr2=0x564c4e0a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcec37fe640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fceac0b5390 0x7fceac0b7850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009fd0 tx=0x7fceb4009760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a00f0 0x564c4e0a04d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcebc009900 tx=0x7fcebc032ba0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 -- 172.21.15.33:0/4213460145 shutdown_connections 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a0a10 0x564c4e0ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/4213460145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a00f0 0x564c4e0a04d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 -- 172.21.15.33:0/4213460145 >> 172.21.15.33:0/4213460145 conn(0x564c4e09df20 msgr2=0x564c4e09f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 -- 172.21.15.33:0/4213460145 shutdown_connections 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 -- 172.21.15.33:0/4213460145 wait complete. 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.515+0000 7fcecc3d5740 1 -- start start 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 1 Processor -- start 2024-09-10T21:15:39.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a00f0 0x564c4e0a9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564c4de67890 con 0x564c4e0ac9a0 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564c4dfa0d40 con 0x564c4e0a00f0 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564c4e0af5f0 con 0x564c4e0a0a10 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcecc3d5740 10 client.0 _ll_register_callbacks cb 0x564c4e004c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a00f0 0x564c4e0a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a00f0 0x564c4e0a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53524/0 (socket says 172.21.15.33:53524) 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 -- 172.21.15.33:0/706460412 learned_addr learned my addr 172.21.15.33:0/706460412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:39.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 -- 172.21.15.33:0/706460412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 msgr2=0x564c4e0a9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0a9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 -- 172.21.15.33:0/706460412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0ac9a0 msgr2=0x564c4e0aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.516+0000 7fcec37fe640 1 -- 172.21.15.33:0/706460412 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4e0b8a30 con 0x564c4e0a00f0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcec37fe640 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a00f0 0x564c4e0a9950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fceb4004470 tx=0x7fceb4004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2296e9b33960dbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcea2ffd640 1 -- 172.21.15.33:0/706460412 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb402e060 con 0x564c4e0a00f0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcea2ffd640 1 -- 172.21.15.33:0/706460412 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fceb4029400 con 0x564c4e0a00f0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcebc004030 con 0x564c4e0a00f0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcea2ffd640 1 -- 172.21.15.33:0/706460412 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fceb4029640 con 0x564c4e0a00f0 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcecc3d5740 10 client.27032 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:39.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564c4e0bb420 con 0x564c4e0a00f0 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcea2ffd640 1 -- 172.21.15.33:0/706460412 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcebc004030 con 0x564c4e0a00f0 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.517+0000 7fcea2ffd640 1 -- 172.21.15.33:0/706460412 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fceb4033070 con 0x564c4e0a00f0 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcecc3d5740 10 client.27032 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcecc3d5740 10 client.27032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcecc3d5740 20 client.27032 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e0bc8e0 0x564c4e0becd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcecc3d5740 4 client.27032 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c4e018db0 con 0x564c4e0bc8e0 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e0bc8e0 0x564c4e0becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e0bc8e0 0x564c4e0becd0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a13f7b0e1ab09b server_cookie=40d2ddcd24754d59 in_seq=0 out_seq=0 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.518+0000 7fcea2ffd640 10 client.27032 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcea2ffd640 1 -- 172.21.15.33:0/706460412 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564c4e018db0 con 0x564c4e0bc8e0 2024-09-10T21:15:39.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcea2ffd640 10 client.27032 handle_command_reply: tid=0 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 client.27032 shutdown 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a00f0 msgr2=0x564c4e0a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a00f0 0x564c4e0a9950 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fceb4004470 tx=0x7fceb4004a40 comp rx=0 tx=0).stop 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 shutdown_connections 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e0bc8e0 0x564c4e0becd0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0ac9a0 0x564c4e0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcea2ffd640 0 client.27032 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0a9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/706460412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a00f0 0x564c4e0a9950 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 >> 172.21.15.33:0/706460412 conn(0x564c4e09df20 msgr2=0x564c4e0a3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 shutdown_connections 2024-09-10T21:15:39.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 1 -- 172.21.15.33:0/706460412 wait complete. 2024-09-10T21:15:39.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.519+0000 7fcecc3d5740 20 client.27032 trim_cache size 0 max 16384 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 Processor -- start 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 -- start start 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a0a10 0x564c4e09ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0ac9a0 0x564c4e0a6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41abfe0 con 0x564c4e0a00f0 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41adb20 con 0x564c4e0a0a10 2024-09-10T21:15:39.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41ac6b0 con 0x564c4e0ac9a0 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcec2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a0a10 0x564c4e09ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0a91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57440/0 (socket says 172.21.15.33:57440) 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcec2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a0a10 0x564c4e09ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53538/0 (socket says 172.21.15.33:53538) 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcec3fff640 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0ac9a0 0x564c4e0a6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.537+0000 7fcec37fe640 1 -- 172.21.15.33:0/3160431495 learned_addr learned my addr 172.21.15.33:0/3160431495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec37fe640 1 -- 172.21.15.33:0/3160431495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0ac9a0 msgr2=0x564c4e0a6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec37fe640 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0ac9a0 0x564c4e0a6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec37fe640 1 -- 172.21.15.33:0/3160431495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a0a10 msgr2=0x564c4e09ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec37fe640 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a0a10 0x564c4e09ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec37fe640 1 -- 172.21.15.33:0/3160431495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c4e09e9a0 con 0x564c4e0a00f0 2024-09-10T21:15:39.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec3fff640 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0ac9a0 0x564c4e0a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:39.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a0a10 0x564c4e09ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:39.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcec37fe640 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0a91e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009fa0 tx=0x7fceb4029590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4e6cdc8662b816 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:39.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcea97fa640 1 -- 172.21.15.33:0/3160431495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4039020 con 0x564c4e0a00f0 2024-09-10T21:15:39.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcea97fa640 1 -- 172.21.15.33:0/3160431495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb4034b30 con 0x564c4e0a00f0 2024-09-10T21:15:39.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.538+0000 7fcea97fa640 1 -- 172.21.15.33:0/3160431495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4029b60 con 0x564c4e0a00f0 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 -- 172.21.15.33:0/3160431495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 msgr2=0x564c4e0a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0a91e0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009fa0 tx=0x7fceb4029590 comp rx=0 tx=0).stop 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 -- 172.21.15.33:0/3160431495 shutdown_connections 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0ac9a0 0x564c4e0a6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0a0a10 0x564c4e09ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/3160431495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0a91e0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 -- 172.21.15.33:0/3160431495 >> 172.21.15.33:0/3160431495 conn(0x564c4e09df20 msgr2=0x564c4e09c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 -- 172.21.15.33:0/3160431495 shutdown_connections 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 -- 172.21.15.33:0/3160431495 wait complete. 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 -- start start 2024-09-10T21:15:39.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.539+0000 7fcecc3d5740 1 Processor -- start 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0b4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0b1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcecc3d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0b1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41ad450 con 0x564c4e0a00f0 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41acd80 con 0x564c4e0ac9a0 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcecc3d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcec41a9fa0 con 0x564c4e0a0a10 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcecc3d5740 10 client.0 _ll_register_callbacks cb 0x7fceb4009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0b4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0b4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57446/0 (socket says 172.21.15.33:57446) 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 -- 172.21.15.33:0/1644045390 learned_addr learned my addr 172.21.15.33:0/1644045390 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:39.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec3fff640 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 -- 172.21.15.33:0/1644045390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 msgr2=0x564c4e0b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 -- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 msgr2=0x564c4e0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0b1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 -- 172.21.15.33:0/1644045390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c4e0b1e30 con 0x564c4e0a00f0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcec37fe640 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0b4e00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fceb40349c0 tx=0x7fceb4009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3d47f02f87f133 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.540+0000 7fcea2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcea2ffd640 1 -- 172.21.15.33:0/1644045390 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fceb4039030 con 0x564c4e0a00f0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcea2ffd640 1 -- 172.21.15.33:0/1644045390 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fceb40297e0 con 0x564c4e0a00f0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fceb4034b30 con 0x564c4e0a00f0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcea2ffd640 1 -- 172.21.15.33:0/1644045390 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fceb402e050 con 0x564c4e0a00f0 2024-09-10T21:15:39.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 10 client.17595 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x564c4e12a5a0 con 0x564c4e0a00f0 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcea2ffd640 1 -- 172.21.15.33:0/1644045390 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fceb4034b30 con 0x564c4e0a00f0 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcea2ffd640 1 -- 172.21.15.33:0/1644045390 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fceb4033070 con 0x564c4e0a00f0 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 10 client.17595 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 10 client.17595 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 20 client.17595 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e12ac30 0x564c4e12d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 4 client.17595 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.541+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564c4e014860 con 0x564c4e12ac30 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.542+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e12ac30 0x564c4e12d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.542+0000 7fcea2ffd640 10 client.17595 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:39.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.542+0000 7fcec2ffd640 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e12ac30 0x564c4e12d050 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7141dbfe709e8f7c server_cookie=ffb943cd39439a59 in_seq=0 out_seq=0 2024-09-10T21:15:39.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.543+0000 7fcea2ffd640 1 -- 172.21.15.33:0/1644045390 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c4e014860 con 0x564c4e12ac30 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.543+0000 7fcea2ffd640 10 client.17595 handle_command_reply: tid=0 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.543+0000 7fcecc3d5740 1 client.17595 shutdown 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 msgr2=0x564c4e0b4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0b4e00 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fceb40349c0 tx=0x7fceb4009aa0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 shutdown_connections 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564c4e12ac30 0x564c4e12d050 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564c4e0ac9a0 0x564c4e0b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564c4e0a0a10 0x564c4e0b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 --2- 172.21.15.33:0/1644045390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564c4e0a00f0 0x564c4e0b4e00 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 >> 172.21.15.33:0/1644045390 conn(0x564c4e09df20 msgr2=0x564c4e09cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:39.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcea2ffd640 0 client.17595 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 shutdown_connections 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 1 -- 172.21.15.33:0/1644045390 wait complete. 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.544+0000 7fcecc3d5740 20 client.17595 trim_cache size 0 max 16384 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e", 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:15:39.553 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:15:39.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fceac0b5390 msgr2=0x7fceac0b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fceac0b5390 0x7fceac0b7850 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009fd0 tx=0x7fceb4009760 comp rx=0 tx=0).stop 2024-09-10T21:15:39.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108620 msgr2=0x7fcec4199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:39.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108620 0x7fcec4199580 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fceb800e9f0 tx=0x7fceb800edf0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 shutdown_connections 2024-09-10T21:15:39.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fceac0b5390 0x7fceac0b7850 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcec4108fa0 0x7fcec419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcec4108620 0x7fcec4199580 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 --2- 172.21.15.33:0/3453213581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcec4102c10 0x7fcec4199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:39.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 >> 172.21.15.33:0/3453213581 conn(0x7fcec40fe6c0 msgr2=0x7fcec4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:39.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 shutdown_connections 2024-09-10T21:15:39.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:39.545+0000 7fcec8d9e640 1 -- 172.21.15.33:0/3453213581 wait complete. 2024-09-10T21:15:39.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 881208 6.66 MB/sec execute 520 sec latency 58.681 ms 2024-09-10T21:15:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:39 smithi104 ceph-mon[35325]: pgmap v696: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-09-10T21:15:40.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:39 smithi174 ceph-mon[35635]: pgmap v696: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-09-10T21:15:40.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:39 smithi033 ceph-mon[30860]: pgmap v696: 97 pgs: 97 active+clean; 335 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 358 op/s 2024-09-10T21:15:40.383 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:15:40.661 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:15:40.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 883833 6.67 MB/sec execute 521 sec latency 19.678 ms 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:40 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:15:41.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.092 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:15:41.093 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.094 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.094 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:40 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:40 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 -- 172.21.15.33:0/3898314359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 msgr2=0x7fc50810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 --2- 172.21.15.33:0/3898314359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50810f560 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc500009ca0 tx=0x7fc50002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 -- 172.21.15.33:0/3898314359 shutdown_connections 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 --2- 172.21.15.33:0/3898314359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50810f560 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 --2- 172.21.15.33:0/3898314359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5081045c0 0x7fc50810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 --2- 172.21.15.33:0/3898314359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc508103ca0 0x7fc508104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 -- 172.21.15.33:0/3898314359 >> 172.21.15.33:0/3898314359 conn(0x7fc5080fd700 msgr2=0x7fc5080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 -- 172.21.15.33:0/3898314359 shutdown_connections 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.185+0000 7fc50ea08640 1 -- 172.21.15.33:0/3898314359 wait complete. 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 Processor -- start 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 -- start start 2024-09-10T21:15:41.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc508103ca0 0x7fc508199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5081045c0 0x7fc5081997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc508112d70 con 0x7fc50810d100 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc508112bf0 con 0x7fc5081045c0 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.186+0000 7fc50ea08640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc508112ef0 con 0x7fc508103ca0 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50819eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57476/0 (socket says 172.21.15.33:57476) 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 -- 172.21.15.33:0/391820338 learned_addr learned my addr 172.21.15.33:0/391820338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc5077fe640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5081045c0 0x7fc5081997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc507fff640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc508103ca0 0x7fc508199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 -- 172.21.15.33:0/391820338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc508103ca0 msgr2=0x7fc508199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc508103ca0 0x7fc508199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 -- 172.21.15.33:0/391820338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5081045c0 msgr2=0x7fc5081997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5081045c0 0x7fc5081997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 -- 172.21.15.33:0/391820338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc50819f310 con 0x7fc50810d100 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc5077fe640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5081045c0 0x7fc5081997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.187+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50819eb40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc500005cc0 tx=0x7fc500031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.188+0000 7fc5057fa640 1 -- 172.21.15.33:0/391820338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc500003b70 con 0x7fc50810d100 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.188+0000 7fc5057fa640 1 -- 172.21.15.33:0/391820338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5000318f0 con 0x7fc50810d100 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.188+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc50819f5a0 con 0x7fc50810d100 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.188+0000 7fc5057fa640 1 -- 172.21.15.33:0/391820338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50002f950 con 0x7fc50810d100 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.188+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a5cc0 con 0x7fc50810d100 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.190+0000 7fc510ddd740 1 Processor -- start 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.190+0000 7fc5057fa640 1 -- 172.21.15.33:0/391820338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc50002fb50 con 0x7fc50810d100 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc510ddd740 1 -- start start 2024-09-10T21:15:41.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcb8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc510ddd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b56fc2ee50 con 0x55b56fcb80a0 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc510ddd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b56fc1e280 con 0x55b56fcb89c0 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc510ddd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b56fc22090 con 0x55b56fcc4950 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc5077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcb8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc507fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc5077fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcb8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57488/0 (socket says 172.21.15.33:57488) 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc5077fe640 1 -- 172.21.15.33:0/1170767552 learned_addr learned my addr 172.21.15.33:0/1170767552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc5077fe640 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc507fff640 1 -- 172.21.15.33:0/1170767552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 msgr2=0x55b56fcc6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc507fff640 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc507fff640 1 -- 172.21.15.33:0/1170767552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 msgr2=0x55b56fcb8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc507fff640 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcb8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc507fff640 1 -- 172.21.15.33:0/1170767552 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56f97c650 con 0x55b56fcb89c0 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc5077fe640 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc5057fa640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4f00b5360 0x7fc4f00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.191+0000 7fc507fff640 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc4410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009c00 tx=0x7fc4f8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68dc75019f9b4abd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc5077fe640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4f00b5360 0x7fc4f00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc5057fa640 1 -- 172.21.15.33:0/391820338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc500106ca0 con 0x7fc50810d100 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc4ee7fc640 1 -- 172.21.15.33:0/1170767552 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8004230 con 0x55b56fcb89c0 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc4ee7fc640 1 -- 172.21.15.33:0/1170767552 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f80043d0 con 0x55b56fcb89c0 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc4ee7fc640 1 -- 172.21.15.33:0/1170767552 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8004c60 con 0x55b56fcb89c0 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc510ddd740 1 -- 172.21.15.33:0/1170767552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 msgr2=0x55b56fcc4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc510ddd740 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc4410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8009c00 tx=0x7fc4f8011810 comp rx=0 tx=0).stop 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.192+0000 7fc5077fe640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4f00b5360 0x7fc4f00b7820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009580 tx=0x7fc4fc009b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- 172.21.15.33:0/1170767552 shutdown_connections 2024-09-10T21:15:41.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc4410 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 --2- 172.21.15.33:0/1170767552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcb8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- 172.21.15.33:0/1170767552 >> 172.21.15.33:0/1170767552 conn(0x55b56fcb5ed0 msgr2=0x55b56fcb7070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- 172.21.15.33:0/1170767552 shutdown_connections 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- 172.21.15.33:0/1170767552 wait complete. 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- start start 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 Processor -- start 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcb89c0 0x55b56fcc1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcc4950 0x55b56fcc1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b56fa7f700 con 0x55b56fcb80a0 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b56fbb9110 con 0x55b56fcc4950 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.193+0000 7fc510ddd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b56fcc75d0 con 0x55b56fcb89c0 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc510ddd740 10 client.0 _ll_register_callbacks cb 0x55b56fbd2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:41.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57500/0 (socket says 172.21.15.33:57500) 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 -- 172.21.15.33:0/3888702872 learned_addr learned my addr 172.21.15.33:0/3888702872 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc5077fe640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcb89c0 0x55b56fcc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcc4950 0x55b56fcc1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 -- 172.21.15.33:0/3888702872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcb89c0 msgr2=0x55b56fcc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcb89c0 0x55b56fcc1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 -- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcc4950 msgr2=0x55b56fcc1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcc4950 0x55b56fcc1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 -- 172.21.15.33:0/3888702872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56fbe6c40 con 0x55b56fcb80a0 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcc4950 0x55b56fcc1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc507fff640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc41b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8019b80 tx=0x7fc4f8019bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ceeaa489e2103d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc4e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc4e77fe640 1 -- 172.21.15.33:0/3888702872 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8028020 con 0x55b56fcb80a0 2024-09-10T21:15:41.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc4e77fe640 1 -- 172.21.15.33:0/3888702872 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc4f8024620 con 0x55b56fcb80a0 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.194+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f80043d0 con 0x55b56fcb80a0 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc4e77fe640 1 -- 172.21.15.33:0/3888702872 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc4f8009e70 con 0x55b56fcb80a0 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 10 client.17607 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b56fcd2eb0 con 0x55b56fcb80a0 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc4e77fe640 1 -- 172.21.15.33:0/3888702872 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f80043d0 con 0x55b56fcb80a0 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc4e77fe640 1 -- 172.21.15.33:0/3888702872 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fc4f8018070 con 0x55b56fcb80a0 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 10 client.17607 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 10 client.17607 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 20 client.17607 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fcd4200 0x55b56fcd65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 4 client.17607 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b56fc30da0 con 0x55b56fcd4200 2024-09-10T21:15:41.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.195+0000 7fc5077fe640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fcd4200 0x55b56fcd65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.196+0000 7fc5077fe640 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fcd4200 0x55b56fcd65f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1185d8ac0e90f958 server_cookie=9b3105b4f219b00d in_seq=0 out_seq=0 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.196+0000 7fc4e77fe640 10 client.17607 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.196+0000 7fc4e77fe640 1 -- 172.21.15.33:0/3888702872 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b56fc30da0 con 0x55b56fcd4200 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.196+0000 7fc4e77fe640 10 client.17607 handle_command_reply: tid=0 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.196+0000 7fc510ddd740 1 client.17607 shutdown 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.196+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 msgr2=0x55b56fcc41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc41b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8019b80 tx=0x7fc4f8019bb0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 shutdown_connections 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fcd4200 0x55b56fcd65f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcc4950 0x55b56fcc1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcb89c0 0x55b56fcc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc4e77fe640 0 client.17607 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:41.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 --2- 172.21.15.33:0/3888702872 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc41b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 >> 172.21.15.33:0/3888702872 conn(0x55b56fcb5ed0 msgr2=0x55b56fcbba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 shutdown_connections 2024-09-10T21:15:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 1 -- 172.21.15.33:0/3888702872 wait complete. 2024-09-10T21:15:41.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.197+0000 7fc510ddd740 20 client.17607 trim_cache size 0 max 16384 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 Processor -- start 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 -- start start 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcd0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcbe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcbed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081ae3a0 con 0x55b56fcb80a0 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081afee0 con 0x55b56fcb89c0 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc510ddd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081aea70 con 0x55b56fcc4950 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcd0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcd0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57506/0 (socket says 172.21.15.33:57506) 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.216+0000 7fc507fff640 1 -- 172.21.15.33:0/2687565243 learned_addr learned my addr 172.21.15.33:0/2687565243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:41.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcbed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc5077fe640 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcbe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc507fff640 1 -- 172.21.15.33:0/2687565243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 msgr2=0x55b56fcbed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc507fff640 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcbed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc507fff640 1 -- 172.21.15.33:0/2687565243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 msgr2=0x55b56fcbe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc507fff640 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcbe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc507fff640 1 -- 172.21.15.33:0/2687565243 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b56fcbc350 con 0x55b56fcb80a0 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcbed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:41.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc507fff640 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcd0830 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8012fd0 tx=0x7fc4f8009ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9073da390a53655 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc4edffb640 1 -- 172.21.15.33:0/2687565243 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8028040 con 0x55b56fcb80a0 2024-09-10T21:15:41.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc4edffb640 1 -- 172.21.15.33:0/2687565243 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f801f040 con 0x55b56fcb80a0 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.217+0000 7fc4edffb640 1 -- 172.21.15.33:0/2687565243 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8018c60 con 0x55b56fcb80a0 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 -- 172.21.15.33:0/2687565243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 msgr2=0x55b56fcd0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcd0830 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8012fd0 tx=0x7fc4f8009ab0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 -- 172.21.15.33:0/2687565243 shutdown_connections 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcbed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcbe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2687565243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcd0830 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 -- 172.21.15.33:0/2687565243 >> 172.21.15.33:0/2687565243 conn(0x55b56fcb5ed0 msgr2=0x55b56fcb6f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 -- 172.21.15.33:0/2687565243 shutdown_connections 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 -- 172.21.15.33:0/2687565243 wait complete. 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.218+0000 7fc510ddd740 1 -- start start 2024-09-10T21:15:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 1 Processor -- start 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081af810 con 0x55b56fcb80a0 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081af140 con 0x55b56fcb89c0 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081ac360 con 0x55b56fcc4950 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc510ddd740 10 client.0 _ll_register_callbacks cb 0x7fc4fc009e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc507fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57520/0 (socket says 172.21.15.33:57520) 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc507fff640 1 -- 172.21.15.33:0/2293406277 learned_addr learned my addr 172.21.15.33:0/2293406277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc5077fe640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc50cfa5640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc507fff640 1 -- 172.21.15.33:0/2293406277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 msgr2=0x55b56fcc9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.219+0000 7fc507fff640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc507fff640 1 -- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 msgr2=0x55b56fcc8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc507fff640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc507fff640 1 -- 172.21.15.33:0/2293406277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b56fcb6950 con 0x55b56fcb80a0 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc5077fe640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc507fff640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc8a00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80059f0 tx=0x7fc4f80186a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e118709112ae75f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc4e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc4e77fe640 1 -- 172.21.15.33:0/2293406277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f8028030 con 0x55b56fcb80a0 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc4e77fe640 1 -- 172.21.15.33:0/2293406277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc4f8018b00 con 0x55b56fcb80a0 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4f801f040 con 0x55b56fcb80a0 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc4e77fe640 1 -- 172.21.15.33:0/2293406277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7fc4f8018d40 con 0x55b56fcb80a0 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc510ddd740 10 client.17613 fetch_fsmap learned FSMap version 58 2024-09-10T21:15:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55b56fd41390 con 0x55b56fcb80a0 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.220+0000 7fc4e77fe640 1 -- 172.21.15.33:0/2293406277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4f801f040 con 0x55b56fcb80a0 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc4e77fe640 1 -- 172.21.15.33:0/2293406277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 58) ==== 8362+0+0 (secure 0 0 0) 0x7fc4f8018070 con 0x55b56fcb80a0 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc510ddd740 10 client.17613 fetch_fsmap finished waiting for FSMap version 58 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc510ddd740 10 client.17613 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc510ddd740 20 client.17613 populate_metadata read hostname 'smithi033' 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fd417a0 0x55b56fd43bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc510ddd740 4 client.17613 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b56fc2c850 con 0x55b56fd417a0 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc5077fe640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fd417a0 0x55b56fd43bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc5077fe640 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fd417a0 0x55b56fd43bc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0a8b488e5290d00 server_cookie=788cc55361ac8889 in_seq=0 out_seq=0 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.221+0000 7fc4e77fe640 10 client.17613 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc4e77fe640 1 -- 172.21.15.33:0/2293406277 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55b56fc2c850 con 0x55b56fd417a0 2024-09-10T21:15:41.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc4e77fe640 10 client.17613 handle_command_reply: tid=0 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 client.17613 shutdown 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 msgr2=0x55b56fcc8a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc8a00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80059f0 tx=0x7fc4f80186a0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 shutdown_connections 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b56fd417a0 0x55b56fd43bc0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b56fcc4950 0x55b56fcc9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b56fcb89c0 0x55b56fcc8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 --2- 172.21.15.33:0/2293406277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b56fcb80a0 0x55b56fcc8a00 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 >> 172.21.15.33:0/2293406277 conn(0x55b56fcb5ed0 msgr2=0x55b56fcc2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc4e77fe640 0 client.17613 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 shutdown_connections 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 1 -- 172.21.15.33:0/2293406277 wait complete. 2024-09-10T21:15:41.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.222+0000 7fc510ddd740 20 client.17613 trim_cache size 0 max 16384 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e", 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:15:41.233 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:15:41.236 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.237 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.239 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:15:41.242 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:15:41.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4f00b5360 msgr2=0x7fc4f00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4f00b5360 0x7fc4f00b7820 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc009580 tx=0x7fc4fc009b40 comp rx=0 tx=0).stop 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 msgr2=0x7fc50819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50819eb40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc500005cc0 tx=0x7fc500031700 comp rx=0 tx=0).stop 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 shutdown_connections 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc4f00b5360 0x7fc4f00b7820 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc50810d100 0x7fc50819eb40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5081045c0 0x7fc5081997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 --2- 172.21.15.33:0/391820338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc508103ca0 0x7fc508199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 >> 172.21.15.33:0/391820338 conn(0x7fc5080fd700 msgr2=0x7fc50810f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 shutdown_connections 2024-09-10T21:15:41.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:15:41.223+0000 7fc50ea08640 1 -- 172.21.15.33:0/391820338 wait complete. 2024-09-10T21:15:41.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 884825 6.67 MB/sec execute 522 sec latency 40.210 ms 2024-09-10T21:15:41.923 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:41 smithi033 ceph-mon[30860]: pgmap v697: 97 pgs: 97 active+clean; 336 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-10T21:15:41.952 INFO:tasks.cephfs.filesystem:scrub status for tag:e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e - {'path': '/', 'tag': 'e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e', 'options': 'recursive,force'} 2024-09-10T21:15:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:41 smithi104 ceph-mon[35325]: pgmap v697: 97 pgs: 97 active+clean; 336 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-10T21:15:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:41 smithi174 ceph-mon[35635]: pgmap v697: 97 pgs: 97 active+clean; 336 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 307 op/s 2024-09-10T21:15:42.753 INFO:tasks.workunit.client.0.smithi033.stdout: 1 887345 6.69 MB/sec execute 523 sec latency 12.614 ms 2024-09-10T21:15:43.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 888616 6.69 MB/sec execute 524 sec latency 16.834 ms 2024-09-10T21:15:44.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:43 smithi104 ceph-mon[35325]: pgmap v698: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 315 op/s 2024-09-10T21:15:44.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:43 smithi174 ceph-mon[35635]: pgmap v698: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 315 op/s 2024-09-10T21:15:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:43 smithi033 ceph-mon[30860]: pgmap v698: 97 pgs: 97 active+clean; 337 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 315 op/s 2024-09-10T21:15:44.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 891074 6.70 MB/sec execute 525 sec latency 12.154 ms 2024-09-10T21:15:45.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 892264 6.70 MB/sec execute 526 sec latency 12.299 ms 2024-09-10T21:15:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:45 smithi104 ceph-mon[35325]: pgmap v699: 97 pgs: 97 active+clean; 341 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 342 op/s 2024-09-10T21:15:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:45 smithi174 ceph-mon[35635]: pgmap v699: 97 pgs: 97 active+clean; 341 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 342 op/s 2024-09-10T21:15:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:45 smithi033 ceph-mon[30860]: pgmap v699: 97 pgs: 97 active+clean; 341 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.1 MiB/s wr, 342 op/s 2024-09-10T21:15:46.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 894484 6.71 MB/sec execute 527 sec latency 55.387 ms 2024-09-10T21:15:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:47.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 895730 6.71 MB/sec execute 528 sec latency 20.488 ms 2024-09-10T21:15:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:47 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:15:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:47 smithi174 ceph-mon[35635]: pgmap v700: 97 pgs: 97 active+clean; 342 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-09-10T21:15:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:47 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:15:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:47 smithi033 ceph-mon[30860]: pgmap v700: 97 pgs: 97 active+clean; 342 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-09-10T21:15:48.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:47 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:15:48.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:47 smithi104 ceph-mon[35325]: pgmap v700: 97 pgs: 97 active+clean; 342 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 297 op/s 2024-09-10T21:15:48.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 898208 6.72 MB/sec execute 529 sec latency 13.164 ms 2024-09-10T21:15:49.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 899335 6.72 MB/sec execute 530 sec latency 14.458 ms 2024-09-10T21:15:50.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:49 smithi174 ceph-mon[35635]: pgmap v701: 97 pgs: 97 active+clean; 342 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 350 op/s 2024-09-10T21:15:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:49 smithi033 ceph-mon[30860]: pgmap v701: 97 pgs: 97 active+clean; 342 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 350 op/s 2024-09-10T21:15:50.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:49 smithi104 ceph-mon[35325]: pgmap v701: 97 pgs: 97 active+clean; 342 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.9 MiB/s wr, 350 op/s 2024-09-10T21:15:50.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 901742 6.73 MB/sec execute 531 sec latency 13.984 ms 2024-09-10T21:15:51.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 902727 6.73 MB/sec execute 532 sec latency 36.584 ms 2024-09-10T21:15:52.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:51 smithi174 ceph-mon[35635]: pgmap v702: 97 pgs: 97 active+clean; 342 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-10T21:15:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:51 smithi033 ceph-mon[30860]: pgmap v702: 97 pgs: 97 active+clean; 342 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-10T21:15:52.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:51 smithi104 ceph-mon[35325]: pgmap v702: 97 pgs: 97 active+clean; 342 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.6 MiB/s wr, 300 op/s 2024-09-10T21:15:52.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 905104 6.74 MB/sec execute 533 sec latency 14.685 ms 2024-09-10T21:15:53.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 906328 6.74 MB/sec execute 534 sec latency 14.652 ms 2024-09-10T21:15:54.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:53 smithi174 ceph-mon[35635]: pgmap v703: 97 pgs: 97 active+clean; 347 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-10T21:15:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:53 smithi033 ceph-mon[30860]: pgmap v703: 97 pgs: 97 active+clean; 347 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-10T21:15:54.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:53 smithi104 ceph-mon[35325]: pgmap v703: 97 pgs: 97 active+clean; 347 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 305 op/s 2024-09-10T21:15:54.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 908818 6.76 MB/sec execute 535 sec latency 14.060 ms 2024-09-10T21:15:55.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 909975 6.75 MB/sec execute 536 sec latency 15.326 ms 2024-09-10T21:15:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:55 smithi174 ceph-mon[35635]: pgmap v704: 97 pgs: 97 active+clean; 348 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T21:15:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:55 smithi033 ceph-mon[30860]: pgmap v704: 97 pgs: 97 active+clean; 348 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T21:15:56.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:55 smithi104 ceph-mon[35325]: pgmap v704: 97 pgs: 97 active+clean; 348 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.0 MiB/s wr, 317 op/s 2024-09-10T21:15:56.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 912326 6.77 MB/sec execute 537 sec latency 40.831 ms 2024-09-10T21:15:57.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 913565 6.76 MB/sec execute 538 sec latency 16.195 ms 2024-09-10T21:15:58.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:57 smithi174 ceph-mon[35635]: pgmap v705: 97 pgs: 97 active+clean; 349 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-09-10T21:15:58.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:57 smithi033 ceph-mon[30860]: pgmap v705: 97 pgs: 97 active+clean; 349 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-09-10T21:15:58.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:57 smithi104 ceph-mon[35325]: pgmap v705: 97 pgs: 97 active+clean; 349 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.5 MiB/s wr, 295 op/s 2024-09-10T21:15:58.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 916130 6.78 MB/sec execute 539 sec latency 13.069 ms 2024-09-10T21:15:59.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 916993 6.79 MB/sec execute 540 sec latency 48.377 ms 2024-09-10T21:16:00.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:15:59 smithi174 ceph-mon[35635]: pgmap v706: 97 pgs: 97 active+clean; 351 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-09-10T21:16:00.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:15:59 smithi033 ceph-mon[30860]: pgmap v706: 97 pgs: 97 active+clean; 351 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-09-10T21:16:00.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:15:59 smithi104 ceph-mon[35325]: pgmap v706: 97 pgs: 97 active+clean; 351 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.9 MiB/s wr, 322 op/s 2024-09-10T21:16:00.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 919447 6.81 MB/sec execute 541 sec latency 10.706 ms 2024-09-10T21:16:01.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 920860 6.82 MB/sec execute 542 sec latency 42.922 ms 2024-09-10T21:16:02.114 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:01 smithi104 ceph-mon[35325]: pgmap v707: 97 pgs: 97 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-09-10T21:16:02.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:01 smithi174 ceph-mon[35635]: pgmap v707: 97 pgs: 97 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-09-10T21:16:02.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:01 smithi033 ceph-mon[30860]: pgmap v707: 97 pgs: 97 active+clean; 350 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 283 op/s 2024-09-10T21:16:02.754 INFO:tasks.workunit.client.0.smithi033.stdout: 1 923236 6.83 MB/sec execute 543 sec latency 15.571 ms 2024-09-10T21:16:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:02 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:03.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:02 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:03.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:02 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:03.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 923595 6.82 MB/sec execute 544 sec latency 884.277 ms 2024-09-10T21:16:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:03 smithi174 ceph-mon[35635]: pgmap v708: 97 pgs: 97 active+clean; 349 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-09-10T21:16:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:03 smithi033 ceph-mon[30860]: pgmap v708: 97 pgs: 97 active+clean; 349 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-09-10T21:16:04.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:03 smithi104 ceph-mon[35325]: pgmap v708: 97 pgs: 97 active+clean; 349 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.7 MiB/s wr, 295 op/s 2024-09-10T21:16:04.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 924029 6.81 MB/sec execute 545 sec latency 1604.583 ms 2024-09-10T21:16:05.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 925601 6.81 MB/sec execute 546 sec latency 16.599 ms 2024-09-10T21:16:06.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:05 smithi174 ceph-mon[35635]: pgmap v709: 97 pgs: 97 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 323 op/s 2024-09-10T21:16:06.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:05 smithi033 ceph-mon[30860]: pgmap v709: 97 pgs: 97 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 323 op/s 2024-09-10T21:16:06.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:05 smithi104 ceph-mon[35325]: pgmap v709: 97 pgs: 97 active+clean; 347 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.0 MiB/s wr, 323 op/s 2024-09-10T21:16:06.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 927646 6.82 MB/sec execute 547 sec latency 28.335 ms 2024-09-10T21:16:07.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 929687 6.82 MB/sec execute 548 sec latency 28.554 ms 2024-09-10T21:16:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:07 smithi174 ceph-mon[35635]: pgmap v710: 97 pgs: 97 active+clean; 343 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-09-10T21:16:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:07 smithi033 ceph-mon[30860]: pgmap v710: 97 pgs: 97 active+clean; 343 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-09-10T21:16:08.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:07 smithi104 ceph-mon[35325]: pgmap v710: 97 pgs: 97 active+clean; 343 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 280 op/s 2024-09-10T21:16:08.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 931310 6.83 MB/sec execute 549 sec latency 12.188 ms 2024-09-10T21:16:09.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 933491 6.84 MB/sec execute 550 sec latency 15.023 ms 2024-09-10T21:16:10.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:09 smithi174 ceph-mon[35635]: pgmap v711: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-09-10T21:16:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:09 smithi033 ceph-mon[30860]: pgmap v711: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-09-10T21:16:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:09 smithi104 ceph-mon[35325]: pgmap v711: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 308 op/s 2024-09-10T21:16:10.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 935051 6.84 MB/sec execute 551 sec latency 44.946 ms 2024-09-10T21:16:11.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 937492 6.85 MB/sec execute 552 sec latency 14.296 ms 2024-09-10T21:16:11.953 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:16:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:11 smithi174 ceph-mon[35635]: pgmap v712: 97 pgs: 97 active+clean; 338 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T21:16:12.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:11 smithi033 ceph-mon[30860]: pgmap v712: 97 pgs: 97 active+clean; 338 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T21:16:12.227 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:12.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:11 smithi104 ceph-mon[35325]: pgmap v712: 97 pgs: 97 active+clean; 338 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-09-10T21:16:12.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 938925 6.86 MB/sec execute 553 sec latency 45.240 ms 2024-09-10T21:16:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.044+0000 7f4578566640 1 -- 172.21.15.33:0/832234317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f457010d120 msgr2=0x7f457010f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.044+0000 7f4578566640 1 --2- 172.21.15.33:0/832234317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f457010d120 0x7f457010f580 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4568009ca0 tx=0x7f456802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- 172.21.15.33:0/832234317 shutdown_connections 2024-09-10T21:16:13.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 --2- 172.21.15.33:0/832234317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f457010d120 0x7f457010f580 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 --2- 172.21.15.33:0/832234317 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701045e0 0x7f457010cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 --2- 172.21.15.33:0/832234317 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4570103cc0 0x7f45701040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- 172.21.15.33:0/832234317 >> 172.21.15.33:0/832234317 conn(0x7f45700fd740 msgr2=0x7f45700ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- 172.21.15.33:0/832234317 shutdown_connections 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- 172.21.15.33:0/832234317 wait complete. 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 Processor -- start 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- start start 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570103cc0 0x7f4570199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701045e0 0x7f4570199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457010d120 0x7f457019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4570112de0 con 0x7f4570103cc0 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4570112c60 con 0x7f45701045e0 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.045+0000 7f4578566640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4570112f60 con 0x7f457010d120 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570103cc0 0x7f4570199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570103cc0 0x7f4570199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36828/0 (socket says 172.21.15.33:36828) 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 -- 172.21.15.33:0/3040148012 learned_addr learned my addr 172.21.15.33:0/3040148012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4575b01640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701045e0 0x7f4570199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576b03640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457010d120 0x7f457019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 -- 172.21.15.33:0/3040148012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457010d120 msgr2=0x7f457019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457010d120 0x7f457019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 -- 172.21.15.33:0/3040148012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701045e0 msgr2=0x7f4570199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701045e0 0x7f4570199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 -- 172.21.15.33:0/3040148012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457019f3e0 con 0x7f4570103cc0 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4575b01640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701045e0 0x7f4570199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4576302640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570103cc0 0x7f4570199330 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9f0 tx=0x7f456c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f45677fe640 1 -- 172.21.15.33:0/3040148012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c00cd80 con 0x7f4570103cc0 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457019f6d0 con 0x7f4570103cc0 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f45677fe640 1 -- 172.21.15.33:0/3040148012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456c0054d0 con 0x7f4570103cc0 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f45677fe640 1 -- 172.21.15.33:0/3040148012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c010690 con 0x7f4570103cc0 2024-09-10T21:16:13.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.046+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701a5d90 con 0x7f4570103cc0 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.049+0000 7f457a93b740 1 Processor -- start 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f457a93b740 1 -- start start 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f45677fe640 1 -- 172.21.15.33:0/3040148012 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f456c004e90 con 0x7f4570103cc0 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f457a93b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e4b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f457a93b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e5bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f457a93b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e5e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f457a93b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558671dc1e50 con 0x558671e4b480 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f457a93b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558671db1280 con 0x558671e4bda0 2024-09-10T21:16:13.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f457a93b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558671db5090 con 0x558671e5c140 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f4576b03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e4b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f4576b03640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e4b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36844/0 (socket says 172.21.15.33:36844) 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f4576b03640 1 -- 172.21.15.33:0/3049740749 learned_addr learned my addr 172.21.15.33:0/3049740749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f4576302640 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e5bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.050+0000 7f4575b01640 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e5e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4575b01640 1 -- 172.21.15.33:0/3049740749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 msgr2=0x558671e5bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4575b01640 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e5bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4575b01640 1 -- 172.21.15.33:0/3049740749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 msgr2=0x558671e4b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4575b01640 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e4b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4575b01640 1 -- 172.21.15.33:0/3049740749 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558671b0f650 con 0x558671e5c140 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4575b01640 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e5e530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4560009f10 tx=0x7f4560011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5282ed76595da2fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4564ff9640 1 -- 172.21.15.33:0/3049740749 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600042f0 con 0x558671e5c140 2024-09-10T21:16:13.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4564ff9640 1 -- 172.21.15.33:0/3049740749 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4560004490 con 0x558671e5c140 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4564ff9640 1 -- 172.21.15.33:0/3049740749 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45600035f0 con 0x558671e5c140 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f45677fe640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f455c0b52e0 0x7f455c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.051+0000 7f4576b03640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f455c0b52e0 0x7f455c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f45677fe640 1 -- 172.21.15.33:0/3040148012 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f456c0e5fe0 con 0x7f4570103cc0 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 -- 172.21.15.33:0/3049740749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 msgr2=0x558671e5e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e5e530 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4560009f10 tx=0x7f4560011810 comp rx=0 tx=0).stop 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 -- 172.21.15.33:0/3049740749 shutdown_connections 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e5e530 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e5bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 --2- 172.21.15.33:0/3049740749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e4b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 -- 172.21.15.33:0/3049740749 >> 172.21.15.33:0/3049740749 conn(0x558671e492b0 msgr2=0x558671e4a450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f4576b03640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f455c0b52e0 0x7f455c0b77a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4568009a60 tx=0x7f4568005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 -- 172.21.15.33:0/3049740749 shutdown_connections 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.052+0000 7f457a93b740 1 -- 172.21.15.33:0/3049740749 wait complete. 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 -- start start 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 Processor -- start 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4b480 0x558671e5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4bda0 0x558671e58e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558671c12700 con 0x558671e4bda0 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558671d4c110 con 0x558671e5c140 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558671e5edf0 con 0x558671e4b480 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f457a93b740 10 client.0 _ll_register_callbacks cb 0x558671d65990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4575b01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4bda0 0x558671e58e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4b480 0x558671e5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4b480 0x558671e5ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49844/0 (socket says 172.21.15.33:49844) 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4575b01640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4bda0 0x558671e58e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36856/0 (socket says 172.21.15.33:36856) 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 -- 172.21.15.33:0/4134737370 learned_addr learned my addr 172.21.15.33:0/4134737370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576b03640 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 -- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 msgr2=0x558671e594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 -- 172.21.15.33:0/4134737370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4bda0 msgr2=0x558671e58e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4bda0 0x558671e58e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.053+0000 7f4576302640 1 -- 172.21.15.33:0/4134737370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558671d79c40 con 0x558671e4b480 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f4576302640 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4b480 0x558671e5ba20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f456c0e9800 tx=0x7f456c0048f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63d605dae0de2f4f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f4559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f4559ffb640 1 -- 172.21.15.33:0/4134737370 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c0ac570 con 0x558671e4b480 2024-09-10T21:16:13.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f4559ffb640 1 -- 172.21.15.33:0/4134737370 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f456c0aca00 con 0x558671e4b480 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4560004490 con 0x558671e4b480 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f4559ffb640 1 -- 172.21.15.33:0/4134737370 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f456c0abf10 con 0x558671e4b480 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f457a93b740 10 client.27019 fetch_fsmap learned FSMap version 59 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558671e6a770 con 0x558671e4b480 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f4559ffb640 1 -- 172.21.15.33:0/4134737370 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4560004490 con 0x558671e4b480 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f4559ffb640 1 -- 172.21.15.33:0/4134737370 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f456c0b58a0 con 0x558671e4b480 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f457a93b740 10 client.27019 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.054+0000 7f457a93b740 10 client.27019 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.055+0000 7f457a93b740 20 client.27019 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.055+0000 7f457a93b740 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671e6bc30 0x558671e6e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.055+0000 7f457a93b740 4 client.27019 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.055+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558671dc3da0 con 0x558671e6bc30 2024-09-10T21:16:13.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.055+0000 7f4575b01640 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671e6bc30 0x558671e6e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.055+0000 7f4575b01640 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671e6bc30 0x558671e6e0d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8566323d5b421102 server_cookie=2f799df174faf04e in_seq=0 out_seq=0 2024-09-10T21:16:13.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.055+0000 7f4559ffb640 10 client.27019 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f4559ffb640 1 -- 172.21.15.33:0/4134737370 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558671dc3da0 con 0x558671e6bc30 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f4559ffb640 10 client.27019 handle_command_reply: tid=0 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 client.27019 shutdown 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4b480 msgr2=0x558671e5ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4b480 0x558671e5ba20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f456c0e9800 tx=0x7f456c0048f0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 shutdown_connections 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671e6bc30 0x558671e6e0d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4bda0 0x558671e58e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 --2- 172.21.15.33:0/4134737370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4b480 0x558671e5ba20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 >> 172.21.15.33:0/4134737370 conn(0x558671e492b0 msgr2=0x558671e4ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f4559ffb640 0 client.27019 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 shutdown_connections 2024-09-10T21:16:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.056+0000 7f457a93b740 1 -- 172.21.15.33:0/4134737370 wait complete. 2024-09-10T21:16:13.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.057+0000 7f457a93b740 20 client.27019 trim_cache size 0 max 16384 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 Processor -- start 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 -- start start 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e68080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4bda0 0x558671e739a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e73ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ac210 con 0x558671e4b480 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f45701add50 con 0x558671e5c140 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f457a93b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ac8e0 con 0x558671e4bda0 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4576302640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e68080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4575b01640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4bda0 0x558671e739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4576302640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e68080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36860/0 (socket says 172.21.15.33:36860) 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4576302640 1 -- 172.21.15.33:0/1349206051 learned_addr learned my addr 172.21.15.33:0/1349206051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4576b03640 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e73ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4575b01640 1 -- 172.21.15.33:0/1349206051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 msgr2=0x558671e73ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4575b01640 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e73ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4575b01640 1 -- 172.21.15.33:0/1349206051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 msgr2=0x558671e68080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4575b01640 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e68080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4575b01640 1 -- 172.21.15.33:0/1349206051 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558671e59810 con 0x558671e4bda0 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.075+0000 7f4576302640 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e68080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.076+0000 7f4575b01640 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4bda0 0x558671e739a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4560009650 tx=0x7f4560004bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=575aaf2c27dbc891 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.076+0000 7f455bfff640 1 -- 172.21.15.33:0/1349206051 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560032070 con 0x558671e4bda0 2024-09-10T21:16:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.076+0000 7f455bfff640 1 -- 172.21.15.33:0/1349206051 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456002b070 con 0x558671e4bda0 2024-09-10T21:16:13.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.076+0000 7f455bfff640 1 -- 172.21.15.33:0/1349206051 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456001e040 con 0x558671e4bda0 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- 172.21.15.33:0/1349206051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4bda0 msgr2=0x558671e739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4bda0 0x558671e739a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4560009650 tx=0x7f4560004bd0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- 172.21.15.33:0/1349206051 shutdown_connections 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e5c140 0x558671e73ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e4bda0 0x558671e739a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 --2- 172.21.15.33:0/1349206051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e68080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- 172.21.15.33:0/1349206051 >> 172.21.15.33:0/1349206051 conn(0x558671e492b0 msgr2=0x558671e5acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- 172.21.15.33:0/1349206051 shutdown_connections 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- 172.21.15.33:0/1349206051 wait complete. 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- start start 2024-09-10T21:16:13.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 Processor -- start 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e64a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e61240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f45701ad680 con 0x558671e4b480 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f45701acfb0 con 0x558671e4bda0 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45701aa1d0 con 0x558671e5c140 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.077+0000 7f457a93b740 10 client.0 _ll_register_callbacks cb 0x7f456c0e9750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4576302640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e64a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4576b03640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e61240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e64a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:35396/0 (socket says 172.21.15.33:35396) 2024-09-10T21:16:13.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 -- 172.21.15.33:0/2877008548 learned_addr learned my addr 172.21.15.33:0/2877008548 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4576302640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e644e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36864/0 (socket says 172.21.15.33:36864) 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 -- 172.21.15.33:0/2877008548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 msgr2=0x558671e61240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e61240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 -- 172.21.15.33:0/2877008548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 msgr2=0x558671e644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 -- 172.21.15.33:0/2877008548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558671e61a70 con 0x558671e4bda0 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4576b03640 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4576302640 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4575b01640 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e64a60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4560005e00 tx=0x7f4560005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b082f64a09ecffd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4559ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4559ffb640 1 -- 172.21.15.33:0/2877008548 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560032070 con 0x558671e4bda0 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f4559ffb640 1 -- 172.21.15.33:0/2877008548 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f45600037f0 con 0x558671e4bda0 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.078+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f456002b070 con 0x558671e4bda0 2024-09-10T21:16:13.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f4559ffb640 1 -- 172.21.15.33:0/2877008548 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f45600039f0 con 0x558671e4bda0 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 10 client.27074 fetch_fsmap learned FSMap version 59 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x558671ed9870 con 0x558671e4bda0 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f4559ffb640 1 -- 172.21.15.33:0/2877008548 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456002b070 con 0x558671e4bda0 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f4559ffb640 1 -- 172.21.15.33:0/2877008548 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7f456003f050 con 0x558671e4bda0 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 10 client.27074 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 10 client.27074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 20 client.27074 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671ed9d80 0x558671edc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 4 client.27074 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558671dbf850 con 0x558671ed9d80 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.079+0000 7f4576302640 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671ed9d80 0x558671edc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f4576302640 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671ed9d80 0x558671edc1a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bb393ac667b5a06 server_cookie=ca601b262dafab0f in_seq=0 out_seq=0 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f4559ffb640 10 client.27074 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f4559ffb640 1 -- 172.21.15.33:0/2877008548 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x558671dbf850 con 0x558671ed9d80 2024-09-10T21:16:13.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f4559ffb640 10 client.27074 handle_command_reply: tid=0 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 client.27074 shutdown 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 msgr2=0x558671e64a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e64a60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4560005e00 tx=0x7f4560005c50 comp rx=0 tx=0).stop 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 shutdown_connections 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558671ed9d80 0x558671edc1a0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558671e5c140 0x558671e61240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f4559ffb640 0 client.27074 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558671e4bda0 0x558671e64a60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 --2- 172.21.15.33:0/2877008548 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558671e4b480 0x558671e644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 >> 172.21.15.33:0/2877008548 conn(0x558671e492b0 msgr2=0x558671e5a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 shutdown_connections 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.080+0000 7f457a93b740 1 -- 172.21.15.33:0/2877008548 wait complete. 2024-09-10T21:16:13.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f457a93b740 20 client.27074 trim_cache size 0 max 16384 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.091 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.092 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.093 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.094 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.095 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.096 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.097 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:16:13.098 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:16:13.099 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:16:13.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f455c0b52e0 msgr2=0x7f455c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f455c0b52e0 0x7f455c0b77a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4568009a60 tx=0x7f4568005d80 comp rx=0 tx=0).stop 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570103cc0 msgr2=0x7f4570199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570103cc0 0x7f4570199330 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f456c00e9f0 tx=0x7f456c00edf0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 shutdown_connections 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f455c0b52e0 0x7f455c0b77a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f457010d120 0x7f457019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701045e0 0x7f4570199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 --2- 172.21.15.33:0/3040148012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570103cc0 0x7f4570199330 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 >> 172.21.15.33:0/3040148012 conn(0x7f45700fd740 msgr2=0x7f45700fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 shutdown_connections 2024-09-10T21:16:13.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:13.081+0000 7f4578566640 1 -- 172.21.15.33:0/3040148012 wait complete. 2024-09-10T21:16:13.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 941670 6.87 MB/sec execute 554 sec latency 17.204 ms 2024-09-10T21:16:13.893 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:13 smithi033 ceph-mon[30860]: pgmap v713: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-10T21:16:13.922 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:16:13.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:16:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:13 smithi174 ceph-mon[35635]: pgmap v713: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-10T21:16:14.199 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:13 smithi104 ceph-mon[35325]: pgmap v713: 97 pgs: 97 active+clean; 339 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-10T21:16:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3235727913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74108020 msgr2=0x7fdb7410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:14.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3235727913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74108020 0x7fdb7410e820 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fdb6c009ca0 tx=0x7fdb6c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3235727913 shutdown_connections 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3235727913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74108020 0x7fdb7410e820 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3235727913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb741076a0 0x7fdb74107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3235727913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb74101c90 0x7fdb74102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3235727913 >> 172.21.15.33:0/3235727913 conn(0x7fdb740fd700 msgr2=0x7fdb740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3235727913 shutdown_connections 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.706+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3235727913 wait complete. 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 Processor -- start 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 -- start start 2024-09-10T21:16:14.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74101c90 0x7fdb741992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb741076a0 0x7fdb74199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb74108020 0x7fdb7419eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74111ce0 con 0x7fdb74101c90 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74111b60 con 0x7fdb74108020 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb7b7de640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb74111e60 con 0x7fdb741076a0 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.707+0000 7fdb78d79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb741076a0 0x7fdb74199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb741076a0 0x7fdb74199800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49884/0 (socket says 172.21.15.33:49884) 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 -- 172.21.15.33:0/3707881343 learned_addr learned my addr 172.21.15.33:0/3707881343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb79d7b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb74108020 0x7fdb7419eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:14.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb7957a640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74101c90 0x7fdb741992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 -- 172.21.15.33:0/3707881343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb74108020 msgr2=0x7fdb7419eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb74108020 0x7fdb7419eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 -- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74101c90 msgr2=0x7fdb741992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74101c90 0x7fdb741992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 -- 172.21.15.33:0/3707881343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7419f370 con 0x7fdb741076a0 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb78d79640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb741076a0 0x7fdb74199800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400ea60 tx=0x7fdb6400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb6a7fc640 1 -- 172.21.15.33:0/3707881343 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6400cde0 con 0x7fdb741076a0 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7419f660 con 0x7fdb741076a0 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb741a5ef0 con 0x7fdb741076a0 2024-09-10T21:16:14.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb6a7fc640 1 -- 172.21.15.33:0/3707881343 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb640035f0 con 0x7fdb741076a0 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.708+0000 7fdb6a7fc640 1 -- 172.21.15.33:0/3707881343 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64018690 con 0x7fdb741076a0 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.709+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb74102070 con 0x7fdb741076a0 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.711+0000 7fdb6a7fc640 1 -- 172.21.15.33:0/3707881343 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdb64003a10 con 0x7fdb741076a0 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.712+0000 7fdb6a7fc640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb600b52e0 0x7fdb600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.712+0000 7fdb6a7fc640 1 -- 172.21.15.33:0/3707881343 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdb640e5ef0 con 0x7fdb741076a0 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.712+0000 7fdb7957a640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb600b52e0 0x7fdb600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.712+0000 7fdb7957a640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb600b52e0 0x7fdb600b77a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009bd0 tx=0x7fdb70005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:14.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.715+0000 7fdb6a7fc640 1 -- 172.21.15.33:0/3707881343 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fdb640ace00 con 0x7fdb741076a0 2024-09-10T21:16:14.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 943389 6.87 MB/sec execute 555 sec latency 20.557 ms 2024-09-10T21:16:14.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.958+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb7410ae10 con 0x7fdb741076a0 2024-09-10T21:16:14.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.958+0000 7fdb6a7fc640 1 -- 172.21.15.33:0/3707881343 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+11974 (secure 0 0 0) 0x7fdb640ac700 con 0x7fdb741076a0 2024-09-10T21:16:14.963 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:16:14.963 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":59,"btime":"2024-09-10T21:16:01:824218+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:16:01.765675+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:16:14.964 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 59 2024-09-10T21:16:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb600b52e0 msgr2=0x7fdb600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb600b52e0 0x7fdb600b77a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdb70009bd0 tx=0x7fdb70005f90 comp rx=0 tx=0).stop 2024-09-10T21:16:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb741076a0 msgr2=0x7fdb74199800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:14.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb741076a0 0x7fdb74199800 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fdb6400ea60 tx=0x7fdb6400ef20 comp rx=0 tx=0).stop 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 shutdown_connections 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb600b52e0 0x7fdb600b77a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb74108020 0x7fdb7419eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb741076a0 0x7fdb74199800 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 --2- 172.21.15.33:0/3707881343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb74101c90 0x7fdb741992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 >> 172.21.15.33:0/3707881343 conn(0x7fdb740fd700 msgr2=0x7fdb7406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 shutdown_connections 2024-09-10T21:16:14.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:14.961+0000 7fdb7b7de640 1 -- 172.21.15.33:0/3707881343 wait complete. 2024-09-10T21:16:15.683 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:16:15.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 945499 6.89 MB/sec execute 556 sec latency 17.180 ms 2024-09-10T21:16:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:15 smithi033 ceph-mon[30860]: pgmap v714: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 3.1 MiB/s wr, 334 op/s 2024-09-10T21:16:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:15 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3707881343' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:16:15.947 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:15 smithi174 ceph-mon[35635]: pgmap v714: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 3.1 MiB/s wr, 334 op/s 2024-09-10T21:16:16.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:15 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3707881343' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:16:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:15 smithi104 ceph-mon[35325]: pgmap v714: 97 pgs: 97 active+clean; 342 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 869 KiB/s rd, 3.1 MiB/s wr, 334 op/s 2024-09-10T21:16:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:15 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3707881343' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:16:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.441+0000 7fb5f5664640 1 -- 172.21.15.33:0/1875976708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f01087f0 msgr2=0x7fb5f0108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.441+0000 7fb5f5664640 1 --2- 172.21.15.33:0/1875976708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f01087f0 0x7fb5f0108c30 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009900 tx=0x7fb5d802a410 comp rx=0 tx=0).stop 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 -- 172.21.15.33:0/1875976708 shutdown_connections 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 --2- 172.21.15.33:0/1875976708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5f0109170 0x7fb5f010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 --2- 172.21.15.33:0/1875976708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f01087f0 0x7fb5f0108c30 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 --2- 172.21.15.33:0/1875976708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5f0102de0 0x7fb5f01031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 -- 172.21.15.33:0/1875976708 >> 172.21.15.33:0/1875976708 conn(0x7fb5f00fe830 msgr2=0x7fb5f0100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 -- 172.21.15.33:0/1875976708 shutdown_connections 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 -- 172.21.15.33:0/1875976708 wait complete. 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 Processor -- start 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.442+0000 7fb5f5664640 1 -- start start 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5f5664640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5f0102de0 0x7fb5f019b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5f5664640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5f01087f0 0x7fb5f019ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5f5664640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f0109170 0x7fb5f01a0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5f5664640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0112e30 con 0x7fb5f0109170 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5f5664640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0112cb0 con 0x7fb5f01087f0 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5f5664640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f0112fb0 con 0x7fb5f0102de0 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f0109170 0x7fb5f01a0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5f0102de0 0x7fb5f019b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f0109170 0x7fb5f01a0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36876/0 (socket says 172.21.15.33:36876) 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/692916845 learned_addr learned my addr 172.21.15.33:0/692916845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5f01087f0 0x7fb5f019ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/692916845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5f0102de0 msgr2=0x7fb5f019b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5f0102de0 0x7fb5f019b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/692916845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5f01087f0 msgr2=0x7fb5f019ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5f01087f0 0x7fb5f019ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/692916845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5f01a15a0 con 0x7fb5f0109170 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5f01087f0 0x7fb5f019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.443+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f0109170 0x7fb5f01a0dd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e970 tx=0x7fb5e800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.444+0000 7fb5e7fff640 1 -- 172.21.15.33:0/692916845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800ccc0 con 0x7fb5f0109170 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.444+0000 7fb5e7fff640 1 -- 172.21.15.33:0/692916845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e800ce60 con 0x7fb5f0109170 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.444+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5f01a1890 con 0x7fb5f0109170 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.444+0000 7fb5e7fff640 1 -- 172.21.15.33:0/692916845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8010430 con 0x7fb5f0109170 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.444+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5f01a7f50 con 0x7fb5f0109170 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.447+0000 7fb5f7a39740 1 Processor -- start 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.447+0000 7fb5e7fff640 1 -- 172.21.15.33:0/692916845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb5e8004ff0 con 0x7fb5f0109170 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.447+0000 7fb5f7a39740 1 -- start start 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.447+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x5580437518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.447+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.447+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804375dd80 0x558043760170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5580436c7ea0 con 0x558043751df0 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5580436b72d0 con 0x55804375dd80 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580436bb0e0 con 0x5580437514d0 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x5580437518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x5580437518b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49912/0 (socket says 172.21.15.33:49912) 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5ee7fc640 1 -- 172.21.15.33:0/1242993292 learned_addr learned my addr 172.21.15.33:0/1242993292 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804375dd80 0x558043760170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5eeffd640 1 -- 172.21.15.33:0/1242993292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 msgr2=0x5580437518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x5580437518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5eeffd640 1 -- 172.21.15.33:0/1242993292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804375dd80 msgr2=0x558043760170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804375dd80 0x558043760170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5eeffd640 1 -- 172.21.15.33:0/1242993292 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558043415650 con 0x558043751df0 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x5580437518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5e7fff640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5dc0b52e0 0x7fb5dc0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5dc0b52e0 0x7fb5dc0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375d840 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009f80 tx=0x7fb5e00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d85335c11dbf48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5e7fff640 1 -- 172.21.15.33:0/692916845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb5e80e5ff0 con 0x7fb5f0109170 2024-09-10T21:16:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.448+0000 7fb5e57fa640 1 -- 172.21.15.33:0/1242993292 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0004d20 con 0x558043751df0 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.449+0000 7fb5e57fa640 1 -- 172.21.15.33:0/1242993292 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0004050 con 0x558043751df0 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.449+0000 7fb5e57fa640 1 -- 172.21.15.33:0/1242993292 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0005260 con 0x558043751df0 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.449+0000 7fb5f7a39740 1 -- 172.21.15.33:0/1242993292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 msgr2=0x55804375d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.449+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5dc0b52e0 0x7fb5dc0b77a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005ec0 tx=0x7fb5d8005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.449+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375d840 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009f80 tx=0x7fb5e00118d0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- 172.21.15.33:0/1242993292 shutdown_connections 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804375dd80 0x558043760170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375d840 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/1242993292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x5580437518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- 172.21.15.33:0/1242993292 >> 172.21.15.33:0/1242993292 conn(0x55804374f300 msgr2=0x5580437504a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- 172.21.15.33:0/1242993292 shutdown_connections 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- 172.21.15.33:0/1242993292 wait complete. 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- start start 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 Processor -- start 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804375d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x55804375aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804375dd80 0x55804375afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558043518700 con 0x55804375dd80 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558043652160 con 0x558043751df0 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558043760a00 con 0x5580437514d0 2024-09-10T21:16:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5f7a39740 10 client.0 _ll_register_callbacks cb 0x55804366b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804375dd80 0x55804375afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804375d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804375d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49916/0 (socket says 172.21.15.33:49916) 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x55804375aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804375dd80 0x55804375afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36894/0 (socket says 172.21.15.33:36894) 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.450+0000 7fb5eeffd640 1 -- 172.21.15.33:0/2956471604 learned_addr learned my addr 172.21.15.33:0/2956471604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5eeffd640 1 -- 172.21.15.33:0/2956471604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 msgr2=0x55804375aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x55804375aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5eeffd640 1 -- 172.21.15.33:0/2956471604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804375dd80 msgr2=0x55804375afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804375dd80 0x55804375afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5eeffd640 1 -- 172.21.15.33:0/2956471604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55804367fc90 con 0x5580437514d0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804375dd80 0x55804375afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804375d5e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5e001b040 tx=0x7fb5e0004cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8a8b52a157b9655 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2956471604 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0028020 con 0x5580437514d0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2956471604 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb5e0003c40 con 0x5580437514d0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5e0004050 con 0x5580437514d0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2956471604 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb5e0003e80 con 0x5580437514d0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.451+0000 7fb5f7a39740 10 client.27043 fetch_fsmap learned FSMap version 59 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x55804376c310 con 0x5580437514d0 2024-09-10T21:16:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2956471604 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5e0004050 con 0x5580437514d0 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2956471604 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fb5e000f4e0 con 0x5580437514d0 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5f7a39740 10 client.27043 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5f7a39740 10 client.27043 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5f7a39740 20 client.27043 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804376d6c0 0x55804376fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5f7a39740 4 client.27043 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580436c9df0 con 0x55804376d6c0 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.452+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804376d6c0 0x55804376fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804376d6c0 0x55804376fab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dd38124523924bff server_cookie=90a63804deb9205d in_seq=0 out_seq=0 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5d67fc640 10 client.27043 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:16:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2956471604 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5580436c9df0 con 0x55804376d6c0 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5d67fc640 10 client.27043 handle_command_reply: tid=0 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5f7a39740 1 client.27043 shutdown 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 msgr2=0x55804375d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804375d5e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb5e001b040 tx=0x7fb5e0004cf0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.453+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 shutdown_connections 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804376d6c0 0x55804376fab0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804375dd80 0x55804375afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x55804375aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5d67fc640 0 client.27043 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2956471604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804375d5e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 >> 172.21.15.33:0/2956471604 conn(0x55804374f300 msgr2=0x558043754ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 shutdown_connections 2024-09-10T21:16:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2956471604 wait complete. 2024-09-10T21:16:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.454+0000 7fb5f7a39740 20 client.27043 trim_cache size 0 max 16384 2024-09-10T21:16:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.467+0000 7fb5f7a39740 1 Processor -- start 2024-09-10T21:16:16.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.467+0000 7fb5f7a39740 1 -- start start 2024-09-10T21:16:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804374f300 0x558043759f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804374e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f01b0640 con 0x558043751df0 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f01b2180 con 0x55804374f300 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f01b0d10 con 0x5580437514d0 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804374e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804374f300 0x558043759f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36898/0 (socket says 172.21.15.33:36898) 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ee7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804374e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49922/0 (socket says 172.21.15.33:49922) 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/2984306577 learned_addr learned my addr 172.21.15.33:0/2984306577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ee7fc640 1 -- 172.21.15.33:0/2984306577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804374f300 msgr2=0x558043759f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804374f300 0x558043759f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ee7fc640 1 -- 172.21.15.33:0/2984306577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 msgr2=0x55804375f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ee7fc640 1 -- 172.21.15.33:0/2984306577 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580437557e0 con 0x5580437514d0 2024-09-10T21:16:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.468+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.469+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804374e310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005c40 tx=0x7fb5d802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d790d7ebee5f9bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.469+0000 7fb5e4ff9640 1 -- 172.21.15.33:0/2984306577 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8035040 con 0x5580437514d0 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.469+0000 7fb5e4ff9640 1 -- 172.21.15.33:0/2984306577 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d8003e20 con 0x5580437514d0 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.469+0000 7fb5e4ff9640 1 -- 172.21.15.33:0/2984306577 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d803dbb0 con 0x5580437514d0 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2984306577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 msgr2=0x55804374e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804374e310 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005c40 tx=0x7fb5d802a410 comp rx=0 tx=0).stop 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2984306577 shutdown_connections 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558043751df0 0x55804375f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x55804374e310 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2984306577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55804374f300 0x558043759f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2984306577 >> 172.21.15.33:0/2984306577 conn(0x5580437563d0 msgr2=0x55804374dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2984306577 shutdown_connections 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2984306577 wait complete. 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 -- start start 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.470+0000 7fb5f7a39740 1 Processor -- start 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804374f300 0x5580437663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x558043760cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5f7a39740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x5580437611f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f01b1ab0 con 0x55804374f300 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f01b13e0 con 0x558043751df0 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5f7a39740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5f01ae600 con 0x5580437514d0 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5f7a39740 10 client.0 _ll_register_callbacks cb 0x7fb5d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804374f300 0x5580437663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x5580437611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x5580437611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:35484/0 (socket says 172.21.15.33:35484) 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5eeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804374f300 0x5580437663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36912/0 (socket says 172.21.15.33:36912) 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/2954992854 learned_addr learned my addr 172.21.15.33:0/2954992854 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ee7fc640 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x558043760cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 msgr2=0x558043760cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x558043760cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/2954992854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804374f300 msgr2=0x5580437663a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804374f300 0x5580437663a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 -- 172.21.15.33:0/2954992854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558043761930 con 0x558043751df0 2024-09-10T21:16:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804374f300 0x5580437663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5ef7fe640 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x5580437611f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80e3a80 tx=0x7fb5e80ac540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3577423afb5e1032 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5d67fc640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.471+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2954992854 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80d3070 con 0x558043751df0 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2954992854 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb5e80ab9d0 con 0x558043751df0 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5d8003e20 con 0x558043751df0 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2954992854 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7fb5e80ac280 con 0x558043751df0 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 10 client.27098 fetch_fsmap learned FSMap version 59 2024-09-10T21:16:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) -- 0x5580437db430 con 0x558043751df0 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2954992854 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d8003e20 con 0x558043751df0 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2954992854 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 59) ==== 8366+0+0 (secure 0 0 0) 0x7fb5e80c3060 con 0x558043751df0 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 10 client.27098 fetch_fsmap finished waiting for FSMap version 59 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 10 client.27098 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 20 client.27098 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804375dd80 0x5580437ddb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 4 client.27098 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5580436c58a0 con 0x55804375dd80 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.472+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804375dd80 0x5580437ddb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5d67fc640 10 client.27098 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5eeffd640 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804375dd80 0x5580437ddb90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0613eb43f364662 server_cookie=35eaf032ced44259 in_seq=0 out_seq=0 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5d67fc640 1 -- 172.21.15.33:0/2954992854 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5580436c58a0 con 0x55804375dd80 2024-09-10T21:16:16.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5d67fc640 10 client.27098 handle_command_reply: tid=0 2024-09-10T21:16:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 client.27098 shutdown 2024-09-10T21:16:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 msgr2=0x5580437611f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x5580437611f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5e80e3a80 tx=0x7fb5e80ac540 comp rx=0 tx=0).stop 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 shutdown_connections 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55804375dd80 0x5580437ddb90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558043751df0 0x5580437611f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580437514d0 0x558043760cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 --2- 172.21.15.33:0/2954992854 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55804374f300 0x5580437663a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5d67fc640 0 client.27098 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 >> 172.21.15.33:0/2954992854 conn(0x5580437563d0 msgr2=0x55804374e420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 shutdown_connections 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 1 -- 172.21.15.33:0/2954992854 wait complete. 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.473+0000 7fb5f7a39740 20 client.27098 trim_cache size 0 max 16384 2024-09-10T21:16:16.483 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:16:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5dc0b52e0 msgr2=0x7fb5dc0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5dc0b52e0 0x7fb5dc0b77a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8005ec0 tx=0x7fb5d8005b50 comp rx=0 tx=0).stop 2024-09-10T21:16:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f0109170 msgr2=0x7fb5f01a0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f0109170 0x7fb5f01a0dd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800e970 tx=0x7fb5e800ed70 comp rx=0 tx=0).stop 2024-09-10T21:16:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 shutdown_connections 2024-09-10T21:16:16.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5dc0b52e0 0x7fb5dc0b77a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5f0109170 0x7fb5f01a0dd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb5f01087f0 0x7fb5f019ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 --2- 172.21.15.33:0/692916845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb5f0102de0 0x7fb5f019b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 >> 172.21.15.33:0/692916845 conn(0x7fb5f00fe830 msgr2=0x7fb5f0075ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 shutdown_connections 2024-09-10T21:16:16.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:16.474+0000 7fb5f5664640 1 -- 172.21.15.33:0/692916845 wait complete. 2024-09-10T21:16:16.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 947547 6.89 MB/sec execute 557 sec latency 57.678 ms 2024-09-10T21:16:17.200 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:16:17.482 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:17.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948455 6.89 MB/sec execute 558 sec latency 578.894 ms 2024-09-10T21:16:18.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:17 smithi033 ceph-mon[30860]: pgmap v715: 97 pgs: 97 active+clean; 341 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-09-10T21:16:18.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:17 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:17 smithi174 ceph-mon[35635]: pgmap v715: 97 pgs: 97 active+clean; 341 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-09-10T21:16:18.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:17 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:18.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 -- 172.21.15.33:0/1264742438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3081076a0 msgr2=0x7fc308107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 --2- 172.21.15.33:0/1264742438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3081076a0 0x7fc308107ae0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009ca0 tx=0x7fc2f802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 -- 172.21.15.33:0/1264742438 shutdown_connections 2024-09-10T21:16:18.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 --2- 172.21.15.33:0/1264742438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308108020 0x7fc30810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 --2- 172.21.15.33:0/1264742438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3081076a0 0x7fc308107ae0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 --2- 172.21.15.33:0/1264742438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc308101c90 0x7fc308102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 -- 172.21.15.33:0/1264742438 >> 172.21.15.33:0/1264742438 conn(0x7fc3080fd700 msgr2=0x7fc3080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 -- 172.21.15.33:0/1264742438 shutdown_connections 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.150+0000 7fc30eff6640 1 -- 172.21.15.33:0/1264742438 wait complete. 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 Processor -- start 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 -- start start 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308101c90 0x7fc3081992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081076a0 0x7fc308199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308108020 0x7fc30819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc308111ca0 con 0x7fc308101c90 2024-09-10T21:16:18.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc308111b20 con 0x7fc3081076a0 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30eff6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc308111e20 con 0x7fc308108020 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308101c90 0x7fc3081992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308101c90 0x7fc3081992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36942/0 (socket says 172.21.15.33:36942) 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30cd92640 1 -- 172.21.15.33:0/3246553511 learned_addr learned my addr 172.21.15.33:0/3246553511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.151+0000 7fc30d593640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308108020 0x7fc30819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc307fff640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081076a0 0x7fc308199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30cd92640 1 -- 172.21.15.33:0/3246553511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308108020 msgr2=0x7fc30819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30cd92640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308108020 0x7fc30819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30cd92640 1 -- 172.21.15.33:0/3246553511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081076a0 msgr2=0x7fc308199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30cd92640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081076a0 0x7fc308199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30cd92640 1 -- 172.21.15.33:0/3246553511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc30819f370 con 0x7fc308101c90 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30cd92640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308101c90 0x7fc3081992c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000d950 tx=0x7fc2f000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc305ffb640 1 -- 172.21.15.33:0/3246553511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0017030 con 0x7fc308101c90 2024-09-10T21:16:18.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc305ffb640 1 -- 172.21.15.33:0/3246553511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f0004020 con 0x7fc308101c90 2024-09-10T21:16:18.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc30819f660 con 0x7fc308101c90 2024-09-10T21:16:18.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc305ffb640 1 -- 172.21.15.33:0/3246553511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00047a0 con 0x7fc308101c90 2024-09-10T21:16:18.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.152+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3081a5ef0 con 0x7fc308101c90 2024-09-10T21:16:18.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc305ffb640 1 -- 172.21.15.33:0/3246553511 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc2f0009a30 con 0x7fc308101c90 2024-09-10T21:16:18.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 Processor -- start 2024-09-10T21:16:18.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 -- start start 2024-09-10T21:16:18.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4405f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f9df0 0x55e440605840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e440605d80 0x55e440608170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e44056fea0 con 0x55e4405f9df0 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e44055f2d0 con 0x55e4405f94d0 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.155+0000 7fc3113cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e4405630e0 con 0x55e440605d80 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f9df0 0x55e440605840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f9df0 0x55e440605840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36948/0 (socket says 172.21.15.33:36948) 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 -- 172.21.15.33:0/432171762 learned_addr learned my addr 172.21.15.33:0/432171762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30d593640 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4405f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc307fff640 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e440605d80 0x55e440608170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 -- 172.21.15.33:0/432171762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e440605d80 msgr2=0x55e440608170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e440605d80 0x55e440608170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 -- 172.21.15.33:0/432171762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 msgr2=0x55e4405f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4405f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 -- 172.21.15.33:0/432171762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4402bd650 con 0x55e4405f9df0 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc307fff640 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e440605d80 0x55e440608170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc305ffb640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2f40b5360 0x7fc2f40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30d593640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2f40b5360 0x7fc2f40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc30cd92640 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f9df0 0x55e440605840 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00eb5e0 tx=0x7fc2f00ebaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff15d7afe9cc209 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.156+0000 7fc305ffb640 1 -- 172.21.15.33:0/3246553511 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc2f00e5630 con 0x7fc308101c90 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc2feffd640 1 -- 172.21.15.33:0/432171762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00a7c90 con 0x55e4405f9df0 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc2feffd640 1 -- 172.21.15.33:0/432171762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f00ab900 con 0x55e4405f9df0 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc2feffd640 1 -- 172.21.15.33:0/432171762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00ab560 con 0x55e4405f9df0 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 -- 172.21.15.33:0/432171762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f9df0 msgr2=0x55e440605840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc30d593640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2f40b5360 0x7fc2f40b7820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc30000b6c0 tx=0x7fc30000b2e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:18.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f9df0 0x55e440605840 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00eb5e0 tx=0x7fc2f00ebaa0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 -- 172.21.15.33:0/432171762 shutdown_connections 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e440605d80 0x55e440608170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f9df0 0x55e440605840 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 --2- 172.21.15.33:0/432171762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4405f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 -- 172.21.15.33:0/432171762 >> 172.21.15.33:0/432171762 conn(0x55e4405f7300 msgr2=0x55e4405f84a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 -- 172.21.15.33:0/432171762 shutdown_connections 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.157+0000 7fc3113cb740 1 -- 172.21.15.33:0/432171762 wait complete. 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 -- start start 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 Processor -- start 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4406056f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e440602b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e440605d80 0x55e4406031c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e4403c0700 con 0x55e440605d80 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e4404fa160 con 0x55e4405f94d0 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e440608a00 con 0x55e4405f9df0 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc3113cb740 10 client.0 _ll_register_callbacks cb 0x55e4405139e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc30d593640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e440605d80 0x55e4406031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc30d593640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e440605d80 0x55e4406031c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36962/0 (socket says 172.21.15.33:36962) 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.158+0000 7fc30d593640 1 -- 172.21.15.33:0/2058136679 learned_addr learned my addr 172.21.15.33:0/2058136679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc307fff640 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e440602b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc30cd92640 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4406056f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc30d593640 1 -- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 msgr2=0x55e440602b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc30d593640 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e440602b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc30d593640 1 -- 172.21.15.33:0/2058136679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 msgr2=0x55e4406056f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc30d593640 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4406056f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc30d593640 1 -- 172.21.15.33:0/2058136679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e440527c90 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc30d593640 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e440605d80 0x55e4406031c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc300003bf0 tx=0x7fc300004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3225544f9b73b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc2e7fff640 1 -- 172.21.15.33:0/2058136679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30002c060 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc2e7fff640 1 -- 172.21.15.33:0/2058136679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc300017030 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f00ab900 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc2e7fff640 1 -- 172.21.15.33:0/2058136679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fc30001fd40 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc3113cb740 10 client.17667 fetch_fsmap learned FSMap version 60 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.159+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55e440614440 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc2e7fff640 1 -- 172.21.15.33:0/2058136679 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f00ab900 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc2e7fff640 1 -- 172.21.15.33:0/2058136679 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fc300031070 con 0x55e440605d80 2024-09-10T21:16:18.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc3113cb740 10 client.17667 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc3113cb740 10 client.17667 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc3113cb740 20 client.17667 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc3113cb740 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e440617d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc3113cb740 4 client.17667 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e440571df0 con 0x55e440615950 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.160+0000 7fc30cd92640 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e440617d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.161+0000 7fc30cd92640 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e440617d70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5228f7eab6b7cfc server_cookie=f8e0dd7602573788 in_seq=0 out_seq=0 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.161+0000 7fc2e7fff640 10 client.17667 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.161+0000 7fc2e7fff640 1 -- 172.21.15.33:0/2058136679 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e440571df0 con 0x55e440615950 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.161+0000 7fc2e7fff640 10 client.17667 handle_command_reply: tid=0 2024-09-10T21:16:18.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.161+0000 7fc3113cb740 1 client.17667 shutdown 2024-09-10T21:16:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e440605d80 msgr2=0x55e4406031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e440605d80 0x55e4406031c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc300003bf0 tx=0x7fc300004730 comp rx=0 tx=0).stop 2024-09-10T21:16:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 shutdown_connections 2024-09-10T21:16:18.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e440605d80 0x55e4406031c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e440617d70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e440602b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc2e7fff640 0 client.17667 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 --2- 172.21.15.33:0/2058136679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e4405f94d0 0x55e4406056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 >> 172.21.15.33:0/2058136679 conn(0x55e4405f7300 msgr2=0x55e4405fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 shutdown_connections 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 1 -- 172.21.15.33:0/2058136679 wait complete. 2024-09-10T21:16:18.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.162+0000 7fc3113cb740 20 client.17667 trim_cache size 0 max 16384 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 Processor -- start 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 -- start start 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e440611c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e4406013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440601900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081ae460 con 0x55e4405f94d0 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081affd0 con 0x55e440605d80 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc3113cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081aeb30 con 0x55e4405f9df0 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e440611c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e440611c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36978/0 (socket says 172.21.15.33:36978) 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 -- 172.21.15.33:0/4130853902 learned_addr learned my addr 172.21.15.33:0/4130853902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30d593640 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440601900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc307fff640 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e4406013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 -- 172.21.15.33:0/4130853902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 msgr2=0x55e4406013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e4406013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 -- 172.21.15.33:0/4130853902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 msgr2=0x55e440601900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440601900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.176+0000 7fc30cd92640 1 -- 172.21.15.33:0/4130853902 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4406180e0 con 0x55e4405f94d0 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.177+0000 7fc30cd92640 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e440611c60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334c16ffc3c45deb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.177+0000 7fc2fe7fc640 1 -- 172.21.15.33:0/4130853902 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00ab740 con 0x55e4405f94d0 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.177+0000 7fc2fe7fc640 1 -- 172.21.15.33:0/4130853902 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f00ab900 con 0x55e4405f94d0 2024-09-10T21:16:18.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.177+0000 7fc2fe7fc640 1 -- 172.21.15.33:0/4130853902 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00abbc0 con 0x55e4405f94d0 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 -- 172.21.15.33:0/4130853902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 msgr2=0x55e440611c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e440611c60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0005bf0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 -- 172.21.15.33:0/4130853902 shutdown_connections 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440601900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e4406013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 --2- 172.21.15.33:0/4130853902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e440611c60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 -- 172.21.15.33:0/4130853902 >> 172.21.15.33:0/4130853902 conn(0x55e4405f7300 msgr2=0x55e4405fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 -- 172.21.15.33:0/4130853902 shutdown_connections 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 -- 172.21.15.33:0/4130853902 wait complete. 2024-09-10T21:16:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 -- start start 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.178+0000 7fc3113cb740 1 Processor -- start 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e44060deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e44060e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc3113cb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440608c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc3113cb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081af8d0 con 0x55e4405f94d0 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc3113cb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081af200 con 0x55e440605d80 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc3113cb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3081ac420 con 0x55e4405f9df0 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc3113cb740 10 client.0 _ll_register_callbacks cb 0x7fc3000043b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e44060deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e44060deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36992/0 (socket says 172.21.15.33:36992) 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 -- 172.21.15.33:0/584293702 learned_addr learned my addr 172.21.15.33:0/584293702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:18.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30d593640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440608c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc307fff640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e44060e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 -- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 msgr2=0x55e44060e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e44060e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 -- 172.21.15.33:0/584293702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 msgr2=0x55e440608c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440608c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 -- 172.21.15.33:0/584293702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e440609370 con 0x55e4405f94d0 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc307fff640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e44060e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc30cd92640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e44060deb0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00e8ef0 tx=0x7fc2f00e8f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec8e510b34c5855 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc2e7fff640 1 -- 172.21.15.33:0/584293702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00ab740 con 0x55e4405f94d0 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.179+0000 7fc2e7fff640 1 -- 172.21.15.33:0/584293702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc2f00041d0 con 0x55e4405f94d0 2024-09-10T21:16:18.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f00ab900 con 0x55e4405f94d0 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc2e7fff640 1 -- 172.21.15.33:0/584293702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fc2f00eb200 con 0x55e4405f94d0 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 10 client.17673 fetch_fsmap learned FSMap version 60 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55e4406831e0 con 0x55e4405f94d0 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc2e7fff640 1 -- 172.21.15.33:0/584293702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f00ab900 con 0x55e4405f94d0 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc2e7fff640 1 -- 172.21.15.33:0/584293702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fc2f0093070 con 0x55e4405f94d0 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 10 client.17673 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 10 client.17673 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 20 client.17673 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e4406853c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 4 client.17673 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e44056d8a0 con 0x55e440615950 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.180+0000 7fc307fff640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e4406853c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc307fff640 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e4406853c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85d828054cf87547 server_cookie=97bf61bfcc59e8b3 in_seq=0 out_seq=0 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc2e7fff640 10 client.17673 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc2e7fff640 1 -- 172.21.15.33:0/584293702 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e44056d8a0 con 0x55e440615950 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc2e7fff640 10 client.17673 handle_command_reply: tid=0 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 client.17673 shutdown 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 msgr2=0x55e44060deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e44060deb0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fc2f00e8ef0 tx=0x7fc2f00e8f20 comp rx=0 tx=0).stop 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 shutdown_connections 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e440605d80 0x55e440608c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e440615950 0x55e4406853c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4405f9df0 0x55e44060e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc2e7fff640 0 client.17673 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 --2- 172.21.15.33:0/584293702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e4405f94d0 0x55e44060deb0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 >> 172.21.15.33:0/584293702 conn(0x55e4405f7300 msgr2=0x55e440601380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:18.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 shutdown_connections 2024-09-10T21:16:18.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 1 -- 172.21.15.33:0/584293702 wait complete. 2024-09-10T21:16:18.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.181+0000 7fc3113cb740 20 client.17673 trim_cache size 0 max 16384 2024-09-10T21:16:18.192 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2f40b5360 msgr2=0x7fc2f40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2f40b5360 0x7fc2f40b7820 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc30000b6c0 tx=0x7fc30000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308101c90 msgr2=0x7fc3081992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308101c90 0x7fc3081992c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000d950 tx=0x7fc2f000de10 comp rx=0 tx=0).stop 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 shutdown_connections 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc2f40b5360 0x7fc2f40b7820 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc308108020 0x7fc30819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3081076a0 0x7fc308199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 --2- 172.21.15.33:0/3246553511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc308101c90 0x7fc3081992c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:18.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 >> 172.21.15.33:0/3246553511 conn(0x7fc3080fd700 msgr2=0x7fc3081047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:18.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 shutdown_connections 2024-09-10T21:16:18.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:18.182+0000 7fc30eff6640 1 -- 172.21.15.33:0/3246553511 wait complete. 2024-09-10T21:16:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:17 smithi104 ceph-mon[35325]: pgmap v715: 97 pgs: 97 active+clean; 341 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 2.3 MiB/s wr, 247 op/s 2024-09-10T21:16:18.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:17 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:18.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948455 6.87 MB/sec execute 559 sec latency 1579.004 ms 2024-09-10T21:16:18.906 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:16:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:19.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 -- 172.21.15.33:0/364284084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 msgr2=0x7fde9c101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/364284084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c101c50 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fde98009900 tx=0x7fde9802a410 comp rx=0 tx=0).stop 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 -- 172.21.15.33:0/364284084 shutdown_connections 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/364284084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde9c10f420 0x7fde9c111810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/364284084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c102190 0x7fde9c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/364284084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c101c50 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 -- 172.21.15.33:0/364284084 >> 172.21.15.33:0/364284084 conn(0x7fde9c0fd700 msgr2=0x7fde9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 -- 172.21.15.33:0/364284084 shutdown_connections 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.627+0000 7fdea3ffc640 1 -- 172.21.15.33:0/364284084 wait complete. 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 Processor -- start 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 -- start start 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c19b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c102190 0x7fde9c19ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde9c10f420 0x7fde9c1a0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c114ff0 con 0x7fde9c101870 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c114e70 con 0x7fde9c10f420 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea3ffc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c115170 con 0x7fde9c102190 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c19b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c19b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60946/0 (socket says 172.21.15.33:60946) 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 -- 172.21.15.33:0/1848757182 learned_addr learned my addr 172.21.15.33:0/1848757182 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea2599640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde9c10f420 0x7fde9c1a0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1597640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c102190 0x7fde9c19ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 -- 172.21.15.33:0/1848757182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c102190 msgr2=0x7fde9c19ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c102190 0x7fde9c19ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 -- 172.21.15.33:0/1848757182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde9c10f420 msgr2=0x7fde9c1a0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde9c10f420 0x7fde9c1a0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.628+0000 7fdea1d98640 1 -- 172.21.15.33:0/1848757182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9c1a15f0 con 0x7fde9c101870 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fdea1597640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c102190 0x7fde9c19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fdea2599640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde9c10f420 0x7fde9c1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fdea1d98640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c19b540 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fde98005940 tx=0x7fde980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fde92ffd640 1 -- 172.21.15.33:0/1848757182 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9803f030 con 0x7fde9c101870 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde9c1a1880 con 0x7fde9c101870 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fde92ffd640 1 -- 172.21.15.33:0/1848757182 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde98003cb0 con 0x7fde9c101870 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fde92ffd640 1 -- 172.21.15.33:0/1848757182 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98035070 con 0x7fde9c101870 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.629+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde9c1a7fa0 con 0x7fde9c101870 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.631+0000 7fde92ffd640 1 -- 172.21.15.33:0/1848757182 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fde98003800 con 0x7fde9c101870 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.631+0000 7fdea63d1740 1 Processor -- start 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea63d1740 1 -- start start 2024-09-10T21:16:19.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e5358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e535df0 0x55647e541840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e541d80 0x55647e544170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea63d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55647e4abea0 con 0x55647e5354d0 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea63d1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55647e49b2d0 con 0x55647e541d80 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea63d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55647e49f0e0 con 0x55647e535df0 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e5358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e5358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60956/0 (socket says 172.21.15.33:60956) 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea2599640 1 -- 172.21.15.33:0/2784962523 learned_addr learned my addr 172.21.15.33:0/2784962523 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea1597640 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e541d80 0x55647e544170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.632+0000 7fdea1d98640 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e535df0 0x55647e541840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fdea2599640 1 -- 172.21.15.33:0/2784962523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e535df0 msgr2=0x55647e541840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fdea2599640 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e535df0 0x55647e541840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fdea2599640 1 -- 172.21.15.33:0/2784962523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e541d80 msgr2=0x55647e544170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fdea2599640 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e541d80 0x55647e544170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fdea2599640 1 -- 172.21.15.33:0/2784962523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647e1f9650 con 0x55647e5354d0 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fde92ffd640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fde880b52e0 0x7fde880b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fdea1d98640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fde880b52e0 0x7fde880b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fdea2599640 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e5358b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fde9400f410 tx=0x7fde9400f8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953f5b9f90357f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fde92ffd640 1 -- 172.21.15.33:0/1848757182 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fde98106850 con 0x7fde9c101870 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fde87fff640 1 -- 172.21.15.33:0/2784962523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94015070 con 0x55647e5354d0 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fde87fff640 1 -- 172.21.15.33:0/2784962523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde94003e20 con 0x55647e5354d0 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.633+0000 7fde87fff640 1 -- 172.21.15.33:0/2784962523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94011500 con 0x55647e5354d0 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 -- 172.21.15.33:0/2784962523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 msgr2=0x55647e5358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea1d98640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fde880b52e0 0x7fde880b77a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fde98005ec0 tx=0x7fde9810c530 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e5358b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fde9400f410 tx=0x7fde9400f8d0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 -- 172.21.15.33:0/2784962523 shutdown_connections 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e541d80 0x55647e544170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e535df0 0x55647e541840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 --2- 172.21.15.33:0/2784962523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e5358b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 -- 172.21.15.33:0/2784962523 >> 172.21.15.33:0/2784962523 conn(0x55647e533300 msgr2=0x55647e5344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 -- 172.21.15.33:0/2784962523 shutdown_connections 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 -- 172.21.15.33:0/2784962523 wait complete. 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.634+0000 7fdea63d1740 1 -- start start 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 1 Processor -- start 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e541620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e53ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 0x55647e53f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55647e2fc700 con 0x55647e5354d0 2024-09-10T21:16:19.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55647e436160 con 0x55647e535df0 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55647e544a00 con 0x55647e541d80 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea63d1740 10 client.0 _ll_register_callbacks cb 0x55647e44f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e541620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e541620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60972/0 (socket says 172.21.15.33:60972) 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 -- 172.21.15.33:0/3108837047 learned_addr learned my addr 172.21.15.33:0/3108837047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea1597640 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e53ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 -- 172.21.15.33:0/3108837047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 msgr2=0x55647e53f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 0x55647e53f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 -- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 msgr2=0x55647e53ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e53ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea2599640 1 -- 172.21.15.33:0/3108837047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647e463c90 con 0x55647e5354d0 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.635+0000 7fdea1597640 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e53ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:19.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea2599640 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e541620 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fde94006010 tx=0x7fde94004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6669416776e68fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fde857fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fde857fa640 1 -- 172.21.15.33:0/3108837047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94015070 con 0x55647e5354d0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fde857fa640 1 -- 172.21.15.33:0/3108837047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fde94003b70 con 0x55647e5354d0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde94003e20 con 0x55647e5354d0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fde857fa640 1 -- 172.21.15.33:0/3108837047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fde940047d0 con 0x55647e5354d0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 10 client.17685 fetch_fsmap learned FSMap version 60 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55647e550320 con 0x55647e5354d0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fde857fa640 1 -- 172.21.15.33:0/3108837047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde94003e20 con 0x55647e5354d0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fde857fa640 1 -- 172.21.15.33:0/3108837047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fde94026590 con 0x55647e5354d0 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 10 client.17685 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 10 client.17685 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:16:19.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 20 client.17685 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e551720 0x55647e553b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 4 client.17685 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.636+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55647e4addf0 con 0x55647e551720 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.637+0000 7fdea1597640 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e551720 0x55647e553b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.637+0000 7fdea1597640 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e551720 0x55647e553b10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81d054f321b3fcf server_cookie=b8e590a46b62c7e6 in_seq=0 out_seq=0 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.637+0000 7fde857fa640 10 client.17685 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fde857fa640 1 -- 172.21.15.33:0/3108837047 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55647e4addf0 con 0x55647e551720 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fde857fa640 10 client.17685 handle_command_reply: tid=0 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 client.17685 shutdown 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 msgr2=0x55647e541620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e541620 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fde94006010 tx=0x7fde94004000 comp rx=0 tx=0).stop 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 shutdown_connections 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 0x55647e53f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e53ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e551720 0x55647e553b10 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 --2- 172.21.15.33:0/3108837047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e541620 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fde857fa640 0 client.17685 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:16:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 >> 172.21.15.33:0/3108837047 conn(0x55647e533300 msgr2=0x55647e538e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 shutdown_connections 2024-09-10T21:16:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 1 -- 172.21.15.33:0/3108837047 wait complete. 2024-09-10T21:16:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.638+0000 7fdea63d1740 20 client.17685 trim_cache size 0 max 16384 2024-09-10T21:16:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 Processor -- start 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 -- start start 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e54dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e532310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 0x55647e53bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b0630 con 0x55647e5354d0 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b21a0 con 0x55647e535df0 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea63d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b0d00 con 0x55647e541d80 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e54dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e54dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60976/0 (socket says 172.21.15.33:60976) 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 -- 172.21.15.33:0/512458565 learned_addr learned my addr 172.21.15.33:0/512458565 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea1597640 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e532310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 -- 172.21.15.33:0/512458565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 msgr2=0x55647e53bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 0x55647e53bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 -- 172.21.15.33:0/512458565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 msgr2=0x55647e532310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e532310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 -- 172.21.15.33:0/512458565 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55647e541250 con 0x55647e5354d0 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.652+0000 7fdea2599640 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e54dc60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fde9400f910 tx=0x7fde94009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad7e114ba973972 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fde877fe640 1 -- 172.21.15.33:0/512458565 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94036070 con 0x55647e5354d0 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fde877fe640 1 -- 172.21.15.33:0/512458565 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde94021070 con 0x55647e5354d0 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fde877fe640 1 -- 172.21.15.33:0/512458565 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94015070 con 0x55647e5354d0 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fdea63d1740 1 -- 172.21.15.33:0/512458565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 msgr2=0x55647e54dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fdea63d1740 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e54dc60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fde9400f910 tx=0x7fde94009fa0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fdea63d1740 1 -- 172.21.15.33:0/512458565 shutdown_connections 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fdea63d1740 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e541d80 0x55647e53bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fdea63d1740 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e532310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fdea63d1740 1 --2- 172.21.15.33:0/512458565 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e5354d0 0x55647e54dc60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.653+0000 7fdea63d1740 1 -- 172.21.15.33:0/512458565 >> 172.21.15.33:0/512458565 conn(0x55647e533300 msgr2=0x55647e531c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 -- 172.21.15.33:0/512458565 shutdown_connections 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 -- 172.21.15.33:0/512458565 wait complete. 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 -- start start 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 Processor -- start 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e5354d0 0x55647e5467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e546d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e541d80 0x55647e5b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b1aa0 con 0x55647e541d80 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1b13d0 con 0x55647e535df0 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde9c1ae5f0 con 0x55647e5354d0 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.654+0000 7fdea63d1740 10 client.0 _ll_register_callbacks cb 0x7fde8c00a150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:19.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e541d80 0x55647e5b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea2599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e541d80 0x55647e5b75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60990/0 (socket says 172.21.15.33:60990) 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea2599640 1 -- 172.21.15.33:0/502261844 learned_addr learned my addr 172.21.15.33:0/502261844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea1597640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e546d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea2599640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e5354d0 0x55647e5467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea1597640 1 -- 172.21.15.33:0/502261844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e5354d0 msgr2=0x55647e5467c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea1597640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e5354d0 0x55647e5467c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea1597640 1 -- 172.21.15.33:0/502261844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e541d80 msgr2=0x55647e5b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea1597640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e541d80 0x55647e5b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea1597640 1 -- 172.21.15.33:0/502261844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55647e5b7cf0 con 0x55647e535df0 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea2599640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e541d80 0x55647e5b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fdea1597640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e546d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55647e436270 tx=0x7fde8c0157f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=388b65f52d78d6c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.655+0000 7fde857fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:16:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fde857fa640 1 -- 172.21.15.33:0/502261844 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0147c0 con 0x55647e535df0 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde94021070 con 0x55647e535df0 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fde857fa640 1 -- 172.21.15.33:0/502261844 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fde8c014960 con 0x55647e535df0 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fde857fa640 1 -- 172.21.15.33:0/502261844 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7fde8c014b60 con 0x55647e535df0 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 10 client.27134 fetch_fsmap learned FSMap version 60 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x55647e5bf1d0 con 0x55647e535df0 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fde857fa640 1 -- 172.21.15.33:0/502261844 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde94021070 con 0x55647e535df0 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fde857fa640 1 -- 172.21.15.33:0/502261844 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7fde8c02f400 con 0x55647e535df0 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 10 client.27134 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 10 client.27134 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 20 client.27134 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e5bf8d0 0x55647e5c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 4 client.27134 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:16:19.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.656+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55647e4a98a0 con 0x55647e5bf8d0 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea2599640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e5bf8d0 0x55647e5c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fde857fa640 10 client.27134 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea2599640 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e5bf8d0 0x55647e5c1cf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1d12c65d0317903 server_cookie=9bf1d331d2fc0df5 in_seq=0 out_seq=0 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fde857fa640 1 -- 172.21.15.33:0/502261844 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55647e4a98a0 con 0x55647e5bf8d0 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fde857fa640 10 client.27134 handle_command_reply: tid=0 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 client.27134 shutdown 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 msgr2=0x55647e546d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e546d60 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x55647e436270 tx=0x7fde8c0157f0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 shutdown_connections 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55647e5bf8d0 0x55647e5c1cf0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55647e541d80 0x55647e5b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55647e535df0 0x55647e546d60 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 --2- 172.21.15.33:0/502261844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55647e5354d0 0x55647e5467c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fde857fa640 0 client.27134 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:16:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.657+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 >> 172.21.15.33:0/502261844 conn(0x55647e533300 msgr2=0x55647e5408c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.658+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 shutdown_connections 2024-09-10T21:16:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.658+0000 7fdea63d1740 1 -- 172.21.15.33:0/502261844 wait complete. 2024-09-10T21:16:19.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.658+0000 7fdea63d1740 20 client.27134 trim_cache size 0 max 16384 2024-09-10T21:16:19.669 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:16:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fde880b52e0 msgr2=0x7fde880b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fde880b52e0 0x7fde880b77a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fde98005ec0 tx=0x7fde9810c530 comp rx=0 tx=0).stop 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 msgr2=0x7fde9c19b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c19b540 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fde98005940 tx=0x7fde980042f0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 shutdown_connections 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fde880b52e0 0x7fde880b77a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fde9c10f420 0x7fde9c1a0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde9c102190 0x7fde9c19ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 --2- 172.21.15.33:0/1848757182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde9c101870 0x7fde9c19b540 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 >> 172.21.15.33:0/1848757182 conn(0x7fde9c0fd700 msgr2=0x7fde9c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 shutdown_connections 2024-09-10T21:16:19.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:19.659+0000 7fdea3ffc640 1 -- 172.21.15.33:0/1848757182 wait complete. 2024-09-10T21:16:19.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948455 6.86 MB/sec execute 560 sec latency 2579.112 ms 2024-09-10T21:16:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:19 smithi174 ceph-mon[35635]: pgmap v716: 97 pgs: 97 active+clean; 344 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-09-10T21:16:20.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:19 smithi033 ceph-mon[30860]: pgmap v716: 97 pgs: 97 active+clean; 344 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-09-10T21:16:20.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:19 smithi104 ceph-mon[35325]: pgmap v716: 97 pgs: 97 active+clean; 344 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.8 MiB/s wr, 282 op/s 2024-09-10T21:16:20.401 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:16:20.664 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:20.755 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948455 6.85 MB/sec execute 561 sec latency 3579.233 ms 2024-09-10T21:16:21.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 -- 172.21.15.33:0/4196818542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 msgr2=0x7efffc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 --2- 172.21.15.33:0/4196818542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1031a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7efff8009900 tx=0x7efff802a410 comp rx=0 tx=0).stop 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 -- 172.21.15.33:0/4196818542 shutdown_connections 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 --2- 172.21.15.33:0/4196818542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc109150 0x7efffc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 --2- 172.21.15.33:0/4196818542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc1087d0 0x7efffc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 --2- 172.21.15.33:0/4196818542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1031a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 -- 172.21.15.33:0/4196818542 >> 172.21.15.33:0/4196818542 conn(0x7efffc0fe830 msgr2=0x7efffc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 -- 172.21.15.33:0/4196818542 shutdown_connections 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.157+0000 7f00038d6640 1 -- 172.21.15.33:0/4196818542 wait complete. 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 Processor -- start 2024-09-10T21:16:21.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 -- start start 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc1087d0 0x7efffc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc109150 0x7efffc19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efffc112e10 con 0x7efffc102dc0 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efffc112c90 con 0x7efffc1087d0 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.158+0000 7f00038d6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efffc112f90 con 0x7efffc109150 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32768/0 (socket says 172.21.15.33:32768) 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 -- 172.21.15.33:0/1191285896 learned_addr learned my addr 172.21.15.33:0/1191285896 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0000e71640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc1087d0 0x7efffc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001e73640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc109150 0x7efffc19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 -- 172.21.15.33:0/1191285896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc109150 msgr2=0x7efffc19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc109150 0x7efffc19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 -- 172.21.15.33:0/1191285896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc1087d0 msgr2=0x7efffc199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc1087d0 0x7efffc199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 -- 172.21.15.33:0/1191285896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc19f380 con 0x7efffc102dc0 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0000e71640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc1087d0 0x7efffc199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.159+0000 7f0001672640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1992d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7efff8005940 tx=0x7efff80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.160+0000 7efff27fc640 1 -- 172.21.15.33:0/1191285896 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff803f030 con 0x7efffc102dc0 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.160+0000 7efff27fc640 1 -- 172.21.15.33:0/1191285896 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff8003cb0 con 0x7efffc102dc0 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.161+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc19f610 con 0x7efffc102dc0 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.161+0000 7efff27fc640 1 -- 172.21.15.33:0/1191285896 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff8035070 con 0x7efffc102dc0 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.161+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc1a5d30 con 0x7efffc102dc0 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 Processor -- start 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 -- start start 2024-09-10T21:16:21.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9f7190 0x559aed9f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559aed95cea0 con 0x559aed9e64d0 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7efff27fc640 1 -- 172.21.15.33:0/1191285896 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efff803bbf0 con 0x7efffc102dc0 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559aed94c2d0 con 0x559aed9f7190 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0005cab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559aed9500e0 con 0x559aed9e6df0 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.164+0000 7f0001e73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001672640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32772/0 (socket says 172.21.15.33:32772) 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 -- 172.21.15.33:0/2418081268 learned_addr learned my addr 172.21.15.33:0/2418081268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0000e71640 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9f7190 0x559aed9f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 -- 172.21.15.33:0/2418081268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 msgr2=0x559aed9f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 -- 172.21.15.33:0/2418081268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9f7190 msgr2=0x559aed9f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9f7190 0x559aed9f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 -- 172.21.15.33:0/2418081268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aed6aa650 con 0x559aed9e64d0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7f0001e73640 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e68b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efff40103a0 tx=0x7efff4010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e75659ae106f0a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.165+0000 7effe77fe640 1 -- 172.21.15.33:0/2418081268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4016070 con 0x559aed9e64d0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7effe77fe640 1 -- 172.21.15.33:0/2418081268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4004d20 con 0x559aed9e64d0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7effe77fe640 1 -- 172.21.15.33:0/2418081268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff40053c0 con 0x559aed9e64d0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7efff27fc640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe80b52e0 0x7effe80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7f0001672640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe80b52e0 0x7effe80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7efff27fc640 1 -- 172.21.15.33:0/1191285896 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efff81066f0 con 0x7efffc102dc0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7f0001672640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe80b52e0 0x7effe80b77a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efff8003fd0 tx=0x7efff80f0040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:21.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7f0005cab740 1 -- 172.21.15.33:0/2418081268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 msgr2=0x559aed9e68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.166+0000 7f0005cab740 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e68b0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7efff40103a0 tx=0x7efff4010860 comp rx=0 tx=0).stop 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 -- 172.21.15.33:0/2418081268 shutdown_connections 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9f7190 0x559aed9f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 --2- 172.21.15.33:0/2418081268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e68b0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 -- 172.21.15.33:0/2418081268 >> 172.21.15.33:0/2418081268 conn(0x559aed9e4300 msgr2=0x559aed9e54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 -- 172.21.15.33:0/2418081268 shutdown_connections 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 -- 172.21.15.33:0/2418081268 wait complete. 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 -- start start 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.167+0000 7f0005cab740 1 Processor -- start 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0005cab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0005cab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0005cab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9f7190 0x559aed9f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0005cab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559aed7ad700 con 0x559aed9f7190 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0005cab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559aed8e7160 con 0x559aed9e64d0 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0005cab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559aed9f9e40 con 0x559aed9e6df0 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0005cab740 10 client.0 _ll_register_callbacks cb 0x559aed9009e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0001e73640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:52796/0 (socket says 172.21.15.33:52796) 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0001e73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9f7190 0x559aed9f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 -- 172.21.15.33:0/669321773 learned_addr learned my addr 172.21.15.33:0/669321773 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 -- 172.21.15.33:0/669321773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 msgr2=0x559aed9f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 -- 172.21.15.33:0/669321773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9f7190 msgr2=0x559aed9f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9f7190 0x559aed9f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0000e71640 1 -- 172.21.15.33:0/669321773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aed914c90 con 0x559aed9e6df0 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.168+0000 7f0001e73640 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9f7190 0x559aed9f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7f0000e71640 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f3f40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effec0116f0 tx=0x7effec011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d41be6e9173d6469 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7effe4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7effe4ff9640 1 -- 172.21.15.33:0/669321773 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec010c20 con 0x559aed9e6df0 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff4004d20 con 0x559aed9e6df0 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7effe4ff9640 1 -- 172.21.15.33:0/669321773 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7effec017070 con 0x559aed9e6df0 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7effe4ff9640 1 -- 172.21.15.33:0/669321773 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7effec010dc0 con 0x559aed9e6df0 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7f0005cab740 10 client.27082 fetch_fsmap learned FSMap version 60 2024-09-10T21:16:21.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x559aeda05870 con 0x559aed9e6df0 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.169+0000 7effe4ff9640 1 -- 172.21.15.33:0/669321773 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4004d20 con 0x559aed9e6df0 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7effe4ff9640 1 -- 172.21.15.33:0/669321773 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7effec004b90 con 0x559aed9e6df0 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7f0005cab740 10 client.27082 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7f0005cab740 10 client.27082 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7f0005cab740 20 client.27082 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7f0005cab740 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda06d40 0x559aeda09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7f0005cab740 4 client.27082 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aed95edf0 con 0x559aeda06d40 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.170+0000 7f0001e73640 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda06d40 0x559aeda09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.171+0000 7f0001e73640 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda06d40 0x559aeda09130 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc3bed6693d229a4 server_cookie=388116a4e3e7b7f4 in_seq=0 out_seq=0 2024-09-10T21:16:21.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.171+0000 7effe4ff9640 10 client.27082 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:16:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.171+0000 7effe4ff9640 1 -- 172.21.15.33:0/669321773 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559aed95edf0 con 0x559aeda06d40 2024-09-10T21:16:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.171+0000 7effe4ff9640 10 client.27082 handle_command_reply: tid=0 2024-09-10T21:16:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.171+0000 7f0005cab740 1 client.27082 shutdown 2024-09-10T21:16:21.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.171+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 msgr2=0x559aed9f3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.171+0000 7f0005cab740 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f3f40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7effec0116f0 tx=0x7effec011bb0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 shutdown_connections 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda06d40 0x559aeda09130 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9f7190 0x559aed9f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9e6df0 0x559aed9f3f40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 --2- 172.21.15.33:0/669321773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7effe4ff9640 0 client.27082 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 >> 172.21.15.33:0/669321773 conn(0x559aed9e4300 msgr2=0x559aed9e3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:21.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 shutdown_connections 2024-09-10T21:16:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 1 -- 172.21.15.33:0/669321773 wait complete. 2024-09-10T21:16:21.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.172+0000 7f0005cab740 20 client.27082 trim_cache size 0 max 16384 2024-09-10T21:16:21.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 Processor -- start 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 -- start start 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e6df0 0x559aeda030a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 0x559aed9f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ac2a0 con 0x559aed9e64d0 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ade10 con 0x559aed9e6df0 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0005cab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ac970 con 0x559aed9f7190 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0001e73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0001e73640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32778/0 (socket says 172.21.15.33:32778) 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0001e73640 1 -- 172.21.15.33:0/824124071 learned_addr learned my addr 172.21.15.33:0/824124071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0000e71640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e6df0 0x559aeda030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.186+0000 7f0001e73640 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 0x559aed9f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7f0000e71640 1 -- 172.21.15.33:0/824124071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 msgr2=0x559aed9f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7f0000e71640 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 0x559aed9f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7f0000e71640 1 -- 172.21.15.33:0/824124071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 msgr2=0x559aed9e3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7f0000e71640 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7f0000e71640 1 -- 172.21.15.33:0/824124071 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aed9e6c20 con 0x559aed9e6df0 2024-09-10T21:16:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7f0001e73640 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:21.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7f0000e71640 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e6df0 0x559aeda030a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7effec01a600 tx=0x7effec0058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18788c4d42a491a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7effe6ffd640 1 -- 172.21.15.33:0/824124071 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec032070 con 0x559aed9e6df0 2024-09-10T21:16:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7effe6ffd640 1 -- 172.21.15.33:0/824124071 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7effec005bb0 con 0x559aed9e6df0 2024-09-10T21:16:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.187+0000 7effe6ffd640 1 -- 172.21.15.33:0/824124071 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec029040 con 0x559aed9e6df0 2024-09-10T21:16:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 -- 172.21.15.33:0/824124071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e6df0 msgr2=0x559aeda030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e6df0 0x559aeda030a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7effec01a600 tx=0x7effec0058d0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 -- 172.21.15.33:0/824124071 shutdown_connections 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 0x559aed9f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e6df0 0x559aeda030a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 --2- 172.21.15.33:0/824124071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e64d0 0x559aed9e3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 -- 172.21.15.33:0/824124071 >> 172.21.15.33:0/824124071 conn(0x559aed9e4300 msgr2=0x559aed9f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 -- 172.21.15.33:0/824124071 shutdown_connections 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 -- 172.21.15.33:0/824124071 wait complete. 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.188+0000 7f0005cab740 1 -- start start 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 1 Processor -- start 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e6df0 0x559aed9ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 0x559aed9fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ad710 con 0x559aed9e6df0 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ad040 con 0x559aed9e64d0 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1aa260 con 0x559aed9f7190 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0005cab740 10 client.0 _ll_register_callbacks cb 0x7effec00bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0000e71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e6df0 0x559aed9ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0001e73640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0001e73640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:37738/0 (socket says 172.21.15.33:37738) 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0000e71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e6df0 0x559aed9ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32788/0 (socket says 172.21.15.33:32788) 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.189+0000 7f0001e73640 1 -- 172.21.15.33:0/3474978441 learned_addr learned my addr 172.21.15.33:0/3474978441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:21.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0001e73640 1 -- 172.21.15.33:0/3474978441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 msgr2=0x559aed9fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0001e73640 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 0x559aed9fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0001e73640 1 -- 172.21.15.33:0/3474978441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e6df0 msgr2=0x559aed9ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0001e73640 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e6df0 0x559aed9ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0001e73640 1 -- 172.21.15.33:0/3474978441 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aed9ffe50 con 0x559aed9e64d0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0001e73640 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9ff100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efff4010370 tx=0x7efff4010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=251bdc4e7bca838e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7effe4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7effe4ff9640 1 -- 172.21.15.33:0/3474978441 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4037070 con 0x559aed9e64d0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effec005bb0 con 0x559aed9e64d0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7effe4ff9640 1 -- 172.21.15.33:0/3474978441 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efff4003b70 con 0x559aed9e64d0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7effe4ff9640 1 -- 172.21.15.33:0/3474978441 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) ==== 24+0+0 (secure 0 0 0) 0x7efff4016040 con 0x559aed9e64d0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0005cab740 10 client.27152 fetch_fsmap learned FSMap version 60 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.190+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) -- 0x559aeda741a0 con 0x559aed9e64d0 2024-09-10T21:16:21.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7effe4ff9640 1 -- 172.21.15.33:0/3474978441 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4003d70 con 0x559aed9e64d0 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7effe4ff9640 1 -- 172.21.15.33:0/3474978441 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 60) ==== 8370+0+0 (secure 0 0 0) 0x7efff403a020 con 0x559aed9e64d0 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7f0005cab740 10 client.27152 fetch_fsmap finished waiting for FSMap version 60 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7f0005cab740 10 client.27152 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7f0005cab740 20 client.27152 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7f0005cab740 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda74780 0x559aeda76ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7f0005cab740 4 client.27152 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7effec005bb0 con 0x559aeda74780 2024-09-10T21:16:21.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.191+0000 7f0000e71640 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda74780 0x559aeda76ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7effe4ff9640 10 client.27152 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:16:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0000e71640 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda74780 0x559aeda76ba0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4aecbb3a4c3e2037 server_cookie=3bffa08ca70d509a in_seq=0 out_seq=0 2024-09-10T21:16:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7effe4ff9640 1 -- 172.21.15.33:0/3474978441 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7effec005bb0 con 0x559aeda74780 2024-09-10T21:16:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7effe4ff9640 10 client.27152 handle_command_reply: tid=0 2024-09-10T21:16:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 client.27152 shutdown 2024-09-10T21:16:21.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 msgr2=0x559aed9ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9ff100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efff4010370 tx=0x7efff4010fd0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 shutdown_connections 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559aed9f7190 0x559aed9fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559aeda74780 0x559aeda76ba0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559aed9e6df0 0x559aed9ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 --2- 172.21.15.33:0/3474978441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559aed9e64d0 0x559aed9ff100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 >> 172.21.15.33:0/3474978441 conn(0x559aed9e4300 msgr2=0x559aed9f8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7effe4ff9640 0 client.27152 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 shutdown_connections 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 1 -- 172.21.15.33:0/3474978441 wait complete. 2024-09-10T21:16:21.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.192+0000 7f0005cab740 20 client.27152 trim_cache size 0 max 16384 2024-09-10T21:16:21.202 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:16:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe80b52e0 msgr2=0x7effe80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe80b52e0 0x7effe80b77a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7efff8003fd0 tx=0x7efff80f0040 comp rx=0 tx=0).stop 2024-09-10T21:16:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 msgr2=0x7efffc1992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1992d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7efff8005940 tx=0x7efff80042f0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 shutdown_connections 2024-09-10T21:16:21.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe80b52e0 0x7effe80b77a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc109150 0x7efffc19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc1087d0 0x7efffc199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 --2- 172.21.15.33:0/1191285896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc102dc0 0x7efffc1992d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 >> 172.21.15.33:0/1191285896 conn(0x7efffc0fe830 msgr2=0x7efffc1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 shutdown_connections 2024-09-10T21:16:21.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:21.193+0000 7f00038d6640 1 -- 172.21.15.33:0/1191285896 wait complete. 2024-09-10T21:16:21.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948455 6.84 MB/sec execute 562 sec latency 4579.305 ms 2024-09-10T21:16:21.926 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:16:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:21 smithi174 ceph-mon[35635]: pgmap v717: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-09-10T21:16:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:16:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:16:22.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:16:22.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:21 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:16:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:21 smithi033 ceph-mon[30860]: pgmap v717: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-09-10T21:16:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:16:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:16:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:16:22.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:21 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:16:22.198 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:22.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:21 smithi104 ceph-mon[35325]: pgmap v717: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.6 MiB/s wr, 263 op/s 2024-09-10T21:16:22.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:16:22.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:16:22.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:16:22.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:21 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:16:22.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 -- 172.21.15.33:0/1235555079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 msgr2=0x7f0ddc10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 --2- 172.21.15.33:0/1235555079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc10f560 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc009ca0 tx=0x7f0dcc02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 -- 172.21.15.33:0/1235555079 shutdown_connections 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 --2- 172.21.15.33:0/1235555079 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc10f560 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 --2- 172.21.15.33:0/1235555079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ddc1045c0 0x7f0ddc10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 --2- 172.21.15.33:0/1235555079 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ddc103ca0 0x7f0ddc104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 -- 172.21.15.33:0/1235555079 >> 172.21.15.33:0/1235555079 conn(0x7f0ddc0fd700 msgr2=0x7f0ddc0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 -- 172.21.15.33:0/1235555079 shutdown_connections 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.641+0000 7f0de0c89640 1 -- 172.21.15.33:0/1235555079 wait complete. 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 Processor -- start 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 -- start start 2024-09-10T21:16:22.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ddc103ca0 0x7f0ddc199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ddc1045c0 0x7f0ddc1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc19eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc112d70 con 0x7f0ddc10d100 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc112bf0 con 0x7f0ddc103ca0 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0de0c89640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc112ef0 con 0x7f0ddc1045c0 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0ddad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc19eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0ddad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc19eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32816/0 (socket says 172.21.15.33:32816) 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0ddad9d640 1 -- 172.21.15.33:0/2150600744 learned_addr learned my addr 172.21.15.33:0/2150600744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ddc1045c0 0x7f0ddc1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.642+0000 7f0dda59c640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ddc103ca0 0x7f0ddc199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0ddad9d640 1 -- 172.21.15.33:0/2150600744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ddc1045c0 msgr2=0x7f0ddc1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0ddad9d640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ddc1045c0 0x7f0ddc1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0ddad9d640 1 -- 172.21.15.33:0/2150600744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ddc103ca0 msgr2=0x7f0ddc199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0ddad9d640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ddc103ca0 0x7f0ddc199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0ddad9d640 1 -- 172.21.15.33:0/2150600744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ddc19f330 con 0x7f0ddc10d100 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0ddad9d640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc19eb60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0098d0 tx=0x7f0dcc0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0dd37fe640 1 -- 172.21.15.33:0/2150600744 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc003b70 con 0x7f0ddc10d100 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0dd37fe640 1 -- 172.21.15.33:0/2150600744 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dcc003d10 con 0x7f0ddc10d100 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ddc19f5c0 con 0x7f0ddc10d100 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0dd37fe640 1 -- 172.21.15.33:0/2150600744 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dcc02fa20 con 0x7f0ddc10d100 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.643+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ddc1a5ce0 con 0x7f0ddc10d100 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0dd37fe640 1 -- 172.21.15.33:0/2150600744 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0dcc02fc40 con 0x7f0ddc10d100 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 Processor -- start 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 -- start start 2024-09-10T21:16:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b2c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2ca10 0x559d26b3c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559d26aa2ea0 con 0x559d26b3cdb0 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559d26a922d0 con 0x559d26b2ca10 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.646+0000 7f0de305e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d26a960e0 con 0x559d26b2c0f0 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32818/0 (socket says 172.21.15.33:32818) 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/2520259526 learned_addr learned my addr 172.21.15.33:0/2520259526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b2c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dda59c640 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2ca10 0x559d26b3c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd37fe640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0dc80b52e0 0x7f0dc80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/2520259526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 msgr2=0x559d26b2c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b2c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0ddad9d640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0dc80b52e0 0x7f0dc80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/2520259526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2ca10 msgr2=0x559d26b3c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2ca10 0x559d26b3c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/2520259526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d267f0650 con 0x559d26b3cdb0 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd37fe640 1 -- 172.21.15.33:0/2150600744 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0dcc106ea0 con 0x7f0ddc10d100 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3f1a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4012770 tx=0x7f0dc400b5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fecc75c647865d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.647+0000 7f0dd0ff9640 1 -- 172.21.15.33:0/2520259526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4012cf0 con 0x559d26b3cdb0 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0dd0ff9640 1 -- 172.21.15.33:0/2520259526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc4005020 con 0x559d26b3cdb0 2024-09-10T21:16:22.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0dd0ff9640 1 -- 172.21.15.33:0/2520259526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40049a0 con 0x559d26b3cdb0 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 -- 172.21.15.33:0/2520259526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 msgr2=0x559d26b3f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3f1a0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4012770 tx=0x7f0dc400b5c0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0ddad9d640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0dc80b52e0 0x7f0dc80b77a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc04d000 tx=0x7f0dcc09ac40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 -- 172.21.15.33:0/2520259526 shutdown_connections 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3f1a0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2ca10 0x559d26b3c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 --2- 172.21.15.33:0/2520259526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b2c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 -- 172.21.15.33:0/2520259526 >> 172.21.15.33:0/2520259526 conn(0x559d26b29f20 msgr2=0x559d26b2b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 -- 172.21.15.33:0/2520259526 shutdown_connections 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.648+0000 7f0de305e740 1 -- 172.21.15.33:0/2520259526 wait complete. 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 -- start start 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 Processor -- start 2024-09-10T21:16:22.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b3c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b39ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559d268f3700 con 0x559d26b3cdb0 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559d26a2d160 con 0x559d26b2c0f0 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d26b3fa60 con 0x559d26b2ca10 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0de305e740 10 client.0 _ll_register_callbacks cb 0x559d26a469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0dd9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0dda59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b39ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.649+0000 7f0dd9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32832/0 (socket says 172.21.15.33:32832) 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/1293383232 learned_addr learned my addr 172.21.15.33:0/1293383232 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b3c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/1293383232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 msgr2=0x559d26b39ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b39ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 msgr2=0x559d26b3c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b3c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/1293383232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d26a5ac90 con 0x559d26b3cdb0 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3a110 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40125b0 tx=0x7f0dc401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56171476b25b01f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/1293383232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4010770 con 0x559d26b3cdb0 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/1293383232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0dc4010910 con 0x559d26b3cdb0 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc4005020 con 0x559d26b3cdb0 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/1293383232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0dc4010b10 con 0x559d26b3cdb0 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0de305e740 10 client.17721 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:22.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.650+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x559d26b4b3e0 con 0x559d26b3cdb0 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/1293383232 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc4005020 con 0x559d26b3cdb0 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/1293383232 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f0dc402e020 con 0x559d26b3cdb0 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0de305e740 10 client.17721 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0de305e740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0de305e740 20 client.17721 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0de305e740 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26b4c780 0x559d26b4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0de305e740 4 client.17721 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d26aa4df0 con 0x559d26b4c780 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0dda59c640 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26b4c780 0x559d26b4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0dc1ffb640 10 client.17721 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:22.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.651+0000 7f0dda59c640 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26b4c780 0x559d26b4eb70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55b970cb2c541bdb server_cookie=25f66830108eef4 in_seq=0 out_seq=0 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/1293383232 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559d26aa4df0 con 0x559d26b4c780 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0dc1ffb640 10 client.17721 handle_command_reply: tid=0 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 client.17721 shutdown 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 msgr2=0x559d26b3a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3a110 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40125b0 tx=0x7f0dc401b040 comp rx=0 tx=0).stop 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 shutdown_connections 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26b4c780 0x559d26b4eb70 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b3a110 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b39ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0dc1ffb640 0 client.17721 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 --2- 172.21.15.33:0/1293383232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b3c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 >> 172.21.15.33:0/1293383232 conn(0x559d26b29f20 msgr2=0x559d26b2fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 shutdown_connections 2024-09-10T21:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 1 -- 172.21.15.33:0/1293383232 wait complete. 2024-09-10T21:16:22.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.652+0000 7f0de305e740 20 client.17721 trim_cache size 0 max 16384 2024-09-10T21:16:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 Processor -- start 2024-09-10T21:16:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 -- start start 2024-09-10T21:16:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b29760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b48cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1ac1a0 con 0x559d26b3cdb0 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1add40 con 0x559d26b2c0f0 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0de305e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1ac870 con 0x559d26b2ca10 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0dd9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0dd9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b2b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32840/0 (socket says 172.21.15.33:32840) 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.666+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/4206892130 learned_addr learned my addr 172.21.15.33:0/4206892130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b29760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dda59c640 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b48cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/4206892130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 msgr2=0x559d26b48cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b48cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/4206892130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 msgr2=0x559d26b29760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b29760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dd9d9b640 1 -- 172.21.15.33:0/4206892130 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d26b29d50 con 0x559d26b3cdb0 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b2b3f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40057c0 tx=0x7f0dc4017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85100e0c2dd68780 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dc3fff640 1 -- 172.21.15.33:0/4206892130 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4004fe0 con 0x559d26b3cdb0 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dc3fff640 1 -- 172.21.15.33:0/4206892130 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc4005180 con 0x559d26b3cdb0 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.667+0000 7f0dc3fff640 1 -- 172.21.15.33:0/4206892130 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4010870 con 0x559d26b3cdb0 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 -- 172.21.15.33:0/4206892130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 msgr2=0x559d26b2b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b2b3f0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40057c0 tx=0x7f0dc4017040 comp rx=0 tx=0).stop 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 -- 172.21.15.33:0/4206892130 shutdown_connections 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b3cdb0 0x559d26b2b3f0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2ca10 0x559d26b48cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 --2- 172.21.15.33:0/4206892130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b2c0f0 0x559d26b29760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 -- 172.21.15.33:0/4206892130 >> 172.21.15.33:0/4206892130 conn(0x559d26b29f20 msgr2=0x559d26b2ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 -- 172.21.15.33:0/4206892130 shutdown_connections 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 -- 172.21.15.33:0/4206892130 wait complete. 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.668+0000 7f0de305e740 1 -- start start 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 1 Processor -- start 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b44d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b2ca10 0x559d26b45290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b3cdb0 0x559d26b41d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1ad610 con 0x559d26b2ca10 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1acf40 con 0x559d26b3cdb0 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ddc1aa160 con 0x559d26b2c0f0 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0de305e740 10 client.0 _ll_register_callbacks cb 0x7f0dd40097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:22.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b2ca10 0x559d26b45290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b2ca10 0x559d26b45290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32846/0 (socket says 172.21.15.33:32846) 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 -- 172.21.15.33:0/4282769108 learned_addr learned my addr 172.21.15.33:0/4282769108 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b3cdb0 0x559d26b41d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b44d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 -- 172.21.15.33:0/4282769108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 msgr2=0x559d26b44d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b44d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 -- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b3cdb0 msgr2=0x559d26b41d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b3cdb0 0x559d26b41d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.669+0000 7f0dda59c640 1 -- 172.21.15.33:0/4282769108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d26b45950 con 0x559d26b2ca10 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0dda59c640 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b2ca10 0x559d26b45290 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4004700 tx=0x7f0dd40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291e1d434bae6bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0dc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/4282769108 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd4029060 con 0x559d26b2ca10 2024-09-10T21:16:22.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/4282769108 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0dd401f610 con 0x559d26b2ca10 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc4005180 con 0x559d26b2ca10 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/4282769108 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0dd401f810 con 0x559d26b2ca10 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 10 client.17727 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x559d26bb9ab0 con 0x559d26b2ca10 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/4282769108 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0dc4005180 con 0x559d26b2ca10 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/4282769108 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f0dd402e070 con 0x559d26b2ca10 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 10 client.17727 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 10 client.17727 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 20 client.17727 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26bba2c0 0x559d26bbc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 4 client.17727 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:16:22.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.670+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d26aa08a0 con 0x559d26bba2c0 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26bba2c0 0x559d26bbc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0dd9d9b640 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26bba2c0 0x559d26bbc6e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8403d25b1dc1b243 server_cookie=3589b3f170e35286 in_seq=0 out_seq=0 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0dc1ffb640 10 client.17727 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0dc1ffb640 1 -- 172.21.15.33:0/4282769108 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d26aa08a0 con 0x559d26bba2c0 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0dc1ffb640 10 client.17727 handle_command_reply: tid=0 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 client.17727 shutdown 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b2ca10 msgr2=0x559d26b45290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b2ca10 0x559d26b45290 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4004700 tx=0x7f0dd40039c0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 shutdown_connections 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559d26bba2c0 0x559d26bbc6e0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559d26b3cdb0 0x559d26b41d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559d26b2ca10 0x559d26b45290 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0dc1ffb640 0 client.17727 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 --2- 172.21.15.33:0/4282769108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d26b2c0f0 0x559d26b44d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 >> 172.21.15.33:0/4282769108 conn(0x559d26b29f20 msgr2=0x559d26b3b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 shutdown_connections 2024-09-10T21:16:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.671+0000 7f0de305e740 1 -- 172.21.15.33:0/4282769108 wait complete. 2024-09-10T21:16:22.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de305e740 20 client.17727 trim_cache size 0 max 16384 2024-09-10T21:16:22.686 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:16:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0dc80b52e0 msgr2=0x7f0dc80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0dc80b52e0 0x7f0dc80b77a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc04d000 tx=0x7f0dcc09ac40 comp rx=0 tx=0).stop 2024-09-10T21:16:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 msgr2=0x7f0ddc19eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc19eb60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc0098d0 tx=0x7f0dcc0317b0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 shutdown_connections 2024-09-10T21:16:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0dc80b52e0 0x7f0dc80b77a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ddc10d100 0x7f0ddc19eb60 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ddc1045c0 0x7f0ddc1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 --2- 172.21.15.33:0/2150600744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0ddc103ca0 0x7f0ddc199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:22.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 >> 172.21.15.33:0/2150600744 conn(0x7f0ddc0fd700 msgr2=0x7f0ddc0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:22.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 shutdown_connections 2024-09-10T21:16:22.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:22.672+0000 7f0de0c89640 1 -- 172.21.15.33:0/2150600744 wait complete. 2024-09-10T21:16:22.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948481 6.83 MB/sec execute 563 sec latency 4929.687 ms 2024-09-10T21:16:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:22 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:22 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:23.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:22 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:16:23.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948481 6.81 MB/sec execute 564 sec latency 1611.496 ms 2024-09-10T21:16:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:23 smithi174 ceph-mon[35635]: pgmap v718: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 2.5 MiB/s wr, 255 op/s 2024-09-10T21:16:24.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:23 smithi033 ceph-mon[30860]: pgmap v718: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 2.5 MiB/s wr, 255 op/s 2024-09-10T21:16:24.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:23 smithi104 ceph-mon[35325]: pgmap v718: 97 pgs: 97 active+clean; 346 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 2.5 MiB/s wr, 255 op/s 2024-09-10T21:16:24.449 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:16:24.449 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:16:24.722 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:24.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948481 6.80 MB/sec execute 565 sec latency 2611.589 ms 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.218+0000 7fd1273f3640 1 -- 172.21.15.33:0/4108004635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120101c90 msgr2=0x7fd120102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.218+0000 7fd1273f3640 1 --2- 172.21.15.33:0/4108004635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120101c90 0x7fd120102070 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd110009ca0 tx=0x7fd11002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 -- 172.21.15.33:0/4108004635 shutdown_connections 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 --2- 172.21.15.33:0/4108004635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd120108020 0x7fd12010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 --2- 172.21.15.33:0/4108004635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1201076a0 0x7fd120107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 --2- 172.21.15.33:0/4108004635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120101c90 0x7fd120102070 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 -- 172.21.15.33:0/4108004635 >> 172.21.15.33:0/4108004635 conn(0x7fd1200fd700 msgr2=0x7fd1200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 -- 172.21.15.33:0/4108004635 shutdown_connections 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 -- 172.21.15.33:0/4108004635 wait complete. 2024-09-10T21:16:25.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 Processor -- start 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.219+0000 7fd1273f3640 1 -- start start 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd1273f3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd120101c90 0x7fd120199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd1273f3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1201076a0 0x7fd1201997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd1273f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120108020 0x7fd12019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd1273f3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd120112e00 con 0x7fd120108020 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd1273f3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd120112c80 con 0x7fd1201076a0 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd1273f3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd120112f80 con 0x7fd120101c90 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120108020 0x7fd12019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120108020 0x7fd12019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32860/0 (socket says 172.21.15.33:32860) 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 -- 172.21.15.33:0/3309377332 learned_addr learned my addr 172.21.15.33:0/3309377332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:25.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd12498e640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1201076a0 0x7fd1201997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd12518f640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd120101c90 0x7fd120199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 -- 172.21.15.33:0/3309377332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd120101c90 msgr2=0x7fd120199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd120101c90 0x7fd120199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 -- 172.21.15.33:0/3309377332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1201076a0 msgr2=0x7fd1201997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1201076a0 0x7fd1201997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 -- 172.21.15.33:0/3309377332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12019f320 con 0x7fd120108020 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd12518f640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd120101c90 0x7fd120199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd125990640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120108020 0x7fd12019eb50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd11800d750 tx=0x7fd11800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd11e7fc640 1 -- 172.21.15.33:0/3309377332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118017070 con 0x7fd120108020 2024-09-10T21:16:25.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd11e7fc640 1 -- 172.21.15.33:0/3309377332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1180054d0 con 0x7fd120108020 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12019f610 con 0x7fd120108020 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.220+0000 7fd11e7fc640 1 -- 172.21.15.33:0/3309377332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118004c60 con 0x7fd120108020 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.221+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1201a5ea0 con 0x7fd120108020 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.223+0000 7fd1297c8740 1 Processor -- start 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.223+0000 7fd11e7fc640 1 -- 172.21.15.33:0/3309377332 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd118004e00 con 0x7fd120108020 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.223+0000 7fd1297c8740 1 -- start start 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fcf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fdb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd1297c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564fa4f45e60 con 0x564fa4fcf800 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd1297c8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564fa4f35290 con 0x564fa4fceee0 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd1297c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564fa4f390a0 con 0x564fa4fdb790 2024-09-10T21:16:25.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fdb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fdb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32876/0 (socket says 172.21.15.33:32876) 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 -- 172.21.15.33:0/3631445999 learned_addr learned my addr 172.21.15.33:0/3631445999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12498e640 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fcf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12498e640 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 -- 172.21.15.33:0/3631445999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 msgr2=0x564fa4fddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 -- 172.21.15.33:0/3631445999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 msgr2=0x564fa4fcf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd11e7fc640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1080b5360 0x7fd1080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fcf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 -- 172.21.15.33:0/3631445999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa4c93930 con 0x564fa4fcf800 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12498e640 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd125990640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1080b5360 0x7fd1080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd12518f640 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fdb250 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fa0 tx=0x7fd1100093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfb24ed24810431 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.224+0000 7fd11e7fc640 1 -- 172.21.15.33:0/3309377332 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd1180e54e0 con 0x7fd120108020 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.225+0000 7fd1077fe640 1 -- 172.21.15.33:0/3631445999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110031cf0 con 0x564fa4fcf800 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.225+0000 7fd1077fe640 1 -- 172.21.15.33:0/3631445999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd110040070 con 0x564fa4fcf800 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.225+0000 7fd1077fe640 1 -- 172.21.15.33:0/3631445999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1100042a0 con 0x564fa4fcf800 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.225+0000 7fd1297c8740 1 -- 172.21.15.33:0/3631445999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 msgr2=0x564fa4fdb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.225+0000 7fd1297c8740 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fdb250 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fa0 tx=0x7fd1100093e0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.225+0000 7fd125990640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1080b5360 0x7fd1080b7820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1180ab9a0 tx=0x7fd118093040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 -- 172.21.15.33:0/3631445999 shutdown_connections 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fdb250 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 --2- 172.21.15.33:0/3631445999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fcf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 -- 172.21.15.33:0/3631445999 >> 172.21.15.33:0/3631445999 conn(0x564fa4fccd10 msgr2=0x564fa4fcdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 -- 172.21.15.33:0/3631445999 shutdown_connections 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 -- 172.21.15.33:0/3631445999 wait complete. 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 -- start start 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.226+0000 7fd1297c8740 1 Processor -- start 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fd8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fd91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564fa4d96890 con 0x564fa4fcf800 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x564fa4ecfd40 con 0x564fa4fceee0 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564fa4fde3e0 con 0x564fa4fdb790 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 10 client.0 _ll_register_callbacks cb 0x564fa4f33c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:25.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32880/0 (socket says 172.21.15.33:32880) 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 -- 172.21.15.33:0/2331819409 learned_addr learned my addr 172.21.15.33:0/2331819409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12498e640 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fd8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 -- 172.21.15.33:0/2331819409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 msgr2=0x564fa4fd91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fd91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 -- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 msgr2=0x564fa4fd8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fd8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 -- 172.21.15.33:0/2331819409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa4fe77f0 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd12518f640 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd8c80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd110009620 tx=0x7fd11002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b492848f2eec4c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd104ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd104ff9640 1 -- 172.21.15.33:0/2331819409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11002f860 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd104ff9640 1 -- 172.21.15.33:0/2331819409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd11002fa00 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.227+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110040070 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd104ff9640 1 -- 172.21.15.33:0/2331819409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd11002fc00 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 10 client.17739 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564fa4fea180 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd104ff9640 1 -- 172.21.15.33:0/2331819409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd110040070 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd104ff9640 1 -- 172.21.15.33:0/2331819409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fd110030cd0 con 0x564fa4fcf800 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 10 client.17739 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:25.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 10 client.17739 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 20 client.17739 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa4feb4d0 0x564fa4fed8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 4 client.17739 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fa4f47db0 con 0x564fa4feb4d0 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.228+0000 7fd12498e640 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa4feb4d0 0x564fa4fed8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.229+0000 7fd12498e640 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa4feb4d0 0x564fa4fed8c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d8c39a8cc0c9300 server_cookie=17802db5627170b9 in_seq=0 out_seq=0 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.229+0000 7fd104ff9640 10 client.17739 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.229+0000 7fd104ff9640 1 -- 172.21.15.33:0/2331819409 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564fa4f47db0 con 0x564fa4feb4d0 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.229+0000 7fd104ff9640 10 client.17739 handle_command_reply: tid=0 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.229+0000 7fd1297c8740 1 client.17739 shutdown 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 msgr2=0x564fa4fd8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd8c80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd110009620 tx=0x7fd11002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 shutdown_connections 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa4feb4d0 0x564fa4fed8c0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fd91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd8c80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 --2- 172.21.15.33:0/2331819409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fd8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 >> 172.21.15.33:0/2331819409 conn(0x564fa4fccd10 msgr2=0x564fa4fcbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd104ff9640 0 client.17739 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 shutdown_connections 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 1 -- 172.21.15.33:0/2331819409 wait complete. 2024-09-10T21:16:25.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.230+0000 7fd1297c8740 20 client.17739 trim_cache size 0 max 16384 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 Processor -- start 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 -- start start 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fd5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4ff2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1201ae3f0 con 0x564fa4fcf800 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1201aff90 con 0x564fa4fceee0 2024-09-10T21:16:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd1297c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1201aeac0 con 0x564fa4fdb790 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32884/0 (socket says 172.21.15.33:32884) 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 -- 172.21.15.33:0/557492022 learned_addr learned my addr 172.21.15.33:0/557492022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12498e640 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4ff2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 -- 172.21.15.33:0/557492022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 msgr2=0x564fa4ff2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4ff2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 -- 172.21.15.33:0/557492022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 msgr2=0x564fa4fd5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fd5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12518f640 1 -- 172.21.15.33:0/557492022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa4fcd790 con 0x564fa4fcf800 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.248+0000 7fd12498e640 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4ff2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd12518f640 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd60a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fa0 tx=0x7fd110031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262ec99a7ab100a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd106ffd640 1 -- 172.21.15.33:0/557492022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110003970 con 0x564fa4fcf800 2024-09-10T21:16:25.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd106ffd640 1 -- 172.21.15.33:0/557492022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd110040040 con 0x564fa4fcf800 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd106ffd640 1 -- 172.21.15.33:0/557492022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110004720 con 0x564fa4fcf800 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd1297c8740 1 -- 172.21.15.33:0/557492022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 msgr2=0x564fa4fd60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd1297c8740 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd60a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fa0 tx=0x7fd110031700 comp rx=0 tx=0).stop 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd1297c8740 1 -- 172.21.15.33:0/557492022 shutdown_connections 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd1297c8740 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4ff2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd1297c8740 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fd60a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.249+0000 7fd1297c8740 1 --2- 172.21.15.33:0/557492022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fd5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 -- 172.21.15.33:0/557492022 >> 172.21.15.33:0/557492022 conn(0x564fa4fccd10 msgr2=0x564fa4fd6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 -- 172.21.15.33:0/557492022 shutdown_connections 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 -- 172.21.15.33:0/557492022 wait complete. 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 -- start start 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 Processor -- start 2024-09-10T21:16:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fe36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fe3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fde660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1201af860 con 0x564fa4fcf800 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1201af190 con 0x564fa4fceee0 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.250+0000 7fd1297c8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1201ac3b0 con 0x564fa4fdb790 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd1297c8740 10 client.0 _ll_register_callbacks cb 0x7fd114009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fe3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fe3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32896/0 (socket says 172.21.15.33:32896) 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 -- 172.21.15.33:0/844519222 learned_addr learned my addr 172.21.15.33:0/844519222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12498e640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fde660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12498e640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fe36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 -- 172.21.15.33:0/844519222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 msgr2=0x564fa4fde660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fde660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 -- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 msgr2=0x564fa4fe36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fe36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 -- 172.21.15.33:0/844519222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa4fdeba0 con 0x564fa4fcf800 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12498e640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fe36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd12518f640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fe3bf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd110006010 tx=0x7fd1100095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703d340743529c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd104ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd104ff9640 1 -- 172.21.15.33:0/844519222 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd110065030 con 0x564fa4fcf800 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd104ff9640 1 -- 172.21.15.33:0/844519222 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd110003970 con 0x564fa4fcf800 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd110040040 con 0x564fa4fcf800 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd104ff9640 1 -- 172.21.15.33:0/844519222 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fd110003b70 con 0x564fa4fcf800 2024-09-10T21:16:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd1297c8740 10 client.17745 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.251+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x564fa5058900 con 0x564fa4fcf800 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd104ff9640 1 -- 172.21.15.33:0/844519222 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd110040040 con 0x564fa4fcf800 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd104ff9640 1 -- 172.21.15.33:0/844519222 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7fd1100046d0 con 0x564fa4fcf800 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd1297c8740 10 client.17745 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd1297c8740 10 client.17745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd1297c8740 20 client.17745 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd1297c8740 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa5059020 0x564fa505b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd1297c8740 4 client.17745 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564fa4f43860 con 0x564fa5059020 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.252+0000 7fd12498e640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa5059020 0x564fa505b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.253+0000 7fd104ff9640 10 client.17745 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.253+0000 7fd12498e640 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa5059020 0x564fa505b440 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f787ad8ffebca0 server_cookie=938e28bc65f29ef in_seq=0 out_seq=0 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.254+0000 7fd104ff9640 1 -- 172.21.15.33:0/844519222 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564fa4f43860 con 0x564fa5059020 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.254+0000 7fd104ff9640 10 client.17745 handle_command_reply: tid=0 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.254+0000 7fd1297c8740 1 client.17745 shutdown 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.254+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 msgr2=0x564fa4fe3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.254+0000 7fd1297c8740 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fe3bf0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd110006010 tx=0x7fd1100095b0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 shutdown_connections 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x564fa5059020 0x564fa505b440 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fa4fdb790 0x564fa4fde660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564fa4fcf800 0x564fa4fe3bf0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 --2- 172.21.15.33:0/844519222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564fa4fceee0 0x564fa4fe36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 >> 172.21.15.33:0/844519222 conn(0x564fa4fccd10 msgr2=0x564fa4fd1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd104ff9640 0 client.17745 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 shutdown_connections 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 1 -- 172.21.15.33:0/844519222 wait complete. 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1297c8740 20 client.17745 trim_cache size 0 max 16384 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "0d738eee-48a0-490a-83f5-59006db2e660", 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:16:25.269 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:16:25.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1080b5360 msgr2=0x7fd1080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1273f3640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1080b5360 0x7fd1080b7820 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd1180ab9a0 tx=0x7fd118093040 comp rx=0 tx=0).stop 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120108020 msgr2=0x7fd12019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.255+0000 7fd1273f3640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120108020 0x7fd12019eb50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd11800d750 tx=0x7fd11800dc10 comp rx=0 tx=0).stop 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 shutdown_connections 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1080b5360 0x7fd1080b7820 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd120108020 0x7fd12019eb50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1201076a0 0x7fd1201997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 --2- 172.21.15.33:0/3309377332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd120101c90 0x7fd120199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 >> 172.21.15.33:0/3309377332 conn(0x7fd1200fd700 msgr2=0x7fd1201058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 shutdown_connections 2024-09-10T21:16:25.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:25.256+0000 7fd1273f3640 1 -- 172.21.15.33:0/3309377332 wait complete. 2024-09-10T21:16:25.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948493 6.79 MB/sec execute 566 sec latency 3168.865 ms 2024-09-10T21:16:26.027 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:16:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:25 smithi174 ceph-mon[35635]: pgmap v719: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-09-10T21:16:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:25 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:25 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:25 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:25 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:25 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:26.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:25 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:25 smithi033 ceph-mon[30860]: pgmap v719: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-09-10T21:16:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:25 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:25 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:25 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:25 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:25 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:26.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:25 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.290 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:26.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:25 smithi104 ceph-mon[35325]: pgmap v719: 97 pgs: 97 active+clean; 347 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 500 KiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-09-10T21:16:26.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:25 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:26.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:25 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:26.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:25 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:25 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:25 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:26.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:25 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:26.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 948493 6.78 MB/sec execute 567 sec latency 1407.080 ms 2024-09-10T21:16:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 -- 172.21.15.33:0/1338605382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be410d140 msgr2=0x7f1be410f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/1338605382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be410d140 0x7f1be410f5a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009ca0 tx=0x7f1bdc02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 -- 172.21.15.33:0/1338605382 shutdown_connections 2024-09-10T21:16:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/1338605382 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be410d140 0x7f1be410f5a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/1338605382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be4104600 0x7f1be410caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/1338605382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1be4103ce0 0x7f1be41040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 -- 172.21.15.33:0/1338605382 >> 172.21.15.33:0/1338605382 conn(0x7f1be40fd720 msgr2=0x7f1be40ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 -- 172.21.15.33:0/1338605382 shutdown_connections 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 -- 172.21.15.33:0/1338605382 wait complete. 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 Processor -- start 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.843+0000 7f1beaaa4640 1 -- start start 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1beaaa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be4103ce0 0x7f1be41992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1beaaa4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be4104600 0x7f1be41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1beaaa4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1be410d140 0x7f1be419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1beaaa4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1be4112db0 con 0x7f1be4103ce0 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1beaaa4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1be4112c30 con 0x7f1be4104600 2024-09-10T21:16:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1beaaa4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1be4112f30 con 0x7f1be410d140 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be4103ce0 0x7f1be41992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be4103ce0 0x7f1be41992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32906/0 (socket says 172.21.15.33:32906) 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 -- 172.21.15.33:0/2483775402 learned_addr learned my addr 172.21.15.33:0/2483775402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be9041640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1be410d140 0x7f1be419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be3fff640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be4104600 0x7f1be41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 -- 172.21.15.33:0/2483775402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1be410d140 msgr2=0x7f1be419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1be410d140 0x7f1be419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 -- 172.21.15.33:0/2483775402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be4104600 msgr2=0x7f1be41997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be4104600 0x7f1be41997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 -- 172.21.15.33:0/2483775402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be419f350 con 0x7f1be4103ce0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be9041640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1be410d140 0x7f1be419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be8840640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be4103ce0 0x7f1be41992a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400d770 tx=0x7f1bd400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be1ffb640 1 -- 172.21.15.33:0/2483775402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4017070 con 0x7f1be4103ce0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be1ffb640 1 -- 172.21.15.33:0/2483775402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd40054d0 con 0x7f1be4103ce0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be419f640 con 0x7f1be4103ce0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.844+0000 7f1be1ffb640 1 -- 172.21.15.33:0/2483775402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4004c30 con 0x7f1be4103ce0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.845+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be41a5d00 con 0x7f1be4103ce0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1be1ffb640 1 -- 172.21.15.33:0/2483775402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1bd4004e10 con 0x7f1be4103ce0 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 Processor -- start 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 -- start start 2024-09-10T21:16:26.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e5742860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e574e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e5751120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8e56b8e50 con 0x55f8e5742480 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f8e56a8280 con 0x55f8e574ed30 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.847+0000 7f1bece79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f8e56ac090 con 0x55f8e5742da0 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e5742860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e5742860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32910/0 (socket says 172.21.15.33:32910) 2024-09-10T21:16:26.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 -- 172.21.15.33:0/3898967819 learned_addr learned my addr 172.21.15.33:0/3898967819 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be8840640 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e574e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be3fff640 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e5751120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be1ffb640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1bd00b52e0 0x7f1bd00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 -- 172.21.15.33:0/3898967819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 msgr2=0x55f8e574e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e574e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 -- 172.21.15.33:0/3898967819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 msgr2=0x55f8e5751120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e5751120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 -- 172.21.15.33:0/3898967819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8e5406650 con 0x55f8e5742480 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be3fff640 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e5751120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:26.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be1ffb640 1 -- 172.21.15.33:0/2483775402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1bd40e5780 con 0x7f1be4103ce0 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e5742860 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009800 tx=0x7f1bdc009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64f5e84b051d795 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1be9041640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1bd00b52e0 0x7f1bd00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.848+0000 7f1bdaffd640 1 -- 172.21.15.33:0/3898967819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc04d070 con 0x55f8e5742480 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bdaffd640 1 -- 172.21.15.33:0/3898967819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bdc036040 con 0x55f8e5742480 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bdaffd640 1 -- 172.21.15.33:0/3898967819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc02f660 con 0x55f8e5742480 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1be9041640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1bd00b52e0 0x7f1bd00b77a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc04a000 tx=0x7f1bdc04b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 -- 172.21.15.33:0/3898967819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 msgr2=0x55f8e5742860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e5742860 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc009800 tx=0x7f1bdc009830 comp rx=0 tx=0).stop 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 -- 172.21.15.33:0/3898967819 shutdown_connections 2024-09-10T21:16:26.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e5751120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e574e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 --2- 172.21.15.33:0/3898967819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e5742860 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 -- 172.21.15.33:0/3898967819 >> 172.21.15.33:0/3898967819 conn(0x55f8e57402b0 msgr2=0x55f8e5741450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 -- 172.21.15.33:0/3898967819 shutdown_connections 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.849+0000 7f1bece79740 1 -- 172.21.15.33:0/3898967819 wait complete. 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 -- start start 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 Processor -- start 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742480 0x55f8e574e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742da0 0x55f8e574ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e574c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f8e5509700 con 0x55f8e5742da0 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f8e5643110 con 0x55f8e574ed30 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f8e57519b0 con 0x55f8e5742480 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1bece79740 10 client.0 _ll_register_callbacks cb 0x55f8e565c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742da0 0x55f8e574ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1be3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742da0 0x55f8e574ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32916/0 (socket says 172.21.15.33:32916) 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1be3fff640 1 -- 172.21.15.33:0/2321729844 learned_addr learned my addr 172.21.15.33:0/2321729844 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1be9041640 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e574c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.850+0000 7f1be8840640 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742480 0x55f8e574e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1be3fff640 1 -- 172.21.15.33:0/2321729844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742480 msgr2=0x55f8e574e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1be3fff640 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742480 0x55f8e574e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1be3fff640 1 -- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 msgr2=0x55f8e574c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1be3fff640 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e574c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1be3fff640 1 -- 172.21.15.33:0/2321729844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8e5670c40 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1be3fff640 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742da0 0x55f8e574ba50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00be70 tx=0x7f1bcc0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc8ababd7db53ee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2321729844 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bcc012650 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2321729844 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1bcc0127f0 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bdc036040 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2321729844 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1bcc0129f0 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bece79740 10 client.17757 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f8e575d350 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.851+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2321729844 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bdc036040 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2321729844 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f1bcc01bc80 con 0x55f8e5742da0 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bece79740 10 client.17757 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:26.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bece79740 10 client.17757 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bece79740 20 client.17757 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bece79740 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e575e820 0x55f8e5760c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bece79740 4 client.17757 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8e56bada0 con 0x55f8e575e820 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1be8840640 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e575e820 0x55f8e5760c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1be8840640 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e575e820 0x55f8e5760c30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3968b2452bb5ffd server_cookie=6a8aec6e632698f9 in_seq=0 out_seq=0 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.852+0000 7f1bc3fff640 10 client.17757 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2321729844 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8e56bada0 con 0x55f8e575e820 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bc3fff640 10 client.17757 handle_command_reply: tid=0 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 client.17757 shutdown 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742da0 msgr2=0x55f8e574ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742da0 0x55f8e574ba50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc00be70 tx=0x7f1bcc0117b0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 shutdown_connections 2024-09-10T21:16:26.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e575e820 0x55f8e5760c30 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e574ed30 0x55f8e574c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742da0 0x55f8e574ba50 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bc3fff640 0 client.17757 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 --2- 172.21.15.33:0/2321729844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742480 0x55f8e574e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 >> 172.21.15.33:0/2321729844 conn(0x55f8e57402b0 msgr2=0x55f8e5745e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 shutdown_connections 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 1 -- 172.21.15.33:0/2321729844 wait complete. 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.853+0000 7f1bece79740 20 client.17757 trim_cache size 0 max 16384 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 Processor -- start 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 -- start start 2024-09-10T21:16:26.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e575ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742da0 0x55f8e5750120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e574ed30 0x55f8e5750660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41ae470 con 0x55f8e5742480 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41b0010 con 0x55f8e5742da0 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1bece79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41aeb40 con 0x55f8e574ed30 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1be8840640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e575ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1be8840640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e575ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32924/0 (socket says 172.21.15.33:32924) 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.871+0000 7f1be8840640 1 -- 172.21.15.33:0/626662147 learned_addr learned my addr 172.21.15.33:0/626662147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1be3fff640 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742da0 0x55f8e5750120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1be8840640 1 -- 172.21.15.33:0/626662147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e574ed30 msgr2=0x55f8e5750660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1be8840640 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e574ed30 0x55f8e5750660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1be8840640 1 -- 172.21.15.33:0/626662147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742da0 msgr2=0x55f8e5750120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1be8840640 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742da0 0x55f8e5750120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1be8840640 1 -- 172.21.15.33:0/626662147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8e5740f90 con 0x55f8e5742480 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1be8840640 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e575ac10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40abc40 tx=0x7f1bd4093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eec2de80bc50925 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bda7fc640 1 -- 172.21.15.33:0/626662147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4020040 con 0x55f8e5742480 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bda7fc640 1 -- 172.21.15.33:0/626662147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd40ac640 con 0x55f8e5742480 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bda7fc640 1 -- 172.21.15.33:0/626662147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd40a81b0 con 0x55f8e5742480 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bece79740 1 -- 172.21.15.33:0/626662147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 msgr2=0x55f8e575ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bece79740 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e575ac10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1bd40abc40 tx=0x7f1bd4093040 comp rx=0 tx=0).stop 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bece79740 1 -- 172.21.15.33:0/626662147 shutdown_connections 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bece79740 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e574ed30 0x55f8e5750660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bece79740 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742da0 0x55f8e5750120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bece79740 1 --2- 172.21.15.33:0/626662147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e5742480 0x55f8e575ac10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.872+0000 7f1bece79740 1 -- 172.21.15.33:0/626662147 >> 172.21.15.33:0/626662147 conn(0x55f8e57402b0 msgr2=0x55f8e5746ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 -- 172.21.15.33:0/626662147 shutdown_connections 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 -- 172.21.15.33:0/626662147 wait complete. 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 -- start start 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 Processor -- start 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742480 0x55f8e5756fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e5757540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e574ed30 0x55f8e5751d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41af8e0 con 0x55f8e574ed30 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41af210 con 0x55f8e5742480 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1be41ac430 con 0x55f8e5742da0 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1bece79740 10 client.0 _ll_register_callbacks cb 0x7f1be406c4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1be9041640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e574ed30 0x55f8e5751d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1be9041640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e574ed30 0x55f8e5751d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32934/0 (socket says 172.21.15.33:32934) 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.873+0000 7f1be9041640 1 -- 172.21.15.33:0/2402170978 learned_addr learned my addr 172.21.15.33:0/2402170978 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be3fff640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e5757540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be8840640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742480 0x55f8e5756fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be9041640 1 -- 172.21.15.33:0/2402170978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 msgr2=0x55f8e5757540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be9041640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e5757540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be9041640 1 -- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742480 msgr2=0x55f8e5756fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be9041640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742480 0x55f8e5756fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be9041640 1 -- 172.21.15.33:0/2402170978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8e5752550 con 0x55f8e574ed30 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be3fff640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e5757540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:16:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1be9041640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e574ed30 0x55f8e5751d20 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000f80 tx=0x7f1bdc005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ab98ccc0ce8934 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bc3fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2402170978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bdc04d040 con 0x55f8e574ed30 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2402170978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1bdc031790 con 0x55f8e574ed30 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bd40ac640 con 0x55f8e574ed30 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2402170978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f1bdc02f660 con 0x55f8e574ed30 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bece79740 10 client.17763 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55f8e57cc370 con 0x55f8e574ed30 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.874+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2402170978 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd40ac640 con 0x55f8e574ed30 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2402170978 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f1bdc043050 con 0x55f8e574ed30 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bece79740 10 client.17763 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bece79740 10 client.17763 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bece79740 20 client.17763 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bece79740 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e57cc770 0x55f8e57ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bece79740 4 client.17763 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8e56b6850 con 0x55f8e57cc770 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "scrub active (0 inodes in the stack)", 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:16:26.958 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "0d738eee-48a0-490a-83f5-59006db2e660", 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.959 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.960 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:16:26.961 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.962 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.963 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.964 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.965 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:16:26.966 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.967 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.968 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.969 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:16:26.970 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1be8840640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e57cc770 0x55f8e57ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1bc3fff640 10 client.17763 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.875+0000 7f1be8840640 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e57cc770 0x55f8e57ceb60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97f113ad48c3e00d server_cookie=1be66b9af9abca13 in_seq=0 out_seq=0 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bc3fff640 1 -- 172.21.15.33:0/2402170978 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5120 (crc 0 0 0) 0x55f8e56b6850 con 0x55f8e57cc770 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bc3fff640 10 client.17763 handle_command_reply: tid=0 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 client.17763 shutdown 2024-09-10T21:16:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e574ed30 msgr2=0x55f8e5751d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e574ed30 0x55f8e5751d20 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc000f80 tx=0x7f1bdc005d80 comp rx=0 tx=0).stop 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 shutdown_connections 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f8e57cc770 0x55f8e57ceb60 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f8e574ed30 0x55f8e5751d20 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8e5742da0 0x55f8e5757540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bc3fff640 0 client.17763 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 --2- 172.21.15.33:0/2402170978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f8e5742480 0x55f8e5756fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 >> 172.21.15.33:0/2402170978 conn(0x55f8e57402b0 msgr2=0x55f8e574d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 shutdown_connections 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 1 -- 172.21.15.33:0/2402170978 wait complete. 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.876+0000 7f1bece79740 20 client.17763 trim_cache size 0 max 16384 2024-09-10T21:16:26.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1bd00b52e0 msgr2=0x7f1bd00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1bd00b52e0 0x7f1bd00b77a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc04a000 tx=0x7f1bdc04b000 comp rx=0 tx=0).stop 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be4103ce0 msgr2=0x7f1be41992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be4103ce0 0x7f1be41992a0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400d770 tx=0x7f1bd400dc30 comp rx=0 tx=0).stop 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 shutdown_connections 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1bd00b52e0 0x7f1bd00b77a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1be410d140 0x7f1be419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be4104600 0x7f1be41997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 --2- 172.21.15.33:0/2483775402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1be4103ce0 0x7f1be41992a0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 >> 172.21.15.33:0/2483775402 conn(0x7f1be40fd720 msgr2=0x7f1be40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 shutdown_connections 2024-09-10T21:16:26.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:26.877+0000 7f1beaaa4640 1 -- 172.21.15.33:0/2483775402 wait complete. 2024-09-10T21:16:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:16:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:27.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:16:27.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:26 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:16:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:27.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:27.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:27.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:27.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:16:27.178 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:26 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:16:27.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:26 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:27.695 INFO:tasks.cephfs.filesystem:scrub status for tag:0d738eee-48a0-490a-83f5-59006db2e660 - {'path': '/', 'tag': '0d738eee-48a0-490a-83f5-59006db2e660', 'options': 'recursive,force'} 2024-09-10T21:16:27.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 949272 6.77 MB/sec execute 568 sec latency 1762.890 ms 2024-09-10T21:16:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:27 smithi033 ceph-mon[30860]: pgmap v720: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-09-10T21:16:28.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:27 smithi174 ceph-mon[35635]: pgmap v720: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-09-10T21:16:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:27 smithi104 ceph-mon[35325]: pgmap v720: 97 pgs: 97 active+clean; 348 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.9 MiB/s wr, 145 op/s 2024-09-10T21:16:28.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 951825 6.78 MB/sec execute 569 sec latency 15.601 ms 2024-09-10T21:16:29.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:28 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:29.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:28 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:29.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:28 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:16:29.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 953417 6.79 MB/sec execute 570 sec latency 10.562 ms 2024-09-10T21:16:30.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:29 smithi033 ceph-mon[30860]: pgmap v721: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.0 MiB/s wr, 160 op/s 2024-09-10T21:16:30.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:29 smithi104 ceph-mon[35325]: pgmap v721: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.0 MiB/s wr, 160 op/s 2024-09-10T21:16:30.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:29 smithi174 ceph-mon[35635]: pgmap v721: 97 pgs: 97 active+clean; 360 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 3.0 MiB/s wr, 160 op/s 2024-09-10T21:16:30.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 955570 6.80 MB/sec execute 571 sec latency 262.835 ms 2024-09-10T21:16:31.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 955570 6.79 MB/sec execute 572 sec latency 1262.952 ms 2024-09-10T21:16:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:31 smithi033 ceph-mon[30860]: pgmap v722: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-09-10T21:16:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:16:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:16:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:16:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:16:32.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:16:32.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:31 smithi104 ceph-mon[35325]: pgmap v722: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-09-10T21:16:32.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:16:32.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:16:32.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:16:32.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:16:32.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:16:32.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:31 smithi174 ceph-mon[35635]: pgmap v722: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 3.6 MiB/s wr, 164 op/s 2024-09-10T21:16:32.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:16:32.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:16:32.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:16:32.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:16:32.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:16:32.756 INFO:tasks.workunit.client.0.smithi033.stdout: 1 956345 6.78 MB/sec execute 573 sec latency 1610.942 ms 2024-09-10T21:16:33.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 958932 6.79 MB/sec execute 574 sec latency 11.362 ms 2024-09-10T21:16:34.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:33 smithi033 ceph-mon[30860]: pgmap v723: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.5 MiB/s wr, 130 op/s 2024-09-10T21:16:34.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:33 smithi104 ceph-mon[35325]: pgmap v723: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.5 MiB/s wr, 130 op/s 2024-09-10T21:16:34.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:33 smithi174 ceph-mon[35635]: pgmap v723: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 3.5 MiB/s wr, 130 op/s 2024-09-10T21:16:34.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 960536 6.79 MB/sec execute 575 sec latency 12.483 ms 2024-09-10T21:16:35.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 963119 6.81 MB/sec execute 576 sec latency 13.061 ms 2024-09-10T21:16:36.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:35 smithi033 ceph-mon[30860]: pgmap v724: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 3.8 MiB/s wr, 192 op/s 2024-09-10T21:16:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:35 smithi104 ceph-mon[35325]: pgmap v724: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 3.8 MiB/s wr, 192 op/s 2024-09-10T21:16:36.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:35 smithi174 ceph-mon[35635]: pgmap v724: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 3.8 MiB/s wr, 192 op/s 2024-09-10T21:16:36.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 965393 6.81 MB/sec execute 577 sec latency 12.605 ms 2024-09-10T21:16:37.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 967038 6.82 MB/sec execute 578 sec latency 30.057 ms 2024-09-10T21:16:38.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:37 smithi033 ceph-mon[30860]: pgmap v725: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-09-10T21:16:38.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:37 smithi104 ceph-mon[35325]: pgmap v725: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-09-10T21:16:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:37 smithi174 ceph-mon[35635]: pgmap v725: 97 pgs: 97 active+clean; 362 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 3.8 MiB/s wr, 188 op/s 2024-09-10T21:16:38.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 969727 6.84 MB/sec execute 579 sec latency 13.452 ms 2024-09-10T21:16:39.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 969753 6.82 MB/sec execute 580 sec latency 969.790 ms 2024-09-10T21:16:40.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:39 smithi033 ceph-mon[30860]: pgmap v726: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-09-10T21:16:40.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:39 smithi104 ceph-mon[35325]: pgmap v726: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-09-10T21:16:40.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:39 smithi174 ceph-mon[35635]: pgmap v726: 97 pgs: 97 active+clean; 364 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 4.0 MiB/s wr, 242 op/s 2024-09-10T21:16:40.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 969753 6.81 MB/sec execute 581 sec latency 1969.909 ms 2024-09-10T21:16:41.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 969753 6.80 MB/sec execute 582 sec latency 2970.033 ms 2024-09-10T21:16:42.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:41 smithi033 ceph-mon[30860]: pgmap v727: 97 pgs: 97 active+clean; 366 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-10T21:16:42.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:41 smithi104 ceph-mon[35325]: pgmap v727: 97 pgs: 97 active+clean; 366 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-10T21:16:42.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:41 smithi174 ceph-mon[35635]: pgmap v727: 97 pgs: 97 active+clean; 366 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 3.0 MiB/s wr, 269 op/s 2024-09-10T21:16:42.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 970530 6.80 MB/sec execute 583 sec latency 3316.808 ms 2024-09-10T21:16:43.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 973120 6.81 MB/sec execute 584 sec latency 22.064 ms 2024-09-10T21:16:44.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:43 smithi033 ceph-mon[30860]: pgmap v728: 97 pgs: 97 active+clean; 365 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-10T21:16:44.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:43 smithi104 ceph-mon[35325]: pgmap v728: 97 pgs: 97 active+clean; 365 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-10T21:16:44.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:43 smithi174 ceph-mon[35635]: pgmap v728: 97 pgs: 97 active+clean; 365 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-09-10T21:16:44.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 974719 6.81 MB/sec execute 585 sec latency 13.869 ms 2024-09-10T21:16:45.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 977319 6.83 MB/sec execute 586 sec latency 13.001 ms 2024-09-10T21:16:46.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:45 smithi033 ceph-mon[30860]: pgmap v729: 97 pgs: 97 active+clean; 367 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.5 MiB/s wr, 262 op/s 2024-09-10T21:16:46.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:45 smithi104 ceph-mon[35325]: pgmap v729: 97 pgs: 97 active+clean; 367 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.5 MiB/s wr, 262 op/s 2024-09-10T21:16:46.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:45 smithi174 ceph-mon[35635]: pgmap v729: 97 pgs: 97 active+clean; 367 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 2.5 MiB/s wr, 262 op/s 2024-09-10T21:16:46.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 979485 6.83 MB/sec execute 587 sec latency 11.432 ms 2024-09-10T21:16:47.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 981167 6.84 MB/sec execute 588 sec latency 37.010 ms 2024-09-10T21:16:48.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:47 smithi033 ceph-mon[30860]: pgmap v730: 97 pgs: 97 active+clean; 367 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-09-10T21:16:48.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:47 smithi104 ceph-mon[35325]: pgmap v730: 97 pgs: 97 active+clean; 367 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-09-10T21:16:48.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:47 smithi174 ceph-mon[35635]: pgmap v730: 97 pgs: 97 active+clean; 367 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 2.1 MiB/s wr, 192 op/s 2024-09-10T21:16:48.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 983732 6.85 MB/sec execute 589 sec latency 16.077 ms 2024-09-10T21:16:49.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 985288 6.85 MB/sec execute 590 sec latency 19.577 ms 2024-09-10T21:16:50.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:49 smithi033 ceph-mon[30860]: pgmap v731: 97 pgs: 97 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-10T21:16:50.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:49 smithi104 ceph-mon[35325]: pgmap v731: 97 pgs: 97 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-10T21:16:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:49 smithi174 ceph-mon[35635]: pgmap v731: 97 pgs: 97 active+clean; 368 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 2.4 MiB/s wr, 271 op/s 2024-09-10T21:16:50.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 987921 6.87 MB/sec execute 591 sec latency 14.098 ms 2024-09-10T21:16:51.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 990126 6.87 MB/sec execute 592 sec latency 12.238 ms 2024-09-10T21:16:52.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:51 smithi033 ceph-mon[30860]: pgmap v732: 97 pgs: 97 active+clean; 369 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-09-10T21:16:52.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:51 smithi104 ceph-mon[35325]: pgmap v732: 97 pgs: 97 active+clean; 369 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-09-10T21:16:52.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:51 smithi174 ceph-mon[35635]: pgmap v732: 97 pgs: 97 active+clean; 369 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 581 KiB/s rd, 2.4 MiB/s wr, 246 op/s 2024-09-10T21:16:52.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 991851 6.88 MB/sec execute 593 sec latency 12.548 ms 2024-09-10T21:16:53.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 994214 6.88 MB/sec execute 594 sec latency 21.941 ms 2024-09-10T21:16:54.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:53 smithi033 ceph-mon[30860]: pgmap v733: 97 pgs: 97 active+clean; 370 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-09-10T21:16:54.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:53 smithi104 ceph-mon[35325]: pgmap v733: 97 pgs: 97 active+clean; 370 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-09-10T21:16:54.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:53 smithi174 ceph-mon[35635]: pgmap v733: 97 pgs: 97 active+clean; 370 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-09-10T21:16:54.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 995526 6.88 MB/sec execute 595 sec latency 14.506 ms 2024-09-10T21:16:55.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 997786 6.89 MB/sec execute 596 sec latency 14.005 ms 2024-09-10T21:16:56.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:55 smithi033 ceph-mon[30860]: pgmap v734: 97 pgs: 97 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-09-10T21:16:56.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:55 smithi104 ceph-mon[35325]: pgmap v734: 97 pgs: 97 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-09-10T21:16:56.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:55 smithi174 ceph-mon[35635]: pgmap v734: 97 pgs: 97 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 713 KiB/s rd, 2.4 MiB/s wr, 300 op/s 2024-09-10T21:16:56.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 998149 6.89 MB/sec execute 597 sec latency 853.441 ms 2024-09-10T21:16:57.696 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:16:57.757 INFO:tasks.workunit.client.0.smithi033.stdout: 1 998828 6.88 MB/sec execute 598 sec latency 1206.531 ms 2024-09-10T21:16:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:58.096 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:57 smithi033 ceph-mon[30860]: pgmap v735: 97 pgs: 97 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-09-10T21:16:58.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:57 smithi104 ceph-mon[35325]: pgmap v735: 97 pgs: 97 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-09-10T21:16:58.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:57 smithi174 ceph-mon[35635]: pgmap v735: 97 pgs: 97 active+clean; 373 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 601 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 -- 172.21.15.33:0/1221327293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 msgr2=0x7f699c101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 --2- 172.21.15.33:0/1221327293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c101a80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f698c009ca0 tx=0x7f698c02a950 comp rx=0 tx=0).stop 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 -- 172.21.15.33:0/1221327293 shutdown_connections 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 --2- 172.21.15.33:0/1221327293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f699c10f250 0x7f699c111640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 --2- 172.21.15.33:0/1221327293 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c101fc0 0x7f699c10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 --2- 172.21.15.33:0/1221327293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c101a80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 -- 172.21.15.33:0/1221327293 >> 172.21.15.33:0/1221327293 conn(0x7f699c0fd5b0 msgr2=0x7f699c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 -- 172.21.15.33:0/1221327293 shutdown_connections 2024-09-10T21:16:58.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.554+0000 7f699b59e640 1 -- 172.21.15.33:0/1221327293 wait complete. 2024-09-10T21:16:58.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.555+0000 7f699b59e640 1 Processor -- start 2024-09-10T21:16:58.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.555+0000 7f699b59e640 1 -- start start 2024-09-10T21:16:58.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.555+0000 7f699b59e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c19b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.555+0000 7f699b59e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c101fc0 0x7f699c19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.555+0000 7f699b59e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f699c10f250 0x7f699c1a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.555+0000 7f699b59e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114dd0 con 0x7f699c1016a0 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.555+0000 7f699b59e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114c50 con 0x7f699c101fc0 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699b59e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f699c114f50 con 0x7f699c10f250 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c19b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c19b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41558/0 (socket says 172.21.15.33:41558) 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 -- 172.21.15.33:0/904507678 learned_addr learned my addr 172.21.15.33:0/904507678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f6999d9b640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c101fc0 0x7f699c19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699ad9d640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f699c10f250 0x7f699c1a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 -- 172.21.15.33:0/904507678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f699c10f250 msgr2=0x7f699c1a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f699c10f250 0x7f699c1a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 -- 172.21.15.33:0/904507678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c101fc0 msgr2=0x7f699c19b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c101fc0 0x7f699c19b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 -- 172.21.15.33:0/904507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699c1a1270 con 0x7f699c1016a0 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f6999d9b640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c101fc0 0x7f699c19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.556+0000 7f699a59c640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c19b1c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f698c005cc0 tx=0x7f698c02f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.557+0000 7f698b7fe640 1 -- 172.21.15.33:0/904507678 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c034040 con 0x7f699c1016a0 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.557+0000 7f698b7fe640 1 -- 172.21.15.33:0/904507678 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f698c0308f0 con 0x7f699c1016a0 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.557+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699c1a1500 con 0x7f699c1016a0 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.557+0000 7f698b7fe640 1 -- 172.21.15.33:0/904507678 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f698c004640 con 0x7f699c1016a0 2024-09-10T21:16:58.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.557+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699c1a7c20 con 0x7f699c1016a0 2024-09-10T21:16:58.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.560+0000 7f69a31d7740 1 Processor -- start 2024-09-10T21:16:58.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.560+0000 7f69a31d7740 1 -- start start 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.560+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff1198a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff11a4f00 0x55cff11a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f69a31d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cff110ee50 con 0x55cff1198650 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f69a31d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cff10fe280 con 0x55cff11a4f00 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f69a31d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cff1102090 con 0x55cff1198f70 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f698b7fe640 1 -- 172.21.15.33:0/904507678 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f698c04d030 con 0x7f699c1016a0 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff1198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff1198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41572/0 (socket says 172.21.15.33:41572) 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 -- 172.21.15.33:0/3283708675 learned_addr learned my addr 172.21.15.33:0/3283708675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:58.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f6999d9b640 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff11a4f00 0x55cff11a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699a59c640 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 -- 172.21.15.33:0/3283708675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 msgr2=0x55cff11a49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 -- 172.21.15.33:0/3283708675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff11a4f00 msgr2=0x55cff11a72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff11a4f00 0x55cff11a72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 -- 172.21.15.33:0/3283708675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff0e5c650 con 0x55cff1198650 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.561+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff1198a30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6994009ca0 tx=0x7f6994011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b05e9ac56877f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.562+0000 7f6988ff9640 1 -- 172.21.15.33:0/3283708675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699401e070 con 0x55cff1198650 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.562+0000 7f6988ff9640 1 -- 172.21.15.33:0/3283708675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6994009e50 con 0x55cff1198650 2024-09-10T21:16:58.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.562+0000 7f6988ff9640 1 -- 172.21.15.33:0/3283708675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400ea50 con 0x55cff1198650 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.562+0000 7f698b7fe640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69840b5410 0x7f69840b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.562+0000 7f699a59c640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69840b5410 0x7f69840b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.562+0000 7f698b7fe640 1 -- 172.21.15.33:0/904507678 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f698c1067b0 con 0x7f699c1016a0 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 -- 172.21.15.33:0/3283708675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 msgr2=0x55cff1198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff1198a30 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6994009ca0 tx=0x7f6994011810 comp rx=0 tx=0).stop 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 -- 172.21.15.33:0/3283708675 shutdown_connections 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff11a4f00 0x55cff11a72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3283708675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff1198a30 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 -- 172.21.15.33:0/3283708675 >> 172.21.15.33:0/3283708675 conn(0x55cff1196480 msgr2=0x55cff1197620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f699a59c640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69840b5410 0x7f69840b78d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f698c004af0 tx=0x7f698c004140 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 -- 172.21.15.33:0/3283708675 shutdown_connections 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.563+0000 7f69a31d7740 1 -- 172.21.15.33:0/3283708675 wait complete. 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 -- start start 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 Processor -- start 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff11a4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198f70 0x55cff11a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 0x55cff11a2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55cff0f5f700 con 0x55cff1198f70 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55cff1099110 con 0x55cff1198650 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cff11a7b80 con 0x55cff11a4f00 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f69a31d7740 10 client.0 _ll_register_callbacks cb 0x55cff10b2990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff11a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff11a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56804/0 (socket says 172.21.15.33:56804) 2024-09-10T21:16:58.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f699ad9d640 1 -- 172.21.15.33:0/3538672763 learned_addr learned my addr 172.21.15.33:0/3538672763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 0x55cff11a2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.564+0000 7f6999d9b640 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198f70 0x55cff11a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f699ad9d640 1 -- 172.21.15.33:0/3538672763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 msgr2=0x55cff11a2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 0x55cff11a2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f699ad9d640 1 -- 172.21.15.33:0/3538672763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198f70 msgr2=0x55cff11a1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198f70 0x55cff11a1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f699ad9d640 1 -- 172.21.15.33:0/3538672763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff10c6c40 con 0x55cff1198650 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff11a4780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f699400e4f0 tx=0x7f699400e5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c7029ac3df7a80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f6981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f6981ffb640 1 -- 172.21.15.33:0/3538672763 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699401e070 con 0x55cff1198650 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6994009e50 con 0x55cff1198650 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f6981ffb640 1 -- 172.21.15.33:0/3538672763 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6994004460 con 0x55cff1198650 2024-09-10T21:16:58.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f6981ffb640 1 -- 172.21.15.33:0/3538672763 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6994005070 con 0x55cff1198650 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f69a31d7740 10 client.27206 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cff11b3480 con 0x55cff1198650 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.565+0000 7f6981ffb640 1 -- 172.21.15.33:0/3538672763 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6994009e50 con 0x55cff1198650 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f6981ffb640 1 -- 172.21.15.33:0/3538672763 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f6994032050 con 0x55cff1198650 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f69a31d7740 10 client.27206 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f69a31d7740 10 client.27206 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f69a31d7740 20 client.27206 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff11b4830 0x55cff11b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f69a31d7740 4 client.27206 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cff1110da0 con 0x55cff11b4830 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f6999d9b640 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff11b4830 0x55cff11b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f6981ffb640 10 client.27206 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:58.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.566+0000 7f6999d9b640 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff11b4830 0x55cff11b6c20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ef11b5272aed764 server_cookie=c7db0a98dbc07712 in_seq=0 out_seq=0 2024-09-10T21:16:58.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f6981ffb640 1 -- 172.21.15.33:0/3538672763 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cff1110da0 con 0x55cff11b4830 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f6981ffb640 10 client.27206 handle_command_reply: tid=0 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 client.27206 shutdown 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 msgr2=0x55cff11a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff11a4780 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f699400e4f0 tx=0x7f699400e5f0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 shutdown_connections 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff11b4830 0x55cff11b6c20 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 0x55cff11a2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198f70 0x55cff11a1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3538672763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff11a4780 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f6981ffb640 0 client.27206 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 >> 172.21.15.33:0/3538672763 conn(0x55cff1196480 msgr2=0x55cff119bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.567+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 shutdown_connections 2024-09-10T21:16:58.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.568+0000 7f69a31d7740 1 -- 172.21.15.33:0/3538672763 wait complete. 2024-09-10T21:16:58.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.568+0000 7f69a31d7740 20 client.27206 trim_cache size 0 max 16384 2024-09-10T21:16:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.587+0000 7f69a31d7740 1 Processor -- start 2024-09-10T21:16:58.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f69a31d7740 1 -- start start 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff1195510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff11a4f00 0x55cff11a6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f69a31d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b02d0 con 0x55cff11a4f00 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f69a31d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b1e70 con 0x55cff1198650 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f69a31d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b09a0 con 0x55cff1198f70 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff11a4f00 0x55cff11a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49316/0 (socket says 172.21.15.33:49316) 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff11a4f00 0x55cff11a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41590/0 (socket says 172.21.15.33:41590) 2024-09-10T21:16:58.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 -- 172.21.15.33:0/1922692839 learned_addr learned my addr 172.21.15.33:0/1922692839 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f699ad9d640 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff1195510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 -- 172.21.15.33:0/1922692839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 msgr2=0x55cff1195510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff1195510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 -- 172.21.15.33:0/1922692839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff11a4f00 msgr2=0x55cff11a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff11a4f00 0x55cff11a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f6999d9b640 1 -- 172.21.15.33:0/1922692839 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff11a2a40 con 0x55cff1198f70 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.588+0000 7f699ad9d640 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff11a4f00 0x55cff11a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:16:58.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.589+0000 7f6999d9b640 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a62f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55cff11b7350 tx=0x7f6990005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99889db2ca3297c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.589+0000 7f6983fff640 1 -- 172.21.15.33:0/1922692839 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990005220 con 0x55cff1198f70 2024-09-10T21:16:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.589+0000 7f6983fff640 1 -- 172.21.15.33:0/1922692839 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69900047a0 con 0x55cff1198f70 2024-09-10T21:16:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.589+0000 7f6983fff640 1 -- 172.21.15.33:0/1922692839 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69900118b0 con 0x55cff1198f70 2024-09-10T21:16:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 -- 172.21.15.33:0/1922692839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 msgr2=0x55cff11a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a62f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55cff11b7350 tx=0x7f6990005f90 comp rx=0 tx=0).stop 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 -- 172.21.15.33:0/1922692839 shutdown_connections 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff11a4f00 0x55cff11a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff1198f70 0x55cff11a62f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 --2- 172.21.15.33:0/1922692839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198650 0x55cff1195510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 -- 172.21.15.33:0/1922692839 >> 172.21.15.33:0/1922692839 conn(0x55cff1196480 msgr2=0x55cff1194e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 -- 172.21.15.33:0/1922692839 shutdown_connections 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 -- 172.21.15.33:0/1922692839 wait complete. 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 -- start start 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.590+0000 7f69a31d7740 1 Processor -- start 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff11ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198f70 0x55cff11ad380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f69a31d7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 0x55cff119e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f69a31d7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b1740 con 0x55cff1198650 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f69a31d7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1b1070 con 0x55cff1198f70 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f69a31d7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f699c1ae290 con 0x55cff11a4f00 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f69a31d7740 10 client.0 _ll_register_callbacks cb 0x7f699401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff11ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff11ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41596/0 (socket says 172.21.15.33:41596) 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 -- 172.21.15.33:0/3468200123 learned_addr learned my addr 172.21.15.33:0/3468200123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f6999d9b640 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198f70 0x55cff11ad380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 -- 172.21.15.33:0/3468200123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 msgr2=0x55cff119e220 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 0x55cff119e220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 -- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198f70 msgr2=0x55cff11ad380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:16:58.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198f70 0x55cff11ad380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.591+0000 7f699ad9d640 1 -- 172.21.15.33:0/3468200123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff119e760 con 0x55cff1198650 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f699ad9d640 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff11ace40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f699400e5c0 tx=0x7f6994003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b8ab4e870d77c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f6981ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f6981ffb640 1 -- 172.21.15.33:0/3468200123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699401e070 con 0x55cff1198650 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f6981ffb640 1 -- 172.21.15.33:0/3468200123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6994003c80 con 0x55cff1198650 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69900047a0 con 0x55cff1198650 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f6981ffb640 1 -- 172.21.15.33:0/3468200123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6994048050 con 0x55cff1198650 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f69a31d7740 10 client.17781 fetch_fsmap learned FSMap version 61 2024-09-10T21:16:58.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.592+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55cff1222c80 con 0x55cff1198650 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f6981ffb640 1 -- 172.21.15.33:0/3468200123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6994011dc0 con 0x55cff1198650 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f6981ffb640 1 -- 172.21.15.33:0/3468200123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f6994032040 con 0x55cff1198650 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f69a31d7740 10 client.17781 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f69a31d7740 10 client.17781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f69a31d7740 20 client.17781 populate_metadata read hostname 'smithi033' 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff1223320 0x55cff1225740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f69a31d7740 4 client.17781 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f69900047a0 con 0x55cff1223320 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.593+0000 7f6999d9b640 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff1223320 0x55cff1225740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f6999d9b640 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff1223320 0x55cff1225740 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=503c949f3191abdc server_cookie=516762806f4b94be in_seq=0 out_seq=0 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f6981ffb640 10 client.17781 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f6981ffb640 1 -- 172.21.15.33:0/3468200123 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x7f69900047a0 con 0x55cff1223320 2024-09-10T21:16:58.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f6981ffb640 10 client.17781 handle_command_reply: tid=0 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 client.17781 shutdown 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 msgr2=0x55cff11ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff11ace40 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f699400e5c0 tx=0x7f6994003c10 comp rx=0 tx=0).stop 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 shutdown_connections 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55cff1223320 0x55cff1225740 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cff11a4f00 0x55cff119e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55cff1198f70 0x55cff11ad380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f6981ffb640 0 client.17781 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 --2- 172.21.15.33:0/3468200123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55cff1198650 0x55cff11ace40 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 >> 172.21.15.33:0/3468200123 conn(0x55cff1196480 msgr2=0x55cff1195620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 shutdown_connections 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 1 -- 172.21.15.33:0/3468200123 wait complete. 2024-09-10T21:16:58.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.594+0000 7f69a31d7740 20 client.17781 trim_cache size 0 max 16384 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.605 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:16:58.606 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:16:58.607 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.609 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:16:58.610 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:16:58.610 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.610 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.610 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.610 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.610 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.613 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:16:58.613 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:16:58.615 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:16:58.617 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:16:58.619 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:16:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69840b5410 msgr2=0x7f69840b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69840b5410 0x7f69840b78d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f698c004af0 tx=0x7f698c004140 comp rx=0 tx=0).stop 2024-09-10T21:16:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 msgr2=0x7f699c19b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:16:58.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c19b1c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f698c005cc0 tx=0x7f698c02f3a0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 shutdown_connections 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69840b5410 0x7f69840b78d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f699c10f250 0x7f699c1a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f699c101fc0 0x7f699c19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 --2- 172.21.15.33:0/904507678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f699c1016a0 0x7f699c19b1c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 >> 172.21.15.33:0/904507678 conn(0x7f699c0fd5b0 msgr2=0x7f699c111400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 shutdown_connections 2024-09-10T21:16:58.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:16:58.595+0000 7f699b59e640 1 -- 172.21.15.33:0/904507678 wait complete. 2024-09-10T21:16:58.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 1000882 6.88 MB/sec execute 599 sec latency 19.530 ms 2024-09-10T21:16:59.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:16:59.547 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:16:59.758 INFO:tasks.workunit.client.0.smithi033.stdout: 1 cleanup 600 sec 2024-09-10T21:16:59.825 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:16:59.947 INFO:tasks.workunit.client.0.smithi033.stdout: 0 cleanup 600 sec 2024-09-10T21:16:59.947 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-09-10T21:16:59.947 INFO:tasks.workunit.client.0.smithi033.stdout: Operation Count AvgLat MaxLat 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: ---------------------------------------- 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: NTCreateX 131511 1.934 4990.152 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: Close 96594 0.002 0.076 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: Rename 5566 5.616 93.231 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: Unlink 26557 1.841 5007.462 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: Deltree 4 88.408 141.421 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: Mkdir 2 0.007 0.008 2024-09-10T21:16:59.948 INFO:tasks.workunit.client.0.smithi033.stdout: Qpathinfo 119170 1.890 4987.628 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: Qfileinfo 20902 0.001 0.054 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: Qfsinfo 21848 0.175 38.828 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: Sfileinfo 10708 0.014 5.926 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: Find 46070 0.150 56.888 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: WriteX 65667 0.014 4.663 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: ReadX 206050 0.010 28.639 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: LockX 428 0.456 1.178 2024-09-10T21:16:59.949 INFO:tasks.workunit.client.0.smithi033.stdout: UnlockX 428 0.435 1.151 2024-09-10T21:16:59.950 INFO:tasks.workunit.client.0.smithi033.stdout: Flush 9214 2.595 97.929 2024-09-10T21:16:59.950 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-09-10T21:16:59.950 INFO:tasks.workunit.client.0.smithi033.stdout:Throughput 6.88344 MB/sec 1 clients 1 procs max_latency=5007.478 ms 2024-09-10T21:16:59.950 INFO:tasks.workunit.client.0.smithi033.stderr:+ dbench 10 2024-09-10T21:16:59.951 INFO:tasks.workunit.client.0.smithi033.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-10T21:16:59.951 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-09-10T21:16:59.951 INFO:tasks.workunit.client.0.smithi033.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-10T21:16:59.951 INFO:tasks.workunit.client.0.smithi033.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-10T21:17:00.049 INFO:tasks.workunit.client.0.smithi033.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-10T21:17:00.049 INFO:tasks.workunit.client.0.smithi033.stdout:releasing clients 2024-09-10T21:17:00.195 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:16:59 smithi174 ceph-mon[35635]: pgmap v736: 97 pgs: 97 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-09-10T21:17:00.263 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:16:59 smithi033 ceph-mon[30860]: pgmap v736: 97 pgs: 97 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-09-10T21:17:00.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:16:59 smithi104 ceph-mon[35325]: pgmap v736: 97 pgs: 97 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 750 KiB/s rd, 1.9 MiB/s wr, 306 op/s 2024-09-10T21:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 -- 172.21.15.33:0/3229151188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541045c0 msgr2=0x7f6d5410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 --2- 172.21.15.33:0/3229151188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541045c0 0x7f6d5410cab0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c009900 tx=0x7f6d3c02a410 comp rx=0 tx=0).stop 2024-09-10T21:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 -- 172.21.15.33:0/3229151188 shutdown_connections 2024-09-10T21:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 --2- 172.21.15.33:0/3229151188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5410d100 0x7f6d5410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 --2- 172.21.15.33:0/3229151188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541045c0 0x7f6d5410cab0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 --2- 172.21.15.33:0/3229151188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d54103ca0 0x7f6d54104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 -- 172.21.15.33:0/3229151188 >> 172.21.15.33:0/3229151188 conn(0x7f6d540fd700 msgr2=0x7f6d540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:00.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 -- 172.21.15.33:0/3229151188 shutdown_connections 2024-09-10T21:17:00.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 -- 172.21.15.33:0/3229151188 wait complete. 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.354+0000 7f6d593a1640 1 Processor -- start 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d593a1640 1 -- start start 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d593a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54103ca0 0x7f6d54199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d593a1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d541045c0 0x7f6d54199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d593a1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5410d100 0x7f6d5419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d593a1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54112d70 con 0x7f6d54103ca0 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d593a1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54112bf0 con 0x7f6d5410d100 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d593a1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d54112ef0 con 0x7f6d541045c0 2024-09-10T21:17:00.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d52ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54103ca0 0x7f6d54199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d527fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d541045c0 0x7f6d54199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5410d100 0x7f6d5419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d527fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d541045c0 0x7f6d54199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54616/0 (socket says 172.21.15.33:54616) 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d527fc640 1 -- 172.21.15.33:0/1199151328 learned_addr learned my addr 172.21.15.33:0/1199151328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d527fc640 1 -- 172.21.15.33:0/1199151328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5410d100 msgr2=0x7f6d5419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d527fc640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5410d100 0x7f6d5419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.355+0000 7f6d527fc640 1 -- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54103ca0 msgr2=0x7f6d54199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d527fc640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54103ca0 0x7f6d54199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d527fc640 1 -- 172.21.15.33:0/1199151328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d5419f3d0 con 0x7f6d541045c0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d527fc640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d541045c0 0x7f6d54199860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c032fd0 tx=0x7f6d3c031970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d4bfff640 1 -- 172.21.15.33:0/1199151328 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c03f030 con 0x7f6d541045c0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d5419f660 con 0x7f6d541045c0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d4bfff640 1 -- 172.21.15.33:0/1199151328 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3c004350 con 0x7f6d541045c0 2024-09-10T21:17:00.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d4bfff640 1 -- 172.21.15.33:0/1199151328 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c035070 con 0x7f6d541045c0 2024-09-10T21:17:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.356+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d54101830 con 0x7f6d541045c0 2024-09-10T21:17:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.357+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d54104080 con 0x7f6d541045c0 2024-09-10T21:17:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.359+0000 7f6d4bfff640 1 -- 172.21.15.33:0/1199151328 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6d3c003bc0 con 0x7f6d541045c0 2024-09-10T21:17:00.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.360+0000 7f6d4bfff640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d400b52e0 0x7f6d400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.360+0000 7f6d52ffd640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d400b52e0 0x7f6d400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.360+0000 7f6d4bfff640 1 -- 172.21.15.33:0/1199151328 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6d3c106530 con 0x7f6d541045c0 2024-09-10T21:17:00.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.360+0000 7f6d52ffd640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d400b52e0 0x7f6d400b77a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6d44005fd0 tx=0x7f6d44012040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:00.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.363+0000 7f6d4bfff640 1 -- 172.21.15.33:0/1199151328 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6d3c0cd3b0 con 0x7f6d541045c0 2024-09-10T21:17:00.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.610+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6d5419a450 con 0x7f6d541045c0 2024-09-10T21:17:00.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.611+0000 7f6d4bfff640 1 -- 172.21.15.33:0/1199151328 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+11977 (secure 0 0 0) 0x7f6d3c0cccb0 con 0x7f6d541045c0 2024-09-10T21:17:00.615 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:17:00.615 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":61,"btime":"2024-09-10T21:16:21:863632+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:16:20.996975+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:17:00.616 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 61 2024-09-10T21:17:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d400b52e0 msgr2=0x7f6d400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d400b52e0 0x7f6d400b77a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6d44005fd0 tx=0x7f6d44012040 comp rx=0 tx=0).stop 2024-09-10T21:17:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d541045c0 msgr2=0x7f6d54199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d541045c0 0x7f6d54199860 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c032fd0 tx=0x7f6d3c031970 comp rx=0 tx=0).stop 2024-09-10T21:17:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 shutdown_connections 2024-09-10T21:17:00.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6d400b52e0 0x7f6d400b77a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d5410d100 0x7f6d5419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d541045c0 0x7f6d54199860 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 --2- 172.21.15.33:0/1199151328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d54103ca0 0x7f6d54199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 >> 172.21.15.33:0/1199151328 conn(0x7f6d540fd700 msgr2=0x7f6d540fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 shutdown_connections 2024-09-10T21:17:00.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:00.613+0000 7f6d593a1640 1 -- 172.21.15.33:0/1199151328 wait complete. 2024-09-10T21:17:01.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 425 150.22 MB/sec warmup 1 sec latency 54.624 ms 2024-09-10T21:17:01.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:00 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1199151328' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:17:01.313 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:00 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1199151328' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:17:01.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:17:01.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:00 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1199151328' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:17:01.737 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:02.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 1442 121.39 MB/sec warmup 2 sec latency 78.407 ms 2024-09-10T21:17:02.158 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:01 smithi033 ceph-mon[30860]: pgmap v737: 97 pgs: 97 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.5 MiB/s wr, 227 op/s 2024-09-10T21:17:02.213 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:01 smithi104 ceph-mon[35325]: pgmap v737: 97 pgs: 97 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.5 MiB/s wr, 227 op/s 2024-09-10T21:17:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 -- 172.21.15.33:0/4172199397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 msgr2=0x7f7c0010cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 --2- 172.21.15.33:0/4172199397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 0x7f7c0010cac0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009860 tx=0x7f7be802a340 comp rx=0 tx=0).stop 2024-09-10T21:17:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 -- 172.21.15.33:0/4172199397 shutdown_connections 2024-09-10T21:17:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 --2- 172.21.15.33:0/4172199397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c0010d110 0x7f7c0010f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 --2- 172.21.15.33:0/4172199397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 0x7f7c0010cac0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 --2- 172.21.15.33:0/4172199397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c00103cb0 0x7f7c00104090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 -- 172.21.15.33:0/4172199397 >> 172.21.15.33:0/4172199397 conn(0x7f7c000fd6f0 msgr2=0x7f7c000ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 -- 172.21.15.33:0/4172199397 shutdown_connections 2024-09-10T21:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 -- 172.21.15.33:0/4172199397 wait complete. 2024-09-10T21:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.262+0000 7f7c06710640 1 Processor -- start 2024-09-10T21:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c06710640 1 -- start start 2024-09-10T21:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c06710640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c00103cb0 0x7f7c0019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c06710640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 0x7f7c0019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c06710640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c0010d110 0x7f7c001a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c06710640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00112d80 con 0x7f7c001045d0 2024-09-10T21:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c06710640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00112c00 con 0x7f7c0010d110 2024-09-10T21:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c06710640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00112f00 con 0x7f7c00103cb0 2024-09-10T21:17:02.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c00103cb0 0x7f7c0019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c00103cb0 0x7f7c0019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54644/0 (socket says 172.21.15.33:54644) 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7bfffff640 1 -- 172.21.15.33:0/1212994295 learned_addr learned my addr 172.21.15.33:0/1212994295 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 0x7f7c0019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.263+0000 7f7c04cad640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c0010d110 0x7f7c001a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfffff640 1 -- 172.21.15.33:0/1212994295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c0010d110 msgr2=0x7f7c001a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfffff640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c0010d110 0x7f7c001a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfffff640 1 -- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 msgr2=0x7f7c0019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfffff640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 0x7f7c0019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfffff640 1 -- 172.21.15.33:0/1212994295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c001a1580 con 0x7f7c00103cb0 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfffff640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c00103cb0 0x7f7c0019b4d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e9c0 tx=0x7f7bf000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfdffb640 1 -- 172.21.15.33:0/1212994295 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf000ccc0 con 0x7f7c00103cb0 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfdffb640 1 -- 172.21.15.33:0/1212994295 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bf00054d0 con 0x7f7c00103cb0 2024-09-10T21:17:02.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7bfdffb640 1 -- 172.21.15.33:0/1212994295 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0010430 con 0x7f7c00103cb0 2024-09-10T21:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c001a1870 con 0x7f7c00103cb0 2024-09-10T21:17:02.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.264+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c001a7f30 con 0x7f7c00103cb0 2024-09-10T21:17:02.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.267+0000 7f7bfdffb640 1 -- 172.21.15.33:0/1212994295 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7bf0004f80 con 0x7f7c00103cb0 2024-09-10T21:17:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.269+0000 7f7bfdffb640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bec0b5360 0x7f7bec0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 Processor -- start 2024-09-10T21:17:02.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bec0b5360 0x7f7bec0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7bfdffb640 1 -- 172.21.15.33:0/1212994295 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7bf00e5f10 con 0x7f7c00103cb0 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 -- start start 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a36ce0 0x5642a5a370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a43050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a45980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5642a59a4ea0 con 0x5642a5a37600 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642a59942d0 con 0x5642a5a36ce0 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5642a59980e0 con 0x5642a5a43590 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a36ce0 0x5642a5a370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a43050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a36ce0 0x5642a5a370c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36112/0 (socket says 172.21.15.33:36112) 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7bfffff640 1 -- 172.21.15.33:0/3882171290 learned_addr learned my addr 172.21.15.33:0/3882171290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bec0b5360 0x7f7bec0b7820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7be80303f0 tx=0x7f7be8030340 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.270+0000 7f7c04cad640 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a45980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bfffff640 1 -- 172.21.15.33:0/3882171290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 msgr2=0x5642a5a45980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bfffff640 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a45980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bfffff640 1 -- 172.21.15.33:0/3882171290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 msgr2=0x5642a5a43050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bfffff640 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a43050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bfffff640 1 -- 172.21.15.33:0/3882171290 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a56f2650 con 0x5642a5a36ce0 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bfffff640 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a36ce0 0x5642a5a370c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00e3950 tx=0x7f7bf00abe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb9559dbc62aad43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bf67fc640 1 -- 172.21.15.33:0/3882171290 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00acdd0 con 0x5642a5a36ce0 2024-09-10T21:17:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bf67fc640 1 -- 172.21.15.33:0/3882171290 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bf00ac6d0 con 0x5642a5a36ce0 2024-09-10T21:17:02.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.271+0000 7f7bf67fc640 1 -- 172.21.15.33:0/3882171290 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00a8280 con 0x5642a5a36ce0 2024-09-10T21:17:02.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.272+0000 7f7c08ae5740 1 -- 172.21.15.33:0/3882171290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a36ce0 msgr2=0x5642a5a370c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.272+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a36ce0 0x5642a5a370c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00e3950 tx=0x7f7bf00abe40 comp rx=0 tx=0).stop 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.272+0000 7f7c08ae5740 1 -- 172.21.15.33:0/3882171290 shutdown_connections 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.272+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a45980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.272+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a43050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.272+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/3882171290 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a36ce0 0x5642a5a370c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.273+0000 7f7c08ae5740 1 -- 172.21.15.33:0/3882171290 >> 172.21.15.33:0/3882171290 conn(0x5642a5a34b10 msgr2=0x5642a5a35cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.273+0000 7f7c08ae5740 1 -- 172.21.15.33:0/3882171290 shutdown_connections 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.273+0000 7f7c08ae5740 1 -- 172.21.15.33:0/3882171290 wait complete. 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.274+0000 7f7c08ae5740 1 -- start start 2024-09-10T21:17:02.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.274+0000 7f7c08ae5740 1 Processor -- start 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.274+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a36ce0 0x5642a5a42ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.274+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a43590 0x5642a5a40920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5642a57f5700 con 0x5642a5a37600 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5642a592f160 con 0x5642a5a43590 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5642a5a46180 con 0x5642a5a36ce0 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c08ae5740 10 client.0 _ll_register_callbacks cb 0x5642a59489e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a402f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51760/0 (socket says 172.21.15.33:51760) 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7bf7fff640 1 -- 172.21.15.33:0/803881492 learned_addr learned my addr 172.21.15.33:0/803881492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c04cad640 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a43590 0x5642a5a40920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7bfffff640 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a36ce0 0x5642a5a42ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c04cad640 1 -- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a36ce0 msgr2=0x5642a5a42ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c04cad640 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a36ce0 0x5642a5a42ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c04cad640 1 -- 172.21.15.33:0/803881492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 msgr2=0x5642a5a402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c04cad640 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.275+0000 7f7c04cad640 1 -- 172.21.15.33:0/803881492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a595cc90 con 0x5642a5a43590 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7c04cad640 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a43590 0x5642a5a40920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005fd0 tx=0x7f7bf800f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f13dd4c85fc6ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7bdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/803881492 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf800fe60 con 0x5642a5a43590 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7bf00ac6d0 con 0x5642a5a43590 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/803881492 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7bf8004b20 con 0x5642a5a43590 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/803881492 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7bf8004d20 con 0x5642a5a43590 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7c08ae5740 10 client.27230 fetch_fsmap learned FSMap version 61 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5642a5a51b90 con 0x5642a5a43590 2024-09-10T21:17:02.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.276+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/803881492 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bf00ac6d0 con 0x5642a5a43590 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/803881492 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f7bf8019070 con 0x5642a5a43590 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7c08ae5740 10 client.27230 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7c08ae5740 10 client.27230 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7c08ae5740 20 client.27230 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5a55430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7c08ae5740 4 client.27230 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642a59a6df0 con 0x5642a5a53040 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7bfffff640 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5a55430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7bfffff640 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5a55430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d10df6ff93718b48 server_cookie=6b721eea74d0a5be in_seq=0 out_seq=0 2024-09-10T21:17:02.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.277+0000 7f7bdf7fe640 10 client.27230 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/803881492 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642a59a6df0 con 0x5642a5a53040 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7bdf7fe640 10 client.27230 handle_command_reply: tid=0 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7c08ae5740 1 client.27230 shutdown 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a43590 msgr2=0x5642a5a40920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a43590 0x5642a5a40920 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8005fd0 tx=0x7f7bf800f790 comp rx=0 tx=0).stop 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 shutdown_connections 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5a55430 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7bdf7fe640 0 client.27230 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.278+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a43590 0x5642a5a40920 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.279+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a37600 0x5642a5a402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.279+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/803881492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a36ce0 0x5642a5a42ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.279+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 >> 172.21.15.33:0/803881492 conn(0x5642a5a34b10 msgr2=0x5642a5a3a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.279+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 shutdown_connections 2024-09-10T21:17:02.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.279+0000 7f7c08ae5740 1 -- 172.21.15.33:0/803881492 wait complete. 2024-09-10T21:17:02.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.279+0000 7f7c08ae5740 20 client.27230 trim_cache size 0 max 16384 2024-09-10T21:17:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.295+0000 7f7c08ae5740 1 Processor -- start 2024-09-10T21:17:02.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.295+0000 7f7c08ae5740 1 -- start start 2024-09-10T21:17:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a408d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a2b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001ac270 con 0x5642a5a36ce0 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001ade10 con 0x5642a5a37600 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001ac940 con 0x5642a5a43590 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a408d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36124/0 (socket says 172.21.15.33:36124) 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2080559744 learned_addr learned my addr 172.21.15.33:0/2080559744 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c04cad640 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a2b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2080559744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 msgr2=0x5642a5a2b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a2b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2080559744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 msgr2=0x5642a5a408d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a408d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7bf7fff640 1 -- 172.21.15.33:0/2080559744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642a5a557a0 con 0x5642a5a37600 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.296+0000 7f7c04cad640 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a2b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.297+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4f3b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7be8035040 tx=0x7f7be803c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82223a8021d6ba07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.297+0000 7f7bf5ffb640 1 -- 172.21.15.33:0/2080559744 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be804f070 con 0x5642a5a37600 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.297+0000 7f7bf5ffb640 1 -- 172.21.15.33:0/2080559744 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8042040 con 0x5642a5a37600 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.297+0000 7f7bf5ffb640 1 -- 172.21.15.33:0/2080559744 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8037040 con 0x5642a5a37600 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 -- 172.21.15.33:0/2080559744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 msgr2=0x5642a5a4f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4f3b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7be8035040 tx=0x7f7be803c000 comp rx=0 tx=0).stop 2024-09-10T21:17:02.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 -- 172.21.15.33:0/2080559744 shutdown_connections 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a2b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4f3b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/2080559744 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a408d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 -- 172.21.15.33:0/2080559744 >> 172.21.15.33:0/2080559744 conn(0x5642a5a34b10 msgr2=0x5642a5a2ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 -- 172.21.15.33:0/2080559744 shutdown_connections 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.298+0000 7f7c08ae5740 1 -- 172.21.15.33:0/2080559744 wait complete. 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 -- start start 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 Processor -- start 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a4b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a3c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001ad6e0 con 0x5642a5a36ce0 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001ad010 con 0x5642a5a37600 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001aa2d0 con 0x5642a5a43590 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7c08ae5740 10 client.0 _ll_register_callbacks cb 0x7f7bf8009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.299+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7c04cad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a3c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bfffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a4b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36140/0 (socket says 172.21.15.33:36140) 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 -- 172.21.15.33:0/906020342 learned_addr learned my addr 172.21.15.33:0/906020342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 -- 172.21.15.33:0/906020342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 msgr2=0x5642a5a3c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a3c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 -- 172.21.15.33:0/906020342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 msgr2=0x5642a5a4b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:02.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a4b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 -- 172.21.15.33:0/906020342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642a5a4c1a0 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bfffff640 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a4b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bf7fff640 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4b950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7be8030bc0 tx=0x7f7be8030a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2efe08ae13e03fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/906020342 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8057020 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/906020342 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7be8032440 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8042040 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.300+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/906020342 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f7be804f070 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 10 client.27236 fetch_fsmap learned FSMap version 61 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x5642a5a3dba0 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/906020342 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8042040 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/906020342 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 61) ==== 8374+0+0 (secure 0 0 0) 0x7f7be803d710 con 0x5642a5a37600 2024-09-10T21:17:02.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 10 client.27236 fetch_fsmap finished waiting for FSMap version 61 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 10 client.27236 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 20 client.27236 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5ac2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 4 client.27236 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5642a59a28a0 con 0x5642a5a53040 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.301+0000 7f7bfffff640 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5ac2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7bfffff640 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5ac2f60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d1498952a08a5083 server_cookie=a32091c352039fe2 in_seq=0 out_seq=0 2024-09-10T21:17:02.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7bdf7fe640 10 client.27236 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:02.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7bdf7fe640 1 -- 172.21.15.33:0/906020342 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5642a59a28a0 con 0x5642a5a53040 2024-09-10T21:17:02.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7bdf7fe640 10 client.27236 handle_command_reply: tid=0 2024-09-10T21:17:02.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 client.27236 shutdown 2024-09-10T21:17:02.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 msgr2=0x5642a5a4b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4b950 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7be8030bc0 tx=0x7f7be8030a40 comp rx=0 tx=0).stop 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 shutdown_connections 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5642a5a53040 0x5642a5ac2f60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5642a5a43590 0x5642a5a3c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5642a5a37600 0x5642a5a4b950 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7bdf7fe640 0 client.27236 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 --2- 172.21.15.33:0/906020342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5642a5a36ce0 0x5642a5a4b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 >> 172.21.15.33:0/906020342 conn(0x5642a5a34b10 msgr2=0x5642a5a42140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 shutdown_connections 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 1 -- 172.21.15.33:0/906020342 wait complete. 2024-09-10T21:17:02.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.302+0000 7f7c08ae5740 20 client.27236 trim_cache size 0 max 16384 2024-09-10T21:17:02.314 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bec0b5360 msgr2=0x7f7bec0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bec0b5360 0x7f7bec0b7820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7be80303f0 tx=0x7f7be8030340 comp rx=0 tx=0).stop 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c00103cb0 msgr2=0x7f7c0019b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c00103cb0 0x7f7c0019b4d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e9c0 tx=0x7f7bf000ee80 comp rx=0 tx=0).stop 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 shutdown_connections 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7bec0b5360 0x7f7bec0b7820 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c0010d110 0x7f7c001a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c001045d0 0x7f7c0019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 --2- 172.21.15.33:0/1212994295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c00103cb0 0x7f7c0019b4d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 >> 172.21.15.33:0/1212994295 conn(0x7f7c000fd6f0 msgr2=0x7f7c000ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:02.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.303+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 shutdown_connections 2024-09-10T21:17:02.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:02.304+0000 7f7c06710640 1 -- 172.21.15.33:0/1212994295 wait complete. 2024-09-10T21:17:02.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:01 smithi174 ceph-mon[35635]: pgmap v737: 97 pgs: 97 active+clean; 376 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 576 KiB/s rd, 1.5 MiB/s wr, 227 op/s 2024-09-10T21:17:03.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 2666 102.65 MB/sec warmup 3 sec latency 52.955 ms 2024-09-10T21:17:03.135 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:17:03.422 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 -- 172.21.15.33:0/4091310249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb81010f3e0 msgr2=0x7fb8101117d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 --2- 172.21.15.33:0/4091310249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb81010f3e0 0x7fb8101117d0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb808009ca0 tx=0x7fb80802a950 comp rx=0 tx=0).stop 2024-09-10T21:17:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 -- 172.21.15.33:0/4091310249 shutdown_connections 2024-09-10T21:17:04.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 --2- 172.21.15.33:0/4091310249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb81010f3e0 0x7fb8101117d0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 --2- 172.21.15.33:0/4091310249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb810102150 0x7fb81010eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 --2- 172.21.15.33:0/4091310249 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb810101830 0x7fb810101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 -- 172.21.15.33:0/4091310249 >> 172.21.15.33:0/4091310249 conn(0x7fb8100fd700 msgr2=0x7fb8100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 -- 172.21.15.33:0/4091310249 shutdown_connections 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.012+0000 7fb816e37640 1 -- 172.21.15.33:0/4091310249 wait complete. 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 Processor -- start 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 -- start start 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb810101830 0x7fb81019b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb810102150 0x7fb81019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb81010f3e0 0x7fb8101a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb810114f40 con 0x7fb810101830 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb810114dc0 con 0x7fb81010f3e0 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb816e37640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8101150c0 con 0x7fb810102150 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb814bd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb810101830 0x7fb81019b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb814bd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb810101830 0x7fb81019b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51786/0 (socket says 172.21.15.33:51786) 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb814bd3640 1 -- 172.21.15.33:0/4070307354 learned_addr learned my addr 172.21.15.33:0/4070307354 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb80ffff640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb810102150 0x7fb81019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb8153d4640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb81010f3e0 0x7fb8101a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb814bd3640 1 -- 172.21.15.33:0/4070307354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb810102150 msgr2=0x7fb81019b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb814bd3640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb810102150 0x7fb81019b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb814bd3640 1 -- 172.21.15.33:0/4070307354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb81010f3e0 msgr2=0x7fb8101a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.013+0000 7fb814bd3640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb81010f3e0 0x7fb8101a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.014+0000 7fb814bd3640 1 -- 172.21.15.33:0/4070307354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8101a14e0 con 0x7fb810101830 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.014+0000 7fb814bd3640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb810101830 0x7fb81019b430 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb80400e9f0 tx=0x7fb80400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.014+0000 7fb80dffb640 1 -- 172.21.15.33:0/4070307354 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80400cd80 con 0x7fb810101830 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.014+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8101a17d0 con 0x7fb810101830 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.014+0000 7fb80dffb640 1 -- 172.21.15.33:0/4070307354 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8040108e0 con 0x7fb810101830 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.014+0000 7fb80dffb640 1 -- 172.21.15.33:0/4070307354 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804018980 con 0x7fb810101830 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.014+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8101a8060 con 0x7fb810101830 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 Processor -- start 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 -- start start 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb80dffb640 1 -- 172.21.15.33:0/4070307354 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb804010430 con 0x7fb810101830 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49605f4d0 0x55c49605f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c49606b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49606bd80 0x55c49606e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c495fd5ea0 con 0x55c49605fdf0 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c495fc52d0 con 0x55c49606bd80 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb81920c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c495fc90e0 con 0x55c49605f4d0 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb814bd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c49606b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb8153d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49605f4d0 0x55c49605f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb8153d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49605f4d0 0x55c49605f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54696/0 (socket says 172.21.15.33:54696) 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb8153d4640 1 -- 172.21.15.33:0/3244121358 learned_addr learned my addr 172.21.15.33:0/3244121358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.017+0000 7fb80ffff640 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49606bd80 0x55c49606e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb8153d4640 1 -- 172.21.15.33:0/3244121358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49606bd80 msgr2=0x55c49606e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb8153d4640 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49606bd80 0x55c49606e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb8153d4640 1 -- 172.21.15.33:0/3244121358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 msgr2=0x55c49606b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb8153d4640 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c49606b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb8153d4640 1 -- 172.21.15.33:0/3244121358 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c495d23650 con 0x55c49605f4d0 2024-09-10T21:17:04.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb814bd3640 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c49606b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb8153d4640 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49605f4d0 0x55c49605f8b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb808035000 tx=0x7fb8080093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d937ce225be801 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb80dffb640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8000b52e0 0x7fb8000b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb80dffb640 1 -- 172.21.15.33:0/4070307354 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb8040e5f20 con 0x7fb810101830 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb814bd3640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8000b52e0 0x7fb8000b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb7faffd640 1 -- 172.21.15.33:0/3244121358 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808048070 con 0x55c49605f4d0 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb7faffd640 1 -- 172.21.15.33:0/3244121358 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb808003e20 con 0x55c49605f4d0 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.018+0000 7fb7faffd640 1 -- 172.21.15.33:0/3244121358 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808034040 con 0x55c49605f4d0 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 -- 172.21.15.33:0/3244121358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49605f4d0 msgr2=0x55c49605f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb814bd3640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8000b52e0 0x7fb8000b77a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb80400c6a0 tx=0x7fb8040e99a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49605f4d0 0x55c49605f8b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb808035000 tx=0x7fb8080093e0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 -- 172.21.15.33:0/3244121358 shutdown_connections 2024-09-10T21:17:04.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49606bd80 0x55c49606e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c49606b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 --2- 172.21.15.33:0/3244121358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49605f4d0 0x55c49605f8b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 -- 172.21.15.33:0/3244121358 >> 172.21.15.33:0/3244121358 conn(0x55c49605d300 msgr2=0x55c49605e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 -- 172.21.15.33:0/3244121358 shutdown_connections 2024-09-10T21:17:04.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.019+0000 7fb81920c740 1 -- 172.21.15.33:0/3244121358 wait complete. 2024-09-10T21:17:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 -- start start 2024-09-10T21:17:04.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 Processor -- start 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c496068b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c495e26700 con 0x55c49605fdf0 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c495f60160 con 0x55c49605f4d0 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c49606e9a0 con 0x55c49606bd80 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb81920c740 10 client.0 _ll_register_callbacks cb 0x55c495f799e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb80ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c496068b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb8153d4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb8153d4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36172/0 (socket says 172.21.15.33:36172) 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb8153d4640 1 -- 172.21.15.33:0/4046105875 learned_addr learned my addr 172.21.15.33:0/4046105875 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:04.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb80ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c496068b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51812/0 (socket says 172.21.15.33:51812) 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.020+0000 7fb8153d4640 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb80ffff640 1 -- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 msgr2=0x55c4960691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb80ffff640 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb80ffff640 1 -- 172.21.15.33:0/4046105875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 msgr2=0x55c49606b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb80ffff640 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb80ffff640 1 -- 172.21.15.33:0/4046105875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c495f8dc90 con 0x55c49605fdf0 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb80ffff640 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c496068b30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0126d0 tx=0x7fb7fc012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83cb027efaf80871 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb7effff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb7effff640 1 -- 172.21.15.33:0/4046105875 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc004020 con 0x55c49605fdf0 2024-09-10T21:17:04.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb7effff640 1 -- 172.21.15.33:0/4046105875 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb7fc0041c0 con 0x55c49605fdf0 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb808003e20 con 0x55c49605fdf0 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb7effff640 1 -- 172.21.15.33:0/4046105875 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fb7fc0043c0 con 0x55c49605fdf0 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb81920c740 10 client.17817 fetch_fsmap learned FSMap version 62 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.021+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55c49607a3e0 con 0x55c49605fdf0 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb7effff640 1 -- 172.21.15.33:0/4046105875 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb808003e20 con 0x55c49605fdf0 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb7effff640 1 -- 172.21.15.33:0/4046105875 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 62) ==== 8378+0+0 (secure 0 0 0) 0x7fb7fc0039b0 con 0x55c49605fdf0 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb81920c740 10 client.17817 fetch_fsmap finished waiting for FSMap version 62 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb81920c740 10 client.17817 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb81920c740 20 client.17817 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb81920c740 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c49607b8a0 0x55c49607dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb81920c740 4 client.17817 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c495fd7df0 con 0x55c49607b8a0 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.022+0000 7fb8153d4640 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c49607b8a0 0x55c49607dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.023+0000 7fb8153d4640 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c49607b8a0 0x55c49607dd00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12761486c2a3589d server_cookie=53568285dbcda8d1 in_seq=0 out_seq=0 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.023+0000 7fb7effff640 10 client.17817 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.023+0000 7fb7effff640 1 -- 172.21.15.33:0/4046105875 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c495fd7df0 con 0x55c49607b8a0 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.023+0000 7fb7effff640 10 client.17817 handle_command_reply: tid=0 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.023+0000 7fb81920c740 1 client.17817 shutdown 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.023+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 msgr2=0x55c496068b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.023+0000 7fb81920c740 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c496068b30 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc0126d0 tx=0x7fb7fc012b90 comp rx=0 tx=0).stop 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 shutdown_connections 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c49607b8a0 0x55c49607dd00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605fdf0 0x55c496068b30 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb7effff640 0 client.17817 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 --2- 172.21.15.33:0/4046105875 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 >> 172.21.15.33:0/4046105875 conn(0x55c49605d300 msgr2=0x55c496062ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 shutdown_connections 2024-09-10T21:17:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 1 -- 172.21.15.33:0/4046105875 wait complete. 2024-09-10T21:17:04.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.024+0000 7fb81920c740 20 client.17817 trim_cache size 0 max 16384 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.038+0000 7fb81920c740 1 Processor -- start 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.038+0000 7fb81920c740 1 -- start start 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb81920c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605f4d0 0x55c496077c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb81920c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb81920c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4960678f0 0x55c49606d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb81920c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8101b0570 con 0x55c49605f4d0 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb81920c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8101b2140 con 0x55c4960678f0 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb81920c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8101b0c40 con 0x55c49606bd80 2024-09-10T21:17:04.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605f4d0 0x55c496077c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605f4d0 0x55c496077c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51814/0 (socket says 172.21.15.33:51814) 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 -- 172.21.15.33:0/2726765772 learned_addr learned my addr 172.21.15.33:0/2726765772 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb80ffff640 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4960678f0 0x55c49606d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 -- 172.21.15.33:0/2726765772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 msgr2=0x55c4960673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 -- 172.21.15.33:0/2726765772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4960678f0 msgr2=0x55c49606d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4960678f0 0x55c49606d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 -- 172.21.15.33:0/2726765772 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c496069520 con 0x55c49605f4d0 2024-09-10T21:17:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.039+0000 7fb8153d4640 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605f4d0 0x55c496077c00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb80802ae70 tx=0x7fb80802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c861a993e6256e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.040+0000 7fb7fa7fc640 1 -- 172.21.15.33:0/2726765772 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb808048040 con 0x55c49605f4d0 2024-09-10T21:17:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.040+0000 7fb7fa7fc640 1 -- 172.21.15.33:0/2726765772 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb808003e20 con 0x55c49605f4d0 2024-09-10T21:17:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.040+0000 7fb7fa7fc640 1 -- 172.21.15.33:0/2726765772 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8080047b0 con 0x55c49605f4d0 2024-09-10T21:17:04.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 -- 172.21.15.33:0/2726765772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605f4d0 msgr2=0x55c496077c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605f4d0 0x55c496077c00 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fb80802ae70 tx=0x7fb80802a950 comp rx=0 tx=0).stop 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 -- 172.21.15.33:0/2726765772 shutdown_connections 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c4960678f0 0x55c49606d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c4960673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 --2- 172.21.15.33:0/2726765772 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c49605f4d0 0x55c496077c00 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 -- 172.21.15.33:0/2726765772 >> 172.21.15.33:0/2726765772 conn(0x55c49605d300 msgr2=0x55c4960583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 -- 172.21.15.33:0/2726765772 shutdown_connections 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 -- 172.21.15.33:0/2726765772 wait complete. 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 -- start start 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 Processor -- start 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.041+0000 7fb81920c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4960678f0 0x55c49606f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb81920c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c49606f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb81920c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb8101b1a10 con 0x55c4960678f0 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb81920c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb8101b1310 con 0x55c49605f4d0 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb81920c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8101ae530 con 0x55c49606bd80 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb81920c740 10 client.0 _ll_register_callbacks cb 0x7fb7fc009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4960678f0 0x55c49606f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4960678f0 0x55c49606f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51822/0 (socket says 172.21.15.33:51822) 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 -- 172.21.15.33:0/3082997429 learned_addr learned my addr 172.21.15.33:0/3082997429 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb8153d4640 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c49606f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb8153d4640 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 -- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 msgr2=0x55c49606f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c49606f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 -- 172.21.15.33:0/3082997429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 msgr2=0x55c49606ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 -- 172.21.15.33:0/3082997429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4960e8a30 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb80ffff640 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4960678f0 0x55c49606f2d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc012040 tx=0x7fb7fc00fb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e48f471362e3508 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb7effff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb7effff640 1 -- 172.21.15.33:0/3082997429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02c040 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.042+0000 7fb7effff640 1 -- 172.21.15.33:0/3082997429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb7fc00fd40 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb808003e20 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb7effff640 1 -- 172.21.15.33:0/3082997429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7fb7fc009d30 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 10 client.17823 fetch_fsmap learned FSMap version 62 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55c4960e9d60 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb7effff640 1 -- 172.21.15.33:0/3082997429 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb808003e20 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb7effff640 1 -- 172.21.15.33:0/3082997429 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 62) ==== 8378+0+0 (secure 0 0 0) 0x7fb7fc0052b0 con 0x55c4960678f0 2024-09-10T21:17:04.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 10 client.17823 fetch_fsmap finished waiting for FSMap version 62 2024-09-10T21:17:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 10 client.17823 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:17:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 20 client.17823 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c4960ea4b0 0x55c4960ec8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 4 client.17823 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c495fd38a0 con 0x55c4960ea4b0 2024-09-10T21:17:04.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.043+0000 7fb8153d4640 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c4960ea4b0 0x55c4960ec8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb7effff640 10 client.17823 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb8153d4640 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c4960ea4b0 0x55c4960ec8d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9afe46cbebbd457f server_cookie=df93f3f956a16d7 in_seq=0 out_seq=0 2024-09-10T21:17:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb7effff640 1 -- 172.21.15.33:0/3082997429 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c495fd38a0 con 0x55c4960ea4b0 2024-09-10T21:17:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb7effff640 10 client.17823 handle_command_reply: tid=0 2024-09-10T21:17:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 client.17823 shutdown 2024-09-10T21:17:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4960678f0 msgr2=0x55c49606f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4960678f0 0x55c49606f2d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc012040 tx=0x7fb7fc00fb50 comp rx=0 tx=0).stop 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 shutdown_connections 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c4960ea4b0 0x55c4960ec8d0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c49606bd80 0x55c49606f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c4960678f0 0x55c49606f2d0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 --2- 172.21.15.33:0/3082997429 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c49605f4d0 0x55c49606ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 >> 172.21.15.33:0/3082997429 conn(0x55c49605d300 msgr2=0x55c49606a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb7effff640 0 client.17823 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 shutdown_connections 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.044+0000 7fb81920c740 1 -- 172.21.15.33:0/3082997429 wait complete. 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb81920c740 20 client.17823 trim_cache size 0 max 16384 2024-09-10T21:17:04.053 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:04.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 3470 89.47 MB/sec warmup 4 sec latency 68.930 ms 2024-09-10T21:17:04.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8000b52e0 msgr2=0x7fb8000b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8000b52e0 0x7fb8000b77a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb80400c6a0 tx=0x7fb8040e99a0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb810101830 msgr2=0x7fb81019b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb810101830 0x7fb81019b430 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb80400e9f0 tx=0x7fb80400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 shutdown_connections 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb8000b52e0 0x7fb8000b77a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb81010f3e0 0x7fb8101a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb810102150 0x7fb81019b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 --2- 172.21.15.33:0/4070307354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb810101830 0x7fb81019b430 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.045+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 >> 172.21.15.33:0/4070307354 conn(0x7fb8100fd700 msgr2=0x7fb8101115d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.046+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 shutdown_connections 2024-09-10T21:17:04.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:04.046+0000 7fb816e37640 1 -- 172.21.15.33:0/4070307354 wait complete. 2024-09-10T21:17:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:03 smithi033 ceph-mon[30860]: pgmap v738: 97 pgs: 97 active+clean; 379 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-09-10T21:17:04.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:03 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:04.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:03 smithi104 ceph-mon[35325]: pgmap v738: 97 pgs: 97 active+clean; 379 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-09-10T21:17:04.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:03 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:04.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:03 smithi174 ceph-mon[35635]: pgmap v738: 97 pgs: 97 active+clean; 379 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 755 KiB/s rd, 1.5 MiB/s wr, 267 op/s 2024-09-10T21:17:04.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:03 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:04.733 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:17:05.024 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:05.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 4578 85.87 MB/sec warmup 5 sec latency 45.272 ms 2024-09-10T21:17:05.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 -- 172.21.15.33:0/1827511925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c101700 msgr2=0x7f493c101ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 --2- 172.21.15.33:0/1827511925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c101700 0x7f493c101ae0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4938009900 tx=0x7f493802a480 comp rx=0 tx=0).stop 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 -- 172.21.15.33:0/1827511925 shutdown_connections 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 --2- 172.21.15.33:0/1827511925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c10f2b0 0x7f493c1116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 --2- 172.21.15.33:0/1827511925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c102020 0x7f493c10ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 --2- 172.21.15.33:0/1827511925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c101700 0x7f493c101ae0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 -- 172.21.15.33:0/1827511925 >> 172.21.15.33:0/1827511925 conn(0x7f493c0fd5b0 msgr2=0x7f493c0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 -- 172.21.15.33:0/1827511925 shutdown_connections 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.545+0000 7f494357e640 1 -- 172.21.15.33:0/1827511925 wait complete. 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 Processor -- start 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 -- start start 2024-09-10T21:17:05.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c101700 0x7f493c19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c102020 0x7f493c19b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c10f2b0 0x7f493c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f493c114db0 con 0x7f493c10f2b0 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f493c114c30 con 0x7f493c102020 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494357e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f493c114f30 con 0x7f493c101700 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494257c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c101700 0x7f493c19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494257c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c101700 0x7f493c19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54742/0 (socket says 172.21.15.33:54742) 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f4942d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c10f2b0 0x7f493c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f494257c640 1 -- 172.21.15.33:0/3517738057 learned_addr learned my addr 172.21.15.33:0/3517738057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.546+0000 7f4941d7b640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c102020 0x7f493c19b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494257c640 1 -- 172.21.15.33:0/3517738057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c102020 msgr2=0x7f493c19b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494257c640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c102020 0x7f493c19b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494257c640 1 -- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c10f2b0 msgr2=0x7f493c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494257c640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c10f2b0 0x7f493c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494257c640 1 -- 172.21.15.33:0/3517738057 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f493c1a1330 con 0x7f493c101700 2024-09-10T21:17:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494257c640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c101700 0x7f493c19b280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4938005940 tx=0x7f49380040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f49337fe640 1 -- 172.21.15.33:0/3517738057 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493803f030 con 0x7f493c101700 2024-09-10T21:17:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f49337fe640 1 -- 172.21.15.33:0/3517738057 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49380044c0 con 0x7f493c101700 2024-09-10T21:17:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f493c1a15c0 con 0x7f493c101700 2024-09-10T21:17:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f49337fe640 1 -- 172.21.15.33:0/3517738057 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938035070 con 0x7f493c101700 2024-09-10T21:17:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.547+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f493c1a7ce0 con 0x7f493c101700 2024-09-10T21:17:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f49337fe640 1 -- 172.21.15.33:0/3517738057 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f493803ba20 con 0x7f493c101700 2024-09-10T21:17:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 Processor -- start 2024-09-10T21:17:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 -- start start 2024-09-10T21:17:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022faea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fbaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fbaf50 0x55e022fbd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e022f24ea0 con 0x55e022fae6a0 2024-09-10T21:17:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e022f142d0 con 0x55e022faefc0 2024-09-10T21:17:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.550+0000 7f4946bb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e022f180e0 con 0x55e022fbaf50 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f4942d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022faea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f4942d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022faea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51854/0 (socket says 172.21.15.33:51854) 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f494257c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fbaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f4942d7d640 1 -- 172.21.15.33:0/2563302310 learned_addr learned my addr 172.21.15.33:0/2563302310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f4941d7b640 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fbaf50 0x55e022fbd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f494257c640 1 -- 172.21.15.33:0/2563302310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fbaf50 msgr2=0x55e022fbd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f494257c640 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fbaf50 0x55e022fbd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f494257c640 1 -- 172.21.15.33:0/2563302310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 msgr2=0x55e022faea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f494257c640 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022faea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f494257c640 1 -- 172.21.15.33:0/2563302310 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e022c72650 con 0x55e022faefc0 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f49337fe640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49280b5390 0x7f49280b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f4941d7b640 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fbaf50 0x55e022fbd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f4942d7d640 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022faea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.551+0000 7f4942d7d640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49280b5390 0x7f49280b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.552+0000 7f494257c640 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fbaa10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4938103c40 tx=0x7f4938109eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc197b2e2969d933 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.552+0000 7f49337fe640 1 -- 172.21.15.33:0/3517738057 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f49381061e0 con 0x7f493c101700 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.552+0000 7f4930ff9640 1 -- 172.21.15.33:0/2563302310 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938049030 con 0x55e022faefc0 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.552+0000 7f4930ff9640 1 -- 172.21.15.33:0/2563302310 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49380c8170 con 0x55e022faefc0 2024-09-10T21:17:05.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.552+0000 7f4930ff9640 1 -- 172.21.15.33:0/2563302310 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380cc360 con 0x55e022faefc0 2024-09-10T21:17:05.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 -- 172.21.15.33:0/2563302310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 msgr2=0x55e022fbaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fbaa10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4938103c40 tx=0x7f4938109eb0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4942d7d640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49280b5390 0x7f49280b7850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f4934009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 -- 172.21.15.33:0/2563302310 shutdown_connections 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fbaf50 0x55e022fbd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fbaa10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 --2- 172.21.15.33:0/2563302310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022faea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 -- 172.21.15.33:0/2563302310 >> 172.21.15.33:0/2563302310 conn(0x55e022fac4d0 msgr2=0x55e022fad670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 -- 172.21.15.33:0/2563302310 shutdown_connections 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 -- 172.21.15.33:0/2563302310 wait complete. 2024-09-10T21:17:05.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 -- start start 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.553+0000 7f4946bb5740 1 Processor -- start 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fae6a0 0x55e022fba880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fb7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fb8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4946bb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e022d75700 con 0x55e022fbaf50 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4946bb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e022eaf160 con 0x55e022faefc0 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4946bb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e022fbd990 con 0x55e022fae6a0 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4946bb5740 10 client.0 _ll_register_callbacks cb 0x55e022ec89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fae6a0 0x55e022fba880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4942d7d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fb8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fae6a0 0x55e022fba880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54762/0 (socket says 172.21.15.33:54762) 2024-09-10T21:17:05.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4941d7b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fb7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 -- 172.21.15.33:0/1524684407 learned_addr learned my addr 172.21.15.33:0/1524684407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 -- 172.21.15.33:0/1524684407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 msgr2=0x55e022fb7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fb7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 -- 172.21.15.33:0/1524684407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 msgr2=0x55e022fb8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fb8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f494257c640 1 -- 172.21.15.33:0/1524684407 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e022edcc90 con 0x55e022fae6a0 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.554+0000 7f4942d7d640 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fb8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:05.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f494257c640 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fae6a0 0x55e022fba880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4938031440 tx=0x7f493802a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84ce950f9a0058c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4925ffb640 1 -- 172.21.15.33:0/1524684407 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938049040 con 0x55e022fae6a0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4925ffb640 1 -- 172.21.15.33:0/1524684407 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4938003cf0 con 0x55e022fae6a0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49380c8170 con 0x55e022fae6a0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4925ffb640 1 -- 172.21.15.33:0/1524684407 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f49380cc180 con 0x55e022fae6a0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 10 client.27199 fetch_fsmap learned FSMap version 63 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55e022fc9240 con 0x55e022fae6a0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4925ffb640 1 -- 172.21.15.33:0/1524684407 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49380c8170 con 0x55e022fae6a0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4925ffb640 1 -- 172.21.15.33:0/1524684407 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f49380c8c80 con 0x55e022fae6a0 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 10 client.27199 fetch_fsmap finished waiting for FSMap version 63 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 10 client.27199 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 20 client.27199 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:05.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e022fcc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 4 client.27199 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.555+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e022f26df0 con 0x55e022fca3c0 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.556+0000 7f4941d7b640 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e022fcc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.556+0000 7f4925ffb640 10 client.27199 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.556+0000 7f4941d7b640 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e022fcc7b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=46caadf12d99e9e0 server_cookie=6af79f10a1fb223c in_seq=0 out_seq=0 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4925ffb640 1 -- 172.21.15.33:0/1524684407 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e022f26df0 con 0x55e022fca3c0 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4925ffb640 10 client.27199 handle_command_reply: tid=0 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 client.27199 shutdown 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fae6a0 msgr2=0x55e022fba880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fae6a0 0x55e022fba880 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4938031440 tx=0x7f493802a480 comp rx=0 tx=0).stop 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 shutdown_connections 2024-09-10T21:17:05.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e022fcc7b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fb8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022faefc0 0x55e022fb7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4925ffb640 0 client.27199 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1524684407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022fae6a0 0x55e022fba880 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 >> 172.21.15.33:0/1524684407 conn(0x55e022fac4d0 msgr2=0x55e022fb2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 shutdown_connections 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.557+0000 7f4946bb5740 1 -- 172.21.15.33:0/1524684407 wait complete. 2024-09-10T21:17:05.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.558+0000 7f4946bb5740 20 client.27199 trim_cache size 0 max 16384 2024-09-10T21:17:05.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 Processor -- start 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 -- start start 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022fabe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fb9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fbaf50 0x55e022fbc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1abf70 con 0x55e022fae6a0 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1adba0 con 0x55e022fbaf50 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4946bb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1ac670 con 0x55e022faefc0 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f494257c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022fabe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f494257c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022fabe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51882/0 (socket says 172.21.15.33:51882) 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f494257c640 1 -- 172.21.15.33:0/3398096162 learned_addr learned my addr 172.21.15.33:0/3398096162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:05.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.573+0000 7f4942d7d640 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fbaf50 0x55e022fbc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f4941d7b640 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fb9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f494257c640 1 -- 172.21.15.33:0/3398096162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 msgr2=0x55e022fb9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f494257c640 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fb9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f494257c640 1 -- 172.21.15.33:0/3398096162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fbaf50 msgr2=0x55e022fbc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f494257c640 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fbaf50 0x55e022fbc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f494257c640 1 -- 172.21.15.33:0/3398096162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e022faedf0 con 0x55e022fae6a0 2024-09-10T21:17:05.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f494257c640 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022fabe60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f49380cc9a0 tx=0x7f493803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5d03f75feb2cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f4927fff640 1 -- 172.21.15.33:0/3398096162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938049070 con 0x55e022fae6a0 2024-09-10T21:17:05.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f4927fff640 1 -- 172.21.15.33:0/3398096162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49380c8170 con 0x55e022fae6a0 2024-09-10T21:17:05.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.574+0000 7f4927fff640 1 -- 172.21.15.33:0/3398096162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49380c9060 con 0x55e022fae6a0 2024-09-10T21:17:05.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 -- 172.21.15.33:0/3398096162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 msgr2=0x55e022fabe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022fabe60 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f49380cc9a0 tx=0x7f493803b460 comp rx=0 tx=0).stop 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 -- 172.21.15.33:0/3398096162 shutdown_connections 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fbaf50 0x55e022fbc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fb9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 --2- 172.21.15.33:0/3398096162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fae6a0 0x55e022fabe60 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 -- 172.21.15.33:0/3398096162 >> 172.21.15.33:0/3398096162 conn(0x55e022fac4d0 msgr2=0x55e022fb9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 -- 172.21.15.33:0/3398096162 shutdown_connections 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.575+0000 7f4946bb5740 1 -- 172.21.15.33:0/3398096162 wait complete. 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 -- start start 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 Processor -- start 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fae6a0 0x55e022fc3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fd1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fd20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1ad470 con 0x55e022fbaf50 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1acd70 con 0x55e022fae6a0 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f493c1a9f50 con 0x55e022faefc0 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f494257c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fae6a0 0x55e022fc3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4941d7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fd1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4946bb5740 10 client.0 _ll_register_callbacks cb 0x7f4934009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f494257c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fae6a0 0x55e022fc3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36246/0 (socket says 172.21.15.33:36246) 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f494257c640 1 -- 172.21.15.33:0/1058777335 learned_addr learned my addr 172.21.15.33:0/1058777335 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.576+0000 7f4942d7d640 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fd20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f494257c640 1 -- 172.21.15.33:0/1058777335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 msgr2=0x55e022fd1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f494257c640 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fd1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f494257c640 1 -- 172.21.15.33:0/1058777335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 msgr2=0x55e022fd20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f494257c640 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fd20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f494257c640 1 -- 172.21.15.33:0/1058777335 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e022fd28e0 con 0x55e022fae6a0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4941d7b640 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fd1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f494257c640 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fae6a0 0x55e022fc3370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49380d1460 tx=0x7f493802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96e27b7e02ab92bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4925ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4925ffb640 1 -- 172.21.15.33:0/1058777335 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4938049070 con 0x55e022fae6a0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4925ffb640 1 -- 172.21.15.33:0/1058777335 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4938109ab0 con 0x55e022fae6a0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49380c8170 con 0x55e022fae6a0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4925ffb640 1 -- 172.21.15.33:0/1058777335 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f49380cc7e0 con 0x55e022fae6a0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4946bb5740 10 client.27272 fetch_fsmap learned FSMap version 63 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) -- 0x55e022fd28e0 con 0x55e022fae6a0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.577+0000 7f4925ffb640 1 -- 172.21.15.33:0/1058777335 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49380c8170 con 0x55e022fae6a0 2024-09-10T21:17:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4925ffb640 1 -- 172.21.15.33:0/1058777335 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 63) ==== 8386+0+0 (secure 0 0 0) 0x7f49380b7020 con 0x55e022fae6a0 2024-09-10T21:17:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4946bb5740 10 client.27272 fetch_fsmap finished waiting for FSMap version 63 2024-09-10T21:17:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4946bb5740 10 client.27272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:17:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4946bb5740 20 client.27272 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e023039520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4946bb5740 4 client.27272 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e022f228a0 con 0x55e022fca3c0 2024-09-10T21:17:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.578+0000 7f4941d7b640 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e023039520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4941d7b640 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e023039520 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2cbacf61061f4638 server_cookie=322ed0dff66bb786 in_seq=0 out_seq=0 2024-09-10T21:17:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4925ffb640 10 client.27272 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4925ffb640 1 -- 172.21.15.33:0/1058777335 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e022f228a0 con 0x55e022fca3c0 2024-09-10T21:17:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4925ffb640 10 client.27272 handle_command_reply: tid=0 2024-09-10T21:17:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 client.27272 shutdown 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fae6a0 msgr2=0x55e022fc3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fae6a0 0x55e022fc3370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f49380d1460 tx=0x7f493802a480 comp rx=0 tx=0).stop 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 shutdown_connections 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e022fca3c0 0x55e023039520 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e022fbaf50 0x55e022fd20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e022faefc0 0x55e022fd1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 --2- 172.21.15.33:0/1058777335 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e022fae6a0 0x55e022fc3370 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4925ffb640 0 client.27272 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 >> 172.21.15.33:0/1058777335 conn(0x55e022fac4d0 msgr2=0x55e022fb9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 shutdown_connections 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 1 -- 172.21.15.33:0/1058777335 wait complete. 2024-09-10T21:17:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.579+0000 7f4946bb5740 20 client.27272 trim_cache size 0 max 16384 2024-09-10T21:17:05.589 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49280b5390 msgr2=0x7f49280b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49280b5390 0x7f49280b7850 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4934009fd0 tx=0x7f4934009500 comp rx=0 tx=0).stop 2024-09-10T21:17:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c101700 msgr2=0x7f493c19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:05.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c101700 0x7f493c19b280 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4938005940 tx=0x7f49380040c0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 shutdown_connections 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49280b5390 0x7f49280b7850 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f493c10f2b0 0x7f493c1a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f493c102020 0x7f493c19b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 --2- 172.21.15.33:0/3517738057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f493c101700 0x7f493c19b280 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 >> 172.21.15.33:0/3517738057 conn(0x7f493c0fd5b0 msgr2=0x7f493c111590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 shutdown_connections 2024-09-10T21:17:05.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:05.580+0000 7f494357e640 1 -- 172.21.15.33:0/3517738057 wait complete. 2024-09-10T21:17:06.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 5046 75.40 MB/sec warmup 6 sec latency 56.770 ms 2024-09-10T21:17:06.280 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:05 smithi033 ceph-mon[30860]: pgmap v739: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.1 MiB/s wr, 326 op/s 2024-09-10T21:17:06.281 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:05 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:06.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:17:06.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:05 smithi104 ceph-mon[35325]: pgmap v739: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.1 MiB/s wr, 326 op/s 2024-09-10T21:17:06.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:05 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:06.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:05 smithi174 ceph-mon[35635]: pgmap v739: 97 pgs: 97 active+clean; 378 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 2.1 MiB/s wr, 326 op/s 2024-09-10T21:17:06.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:05 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:06.623 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:07.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 5773 68.36 MB/sec warmup 7 sec latency 181.490 ms 2024-09-10T21:17:07.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.160+0000 7fb0bf330640 1 -- 172.21.15.33:0/3572229329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b8109160 msgr2=0x7fb0b810f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.160+0000 7fb0bf330640 1 --2- 172.21.15.33:0/3572229329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b8109160 0x7fb0b810f960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0009be0 tx=0x7fb0b002a6c0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.161+0000 7fb0bf330640 1 -- 172.21.15.33:0/3572229329 shutdown_connections 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.161+0000 7fb0bf330640 1 --2- 172.21.15.33:0/3572229329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b8109160 0x7fb0b810f960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.161+0000 7fb0bf330640 1 --2- 172.21.15.33:0/3572229329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b81087e0 0x7fb0b8108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.161+0000 7fb0bf330640 1 --2- 172.21.15.33:0/3572229329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8102dd0 0x7fb0b81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.161+0000 7fb0bf330640 1 -- 172.21.15.33:0/3572229329 >> 172.21.15.33:0/3572229329 conn(0x7fb0b80fe800 msgr2=0x7fb0b8100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.161+0000 7fb0bf330640 1 -- 172.21.15.33:0/3572229329 shutdown_connections 2024-09-10T21:17:07.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.161+0000 7fb0bf330640 1 -- 172.21.15.33:0/3572229329 wait complete. 2024-09-10T21:17:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.162+0000 7fb0bf330640 1 Processor -- start 2024-09-10T21:17:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.162+0000 7fb0bf330640 1 -- start start 2024-09-10T21:17:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.162+0000 7fb0bf330640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8102dd0 0x7fb0b819b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.162+0000 7fb0bf330640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b81087e0 0x7fb0b819ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.162+0000 7fb0bf330640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b8109160 0x7fb0b81a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bf330640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8076b30 con 0x7fb0b8109160 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bf330640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b80769b0 con 0x7fb0b81087e0 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bf330640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b8076cb0 con 0x7fb0b8102dd0 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b8109160 0x7fb0b81a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b8109160 0x7fb0b81a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51908/0 (socket says 172.21.15.33:51908) 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/2300322345 learned_addr learned my addr 172.21.15.33:0/2300322345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b81087e0 0x7fb0b819ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8102dd0 0x7fb0b819b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/2300322345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8102dd0 msgr2=0x7fb0b819b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8102dd0 0x7fb0b819b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/2300322345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b81087e0 msgr2=0x7fb0b819ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b81087e0 0x7fb0b819ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.163+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/2300322345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0b81a15c0 con 0x7fb0b8109160 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.164+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8102dd0 0x7fb0b819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.164+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b8109160 0x7fb0b81a0df0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0004af0 tx=0x7fb0b0003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.164+0000 7fb0ae7fc640 1 -- 172.21.15.33:0/2300322345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0041020 con 0x7fb0b8109160 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.164+0000 7fb0ae7fc640 1 -- 172.21.15.33:0/2300322345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0b0030970 con 0x7fb0b8109160 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.164+0000 7fb0ae7fc640 1 -- 172.21.15.33:0/2300322345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0036040 con 0x7fb0b8109160 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.165+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0b81a1850 con 0x7fb0b8109160 2024-09-10T21:17:07.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.165+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0b81a1c20 con 0x7fb0b8109160 2024-09-10T21:17:07.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.169+0000 7fb0c1705740 1 Processor -- start 2024-09-10T21:17:07.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.170+0000 7fb0ae7fc640 1 -- 172.21.15.33:0/2300322345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb0b0004460 con 0x7fb0b8109160 2024-09-10T21:17:07.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0ae7fc640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0a40b53e0 0x7fb0a40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0c1705740 1 -- start start 2024-09-10T21:17:07.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0ae7fc640 1 -- 172.21.15.33:0/2300322345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb0b0106b20 con 0x7fb0b8109160 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0a40b53e0 0x7fb0a40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdb34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdc1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0c1705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55deebd29ea0 con 0x55deebdb30f0 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0c1705740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55deebd192d0 con 0x55deebdbf9a0 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.172+0000 7fb0c1705740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55deebd1d0e0 con 0x55deebdb3a10 2024-09-10T21:17:07.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bc8cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bc8cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54814/0 (socket says 172.21.15.33:54814) 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/547608261 learned_addr learned my addr 172.21.15.33:0/547608261 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdb34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/547608261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 msgr2=0x55deebdc1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdc1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdc1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/547608261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 msgr2=0x55deebdb34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.173+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdb34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/547608261 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deeba77650 con 0x55deebdb3a10 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdc1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdb34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:07.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0a40b53e0 0x7fb0a40b78a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80139d0 tx=0x7fb0a8009b70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbf460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009700 tx=0x7fb0a801d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dcb0374df7405968 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0a37fe640 1 -- 172.21.15.33:0/547608261 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8013170 con 0x55deebdb3a10 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0a37fe640 1 -- 172.21.15.33:0/547608261 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a8013310 con 0x55deebdb3a10 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.174+0000 7fb0a37fe640 1 -- 172.21.15.33:0/547608261 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a800e7b0 con 0x55deebdb3a10 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 -- 172.21.15.33:0/547608261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 msgr2=0x55deebdbf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbf460 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009700 tx=0x7fb0a801d000 comp rx=0 tx=0).stop 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 -- 172.21.15.33:0/547608261 shutdown_connections 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdc1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbf460 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 --2- 172.21.15.33:0/547608261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdb34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 -- 172.21.15.33:0/547608261 >> 172.21.15.33:0/547608261 conn(0x55deebdb0f20 msgr2=0x55deebdb20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 -- 172.21.15.33:0/547608261 shutdown_connections 2024-09-10T21:17:07.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.176+0000 7fb0c1705740 1 -- 172.21.15.33:0/547608261 wait complete. 2024-09-10T21:17:07.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.177+0000 7fb0c1705740 1 -- start start 2024-09-10T21:17:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.177+0000 7fb0c1705740 1 Processor -- start 2024-09-10T21:17:07.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.178+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdbf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.178+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdbce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0c1705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55deebb7a700 con 0x55deebdb30f0 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0c1705740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55deebcb4160 con 0x55deebdbf9a0 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0bd8cd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdbce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0bd0cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdbf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0bc8cb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0bd0cc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdbf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51930/0 (socket says 172.21.15.33:51930) 2024-09-10T21:17:07.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0bd0cc640 1 -- 172.21.15.33:0/2052854604 learned_addr learned my addr 172.21.15.33:0/2052854604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55deebdcad70 con 0x55deebdb3a10 2024-09-10T21:17:07.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.179+0000 7fb0c1705740 10 client.0 _ll_register_callbacks cb 0x55deebccd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:07.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/2052854604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 msgr2=0x55deebdbc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/2052854604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 msgr2=0x55deebdbf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdbf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/2052854604 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deebce1c90 con 0x55deebdbf9a0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdbf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdbce70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0b01045e0 tx=0x7fb0b00cca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98af49d855778a3e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a8013310 con 0x55deebdbf9a0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/2052854604 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00cd2a0 con 0x55deebdbf9a0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/2052854604 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb0b00f4140 con 0x55deebdbf9a0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/2052854604 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb0b00d1d80 con 0x55deebdbf9a0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0c1705740 10 client.27281 fetch_fsmap learned FSMap version 64 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.180+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=64}) -- 0x55deebdd6680 con 0x55deebdbf9a0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.181+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/2052854604 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 64) ==== 8390+0+0 (secure 0 0 0) 0x7fb0b00f00c0 con 0x55deebdbf9a0 2024-09-10T21:17:07.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.181+0000 7fb0c1705740 10 client.27281 fetch_fsmap finished waiting for FSMap version 64 2024-09-10T21:17:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.181+0000 7fb0c1705740 10 client.27281 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:17:07.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.181+0000 7fb0c1705740 20 client.27281 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.181+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebdd7a20 0x55deebdd9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.181+0000 7fb0c1705740 4 client.27281 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.181+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deebd2bdf0 con 0x55deebdd7a20 2024-09-10T21:17:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.182+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebdd7a20 0x55deebdd9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.182+0000 7fb0a0ff9640 10 client.27281 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:07.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.182+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebdd7a20 0x55deebdd9e10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65904f353440b4be server_cookie=e665145052a1d5e in_seq=0 out_seq=0 2024-09-10T21:17:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/2052854604 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55deebd2bdf0 con 0x55deebdd7a20 2024-09-10T21:17:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0a0ff9640 10 client.27281 handle_command_reply: tid=0 2024-09-10T21:17:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 client.27281 shutdown 2024-09-10T21:17:07.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 msgr2=0x55deebdbce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdbce70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb0b01045e0 tx=0x7fb0b00cca10 comp rx=0 tx=0).stop 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 shutdown_connections 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdbf9a0 0x55deebdbce70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdbc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebdd7a20 0x55deebdd9e10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2052854604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb30f0 0x55deebdbf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 >> 172.21.15.33:0/2052854604 conn(0x55deebdb0f20 msgr2=0x55deebdb6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0a0ff9640 0 client.27281 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 shutdown_connections 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.185+0000 7fb0c1705740 1 -- 172.21.15.33:0/2052854604 wait complete. 2024-09-10T21:17:07.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.186+0000 7fb0c1705740 20 client.27281 trim_cache size 0 max 16384 2024-09-10T21:17:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.205+0000 7fb0c1705740 1 Processor -- start 2024-09-10T21:17:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.205+0000 7fb0c1705740 1 -- start start 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.205+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb3a10 0x55deebdb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdbf9a0 0x55deebdb7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0c1705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81ac120 con 0x55deebdb3a10 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0c1705740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81add80 con 0x55deebdb30f0 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0c1705740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81ac820 con 0x55deebdbf9a0 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb3a10 0x55deebdb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb3a10 0x55deebdb79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51934/0 (socket says 172.21.15.33:51934) 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/2021523888 learned_addr learned my addr 172.21.15.33:0/2021523888 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bd0cc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdbf9a0 0x55deebdb7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/2021523888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdbf9a0 msgr2=0x55deebdb7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdbf9a0 0x55deebdb7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/2021523888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 msgr2=0x55deebdd3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 -- 172.21.15.33:0/2021523888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deebdb19a0 con 0x55deebdb3a10 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdbf9a0 0x55deebdb7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.206+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb3a10 0x55deebdb79f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8023040 tx=0x7fb0a8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167bda04e05ded29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.207+0000 7fb0a2ffd640 1 -- 172.21.15.33:0/2021523888 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8030020 con 0x55deebdb3a10 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.207+0000 7fb0a2ffd640 1 -- 172.21.15.33:0/2021523888 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a80054f0 con 0x55deebdb3a10 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.207+0000 7fb0a2ffd640 1 -- 172.21.15.33:0/2021523888 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8004630 con 0x55deebdb3a10 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 -- 172.21.15.33:0/2021523888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb3a10 msgr2=0x55deebdb79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb3a10 0x55deebdb79f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8023040 tx=0x7fb0a8005850 comp rx=0 tx=0).stop 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 -- 172.21.15.33:0/2021523888 shutdown_connections 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdbf9a0 0x55deebdb7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdb3a10 0x55deebdb79f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 --2- 172.21.15.33:0/2021523888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 -- 172.21.15.33:0/2021523888 >> 172.21.15.33:0/2021523888 conn(0x55deebdb0f20 msgr2=0x55deebdd9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 -- 172.21.15.33:0/2021523888 shutdown_connections 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 -- 172.21.15.33:0/2021523888 wait complete. 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.208+0000 7fb0c1705740 1 -- start start 2024-09-10T21:17:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 1 Processor -- start 2024-09-10T21:17:07.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdd05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdbf9a0 0x55deebdcb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81ad620 con 0x55deebdbf9a0 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0bd8cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdbf9a0 0x55deebdcb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0bd0cc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0bd8cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdbf9a0 0x55deebdcb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51950/0 (socket says 172.21.15.33:51950) 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/3099026613 learned_addr learned my addr 172.21.15.33:0/3099026613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81acf20 con 0x55deebdb30f0 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0b81aa100 con 0x55deebdb3a10 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0bc8cb640 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdd05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.209+0000 7fb0c1705740 10 client.0 _ll_register_callbacks cb 0x7fb0b0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:07.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 msgr2=0x55deebdd05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdd05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/3099026613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 msgr2=0x55deebdd0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0bd8cd640 1 -- 172.21.15.33:0/3099026613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deebdd0d30 con 0x55deebdbf9a0 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0bd8cd640 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdbf9a0 0x55deebdcb010 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006010 tx=0x7fb0b01044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f6ff677c1b290c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/3099026613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00d65a0 con 0x55deebdbf9a0 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0a80054f0 con 0x55deebdbf9a0 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/3099026613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb0b0004b40 con 0x55deebdbf9a0 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.210+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/3099026613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) ==== 24+0+0 (secure 0 0 0) 0x7fb0b00ccdb0 con 0x55deebdbf9a0 2024-09-10T21:17:07.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 10 client.17859 fetch_fsmap learned FSMap version 64 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) -- 0x55deebe44e10 con 0x55deebdbf9a0 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/3099026613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0a80054f0 con 0x55deebdbf9a0 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/3099026613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 64) ==== 8390+0+0 (secure 0 0 0) 0x7fb0b00c8db0 con 0x55deebdbf9a0 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 10 client.17859 fetch_fsmap finished waiting for FSMap version 64 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 10 client.17859 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 20 client.17859 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebe453a0 0x55deebe47790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 4 client.17859 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55deebd278a0 con 0x55deebe453a0 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.211+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebe453a0 0x55deebe47790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.212+0000 7fb0a0ff9640 10 client.17859 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:07.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.212+0000 7fb0bd0cc640 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebe453a0 0x55deebe47790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aefc56b1b550a47d server_cookie=26e57c96ad17b233 in_seq=0 out_seq=0 2024-09-10T21:17:07.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.214+0000 7fb0a0ff9640 1 -- 172.21.15.33:0/3099026613 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55deebd278a0 con 0x55deebe453a0 2024-09-10T21:17:07.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.214+0000 7fb0a0ff9640 10 client.17859 handle_command_reply: tid=0 2024-09-10T21:17:07.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.214+0000 7fb0c1705740 1 client.17859 shutdown 2024-09-10T21:17:07.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.214+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdbf9a0 msgr2=0x55deebdcb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.214+0000 7fb0c1705740 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdbf9a0 0x55deebdcb010 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0006010 tx=0x7fb0b01044b0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.214+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 shutdown_connections 2024-09-10T21:17:07.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.214+0000 7fb0c1705740 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55deebe453a0 0x55deebe47790 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0c1705740 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55deebdbf9a0 0x55deebdcb010 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0c1705740 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deebdb3a10 0x55deebdd05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0c1705740 1 --2- 172.21.15.33:0/3099026613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55deebdb30f0 0x55deebdd0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 >> 172.21.15.33:0/3099026613 conn(0x55deebdb0f20 msgr2=0x55deebdb79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0a0ff9640 0 client.17859 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 shutdown_connections 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0c1705740 1 -- 172.21.15.33:0/3099026613 wait complete. 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.215+0000 7fb0c1705740 20 client.17859 trim_cache size 0 max 16384 2024-09-10T21:17:07.222 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0a40b53e0 msgr2=0x7fb0a40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0a40b53e0 0x7fb0a40b78a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80139d0 tx=0x7fb0a8009b70 comp rx=0 tx=0).stop 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b8109160 msgr2=0x7fb0b81a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b8109160 0x7fb0b81a0df0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb0b0004af0 tx=0x7fb0b0003c90 comp rx=0 tx=0).stop 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 shutdown_connections 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0a40b53e0 0x7fb0a40b78a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0b8109160 0x7fb0b81a0df0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b81087e0 0x7fb0b819ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 --2- 172.21.15.33:0/2300322345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0b8102dd0 0x7fb0b819b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:07.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 >> 172.21.15.33:0/2300322345 conn(0x7fb0b80fe800 msgr2=0x7fb0b81013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:07.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 shutdown_connections 2024-09-10T21:17:07.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:07.216+0000 7fb0bf330640 1 -- 172.21.15.33:0/2300322345 wait complete. 2024-09-10T21:17:07.946 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:17:08.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 7092 69.66 MB/sec warmup 8 sec latency 774.196 ms 2024-09-10T21:17:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:07 smithi033 ceph-mon[30860]: pgmap v740: 97 pgs: 97 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.1 MiB/s wr, 244 op/s 2024-09-10T21:17:08.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:07 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:08.278 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:08.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:07 smithi104 ceph-mon[35325]: pgmap v740: 97 pgs: 97 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.1 MiB/s wr, 244 op/s 2024-09-10T21:17:08.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:07 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:08.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:07 smithi174 ceph-mon[35635]: pgmap v740: 97 pgs: 97 active+clean; 379 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.1 MiB/s wr, 244 op/s 2024-09-10T21:17:08.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:07 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 -- 172.21.15.33:0/2440168397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8908103ca0 msgr2=0x7f8908104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 --2- 172.21.15.33:0/2440168397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8908103ca0 0x7f8908104080 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f8904009900 tx=0x7f890402a410 comp rx=0 tx=0).stop 2024-09-10T21:17:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 -- 172.21.15.33:0/2440168397 shutdown_connections 2024-09-10T21:17:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 --2- 172.21.15.33:0/2440168397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f890810d100 0x7f890810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 --2- 172.21.15.33:0/2440168397 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89081045c0 0x7f890810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 --2- 172.21.15.33:0/2440168397 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8908103ca0 0x7f8908104080 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 -- 172.21.15.33:0/2440168397 >> 172.21.15.33:0/2440168397 conn(0x7f89080fd700 msgr2=0x7f89080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 -- 172.21.15.33:0/2440168397 shutdown_connections 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.831+0000 7f890f80d640 1 -- 172.21.15.33:0/2440168397 wait complete. 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 Processor -- start 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 -- start start 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8908103ca0 0x7f8908199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89081045c0 0x7f89081997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f890810d100 0x7f890819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8908112dc0 con 0x7f890810d100 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8908112c40 con 0x7f8908103ca0 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.832+0000 7f890f80d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8908112f40 con 0x7f89081045c0 2024-09-10T21:17:08.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8908103ca0 0x7f8908199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8908103ca0 0x7f8908199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:36322/0 (socket says 172.21.15.33:36322) 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 -- 172.21.15.33:0/2725735568 learned_addr learned my addr 172.21.15.33:0/2725735568 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890ddaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f890810d100 0x7f890819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890cda8640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89081045c0 0x7f89081997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 -- 172.21.15.33:0/2725735568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89081045c0 msgr2=0x7f89081997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89081045c0 0x7f89081997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 -- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f890810d100 msgr2=0x7f890819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f890810d100 0x7f890819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 -- 172.21.15.33:0/2725735568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f890819f320 con 0x7f8908103ca0 2024-09-10T21:17:08.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890ddaa640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f890810d100 0x7f890819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:08.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.833+0000 7f890d5a9640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8908103ca0 0x7f8908199270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8904005ab0 tx=0x7f8904003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.834+0000 7f88fe7fc640 1 -- 172.21.15.33:0/2725735568 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890403f070 con 0x7f8908103ca0 2024-09-10T21:17:08.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.834+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f890819f5b0 con 0x7f8908103ca0 2024-09-10T21:17:08.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.834+0000 7f88fe7fc640 1 -- 172.21.15.33:0/2725735568 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8904031b60 con 0x7f8908103ca0 2024-09-10T21:17:08.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.834+0000 7f88fe7fc640 1 -- 172.21.15.33:0/2725735568 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904035030 con 0x7f8908103ca0 2024-09-10T21:17:08.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.834+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89081a5cd0 con 0x7f8908103ca0 2024-09-10T21:17:08.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.836+0000 7f88fe7fc640 1 -- 172.21.15.33:0/2725735568 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f890403b460 con 0x7f8908103ca0 2024-09-10T21:17:08.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.837+0000 7f8911be2740 1 Processor -- start 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.837+0000 7f8911be2740 1 -- start start 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.837+0000 7f8911be2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a0590f0 0x559a9a0594d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.837+0000 7f8911be2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a059a10 0x559a9a069870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f88fe7fc640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88f40b52e0 0x7f88f40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f8911be2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a06c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890ddaa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a0590f0 0x559a9a0594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890d5a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a059a10 0x559a9a069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890ddaa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a0590f0 0x559a9a0594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54868/0 (socket says 172.21.15.33:54868) 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890ddaa640 1 -- 172.21.15.33:0/1119403137 learned_addr learned my addr 172.21.15.33:0/1119403137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a99fcfea0 con 0x559a9a069db0 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a99fbf2d0 con 0x559a9a059a10 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a99fc30e0 con 0x559a9a0590f0 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890cda8640 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a06c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890ddaa640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88f40b52e0 0x7f88f40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f88fe7fc640 1 -- 172.21.15.33:0/2725735568 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8904106590 con 0x7f8908103ca0 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890cda8640 1 -- 172.21.15.33:0/1119403137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a0590f0 msgr2=0x559a9a0594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890cda8640 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a0590f0 0x559a9a0594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890cda8640 1 -- 172.21.15.33:0/1119403137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a059a10 msgr2=0x559a9a069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890cda8640 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a059a10 0x559a9a069870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890cda8640 1 -- 172.21.15.33:0/1119403137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a99d1d650 con 0x559a9a069db0 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.838+0000 7f890d5a9640 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a059a10 0x559a9a069870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.839+0000 7f890cda8640 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a06c1a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f890806b950 tx=0x7f88f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5d9b708d9ad700 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.839+0000 7f890ddaa640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88f40b52e0 0x7f88f40b77a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f89000099e0 tx=0x7f8900009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.839+0000 7f88f37fe640 1 -- 172.21.15.33:0/1119403137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8009d60 con 0x559a9a069db0 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.839+0000 7f88f37fe640 1 -- 172.21.15.33:0/1119403137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f8005020 con 0x559a9a069db0 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.839+0000 7f88f37fe640 1 -- 172.21.15.33:0/1119403137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8004b10 con 0x559a9a069db0 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.839+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 msgr2=0x559a9a06c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.839+0000 7f8911be2740 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a06c1a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f890806b950 tx=0x7f88f8011810 comp rx=0 tx=0).stop 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 shutdown_connections 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a06c1a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a059a10 0x559a9a069870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 --2- 172.21.15.33:0/1119403137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a0590f0 0x559a9a0594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 >> 172.21.15.33:0/1119403137 conn(0x559a9a056f20 msgr2=0x559a9a0580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 shutdown_connections 2024-09-10T21:17:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 -- 172.21.15.33:0/1119403137 wait complete. 2024-09-10T21:17:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 -- start start 2024-09-10T21:17:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.840+0000 7f8911be2740 1 Processor -- start 2024-09-10T21:17:08.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f8911be2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a0590f0 0x559a9a069620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f8911be2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f8911be2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a069db0 0x559a9a066ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f8911be2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559a99e20700 con 0x559a9a0590f0 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f8911be2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559a99f5a160 con 0x559a9a069db0 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f8911be2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a9a06ca60 con 0x559a9a059a10 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f8911be2740 10 client.0 _ll_register_callbacks cb 0x559a99f739e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f890d5a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a0590f0 0x559a9a069620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f890d5a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a0590f0 0x559a9a069620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51982/0 (socket says 172.21.15.33:51982) 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f890d5a9640 1 -- 172.21.15.33:0/1476088732 learned_addr learned my addr 172.21.15.33:0/1476088732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f890cda8640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.841+0000 7f890ddaa640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a069db0 0x559a9a066ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890d5a9640 1 -- 172.21.15.33:0/1476088732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 msgr2=0x559a9a066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890d5a9640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890d5a9640 1 -- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a069db0 msgr2=0x559a9a066ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890d5a9640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a069db0 0x559a9a066ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890d5a9640 1 -- 172.21.15.33:0/1476088732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a99f87c90 con 0x559a9a0590f0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890ddaa640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a069db0 0x559a9a066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890cda8640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f890d5a9640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a0590f0 0x559a9a069620 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f89040c9390 tx=0x7f8904095f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec2f10c65b3d6ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f88f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f88f0ff9640 1 -- 172.21.15.33:0/1476088732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904049040 con 0x559a9a0590f0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f88f0ff9640 1 -- 172.21.15.33:0/1476088732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f89040b2be0 con 0x559a9a0590f0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f8005020 con 0x559a9a0590f0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f88f0ff9640 1 -- 172.21.15.33:0/1476088732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f890409a0d0 con 0x559a9a0590f0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f8911be2740 10 client.17871 fetch_fsmap learned FSMap version 65 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.842+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559a9a078310 con 0x559a9a0590f0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f88f0ff9640 1 -- 172.21.15.33:0/1476088732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f8005020 con 0x559a9a0590f0 2024-09-10T21:17:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f88f0ff9640 1 -- 172.21.15.33:0/1476088732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 65) ==== 8394+0+0 (secure 0 0 0) 0x7f89040cd020 con 0x559a9a0590f0 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f8911be2740 10 client.17871 fetch_fsmap finished waiting for FSMap version 65 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f8911be2740 10 client.17871 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f8911be2740 20 client.17871 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f8911be2740 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a07bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f8911be2740 4 client.17871 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a99fd1df0 con 0x559a9a079750 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.843+0000 7f890cda8640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a07bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.844+0000 7f88f0ff9640 10 client.17871 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.844+0000 7f890cda8640 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a07bbf0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd9da4707add8ee server_cookie=8f9e0837e1e5670c in_seq=0 out_seq=0 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.844+0000 7f88f0ff9640 1 -- 172.21.15.33:0/1476088732 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559a99fd1df0 con 0x559a9a079750 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.844+0000 7f88f0ff9640 10 client.17871 handle_command_reply: tid=0 2024-09-10T21:17:08.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 client.17871 shutdown 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a0590f0 msgr2=0x559a9a069620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a0590f0 0x559a9a069620 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f89040c9390 tx=0x7f8904095f60 comp rx=0 tx=0).stop 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 shutdown_connections 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a07bbf0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a069db0 0x559a9a066ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f88f0ff9640 0 client.17871 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a066a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 --2- 172.21.15.33:0/1476088732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a0590f0 0x559a9a069620 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.845+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 >> 172.21.15.33:0/1476088732 conn(0x559a9a056f20 msgr2=0x559a9a05ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.846+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 shutdown_connections 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.846+0000 7f8911be2740 1 -- 172.21.15.33:0/1476088732 wait complete. 2024-09-10T21:17:08.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.846+0000 7f8911be2740 20 client.17871 trim_cache size 0 max 16384 2024-09-10T21:17:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 Processor -- start 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 -- start start 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a075c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a058680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a0814c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89081ac130 con 0x559a9a069db0 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89081addc0 con 0x559a9a0590f0 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.862+0000 7f8911be2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89081ac830 con 0x559a9a059a10 2024-09-10T21:17:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890ddaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a0814c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a058680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a058680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54880/0 (socket says 172.21.15.33:54880) 2024-09-10T21:17:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890ddaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a0814c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51992/0 (socket says 172.21.15.33:51992) 2024-09-10T21:17:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 -- 172.21.15.33:0/415540588 learned_addr learned my addr 172.21.15.33:0/415540588 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890d5a9640 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a075c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 -- 172.21.15.33:0/415540588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 msgr2=0x559a9a075c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a075c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 -- 172.21.15.33:0/415540588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 msgr2=0x559a9a0814c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a0814c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 -- 172.21.15.33:0/415540588 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9a067360 con 0x559a9a059a10 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.863+0000 7f890cda8640 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a058680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f890806b950 tx=0x7f88f8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf4fb8c118a0adb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.864+0000 7f88f2ffd640 1 -- 172.21.15.33:0/415540588 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8033070 con 0x559a9a059a10 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.864+0000 7f88f2ffd640 1 -- 172.21.15.33:0/415540588 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f8009790 con 0x559a9a059a10 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.864+0000 7f88f2ffd640 1 -- 172.21.15.33:0/415540588 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88f8004c00 con 0x559a9a059a10 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 -- 172.21.15.33:0/415540588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 msgr2=0x559a9a058680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a058680 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f890806b950 tx=0x7f88f8009310 comp rx=0 tx=0).stop 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 -- 172.21.15.33:0/415540588 shutdown_connections 2024-09-10T21:17:08.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a069db0 0x559a9a0814c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a059a10 0x559a9a058680 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 --2- 172.21.15.33:0/415540588 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a075c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 -- 172.21.15.33:0/415540588 >> 172.21.15.33:0/415540588 conn(0x559a9a056f20 msgr2=0x559a9a057f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 -- 172.21.15.33:0/415540588 shutdown_connections 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 -- 172.21.15.33:0/415540588 wait complete. 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 -- start start 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.865+0000 7f8911be2740 1 Processor -- start 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f8911be2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f8911be2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a059a10 0x559a9a072230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f8911be2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a069db0 0x559a9a06ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f8911be2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89081ad660 con 0x559a9a059a10 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f8911be2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89081acf30 con 0x559a9a0590f0 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f8911be2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89081aa0f0 con 0x559a9a069db0 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f8911be2740 10 client.0 _ll_register_callbacks cb 0x7f8904009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f890cda8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a059a10 0x559a9a072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f890d5a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f890cda8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a059a10 0x559a9a072230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52002/0 (socket says 172.21.15.33:52002) 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f890cda8640 1 -- 172.21.15.33:0/793802125 learned_addr learned my addr 172.21.15.33:0/793802125 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.866+0000 7f890ddaa640 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a069db0 0x559a9a06ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f890d5a9640 1 -- 172.21.15.33:0/793802125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a069db0 msgr2=0x559a9a06ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f890d5a9640 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a069db0 0x559a9a06ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f890d5a9640 1 -- 172.21.15.33:0/793802125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a059a10 msgr2=0x559a9a072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f890d5a9640 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a059a10 0x559a9a072230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f890d5a9640 1 -- 172.21.15.33:0/793802125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9a06f200 con 0x559a9a0590f0 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f890cda8640 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a059a10 0x559a9a072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f890d5a9640 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a071cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f890402a9d0 tx=0x7f89040d6040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a799d899ac51e1fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f88f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f88f0ff9640 1 -- 172.21.15.33:0/793802125 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8904049070 con 0x559a9a0590f0 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.867+0000 7f88f0ff9640 1 -- 172.21.15.33:0/793802125 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f89041060d0 con 0x559a9a0590f0 2024-09-10T21:17:08.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f88f8009790 con 0x559a9a0590f0 2024-09-10T21:17:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f88f0ff9640 1 -- 172.21.15.33:0/793802125 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f89040c8c60 con 0x559a9a0590f0 2024-09-10T21:17:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 10 client.27305 fetch_fsmap learned FSMap version 65 2024-09-10T21:17:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x559a9a0e6eb0 con 0x559a9a0590f0 2024-09-10T21:17:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f88f0ff9640 1 -- 172.21.15.33:0/793802125 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88f8009790 con 0x559a9a0590f0 2024-09-10T21:17:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f88f0ff9640 1 -- 172.21.15.33:0/793802125 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 65) ==== 8394+0+0 (secure 0 0 0) 0x7f89040cc020 con 0x559a9a0590f0 2024-09-10T21:17:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 10 client.27305 fetch_fsmap finished waiting for FSMap version 65 2024-09-10T21:17:08.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 10 client.27305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 20 client.27305 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a0e9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 4 client.27305 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.868+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a99fcd8a0 con 0x559a9a079750 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.869+0000 7f890cda8640 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a0e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.870+0000 7f88f0ff9640 10 client.27305 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.870+0000 7f890cda8640 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a0e9560 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=908889c825fb8b9c server_cookie=c98a1075e8588354 in_seq=0 out_seq=0 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.870+0000 7f88f0ff9640 1 -- 172.21.15.33:0/793802125 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a99fcd8a0 con 0x559a9a079750 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.870+0000 7f88f0ff9640 10 client.27305 handle_command_reply: tid=0 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.870+0000 7f8911be2740 1 client.27305 shutdown 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.870+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 msgr2=0x559a9a071cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a071cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f890402a9d0 tx=0x7f89040d6040 comp rx=0 tx=0).stop 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 shutdown_connections 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x559a9a079750 0x559a9a0e9560 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a9a069db0 0x559a9a06ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559a9a059a10 0x559a9a072230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 --2- 172.21.15.33:0/793802125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559a9a0590f0 0x559a9a071cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 >> 172.21.15.33:0/793802125 conn(0x559a9a056f20 msgr2=0x559a9a055750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f88f0ff9640 0 client.27305 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.871+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 shutdown_connections 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.872+0000 7f8911be2740 1 -- 172.21.15.33:0/793802125 wait complete. 2024-09-10T21:17:08.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.872+0000 7f8911be2740 20 client.27305 trim_cache size 0 max 16384 2024-09-10T21:17:08.881 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.873+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88f40b52e0 msgr2=0x7f88f40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.873+0000 7f890f80d640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88f40b52e0 0x7f88f40b77a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f89000099e0 tx=0x7f8900009500 comp rx=0 tx=0).stop 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.873+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8908103ca0 msgr2=0x7f8908199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.873+0000 7f890f80d640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8908103ca0 0x7f8908199270 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8904005ab0 tx=0x7f8904003c10 comp rx=0 tx=0).stop 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 shutdown_connections 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f88f40b52e0 0x7f88f40b77a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f890810d100 0x7f890819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89081045c0 0x7f89081997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 --2- 172.21.15.33:0/2725735568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8908103ca0 0x7f8908199270 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:08.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 >> 172.21.15.33:0/2725735568 conn(0x7f89080fd700 msgr2=0x7f890810f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:08.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 shutdown_connections 2024-09-10T21:17:08.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:08.874+0000 7f890f80d640 1 -- 172.21.15.33:0/2725735568 wait complete. 2024-09-10T21:17:09.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 7928 67.09 MB/sec warmup 9 sec latency 1774.230 ms 2024-09-10T21:17:09.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:08 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:08 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:09.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:08 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:10.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 9102 66.07 MB/sec warmup 10 sec latency 1813.058 ms 2024-09-10T21:17:10.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:09 smithi033 ceph-mon[30860]: pgmap v741: 97 pgs: 97 active+clean; 385 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 347 op/s 2024-09-10T21:17:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:09 smithi104 ceph-mon[35325]: pgmap v741: 97 pgs: 97 active+clean; 385 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 347 op/s 2024-09-10T21:17:10.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:09 smithi174 ceph-mon[35635]: pgmap v741: 97 pgs: 97 active+clean; 385 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 347 op/s 2024-09-10T21:17:10.787 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:17:10.787 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:17:11.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 10026 65.79 MB/sec warmup 11 sec latency 1887.345 ms 2024-09-10T21:17:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:11.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:10 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:11.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:10 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:10 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 -- 172.21.15.33:0/3144715417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d01045c0 msgr2=0x7f87d010cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/3144715417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d01045c0 0x7f87d010cab0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f87b8009900 tx=0x7f87b802a410 comp rx=0 tx=0).stop 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 -- 172.21.15.33:0/3144715417 shutdown_connections 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/3144715417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d010d100 0x7f87d010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/3144715417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d01045c0 0x7f87d010cab0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/3144715417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0103ca0 0x7f87d0104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 -- 172.21.15.33:0/3144715417 >> 172.21.15.33:0/3144715417 conn(0x7f87d00fd700 msgr2=0x7f87d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 -- 172.21.15.33:0/3144715417 shutdown_connections 2024-09-10T21:17:11.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.664+0000 7f87d4ca3640 1 -- 172.21.15.33:0/3144715417 wait complete. 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.665+0000 7f87d4ca3640 1 Processor -- start 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87d4ca3640 1 -- start start 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87d4ca3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0103ca0 0x7f87d01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87d4ca3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d01045c0 0x7f87d0199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87d4ca3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d010d100 0x7f87d019ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87d4ca3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0112dc0 con 0x7f87d010d100 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87d4ca3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0112c40 con 0x7f87d01045c0 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87d4ca3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0112f40 con 0x7f87d0103ca0 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87c7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d01045c0 0x7f87d0199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87c7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d01045c0 0x7f87d0199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49682/0 (socket says 172.21.15.33:49682) 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87c7fff640 1 -- 172.21.15.33:0/2074349605 learned_addr learned my addr 172.21.15.33:0/2074349605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:11.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87ce59c640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0103ca0 0x7f87d01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87ced9d640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d010d100 0x7f87d019ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.666+0000 7f87c7fff640 1 -- 172.21.15.33:0/2074349605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0103ca0 msgr2=0x7f87d01992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c7fff640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0103ca0 0x7f87d01992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c7fff640 1 -- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d010d100 msgr2=0x7f87d019ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c7fff640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d010d100 0x7f87d019ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c7fff640 1 -- 172.21.15.33:0/2074349605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d019f3a0 con 0x7f87d01045c0 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87ced9d640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d010d100 0x7f87d019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87ce59c640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0103ca0 0x7f87d01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:11.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c7fff640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d01045c0 0x7f87d0199830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f87b8005ab0 tx=0x7f87b8003ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c77fe640 1 -- 172.21.15.33:0/2074349605 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b803f070 con 0x7f87d01045c0 2024-09-10T21:17:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d019f630 con 0x7f87d01045c0 2024-09-10T21:17:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c77fe640 1 -- 172.21.15.33:0/2074349605 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b8031b50 con 0x7f87d01045c0 2024-09-10T21:17:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d0101830 con 0x7f87d01045c0 2024-09-10T21:17:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.667+0000 7f87c77fe640 1 -- 172.21.15.33:0/2074349605 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8035030 con 0x7f87d01045c0 2024-09-10T21:17:11.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.670+0000 7f87d7078740 1 Processor -- start 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87d7078740 1 -- start start 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87c77fe640 1 -- 172.21.15.33:0/2074349605 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f87b803bb70 con 0x7f87d01045c0 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87d7078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87d7078740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87d7078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57aa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87d7078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f9f5709e60 con 0x55f9f57a7f80 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87d7078740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f9f56f9290 con 0x55f9f57937e0 2024-09-10T21:17:11.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87d7078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9f56fd0a0 con 0x55f9f5792ec0 2024-09-10T21:17:11.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57aa370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41222/0 (socket says 172.21.15.33:41222) 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87c7fff640 1 -- 172.21.15.33:0/735495407 learned_addr learned my addr 172.21.15.33:0/735495407 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87ced9d640 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87c7fff640 1 -- 172.21.15.33:0/735495407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 msgr2=0x55f9f57932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87c7fff640 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57932a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.671+0000 7f87c7fff640 1 -- 172.21.15.33:0/735495407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 msgr2=0x55f9f57a7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.672+0000 7f87ce59c640 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.672+0000 7f87c77fe640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87bc0b5360 0x7f87bc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.672+0000 7f87c7fff640 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57a7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.672+0000 7f87c7fff640 1 -- 172.21.15.33:0/735495407 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f5457930 con 0x55f9f57a7f80 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.672+0000 7f87ced9d640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87bc0b5360 0x7f87bc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.672+0000 7f87c7fff640 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57aa370 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f87b80d8e90 tx=0x7f87b80c8a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817b7bde1e16b888 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.672+0000 7f87c77fe640 1 -- 172.21.15.33:0/2074349605 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87b8106350 con 0x7f87d01045c0 2024-09-10T21:17:11.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.673+0000 7f87c4ff9640 1 -- 172.21.15.33:0/735495407 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80cc510 con 0x55f9f57a7f80 2024-09-10T21:17:11.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.673+0000 7f87c4ff9640 1 -- 172.21.15.33:0/735495407 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b80d8720 con 0x55f9f57a7f80 2024-09-10T21:17:11.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.673+0000 7f87c4ff9640 1 -- 172.21.15.33:0/735495407 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80d8cf0 con 0x55f9f57a7f80 2024-09-10T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.673+0000 7f87d7078740 1 -- 172.21.15.33:0/735495407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 msgr2=0x55f9f57aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.673+0000 7f87d7078740 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57aa370 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f87b80d8e90 tx=0x7f87b80c8a60 comp rx=0 tx=0).stop 2024-09-10T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.673+0000 7f87ced9d640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87bc0b5360 0x7f87bc0b7820 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009fd0 tx=0x7f87c8009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 -- 172.21.15.33:0/735495407 shutdown_connections 2024-09-10T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57aa370 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57a7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 --2- 172.21.15.33:0/735495407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 -- 172.21.15.33:0/735495407 >> 172.21.15.33:0/735495407 conn(0x55f9f5790cf0 msgr2=0x55f9f5791e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 -- 172.21.15.33:0/735495407 shutdown_connections 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 -- 172.21.15.33:0/735495407 wait complete. 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 -- start start 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.674+0000 7f87d7078740 1 Processor -- start 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87d7078740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f5792ec0 0x55f9f57a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87d7078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f57937e0 0x55f9f57a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87d7078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87d7078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f9f555a890 con 0x55f9f57a7f80 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87d7078740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f9f5693d40 con 0x55f9f5792ec0 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87d7078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f9f57aa9c0 con 0x55f9f57937e0 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87d7078740 10 client.0 _ll_register_callbacks cb 0x55f9f56f7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f5792ec0 0x55f9f57a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f5792ec0 0x55f9f57a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49694/0 (socket says 172.21.15.33:49694) 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 -- 172.21.15.33:0/1413017848 learned_addr learned my addr 172.21.15.33:0/1413017848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ced9d640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87c7fff640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f57937e0 0x55f9f57a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 -- 172.21.15.33:0/1413017848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f57937e0 msgr2=0x55f9f57a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f57937e0 0x55f9f57a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 -- 172.21.15.33:0/1413017848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 msgr2=0x55f9f57a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 -- 172.21.15.33:0/1413017848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f57b3d40 con 0x55f9f5792ec0 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ced9d640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.675+0000 7f87ce59c640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f5792ec0 0x55f9f57a4f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009eb0 tx=0x7f87c000e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf4b3a606aa1a684 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87b5ffb640 1 -- 172.21.15.33:0/1413017848 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c0019420 con 0x55f9f5792ec0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9f570bdb0 con 0x55f9f5792ec0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87b5ffb640 1 -- 172.21.15.33:0/1413017848 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87c00195c0 con 0x55f9f5792ec0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87b5ffb640 1 -- 172.21.15.33:0/1413017848 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f87c00197c0 con 0x55f9f5792ec0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87d7078740 10 client.27314 fetch_fsmap learned FSMap version 66 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55f9f57b67a0 con 0x55f9f5792ec0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87b5ffb640 1 -- 172.21.15.33:0/1413017848 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87c0019e10 con 0x55f9f5792ec0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.676+0000 7f87b5ffb640 1 -- 172.21.15.33:0/1413017848 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 66) ==== 8398+0+0 (secure 0 0 0) 0x7f87c0010ce0 con 0x55f9f5792ec0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87d7078740 10 client.27314 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87d7078740 10 client.27314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87d7078740 20 client.27314 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87d7078740 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f57b7cf0 0x55f9f57ba170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87d7078740 4 client.27314 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9f570bdb0 con 0x55f9f57b7cf0 2024-09-10T21:17:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87c7fff640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f57b7cf0 0x55f9f57ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87c7fff640 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f57b7cf0 0x55f9f57ba170 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd99c1587c660359 server_cookie=59505c287064bf6b in_seq=0 out_seq=0 2024-09-10T21:17:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.677+0000 7f87b5ffb640 10 client.27314 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87b5ffb640 1 -- 172.21.15.33:0/1413017848 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f9f570bdb0 con 0x55f9f57b7cf0 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87b5ffb640 10 client.27314 handle_command_reply: tid=0 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 client.27314 shutdown 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f5792ec0 msgr2=0x55f9f57a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f5792ec0 0x55f9f57a4f70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f87c0009eb0 tx=0x7f87c000e720 comp rx=0 tx=0).stop 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 shutdown_connections 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f57b7cf0 0x55f9f57ba170 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f57937e0 0x55f9f57a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87b5ffb640 0 client.27314 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 --2- 172.21.15.33:0/1413017848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f5792ec0 0x55f9f57a4f70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 >> 172.21.15.33:0/1413017848 conn(0x55f9f5790cf0 msgr2=0x55f9f578ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 shutdown_connections 2024-09-10T21:17:11.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 1 -- 172.21.15.33:0/1413017848 wait complete. 2024-09-10T21:17:11.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.678+0000 7f87d7078740 20 client.27314 trim_cache size 0 max 16384 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 Processor -- start 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 -- start start 2024-09-10T21:17:11.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57937e0 0x55f9f57a5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01ac210 con 0x55f9f57937e0 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01aded0 con 0x55f9f57a7f80 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87d7078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01ac910 con 0x55f9f5792ec0 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.698+0000 7f87c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57937e0 0x55f9f57a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87ced9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57937e0 0x55f9f57a5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41234/0 (socket says 172.21.15.33:41234) 2024-09-10T21:17:11.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 -- 172.21.15.33:0/1561001522 learned_addr learned my addr 172.21.15.33:0/1561001522 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:11.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87ce59c640 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 -- 172.21.15.33:0/1561001522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 msgr2=0x55f9f57a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 -- 172.21.15.33:0/1561001522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57a7f80 msgr2=0x55f9f57a59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 -- 172.21.15.33:0/1561001522 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f57ba7f0 con 0x55f9f57937e0 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87ced9d640 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87c7fff640 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57937e0 0x55f9f57a5470 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f87b80c9e30 tx=0x7f87b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f99400ece14f5df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87b7fff640 1 -- 172.21.15.33:0/1561001522 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b8049040 con 0x55f9f57937e0 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87b7fff640 1 -- 172.21.15.33:0/1561001522 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87b803b460 con 0x55f9f57937e0 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.699+0000 7f87b7fff640 1 -- 172.21.15.33:0/1561001522 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87b80cc330 con 0x55f9f57937e0 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 -- 172.21.15.33:0/1561001522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57937e0 msgr2=0x55f9f57a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57937e0 0x55f9f57a5470 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f87b80c9e30 tx=0x7f87b8009260 comp rx=0 tx=0).stop 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 -- 172.21.15.33:0/1561001522 shutdown_connections 2024-09-10T21:17:11.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57a7f80 0x55f9f57a59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57937e0 0x55f9f57a5470 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 --2- 172.21.15.33:0/1561001522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 -- 172.21.15.33:0/1561001522 >> 172.21.15.33:0/1561001522 conn(0x55f9f5790cf0 msgr2=0x55f9f57a37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 -- 172.21.15.33:0/1561001522 shutdown_connections 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.700+0000 7f87d7078740 1 -- 172.21.15.33:0/1561001522 wait complete. 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 -- start start 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 Processor -- start 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57b0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57bea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01ad770 con 0x55f9f57a7f80 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01ad040 con 0x55f9f57937e0 2024-09-10T21:17:11.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87d01aa1d0 con 0x55f9f5792ec0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87d7078740 10 client.0 _ll_register_callbacks cb 0x7f87c0009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87c7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87ced9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57bea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87ced9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57bea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41248/0 (socket says 172.21.15.33:41248) 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87ced9d640 1 -- 172.21.15.33:0/3883284614 learned_addr learned my addr 172.21.15.33:0/3883284614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87ce59c640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.701+0000 7f87c7fff640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49710/0 (socket says 172.21.15.33:49710) 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87ced9d640 1 -- 172.21.15.33:0/3883284614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 msgr2=0x55f9f57afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87ced9d640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87ced9d640 1 -- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 msgr2=0x55f9f57b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87ced9d640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87ced9d640 1 -- 172.21.15.33:0/3883284614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f57bef70 con 0x55f9f57a7f80 2024-09-10T21:17:11.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87c7fff640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87ced9d640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57bea30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f87c800fdc0 tx=0x7f87c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91410ba1c5c3a35e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87b5ffb640 1 -- 172.21.15.33:0/3883284614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c8004b20 con 0x55f9f57a7f80 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87b5ffb640 1 -- 172.21.15.33:0/3883284614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f87c8018030 con 0x55f9f57a7f80 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9f5707860 con 0x55f9f57a7f80 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87b5ffb640 1 -- 172.21.15.33:0/3883284614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f87c8004cc0 con 0x55f9f57a7f80 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87d7078740 10 client.17895 fetch_fsmap learned FSMap version 66 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.702+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x55f9f57c02b0 con 0x55f9f57a7f80 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87b5ffb640 1 -- 172.21.15.33:0/3883284614 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f9f5707860 con 0x55f9f57a7f80 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87b5ffb640 1 -- 172.21.15.33:0/3883284614 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 66) ==== 8398+0+0 (secure 0 0 0) 0x7f87c8033020 con 0x55f9f57a7f80 2024-09-10T21:17:11.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87d7078740 10 client.17895 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87d7078740 10 client.17895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87d7078740 20 client.17895 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87d7078740 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f5825d90 0x55f9f58281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87d7078740 4 client.17895 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f9f5704c90 con 0x55f9f5825d90 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.703+0000 7f87ce59c640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f5825d90 0x55f9f58281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.704+0000 7f87ce59c640 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f5825d90 0x55f9f58281b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4367a2ff849ae83 server_cookie=8e89ca59b664321 in_seq=0 out_seq=0 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.704+0000 7f87b5ffb640 10 client.17895 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87b5ffb640 1 -- 172.21.15.33:0/3883284614 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9f5704c90 con 0x55f9f5825d90 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87b5ffb640 10 client.17895 handle_command_reply: tid=0 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 client.17895 shutdown 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 msgr2=0x55f9f57bea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57bea30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f87c800fdc0 tx=0x7f87c8004a40 comp rx=0 tx=0).stop 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 shutdown_connections 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f9f5825d90 0x55f9f58281b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f9f57a7f80 0x55f9f57bea30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f9f57937e0 0x55f9f57b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 --2- 172.21.15.33:0/3883284614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f9f5792ec0 0x55f9f57afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87b5ffb640 0 client.17895 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 >> 172.21.15.33:0/3883284614 conn(0x55f9f5790cf0 msgr2=0x55f9f57a65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 shutdown_connections 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:17:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:17:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "df90b8ee-36ca-4b4b-b236-67440e7f292e", 2024-09-10T21:17:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:17:11.714 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:17:11.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.705+0000 7f87d7078740 1 -- 172.21.15.33:0/3883284614 wait complete. 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d7078740 20 client.17895 trim_cache size 0 max 16384 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87bc0b5360 msgr2=0x7f87bc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87bc0b5360 0x7f87bc0b7820 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009fd0 tx=0x7f87c8009500 comp rx=0 tx=0).stop 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d01045c0 msgr2=0x7f87d0199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d01045c0 0x7f87d0199830 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f87b8005ab0 tx=0x7f87b8003ae0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 shutdown_connections 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f87bc0b5360 0x7f87bc0b7820 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87d010d100 0x7f87d019ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87d01045c0 0x7f87d0199830 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 --2- 172.21.15.33:0/2074349605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87d0103ca0 0x7f87d01992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 >> 172.21.15.33:0/2074349605 conn(0x7f87d00fd700 msgr2=0x7f87d00fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 shutdown_connections 2024-09-10T21:17:11.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:11.706+0000 7f87d4ca3640 1 -- 172.21.15.33:0/2074349605 wait complete. 2024-09-10T21:17:12.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 10443 61.30 MB/sec warmup 12 sec latency 396.401 ms 2024-09-10T21:17:12.266 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:11 smithi033 ceph-mon[30860]: pgmap v742: 97 pgs: 97 active+clean; 392 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T21:17:12.266 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:11 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:17:12.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:11 smithi104 ceph-mon[35325]: pgmap v742: 97 pgs: 97 active+clean; 392 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T21:17:12.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:11 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:17:12.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:11 smithi174 ceph-mon[35635]: pgmap v742: 97 pgs: 97 active+clean; 392 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 410 op/s 2024-09-10T21:17:12.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:11 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:17:12.429 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:17:12.749 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:13.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 11067 58.77 MB/sec warmup 13 sec latency 435.619 ms 2024-09-10T21:17:13.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:17:13.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:17:13.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:17:13.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:17:13.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:17:13.174 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:17:13.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: pgmap v743: 97 pgs: 97 active+clean; 406 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 484 op/s 2024-09-10T21:17:13.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:13 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:17:13.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: pgmap v743: 97 pgs: 97 active+clean; 406 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 484 op/s 2024-09-10T21:17:13.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:13 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:17:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:17:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:17:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:17:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:17:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:17:13.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: pgmap v743: 97 pgs: 97 active+clean; 406 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 484 op/s 2024-09-10T21:17:13.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:13 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:13.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 -- 172.21.15.33:0/2081937430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 msgr2=0x7f2374107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 --2- 172.21.15.33:0/2081937430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 0x7f2374107b10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f2360009900 tx=0x7f236002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 -- 172.21.15.33:0/2081937430 shutdown_connections 2024-09-10T21:17:13.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 --2- 172.21.15.33:0/2081937430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2374108050 0x7f237410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 --2- 172.21.15.33:0/2081937430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 0x7f2374107b10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 --2- 172.21.15.33:0/2081937430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 0x7f23741020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 -- 172.21.15.33:0/2081937430 >> 172.21.15.33:0/2081937430 conn(0x7f23740fd770 msgr2=0x7f23740ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 -- 172.21.15.33:0/2081937430 shutdown_connections 2024-09-10T21:17:13.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.553+0000 7f237af0d640 1 -- 172.21.15.33:0/2081937430 wait complete. 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.554+0000 7f237af0d640 1 Processor -- start 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.554+0000 7f237af0d640 1 -- start start 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f237af0d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 0x7f237419a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f237af0d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 0x7f237419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f237af0d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2374108050 0x7f23741a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f237af0d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2374111d10 con 0x7f23741076d0 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f237af0d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2374111b90 con 0x7f2374108050 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f237af0d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2374111e90 con 0x7f2374101cc0 2024-09-10T21:17:13.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2373fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 0x7f237419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 0x7f237419a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 0x7f237419a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:57882/0 (socket says 172.21.15.33:57882) 2024-09-10T21:17:13.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 -- 172.21.15.33:0/2209752552 learned_addr learned my addr 172.21.15.33:0/2209752552 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 -- 172.21.15.33:0/2209752552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2374108050 msgr2=0x7f23741a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2374108050 0x7f23741a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 -- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 msgr2=0x7f237419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 0x7f237419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.555+0000 7f2378ca9640 1 -- 172.21.15.33:0/2209752552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2374197ef0 con 0x7f2374101cc0 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.556+0000 7f2378ca9640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 0x7f237419a100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f236800d950 tx=0x7f236800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.556+0000 7f2371ffb640 1 -- 172.21.15.33:0/2209752552 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368017070 con 0x7f2374101cc0 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.557+0000 7f2371ffb640 1 -- 172.21.15.33:0/2209752552 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2368009c30 con 0x7f2374101cc0 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.557+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23741981e0 con 0x7f2374101cc0 2024-09-10T21:17:13.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.557+0000 7f2371ffb640 1 -- 172.21.15.33:0/2209752552 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236800f620 con 0x7f2374101cc0 2024-09-10T21:17:13.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.557+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2374198700 con 0x7f2374101cc0 2024-09-10T21:17:13.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.561+0000 7f2371ffb640 1 -- 172.21.15.33:0/2209752552 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2368010460 con 0x7f2374101cc0 2024-09-10T21:17:13.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.561+0000 7f237d2e2740 1 Processor -- start 2024-09-10T21:17:13.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.561+0000 7f237d2e2740 1 -- start start 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8480 0x5565ba9f8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565baa047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa07120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f237d2e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565ba96ee50 con 0x5565ba9f8480 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f237d2e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5565ba95e280 con 0x5565ba9f8da0 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f237d2e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565ba962090 con 0x5565baa04d30 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f2371ffb640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23640b5360 0x7f23640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565baa047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565baa047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49728/0 (socket says 172.21.15.33:49728) 2024-09-10T21:17:13.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f2378ca9640 1 -- 172.21.15.33:0/3472807606 learned_addr learned my addr 172.21.15.33:0/3472807606 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:13.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f2373fff640 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa07120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f23794aa640 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8480 0x5565ba9f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.562+0000 7f23794aa640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23640b5360 0x7f23640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f2371ffb640 1 -- 172.21.15.33:0/2209752552 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f23680e57c0 con 0x7f2374101cc0 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f2378ca9640 1 -- 172.21.15.33:0/3472807606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 msgr2=0x5565baa07120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa07120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f2378ca9640 1 -- 172.21.15.33:0/3472807606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8480 msgr2=0x5565ba9f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8480 0x5565ba9f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f2378ca9640 1 -- 172.21.15.33:0/3472807606 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565ba6bc650 con 0x5565ba9f8da0 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565baa047f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f23680a91f0 tx=0x7f2368010cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6ae4a355e5d4a7b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f235effd640 1 -- 172.21.15.33:0/3472807606 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368020070 con 0x5565ba9f8da0 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f235effd640 1 -- 172.21.15.33:0/3472807606 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23680aba90 con 0x5565ba9f8da0 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.563+0000 7f235effd640 1 -- 172.21.15.33:0/3472807606 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23680ab440 con 0x5565ba9f8da0 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f23794aa640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23640b5360 0x7f23640b7820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f236c008fd0 tx=0x7f236c006ce0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 -- 172.21.15.33:0/3472807606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 msgr2=0x5565baa047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565baa047f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f23680a91f0 tx=0x7f2368010cb0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 -- 172.21.15.33:0/3472807606 shutdown_connections 2024-09-10T21:17:13.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa07120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565baa047f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3472807606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8480 0x5565ba9f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 -- 172.21.15.33:0/3472807606 >> 172.21.15.33:0/3472807606 conn(0x5565ba9f62b0 msgr2=0x5565ba9f7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.564+0000 7f237d2e2740 1 -- 172.21.15.33:0/3472807606 shutdown_connections 2024-09-10T21:17:13.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.565+0000 7f237d2e2740 1 -- 172.21.15.33:0/3472807606 wait complete. 2024-09-10T21:17:13.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.565+0000 7f237d2e2740 1 -- start start 2024-09-10T21:17:13.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.565+0000 7f237d2e2740 1 Processor -- start 2024-09-10T21:17:13.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8480 0x5565baa04650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa01aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f237d2e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5565ba7bf700 con 0x5565ba9f8da0 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8480 0x5565baa04650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f237d2e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5565ba8f9110 con 0x5565ba9f8480 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f237d2e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565baa07950 con 0x5565baa04d30 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8480 0x5565baa04650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49738/0 (socket says 172.21.15.33:49738) 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 -- 172.21.15.33:0/3739485139 learned_addr learned my addr 172.21.15.33:0/3739485139 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f237d2e2740 10 client.0 _ll_register_callbacks cb 0x5565ba912990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2373fff640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa01aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f23794aa640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 -- 172.21.15.33:0/3739485139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 msgr2=0x5565baa020d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa020d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 -- 172.21.15.33:0/3739485139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 msgr2=0x5565baa01aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa01aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2378ca9640 1 -- 172.21.15.33:0/3739485139 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ba926c40 con 0x5565ba9f8480 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f23794aa640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.566+0000 7f2373fff640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa01aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8480 0x5565baa04650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23680abf10 tx=0x7f23680109b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c13393cf3d8efd80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f2353fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f2353fff640 1 -- 172.21.15.33:0/3739485139 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368020070 con 0x5565ba9f8480 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f2353fff640 1 -- 172.21.15.33:0/3739485139 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f23680930e0 con 0x5565ba9f8480 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23680aba90 con 0x5565ba9f8480 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f2353fff640 1 -- 172.21.15.33:0/3739485139 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2368009de0 con 0x5565ba9f8480 2024-09-10T21:17:13.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f237d2e2740 10 client.27326 fetch_fsmap learned FSMap version 66 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5565baa1baa0 con 0x5565ba9f8480 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.567+0000 7f2353fff640 1 -- 172.21.15.33:0/3739485139 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23680b5510 con 0x5565ba9f8480 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.568+0000 7f2353fff640 1 -- 172.21.15.33:0/3739485139 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 66) ==== 8398+0+0 (secure 0 0 0) 0x7f23680aac50 con 0x5565ba9f8480 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.568+0000 7f237d2e2740 10 client.27326 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.568+0000 7f237d2e2740 10 client.27326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.568+0000 7f237d2e2740 20 client.27326 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.568+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa1cdf0 0x5565baa1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.568+0000 7f237d2e2740 4 client.27326 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.568+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f23680aba90 con 0x5565baa1cdf0 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.569+0000 7f2373fff640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa1cdf0 0x5565baa1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.569+0000 7f2373fff640 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa1cdf0 0x5565baa1f1e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4915d815cde4c847 server_cookie=d7d18ec39ea4bce0 in_seq=0 out_seq=0 2024-09-10T21:17:13.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.569+0000 7f2353fff640 10 client.27326 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f2353fff640 1 -- 172.21.15.33:0/3739485139 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f23680aba90 con 0x5565baa1cdf0 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f2353fff640 10 client.27326 handle_command_reply: tid=0 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 client.27326 shutdown 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8480 msgr2=0x5565baa04650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8480 0x5565baa04650 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f23680abf10 tx=0x7f23680109b0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 shutdown_connections 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa1cdf0 0x5565baa1f1e0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565baa04d30 0x5565baa020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa01aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3739485139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8480 0x5565baa04650 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f2353fff640 0 client.27326 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:13.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.570+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 >> 172.21.15.33:0/3739485139 conn(0x5565ba9f62b0 msgr2=0x5565ba9fbe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.571+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 shutdown_connections 2024-09-10T21:17:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.571+0000 7f237d2e2740 1 -- 172.21.15.33:0/3739485139 wait complete. 2024-09-10T21:17:13.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.571+0000 7f237d2e2740 20 client.27326 trim_cache size 0 max 16384 2024-09-10T21:17:13.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.598+0000 7f237d2e2740 1 Processor -- start 2024-09-10T21:17:13.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.598+0000 7f237d2e2740 1 -- start start 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa00f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565ba9f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565baa04d30 0x5565ba9f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f237d2e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23741ac2d0 con 0x5565baa04d30 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f237d2e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23741adf90 con 0x5565ba9f8da0 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f237d2e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23741ac9d0 con 0x5565ba9f8480 2024-09-10T21:17:13.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f23794aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565baa04d30 0x5565ba9f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa00f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f2373fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565ba9f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa00f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:57912/0 (socket says 172.21.15.33:57912) 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f2373fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565ba9f5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49752/0 (socket says 172.21.15.33:49752) 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.599+0000 7f2378ca9640 1 -- 172.21.15.33:0/591013970 learned_addr learned my addr 172.21.15.33:0/591013970 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f2373fff640 1 -- 172.21.15.33:0/591013970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 msgr2=0x5565baa00f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f2373fff640 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa00f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f2373fff640 1 -- 172.21.15.33:0/591013970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565baa04d30 msgr2=0x5565ba9f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f2373fff640 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565baa04d30 0x5565ba9f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f2373fff640 1 -- 172.21.15.33:0/591013970 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565baa00a00 con 0x5565ba9f8da0 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f2373fff640 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565ba9f5bc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2360009230 tx=0x7f2360005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6be82ec7010f1174 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f235e7fc640 1 -- 172.21.15.33:0/591013970 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360049020 con 0x5565ba9f8da0 2024-09-10T21:17:13.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f235e7fc640 1 -- 172.21.15.33:0/591013970 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2360032900 con 0x5565ba9f8da0 2024-09-10T21:17:13.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f235e7fc640 1 -- 172.21.15.33:0/591013970 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360035040 con 0x5565ba9f8da0 2024-09-10T21:17:13.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.600+0000 7f23794aa640 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565baa04d30 0x5565ba9f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:13.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 -- 172.21.15.33:0/591013970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 msgr2=0x5565ba9f5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565ba9f5bc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2360009230 tx=0x7f2360005e60 comp rx=0 tx=0).stop 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 -- 172.21.15.33:0/591013970 shutdown_connections 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565baa04d30 0x5565ba9f76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565ba9f8da0 0x5565ba9f5bc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 --2- 172.21.15.33:0/591013970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa00f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 -- 172.21.15.33:0/591013970 >> 172.21.15.33:0/591013970 conn(0x5565ba9f62b0 msgr2=0x5565ba9f6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 -- 172.21.15.33:0/591013970 shutdown_connections 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.601+0000 7f237d2e2740 1 -- 172.21.15.33:0/591013970 wait complete. 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 -- start start 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 Processor -- start 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565baa04d30 0x5565baa1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23741ad830 con 0x5565ba9f8da0 2024-09-10T21:17:13.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23741ad100 con 0x5565baa04d30 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.602+0000 7f237d2e2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23741aa290 con 0x5565ba9f8480 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f237d2e2740 10 client.0 _ll_register_callbacks cb 0x7f236c00bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa224a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:57924/0 (socket says 172.21.15.33:57924) 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 -- 172.21.15.33:0/3104378947 learned_addr learned my addr 172.21.15.33:0/3104378947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2373fff640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f23794aa640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565baa04d30 0x5565baa1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 -- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565baa04d30 msgr2=0x5565baa1f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565baa04d30 0x5565baa1f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 -- 172.21.15.33:0/3104378947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 msgr2=0x5565baa229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 -- 172.21.15.33:0/3104378947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565baa02440 con 0x5565ba9f8480 2024-09-10T21:17:13.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f23794aa640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565baa04d30 0x5565baa1f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2378ca9640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa224a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23680ab170 tx=0x7f23680ac680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8b57fa2a4c9c6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.603+0000 7f2353fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.604+0000 7f2353fff640 1 -- 172.21.15.33:0/3104378947 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2368014020 con 0x5565ba9f8480 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.604+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2360032900 con 0x5565ba9f8480 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.604+0000 7f2353fff640 1 -- 172.21.15.33:0/3104378947 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f23680932b0 con 0x5565ba9f8480 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.604+0000 7f2353fff640 1 -- 172.21.15.33:0/3104378947 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) ==== 24+0+0 (secure 0 0 0) 0x7f2368020070 con 0x5565ba9f8480 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.604+0000 7f237d2e2740 10 client.27268 fetch_fsmap learned FSMap version 66 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.604+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) -- 0x5565baa8a290 con 0x5565ba9f8480 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.604+0000 7f2353fff640 1 -- 172.21.15.33:0/3104378947 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23680aba90 con 0x5565ba9f8480 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.605+0000 7f2353fff640 1 -- 172.21.15.33:0/3104378947 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 66) ==== 8398+0+0 (secure 0 0 0) 0x7f23680971c0 con 0x5565ba9f8480 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.605+0000 7f237d2e2740 10 client.27268 fetch_fsmap finished waiting for FSMap version 66 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.605+0000 7f237d2e2740 10 client.27268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:13.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.605+0000 7f237d2e2740 20 client.27268 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:13.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.605+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa8a8f0 0x5565baa8cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:13.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.605+0000 7f237d2e2740 4 client.27268 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:17:13.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.605+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f2360032900 con 0x5565baa8a8f0 2024-09-10T21:17:13.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.606+0000 7f2373fff640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa8a8f0 0x5565baa8cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:13.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.606+0000 7f2373fff640 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa8a8f0 0x5565baa8cd10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e035936e39f4f56 server_cookie=e0f71a0ef9c6ac95 in_seq=0 out_seq=0 2024-09-10T21:17:13.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.606+0000 7f2353fff640 10 client.27268 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:13.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f2353fff640 1 -- 172.21.15.33:0/3104378947 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x7f2360032900 con 0x5565baa8a8f0 2024-09-10T21:17:13.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f2353fff640 10 client.27268 handle_command_reply: tid=0 2024-09-10T21:17:13.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 client.27268 shutdown 2024-09-10T21:17:13.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 msgr2=0x5565baa224a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa224a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f23680ab170 tx=0x7f23680ac680 comp rx=0 tx=0).stop 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 shutdown_connections 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5565baa8a8f0 0x5565baa8cd10 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f2353fff640 0 client.27268 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5565baa04d30 0x5565baa1f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5565ba9f8da0 0x5565baa229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 --2- 172.21.15.33:0/3104378947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565ba9f8480 0x5565baa224a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.608+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 >> 172.21.15.33:0/3104378947 conn(0x5565ba9f62b0 msgr2=0x5565baa038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.609+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 shutdown_connections 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.609+0000 7f237d2e2740 1 -- 172.21.15.33:0/3104378947 wait complete. 2024-09-10T21:17:13.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.609+0000 7f237d2e2740 20 client.27268 trim_cache size 0 max 16384 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "df90b8ee-36ca-4b4b-b236-67440e7f292e", 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:17:13.616 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:17:13.618 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:17:13.622 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:17:13.623 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:13.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:17:13.627 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:17:13.627 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23640b5360 msgr2=0x7f23640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23640b5360 0x7f23640b7820 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f236c008fd0 tx=0x7f236c006ce0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 msgr2=0x7f237419a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 0x7f237419a100 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f236800d950 tx=0x7f236800de10 comp rx=0 tx=0).stop 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 shutdown_connections 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23640b5360 0x7f23640b7820 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2374108050 0x7f23741a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23741076d0 0x7f237419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 --2- 172.21.15.33:0/2209752552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2374101cc0 0x7f237419a100 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 >> 172.21.15.33:0/2209752552 conn(0x7f23740fd770 msgr2=0x7f23740ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 shutdown_connections 2024-09-10T21:17:13.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:13.610+0000 7f237af0d640 1 -- 172.21.15.33:0/2209752552 wait complete. 2024-09-10T21:17:14.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 12325 59.63 MB/sec warmup 14 sec latency 419.226 ms 2024-09-10T21:17:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:14 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:14.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:14.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:14 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:14.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:14.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:14 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:17:15.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 13370 60.39 MB/sec warmup 15 sec latency 1419.284 ms 2024-09-10T21:17:15.177 INFO:tasks.cephfs.filesystem:scrub status for tag:df90b8ee-36ca-4b4b-b236-67440e7f292e - {'path': '/', 'tag': 'df90b8ee-36ca-4b4b-b236-67440e7f292e', 'options': 'recursive,force'} 2024-09-10T21:17:15.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:15 smithi104 ceph-mon[35325]: pgmap v744: 97 pgs: 97 active+clean; 451 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 575 op/s 2024-09-10T21:17:15.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:15 smithi174 ceph-mon[35635]: pgmap v744: 97 pgs: 97 active+clean; 451 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 575 op/s 2024-09-10T21:17:15.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:15 smithi033 ceph-mon[30860]: pgmap v744: 97 pgs: 97 active+clean; 451 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 10 MiB/s wr, 575 op/s 2024-09-10T21:17:16.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 14191 58.71 MB/sec warmup 16 sec latency 2389.574 ms 2024-09-10T21:17:17.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 15414 58.62 MB/sec warmup 17 sec latency 39.003 ms 2024-09-10T21:17:17.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:17 smithi174 ceph-mon[35635]: pgmap v745: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-09-10T21:17:17.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:17 smithi033 ceph-mon[30860]: pgmap v745: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-09-10T21:17:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:17 smithi104 ceph-mon[35325]: pgmap v745: 97 pgs: 97 active+clean; 471 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 11 MiB/s wr, 588 op/s 2024-09-10T21:17:18.049 INFO:tasks.workunit.client.0.smithi033.stdout: 10 16842 60.32 MB/sec warmup 18 sec latency 34.854 ms 2024-09-10T21:17:19.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 17633 59.64 MB/sec warmup 19 sec latency 76.001 ms 2024-09-10T21:17:19.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:19 smithi174 ceph-mon[35635]: pgmap v746: 97 pgs: 97 active+clean; 481 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-09-10T21:17:19.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:19 smithi033 ceph-mon[30860]: pgmap v746: 97 pgs: 97 active+clean; 481 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-09-10T21:17:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:19 smithi104 ceph-mon[35325]: pgmap v746: 97 pgs: 97 active+clean; 481 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 12 MiB/s wr, 676 op/s 2024-09-10T21:17:20.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 18724 58.98 MB/sec warmup 20 sec latency 532.641 ms 2024-09-10T21:17:21.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 19673 58.89 MB/sec warmup 21 sec latency 1493.287 ms 2024-09-10T21:17:21.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:21 smithi174 ceph-mon[35635]: pgmap v747: 97 pgs: 97 active+clean; 493 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-09-10T21:17:21.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:17:21.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:17:21.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:17:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:21 smithi033 ceph-mon[30860]: pgmap v747: 97 pgs: 97 active+clean; 493 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-09-10T21:17:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:17:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:17:21.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:17:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:21 smithi104 ceph-mon[35325]: pgmap v747: 97 pgs: 97 active+clean; 493 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-09-10T21:17:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:17:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:17:21.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:17:22.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 20654 57.88 MB/sec warmup 22 sec latency 162.225 ms 2024-09-10T21:17:22.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:22 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:17:22.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:22 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:17:22.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:17:22.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:17:22.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:17:22.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:17:22.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:22 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:17:22.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:22 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:17:22.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:17:22.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:17:22.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:17:22.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:17:22.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:22 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:17:22.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:22 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:17:22.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:17:22.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:17:22.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:17:22.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:17:23.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 21939 58.66 MB/sec warmup 23 sec latency 32.930 ms 2024-09-10T21:17:23.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:23 smithi174 ceph-mon[35635]: pgmap v748: 97 pgs: 97 active+clean; 496 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-09-10T21:17:23.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:23 smithi033 ceph-mon[30860]: pgmap v748: 97 pgs: 97 active+clean; 496 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-09-10T21:17:23.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:23 smithi104 ceph-mon[35325]: pgmap v748: 97 pgs: 97 active+clean; 496 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-09-10T21:17:24.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 22668 58.34 MB/sec warmup 24 sec latency 335.008 ms 2024-09-10T21:17:25.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 23475 58.00 MB/sec warmup 25 sec latency 64.534 ms 2024-09-10T21:17:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:25 smithi174 ceph-mon[35635]: pgmap v749: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 795 op/s 2024-09-10T21:17:25.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:25 smithi033 ceph-mon[30860]: pgmap v749: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 795 op/s 2024-09-10T21:17:25.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:25 smithi104 ceph-mon[35325]: pgmap v749: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 15 MiB/s wr, 795 op/s 2024-09-10T21:17:26.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 24121 56.50 MB/sec warmup 26 sec latency 648.285 ms 2024-09-10T21:17:27.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 25458 56.85 MB/sec warmup 27 sec latency 134.880 ms 2024-09-10T21:17:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:27 smithi174 ceph-mon[35635]: pgmap v750: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 683 op/s 2024-09-10T21:17:27.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:27 smithi033 ceph-mon[30860]: pgmap v750: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 683 op/s 2024-09-10T21:17:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:27 smithi104 ceph-mon[35325]: pgmap v750: 97 pgs: 97 active+clean; 582 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 14 MiB/s wr, 683 op/s 2024-09-10T21:17:28.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 26610 57.45 MB/sec warmup 28 sec latency 86.635 ms 2024-09-10T21:17:28.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:28 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:28.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:28 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:28 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:29.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 27451 57.13 MB/sec warmup 29 sec latency 39.257 ms 2024-09-10T21:17:29.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:29 smithi033 ceph-mon[30860]: pgmap v751: 97 pgs: 97 active+clean; 662 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 856 op/s 2024-09-10T21:17:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:29 smithi104 ceph-mon[35325]: pgmap v751: 97 pgs: 97 active+clean; 662 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 856 op/s 2024-09-10T21:17:29.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:29 smithi174 ceph-mon[35635]: pgmap v751: 97 pgs: 97 active+clean; 662 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 21 MiB/s wr, 856 op/s 2024-09-10T21:17:30.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 28413 56.58 MB/sec warmup 30 sec latency 72.032 ms 2024-09-10T21:17:31.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 29709 57.32 MB/sec warmup 31 sec latency 60.618 ms 2024-09-10T21:17:31.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:31 smithi033 ceph-mon[30860]: pgmap v752: 97 pgs: 97 active+clean; 662 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 21 MiB/s wr, 808 op/s 2024-09-10T21:17:31.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:31 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:31.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:31 smithi174 ceph-mon[35635]: pgmap v752: 97 pgs: 97 active+clean; 662 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 21 MiB/s wr, 808 op/s 2024-09-10T21:17:31.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:31 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:31 smithi104 ceph-mon[35325]: pgmap v752: 97 pgs: 97 active+clean; 662 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 21 MiB/s wr, 808 op/s 2024-09-10T21:17:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:31 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:32.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 30528 56.79 MB/sec warmup 32 sec latency 71.867 ms 2024-09-10T21:17:33.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 31645 56.69 MB/sec warmup 33 sec latency 69.644 ms 2024-09-10T21:17:33.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:33 smithi033 ceph-mon[30860]: pgmap v753: 97 pgs: 97 active+clean; 677 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 860 op/s 2024-09-10T21:17:33.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:33 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:33 smithi104 ceph-mon[35325]: pgmap v753: 97 pgs: 97 active+clean; 677 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 860 op/s 2024-09-10T21:17:33.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:33 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:33.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:33 smithi174 ceph-mon[35635]: pgmap v753: 97 pgs: 97 active+clean; 677 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 860 op/s 2024-09-10T21:17:33.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:33 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:17:34.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 32708 57.08 MB/sec warmup 34 sec latency 48.994 ms 2024-09-10T21:17:35.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 33552 56.59 MB/sec warmup 35 sec latency 44.062 ms 2024-09-10T21:17:35.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:35 smithi033 ceph-mon[30860]: pgmap v754: 97 pgs: 97 active+clean; 686 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 941 op/s 2024-09-10T21:17:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:35 smithi104 ceph-mon[35325]: pgmap v754: 97 pgs: 97 active+clean; 686 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 941 op/s 2024-09-10T21:17:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:35 smithi174 ceph-mon[35635]: pgmap v754: 97 pgs: 97 active+clean; 686 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 22 MiB/s wr, 941 op/s 2024-09-10T21:17:36.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 34690 56.54 MB/sec warmup 36 sec latency 67.848 ms 2024-09-10T21:17:37.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 35703 56.95 MB/sec warmup 37 sec latency 69.855 ms 2024-09-10T21:17:37.680 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:37 smithi033 ceph-mon[30860]: pgmap v755: 97 pgs: 97 active+clean; 690 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-09-10T21:17:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:37 smithi104 ceph-mon[35325]: pgmap v755: 97 pgs: 97 active+clean; 690 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-09-10T21:17:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:37 smithi174 ceph-mon[35635]: pgmap v755: 97 pgs: 97 active+clean; 690 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 779 op/s 2024-09-10T21:17:38.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 36410 56.26 MB/sec warmup 38 sec latency 96.560 ms 2024-09-10T21:17:39.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 37395 55.87 MB/sec warmup 39 sec latency 168.064 ms 2024-09-10T21:17:39.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:39 smithi033 ceph-mon[30860]: pgmap v756: 97 pgs: 97 active+clean; 703 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 894 op/s 2024-09-10T21:17:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:39 smithi104 ceph-mon[35325]: pgmap v756: 97 pgs: 97 active+clean; 703 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 894 op/s 2024-09-10T21:17:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:39 smithi174 ceph-mon[35635]: pgmap v756: 97 pgs: 97 active+clean; 703 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 14 MiB/s wr, 894 op/s 2024-09-10T21:17:40.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 38466 56.12 MB/sec warmup 40 sec latency 47.480 ms 2024-09-10T21:17:41.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 39515 56.35 MB/sec warmup 41 sec latency 52.704 ms 2024-09-10T21:17:41.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:41 smithi033 ceph-mon[30860]: pgmap v757: 97 pgs: 97 active+clean; 709 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-09-10T21:17:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:41 smithi104 ceph-mon[35325]: pgmap v757: 97 pgs: 97 active+clean; 709 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-09-10T21:17:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:41 smithi174 ceph-mon[35635]: pgmap v757: 97 pgs: 97 active+clean; 709 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.1 MiB/s wr, 721 op/s 2024-09-10T21:17:42.050 INFO:tasks.workunit.client.0.smithi033.stdout: 10 40616 56.47 MB/sec warmup 42 sec latency 88.506 ms 2024-09-10T21:17:43.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 41358 55.90 MB/sec warmup 43 sec latency 69.019 ms 2024-09-10T21:17:43.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:43 smithi033 ceph-mon[30860]: pgmap v758: 97 pgs: 97 active+clean; 720 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.2 MiB/s wr, 821 op/s 2024-09-10T21:17:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:43 smithi104 ceph-mon[35325]: pgmap v758: 97 pgs: 97 active+clean; 720 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.2 MiB/s wr, 821 op/s 2024-09-10T21:17:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:43 smithi174 ceph-mon[35635]: pgmap v758: 97 pgs: 97 active+clean; 720 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.2 MiB/s wr, 821 op/s 2024-09-10T21:17:44.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 42522 55.64 MB/sec warmup 44 sec latency 83.495 ms 2024-09-10T21:17:45.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 42550 54.43 MB/sec warmup 45 sec latency 944.037 ms 2024-09-10T21:17:45.178 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:17:45.530 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:45.660 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:45 smithi033 ceph-mon[30860]: pgmap v759: 97 pgs: 97 active+clean; 735 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.1 MiB/s wr, 755 op/s 2024-09-10T21:17:45.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:45 smithi104 ceph-mon[35325]: pgmap v759: 97 pgs: 97 active+clean; 735 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.1 MiB/s wr, 755 op/s 2024-09-10T21:17:45.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:45 smithi174 ceph-mon[35635]: pgmap v759: 97 pgs: 97 active+clean; 735 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.1 MiB/s wr, 755 op/s 2024-09-10T21:17:46.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 43337 54.53 MB/sec warmup 46 sec latency 832.453 ms 2024-09-10T21:17:46.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.113+0000 7f09beffd640 1 -- 172.21.15.33:0/1046059332 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4035020 con 0x7f09c010bb80 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.113+0000 7f09c66f4640 1 -- 172.21.15.33:0/1046059332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010bb80 msgr2=0x7f09c01142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.114+0000 7f09c66f4640 1 --2- 172.21.15.33:0/1046059332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010bb80 0x7f09c01142d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402a480 comp rx=0 tx=0).stop 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.114+0000 7f09c66f4640 1 -- 172.21.15.33:0/1046059332 shutdown_connections 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.114+0000 7f09c66f4640 1 --2- 172.21.15.33:0/1046059332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c0114810 0x7f09c0116c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.114+0000 7f09c66f4640 1 --2- 172.21.15.33:0/1046059332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010bb80 0x7f09c01142d0 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f09b4009900 tx=0x7f09b402a480 comp rx=0 tx=0).stop 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.114+0000 7f09c66f4640 1 --2- 172.21.15.33:0/1046059332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09c010b260 0x7f09c010b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.114+0000 7f09c66f4640 1 -- 172.21.15.33:0/1046059332 >> 172.21.15.33:0/1046059332 conn(0x7f09c006e2e0 msgr2=0x7f09c006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.114+0000 7f09c66f4640 1 -- 172.21.15.33:0/1046059332 shutdown_connections 2024-09-10T21:17:46.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.115+0000 7f09c66f4640 1 -- 172.21.15.33:0/1046059332 wait complete. 2024-09-10T21:17:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.116+0000 7f09c66f4640 1 Processor -- start 2024-09-10T21:17:46.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.116+0000 7f09c66f4640 1 -- start start 2024-09-10T21:17:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.117+0000 7f09c66f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010b260 0x7f09c0110740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.117+0000 7f09c66f4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09c0114810 0x7f09c0110c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09c66f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c01111c0 0x7f09c01b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09bffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010b260 0x7f09c0110740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09bffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010b260 0x7f09c0110740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54968/0 (socket says 172.21.15.33:54968) 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09bf7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09c0114810 0x7f09c0110c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09bffff640 1 -- 172.21.15.33:0/606139551 learned_addr learned my addr 172.21.15.33:0/606139551 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0119380 con 0x7f09c01111c0 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0119200 con 0x7f09c0114810 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09c0119500 con 0x7f09c010b260 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.118+0000 7f09c4c91640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c01111c0 0x7f09c01b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bffff640 1 -- 172.21.15.33:0/606139551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09c0114810 msgr2=0x7f09c0110c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bffff640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09c0114810 0x7f09c0110c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bffff640 1 -- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c01111c0 msgr2=0x7f09c01b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bffff640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c01111c0 0x7f09c01b40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bffff640 1 -- 172.21.15.33:0/606139551 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c01b4790 con 0x7f09c010b260 2024-09-10T21:17:46.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bffff640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010b260 0x7f09c0110740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09b800cce0 tx=0x7f09b800a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bd7fa640 1 -- 172.21.15.33:0/606139551 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b800f650 con 0x7f09c010b260 2024-09-10T21:17:46.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c01b4a80 con 0x7f09c010b260 2024-09-10T21:17:46.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.119+0000 7f09bd7fa640 1 -- 172.21.15.33:0/606139551 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b800f7f0 con 0x7f09c010b260 2024-09-10T21:17:46.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.120+0000 7f09bd7fa640 1 -- 172.21.15.33:0/606139551 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b8005280 con 0x7f09c010b260 2024-09-10T21:17:46.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.120+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01b4f80 con 0x7f09c010b260 2024-09-10T21:17:46.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 Processor -- start 2024-09-10T21:17:46.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 -- start start 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af3b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af3b9c0 0x55f51af47410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af47950 0x55f51af49d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f51aeb1e50 con 0x55f51af3b9c0 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f51aea1280 con 0x55f51af47950 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.124+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f51aea5090 con 0x55f51af3b0a0 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.125+0000 7f09c4c91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af3b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.125+0000 7f09c4c91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af3b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54982/0 (socket says 172.21.15.33:54982) 2024-09-10T21:17:46.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.125+0000 7f09c4c91640 1 -- 172.21.15.33:0/681314834 learned_addr learned my addr 172.21.15.33:0/681314834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.125+0000 7f09bffff640 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af3b9c0 0x55f51af47410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af47950 0x55f51af49d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09bd7fa640 1 -- 172.21.15.33:0/606139551 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f09b800fbc0 con 0x7f09c010b260 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09c4c91640 1 -- 172.21.15.33:0/681314834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af47950 msgr2=0x55f51af49d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09c4c91640 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af47950 0x55f51af49d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09c4c91640 1 -- 172.21.15.33:0/681314834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af3b9c0 msgr2=0x55f51af47410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09c4c91640 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af3b9c0 0x55f51af47410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09c4c91640 1 -- 172.21.15.33:0/681314834 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51abff650 con 0x55f51af3b0a0 2024-09-10T21:17:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09bffff640 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af3b9c0 0x55f51af47410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af47950 0x55f51af49d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09c4c91640 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af3b480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009c00 tx=0x7f09b00117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94e829fdd89ed73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.126+0000 7f09a67fc640 1 -- 172.21.15.33:0/681314834 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0012450 con 0x55f51af3b0a0 2024-09-10T21:17:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.127+0000 7f09a67fc640 1 -- 172.21.15.33:0/681314834 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b0012aa0 con 0x55f51af3b0a0 2024-09-10T21:17:46.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.127+0000 7f09a67fc640 1 -- 172.21.15.33:0/681314834 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b000faf0 con 0x55f51af3b0a0 2024-09-10T21:17:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09bd7fa640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09a80b5360 0x7f09a80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09c8ac9740 1 -- 172.21.15.33:0/681314834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 msgr2=0x55f51af3b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af3b480 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009c00 tx=0x7f09b00117b0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09c8ac9740 1 -- 172.21.15.33:0/681314834 shutdown_connections 2024-09-10T21:17:46.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af47950 0x55f51af49d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af3b9c0 0x55f51af47410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/681314834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af3b480 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.128+0000 7f09c8ac9740 1 -- 172.21.15.33:0/681314834 >> 172.21.15.33:0/681314834 conn(0x55f51af38ed0 msgr2=0x55f51af3a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.129+0000 7f09c8ac9740 1 -- 172.21.15.33:0/681314834 shutdown_connections 2024-09-10T21:17:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.129+0000 7f09bffff640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09a80b5360 0x7f09a80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.129+0000 7f09bd7fa640 1 -- 172.21.15.33:0/606139551 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f09b80e6120 con 0x7f09c010b260 2024-09-10T21:17:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.129+0000 7f09c8ac9740 1 -- 172.21.15.33:0/681314834 wait complete. 2024-09-10T21:17:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.130+0000 7f09c8ac9740 1 -- start start 2024-09-10T21:17:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.130+0000 7f09c8ac9740 1 Processor -- start 2024-09-10T21:17:46.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09bffff640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09a80b5360 0x7f09a80b7820 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f09b80a8e70 tx=0x7f09b80a8040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b0a0 0x55f51af471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b9c0 0x55f51af44640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af44c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55f51ad02700 con 0x55f51af47950 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55f51ae3c110 con 0x55f51af3b0a0 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f51af4a620 con 0x55f51af3b9c0 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c8ac9740 10 client.0 _ll_register_callbacks cb 0x55f51ae55990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.131+0000 7f09c4c91640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b0a0 0x55f51af471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09bf7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b9c0 0x55f51af44640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af44c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af44c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36014/0 (socket says 172.21.15.33:36014) 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 -- 172.21.15.33:0/321295631 learned_addr learned my addr 172.21.15.33:0/321295631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 -- 172.21.15.33:0/321295631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b9c0 msgr2=0x55f51af44640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b9c0 0x55f51af44640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 -- 172.21.15.33:0/321295631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 msgr2=0x55f51af44c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af44c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.132+0000 7f09c4c91640 1 -- 172.21.15.33:0/321295631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51ae69c40 con 0x55f51af3b0a0 2024-09-10T21:17:46.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.133+0000 7f09c4c91640 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b0a0 0x55f51af471f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009fa0 tx=0x7f09b0014040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f668d6d0e0dca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.134+0000 7f099f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.134+0000 7f099f7fe640 1 -- 172.21.15.33:0/321295631 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0012760 con 0x55f51af3b0a0 2024-09-10T21:17:46.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.134+0000 7f099f7fe640 1 -- 172.21.15.33:0/321295631 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f09b000f890 con 0x55f51af3b0a0 2024-09-10T21:17:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.135+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b0012aa0 con 0x55f51af3b0a0 2024-09-10T21:17:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.135+0000 7f099f7fe640 1 -- 172.21.15.33:0/321295631 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f09b0012900 con 0x55f51af3b0a0 2024-09-10T21:17:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.135+0000 7f09c8ac9740 10 client.27341 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:46.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.135+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55f51af55f60 con 0x55f51af3b0a0 2024-09-10T21:17:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f099f7fe640 1 -- 172.21.15.33:0/321295631 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b002e070 con 0x55f51af3b0a0 2024-09-10T21:17:46.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f099f7fe640 1 -- 172.21.15.33:0/321295631 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f09b0026040 con 0x55f51af3b0a0 2024-09-10T21:17:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f09c8ac9740 10 client.27341 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f09c8ac9740 10 client.27341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f09c8ac9740 20 client.27341 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51af59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f09c8ac9740 4 client.27341 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.136+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f09b0012aa0 con 0x55f51af57360 2024-09-10T21:17:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.137+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51af59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.138+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51af59750 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a6d215e7b9f5d93 server_cookie=c542af8ea5200585 in_seq=0 out_seq=0 2024-09-10T21:17:46.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.138+0000 7f099f7fe640 10 client.27341 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.139+0000 7f099f7fe640 1 -- 172.21.15.33:0/321295631 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f09b0012aa0 con 0x55f51af57360 2024-09-10T21:17:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.139+0000 7f099f7fe640 10 client.27341 handle_command_reply: tid=0 2024-09-10T21:17:46.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.139+0000 7f09c8ac9740 1 client.27341 shutdown 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b0a0 msgr2=0x55f51af471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b0a0 0x55f51af471f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009fa0 tx=0x7f09b0014040 comp rx=0 tx=0).stop 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 shutdown_connections 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51af59750 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af44c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f099f7fe640 0 client.27341 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b9c0 0x55f51af44640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.140+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/321295631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b0a0 0x55f51af471f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.141+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 >> 172.21.15.33:0/321295631 conn(0x55f51af38ed0 msgr2=0x55f51af3ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.141+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 shutdown_connections 2024-09-10T21:17:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.142+0000 7f09c8ac9740 1 -- 172.21.15.33:0/321295631 wait complete. 2024-09-10T21:17:46.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.142+0000 7f09c8ac9740 20 client.27341 trim_cache size 0 max 16384 2024-09-10T21:17:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.177+0000 7f09c8ac9740 1 Processor -- start 2024-09-10T21:17:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.177+0000 7f09c8ac9740 1 -- start start 2024-09-10T21:17:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af42ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af43400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af39de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01bd8b0 con 0x55f51af47950 2024-09-10T21:17:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01bf600 con 0x55f51af3b9c0 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09c8ac9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01bdfb0 con 0x55f51af3b0a0 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09bf7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af43400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09bf7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af43400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58124/0 (socket says 172.21.15.33:58124) 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09bf7fe640 1 -- 172.21.15.33:0/2680015731 learned_addr learned my addr 172.21.15.33:0/2680015731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.178+0000 7f09c4c91640 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af42ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09c4c91640 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af39de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09bf7fe640 1 -- 172.21.15.33:0/2680015731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 msgr2=0x55f51af42ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af42ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09bf7fe640 1 -- 172.21.15.33:0/2680015731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 msgr2=0x55f51af39de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af39de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09bf7fe640 1 -- 172.21.15.33:0/2680015731 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f51af3b7f0 con 0x55f51af3b9c0 2024-09-10T21:17:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af43400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005ab0 tx=0x7f09b4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85b041c9569f33c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.179+0000 7f09a5ffb640 1 -- 172.21.15.33:0/2680015731 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4035020 con 0x55f51af3b9c0 2024-09-10T21:17:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.180+0000 7f09a5ffb640 1 -- 172.21.15.33:0/2680015731 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09b40316d0 con 0x55f51af3b9c0 2024-09-10T21:17:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.180+0000 7f09a5ffb640 1 -- 172.21.15.33:0/2680015731 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4004920 con 0x55f51af3b9c0 2024-09-10T21:17:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.180+0000 7f09c8ac9740 1 -- 172.21.15.33:0/2680015731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 msgr2=0x55f51af43400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.181+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af43400 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005ab0 tx=0x7f09b4005fd0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.182+0000 7f09c8ac9740 1 -- 172.21.15.33:0/2680015731 shutdown_connections 2024-09-10T21:17:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.182+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af39de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.182+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af43400 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.182+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/2680015731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af42ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.182+0000 7f09c8ac9740 1 -- 172.21.15.33:0/2680015731 >> 172.21.15.33:0/2680015731 conn(0x55f51af38ed0 msgr2=0x55f51af443f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.183+0000 7f09c8ac9740 1 -- 172.21.15.33:0/2680015731 shutdown_connections 2024-09-10T21:17:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.183+0000 7f09c8ac9740 1 -- 172.21.15.33:0/2680015731 wait complete. 2024-09-10T21:17:46.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.183+0000 7f09c8ac9740 1 -- start start 2024-09-10T21:17:46.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.184+0000 7f09c8ac9740 1 Processor -- start 2024-09-10T21:17:46.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09c8ac9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af4f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09c4c91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af4f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09c4c91640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af4f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55022/0 (socket says 172.21.15.33:55022) 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09c4c91640 1 -- 172.21.15.33:0/3214041242 learned_addr learned my addr 172.21.15.33:0/3214041242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af4fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af4a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af4fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.185+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01beea0 con 0x55f51af47950 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01be710 con 0x55f51af3b9c0 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09c01bb870 con 0x55f51af3b0a0 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c4c91640 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af4a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c8ac9740 10 client.0 _ll_register_callbacks cb 0x7f09b80e9a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c4c91640 1 -- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 msgr2=0x55f51af4fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c4c91640 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af4fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c4c91640 1 -- 172.21.15.33:0/3214041242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 msgr2=0x55f51af4a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c4c91640 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af4a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c4c91640 1 -- 172.21.15.33:0/3214041242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f51af50410 con 0x55f51af3b0a0 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.186+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b40316d0 con 0x55f51af3b0a0 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.187+0000 7f09c4c91640 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af4f910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f09b0012fd0 tx=0x7f09b0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43567e1b39899079 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.187+0000 7f099f7fe640 10 client.27283 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:17:46.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.188+0000 7f099f7fe640 1 -- 172.21.15.33:0/3214041242 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0012760 con 0x55f51af3b0a0 2024-09-10T21:17:46.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.188+0000 7f099f7fe640 1 -- 172.21.15.33:0/3214041242 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f09b0005230 con 0x55f51af3b0a0 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.188+0000 7f099f7fe640 1 -- 172.21.15.33:0/3214041242 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f09b0005470 con 0x55f51af3b0a0 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.189+0000 7f09c8ac9740 10 client.27283 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.189+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=69}) -- 0x55f51afc43f0 con 0x55f51af3b0a0 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f099f7fe640 1 -- 172.21.15.33:0/3214041242 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f09b003b040 con 0x55f51af3b0a0 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f09c8ac9740 10 client.27283 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f09c8ac9740 10 client.27283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f09c8ac9740 20 client.27283 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:46.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51afc6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f09c8ac9740 4 client.27283 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:17:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f51aeb3da0 con 0x55f51af57360 2024-09-10T21:17:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.190+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51afc6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.191+0000 7f09bf7fe640 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51afc6a90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fcc43ea97cdf9bc server_cookie=eafb81671feedff1 in_seq=0 out_seq=0 2024-09-10T21:17:46.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.191+0000 7f099f7fe640 10 client.27283 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:46.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.192+0000 7f099f7fe640 1 -- 172.21.15.33:0/3214041242 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x55f51aeb3da0 con 0x55f51af57360 2024-09-10T21:17:46.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.192+0000 7f099f7fe640 10 client.27283 handle_command_reply: tid=0 2024-09-10T21:17:46.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.192+0000 7f09c8ac9740 1 client.27283 shutdown 2024-09-10T21:17:46.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 msgr2=0x55f51af4f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af4f910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f09b0012fd0 tx=0x7f09b0005f90 comp rx=0 tx=0).stop 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 shutdown_connections 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55f51af57360 0x55f51afc6a90 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55f51af47950 0x55f51af4a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55f51af3b9c0 0x55f51af4fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 --2- 172.21.15.33:0/3214041242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f51af3b0a0 0x55f51af4f910 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.193+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 >> 172.21.15.33:0/3214041242 conn(0x55f51af38ed0 msgr2=0x55f51af42e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.195+0000 7f099f7fe640 0 client.27283 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.195+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 shutdown_connections 2024-09-10T21:17:46.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.195+0000 7f09c8ac9740 1 -- 172.21.15.33:0/3214041242 wait complete. 2024-09-10T21:17:46.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.196+0000 7f09c8ac9740 20 client.27283 trim_cache size 0 max 16384 2024-09-10T21:17:46.203 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:17:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:17:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:17:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:17:46.203 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.204 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:17:46.205 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.207 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.208 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:17:46.209 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.210 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.211 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:17:46.212 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.213 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:17:46.214 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:17:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09a80b5360 msgr2=0x7f09a80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09a80b5360 0x7f09a80b7820 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f09b80a8e70 tx=0x7f09b80a8040 comp rx=0 tx=0).stop 2024-09-10T21:17:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010b260 msgr2=0x7f09c0110740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010b260 0x7f09c0110740 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f09b800cce0 tx=0x7f09b800a620 comp rx=0 tx=0).stop 2024-09-10T21:17:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 shutdown_connections 2024-09-10T21:17:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09a80b5360 0x7f09a80b7820 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f09c01111c0 0x7f09c01b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f09c0114810 0x7f09c0110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 --2- 172.21.15.33:0/606139551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c010b260 0x7f09c0110740 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.200+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 >> 172.21.15.33:0/606139551 conn(0x7f09c006e2e0 msgr2=0x7f09c0072930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.201+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 shutdown_connections 2024-09-10T21:17:46.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:46.201+0000 7f09c66f4640 1 -- 172.21.15.33:0/606139551 wait complete. 2024-09-10T21:17:47.017 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:17:47.017 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:17:47.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 43893 54.17 MB/sec warmup 47 sec latency 171.339 ms 2024-09-10T21:17:47.358 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:47.792 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:47 smithi033 ceph-mon[30860]: pgmap v760: 97 pgs: 97 active+clean; 769 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 753 op/s 2024-09-10T21:17:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:47 smithi104 ceph-mon[35325]: pgmap v760: 97 pgs: 97 active+clean; 769 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 753 op/s 2024-09-10T21:17:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:47 smithi174 ceph-mon[35635]: pgmap v760: 97 pgs: 97 active+clean; 769 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.0 MiB/s wr, 753 op/s 2024-09-10T21:17:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.883+0000 7f4f327f2640 1 -- 172.21.15.33:0/2141964117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 msgr2=0x7f4f2c107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.883+0000 7f4f327f2640 1 --2- 172.21.15.33:0/2141964117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 0x7f4f2c107b10 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f4f20009900 tx=0x7f4f2002a410 comp rx=0 tx=0).stop 2024-09-10T21:17:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.885+0000 7f4f327f2640 1 -- 172.21.15.33:0/2141964117 shutdown_connections 2024-09-10T21:17:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.885+0000 7f4f327f2640 1 --2- 172.21.15.33:0/2141964117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f2c108050 0x7f4f2c10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.885+0000 7f4f327f2640 1 --2- 172.21.15.33:0/2141964117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 0x7f4f2c107b10 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.885+0000 7f4f327f2640 1 --2- 172.21.15.33:0/2141964117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f2c101cc0 0x7f4f2c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.885+0000 7f4f327f2640 1 -- 172.21.15.33:0/2141964117 >> 172.21.15.33:0/2141964117 conn(0x7f4f2c0fd750 msgr2=0x7f4f2c0ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.885+0000 7f4f327f2640 1 -- 172.21.15.33:0/2141964117 shutdown_connections 2024-09-10T21:17:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.885+0000 7f4f327f2640 1 -- 172.21.15.33:0/2141964117 wait complete. 2024-09-10T21:17:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.886+0000 7f4f327f2640 1 Processor -- start 2024-09-10T21:17:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.886+0000 7f4f327f2640 1 -- start start 2024-09-10T21:17:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f327f2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f2c101cc0 0x7f4f2c19b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f327f2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 0x7f4f2c19ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f327f2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f2c108050 0x7f4f2c1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f2c101cc0 0x7f4f2c19b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f327f2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c111d60 con 0x7f4f2c1076d0 2024-09-10T21:17:47.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f2bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f2c101cc0 0x7f4f2c19b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58148/0 (socket says 172.21.15.33:58148) 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f327f2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c111be0 con 0x7f4f2c101cc0 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2c111ee0 con 0x7f4f2c108050 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.887+0000 7f4f2bfff640 1 -- 172.21.15.33:0/3053517783 learned_addr learned my addr 172.21.15.33:0/3053517783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.888+0000 7f4f30d8f640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f2c108050 0x7f4f2c1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.888+0000 7f4f2b7fe640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 0x7f4f2c19ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.888+0000 7f4f30d8f640 1 -- 172.21.15.33:0/3053517783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f2c101cc0 msgr2=0x7f4f2c19b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.888+0000 7f4f30d8f640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f2c101cc0 0x7f4f2c19b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.888+0000 7f4f30d8f640 1 -- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 msgr2=0x7f4f2c19ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.888+0000 7f4f30d8f640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 0x7f4f2c19ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.888+0000 7f4f30d8f640 1 -- 172.21.15.33:0/3053517783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2c1a15e0 con 0x7f4f2c108050 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.889+0000 7f4f30d8f640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f2c108050 0x7f4f2c1a0e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400be20 tx=0x7f4f2400bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.889+0000 7f4f297fa640 1 -- 172.21.15.33:0/3053517783 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24019070 con 0x7f4f2c108050 2024-09-10T21:17:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.889+0000 7f4f297fa640 1 -- 172.21.15.33:0/3053517783 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f240148f0 con 0x7f4f2c108050 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.889+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2c1a18d0 con 0x7f4f2c108050 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.889+0000 7f4f297fa640 1 -- 172.21.15.33:0/3053517783 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24012840 con 0x7f4f2c108050 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.889+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f2c1a7f90 con 0x7f4f2c108050 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.891+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2c1020a0 con 0x7f4f2c108050 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.893+0000 7f4f297fa640 1 -- 172.21.15.33:0/3053517783 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4f24014440 con 0x7f4f2c108050 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.894+0000 7f4f297fa640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4f140b5360 0x7f4f140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.895+0000 7f4f297fa640 1 -- 172.21.15.33:0/3053517783 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4f240e5a40 con 0x7f4f2c108050 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.895+0000 7f4f2bfff640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4f140b5360 0x7f4f140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.895+0000 7f4f2bfff640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4f140b5360 0x7f4f140b7820 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c005fd0 tx=0x7f4f1c00b480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:47.904+0000 7f4f297fa640 1 -- 172.21.15.33:0/3053517783 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f4f240ac270 con 0x7f4f2c108050 2024-09-10T21:17:48.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 44458 53.65 MB/sec warmup 48 sec latency 264.136 ms 2024-09-10T21:17:48.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.172+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f2c19c660 con 0x7f4f2c108050 2024-09-10T21:17:48.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.173+0000 7f4f297fa640 1 -- 172.21.15.33:0/3053517783 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11998 (secure 0 0 0) 0x7f4f2c19c660 con 0x7f4f2c108050 2024-09-10T21:17:48.179 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:17:48.179 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":69,"btime":"2024-09-10T21:17:32:403122+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:17:31.774879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:17:48.180 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 69 2024-09-10T21:17:48.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4f140b5360 msgr2=0x7f4f140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:48.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4f140b5360 0x7f4f140b7820 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c005fd0 tx=0x7f4f1c00b480 comp rx=0 tx=0).stop 2024-09-10T21:17:48.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f2c108050 msgr2=0x7f4f2c1a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:48.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f2c108050 0x7f4f2c1a0e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400be20 tx=0x7f4f2400bf10 comp rx=0 tx=0).stop 2024-09-10T21:17:48.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 shutdown_connections 2024-09-10T21:17:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4f140b5360 0x7f4f140b7820 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f2c108050 0x7f4f2c1a0e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4f2c1076d0 0x7f4f2c19ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 --2- 172.21.15.33:0/3053517783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f2c101cc0 0x7f4f2c19b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 >> 172.21.15.33:0/3053517783 conn(0x7f4f2c0fd750 msgr2=0x7f4f2c104cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 shutdown_connections 2024-09-10T21:17:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:48.176+0000 7f4f327f2640 1 -- 172.21.15.33:0/3053517783 wait complete. 2024-09-10T21:17:48.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:48 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3053517783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:17:48.864 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:48 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3053517783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:17:48.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:48 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3053517783' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:17:48.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:17:49.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 45198 53.11 MB/sec warmup 49 sec latency 178.285 ms 2024-09-10T21:17:49.248 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:49.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:49 smithi033 ceph-mon[30860]: pgmap v761: 97 pgs: 97 active+clean; 774 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-10T21:17:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:49 smithi104 ceph-mon[35325]: pgmap v761: 97 pgs: 97 active+clean; 774 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-10T21:17:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.838+0000 7fbea2ce6640 1 -- 172.21.15.33:0/3168566306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c1076c0 msgr2=0x7fbe9c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.838+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/3168566306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c1076c0 0x7fbe9c107b00 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009860 tx=0x7fbe8802a340 comp rx=0 tx=0).stop 2024-09-10T21:17:49.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.838+0000 7fbea2ce6640 1 -- 172.21.15.33:0/3168566306 shutdown_connections 2024-09-10T21:17:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.838+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/3168566306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe9c108040 0x7fbe9c10e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.838+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/3168566306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c1076c0 0x7fbe9c107b00 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.838+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/3168566306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe9c101cb0 0x7fbe9c102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.838+0000 7fbea2ce6640 1 -- 172.21.15.33:0/3168566306 >> 172.21.15.33:0/3168566306 conn(0x7fbe9c0fd700 msgr2=0x7fbe9c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.839+0000 7fbea2ce6640 1 -- 172.21.15.33:0/3168566306 shutdown_connections 2024-09-10T21:17:49.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.839+0000 7fbea2ce6640 1 -- 172.21.15.33:0/3168566306 wait complete. 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.839+0000 7fbea2ce6640 1 Processor -- start 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea2ce6640 1 -- start start 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea2ce6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101cb0 0x7fbe9c0fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea2ce6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe9c1076c0 0x7fbe9c0fe140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea2ce6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe9c108040 0x7fbe9c0fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea2ce6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c111d00 con 0x7fbe9c101cb0 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea2ce6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c111b80 con 0x7fbe9c108040 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea2ce6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c111e80 con 0x7fbe9c1076c0 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101cb0 0x7fbe9c0fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101cb0 0x7fbe9c0fdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45386/0 (socket says 172.21.15.33:45386) 2024-09-10T21:17:49.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 -- 172.21.15.33:0/541636168 learned_addr learned my addr 172.21.15.33:0/541636168 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbe9bfff640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe9c1076c0 0x7fbe9c0fe140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 -- 172.21.15.33:0/541636168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe9c1076c0 msgr2=0x7fbe9c0fe140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe9c1076c0 0x7fbe9c0fe140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 -- 172.21.15.33:0/541636168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe9c108040 msgr2=0x7fbe9c0fe680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe9c108040 0x7fbe9c0fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.840+0000 7fbea0a82640 1 -- 172.21.15.33:0/541636168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe9c1a76d0 con 0x7fbe9c101cb0 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.841+0000 7fbea0a82640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101cb0 0x7fbe9c0fdc00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e9f0 tx=0x7fbe9000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.841+0000 7fbe99ffb640 1 -- 172.21.15.33:0/541636168 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000cd80 con 0x7fbe9c101cb0 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.841+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe9c1a79c0 con 0x7fbe9c101cb0 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.841+0000 7fbe99ffb640 1 -- 172.21.15.33:0/541636168 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe900108e0 con 0x7fbe9c101cb0 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.841+0000 7fbe99ffb640 1 -- 172.21.15.33:0/541636168 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe90018980 con 0x7fbe9c101cb0 2024-09-10T21:17:49.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.841+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9c1a7ed0 con 0x7fbe9c101cb0 2024-09-10T21:17:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.844+0000 7fbea50bb740 1 Processor -- start 2024-09-10T21:17:49.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.844+0000 7fbea50bb740 1 -- start start 2024-09-10T21:17:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.844+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b2fb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.845+0000 7fbe99ffb640 1 -- 172.21.15.33:0/541636168 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbe90010430 con 0x7fbe9c101cb0 2024-09-10T21:17:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.845+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fba10 0x55c52b307460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.845+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b309d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.845+0000 7fbea50bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c52b271ea0 con 0x55c52b2fba10 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.845+0000 7fbea50bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c52b2612d0 con 0x55c52b3079a0 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.845+0000 7fbea50bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c52b2650e0 con 0x55c52b2fb0f0 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.846+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b309d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.846+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b309d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38476/0 (socket says 172.21.15.33:38476) 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.846+0000 7fbe9bfff640 1 -- 172.21.15.33:0/280037956 learned_addr learned my addr 172.21.15.33:0/280037956 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.846+0000 7fbea1283640 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b2fb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.846+0000 7fbe99ffb640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe8c0b52e0 0x7fbe8c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.846+0000 7fbe99ffb640 1 -- 172.21.15.33:0/541636168 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbe900e5fe0 con 0x7fbe9c101cb0 2024-09-10T21:17:49.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.846+0000 7fbea0a82640 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fba10 0x55c52b307460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea1283640 1 -- 172.21.15.33:0/280037956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 msgr2=0x55c52b309d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea1283640 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b309d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea1283640 1 -- 172.21.15.33:0/280037956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fba10 msgr2=0x55c52b307460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea1283640 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fba10 0x55c52b307460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea1283640 1 -- 172.21.15.33:0/280037956 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c52afbf650 con 0x55c52b2fb0f0 2024-09-10T21:17:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea1283640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe8c0b52e0 0x7fbe8c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea0a82640 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fba10 0x55c52b307460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.847+0000 7fbea1283640 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b2fb4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005fd0 tx=0x7fbe940108a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3827574c5d0d66d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.848+0000 7fbea1283640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe8c0b52e0 0x7fbe8c0b77a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe94017000 tx=0x7fbe94019000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.848+0000 7fbe86ffd640 1 -- 172.21.15.33:0/280037956 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94010080 con 0x55c52b2fb0f0 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.848+0000 7fbe86ffd640 1 -- 172.21.15.33:0/280037956 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9401e020 con 0x55c52b2fb0f0 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.848+0000 7fbe86ffd640 1 -- 172.21.15.33:0/280037956 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400e640 con 0x55c52b2fb0f0 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 -- 172.21.15.33:0/280037956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 msgr2=0x55c52b2fb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b2fb4d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbe94005fd0 tx=0x7fbe940108a0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 -- 172.21.15.33:0/280037956 shutdown_connections 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b309d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fba10 0x55c52b307460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 --2- 172.21.15.33:0/280037956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b2fb4d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 -- 172.21.15.33:0/280037956 >> 172.21.15.33:0/280037956 conn(0x55c52b2f8f20 msgr2=0x55c52b2fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 -- 172.21.15.33:0/280037956 shutdown_connections 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.849+0000 7fbea50bb740 1 -- 172.21.15.33:0/280037956 wait complete. 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.850+0000 7fbea50bb740 1 -- start start 2024-09-10T21:17:49.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.850+0000 7fbea50bb740 1 Processor -- start 2024-09-10T21:17:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b307270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fba10 0x55c52b3046c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b304ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea50bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c52b0c2700 con 0x55c52b3079a0 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea50bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c52b1fc160 con 0x55c52b2fba10 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea50bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c52b30a620 con 0x55c52b2fb0f0 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea50bb740 10 client.0 _ll_register_callbacks cb 0x55c52b2159e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fba10 0x55c52b3046c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea1283640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b304ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fba10 0x55c52b3046c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38492/0 (socket says 172.21.15.33:38492) 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 -- 172.21.15.33:0/264432816 learned_addr learned my addr 172.21.15.33:0/264432816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbea0a82640 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b307270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 -- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 msgr2=0x55c52b307270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b307270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 -- 172.21.15.33:0/264432816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 msgr2=0x55c52b304ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b304ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.851+0000 7fbe9bfff640 1 -- 172.21.15.33:0/264432816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c52b229c90 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbe7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbe9bfff640 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fba10 0x55c52b3046c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbe8802a930 tx=0x7fbe880097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f57a62cb15f23d9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbe7bfff640 1 -- 172.21.15.33:0/264432816 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88041070 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbe7bfff640 1 -- 172.21.15.33:0/264432816 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbe88030450 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe9401e020 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbe7bfff640 1 -- 172.21.15.33:0/264432816 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbe8804d050 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbea50bb740 10 client.27362 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c52b315ea0 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.852+0000 7fbe7bfff640 1 -- 172.21.15.33:0/264432816 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe9401e020 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.853+0000 7fbe7bfff640 1 -- 172.21.15.33:0/264432816 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fbe880046c0 con 0x55c52b2fba10 2024-09-10T21:17:49.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.853+0000 7fbea50bb740 10 client.27362 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.853+0000 7fbea50bb740 10 client.27362 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:17:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.853+0000 7fbea50bb740 20 client.27362 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.853+0000 7fbea50bb740 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b3174e0 0x55c52b3198d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.853+0000 7fbea50bb740 4 client.27362 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:49.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.853+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c52b273df0 con 0x55c52b3174e0 2024-09-10T21:17:49.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.854+0000 7fbea0a82640 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b3174e0 0x55c52b3198d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.854+0000 7fbea0a82640 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b3174e0 0x55c52b3198d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a422b9edb730fcb4 server_cookie=7910c7da95205916 in_seq=0 out_seq=0 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.854+0000 7fbe7bfff640 10 client.27362 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.855+0000 7fbe7bfff640 1 -- 172.21.15.33:0/264432816 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c52b273df0 con 0x55c52b3174e0 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.855+0000 7fbe7bfff640 10 client.27362 handle_command_reply: tid=0 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.855+0000 7fbea50bb740 1 client.27362 shutdown 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fba10 msgr2=0x55c52b3046c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fba10 0x55c52b3046c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbe8802a930 tx=0x7fbe880097b0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 shutdown_connections 2024-09-10T21:17:49.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b3174e0 0x55c52b3198d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b304ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fba10 0x55c52b3046c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 --2- 172.21.15.33:0/264432816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fb0f0 0x55c52b307270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 >> 172.21.15.33:0/264432816 conn(0x55c52b2f8f20 msgr2=0x55c52b2feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbe7bfff640 0 client.27362 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 shutdown_connections 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 1 -- 172.21.15.33:0/264432816 wait complete. 2024-09-10T21:17:49.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.856+0000 7fbea50bb740 20 client.27362 trim_cache size 0 max 16384 2024-09-10T21:17:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 Processor -- start 2024-09-10T21:17:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 -- start start 2024-09-10T21:17:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fb0f0 0x55c52b2f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b302f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b303480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b0680 con 0x55c52b3079a0 2024-09-10T21:17:49.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b23d0 con 0x55c52b2fb0f0 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea50bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b0d80 con 0x55c52b2fba10 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea0a82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fb0f0 0x55c52b2f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea0a82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fb0f0 0x55c52b2f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38506/0 (socket says 172.21.15.33:38506) 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea0a82640 1 -- 172.21.15.33:0/2908967019 learned_addr learned my addr 172.21.15.33:0/2908967019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea1283640 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b303480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbe9bfff640 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b302f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.872+0000 7fbea0a82640 1 -- 172.21.15.33:0/2908967019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 msgr2=0x55c52b302f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbea0a82640 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b302f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbea0a82640 1 -- 172.21.15.33:0/2908967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 msgr2=0x55c52b303480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbea0a82640 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b303480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbea0a82640 1 -- 172.21.15.33:0/2908967019 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c52b2f9c00 con 0x55c52b2fb0f0 2024-09-10T21:17:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbea0a82640 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fb0f0 0x55c52b2f8760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbe900e3b80 tx=0x7fbe900e3a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37124e83c978f9f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbe867fc640 1 -- 172.21.15.33:0/2908967019 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe900acf40 con 0x55c52b2fb0f0 2024-09-10T21:17:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbe867fc640 1 -- 172.21.15.33:0/2908967019 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe900ac840 con 0x55c52b2fb0f0 2024-09-10T21:17:49.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.873+0000 7fbe867fc640 1 -- 172.21.15.33:0/2908967019 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe900adb70 con 0x55c52b2fb0f0 2024-09-10T21:17:49.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 -- 172.21.15.33:0/2908967019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fb0f0 msgr2=0x55c52b2f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fb0f0 0x55c52b2f8760 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbe900e3b80 tx=0x7fbe900e3a50 comp rx=0 tx=0).stop 2024-09-10T21:17:49.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 -- 172.21.15.33:0/2908967019 shutdown_connections 2024-09-10T21:17:49.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b3079a0 0x55c52b303480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b302f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 --2- 172.21.15.33:0/2908967019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b2fb0f0 0x55c52b2f8760 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 -- 172.21.15.33:0/2908967019 >> 172.21.15.33:0/2908967019 conn(0x55c52b2f8f20 msgr2=0x55c52b319450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 -- 172.21.15.33:0/2908967019 shutdown_connections 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.874+0000 7fbea50bb740 1 -- 172.21.15.33:0/2908967019 wait complete. 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 -- start start 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 Processor -- start 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fb0f0 0x55c52b310200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b31ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b31f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b1c70 con 0x55c52b2fb0f0 2024-09-10T21:17:49.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1b14e0 con 0x55c52b3079a0 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9c1ae640 con 0x55c52b2fba10 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.875+0000 7fbea50bb740 10 client.0 _ll_register_callbacks cb 0x7fbe94009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b31f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b31ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea0a82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fb0f0 0x55c52b310200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea0a82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fb0f0 0x55c52b310200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45422/0 (socket says 172.21.15.33:45422) 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b31f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38512/0 (socket says 172.21.15.33:38512) 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbe9bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b31ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59428/0 (socket says 172.21.15.33:59428) 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea0a82640 1 -- 172.21.15.33:0/289931130 learned_addr learned my addr 172.21.15.33:0/289931130 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 -- 172.21.15.33:0/289931130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 msgr2=0x55c52b31ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b31ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 -- 172.21.15.33:0/289931130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fb0f0 msgr2=0x55c52b310200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:49.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fb0f0 0x55c52b310200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 -- 172.21.15.33:0/289931130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c52b30ee60 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbea1283640 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b31f5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e480 tx=0x7fbe940102b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aff7a05d14a6b392 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbe7bfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbe7bfff640 1 -- 172.21.15.33:0/289931130 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe94026030 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe900ac840 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.876+0000 7fbe7bfff640 1 -- 172.21.15.33:0/289931130 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbe94015050 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbe7bfff640 1 -- 172.21.15.33:0/289931130 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbe94019a60 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 10 client.27368 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c52b3844a0 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbe7bfff640 1 -- 172.21.15.33:0/289931130 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe900ac840 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbe7bfff640 1 -- 172.21.15.33:0/289931130 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fbe9400b040 con 0x55c52b3079a0 2024-09-10T21:17:49.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 10 client.27368 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 10 client.27368 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 20 client.27368 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b384b50 0x55c52b386f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 4 client.27368 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.877+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c52b26f8a0 con 0x55c52b384b50 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.878+0000 7fbea0a82640 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b384b50 0x55c52b386f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.878+0000 7fbe7bfff640 10 client.27368 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:49.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.878+0000 7fbea0a82640 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b384b50 0x55c52b386f70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19486cc3de98183e server_cookie=500843965cfae5b2 in_seq=0 out_seq=0 2024-09-10T21:17:49.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbe7bfff640 1 -- 172.21.15.33:0/289931130 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c52b26f8a0 con 0x55c52b384b50 2024-09-10T21:17:49.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbe7bfff640 10 client.27368 handle_command_reply: tid=0 2024-09-10T21:17:49.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 client.27368 shutdown 2024-09-10T21:17:49.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 msgr2=0x55c52b31f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b31f5e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400e480 tx=0x7fbe940102b0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 shutdown_connections 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55c52b384b50 0x55c52b386f70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c52b3079a0 0x55c52b31f5e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c52b2fba10 0x55c52b31ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 --2- 172.21.15.33:0/289931130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c52b2fb0f0 0x55c52b310200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbe7bfff640 0 client.27368 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 >> 172.21.15.33:0/289931130 conn(0x55c52b2f8f20 msgr2=0x55c52b319450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 shutdown_connections 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 1 -- 172.21.15.33:0/289931130 wait complete. 2024-09-10T21:17:49.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.880+0000 7fbea50bb740 20 client.27368 trim_cache size 0 max 16384 2024-09-10T21:17:49.891 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe8c0b52e0 msgr2=0x7fbe8c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe8c0b52e0 0x7fbe8c0b77a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbe94017000 tx=0x7fbe94019000 comp rx=0 tx=0).stop 2024-09-10T21:17:49.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101cb0 msgr2=0x7fbe9c0fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101cb0 0x7fbe9c0fdc00 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000e9f0 tx=0x7fbe9000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 shutdown_connections 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe8c0b52e0 0x7fbe8c0b77a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe9c108040 0x7fbe9c0fe680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe9c1076c0 0x7fbe9c0fe140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 --2- 172.21.15.33:0/541636168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe9c101cb0 0x7fbe9c0fdc00 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 >> 172.21.15.33:0/541636168 conn(0x7fbe9c0fd700 msgr2=0x7fbe9c06b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 shutdown_connections 2024-09-10T21:17:49.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:49.881+0000 7fbea2ce6640 1 -- 172.21.15.33:0/541636168 wait complete. 2024-09-10T21:17:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:49 smithi174 ceph-mon[35635]: pgmap v761: 97 pgs: 97 active+clean; 774 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.07k op/s 2024-09-10T21:17:50.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 46102 53.17 MB/sec warmup 50 sec latency 257.666 ms 2024-09-10T21:17:50.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:17:51.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 47013 53.02 MB/sec warmup 51 sec latency 226.133 ms 2024-09-10T21:17:51.053 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 -- 172.21.15.33:0/2670394133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 msgr2=0x7fbf00108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 --2- 172.21.15.33:0/2670394133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 0x7fbf00108c10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402a410 comp rx=0 tx=0).stop 2024-09-10T21:17:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 -- 172.21.15.33:0/2670394133 shutdown_connections 2024-09-10T21:17:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 --2- 172.21.15.33:0/2670394133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf00109150 0x7fbf0010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 --2- 172.21.15.33:0/2670394133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 0x7fbf00108c10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 --2- 172.21.15.33:0/2670394133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 0x7fbf001031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 -- 172.21.15.33:0/2670394133 >> 172.21.15.33:0/2670394133 conn(0x7fbf000fe830 msgr2=0x7fbf00100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 -- 172.21.15.33:0/2670394133 shutdown_connections 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.599+0000 7fbf05a35640 1 -- 172.21.15.33:0/2670394133 wait complete. 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.600+0000 7fbf05a35640 1 Processor -- start 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.600+0000 7fbf05a35640 1 -- start start 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbf05a35640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 0x7fbf00199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbf05a35640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 0x7fbf001997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbf05a35640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf00109150 0x7fbf0019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbf05a35640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00112da0 con 0x7fbf001087d0 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbf05a35640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00112c20 con 0x7fbf00102dc0 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbf05a35640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf00112f20 con 0x7fbf00109150 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 0x7fbf00199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 0x7fbf00199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38538/0 (socket says 172.21.15.33:38538) 2024-09-10T21:17:51.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 -- 172.21.15.33:0/3740875222 learned_addr learned my addr 172.21.15.33:0/3740875222 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:51.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 0x7fbf001997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 -- 172.21.15.33:0/3740875222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf00109150 msgr2=0x7fbf0019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf00109150 0x7fbf0019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 -- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 msgr2=0x7fbf001997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 0x7fbf001997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefeffd640 1 -- 172.21.15.33:0/3740875222 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0019f310 con 0x7fbf00102dc0 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.601+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 0x7fbf001997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.602+0000 7fbefeffd640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 0x7fbf00199260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbef000d950 tx=0x7fbef000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.602+0000 7fbf04a33640 1 -- 172.21.15.33:0/3740875222 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0017070 con 0x7fbf00102dc0 2024-09-10T21:17:51.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.602+0000 7fbf04a33640 1 -- 172.21.15.33:0/3740875222 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef0009c30 con 0x7fbf00102dc0 2024-09-10T21:17:51.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.603+0000 7fbf04a33640 1 -- 172.21.15.33:0/3740875222 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef000f620 con 0x7fbf00102dc0 2024-09-10T21:17:51.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.603+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0019f600 con 0x7fbf00102dc0 2024-09-10T21:17:51.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.603+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf001a5cc0 con 0x7fbf00102dc0 2024-09-10T21:17:51.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.605+0000 7fbf07e0a740 1 Processor -- start 2024-09-10T21:17:51.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.605+0000 7fbf07e0a740 1 -- start start 2024-09-10T21:17:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada6488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada654840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada657170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbf04a33640 1 -- 172.21.15.33:0/3740875222 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbef0010460 con 0x7fbf00102dc0 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eada5beea0 con 0x55eada6484d0 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eada5ae2d0 con 0x55eada648df0 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eada5b20e0 con 0x55eada654d80 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada6488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada6488b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45456/0 (socket says 172.21.15.33:45456) 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 -- 172.21.15.33:0/4277141240 learned_addr learned my addr 172.21.15.33:0/4277141240 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada657170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbefeffd640 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada654840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 -- 172.21.15.33:0/4277141240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 msgr2=0x55eada657170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada657170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 -- 172.21.15.33:0/4277141240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 msgr2=0x55eada654840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada654840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.606+0000 7fbeff7fe640 1 -- 172.21.15.33:0/4277141240 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eada30c650 con 0x55eada6484d0 2024-09-10T21:17:51.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.607+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada6488b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009d80 tx=0x7fbef800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=377fb812661d6913 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.607+0000 7fbee5ffb640 1 -- 172.21.15.33:0/4277141240 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8016070 con 0x55eada6484d0 2024-09-10T21:17:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.607+0000 7fbee5ffb640 1 -- 172.21.15.33:0/4277141240 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef8005020 con 0x55eada6484d0 2024-09-10T21:17:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.607+0000 7fbee5ffb640 1 -- 172.21.15.33:0/4277141240 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80116d0 con 0x55eada6484d0 2024-09-10T21:17:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.607+0000 7fbf04a33640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbee80b5360 0x7fbee80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.608+0000 7fbf04a33640 1 -- 172.21.15.33:0/3740875222 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbef00e57f0 con 0x7fbf00102dc0 2024-09-10T21:17:51.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.608+0000 7fbefeffd640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbee80b5360 0x7fbee80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.608+0000 7fbf07e0a740 1 -- 172.21.15.33:0/4277141240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 msgr2=0x55eada6488b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.608+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada6488b0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009d80 tx=0x7fbef800e6c0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbf07e0a740 1 -- 172.21.15.33:0/4277141240 shutdown_connections 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbefeffd640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbee80b5360 0x7fbee80b7820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbef00a9220 tx=0x7fbef0010cb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada657170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada654840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/4277141240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada6488b0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbf07e0a740 1 -- 172.21.15.33:0/4277141240 >> 172.21.15.33:0/4277141240 conn(0x55eada646300 msgr2=0x55eada6474a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbf07e0a740 1 -- 172.21.15.33:0/4277141240 shutdown_connections 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.609+0000 7fbf07e0a740 1 -- 172.21.15.33:0/4277141240 wait complete. 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 -- start start 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 Processor -- start 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada654680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada651ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada6520e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55eada40f700 con 0x55eada6484d0 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55eada549160 con 0x55eada648df0 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eada657a00 con 0x55eada654d80 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbf07e0a740 10 client.0 _ll_register_callbacks cb 0x55eada5629e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada654680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbefe7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada651ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada654680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45462/0 (socket says 172.21.15.33:45462) 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.610+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3557521577 learned_addr learned my addr 172.21.15.33:0/3557521577 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:51.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada6520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbefe7fc640 1 -- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 msgr2=0x55eada6520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada6520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbefe7fc640 1 -- 172.21.15.33:0/3557521577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 msgr2=0x55eada654680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada654680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbefe7fc640 1 -- 172.21.15.33:0/3557521577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eada576c90 con 0x55eada648df0 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada654680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada651ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009230 tx=0x7fbef4031810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2392b42c5ac8e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbedeffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbedeffd640 1 -- 172.21.15.33:0/3557521577 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4035040 con 0x55eada648df0 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef8005020 con 0x55eada648df0 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.611+0000 7fbedeffd640 1 -- 172.21.15.33:0/3557521577 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbef4004240 con 0x55eada648df0 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbedeffd640 1 -- 172.21.15.33:0/3557521577 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbef4004440 con 0x55eada648df0 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 10 client.27380 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:51.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55eada66bb90 con 0x55eada648df0 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbedeffd640 1 -- 172.21.15.33:0/3557521577 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef8005020 con 0x55eada648df0 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbedeffd640 1 -- 172.21.15.33:0/3557521577 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fbef4004bc0 con 0x55eada648df0 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 10 client.27380 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 10 client.27380 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 20 client.27380 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada66d0a0 0x55eada66f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 4 client.27380 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.612+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eada5c0df0 con 0x55eada66d0a0 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.613+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada66d0a0 0x55eada66f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.613+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada66d0a0 0x55eada66f490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e66e82362897041c server_cookie=101499e6b0b5c246 in_seq=0 out_seq=0 2024-09-10T21:17:51.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.614+0000 7fbedeffd640 10 client.27380 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbedeffd640 1 -- 172.21.15.33:0/3557521577 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eada5c0df0 con 0x55eada66d0a0 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbedeffd640 10 client.27380 handle_command_reply: tid=0 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 client.27380 shutdown 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 msgr2=0x55eada651ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada651ad0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009230 tx=0x7fbef4031810 comp rx=0 tx=0).stop 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 shutdown_connections 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada66d0a0 0x55eada66f490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada6520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada651ad0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbedeffd640 0 client.27380 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3557521577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada6484d0 0x55eada654680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 >> 172.21.15.33:0/3557521577 conn(0x55eada646300 msgr2=0x55eada64be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.618+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 shutdown_connections 2024-09-10T21:17:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.619+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3557521577 wait complete. 2024-09-10T21:17:51.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.619+0000 7fbf07e0a740 20 client.27380 trim_cache size 0 max 16384 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.642+0000 7fbf07e0a740 1 Processor -- start 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.642+0000 7fbf07e0a740 1 -- start start 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada6484d0 0x55eada647430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada647970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada654d80 0x55eada656170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001ac230 con 0x55eada654d80 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001adf80 con 0x55eada648df0 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001ac930 con 0x55eada6484d0 2024-09-10T21:17:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada654d80 0x55eada656170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada654d80 0x55eada656170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45474/0 (socket says 172.21.15.33:45474) 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3762464451 learned_addr learned my addr 172.21.15.33:0/3762464451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada647970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3762464451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada6484d0 msgr2=0x55eada647430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada6484d0 0x55eada647430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3762464451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 msgr2=0x55eada647970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada647970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3762464451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eada66faf0 con 0x55eada654d80 2024-09-10T21:17:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.643+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada654d80 0x55eada656170 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009d80 tx=0x7fbef8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77461c4bdac0d48f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.644+0000 7fbee57fa640 1 -- 172.21.15.33:0/3762464451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8016070 con 0x55eada654d80 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.644+0000 7fbee57fa640 1 -- 172.21.15.33:0/3762464451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef800ee50 con 0x55eada654d80 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.644+0000 7fbee57fa640 1 -- 172.21.15.33:0/3762464451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8023040 con 0x55eada654d80 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3762464451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada654d80 msgr2=0x55eada656170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada654d80 0x55eada656170 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009d80 tx=0x7fbef8004a40 comp rx=0 tx=0).stop 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3762464451 shutdown_connections 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada654d80 0x55eada656170 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada648df0 0x55eada647970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3762464451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada6484d0 0x55eada647430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3762464451 >> 172.21.15.33:0/3762464451 conn(0x55eada646300 msgr2=0x55eada64cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3762464451 shutdown_connections 2024-09-10T21:17:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3762464451 wait complete. 2024-09-10T21:17:51.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.645+0000 7fbf07e0a740 1 -- start start 2024-09-10T21:17:51.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbf07e0a740 1 Processor -- start 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada6484d0 0x55eada665e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada648df0 0x55eada660510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbf07e0a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada660bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001ad820 con 0x55eada648df0 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001ad090 con 0x55eada6484d0 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbf07e0a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf001aa1f0 con 0x55eada654d80 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada6484d0 0x55eada665e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbefe7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada648df0 0x55eada660510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbeff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada6484d0 0x55eada665e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38584/0 (socket says 172.21.15.33:38584) 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.646+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3555220712 learned_addr learned my addr 172.21.15.33:0/3555220712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada660bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbf07e0a740 10 client.0 _ll_register_callbacks cb 0x7fbef4005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3555220712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada6484d0 msgr2=0x55eada665e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada6484d0 0x55eada665e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3555220712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada648df0 msgr2=0x55eada660510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada648df0 0x55eada660510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbeff7fe640 1 -- 172.21.15.33:0/3555220712 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eada641990 con 0x55eada654d80 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada648df0 0x55eada660510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbeff7fe640 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada660bc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009cd0 tx=0x7fbef800e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0bfe020072a8fdd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.647+0000 7fbedeffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:17:51.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.648+0000 7fbedeffd640 1 -- 172.21.15.33:0/3555220712 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8016070 con 0x55eada654d80 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.648+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef800ee50 con 0x55eada654d80 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.648+0000 7fbedeffd640 1 -- 172.21.15.33:0/3555220712 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbef80042d0 con 0x55eada654d80 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.648+0000 7fbedeffd640 1 -- 172.21.15.33:0/3555220712 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbef80044d0 con 0x55eada654d80 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.648+0000 7fbf07e0a740 10 client.27316 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.648+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55eada6526c0 con 0x55eada654d80 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.649+0000 7fbedeffd640 1 -- 172.21.15.33:0/3555220712 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef800ee50 con 0x55eada654d80 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.649+0000 7fbedeffd640 1 -- 172.21.15.33:0/3555220712 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fbef802f070 con 0x55eada654d80 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.649+0000 7fbf07e0a740 10 client.27316 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.650+0000 7fbf07e0a740 10 client.27316 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.650+0000 7fbf07e0a740 20 client.27316 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.650+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada6db230 0x55eada6dd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:51.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.650+0000 7fbf07e0a740 4 client.27316 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.650+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eada5bc8a0 con 0x55eada6db230 2024-09-10T21:17:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.651+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada6db230 0x55eada6dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.653+0000 7fbefe7fc640 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada6db230 0x55eada6dd650 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b5e5ffb9467cbb3 server_cookie=e9252f5fec019309 in_seq=0 out_seq=0 2024-09-10T21:17:51.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.653+0000 7fbedeffd640 10 client.27316 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbedeffd640 1 -- 172.21.15.33:0/3555220712 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eada5bc8a0 con 0x55eada6db230 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbedeffd640 10 client.27316 handle_command_reply: tid=0 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 client.27316 shutdown 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 msgr2=0x55eada660bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada660bc0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009cd0 tx=0x7fbef800e6c0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 shutdown_connections 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55eada6db230 0x55eada6dd650 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eada654d80 0x55eada660bc0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55eada648df0 0x55eada660510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbedeffd640 0 client.27316 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:17:51.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 --2- 172.21.15.33:0/3555220712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55eada6484d0 0x55eada665e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 >> 172.21.15.33:0/3555220712 conn(0x55eada646300 msgr2=0x55eada647350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 shutdown_connections 2024-09-10T21:17:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 1 -- 172.21.15.33:0/3555220712 wait complete. 2024-09-10T21:17:51.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.656+0000 7fbf07e0a740 20 client.27316 trim_cache size 0 max 16384 2024-09-10T21:17:51.663 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:51.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbee80b5360 msgr2=0x7fbee80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbee80b5360 0x7fbee80b7820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbef00a9220 tx=0x7fbef0010cb0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 msgr2=0x7fbf00199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:51.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 0x7fbf00199260 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbef000d950 tx=0x7fbef000de10 comp rx=0 tx=0).stop 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 shutdown_connections 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbee80b5360 0x7fbee80b7820 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf00109150 0x7fbf0019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbf001087d0 0x7fbf001997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 --2- 172.21.15.33:0/3740875222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbf00102dc0 0x7fbf00199260 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.657+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 >> 172.21.15.33:0/3740875222 conn(0x7fbf000fe830 msgr2=0x7fbf00100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.658+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 shutdown_connections 2024-09-10T21:17:51.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:51.658+0000 7fbf05a35640 1 -- 172.21.15.33:0/3740875222 wait complete. 2024-09-10T21:17:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:51 smithi104 ceph-mon[35325]: pgmap v762: 97 pgs: 97 active+clean; 744 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 1.14k op/s 2024-09-10T21:17:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:51 smithi174 ceph-mon[35635]: pgmap v762: 97 pgs: 97 active+clean; 744 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 1.14k op/s 2024-09-10T21:17:51.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:51 smithi033 ceph-mon[30860]: pgmap v762: 97 pgs: 97 active+clean; 744 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 20 MiB/s wr, 1.14k op/s 2024-09-10T21:17:52.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 47739 52.84 MB/sec warmup 52 sec latency 245.842 ms 2024-09-10T21:17:53.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 48590 52.69 MB/sec warmup 53 sec latency 111.951 ms 2024-09-10T21:17:53.283 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:17:53.571 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:53.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:53 smithi033 ceph-mon[30860]: pgmap v763: 97 pgs: 97 active+clean; 730 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.25k op/s 2024-09-10T21:17:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:53 smithi104 ceph-mon[35325]: pgmap v763: 97 pgs: 97 active+clean; 730 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.25k op/s 2024-09-10T21:17:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:53 smithi174 ceph-mon[35635]: pgmap v763: 97 pgs: 97 active+clean; 730 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.25k op/s 2024-09-10T21:17:54.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 49412 52.57 MB/sec warmup 54 sec latency 309.540 ms 2024-09-10T21:17:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.106+0000 7f5f63b85640 1 -- 172.21.15.33:0/3440258760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 msgr2=0x7f5f5c10e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.106+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3440258760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 0x7f5f5c10e860 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009ca0 tx=0x7f5f5402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.106+0000 7f5f63b85640 1 -- 172.21.15.33:0/3440258760 shutdown_connections 2024-09-10T21:17:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.106+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3440258760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 0x7f5f5c10e860 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.106+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3440258760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 0x7f5f5c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.106+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3440258760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 0x7f5f5c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.106+0000 7f5f63b85640 1 -- 172.21.15.33:0/3440258760 >> 172.21.15.33:0/3440258760 conn(0x7f5f5c0fd740 msgr2=0x7f5f5c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.107+0000 7f5f63b85640 1 -- 172.21.15.33:0/3440258760 shutdown_connections 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.107+0000 7f5f63b85640 1 -- 172.21.15.33:0/3440258760 wait complete. 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.107+0000 7f5f63b85640 1 Processor -- start 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.107+0000 7f5f63b85640 1 -- start start 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.107+0000 7f5f63b85640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 0x7f5f5c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.107+0000 7f5f63b85640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 0x7f5f5c1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.107+0000 7f5f63b85640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 0x7f5f5c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f63b85640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c112e20 con 0x7f5f5c108060 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f63b85640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c112ca0 con 0x7f5f5c101cd0 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f63b85640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c112fa0 con 0x7f5f5c1076e0 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61921640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 0x7f5f5c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 0x7f5f5c1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61921640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 0x7f5f5c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38604/0 (socket says 172.21.15.33:38604) 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f62122640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 0x7f5f5c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61921640 1 -- 172.21.15.33:0/3783145076 learned_addr learned my addr 172.21.15.33:0/3783145076 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 0x7f5f5c1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59522/0 (socket says 172.21.15.33:59522) 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 -- 172.21.15.33:0/3783145076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 msgr2=0x7f5f5c199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 0x7f5f5c199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 -- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 msgr2=0x7f5f5c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 0x7f5f5c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 -- 172.21.15.33:0/3783145076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f5c19f340 con 0x7f5f5c1076e0 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61921640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 0x7f5f5c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.108+0000 7f5f61120640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 0x7f5f5c1997d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009bc0 tx=0x7f5f4c00b690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.109+0000 7f5f52ffd640 1 -- 172.21.15.33:0/3783145076 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c01e070 con 0x7f5f5c1076e0 2024-09-10T21:17:54.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.109+0000 7f5f52ffd640 1 -- 172.21.15.33:0/3783145076 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f4c0198c0 con 0x7f5f5c1076e0 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.109+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f5c19f630 con 0x7f5f5c1076e0 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.109+0000 7f5f52ffd640 1 -- 172.21.15.33:0/3783145076 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4c0187d0 con 0x7f5f5c1076e0 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.109+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f5c10f0a0 con 0x7f5f5c1076e0 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.111+0000 7f5f65f5a740 1 Processor -- start 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f65f5a740 1 -- start start 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f52ffd640 1 -- 172.21.15.33:0/3783145076 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5f4c019410 con 0x7f5f5c1076e0 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366788b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36684840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36687170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf365eeea0 con 0x55bf366784d0 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf365de2d0 con 0x55bf36684d80 2024-09-10T21:17:54.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf365e20e0 con 0x55bf36678df0 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f62122640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366788b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f61120640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36687170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f62122640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366788b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45510/0 (socket says 172.21.15.33:45510) 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f62122640 1 -- 172.21.15.33:0/1327400228 learned_addr learned my addr 172.21.15.33:0/1327400228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.112+0000 7f5f61921640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36684840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f62122640 1 -- 172.21.15.33:0/1327400228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 msgr2=0x55bf36684840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f62122640 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36684840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f62122640 1 -- 172.21.15.33:0/1327400228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 msgr2=0x55bf36687170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f62122640 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36687170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f62122640 1 -- 172.21.15.33:0/1327400228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3633c650 con 0x55bf366784d0 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f61921640 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36684840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:54.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f61120640 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36687170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f62122640 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366788b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009410 tx=0x7f5f54005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fe9e4588980832 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f52ffd640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f480b52e0 0x7f5f480b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.113+0000 7f5f47fff640 1 -- 172.21.15.33:0/1327400228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54051070 con 0x55bf366784d0 2024-09-10T21:17:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.114+0000 7f5f61921640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f480b52e0 0x7f5f480b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.114+0000 7f5f52ffd640 1 -- 172.21.15.33:0/3783145076 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f4c0e9630 con 0x7f5f5c1076e0 2024-09-10T21:17:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.114+0000 7f5f47fff640 1 -- 172.21.15.33:0/1327400228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54036070 con 0x55bf366784d0 2024-09-10T21:17:54.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.114+0000 7f5f47fff640 1 -- 172.21.15.33:0/1327400228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54003e50 con 0x55bf366784d0 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1327400228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 msgr2=0x55bf366788b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366788b0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5f54009410 tx=0x7f5f54005ef0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f61921640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f480b52e0 0x7f5f480b77a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009620 tx=0x7f5f58005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1327400228 shutdown_connections 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36687170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36684840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1327400228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366788b0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1327400228 >> 172.21.15.33:0/1327400228 conn(0x55bf36676300 msgr2=0x55bf366774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1327400228 shutdown_connections 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1327400228 wait complete. 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 -- start start 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.115+0000 7f5f65f5a740 1 Processor -- start 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f61921640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f61921640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366845d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45520/0 (socket says 172.21.15.33:45520) 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f61921640 1 -- 172.21.15.33:0/390141857 learned_addr learned my addr 172.21.15.33:0/390141857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36681a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36681f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf3643f700 con 0x55bf366784d0 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf36579160 con 0x55bf36684d80 2024-09-10T21:17:54.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf36687820 con 0x55bf36678df0 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.116+0000 7f5f65f5a740 10 client.0 _ll_register_callbacks cb 0x55bf365929e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f62122640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36681f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f61120640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36681a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f62122640 1 -- 172.21.15.33:0/390141857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 msgr2=0x55bf36681a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f62122640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36681a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f62122640 1 -- 172.21.15.33:0/390141857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 msgr2=0x55bf366845d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f62122640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366845d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f62122640 1 -- 172.21.15.33:0/390141857 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf365a6c90 con 0x55bf36684d80 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f61921640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f62122640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36681f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f54006010 tx=0x7f5f54003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20851dcb04ba2256 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.117+0000 7f5f457fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f457fa640 1 -- 172.21.15.33:0/390141857 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54051040 con 0x55bf36684d80 2024-09-10T21:17:54.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f457fa640 1 -- 172.21.15.33:0/390141857 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f5402f660 con 0x55bf36684d80 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54036070 con 0x55bf36684d80 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f457fa640 1 -- 172.21.15.33:0/390141857 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5f54031e40 con 0x55bf36684d80 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 10 client.27398 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55bf366930f0 con 0x55bf36684d80 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f457fa640 1 -- 172.21.15.33:0/390141857 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f5402fd10 con 0x55bf36684d80 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f457fa640 1 -- 172.21.15.33:0/390141857 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f5f5404b030 con 0x55bf36684d80 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 10 client.27398 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 10 client.27398 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 20 client.27398 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36694440 0x55bf36696830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 4 client.27398 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:54.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.118+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5f54036070 con 0x55bf36694440 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.119+0000 7f5f61921640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36694440 0x55bf36696830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.119+0000 7f5f457fa640 10 client.27398 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.119+0000 7f5f61921640 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36694440 0x55bf36696830 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fafdf13dffde4983 server_cookie=14c85a1408029cc0 in_seq=0 out_seq=0 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f457fa640 1 -- 172.21.15.33:0/390141857 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f5f54036070 con 0x55bf36694440 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f457fa640 10 client.27398 handle_command_reply: tid=0 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 client.27398 shutdown 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 msgr2=0x55bf36681f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36681f80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5f54006010 tx=0x7f5f54003970 comp rx=0 tx=0).stop 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 shutdown_connections 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36684d80 0x55bf36681f80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36678df0 0x55bf36681a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36694440 0x55bf36696830 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f457fa640 0 client.27398 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/390141857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf366845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.120+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 >> 172.21.15.33:0/390141857 conn(0x55bf36676300 msgr2=0x55bf3667be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.121+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 shutdown_connections 2024-09-10T21:17:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.121+0000 7f5f65f5a740 1 -- 172.21.15.33:0/390141857 wait complete. 2024-09-10T21:17:54.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.121+0000 7f5f65f5a740 20 client.27398 trim_cache size 0 max 16384 2024-09-10T21:17:54.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.135+0000 7f5f65f5a740 1 Processor -- start 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.135+0000 7f5f65f5a740 1 -- start start 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf366784d0 0x55bf366773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf36677900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf36684d80 0x55bf366817d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1ac170 con 0x55bf36684d80 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1adec0 con 0x55bf36678df0 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1ac870 con 0x55bf366784d0 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf36684d80 0x55bf366817d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf36684d80 0x55bf366817d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45530/0 (socket says 172.21.15.33:45530) 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f61120640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf36677900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 -- 172.21.15.33:0/1631670278 learned_addr learned my addr 172.21.15.33:0/1631670278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:54.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f61921640 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf366784d0 0x55bf366773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 -- 172.21.15.33:0/1631670278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf366784d0 msgr2=0x55bf366773c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf366784d0 0x55bf366773c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 -- 172.21.15.33:0/1631670278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 msgr2=0x55bf36677900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf36677900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f62122640 1 -- 172.21.15.33:0/1631670278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3667c780 con 0x55bf36684d80 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.136+0000 7f5f61120640 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf36677900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.137+0000 7f5f62122640 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf36684d80 0x55bf366817d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5f54005e30 tx=0x7f5f54057470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d45dbb315a07732 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.137+0000 7f5f477fe640 1 -- 172.21.15.33:0/1631670278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54048020 con 0x55bf36684d80 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.137+0000 7f5f477fe640 1 -- 172.21.15.33:0/1631670278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54036070 con 0x55bf36684d80 2024-09-10T21:17:54.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.137+0000 7f5f477fe640 1 -- 172.21.15.33:0/1631670278 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54051030 con 0x55bf36684d80 2024-09-10T21:17:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1631670278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf36684d80 msgr2=0x55bf366817d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf36684d80 0x55bf366817d0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f5f54005e30 tx=0x7f5f54057470 comp rx=0 tx=0).stop 2024-09-10T21:17:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1631670278 shutdown_connections 2024-09-10T21:17:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf36684d80 0x55bf366817d0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf36677900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1631670278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf366784d0 0x55bf366773c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1631670278 >> 172.21.15.33:0/1631670278 conn(0x55bf36676300 msgr2=0x55bf3667cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1631670278 shutdown_connections 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1631670278 wait complete. 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 -- start start 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.138+0000 7f5f65f5a740 1 Processor -- start 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf3668cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf3668d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f65f5a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36684d80 0x55bf36687ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1ad760 con 0x55bf366784d0 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1acfd0 con 0x55bf36678df0 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f65f5a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f5c1aa130 con 0x55bf36684d80 2024-09-10T21:17:54.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f65f5a740 10 client.0 _ll_register_callbacks cb 0x7f5f5404f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f61120640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf3668d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f62122640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36684d80 0x55bf36687ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f62122640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36684d80 0x55bf36687ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59536/0 (socket says 172.21.15.33:59536) 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f62122640 1 -- 172.21.15.33:0/1268341444 learned_addr learned my addr 172.21.15.33:0/1268341444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.139+0000 7f5f61921640 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf3668cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f61921640 1 -- 172.21.15.33:0/1268341444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36684d80 msgr2=0x55bf36687ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f61921640 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36684d80 0x55bf36687ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f61921640 1 -- 172.21.15.33:0/1268341444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 msgr2=0x55bf3668d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f61921640 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf3668d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f61921640 1 -- 172.21.15.33:0/1268341444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3667c590 con 0x55bf366784d0 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f62122640 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36684d80 0x55bf36687ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f61921640 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf3668cb20 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55bf365de390 tx=0x7f5f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9661f56698e6c8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:54.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.140+0000 7f5f457fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f457fa640 1 -- 172.21.15.33:0/1268341444 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800fe20 con 0x55bf366784d0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f457fa640 1 -- 172.21.15.33:0/1268341444 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f58003ac0 con 0x55bf366784d0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f54036070 con 0x55bf366784d0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f457fa640 1 -- 172.21.15.33:0/1268341444 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f5f58003cc0 con 0x55bf366784d0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f65f5a740 10 client.17982 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55bf36701890 con 0x55bf366784d0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f457fa640 1 -- 172.21.15.33:0/1268341444 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f54036070 con 0x55bf366784d0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f457fa640 1 -- 172.21.15.33:0/1268341444 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f5f5803a050 con 0x55bf366784d0 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f65f5a740 10 client.17982 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f65f5a740 10 client.17982 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:17:54.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.141+0000 7f5f65f5a740 20 client.17982 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.142+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36702050 0x55bf36704470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.142+0000 7f5f65f5a740 4 client.17982 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.142+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf365f0df0 con 0x55bf36702050 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.142+0000 7f5f61120640 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36702050 0x55bf36704470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.142+0000 7f5f457fa640 10 client.17982 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.142+0000 7f5f61120640 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36702050 0x55bf36704470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cbd9f89b773c8f7 server_cookie=5eb5b76e5d65f930 in_seq=0 out_seq=0 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f457fa640 1 -- 172.21.15.33:0/1268341444 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf365f0df0 con 0x55bf36702050 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f457fa640 10 client.17982 handle_command_reply: tid=0 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 client.17982 shutdown 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 msgr2=0x55bf3668cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf3668cb20 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55bf365de390 tx=0x7f5f580046c0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 shutdown_connections 2024-09-10T21:17:54.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf36684d80 0x55bf36687ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf36678df0 0x55bf3668d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bf36702050 0x55bf36704470 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 --2- 172.21.15.33:0/1268341444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf366784d0 0x55bf3668cb20 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 >> 172.21.15.33:0/1268341444 conn(0x55bf36676300 msgr2=0x55bf366773c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f457fa640 0 client.17982 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 shutdown_connections 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 1 -- 172.21.15.33:0/1268341444 wait complete. 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.143+0000 7f5f65f5a740 20 client.17982 trim_cache size 0 max 16384 2024-09-10T21:17:54.152 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f480b52e0 msgr2=0x7f5f480b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f480b52e0 0x7f5f480b77a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5f58009620 tx=0x7f5f58005f90 comp rx=0 tx=0).stop 2024-09-10T21:17:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 msgr2=0x7f5f5c1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 0x7f5f5c1997d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c009bc0 tx=0x7f5f4c00b690 comp rx=0 tx=0).stop 2024-09-10T21:17:54.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 shutdown_connections 2024-09-10T21:17:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f480b52e0 0x7f5f480b77a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f5c108060 0x7f5f5c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f5c1076e0 0x7f5f5c1997d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 --2- 172.21.15.33:0/3783145076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f5c101cd0 0x7f5f5c199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 >> 172.21.15.33:0/3783145076 conn(0x7f5f5c0fd740 msgr2=0x7f5f5c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 shutdown_connections 2024-09-10T21:17:54.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:54.144+0000 7f5f63b85640 1 -- 172.21.15.33:0/3783145076 wait complete. 2024-09-10T21:17:55.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 50117 52.29 MB/sec warmup 55 sec latency 71.919 ms 2024-09-10T21:17:55.193 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:17:55.517 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:55 smithi104 ceph-mon[35325]: pgmap v764: 97 pgs: 97 active+clean; 737 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.18k op/s 2024-09-10T21:17:55.858 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:55 smithi033 ceph-mon[30860]: pgmap v764: 97 pgs: 97 active+clean; 737 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.18k op/s 2024-09-10T21:17:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:55 smithi174 ceph-mon[35635]: pgmap v764: 97 pgs: 97 active+clean; 737 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.18k op/s 2024-09-10T21:17:56.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 50903 52.09 MB/sec warmup 56 sec latency 82.090 ms 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 -- 172.21.15.33:0/1287475834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c10d140 msgr2=0x7f248c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 --2- 172.21.15.33:0/1287475834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c10d140 0x7f248c10f550 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2484009ca0 tx=0x7f248402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 -- 172.21.15.33:0/1287475834 shutdown_connections 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 --2- 172.21.15.33:0/1287475834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c10d140 0x7f248c10f550 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 --2- 172.21.15.33:0/1287475834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f248c104590 0x7f248c10cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 --2- 172.21.15.33:0/1287475834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f248c103b90 0x7f248c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 -- 172.21.15.33:0/1287475834 >> 172.21.15.33:0/1287475834 conn(0x7f248c0fd700 msgr2=0x7f248c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 -- 172.21.15.33:0/1287475834 shutdown_connections 2024-09-10T21:17:56.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.187+0000 7f2493a4c640 1 -- 172.21.15.33:0/1287475834 wait complete. 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.188+0000 7f2493a4c640 1 Processor -- start 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.188+0000 7f2493a4c640 1 -- start start 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2493a4c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c103b90 0x7f248c1034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2493a4c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f248c104590 0x7f248c101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2493a4c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f248c10d140 0x7f248c102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2493a4c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112d60 con 0x7f248c103b90 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2493a4c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112be0 con 0x7f248c10d140 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2493a4c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f248c112ee0 con 0x7f248c104590 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f24917e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c103b90 0x7f248c1034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2491fe9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f248c10d140 0x7f248c102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f24917e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c103b90 0x7f248c1034c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45568/0 (socket says 172.21.15.33:45568) 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f24917e8640 1 -- 172.21.15.33:0/3986518316 learned_addr learned my addr 172.21.15.33:0/3986518316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:56.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.189+0000 7f2490fe7640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f248c104590 0x7f248c101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2490fe7640 1 -- 172.21.15.33:0/3986518316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f248c10d140 msgr2=0x7f248c102040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2490fe7640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f248c10d140 0x7f248c102040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2490fe7640 1 -- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c103b90 msgr2=0x7f248c1034c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2490fe7640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c103b90 0x7f248c1034c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2490fe7640 1 -- 172.21.15.33:0/3986518316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248c1abbf0 con 0x7f248c104590 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2491fe9640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f248c10d140 0x7f248c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2490fe7640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f248c104590 0x7f248c101a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9f0 tx=0x7f247c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f24827fc640 1 -- 172.21.15.33:0/3986518316 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c00ccc0 con 0x7f248c104590 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248c1abdc0 con 0x7f248c104590 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f24827fc640 1 -- 172.21.15.33:0/3986518316 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247c005020 con 0x7f248c104590 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f24827fc640 1 -- 172.21.15.33:0/3986518316 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c004a00 con 0x7f248c104590 2024-09-10T21:17:56.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.190+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f248c1ac270 con 0x7f248c104590 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.193+0000 7f2495e21740 1 Processor -- start 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2495e21740 1 -- start start 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2495e21740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d07c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2495e21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d088840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2495e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2495e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56207cff2ea0 con 0x56207d088d80 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2495e21740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56207cfe22d0 con 0x56207d07c4d0 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2495e21740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56207cfe60e0 con 0x56207d07cdf0 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f24827fc640 1 -- 172.21.15.33:0/3986518316 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f247c004ba0 con 0x7f248c104590 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2491fe9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d07c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f24917e8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d088840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2491fe9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d07c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38678/0 (socket says 172.21.15.33:38678) 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2491fe9640 1 -- 172.21.15.33:0/1600983485 learned_addr learned my addr 172.21.15.33:0/1600983485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2490fe7640 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.194+0000 7f2491fe9640 1 -- 172.21.15.33:0/1600983485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 msgr2=0x56207d088840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.195+0000 7f2491fe9640 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d088840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.195+0000 7f2491fe9640 1 -- 172.21.15.33:0/1600983485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 msgr2=0x56207d08b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.195+0000 7f2491fe9640 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.195+0000 7f2491fe9640 1 -- 172.21.15.33:0/1600983485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207cd40650 con 0x56207d07c4d0 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.195+0000 7f2490fe7640 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.195+0000 7f2491fe9640 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d07c8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2484035000 tx=0x7f248402f310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90caa3110c0735fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.196+0000 7f24777fe640 1 -- 172.21.15.33:0/1600983485 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484031450 con 0x56207d07c4d0 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.196+0000 7f24777fe640 1 -- 172.21.15.33:0/1600983485 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2484036040 con 0x56207d07c4d0 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.196+0000 7f24777fe640 1 -- 172.21.15.33:0/1600983485 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484004320 con 0x56207d07c4d0 2024-09-10T21:17:56.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.196+0000 7f24827fc640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.196+0000 7f24917e8640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.196+0000 7f24827fc640 1 -- 172.21.15.33:0/3986518316 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f247c0e5ad0 con 0x7f248c104590 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f24917e8640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24780b52e0 0x7f24780b77a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x56207d07b1e0 tx=0x7f2488005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 -- 172.21.15.33:0/1600983485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 msgr2=0x56207d07c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d07c8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2484035000 tx=0x7f248402f310 comp rx=0 tx=0).stop 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 -- 172.21.15.33:0/1600983485 shutdown_connections 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d088840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 --2- 172.21.15.33:0/1600983485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d07c8b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 -- 172.21.15.33:0/1600983485 >> 172.21.15.33:0/1600983485 conn(0x56207d07a300 msgr2=0x56207d07b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.197+0000 7f2495e21740 1 -- 172.21.15.33:0/1600983485 shutdown_connections 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.198+0000 7f2495e21740 1 -- 172.21.15.33:0/1600983485 wait complete. 2024-09-10T21:17:56.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.198+0000 7f2495e21740 1 -- start start 2024-09-10T21:17:56.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.199+0000 7f2495e21740 1 Processor -- start 2024-09-10T21:17:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2495e21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07c4d0 0x56207d0886b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2495e21740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07cdf0 0x56207d085b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2495e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d086180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2495e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56207ce43700 con 0x56207d088d80 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2495e21740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56207cf7d160 con 0x56207d07cdf0 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2495e21740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56207d08b820 con 0x56207d07c4d0 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2495e21740 10 client.0 _ll_register_callbacks cb 0x56207cf969e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07cdf0 0x56207d085b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07cdf0 0x56207d085b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38682/0 (socket says 172.21.15.33:38682) 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 -- 172.21.15.33:0/3127378126 learned_addr learned my addr 172.21.15.33:0/3127378126 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f24917e8640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07c4d0 0x56207d0886b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2491fe9640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d086180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 -- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07c4d0 msgr2=0x56207d0886b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07c4d0 0x56207d0886b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 -- 172.21.15.33:0/3127378126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 msgr2=0x56207d086180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d086180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 -- 172.21.15.33:0/3127378126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207cfaac90 con 0x56207d07cdf0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2491fe9640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d086180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2490fe7640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07cdf0 0x56207d085b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f247c009230 tx=0x7f247c0a8130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953fbdacc27ebfa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.200+0000 7f2474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.201+0000 7f2474ff9640 1 -- 172.21.15.33:0/3127378126 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247c01c070 con 0x56207d07cdf0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.201+0000 7f2474ff9640 1 -- 172.21.15.33:0/3127378126 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f247c0cf0c0 con 0x56207d07cdf0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.201+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2484036040 con 0x56207d07cdf0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.201+0000 7f2474ff9640 1 -- 172.21.15.33:0/3127378126 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f247c0aba00 con 0x56207d07cdf0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.201+0000 7f2495e21740 10 client.27416 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.201+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56207d097190 con 0x56207d07cdf0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.201+0000 7f2474ff9640 1 -- 172.21.15.33:0/3127378126 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2484036040 con 0x56207d07cdf0 2024-09-10T21:17:56.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f2474ff9640 1 -- 172.21.15.33:0/3127378126 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f247c0b5390 con 0x56207d07cdf0 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f2495e21740 10 client.27416 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f2495e21740 10 client.27416 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f2495e21740 20 client.27416 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f2495e21740 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d098be0 0x56207d09b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f2495e21740 4 client.27416 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56207cff4df0 con 0x56207d098be0 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.202+0000 7f24917e8640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d098be0 0x56207d09b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.203+0000 7f24917e8640 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d098be0 0x56207d09b000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ee0d09e17bd70600 server_cookie=25ecd47aa730c51c in_seq=0 out_seq=0 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.203+0000 7f2474ff9640 10 client.27416 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.203+0000 7f2474ff9640 1 -- 172.21.15.33:0/3127378126 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56207cff4df0 con 0x56207d098be0 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.203+0000 7f2474ff9640 10 client.27416 handle_command_reply: tid=0 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.203+0000 7f2495e21740 1 client.27416 shutdown 2024-09-10T21:17:56.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07cdf0 msgr2=0x56207d085b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07cdf0 0x56207d085b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f247c009230 tx=0x7f247c0a8130 comp rx=0 tx=0).stop 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 shutdown_connections 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d098be0 0x56207d09b000 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d086180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07cdf0 0x56207d085b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 --2- 172.21.15.33:0/3127378126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07c4d0 0x56207d0886b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2474ff9640 0 client.27416 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 >> 172.21.15.33:0/3127378126 conn(0x56207d07a300 msgr2=0x56207d07fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 shutdown_connections 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 1 -- 172.21.15.33:0/3127378126 wait complete. 2024-09-10T21:17:56.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.204+0000 7f2495e21740 20 client.27416 trim_cache size 0 max 16384 2024-09-10T21:17:56.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 Processor -- start 2024-09-10T21:17:56.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 -- start start 2024-09-10T21:17:56.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d094a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d08a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1b49d0 con 0x56207d088d80 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1b6720 con 0x56207d07c4d0 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.221+0000 7f2495e21740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1b50d0 con 0x56207d07cdf0 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2490fe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d08a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2491fe9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2490fe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d08a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59558/0 (socket says 172.21.15.33:59558) 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2490fe7640 1 -- 172.21.15.33:0/1002300193 learned_addr learned my addr 172.21.15.33:0/1002300193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f24917e8640 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d094a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2491fe9640 1 -- 172.21.15.33:0/1002300193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 msgr2=0x56207d08a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2491fe9640 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d08a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2491fe9640 1 -- 172.21.15.33:0/1002300193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 msgr2=0x56207d094a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2491fe9640 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d094a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2491fe9640 1 -- 172.21.15.33:0/1002300193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56207d07cc20 con 0x56207d088d80 2024-09-10T21:17:56.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2490fe7640 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d08a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:56.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f24917e8640 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d094a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:17:56.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.222+0000 7f2491fe9640 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08a6b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2484005e30 tx=0x7f2484003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5df1a88fd2bc39 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:56.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.223+0000 7f2476ffd640 1 -- 172.21.15.33:0/1002300193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484003970 con 0x56207d088d80 2024-09-10T21:17:56.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.223+0000 7f2476ffd640 1 -- 172.21.15.33:0/1002300193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2484036040 con 0x56207d088d80 2024-09-10T21:17:56.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.223+0000 7f2476ffd640 1 -- 172.21.15.33:0/1002300193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f248402fba0 con 0x56207d088d80 2024-09-10T21:17:56.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 -- 172.21.15.33:0/1002300193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 msgr2=0x56207d08a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08a6b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2484005e30 tx=0x7f2484003f40 comp rx=0 tx=0).stop 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 -- 172.21.15.33:0/1002300193 shutdown_connections 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d088d80 0x56207d08a6b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d08a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 --2- 172.21.15.33:0/1002300193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d07c4d0 0x56207d094a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 -- 172.21.15.33:0/1002300193 >> 172.21.15.33:0/1002300193 conn(0x56207d07a300 msgr2=0x56207d084380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.224+0000 7f2495e21740 1 -- 172.21.15.33:0/1002300193 shutdown_connections 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.225+0000 7f2495e21740 1 -- 172.21.15.33:0/1002300193 wait complete. 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.225+0000 7f2495e21740 1 -- start start 2024-09-10T21:17:56.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.225+0000 7f2495e21740 1 Processor -- start 2024-09-10T21:17:56.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2495e21740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d07c4d0 0x56207d090ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2495e21740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d091000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2495e21740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d088d80 0x56207d08ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f24917e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d07c4d0 0x56207d090ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2495e21740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1b5fc0 con 0x56207d07c4d0 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2490fe7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d091000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f24917e8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d07c4d0 0x56207d090ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45612/0 (socket says 172.21.15.33:45612) 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f24917e8640 1 -- 172.21.15.33:0/542810313 learned_addr learned my addr 172.21.15.33:0/542810313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1b5830 con 0x56207d088d80 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f248c1b2990 con 0x56207d07cdf0 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2491fe9640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d088d80 0x56207d08ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2495e21740 10 client.0 _ll_register_callbacks cb 0x7f247c0ea040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2491fe9640 1 -- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 msgr2=0x56207d091000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2491fe9640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d091000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2491fe9640 1 -- 172.21.15.33:0/542810313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d07c4d0 msgr2=0x56207d090ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2491fe9640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d07c4d0 0x56207d090ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f2491fe9640 1 -- 172.21.15.33:0/542810313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56207d0915f0 con 0x56207d088d80 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.226+0000 7f24917e8640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d07c4d0 0x56207d090ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.227+0000 7f2490fe7640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d091000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.227+0000 7f2491fe9640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d088d80 0x56207d08ba70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f248402ade0 tx=0x7f2484003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a3f86bb5888d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.227+0000 7f2474ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.227+0000 7f2474ff9640 1 -- 172.21.15.33:0/542810313 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2484031860 con 0x56207d088d80 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.227+0000 7f2474ff9640 1 -- 172.21.15.33:0/542810313 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f248402f7e0 con 0x56207d088d80 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.227+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2484036040 con 0x56207d088d80 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2474ff9640 1 -- 172.21.15.33:0/542810313 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f248402f9e0 con 0x56207d088d80 2024-09-10T21:17:56.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2495e21740 10 client.27422 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56207d105e40 con 0x56207d088d80 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2474ff9640 1 -- 172.21.15.33:0/542810313 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2484003fb0 con 0x56207d088d80 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2474ff9640 1 -- 172.21.15.33:0/542810313 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f2484060030 con 0x56207d088d80 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2495e21740 10 client.27422 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2495e21740 10 client.27422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2495e21740 20 client.27422 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.228+0000 7f2495e21740 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d1062c0 0x56207d1086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.229+0000 7f2495e21740 4 client.27422 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:56.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.229+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2484036040 con 0x56207d1062c0 2024-09-10T21:17:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.229+0000 7f24917e8640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d1062c0 0x56207d1086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.230+0000 7f24917e8640 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d1062c0 0x56207d1086e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6590be626d452066 server_cookie=9c003b1c3653174d in_seq=0 out_seq=0 2024-09-10T21:17:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.230+0000 7f2474ff9640 10 client.27422 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2474ff9640 1 -- 172.21.15.33:0/542810313 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2484036040 con 0x56207d1062c0 2024-09-10T21:17:56.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2474ff9640 10 client.27422 handle_command_reply: tid=0 2024-09-10T21:17:56.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 client.27422 shutdown 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d088d80 msgr2=0x56207d08ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d088d80 0x56207d08ba70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f248402ade0 tx=0x7f2484003ba0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 shutdown_connections 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56207d088d80 0x56207d08ba70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56207d07cdf0 0x56207d091000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56207d1062c0 0x56207d1086e0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 --2- 172.21.15.33:0/542810313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56207d07c4d0 0x56207d090ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2474ff9640 0 client.27422 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.233+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 >> 172.21.15.33:0/542810313 conn(0x56207d07a300 msgr2=0x56207d079420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.234+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 shutdown_connections 2024-09-10T21:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.234+0000 7f2495e21740 1 -- 172.21.15.33:0/542810313 wait complete. 2024-09-10T21:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.234+0000 7f2495e21740 20 client.27422 trim_cache size 0 max 16384 2024-09-10T21:17:56.240 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24780b52e0 msgr2=0x7f24780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24780b52e0 0x7f24780b77a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x56207d07b1e0 tx=0x7f2488005f90 comp rx=0 tx=0).stop 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f248c104590 msgr2=0x7f248c101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f248c104590 0x7f248c101a10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f247c00e9f0 tx=0x7f247c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 shutdown_connections 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24780b52e0 0x7f24780b77a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f248c10d140 0x7f248c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f248c104590 0x7f248c101a10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 --2- 172.21.15.33:0/3986518316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f248c103b90 0x7f248c1034c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 >> 172.21.15.33:0/3986518316 conn(0x7f248c0fd700 msgr2=0x7f248c0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 shutdown_connections 2024-09-10T21:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:56.235+0000 7f2493a4c640 1 -- 172.21.15.33:0/3986518316 wait complete. 2024-09-10T21:17:57.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 51880 52.08 MB/sec warmup 57 sec latency 155.004 ms 2024-09-10T21:17:57.263 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:17:57.588 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:17:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:57 smithi033 ceph-mon[30860]: pgmap v765: 97 pgs: 97 active+clean; 743 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.10k op/s 2024-09-10T21:17:57.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:57 smithi104 ceph-mon[35325]: pgmap v765: 97 pgs: 97 active+clean; 743 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.10k op/s 2024-09-10T21:17:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:57 smithi174 ceph-mon[35635]: pgmap v765: 97 pgs: 97 active+clean; 743 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 1.10k op/s 2024-09-10T21:17:58.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 52896 52.15 MB/sec warmup 58 sec latency 165.501 ms 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 -- 172.21.15.33:0/899033381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 msgr2=0x7f6938107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 --2- 172.21.15.33:0/899033381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 0x7f6938107ae0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f692c009900 tx=0x7f692c02a410 comp rx=0 tx=0).stop 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 -- 172.21.15.33:0/899033381 shutdown_connections 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 --2- 172.21.15.33:0/899033381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6938108020 0x7f693810e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 --2- 172.21.15.33:0/899033381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 0x7f6938107ae0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 --2- 172.21.15.33:0/899033381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 0x7f6938102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 -- 172.21.15.33:0/899033381 >> 172.21.15.33:0/899033381 conn(0x7f69380fd700 msgr2=0x7f69380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 -- 172.21.15.33:0/899033381 shutdown_connections 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.273+0000 7f693ccaa640 1 -- 172.21.15.33:0/899033381 wait complete. 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 Processor -- start 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 -- start start 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 0x7f6938108810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 0x7f6938108d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6938108020 0x7f6938109290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f6935d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 0x7f6938108d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 0x7f6938108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 0x7f6938108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38730/0 (socket says 172.21.15.33:38730) 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f6935d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 0x7f6938108d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45628/0 (socket says 172.21.15.33:45628) 2024-09-10T21:17:58.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693659c640 1 -- 172.21.15.33:0/2010303589 learned_addr learned my addr 172.21.15.33:0/2010303589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111c70 con 0x7f69381076a0 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111af0 con 0x7f6938101c90 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.275+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6938111df0 con 0x7f6938108020 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f693659c640 1 -- 172.21.15.33:0/2010303589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6938108020 msgr2=0x7f6938109290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f693659c640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6938108020 0x7f6938109290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f693659c640 1 -- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 msgr2=0x7f6938108d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f693659c640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 0x7f6938108d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f693659c640 1 -- 172.21.15.33:0/2010303589 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69381aba70 con 0x7f6938101c90 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f693659c640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 0x7f6938108810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f692800e9f0 tx=0x7f692800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f69277fe640 1 -- 172.21.15.33:0/2010303589 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692800cd80 con 0x7f6938101c90 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f69277fe640 1 -- 172.21.15.33:0/2010303589 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69280108e0 con 0x7f6938101c90 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f69277fe640 1 -- 172.21.15.33:0/2010303589 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928018980 con 0x7f6938101c90 2024-09-10T21:17:58.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.276+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69381abd60 con 0x7f6938101c90 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.277+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69381ac1c0 con 0x7f6938101c90 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.281+0000 7f693f07f740 1 Processor -- start 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.281+0000 7f69277fe640 1 -- 172.21.15.33:0/2010303589 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6928010430 con 0x7f6938101c90 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.281+0000 7f693f07f740 1 -- start start 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.281+0000 7f693f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a7d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f693f07f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a89840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f693f07f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f6936d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a7d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f6936d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a7d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59596/0 (socket says 172.21.15.33:59596) 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f693659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a89840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b2d59f3ea0 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2d59e32d0 con 0x55b2d5a89d80 2024-09-10T21:17:58.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b2d59e70e0 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f6936d9d640 1 -- 172.21.15.33:0/1953954439 learned_addr learned my addr 172.21.15.33:0/1953954439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f6935d9b640 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.282+0000 7f69277fe640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69200b53e0 0x7f69200b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6935d9b640 1 -- 172.21.15.33:0/1953954439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a7d4d0 msgr2=0x55b2d5a7d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6935d9b640 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a7d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6935d9b640 1 -- 172.21.15.33:0/1953954439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 msgr2=0x55b2d5a89840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6935d9b640 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a89840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6935d9b640 1 -- 172.21.15.33:0/1953954439 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d5741650 con 0x55b2d5a89d80 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f69277fe640 1 -- 172.21.15.33:0/2010303589 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f69280e5f90 con 0x7f6938101c90 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6936d9d640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69200b53e0 0x7f69200b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6935d9b640 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8c170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f692c009230 tx=0x7f692c032b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28febfbcb5c18f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6924ff9640 1 -- 172.21.15.33:0/1953954439 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c040070 con 0x55b2d5a89d80 2024-09-10T21:17:58.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.283+0000 7f6936d9d640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69200b53e0 0x7f69200b78a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f69300049d0 tx=0x7f6930009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.284+0000 7f6924ff9640 1 -- 172.21.15.33:0/1953954439 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f692c004030 con 0x55b2d5a89d80 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.284+0000 7f6924ff9640 1 -- 172.21.15.33:0/1953954439 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c035040 con 0x55b2d5a89d80 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a89d80 msgr2=0x55b2d5a8c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8c170 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f692c009230 tx=0x7f692c032b20 comp rx=0 tx=0).stop 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 shutdown_connections 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8c170 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a89840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 --2- 172.21.15.33:0/1953954439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a7d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 >> 172.21.15.33:0/1953954439 conn(0x55b2d5a7b300 msgr2=0x55b2d5a7c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 shutdown_connections 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.285+0000 7f693f07f740 1 -- 172.21.15.33:0/1953954439 wait complete. 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.286+0000 7f693f07f740 1 -- start start 2024-09-10T21:17:58.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.286+0000 7f693f07f740 1 Processor -- start 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.286+0000 7f693f07f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a896d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693f07f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a86b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a896d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a87180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a896d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38750/0 (socket says 172.21.15.33:38750) 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693659c640 1 -- 172.21.15.33:0/1983844867 learned_addr learned my addr 172.21.15.33:0/1983844867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b2d5844700 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b2d597e160 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b2d5a8c9a0 con 0x55b2d5a89d80 2024-09-10T21:17:58.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f693f07f740 10 client.0 _ll_register_callbacks cb 0x55b2d59979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f6936d9d640 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a87180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.287+0000 7f6935d9b640 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a86b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f693659c640 1 -- 172.21.15.33:0/1983844867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 msgr2=0x55b2d5a87180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f693659c640 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a87180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f693659c640 1 -- 172.21.15.33:0/1983844867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 msgr2=0x55b2d5a86b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f693659c640 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a86b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f693659c640 1 -- 172.21.15.33:0/1983844867 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d59abc90 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f693659c640 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a896d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f69280ac6a0 tx=0x7f6928010cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d16735ec5e93c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f691dffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f691dffb640 1 -- 172.21.15.33:0/1983844867 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69280abe80 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f691dffb640 1 -- 172.21.15.33:0/1983844867 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6928018b20 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.288+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692c004030 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.289+0000 7f691dffb640 1 -- 172.21.15.33:0/1983844867 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f69280ac220 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.289+0000 7f693f07f740 10 client.27434 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.289+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55b2d5a983e0 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.289+0000 7f691dffb640 1 -- 172.21.15.33:0/1983844867 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f692c004030 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.289+0000 7f691dffb640 1 -- 172.21.15.33:0/1983844867 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f69280e9020 con 0x55b2d5a7d4d0 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.290+0000 7f693f07f740 10 client.27434 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.290+0000 7f693f07f740 10 client.27434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.290+0000 7f693f07f740 20 client.27434 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.290+0000 7f693f07f740 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5a99cb0 0x55b2d5a9c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.290+0000 7f693f07f740 4 client.27434 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.290+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2d59f5df0 con 0x55b2d5a99cb0 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.290+0000 7f6935d9b640 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5a99cb0 0x55b2d5a9c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.291+0000 7f691dffb640 10 client.27434 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:58.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.291+0000 7f6935d9b640 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5a99cb0 0x55b2d5a9c170 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c59984a8ad45ed62 server_cookie=1042ecedf2969921 in_seq=0 out_seq=0 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.292+0000 7f691dffb640 1 -- 172.21.15.33:0/1983844867 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b2d59f5df0 con 0x55b2d5a99cb0 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f691dffb640 10 client.27434 handle_command_reply: tid=0 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 client.27434 shutdown 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7d4d0 msgr2=0x55b2d5a896d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a896d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f69280ac6a0 tx=0x7f6928010cc0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 shutdown_connections 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5a99cb0 0x55b2d5a9c170 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a87180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a86b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 --2- 172.21.15.33:0/1983844867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7d4d0 0x55b2d5a896d0 secure :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f69280ac6a0 tx=0x7f6928010cc0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f691dffb640 0 client.27434 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 >> 172.21.15.33:0/1983844867 conn(0x55b2d5a7b300 msgr2=0x55b2d5a80ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 shutdown_connections 2024-09-10T21:17:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.293+0000 7f693f07f740 1 -- 172.21.15.33:0/1983844867 wait complete. 2024-09-10T21:17:58.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.294+0000 7f693f07f740 20 client.27434 trim_cache size 0 max 16384 2024-09-10T21:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 Processor -- start 2024-09-10T21:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 -- start start 2024-09-10T21:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a7c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a8b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69381b4a60 con 0x55b2d5a89d80 2024-09-10T21:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69381b67b0 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.310+0000 7f693f07f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69381b5160 con 0x55b2d5a85bc0 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45642/0 (socket says 172.21.15.33:45642) 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 -- 172.21.15.33:0/549668690 learned_addr learned my addr 172.21.15.33:0/549668690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f693659c640 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a7c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 -- 172.21.15.33:0/549668690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a85bc0 msgr2=0x55b2d5a8b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a8b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 -- 172.21.15.33:0/549668690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 msgr2=0x55b2d5a7c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a7c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 -- 172.21.15.33:0/549668690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2d5a89a40 con 0x55b2d5a89d80 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f693659c640 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a7c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f6935d9b640 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8b170 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f692c02a930 tx=0x7f692c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7253f6db83996b77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f691ffff640 1 -- 172.21.15.33:0/549668690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c035040 con 0x55b2d5a89d80 2024-09-10T21:17:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.311+0000 7f691ffff640 1 -- 172.21.15.33:0/549668690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f692c050020 con 0x55b2d5a89d80 2024-09-10T21:17:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.312+0000 7f691ffff640 1 -- 172.21.15.33:0/549668690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c040040 con 0x55b2d5a89d80 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.312+0000 7f693f07f740 1 -- 172.21.15.33:0/549668690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a89d80 msgr2=0x55b2d5a8b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.312+0000 7f693f07f740 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8b170 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f692c02a930 tx=0x7f692c009260 comp rx=0 tx=0).stop 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 -- 172.21.15.33:0/549668690 shutdown_connections 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a8b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8b170 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 --2- 172.21.15.33:0/549668690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a7c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 -- 172.21.15.33:0/549668690 >> 172.21.15.33:0/549668690 conn(0x55b2d5a7b300 msgr2=0x55b2d5a7bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 -- 172.21.15.33:0/549668690 shutdown_connections 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 -- 172.21.15.33:0/549668690 wait complete. 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 -- start start 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.313+0000 7f693f07f740 1 Processor -- start 2024-09-10T21:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693f07f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a91f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693f07f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a924a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693f07f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693f07f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f69381b6050 con 0x55b2d5a85bc0 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f6935d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a924a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693f07f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f69381b58c0 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693f07f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69381b2a20 con 0x55b2d5a89d80 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a91f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f6935d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a924a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45646/0 (socket says 172.21.15.33:45646) 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f6935d9b640 1 -- 172.21.15.33:0/63345039 learned_addr learned my addr 172.21.15.33:0/63345039 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f6936d9d640 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693659c640 1 -- 172.21.15.33:0/63345039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 msgr2=0x55b2d5a8cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693659c640 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693659c640 1 -- 172.21.15.33:0/63345039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a85bc0 msgr2=0x55b2d5a924a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693f07f740 10 client.0 _ll_register_callbacks cb 0x7f6928009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693659c640 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a924a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.314+0000 7f693659c640 1 -- 172.21.15.33:0/63345039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2d5a8d4a0 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f6935d9b640 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a924a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f692c050020 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f693659c640 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a91f40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f69280ac410 tx=0x7f69280abdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2f06aefe03ad02d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:17:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f691dffb640 10 client.27440 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:17:58.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f691dffb640 1 -- 172.21.15.33:0/63345039 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69280a8260 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f691dffb640 1 -- 172.21.15.33:0/63345039 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f69280b2380 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f691dffb640 1 -- 172.21.15.33:0/63345039 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f69280b2580 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f693f07f740 10 client.27440 fetch_fsmap learned FSMap version 69 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.315+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=69}) -- 0x55b2d5b076c0 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f691dffb640 1 -- 172.21.15.33:0/63345039 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f69280cfa60 con 0x55b2d5a7ddf0 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f693f07f740 10 client.27440 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f693f07f740 10 client.27440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f693f07f740 20 client.27440 populate_metadata read hostname 'smithi033' 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f693f07f740 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5b07df0 0x55b2d5b0a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f693f07f740 4 client.27440 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b2d59f18a0 con 0x55b2d5b07df0 2024-09-10T21:17:58.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f6935d9b640 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5b07df0 0x55b2d5b0a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f6935d9b640 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5b07df0 0x55b2d5b0a220 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea74236133d85f3 server_cookie=dab21b7adb8f535f in_seq=0 out_seq=0 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.316+0000 7f691dffb640 10 client.27440 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f691dffb640 1 -- 172.21.15.33:0/63345039 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b2d59f18a0 con 0x55b2d5b07df0 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f691dffb640 10 client.27440 handle_command_reply: tid=0 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 client.27440 shutdown 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 msgr2=0x55b2d5a91f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a91f40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f69280ac410 tx=0x7f69280abdb0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 shutdown_connections 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b2d5b07df0 0x55b2d5b0a220 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b2d5a89d80 0x55b2d5a8cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b2d5a85bc0 0x55b2d5a924a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 --2- 172.21.15.33:0/63345039 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b2d5a7ddf0 0x55b2d5a91f40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 >> 172.21.15.33:0/63345039 conn(0x55b2d5a7b300 msgr2=0x55b2d5a7c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f691dffb640 0 client.27440 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:17:58.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 shutdown_connections 2024-09-10T21:17:58.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 1 -- 172.21.15.33:0/63345039 wait complete. 2024-09-10T21:17:58.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.317+0000 7f693f07f740 20 client.27440 trim_cache size 0 max 16384 2024-09-10T21:17:58.326 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69200b53e0 msgr2=0x7f69200b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69200b53e0 0x7f69200b78a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f69300049d0 tx=0x7f6930009500 comp rx=0 tx=0).stop 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 msgr2=0x7f6938108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 0x7f6938108810 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f692800e9f0 tx=0x7f692800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 shutdown_connections 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f69200b53e0 0x7f69200b78a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6938108020 0x7f6938109290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69381076a0 0x7f6938108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 --2- 172.21.15.33:0/2010303589 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6938101c90 0x7f6938108810 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:17:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 >> 172.21.15.33:0/2010303589 conn(0x7f69380fd700 msgr2=0x7f693806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:17:58.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 shutdown_connections 2024-09-10T21:17:58.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:17:58.318+0000 7f693ccaa640 1 -- 172.21.15.33:0/2010303589 wait complete. 2024-09-10T21:17:59.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 53890 52.22 MB/sec warmup 59 sec latency 80.799 ms 2024-09-10T21:17:59.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:17:59 smithi104 ceph-mon[35325]: pgmap v766: 97 pgs: 97 active+clean; 750 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-09-10T21:17:59.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:17:59 smithi174 ceph-mon[35635]: pgmap v766: 97 pgs: 97 active+clean; 750 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-09-10T21:17:59.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:17:59 smithi033 ceph-mon[30860]: pgmap v766: 97 pgs: 97 active+clean; 750 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.08k op/s 2024-09-10T21:18:00.044 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:18:00.044 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:18:00.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 54784 52.09 MB/sec warmup 60 sec latency 64.537 ms 2024-09-10T21:18:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 -- 172.21.15.33:0/2356245885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0400108040 msgr2=0x7f040010e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 --2- 172.21.15.33:0/2356245885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0400108040 0x7f040010e840 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009be0 tx=0x7f03f802a6c0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 -- 172.21.15.33:0/2356245885 shutdown_connections 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 --2- 172.21.15.33:0/2356245885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0400108040 0x7f040010e840 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 --2- 172.21.15.33:0/2356245885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04001076c0 0x7f0400107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 --2- 172.21.15.33:0/2356245885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400101cb0 0x7f0400102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 -- 172.21.15.33:0/2356245885 >> 172.21.15.33:0/2356245885 conn(0x7f04000fd700 msgr2=0x7f04000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 -- 172.21.15.33:0/2356245885 shutdown_connections 2024-09-10T21:18:01.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.975+0000 7f04062f8640 1 -- 172.21.15.33:0/2356245885 wait complete. 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.976+0000 7f04062f8640 1 Processor -- start 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.976+0000 7f04062f8640 1 -- start start 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.976+0000 7f04062f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0400101cb0 0x7f0400199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f04062f8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04001076c0 0x7f0400199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f04062f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400108040 0x7f040019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f04062f8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111d00 con 0x7f0400101cb0 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f04062f8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111b80 con 0x7f04001076c0 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f04062f8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0400111e80 con 0x7f0400108040 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0400101cb0 0x7f0400199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04001076c0 0x7f0400199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04001076c0 0x7f0400199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54760/0 (socket says 172.21.15.33:54760) 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 -- 172.21.15.33:0/1754298633 learned_addr learned my addr 172.21.15.33:0/1754298633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03fffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0400101cb0 0x7f0400199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47198/0 (socket says 172.21.15.33:47198) 2024-09-10T21:18:01.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f0404895640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400108040 0x7f040019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 -- 172.21.15.33:0/1754298633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400108040 msgr2=0x7f040019eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400108040 0x7f040019eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 -- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0400101cb0 msgr2=0x7f0400199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0400101cb0 0x7f0400199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 -- 172.21.15.33:0/1754298633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040019f2f0 con 0x7f04001076c0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.977+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04001076c0 0x7f0400199780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03f400e9c0 tx=0x7f03f400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.978+0000 7f03fd7fa640 1 -- 172.21.15.33:0/1754298633 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f400ccc0 con 0x7f04001076c0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.978+0000 7f03fd7fa640 1 -- 172.21.15.33:0/1754298633 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f40054d0 con 0x7f04001076c0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.978+0000 7f03fd7fa640 1 -- 172.21.15.33:0/1754298633 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4010430 con 0x7f04001076c0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.978+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040019f5e0 con 0x7f04001076c0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.978+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04001a5ca0 con 0x7f04001076c0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.981+0000 7f03fd7fa640 1 -- 172.21.15.33:0/1754298633 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f03f4004f80 con 0x7f04001076c0 2024-09-10T21:18:01.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.981+0000 7f04086cd740 1 Processor -- start 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f04086cd740 1 -- start start 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f04086cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e39960 0x5576d1e39d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f04086cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49280 0x5576d1e496c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f04086cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 0x5576d1e535f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f04086cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576d1db2e60 con 0x5576d1e39960 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f04086cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5576d1da2290 con 0x5576d1e49c00 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f04086cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5576d1da60a0 con 0x5576d1e49280 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f0404895640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e39960 0x5576d1e39d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.982+0000 7f03fffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49280 0x5576d1e496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49280 0x5576d1e496c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49080/0 (socket says 172.21.15.33:49080) 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 -- 172.21.15.33:0/725690663 learned_addr learned my addr 172.21.15.33:0/725690663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fd7fa640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03e80b53e0 0x7f03e80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fd7fa640 1 -- 172.21.15.33:0/1754298633 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f03f40e5f30 con 0x7f04001076c0 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 -- 172.21.15.33:0/725690663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 msgr2=0x5576d1e535f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 --2- 172.21.15.33:0/725690663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 0x5576d1e535f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 -- 172.21.15.33:0/725690663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e39960 msgr2=0x5576d1e39d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 --2- 172.21.15.33:0/725690663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e39960 0x5576d1e39d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 -- 172.21.15.33:0/725690663 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d1b00930 con 0x5576d1e49280 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f0404895640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03e80b53e0 0x7f03e80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.983+0000 7f03fffff640 1 --2- 172.21.15.33:0/725690663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49280 0x5576d1e496c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03f000b760 tx=0x7f03f000bc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee0a00506d110f13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.984+0000 7f03e67fc640 1 -- 172.21.15.33:0/725690663 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0003dd0 con 0x5576d1e49280 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.984+0000 7f03e67fc640 1 -- 172.21.15.33:0/725690663 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f0015070 con 0x5576d1e49280 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.984+0000 7f03e67fc640 1 -- 172.21.15.33:0/725690663 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0004d70 con 0x5576d1e49280 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 -- 172.21.15.33:0/725690663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49280 msgr2=0x5576d1e496c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 --2- 172.21.15.33:0/725690663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49280 0x5576d1e496c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f03f000b760 tx=0x7f03f000bc20 comp rx=0 tx=0).stop 2024-09-10T21:18:01.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f0404895640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03e80b53e0 0x7f03e80b78a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009370 tx=0x7f03f802a6c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 -- 172.21.15.33:0/725690663 shutdown_connections 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 --2- 172.21.15.33:0/725690663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 0x5576d1e535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 --2- 172.21.15.33:0/725690663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49280 0x5576d1e496c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 --2- 172.21.15.33:0/725690663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e39960 0x5576d1e39d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 -- 172.21.15.33:0/725690663 >> 172.21.15.33:0/725690663 conn(0x5576d1e3be80 msgr2=0x5576d1e3c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 -- 172.21.15.33:0/725690663 shutdown_connections 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.985+0000 7f04086cd740 1 -- 172.21.15.33:0/725690663 wait complete. 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.986+0000 7f04086cd740 1 -- start start 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.986+0000 7f04086cd740 1 Processor -- start 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.986+0000 7f04086cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e50b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f04086cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f04086cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f04086cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5576d1c03890 con 0x5576d1e49280 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f04086cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5576d1d3cd40 con 0x5576d1e39960 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f04086cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5576d1e53ce0 con 0x5576d1e49c00 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e50b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e50b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54788/0 (socket says 172.21.15.33:54788) 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 -- 172.21.15.33:0/1399231138 learned_addr learned my addr 172.21.15.33:0/1399231138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f04086cd740 10 client.0 _ll_register_callbacks cb 0x5576d1da0c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f0404895640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 -- 172.21.15.33:0/1399231138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 msgr2=0x5576d1e515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 -- 172.21.15.33:0/1399231138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 msgr2=0x5576d1e51070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 -- 172.21.15.33:0/1399231138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d1e5d0b0 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f0404895640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03fffff640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e50b30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f03f00116a0 tx=0x7f03f00116d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3c6e91820fdae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.987+0000 7f03df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.988+0000 7f03df7fe640 1 -- 172.21.15.33:0/1399231138 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0028060 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.988+0000 7f03df7fe640 1 -- 172.21.15.33:0/1399231138 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f03f0011d00 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.988+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f0015070 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.988+0000 7f03df7fe640 1 -- 172.21.15.33:0/1399231138 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f03f00046c0 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.988+0000 7f04086cd740 10 client.27449 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.988+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5576d1e5fa20 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f03df7fe640 1 -- 172.21.15.33:0/1399231138 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f0015070 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f03df7fe640 1 -- 172.21.15.33:0/1399231138 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f03f002d070 con 0x5576d1e39960 2024-09-10T21:18:01.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f04086cd740 10 client.27449 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f04086cd740 10 client.27449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f04086cd740 20 client.27449 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f04086cd740 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e60d70 0x5576d1e63160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f04086cd740 4 client.27449 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576d1db4db0 con 0x5576d1e60d70 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.989+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e60d70 0x5576d1e63160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.990+0000 7f03df7fe640 10 client.27449 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.990+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e60d70 0x5576d1e63160 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863e58f7358e71a4 server_cookie=5932b613bbd63fe0 in_seq=0 out_seq=0 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.990+0000 7f03df7fe640 1 -- 172.21.15.33:0/1399231138 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5576d1db4db0 con 0x5576d1e60d70 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.990+0000 7f03df7fe640 10 client.27449 handle_command_reply: tid=0 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.990+0000 7f04086cd740 1 client.27449 shutdown 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.990+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 msgr2=0x5576d1e50b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.990+0000 7f04086cd740 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e50b30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f03f00116a0 tx=0x7f03f00116d0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 shutdown_connections 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e60d70 0x5576d1e63160 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f03df7fe640 0 client.27449 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 --2- 172.21.15.33:0/1399231138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e50b30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 >> 172.21.15.33:0/1399231138 conn(0x5576d1e3be80 msgr2=0x5576d1e38f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 shutdown_connections 2024-09-10T21:18:01.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 1 -- 172.21.15.33:0/1399231138 wait complete. 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591", 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:00.991+0000 7f04086cd740 20 client.27449 trim_cache size 0 max 16384 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.018+0000 7f04086cd740 1 Processor -- start 2024-09-10T21:18:01.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.018+0000 7f04086cd740 1 -- start start 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f04086cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e52820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f04086cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f04086cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e51570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f04086cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04001ac330 con 0x5576d1e49280 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f04086cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04001ae080 con 0x5576d1e39960 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f04086cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04001aca30 con 0x5576d1e49c00 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f03fffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e52820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f03fffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e52820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54802/0 (socket says 172.21.15.33:54802) 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f03fffff640 1 -- 172.21.15.33:0/2213934992 learned_addr learned my addr 172.21.15.33:0/2213934992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.019+0000 7f0404895640 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e51570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.020+0000 7f03fffff640 1 -- 172.21.15.33:0/2213934992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 msgr2=0x5576d1e51570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.020+0000 7f03fffff640 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e51570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.020+0000 7f03fffff640 1 -- 172.21.15.33:0/2213934992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 msgr2=0x5576d1e51030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.020+0000 7f03fffff640 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.020+0000 7f03fffff640 1 -- 172.21.15.33:0/2213934992 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d1e3a8e0 con 0x5576d1e39960 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.020+0000 7f03fffff640 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e52820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03f0003b90 tx=0x7f03f000bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2b27698fa4957b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.021+0000 7f03e5ffb640 1 -- 172.21.15.33:0/2213934992 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0033070 con 0x5576d1e39960 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.021+0000 7f03e5ffb640 1 -- 172.21.15.33:0/2213934992 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03f0015070 con 0x5576d1e39960 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.021+0000 7f03e5ffb640 1 -- 172.21.15.33:0/2213934992 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0028070 con 0x5576d1e39960 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 -- 172.21.15.33:0/2213934992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 msgr2=0x5576d1e52820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e52820 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f03f0003b90 tx=0x7f03f000bc20 comp rx=0 tx=0).stop 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 -- 172.21.15.33:0/2213934992 shutdown_connections 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e49c00 0x5576d1e51570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e51030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 --2- 172.21.15.33:0/2213934992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e39960 0x5576d1e52820 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 -- 172.21.15.33:0/2213934992 >> 172.21.15.33:0/2213934992 conn(0x5576d1e3be80 msgr2=0x5576d1e52110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 -- 172.21.15.33:0/2213934992 shutdown_connections 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 -- 172.21.15.33:0/2213934992 wait complete. 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.022+0000 7f04086cd740 1 -- start start 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 1 Processor -- start 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e39960 0x5576d1e58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e586c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 0x5576d1e58e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f04001ad920 con 0x5576d1e49280 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f04001ad190 con 0x5576d1e49c00 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f04001aa2f0 con 0x5576d1e39960 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f04086cd740 10 client.0 _ll_register_callbacks cb 0x7f03f4005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f03ff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47214/0 (socket says 172.21.15.33:47214) 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.023+0000 7f03ff7fe640 1 -- 172.21.15.33:0/3468053978 learned_addr learned my addr 172.21.15.33:0/3468053978 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f0404895640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 0x5576d1e58e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03fffff640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e39960 0x5576d1e58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03ff7fe640 1 -- 172.21.15.33:0/3468053978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e39960 msgr2=0x5576d1e58140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e39960 0x5576d1e58140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03ff7fe640 1 -- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 msgr2=0x5576d1e58e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 0x5576d1e58e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03ff7fe640 1 -- 172.21.15.33:0/3468053978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d1e59680 con 0x5576d1e49280 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03fffff640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e39960 0x5576d1e58140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03ff7fe640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e586c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f03f40ac3b0 tx=0x7f03f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c218cb1ccdf43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.024+0000 7f03df7fe640 1 -- 172.21.15.33:0/3468053978 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f40acdb0 con 0x5576d1e49280 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f03df7fe640 1 -- 172.21.15.33:0/3468053978 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f03f40a8920 con 0x5576d1e49280 2024-09-10T21:18:01.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f0015070 con 0x5576d1e49280 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f03df7fe640 1 -- 172.21.15.33:0/3468053978 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f03f40ac6b0 con 0x5576d1e49280 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 10 client.18036 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=69}) -- 0x5576d1e42480 con 0x5576d1e49280 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f03df7fe640 1 -- 172.21.15.33:0/3468053978 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f03f40ab910 con 0x5576d1e49280 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 10 client.18036 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 10 client.18036 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 20 client.18036 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e42b30 0x5576d1e44f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.025+0000 7f04086cd740 4 client.18036 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.026+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f03f0015070 con 0x5576d1e42b30 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.026+0000 7f03fffff640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e42b30 0x5576d1e44f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.026+0000 7f03fffff640 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e42b30 0x5576d1e44f50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24fdddb299d51657 server_cookie=5b732827dcd777f7 in_seq=0 out_seq=0 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.026+0000 7f03df7fe640 10 client.18036 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:01.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.027+0000 7f03df7fe640 1 -- 172.21.15.33:0/3468053978 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f03f0015070 con 0x5576d1e42b30 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.027+0000 7f03df7fe640 10 client.18036 handle_command_reply: tid=0 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.027+0000 7f04086cd740 1 client.18036 shutdown 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 msgr2=0x5576d1e586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e586c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f03f40ac3b0 tx=0x7f03f4005880 comp rx=0 tx=0).stop 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 shutdown_connections 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5576d1e42b30 0x5576d1e44f50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f03df7fe640 0 client.18036 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5576d1e49c00 0x5576d1e58e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5576d1e49280 0x5576d1e586c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 --2- 172.21.15.33:0/3468053978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5576d1e39960 0x5576d1e58140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 >> 172.21.15.33:0/3468053978 conn(0x5576d1e3be80 msgr2=0x5576d1e48a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 shutdown_connections 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 1 -- 172.21.15.33:0/3468053978 wait complete. 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.028+0000 7f04086cd740 20 client.18036 trim_cache size 0 max 16384 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03e80b53e0 msgr2=0x7f03e80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03e80b53e0 0x7f03e80b78a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009370 tx=0x7f03f802a6c0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04001076c0 msgr2=0x7f0400199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04001076c0 0x7f0400199780 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f03f400e9c0 tx=0x7f03f400ee80 comp rx=0 tx=0).stop 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 shutdown_connections 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f03e80b53e0 0x7f03e80b78a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0400108040 0x7f040019eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f04001076c0 0x7f0400199780 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 --2- 172.21.15.33:0/1754298633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0400101cb0 0x7f0400199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:01.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.030+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 >> 172.21.15.33:0/1754298633 conn(0x7f04000fd700 msgr2=0x7f040006b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:01.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.031+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 shutdown_connections 2024-09-10T21:18:01.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:01.031+0000 7f04062f8640 1 -- 172.21.15.33:0/1754298633 wait complete. 2024-09-10T21:18:01.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 55764 52.15 MB/sec warmup 61 sec latency 60.599 ms 2024-09-10T21:18:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:01 smithi104 ceph-mon[35325]: pgmap v767: 97 pgs: 97 active+clean; 758 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.2 MiB/s wr, 810 op/s 2024-09-10T21:18:01.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:01 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:18:01.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:01 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:01.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:01 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:01.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:01 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:01.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:01 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:01 smithi174 ceph-mon[35635]: pgmap v767: 97 pgs: 97 active+clean; 758 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.2 MiB/s wr, 810 op/s 2024-09-10T21:18:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:01 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:18:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:01 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:01 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:01 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:01 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:01 smithi033 ceph-mon[30860]: pgmap v767: 97 pgs: 97 active+clean; 758 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.2 MiB/s wr, 810 op/s 2024-09-10T21:18:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:01 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:18:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:01 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:01 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:01 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:01.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:01 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:02.051 INFO:tasks.workunit.client.0.smithi033.stdout: 10 56281 51.77 MB/sec warmup 62 sec latency 283.035 ms 2024-09-10T21:18:02.635 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:18:02.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:02 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:02.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:02 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:02.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:02 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:02 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:02 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:02 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:02 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:02 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:02 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:02 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:02 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:02 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:02 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:02 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:02 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:02 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:02 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:02 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:02 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:02 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:02.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:02 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:03.025 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:03.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 57119 51.59 MB/sec warmup 63 sec latency 437.699 ms 2024-09-10T21:18:03.713 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:03 smithi104 ceph-mon[35325]: pgmap v768: 97 pgs: 97 active+clean; 766 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 708 op/s 2024-09-10T21:18:03.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:03 smithi174 ceph-mon[35635]: pgmap v768: 97 pgs: 97 active+clean; 766 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 708 op/s 2024-09-10T21:18:03.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:03 smithi033 ceph-mon[30860]: pgmap v768: 97 pgs: 97 active+clean; 766 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 708 op/s 2024-09-10T21:18:04.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 58227 51.79 MB/sec warmup 64 sec latency 158.976 ms 2024-09-10T21:18:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.717+0000 7ff1a20c3640 1 -- 172.21.15.33:0/2708800111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a150 msgr2=0x7ff19c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.717+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/2708800111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a150 0x7ff19c1111b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff198009860 tx=0x7ff19802a380 comp rx=0 tx=0).stop 2024-09-10T21:18:04.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.717+0000 7ff1a20c3640 1 -- 172.21.15.33:0/2708800111 shutdown_connections 2024-09-10T21:18:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.717+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/2708800111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff19c11c0a0 0x7ff19c11e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.717+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/2708800111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff19c10a530 0x7ff19c10a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.717+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/2708800111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a150 0x7ff19c1111b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.717+0000 7ff1a20c3640 1 -- 172.21.15.33:0/2708800111 >> 172.21.15.33:0/2708800111 conn(0x7ff19c06cf20 msgr2=0x7ff19c06d330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.722+0000 7ff1a20c3640 1 -- 172.21.15.33:0/2708800111 shutdown_connections 2024-09-10T21:18:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.723+0000 7ff1a20c3640 1 -- 172.21.15.33:0/2708800111 wait complete. 2024-09-10T21:18:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 Processor -- start 2024-09-10T21:18:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 -- start start 2024-09-10T21:18:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff19c10a150 0x7ff19c1152c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a530 0x7ff19c115800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff19c11c0a0 0x7ff19c11ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c111d50 con 0x7ff19c11c0a0 2024-09-10T21:18:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c111bd0 con 0x7ff19c10a150 2024-09-10T21:18:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.724+0000 7ff1a20c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c111ed0 con 0x7ff19c10a530 2024-09-10T21:18:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.727+0000 7ff1a08c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a530 0x7ff19c115800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.727+0000 7ff1a08c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a530 0x7ff19c115800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49128/0 (socket says 172.21.15.33:49128) 2024-09-10T21:18:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.727+0000 7ff1a08c0640 1 -- 172.21.15.33:0/1820505166 learned_addr learned my addr 172.21.15.33:0/1820505166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.730+0000 7ff1a10c1640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff19c10a150 0x7ff19c1152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.731+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff19c11c0a0 0x7ff19c11ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.733+0000 7ff1a08c0640 1 -- 172.21.15.33:0/1820505166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff19c10a150 msgr2=0x7ff19c1152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.733+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff19c10a150 0x7ff19c1152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.733+0000 7ff1a08c0640 1 -- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff19c11c0a0 msgr2=0x7ff19c11ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.733+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff19c11c0a0 0x7ff19c11ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.733+0000 7ff1a08c0640 1 -- 172.21.15.33:0/1820505166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c11b2f0 con 0x7ff19c10a530 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.734+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a530 0x7ff19c115800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00e750 tx=0x7ff18c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.735+0000 7ff1927fc640 1 -- 172.21.15.33:0/1820505166 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c00cc60 con 0x7ff19c10a530 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.735+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c11b5e0 con 0x7ff19c10a530 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.735+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c11bb20 con 0x7ff19c10a530 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.735+0000 7ff1927fc640 1 -- 172.21.15.33:0/1820505166 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff18c00ce00 con 0x7ff19c10a530 2024-09-10T21:18:04.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.735+0000 7ff1927fc640 1 -- 172.21.15.33:0/1820505166 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c003e50 con 0x7ff19c10a530 2024-09-10T21:18:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.742+0000 7ff1a56fa740 1 Processor -- start 2024-09-10T21:18:04.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.742+0000 7ff1a56fa740 1 -- start start 2024-09-10T21:18:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.742+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.742+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.743+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.743+0000 7ff1a56fa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555fda15ee50 con 0x555fda1e8f70 2024-09-10T21:18:04.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.743+0000 7ff1a56fa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555fda14e280 con 0x555fda1f4f00 2024-09-10T21:18:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.743+0000 7ff1a56fa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fda152090 con 0x555fda1e8650 2024-09-10T21:18:04.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.744+0000 7ff1a08c0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.744+0000 7ff1a08c0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54820/0 (socket says 172.21.15.33:54820) 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.744+0000 7ff1a08c0640 1 -- 172.21.15.33:0/2517350687 learned_addr learned my addr 172.21.15.33:0/2517350687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.744+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.744+0000 7ff1a10c1640 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.744+0000 7ff1927fc640 1 -- 172.21.15.33:0/1820505166 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff18c0035f0 con 0x7ff19c10a530 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.745+0000 7ff1a08c0640 1 -- 172.21.15.33:0/2517350687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 msgr2=0x555fda1e8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.745+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.745+0000 7ff1a08c0640 1 -- 172.21.15.33:0/2517350687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 msgr2=0x555fda1f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.745+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.745+0000 7ff1a08c0640 1 -- 172.21.15.33:0/2517350687 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fd9eac650 con 0x555fda1f4f00 2024-09-10T21:18:04.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.745+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f72f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0e9ca0 tx=0x7ff18c0ee750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1114cecc960ea73 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.746+0000 7ff1877fe640 1 -- 172.21.15.33:0/2517350687 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0f8670 con 0x555fda1f4f00 2024-09-10T21:18:04.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.747+0000 7ff1877fe640 1 -- 172.21.15.33:0/2517350687 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff18c0f3070 con 0x555fda1f4f00 2024-09-10T21:18:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.749+0000 7ff1a56fa740 1 -- 172.21.15.33:0/2517350687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 msgr2=0x555fda1f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.749+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f72f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0e9ca0 tx=0x7ff18c0ee750 comp rx=0 tx=0).stop 2024-09-10T21:18:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.749+0000 7ff1927fc640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1880b5740 0x7ff1880b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.749+0000 7ff1927fc640 1 -- 172.21.15.33:0/1820505166 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff18c0e5cc0 con 0x7ff19c10a530 2024-09-10T21:18:04.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.751+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1880b5740 0x7ff1880b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.753+0000 7ff1a56fa740 1 -- 172.21.15.33:0/2517350687 shutdown_connections 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.753+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f72f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.753+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.753+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/2517350687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.753+0000 7ff1a56fa740 1 -- 172.21.15.33:0/2517350687 >> 172.21.15.33:0/2517350687 conn(0x555fda1e6480 msgr2=0x555fda1e7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.753+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1880b5740 0x7ff1880b7c00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff194009fd0 tx=0x7ff19400e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.755+0000 7ff1a56fa740 1 -- 172.21.15.33:0/2517350687 shutdown_connections 2024-09-10T21:18:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.755+0000 7ff1a56fa740 1 -- 172.21.15.33:0/2517350687 wait complete. 2024-09-10T21:18:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.755+0000 7ff1a56fa740 1 -- start start 2024-09-10T21:18:04.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.755+0000 7ff1a56fa740 1 Processor -- start 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1f4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a56fa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555fd9faf700 con 0x555fda1e8f70 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a56fa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555fda0e9110 con 0x555fda1f4f00 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a56fa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555fda1f7bd0 con 0x555fda1e8650 2024-09-10T21:18:04.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a56fa740 10 client.0 _ll_register_callbacks cb 0x555fda102990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a10c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a10c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49154/0 (socket says 172.21.15.33:49154) 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.756+0000 7ff1a10c1640 1 -- 172.21.15.33:0/3472190511 learned_addr learned my addr 172.21.15.33:0/3472190511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a18c2640 1 -- 172.21.15.33:0/3472190511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 msgr2=0x555fda1f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a18c2640 1 -- 172.21.15.33:0/3472190511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 msgr2=0x555fda1f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a18c2640 1 -- 172.21.15.33:0/3472190511 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fda116c40 con 0x555fda1f4f00 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.757+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f2130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff194009520 tx=0x7ff194004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62ecd0a3c73aae2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.758+0000 7ff184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.758+0000 7ff184ff9640 1 -- 172.21.15.33:0/3472190511 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194028060 con 0x555fda1f4f00 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.758+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff18c0f3070 con 0x555fda1f4f00 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.758+0000 7ff184ff9640 1 -- 172.21.15.33:0/3472190511 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff194016cf0 con 0x555fda1f4f00 2024-09-10T21:18:04.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.758+0000 7ff184ff9640 1 -- 172.21.15.33:0/3472190511 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff194023420 con 0x555fda1f4f00 2024-09-10T21:18:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.758+0000 7ff1a56fa740 10 client.27464 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.758+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x555fda203610 con 0x555fda1f4f00 2024-09-10T21:18:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff184ff9640 1 -- 172.21.15.33:0/3472190511 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff18c0f3070 con 0x555fda1f4f00 2024-09-10T21:18:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff184ff9640 1 -- 172.21.15.33:0/3472190511 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7ff194016070 con 0x555fda1f4f00 2024-09-10T21:18:04.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff1a56fa740 10 client.27464 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff1a56fa740 10 client.27464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff1a56fa740 20 client.27464 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda204f80 0x555fda2073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff1a56fa740 4 client.27464 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.760+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fda160da0 con 0x555fda204f80 2024-09-10T21:18:04.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.762+0000 7ff1a10c1640 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda204f80 0x555fda2073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.763+0000 7ff1a10c1640 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda204f80 0x555fda2073c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f8d730e40cf377 server_cookie=deb8dbd479f0bddf in_seq=0 out_seq=0 2024-09-10T21:18:04.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.763+0000 7ff184ff9640 10 client.27464 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.765+0000 7ff184ff9640 1 -- 172.21.15.33:0/3472190511 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fda160da0 con 0x555fda204f80 2024-09-10T21:18:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.765+0000 7ff184ff9640 10 client.27464 handle_command_reply: tid=0 2024-09-10T21:18:04.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.765+0000 7ff1a56fa740 1 client.27464 shutdown 2024-09-10T21:18:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.768+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 msgr2=0x555fda1f2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.768+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f2130 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff194009520 tx=0x7ff194004a40 comp rx=0 tx=0).stop 2024-09-10T21:18:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 shutdown_connections 2024-09-10T21:18:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda204f80 0x555fda2073c0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1f4f00 0x555fda1f2130 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/3472190511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 >> 172.21.15.33:0/3472190511 conn(0x555fda1e6480 msgr2=0x555fda1ec020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff184ff9640 0 client.27464 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:04.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 shutdown_connections 2024-09-10T21:18:04.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.770+0000 7ff1a56fa740 1 -- 172.21.15.33:0/3472190511 wait complete. 2024-09-10T21:18:04.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.772+0000 7ff1a56fa740 20 client.27464 trim_cache size 0 max 16384 2024-09-10T21:18:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.818+0000 7ff1a56fa740 1 Processor -- start 2024-09-10T21:18:04.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.818+0000 7ff1a56fa740 1 -- start start 2024-09-10T21:18:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.818+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.819+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8f70 0x555fda1f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.819+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1f4f00 0x555fda1f6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.819+0000 7ff1a10c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.819+0000 7ff1a08c0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8f70 0x555fda1f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.820+0000 7ff1a08c0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8f70 0x555fda1f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54842/0 (socket says 172.21.15.33:54842) 2024-09-10T21:18:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.820+0000 7ff1a08c0640 1 -- 172.21.15.33:0/1544960301 learned_addr learned my addr 172.21.15.33:0/1544960301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.820+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff18000e550 con 0x555fda1f4f00 2024-09-10T21:18:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.820+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800102a0 con 0x555fda1e8f70 2024-09-10T21:18:04.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.820+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff18000ec50 con 0x555fda1e8650 2024-09-10T21:18:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.821+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1f4f00 0x555fda1f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.821+0000 7ff1a18c2640 1 -- 172.21.15.33:0/1544960301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 msgr2=0x555fda1e5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.821+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.821+0000 7ff1a18c2640 1 -- 172.21.15.33:0/1544960301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8f70 msgr2=0x555fda1f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.821+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8f70 0x555fda1f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.821+0000 7ff1a18c2640 1 -- 172.21.15.33:0/1544960301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fda1f2340 con 0x555fda1f4f00 2024-09-10T21:18:04.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.822+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1f4f00 0x555fda1f6830 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff194004b90 tx=0x7ff19402c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3bd0dccf2f4cc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.824+0000 7ff186ffd640 1 -- 172.21.15.33:0/1544960301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194032080 con 0x555fda1f4f00 2024-09-10T21:18:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.824+0000 7ff186ffd640 1 -- 172.21.15.33:0/1544960301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff194009cc0 con 0x555fda1f4f00 2024-09-10T21:18:04.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.824+0000 7ff186ffd640 1 -- 172.21.15.33:0/1544960301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff194016e50 con 0x555fda1f4f00 2024-09-10T21:18:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.825+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1f4f00 msgr2=0x555fda1f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.825+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1f4f00 0x555fda1f6830 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7ff194004b90 tx=0x7ff19402c550 comp rx=0 tx=0).stop 2024-09-10T21:18:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.825+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 shutdown_connections 2024-09-10T21:18:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.825+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1f4f00 0x555fda1f6830 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.825+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8f70 0x555fda1f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.825+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/1544960301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1e8650 0x555fda1e5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.825+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 >> 172.21.15.33:0/1544960301 conn(0x555fda1e6480 msgr2=0x555fda1f0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.826+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 shutdown_connections 2024-09-10T21:18:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.826+0000 7ff1a56fa740 1 -- 172.21.15.33:0/1544960301 wait complete. 2024-09-10T21:18:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.826+0000 7ff1a56fa740 1 -- start start 2024-09-10T21:18:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.827+0000 7ff1a56fa740 1 Processor -- start 2024-09-10T21:18:04.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.828+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8650 0x555fda1fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.828+0000 7ff1a56fa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1fd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.828+0000 7ff1a10c1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8650 0x555fda1fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.828+0000 7ff1a10c1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8650 0x555fda1fcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54856/0 (socket says 172.21.15.33:54856) 2024-09-10T21:18:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.829+0000 7ff1a10c1640 1 -- 172.21.15.33:0/938806852 learned_addr learned my addr 172.21.15.33:0/938806852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.829+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1f4f00 0x555fda1f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.829+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff18000fb40 con 0x555fda1e8f70 2024-09-10T21:18:04.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.829+0000 7ff1a18c2640 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1f4f00 0x555fda1f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.829+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff18000f3b0 con 0x555fda1e8650 2024-09-10T21:18:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.830+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.830+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff18000c510 con 0x555fda1f4f00 2024-09-10T21:18:04.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.831+0000 7ff1a56fa740 10 client.0 _ll_register_callbacks cb 0x7ff194009c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.831+0000 7ff1a08c0640 1 -- 172.21.15.33:0/938806852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1f4f00 msgr2=0x555fda1f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.831+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1f4f00 0x555fda1f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.831+0000 7ff1a08c0640 1 -- 172.21.15.33:0/938806852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8650 msgr2=0x555fda1fcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.831+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8650 0x555fda1fcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.831+0000 7ff1a08c0640 1 -- 172.21.15.33:0/938806852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fda1f8410 con 0x555fda1e8f70 2024-09-10T21:18:04.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.832+0000 7ff1a08c0640 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1fd460 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0eec50 tx=0x7ff18c0e97b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628370ebbcc8d3de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:04.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.832+0000 7ff184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:04.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.832+0000 7ff184ff9640 1 -- 172.21.15.33:0/938806852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0a8a90 con 0x555fda1e8f70 2024-09-10T21:18:04.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.832+0000 7ff184ff9640 1 -- 172.21.15.33:0/938806852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff18c0ac8a0 con 0x555fda1e8f70 2024-09-10T21:18:04.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.832+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff194009cc0 con 0x555fda1e8f70 2024-09-10T21:18:04.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.833+0000 7ff184ff9640 1 -- 172.21.15.33:0/938806852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff18c0abbb0 con 0x555fda1e8f70 2024-09-10T21:18:04.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.833+0000 7ff1a56fa740 10 client.18051 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:04.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.833+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x555fda2725a0 con 0x555fda1e8f70 2024-09-10T21:18:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff184ff9640 1 -- 172.21.15.33:0/938806852 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff194009cc0 con 0x555fda1e8f70 2024-09-10T21:18:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff184ff9640 1 -- 172.21.15.33:0/938806852 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7ff18c0d3190 con 0x555fda1e8f70 2024-09-10T21:18:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff1a56fa740 10 client.18051 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:04.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff1a56fa740 10 client.18051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff1a56fa740 20 client.18051 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda2728e0 0x555fda274cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff1a56fa740 4 client.18051 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:18:04.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.834+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fda15c850 con 0x555fda2728e0 2024-09-10T21:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.838+0000 7ff1a10c1640 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda2728e0 0x555fda274cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:04.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.841+0000 7ff1a10c1640 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda2728e0 0x555fda274cd0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db25c6762d7d7309 server_cookie=1f2d3b4d286c2e89 in_seq=0 out_seq=0 2024-09-10T21:18:04.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.841+0000 7ff184ff9640 10 client.18051 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.842+0000 7ff184ff9640 1 -- 172.21.15.33:0/938806852 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x555fda15c850 con 0x555fda2728e0 2024-09-10T21:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.842+0000 7ff184ff9640 10 client.18051 handle_command_reply: tid=0 2024-09-10T21:18:04.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.842+0000 7ff1a56fa740 1 client.18051 shutdown 2024-09-10T21:18:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.845+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 msgr2=0x555fda1fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.845+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1fd460 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7ff18c0eec50 tx=0x7ff18c0e97b0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.846+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 shutdown_connections 2024-09-10T21:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.846+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x555fda2728e0 0x555fda274cd0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.846+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555fda1f4f00 0x555fda1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.846+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555fda1e8f70 0x555fda1fd460 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.846+0000 7ff1a56fa740 1 --2- 172.21.15.33:0/938806852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555fda1e8650 0x555fda1fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.846+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 >> 172.21.15.33:0/938806852 conn(0x555fda1e6480 msgr2=0x555fda1f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.846+0000 7ff184ff9640 0 client.18051 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.847+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 shutdown_connections 2024-09-10T21:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.847+0000 7ff1a56fa740 1 -- 172.21.15.33:0/938806852 wait complete. 2024-09-10T21:18:04.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.847+0000 7ff1a56fa740 20 client.18051 trim_cache size 0 max 16384 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591", 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:04.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:18:04.858 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:18:04.860 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.862 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.863 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:18:04.864 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:04.865 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:04.865 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:04.865 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:04.865 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:04.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:04.865 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:04.865 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.850+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1880b5740 msgr2=0x7ff1880b7c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.850+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1880b5740 0x7ff1880b7c00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff194009fd0 tx=0x7ff19400e040 comp rx=0 tx=0).stop 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.850+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a530 msgr2=0x7ff19c115800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.850+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a530 0x7ff19c115800 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00e750 tx=0x7ff18c00ec10 comp rx=0 tx=0).stop 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.852+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 shutdown_connections 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.852+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1880b5740 0x7ff1880b7c00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.852+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff19c11c0a0 0x7ff19c11ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.852+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff19c10a530 0x7ff19c115800 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.852+0000 7ff1a20c3640 1 --2- 172.21.15.33:0/1820505166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff19c10a150 0x7ff19c1152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:04.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.852+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 >> 172.21.15.33:0/1820505166 conn(0x7ff19c06cf20 msgr2=0x7ff19c071560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:04.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.854+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 shutdown_connections 2024-09-10T21:18:04.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:04.854+0000 7ff1a20c3640 1 -- 172.21.15.33:0/1820505166 wait complete. 2024-09-10T21:18:05.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 58609 51.31 MB/sec warmup 65 sec latency 574.915 ms 2024-09-10T21:18:05.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:05 smithi104 ceph-mon[35325]: pgmap v769: 97 pgs: 97 active+clean; 777 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 672 op/s 2024-09-10T21:18:05.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:05 smithi174 ceph-mon[35635]: pgmap v769: 97 pgs: 97 active+clean; 777 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 672 op/s 2024-09-10T21:18:05.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:05 smithi033 ceph-mon[30860]: pgmap v769: 97 pgs: 97 active+clean; 777 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.1 MiB/s wr, 672 op/s 2024-09-10T21:18:06.006 INFO:tasks.cephfs.filesystem:scrub status for tag:7c5a98b8-eef5-45b6-8fa8-b9cd5976f591 - {'path': '/', 'tag': '7c5a98b8-eef5-45b6-8fa8-b9cd5976f591', 'options': 'recursive,force'} 2024-09-10T21:18:06.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 58758 50.63 MB/sec warmup 66 sec latency 778.511 ms 2024-09-10T21:18:07.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 59361 50.34 MB/sec warmup 67 sec latency 110.884 ms 2024-09-10T21:18:07.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:07 smithi104 ceph-mon[35325]: pgmap v770: 97 pgs: 97 active+clean; 796 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 701 op/s 2024-09-10T21:18:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:07 smithi174 ceph-mon[35635]: pgmap v770: 97 pgs: 97 active+clean; 796 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 701 op/s 2024-09-10T21:18:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:07 smithi033 ceph-mon[30860]: pgmap v770: 97 pgs: 97 active+clean; 796 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 701 op/s 2024-09-10T21:18:08.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 59987 50.08 MB/sec warmup 68 sec latency 324.963 ms 2024-09-10T21:18:09.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 60556 49.79 MB/sec warmup 69 sec latency 809.241 ms 2024-09-10T21:18:09.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:09 smithi033 ceph-mon[30860]: pgmap v771: 97 pgs: 97 active+clean; 804 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 983 op/s 2024-09-10T21:18:09.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:09 smithi104 ceph-mon[35325]: pgmap v771: 97 pgs: 97 active+clean; 804 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 983 op/s 2024-09-10T21:18:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:09 smithi174 ceph-mon[35635]: pgmap v771: 97 pgs: 97 active+clean; 804 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 15 MiB/s wr, 983 op/s 2024-09-10T21:18:10.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 61635 50.01 MB/sec warmup 70 sec latency 174.565 ms 2024-09-10T21:18:11.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 62368 49.89 MB/sec warmup 71 sec latency 246.929 ms 2024-09-10T21:18:11.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:11 smithi104 ceph-mon[35325]: pgmap v772: 97 pgs: 97 active+clean; 787 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.10k op/s 2024-09-10T21:18:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:11 smithi174 ceph-mon[35635]: pgmap v772: 97 pgs: 97 active+clean; 787 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.10k op/s 2024-09-10T21:18:11.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:11 smithi033 ceph-mon[30860]: pgmap v772: 97 pgs: 97 active+clean; 787 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.10k op/s 2024-09-10T21:18:12.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 63193 49.74 MB/sec warmup 72 sec latency 134.555 ms 2024-09-10T21:18:13.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 64020 49.55 MB/sec warmup 73 sec latency 224.902 ms 2024-09-10T21:18:13.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:13 smithi104 ceph-mon[35325]: pgmap v773: 97 pgs: 97 active+clean; 772 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-09-10T21:18:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:13 smithi174 ceph-mon[35635]: pgmap v773: 97 pgs: 97 active+clean; 772 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-09-10T21:18:13.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:13 smithi033 ceph-mon[30860]: pgmap v773: 97 pgs: 97 active+clean; 772 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 20 MiB/s wr, 1.12k op/s 2024-09-10T21:18:14.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 65048 49.68 MB/sec warmup 74 sec latency 258.391 ms 2024-09-10T21:18:15.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 65865 49.62 MB/sec warmup 75 sec latency 264.524 ms 2024-09-10T21:18:15.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:15 smithi104 ceph-mon[35325]: pgmap v774: 97 pgs: 97 active+clean; 785 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-10T21:18:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:15 smithi174 ceph-mon[35635]: pgmap v774: 97 pgs: 97 active+clean; 785 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-10T21:18:15.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:15 smithi033 ceph-mon[30860]: pgmap v774: 97 pgs: 97 active+clean; 785 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 20 MiB/s wr, 1.16k op/s 2024-09-10T21:18:16.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 66813 49.73 MB/sec warmup 76 sec latency 213.901 ms 2024-09-10T21:18:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:16 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:18:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:16 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:18:16.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:16 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:18:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:16 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:18:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:16 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:18:16.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:16 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:18:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:16 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:18:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:16 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:18:16.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:16 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:18:17.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 67560 49.63 MB/sec warmup 77 sec latency 127.613 ms 2024-09-10T21:18:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:17 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:18:17.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:17 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:18:17.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:17 smithi104 ceph-mon[35325]: pgmap v775: 97 pgs: 97 active+clean; 789 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-09-10T21:18:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:17 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:18:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:17 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:18:17.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:17 smithi174 ceph-mon[35635]: pgmap v775: 97 pgs: 97 active+clean; 789 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-09-10T21:18:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:17 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:18:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:17 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:18:17.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:17 smithi033 ceph-mon[30860]: pgmap v775: 97 pgs: 97 active+clean; 789 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 1.05k op/s 2024-09-10T21:18:18.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 68090 49.26 MB/sec warmup 78 sec latency 324.211 ms 2024-09-10T21:18:19.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 69012 49.25 MB/sec warmup 79 sec latency 66.775 ms 2024-09-10T21:18:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:19 smithi104 ceph-mon[35325]: pgmap v776: 97 pgs: 97 active+clean; 800 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2024-09-10T21:18:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:19 smithi174 ceph-mon[35635]: pgmap v776: 97 pgs: 97 active+clean; 800 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2024-09-10T21:18:19.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:19 smithi033 ceph-mon[30860]: pgmap v776: 97 pgs: 97 active+clean; 800 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 17 MiB/s wr, 1.12k op/s 2024-09-10T21:18:20.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 69899 49.19 MB/sec warmup 80 sec latency 251.604 ms 2024-09-10T21:18:21.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 70920 49.36 MB/sec warmup 81 sec latency 131.152 ms 2024-09-10T21:18:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:21 smithi104 ceph-mon[35325]: pgmap v777: 97 pgs: 97 active+clean; 802 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.2 MiB/s wr, 786 op/s 2024-09-10T21:18:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:21 smithi174 ceph-mon[35635]: pgmap v777: 97 pgs: 97 active+clean; 802 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.2 MiB/s wr, 786 op/s 2024-09-10T21:18:21.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:21 smithi033 ceph-mon[30860]: pgmap v777: 97 pgs: 97 active+clean; 802 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.2 MiB/s wr, 786 op/s 2024-09-10T21:18:22.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 71622 49.28 MB/sec warmup 82 sec latency 207.746 ms 2024-09-10T21:18:22.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:22 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:18:22.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:22 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:18:22.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:22 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:18:23.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 72345 49.09 MB/sec warmup 83 sec latency 190.545 ms 2024-09-10T21:18:23.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:23 smithi104 ceph-mon[35325]: pgmap v778: 97 pgs: 97 active+clean; 815 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.2 MiB/s wr, 753 op/s 2024-09-10T21:18:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:23 smithi174 ceph-mon[35635]: pgmap v778: 97 pgs: 97 active+clean; 815 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.2 MiB/s wr, 753 op/s 2024-09-10T21:18:23.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:23 smithi033 ceph-mon[30860]: pgmap v778: 97 pgs: 97 active+clean; 815 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.2 MiB/s wr, 753 op/s 2024-09-10T21:18:24.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 73073 48.93 MB/sec warmup 84 sec latency 64.403 ms 2024-09-10T21:18:25.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 73915 48.88 MB/sec warmup 85 sec latency 242.158 ms 2024-09-10T21:18:25.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:25 smithi104 ceph-mon[35325]: pgmap v779: 97 pgs: 97 active+clean; 821 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.2 MiB/s wr, 719 op/s 2024-09-10T21:18:25.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:25 smithi174 ceph-mon[35635]: pgmap v779: 97 pgs: 97 active+clean; 821 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.2 MiB/s wr, 719 op/s 2024-09-10T21:18:25.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:25 smithi033 ceph-mon[30860]: pgmap v779: 97 pgs: 97 active+clean; 821 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.2 MiB/s wr, 719 op/s 2024-09-10T21:18:26.052 INFO:tasks.workunit.client.0.smithi033.stdout: 10 74666 48.80 MB/sec warmup 86 sec latency 323.651 ms 2024-09-10T21:18:27.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 75372 48.69 MB/sec warmup 87 sec latency 340.764 ms 2024-09-10T21:18:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:27 smithi104 ceph-mon[35325]: pgmap v780: 97 pgs: 97 active+clean; 826 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 615 op/s 2024-09-10T21:18:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:27 smithi174 ceph-mon[35635]: pgmap v780: 97 pgs: 97 active+clean; 826 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 615 op/s 2024-09-10T21:18:27.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:27 smithi033 ceph-mon[30860]: pgmap v780: 97 pgs: 97 active+clean; 826 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 4.6 MiB/s wr, 615 op/s 2024-09-10T21:18:28.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 76141 48.60 MB/sec warmup 88 sec latency 268.712 ms 2024-09-10T21:18:29.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 76518 48.26 MB/sec warmup 89 sec latency 887.503 ms 2024-09-10T21:18:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:29 smithi104 ceph-mon[35325]: pgmap v781: 97 pgs: 97 active+clean; 837 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-09-10T21:18:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:29 smithi174 ceph-mon[35635]: pgmap v781: 97 pgs: 97 active+clean; 837 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-09-10T21:18:29.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:29 smithi033 ceph-mon[30860]: pgmap v781: 97 pgs: 97 active+clean; 837 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.5 MiB/s wr, 792 op/s 2024-09-10T21:18:30.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 77477 48.42 MB/sec warmup 90 sec latency 44.802 ms 2024-09-10T21:18:31.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 77999 48.09 MB/sec warmup 91 sec latency 253.158 ms 2024-09-10T21:18:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:31 smithi104 ceph-mon[35325]: pgmap v782: 97 pgs: 97 active+clean; 837 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.6 MiB/s wr, 662 op/s 2024-09-10T21:18:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:18:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:18:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:18:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:31 smithi174 ceph-mon[35635]: pgmap v782: 97 pgs: 97 active+clean; 837 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.6 MiB/s wr, 662 op/s 2024-09-10T21:18:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:18:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:18:31.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:18:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:31 smithi033 ceph-mon[30860]: pgmap v782: 97 pgs: 97 active+clean; 837 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 6.6 MiB/s wr, 662 op/s 2024-09-10T21:18:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:18:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:18:31.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:18:32.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 78776 48.03 MB/sec warmup 92 sec latency 205.122 ms 2024-09-10T21:18:33.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 79729 47.95 MB/sec warmup 93 sec latency 146.322 ms 2024-09-10T21:18:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:33 smithi104 ceph-mon[35325]: pgmap v783: 97 pgs: 97 active+clean; 846 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 774 op/s 2024-09-10T21:18:33.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:33 smithi174 ceph-mon[35635]: pgmap v783: 97 pgs: 97 active+clean; 846 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 774 op/s 2024-09-10T21:18:33.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:33 smithi033 ceph-mon[30860]: pgmap v783: 97 pgs: 97 active+clean; 846 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 774 op/s 2024-09-10T21:18:34.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 80826 48.22 MB/sec warmup 94 sec latency 53.347 ms 2024-09-10T21:18:35.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 81796 48.19 MB/sec warmup 95 sec latency 63.645 ms 2024-09-10T21:18:35.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:35 smithi104 ceph-mon[35325]: pgmap v784: 97 pgs: 97 active+clean; 851 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.4 MiB/s wr, 774 op/s 2024-09-10T21:18:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:35 smithi174 ceph-mon[35635]: pgmap v784: 97 pgs: 97 active+clean; 851 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.4 MiB/s wr, 774 op/s 2024-09-10T21:18:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:35 smithi033 ceph-mon[30860]: pgmap v784: 97 pgs: 97 active+clean; 851 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.4 MiB/s wr, 774 op/s 2024-09-10T21:18:36.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:18:36.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 82585 48.21 MB/sec warmup 96 sec latency 208.432 ms 2024-09-10T21:18:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:36.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 -- 172.21.15.33:0/4121964066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc62810d100 msgr2=0x7fc62810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:36.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/4121964066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc62810d100 0x7fc62810f560 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc620009ca0 tx=0x7fc62002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 -- 172.21.15.33:0/4121964066 shutdown_connections 2024-09-10T21:18:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/4121964066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc62810d100 0x7fc62810f560 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/4121964066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc6281045c0 0x7fc62810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/4121964066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc628103ca0 0x7fc628104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 -- 172.21.15.33:0/4121964066 >> 172.21.15.33:0/4121964066 conn(0x7fc6280fd700 msgr2=0x7fc6280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 -- 172.21.15.33:0/4121964066 shutdown_connections 2024-09-10T21:18:36.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.943+0000 7fc62cfd3640 1 -- 172.21.15.33:0/4121964066 wait complete. 2024-09-10T21:18:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.944+0000 7fc62cfd3640 1 Processor -- start 2024-09-10T21:18:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.944+0000 7fc62cfd3640 1 -- start start 2024-09-10T21:18:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62cfd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc628103ca0 0x7fc628199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62cfd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6281045c0 0x7fc6281997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6281045c0 0x7fc6281997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc628103ca0 0x7fc628199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc628103ca0 0x7fc628199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39174/0 (socket says 172.21.15.33:39174) 2024-09-10T21:18:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6281045c0 0x7fc6281997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46044/0 (socket says 172.21.15.33:46044) 2024-09-10T21:18:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62659c640 1 -- 172.21.15.33:0/2360117481 learned_addr learned my addr 172.21.15.33:0/2360117481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc62810d100 0x7fc62819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc628112e00 con 0x7fc6281045c0 2024-09-10T21:18:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc628112c80 con 0x7fc628103ca0 2024-09-10T21:18:36.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.945+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc628112f80 con 0x7fc62810d100 2024-09-10T21:18:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.946+0000 7fc626d9d640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc62810d100 0x7fc62819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.946+0000 7fc62659c640 1 -- 172.21.15.33:0/2360117481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc62810d100 msgr2=0x7fc62819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:36.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.946+0000 7fc62659c640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc62810d100 0x7fc62819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.946+0000 7fc62659c640 1 -- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6281045c0 msgr2=0x7fc6281997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.946+0000 7fc62659c640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6281045c0 0x7fc6281997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.946+0000 7fc62659c640 1 -- 172.21.15.33:0/2360117481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62819f340 con 0x7fc628103ca0 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.947+0000 7fc62659c640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc628103ca0 0x7fc628199290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b4a0 tx=0x7fc61800b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.947+0000 7fc61ffff640 1 -- 172.21.15.33:0/2360117481 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618014070 con 0x7fc628103ca0 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.947+0000 7fc61ffff640 1 -- 172.21.15.33:0/2360117481 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc618010ac0 con 0x7fc628103ca0 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.947+0000 7fc61ffff640 1 -- 172.21.15.33:0/2360117481 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc618017c30 con 0x7fc628103ca0 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.947+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62819f630 con 0x7fc628103ca0 2024-09-10T21:18:36.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.947+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6281018b0 con 0x7fc628103ca0 2024-09-10T21:18:36.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 Processor -- start 2024-09-10T21:18:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 -- start start 2024-09-10T21:18:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad0a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad16410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad18d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ac80e50 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ac70280 con 0x55fb5ad16950 2024-09-10T21:18:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ac74090 con 0x55fb5ad0a0a0 2024-09-10T21:18:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.953+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad18d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad18d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39186/0 (socket says 172.21.15.33:39186) 2024-09-10T21:18:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc61dd9b640 1 -- 172.21.15.33:0/4017704138 learned_addr learned my addr 172.21.15.33:0/4017704138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:36.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc626d9d640 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad0a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc61dd9b640 1 -- 172.21.15.33:0/4017704138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 msgr2=0x55fb5ad0a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad0a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc61dd9b640 1 -- 172.21.15.33:0/4017704138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a9c0 msgr2=0x55fb5ad16410 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad16410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.954+0000 7fc61dd9b640 1 -- 172.21.15.33:0/4017704138 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5a9ce650 con 0x55fb5ad16950 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.955+0000 7fc61ffff640 1 -- 172.21.15.33:0/2360117481 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc618010610 con 0x7fc628103ca0 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.955+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad18d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc610009c50 tx=0x7fc610011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=334366a24f03d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.955+0000 7fc61cd99640 1 -- 172.21.15.33:0/4017704138 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610004220 con 0x55fb5ad16950 2024-09-10T21:18:36.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.955+0000 7fc61cd99640 1 -- 172.21.15.33:0/4017704138 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6100043c0 con 0x55fb5ad16950 2024-09-10T21:18:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.955+0000 7fc61cd99640 1 -- 172.21.15.33:0/4017704138 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610004cd0 con 0x55fb5ad16950 2024-09-10T21:18:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.956+0000 7fc61ffff640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6140b5360 0x7fc6140b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.956+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4017704138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 msgr2=0x55fb5ad18d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:36.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.956+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad18d40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc610009c50 tx=0x7fc610011810 comp rx=0 tx=0).stop 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc61ffff640 1 -- 172.21.15.33:0/2360117481 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc6180e4fe0 con 0x7fc628103ca0 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4017704138 shutdown_connections 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad18d40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad16410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4017704138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad0a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4017704138 >> 172.21.15.33:0/4017704138 conn(0x55fb5ad07ed0 msgr2=0x55fb5ad09070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc626d9d640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6140b5360 0x7fc6140b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4017704138 shutdown_connections 2024-09-10T21:18:36.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.957+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4017704138 wait complete. 2024-09-10T21:18:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.958+0000 7fc62f3a8740 1 -- start start 2024-09-10T21:18:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.958+0000 7fc626d9d640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6140b5360 0x7fc6140b7820 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc620006010 tx=0x7fc620005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:36.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.958+0000 7fc62f3a8740 1 Processor -- start 2024-09-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad162d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad13720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad13df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55fb5aad1700 con 0x55fb5ad0a0a0 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ac0b110 con 0x55fb5ad16950 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad13720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad13720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:48294/0 (socket says 172.21.15.33:48294) 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46058/0 (socket says 172.21.15.33:46058) 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc61dd9b640 1 -- 172.21.15.33:0/2981933488 learned_addr learned my addr 172.21.15.33:0/2981933488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc626d9d640 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad13df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fb5ad195d0 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.959+0000 7fc62f3a8740 10 client.0 _ll_register_callbacks cb 0x55fb5ac24990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc61dd9b640 1 -- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 msgr2=0x55fb5ad13df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:36.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad13df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc61dd9b640 1 -- 172.21.15.33:0/2981933488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a0a0 msgr2=0x55fb5ad162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc61dd9b640 1 -- 172.21.15.33:0/2981933488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5ac38c40 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad13720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc610005f60 tx=0x7fc610019d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a846ad9bcf477b3f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc60dffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc60dffb640 1 -- 172.21.15.33:0/2981933488 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610027030 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6100043c0 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.960+0000 7fc60dffb640 1 -- 172.21.15.33:0/2981933488 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc61002c410 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc60dffb640 1 -- 172.21.15.33:0/2981933488 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc61001d040 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 10 client.27406 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:36.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55fb5ad24fb0 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc60dffb640 1 -- 172.21.15.33:0/2981933488 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6100043c0 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc60dffb640 1 -- 172.21.15.33:0/2981933488 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fc610023ba0 con 0x55fb5ad0a9c0 2024-09-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 10 client.27406 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:36.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 10 client.27406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 20 client.27406 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad26360 0x55fb5ad28750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 4 client.27406 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.961+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb5ac82da0 con 0x55fb5ad26360 2024-09-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.962+0000 7fc62659c640 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad26360 0x55fb5ad28750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.962+0000 7fc60dffb640 10 client.27406 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:36.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.963+0000 7fc62659c640 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad26360 0x55fb5ad28750 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9cc5b97e8079b7 server_cookie=4d80565a934a9bc1 in_seq=0 out_seq=0 2024-09-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc60dffb640 1 -- 172.21.15.33:0/2981933488 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fb5ac82da0 con 0x55fb5ad26360 2024-09-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc60dffb640 10 client.27406 handle_command_reply: tid=0 2024-09-10T21:18:36.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 client.27406 shutdown 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 msgr2=0x55fb5ad13720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad13720 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc610005f60 tx=0x7fc610019d90 comp rx=0 tx=0).stop 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 shutdown_connections 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad26360 0x55fb5ad28750 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc60dffb640 0 client.27406 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad16950 0x55fb5ad13df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad13720 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/2981933488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 >> 172.21.15.33:0/2981933488 conn(0x55fb5ad07ed0 msgr2=0x55fb5ad0da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 shutdown_connections 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.966+0000 7fc62f3a8740 1 -- 172.21.15.33:0/2981933488 wait complete. 2024-09-10T21:18:36.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.967+0000 7fc62f3a8740 20 client.27406 trim_cache size 0 max 16384 2024-09-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.994+0000 7fc62f3a8740 1 Processor -- start 2024-09-10T21:18:36.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62f3a8740 1 -- start start 2024-09-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad22830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:36.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad17d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad18280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b0680 con 0x55fb5ad16950 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b23d0 con 0x55fb5ad0a9c0 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b0d80 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad17d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc61dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad17d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39190/0 (socket says 172.21.15.33:39190) 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc626d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad18280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc61dd9b640 1 -- 172.21.15.33:0/3072549421 learned_addr learned my addr 172.21.15.33:0/3072549421 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc62659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad22830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc61dd9b640 1 -- 172.21.15.33:0/3072549421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 msgr2=0x55fb5ad22830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.995+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad22830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.996+0000 7fc61dd9b640 1 -- 172.21.15.33:0/3072549421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 msgr2=0x55fb5ad18280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.996+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad18280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.996+0000 7fc61dd9b640 1 -- 172.21.15.33:0/3072549421 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb5ad0a7f0 con 0x55fb5ad0a9c0 2024-09-10T21:18:37.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.996+0000 7fc626d9d640 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad18280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.996+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad17d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc610012fd0 tx=0x7fc610019780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfa6f07ff3566a7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.996+0000 7fc60ffff640 1 -- 172.21.15.33:0/3072549421 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610027070 con 0x55fb5ad0a9c0 2024-09-10T21:18:37.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.996+0000 7fc60ffff640 1 -- 172.21.15.33:0/3072549421 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc61002e4f0 con 0x55fb5ad0a9c0 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.997+0000 7fc60ffff640 1 -- 172.21.15.33:0/3072549421 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc610005030 con 0x55fb5ad0a9c0 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 -- 172.21.15.33:0/3072549421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a9c0 msgr2=0x55fb5ad17d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad17d40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc610012fd0 tx=0x7fc610019780 comp rx=0 tx=0).stop 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 -- 172.21.15.33:0/3072549421 shutdown_connections 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad18280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad17d40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/3072549421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad22830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 -- 172.21.15.33:0/3072549421 >> 172.21.15.33:0/3072549421 conn(0x55fb5ad07ed0 msgr2=0x55fb5ad15570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.998+0000 7fc62f3a8740 1 -- 172.21.15.33:0/3072549421 shutdown_connections 2024-09-10T21:18:37.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.999+0000 7fc62f3a8740 1 -- 172.21.15.33:0/3072549421 wait complete. 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.999+0000 7fc62f3a8740 1 -- start start 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:36.999+0000 7fc62f3a8740 1 Processor -- start 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad1e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad1ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62f3a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad19850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad1e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b1c70 con 0x55fb5ad16950 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281b14e0 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62f3a8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6281ae640 con 0x55fb5ad0a9c0 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62f3a8740 10 client.0 _ll_register_callbacks cb 0x7fc620009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad1e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39200/0 (socket says 172.21.15.33:39200) 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.000+0000 7fc62659c640 1 -- 172.21.15.33:0/4087110738 learned_addr learned my addr 172.21.15.33:0/4087110738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad1ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc626d9d640 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad19850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc62659c640 1 -- 172.21.15.33:0/4087110738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 msgr2=0x55fb5ad1ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc62659c640 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad1ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc62659c640 1 -- 172.21.15.33:0/4087110738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 msgr2=0x55fb5ad19850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc62659c640 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad19850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc62659c640 1 -- 172.21.15.33:0/4087110738 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb5ad19d90 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc62659c640 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad1e8a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6180a8ac0 tx=0x7fc6180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3238f53d29f5d172 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc60dffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.001+0000 7fc60dffb640 1 -- 172.21.15.33:0/4087110738 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61801e040 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.002+0000 7fc60dffb640 1 -- 172.21.15.33:0/4087110738 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc6180aaab0 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.002+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61002e4f0 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.002+0000 7fc60dffb640 1 -- 172.21.15.33:0/4087110738 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fc618017360 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.002+0000 7fc62f3a8740 10 client.27488 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.002+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55fb5ad9c100 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc60dffb640 1 -- 172.21.15.33:0/4087110738 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fb5ac82da0 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc60dffb640 1 -- 172.21.15.33:0/4087110738 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fc6180a71a0 con 0x55fb5ad0a0a0 2024-09-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc62f3a8740 10 client.27488 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc62f3a8740 10 client.27488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc62f3a8740 20 client.27488 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad9c8c0 0x55fb5ad9ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc62f3a8740 4 client.27488 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.003+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc61002e4f0 con 0x55fb5ad9c8c0 2024-09-10T21:18:37.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.004+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad9c8c0 0x55fb5ad9ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:37.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.005+0000 7fc61dd9b640 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad9c8c0 0x55fb5ad9ece0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a156a357f8a705f server_cookie=527e514efc20490d in_seq=0 out_seq=0 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.005+0000 7fc60dffb640 10 client.27488 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.005+0000 7fc60dffb640 1 -- 172.21.15.33:0/4087110738 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x7fc61002e4f0 con 0x55fb5ad9c8c0 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.005+0000 7fc60dffb640 10 client.27488 handle_command_reply: tid=0 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.005+0000 7fc62f3a8740 1 client.27488 shutdown 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a0a0 msgr2=0x55fb5ad1e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad1e8a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc6180a8ac0 tx=0x7fc6180046c0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 shutdown_connections 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55fb5ad9c8c0 0x55fb5ad9ece0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55fb5ad16950 0x55fb5ad19850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fb5ad0a9c0 0x55fb5ad1ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 --2- 172.21.15.33:0/4087110738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55fb5ad0a0a0 0x55fb5ad1e8a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 >> 172.21.15.33:0/4087110738 conn(0x55fb5ad07ed0 msgr2=0x55fb5ad06700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc60dffb640 0 client.27488 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 shutdown_connections 2024-09-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.006+0000 7fc62f3a8740 1 -- 172.21.15.33:0/4087110738 wait complete. 2024-09-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.007+0000 7fc62f3a8740 20 client.27488 trim_cache size 0 max 16384 2024-09-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:18:37.012 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:18:37.013 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:18:37.015 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.016 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.017 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.018 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:18:37.019 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.020 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:18:37.021 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.023 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:18:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.026 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:37.027 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6140b5360 msgr2=0x7fc6140b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6140b5360 0x7fc6140b7820 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc620006010 tx=0x7fc620005d80 comp rx=0 tx=0).stop 2024-09-10T21:18:37.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc628103ca0 msgr2=0x7fc628199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc628103ca0 0x7fc628199290 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc61800b4a0 tx=0x7fc61800b960 comp rx=0 tx=0).stop 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 shutdown_connections 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc6140b5360 0x7fc6140b7820 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc62810d100 0x7fc62819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc6281045c0 0x7fc6281997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 --2- 172.21.15.33:0/2360117481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc628103ca0 0x7fc628199290 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 >> 172.21.15.33:0/2360117481 conn(0x7fc6280fd700 msgr2=0x7fc62810f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 shutdown_connections 2024-09-10T21:18:37.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:37.008+0000 7fc62cfd3640 1 -- 172.21.15.33:0/2360117481 wait complete. 2024-09-10T21:18:37.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 83477 48.23 MB/sec warmup 97 sec latency 102.559 ms 2024-09-10T21:18:37.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:37 smithi104 ceph-mon[35325]: pgmap v785: 97 pgs: 97 active+clean; 851 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 751 op/s 2024-09-10T21:18:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:37 smithi174 ceph-mon[35635]: pgmap v785: 97 pgs: 97 active+clean; 851 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 751 op/s 2024-09-10T21:18:37.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:37 smithi033 ceph-mon[30860]: pgmap v785: 97 pgs: 97 active+clean; 851 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.5 MiB/s wr, 751 op/s 2024-09-10T21:18:37.959 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:18:37.959 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:18:38.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 84362 48.18 MB/sec warmup 98 sec latency 64.833 ms 2024-09-10T21:18:38.329 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:39.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 85474 48.28 MB/sec warmup 99 sec latency 128.333 ms 2024-09-10T21:18:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.118+0000 7f8399ffb640 1 -- 172.21.15.33:0/3960588152 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8390037070 con 0x7f839c1076a0 2024-09-10T21:18:39.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 -- 172.21.15.33:0/3960588152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c1076a0 msgr2=0x7f839c107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 --2- 172.21.15.33:0/3960588152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c1076a0 0x7f839c107ae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8390009860 tx=0x7f839002a340 comp rx=0 tx=0).stop 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 -- 172.21.15.33:0/3960588152 shutdown_connections 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 --2- 172.21.15.33:0/3960588152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 0x7f839c10e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 --2- 172.21.15.33:0/3960588152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c1076a0 0x7f839c107ae0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 --2- 172.21.15.33:0/3960588152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f839c101c90 0x7f839c102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 -- 172.21.15.33:0/3960588152 >> 172.21.15.33:0/3960588152 conn(0x7f839c0fd700 msgr2=0x7f839c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 -- 172.21.15.33:0/3960588152 shutdown_connections 2024-09-10T21:18:39.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.119+0000 7f83a1736640 1 -- 172.21.15.33:0/3960588152 wait complete. 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.120+0000 7f83a1736640 1 Processor -- start 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f83a1736640 1 -- start start 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f83a1736640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c101c90 0x7f839c0748c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f83a1736640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f839c1076a0 0x7f839c074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f83a1736640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 0x7f839c06ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f83a1736640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f839c111d30 con 0x7f839c108020 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f83a1736640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f839c111bb0 con 0x7f839c101c90 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f83a1736640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f839c111eb0 con 0x7f839c1076a0 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 0x7f839c06ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f839b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 0x7f839c06ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46100/0 (socket says 172.21.15.33:46100) 2024-09-10T21:18:39.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.121+0000 7f839b7fe640 1 -- 172.21.15.33:0/825823388 learned_addr learned my addr 172.21.15.33:0/825823388 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.122+0000 7f839a7fc640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f839c1076a0 0x7f839c074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.122+0000 7f839affd640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c101c90 0x7f839c0748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.122+0000 7f839b7fe640 1 -- 172.21.15.33:0/825823388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f839c1076a0 msgr2=0x7f839c074e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.122+0000 7f839b7fe640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f839c1076a0 0x7f839c074e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.122+0000 7f839b7fe640 1 -- 172.21.15.33:0/825823388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c101c90 msgr2=0x7f839c0748c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.122+0000 7f839b7fe640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c101c90 0x7f839c0748c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.122+0000 7f839b7fe640 1 -- 172.21.15.33:0/825823388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f839c06f620 con 0x7f839c108020 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.123+0000 7f839b7fe640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 0x7f839c06ed60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8394030d60 tx=0x7f839402e750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.124+0000 7f8383fff640 1 -- 172.21.15.33:0/825823388 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839402ee60 con 0x7f839c108020 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.124+0000 7f8383fff640 1 -- 172.21.15.33:0/825823388 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8394003b70 con 0x7f839c108020 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.124+0000 7f8383fff640 1 -- 172.21.15.33:0/825823388 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839400ba10 con 0x7f839c108020 2024-09-10T21:18:39.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.124+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f839c06f910 con 0x7f839c108020 2024-09-10T21:18:39.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.125+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f839c1127d0 con 0x7f839c108020 2024-09-10T21:18:39.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.127+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f839c102f40 con 0x7f839c108020 2024-09-10T21:18:39.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.134+0000 7f8383fff640 1 -- 172.21.15.33:0/825823388 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f839400b040 con 0x7f839c108020 2024-09-10T21:18:39.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.136+0000 7f8383fff640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83840b54b0 0x7f83840b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:39.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.136+0000 7f8383fff640 1 -- 172.21.15.33:0/825823388 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f83941085c0 con 0x7f839c108020 2024-09-10T21:18:39.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.136+0000 7f839affd640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83840b54b0 0x7f83840b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.137+0000 7f839affd640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83840b54b0 0x7f83840b7970 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f838c009990 tx=0x7f838c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:39.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.139+0000 7f8383fff640 1 -- 172.21.15.33:0/825823388 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f8394046030 con 0x7f839c108020 2024-09-10T21:18:39.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.445+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f839c102070 con 0x7f839c108020 2024-09-10T21:18:39.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.445+0000 7f8383fff640 1 -- 172.21.15.33:0/825823388 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+11998 (secure 0 0 0) 0x7f839c102070 con 0x7f839c108020 2024-09-10T21:18:39.449 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:18:39.449 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":69,"btime":"2024-09-10T21:17:32:403122+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":69,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:17:31.774879+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:18:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 69 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83840b54b0 msgr2=0x7f83840b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83840b54b0 0x7f83840b7970 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f838c009990 tx=0x7f838c009110 comp rx=0 tx=0).stop 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 msgr2=0x7f839c06ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 0x7f839c06ed60 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8394030d60 tx=0x7f839402e750 comp rx=0 tx=0).stop 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 shutdown_connections 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83840b54b0 0x7f83840b7970 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f839c108020 0x7f839c06ed60 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f839c1076a0 0x7f839c074e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 --2- 172.21.15.33:0/825823388 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f839c101c90 0x7f839c0748c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 >> 172.21.15.33:0/825823388 conn(0x7f839c0fd700 msgr2=0x7f839c105b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 shutdown_connections 2024-09-10T21:18:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:39.448+0000 7f83a1736640 1 -- 172.21.15.33:0/825823388 wait complete. 2024-09-10T21:18:39.783 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:39 smithi033 ceph-mon[30860]: pgmap v786: 97 pgs: 97 active+clean; 865 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 951 op/s 2024-09-10T21:18:39.784 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:39 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/825823388' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:18:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:39 smithi104 ceph-mon[35325]: pgmap v786: 97 pgs: 97 active+clean; 865 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 951 op/s 2024-09-10T21:18:39.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:39 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/825823388' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:18:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:39 smithi174 ceph-mon[35635]: pgmap v786: 97 pgs: 97 active+clean; 865 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 951 op/s 2024-09-10T21:18:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:39 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/825823388' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:18:40.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 86366 48.25 MB/sec warmup 100 sec latency 240.191 ms 2024-09-10T21:18:40.152 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:18:40.550 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:41.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 87071 48.26 MB/sec warmup 101 sec latency 199.223 ms 2024-09-10T21:18:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 -- 172.21.15.33:0/1680316684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a41045c0 msgr2=0x7f07a410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 --2- 172.21.15.33:0/1680316684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a41045c0 0x7f07a410cab0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f078c009900 tx=0x7f078c02a410 comp rx=0 tx=0).stop 2024-09-10T21:18:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 -- 172.21.15.33:0/1680316684 shutdown_connections 2024-09-10T21:18:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 --2- 172.21.15.33:0/1680316684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a410d100 0x7f07a410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 --2- 172.21.15.33:0/1680316684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a41045c0 0x7f07a410cab0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 --2- 172.21.15.33:0/1680316684 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07a4103ca0 0x7f07a4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 -- 172.21.15.33:0/1680316684 >> 172.21.15.33:0/1680316684 conn(0x7f07a40fd700 msgr2=0x7f07a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 -- 172.21.15.33:0/1680316684 shutdown_connections 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.119+0000 7f07a887e640 1 -- 172.21.15.33:0/1680316684 wait complete. 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 Processor -- start 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 -- start start 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a4103ca0 0x7f07a4101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07a41045c0 0x7f07a4101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a410d100 0x7f07a41023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4112e00 con 0x7f07a4103ca0 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4112c80 con 0x7f07a41045c0 2024-09-10T21:18:41.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a887e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4112f80 con 0x7f07a410d100 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a4103ca0 0x7f07a4101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a4103ca0 0x7f07a4101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41278/0 (socket says 172.21.15.33:41278) 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a259c640 1 -- 172.21.15.33:0/3919233519 learned_addr learned my addr 172.21.15.33:0/3919233519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f079bfff640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07a41045c0 0x7f07a4101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.120+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a410d100 0x7f07a41023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a259c640 1 -- 172.21.15.33:0/3919233519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a410d100 msgr2=0x7f07a41023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a259c640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a410d100 0x7f07a41023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a259c640 1 -- 172.21.15.33:0/3919233519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07a41045c0 msgr2=0x7f07a4101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a259c640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07a41045c0 0x7f07a4101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a259c640 1 -- 172.21.15.33:0/3919233519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a41a7710 con 0x7f07a4103ca0 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a259c640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a4103ca0 0x7f07a4101950 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f079400e9f0 tx=0x7f079400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f079b7fe640 1 -- 172.21.15.33:0/3919233519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079400cd80 con 0x7f07a4103ca0 2024-09-10T21:18:41.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a41a7a00 con 0x7f07a4103ca0 2024-09-10T21:18:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f079b7fe640 1 -- 172.21.15.33:0/3919233519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940108e0 con 0x7f07a4103ca0 2024-09-10T21:18:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f079b7fe640 1 -- 172.21.15.33:0/3919233519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0794018980 con 0x7f07a4103ca0 2024-09-10T21:18:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.121+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a41a7f90 con 0x7f07a4103ca0 2024-09-10T21:18:41.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 Processor -- start 2024-09-10T21:18:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 -- start start 2024-09-10T21:18:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a158b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a2c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c9798bea0 con 0x563c97a154d0 2024-09-10T21:18:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563c9797b2d0 con 0x563c97a2a590 2024-09-10T21:18:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.125+0000 7f07aac53740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c9797f0e0 con 0x563c97a15df0 2024-09-10T21:18:41.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a2c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a2c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38892/0 (socket says 172.21.15.33:38892) 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 -- 172.21.15.33:0/2120982908 learned_addr learned my addr 172.21.15.33:0/2120982908 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a158b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 -- 172.21.15.33:0/2120982908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 msgr2=0x563c97a2a050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2a050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 -- 172.21.15.33:0/2120982908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 msgr2=0x563c97a158b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a158b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 -- 172.21.15.33:0/2120982908 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c976d9650 con 0x563c97a2a590 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079bfff640 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a2c980 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f078c009900 tx=0x7f078c0042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb0fc3b303b6edfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f079b7fe640 1 -- 172.21.15.33:0/3919233519 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0794010430 con 0x7f07a4103ca0 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.126+0000 7f0798ff9640 1 -- 172.21.15.33:0/2120982908 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c035040 con 0x563c97a2a590 2024-09-10T21:18:41.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.127+0000 7f0798ff9640 1 -- 172.21.15.33:0/2120982908 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f078c0044d0 con 0x563c97a2a590 2024-09-10T21:18:41.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.127+0000 7f0798ff9640 1 -- 172.21.15.33:0/2120982908 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f078c003850 con 0x563c97a2a590 2024-09-10T21:18:41.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.127+0000 7f079b7fe640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07900b52e0 0x7f07900b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07aac53740 1 -- 172.21.15.33:0/2120982908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 msgr2=0x563c97a2c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07aac53740 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a2c980 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f078c009900 tx=0x7f078c0042e0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f079b7fe640 1 -- 172.21.15.33:0/3919233519 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f07940e5f70 con 0x7f07a4103ca0 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07900b52e0 0x7f07900b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07aac53740 1 -- 172.21.15.33:0/2120982908 shutdown_connections 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07aac53740 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a2c980 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07aac53740 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07aac53740 1 --2- 172.21.15.33:0/2120982908 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.128+0000 7f07aac53740 1 -- 172.21.15.33:0/2120982908 >> 172.21.15.33:0/2120982908 conn(0x563c97a13300 msgr2=0x563c97a144a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.129+0000 7f07aac53740 1 -- 172.21.15.33:0/2120982908 shutdown_connections 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.129+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07900b52e0 0x7f07900b77a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f07a4103a10 tx=0x7f079c00e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.129+0000 7f07aac53740 1 -- 172.21.15.33:0/2120982908 wait complete. 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.130+0000 7f07aac53740 1 -- start start 2024-09-10T21:18:41.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.130+0000 7f07aac53740 1 Processor -- start 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.130+0000 7f07aac53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a29dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.130+0000 7f07aac53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.130+0000 7f07aac53740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a27750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a29dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a29dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41308/0 (socket says 172.21.15.33:41308) 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 -- 172.21.15.33:0/919240245 learned_addr learned my addr 172.21.15.33:0/919240245 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.130+0000 7f07aac53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563c977dc700 con 0x563c97a154d0 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563c97916160 con 0x563c97a2a590 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c97a2d230 con 0x563c97a15df0 2024-09-10T21:18:41.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a27750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07aac53740 10 client.0 _ll_register_callbacks cb 0x563c9792f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f079bfff640 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 -- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 msgr2=0x563c97a27210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 -- 172.21.15.33:0/919240245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 msgr2=0x563c97a27750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a27750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 -- 172.21.15.33:0/919240245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c97943c90 con 0x563c97a154d0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f07a259c640 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a29dc0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f07940a99a0 tx=0x7f07940ac080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e481acd6690b1ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f0789ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f0789ffb640 1 -- 172.21.15.33:0/919240245 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07940abb90 con 0x563c97a154d0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.131+0000 7f0789ffb640 1 -- 172.21.15.33:0/919240245 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0794018b20 con 0x563c97a154d0 2024-09-10T21:18:41.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f078c0044d0 con 0x563c97a154d0 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f0789ffb640 1 -- 172.21.15.33:0/919240245 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f07940ace30 con 0x563c97a154d0 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 10 client.18084 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563c97a38af0 con 0x563c97a154d0 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f0789ffb640 1 -- 172.21.15.33:0/919240245 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07940ac730 con 0x563c97a154d0 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f0789ffb640 1 -- 172.21.15.33:0/919240245 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f07940ab400 con 0x563c97a154d0 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 10 client.18084 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 10 client.18084 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 20 client.18084 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97a39f00 0x563c97a3c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 4 client.18084 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.132+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f078c0044d0 con 0x563c97a39f00 2024-09-10T21:18:41.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.133+0000 7f079bfff640 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97a39f00 0x563c97a3c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.133+0000 7f079bfff640 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97a39f00 0x563c97a3c2f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a25d7e20c693caa1 server_cookie=22e84cd9767a2a7a in_seq=0 out_seq=0 2024-09-10T21:18:41.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.133+0000 7f0789ffb640 10 client.18084 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:18:41.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f0789ffb640 1 -- 172.21.15.33:0/919240245 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f078c0044d0 con 0x563c97a39f00 2024-09-10T21:18:41.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f0789ffb640 10 client.18084 handle_command_reply: tid=0 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 client.18084 shutdown 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 msgr2=0x563c97a29dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a29dc0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f07940a99a0 tx=0x7f07940ac080 comp rx=0 tx=0).stop 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 shutdown_connections 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97a39f00 0x563c97a3c2f0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a2a590 0x563c97a27750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f0789ffb640 0 client.18084 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 --2- 172.21.15.33:0/919240245 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a154d0 0x563c97a29dc0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 >> 172.21.15.33:0/919240245 conn(0x563c97a13300 msgr2=0x563c97a18ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.137+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 shutdown_connections 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.138+0000 7f07aac53740 1 -- 172.21.15.33:0/919240245 wait complete. 2024-09-10T21:18:41.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.138+0000 7f07aac53740 20 client.18084 trim_cache size 0 max 16384 2024-09-10T21:18:41.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 Processor -- start 2024-09-10T21:18:41.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 -- start start 2024-09-10T21:18:41.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a26fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a41bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07a2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a41bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07a2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a41bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41324/0 (socket says 172.21.15.33:41324) 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07a2d9d640 1 -- 172.21.15.33:0/2070677506 learned_addr learned my addr 172.21.15.33:0/2070677506 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07a259c640 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a26fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41ac420 con 0x563c97a2a590 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41ae170 con 0x563c97a154d0 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.155+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41acb20 con 0x563c97a15df0 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.156+0000 7f079bfff640 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.156+0000 7f07a2d9d640 1 -- 172.21.15.33:0/2070677506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 msgr2=0x563c97a27500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:41.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.156+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.156+0000 7f07a2d9d640 1 -- 172.21.15.33:0/2070677506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 msgr2=0x563c97a26fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:41.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.156+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a26fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.156+0000 7f07a2d9d640 1 -- 172.21.15.33:0/2070677506 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c97a13130 con 0x563c97a2a590 2024-09-10T21:18:41.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.156+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a41bc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f079c018930 tx=0x7f079c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d9f24933fb8cfa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.157+0000 7f078bfff640 1 -- 172.21.15.33:0/2070677506 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c017b10 con 0x563c97a2a590 2024-09-10T21:18:41.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.157+0000 7f078bfff640 1 -- 172.21.15.33:0/2070677506 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f079c017cb0 con 0x563c97a2a590 2024-09-10T21:18:41.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.157+0000 7f078bfff640 1 -- 172.21.15.33:0/2070677506 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c022680 con 0x563c97a2a590 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.158+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 msgr2=0x563c97a41bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.158+0000 7f07aac53740 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a41bc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f079c018930 tx=0x7f079c013000 comp rx=0 tx=0).stop 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.158+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 shutdown_connections 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.158+0000 7f07aac53740 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a41bc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.158+0000 7f07aac53740 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a27500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.158+0000 7f07aac53740 1 --2- 172.21.15.33:0/2070677506 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a26fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.159+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 >> 172.21.15.33:0/2070677506 conn(0x563c97a13300 msgr2=0x563c97a25a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.159+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 shutdown_connections 2024-09-10T21:18:41.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.159+0000 7f07aac53740 1 -- 172.21.15.33:0/2070677506 wait complete. 2024-09-10T21:18:41.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.159+0000 7f07aac53740 1 -- start start 2024-09-10T21:18:41.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.159+0000 7f07aac53740 1 Processor -- start 2024-09-10T21:18:41.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07aac53740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07aac53740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07aac53740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a2fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07aac53740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41ada10 con 0x563c97a2a590 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07aac53740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41ad280 con 0x563c97a154d0 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07aac53740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07a41aa420 con 0x563c97a15df0 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07aac53740 10 client.0 _ll_register_callbacks cb 0x7f079400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07a2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a2fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f079bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.160+0000 7f07a2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a2fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41340/0 (socket says 172.21.15.33:41340) 2024-09-10T21:18:41.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f079bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40756/0 (socket says 172.21.15.33:40756) 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a2d9d640 1 -- 172.21.15.33:0/1656739191 learned_addr learned my addr 172.21.15.33:0/1656739191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a2d9d640 1 -- 172.21.15.33:0/1656739191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 msgr2=0x563c97a2f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a2d9d640 1 -- 172.21.15.33:0/1656739191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 msgr2=0x563c97a329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a2d9d640 1 -- 172.21.15.33:0/1656739191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c97a2ff90 con 0x563c97a2a590 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.161+0000 7f07a259c640 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.162+0000 7f07a2d9d640 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a2fa20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f079c00bd80 tx=0x7f079c004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ab3ca0a3886f6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:41.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.162+0000 7f0789ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.162+0000 7f0789ffb640 1 -- 172.21.15.33:0/1656739191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f079c032030 con 0x563c97a2a590 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.162+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f079c017cb0 con 0x563c97a2a590 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.162+0000 7f0789ffb640 1 -- 172.21.15.33:0/1656739191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f079c029040 con 0x563c97a2a590 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.163+0000 7f0789ffb640 1 -- 172.21.15.33:0/1656739191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f079c017b10 con 0x563c97a2a590 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.163+0000 7f07aac53740 10 client.18090 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.164+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563c97aa7b50 con 0x563c97a2a590 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.164+0000 7f0789ffb640 1 -- 172.21.15.33:0/1656739191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f079c037900 con 0x563c97a2a590 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.164+0000 7f0789ffb640 1 -- 172.21.15.33:0/1656739191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f079c022910 con 0x563c97a2a590 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.164+0000 7f07aac53740 10 client.18090 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.164+0000 7f07aac53740 10 client.18090 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:18:41.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.164+0000 7f07aac53740 20 client.18090 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:41.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.164+0000 7f07aac53740 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97aa81e0 0x563c97aaa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:41.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.165+0000 7f07aac53740 4 client.18090 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:18:41.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.165+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f079c017cb0 con 0x563c97aa81e0 2024-09-10T21:18:41.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.165+0000 7f07a259c640 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97aa81e0 0x563c97aaa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:41.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.167+0000 7f07a259c640 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97aa81e0 0x563c97aaa600 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c73dc403b32bd05f server_cookie=e51dc41475f4e894 in_seq=0 out_seq=0 2024-09-10T21:18:41.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.167+0000 7f0789ffb640 10 client.18090 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:18:41.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.179+0000 7f0789ffb640 1 -- 172.21.15.33:0/1656739191 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f079c017cb0 con 0x563c97aa81e0 2024-09-10T21:18:41.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f0789ffb640 10 client.18090 handle_command_reply: tid=0 2024-09-10T21:18:41.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 client.18090 shutdown 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 msgr2=0x563c97a2fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a2fa20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f079c00bd80 tx=0x7f079c004b50 comp rx=0 tx=0).stop 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 shutdown_connections 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x563c97aa81e0 0x563c97aaa600 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f0789ffb640 0 client.18090 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563c97a2a590 0x563c97a2fa20 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c97a15df0 0x563c97a2f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 --2- 172.21.15.33:0/1656739191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563c97a154d0 0x563c97a329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.180+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 >> 172.21.15.33:0/1656739191 conn(0x563c97a13300 msgr2=0x563c97a28840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.182+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 shutdown_connections 2024-09-10T21:18:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.182+0000 7f07aac53740 1 -- 172.21.15.33:0/1656739191 wait complete. 2024-09-10T21:18:41.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.183+0000 7f07aac53740 20 client.18090 trim_cache size 0 max 16384 2024-09-10T21:18:41.187 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:18:41.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.184+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07900b52e0 msgr2=0x7f07900b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.184+0000 7f07a887e640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07900b52e0 0x7f07900b77a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f07a4103a10 tx=0x7f079c00e040 comp rx=0 tx=0).stop 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.184+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a4103ca0 msgr2=0x7f07a4101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.184+0000 7f07a887e640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a4103ca0 0x7f07a4101950 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f079400e9f0 tx=0x7f079400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.184+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 shutdown_connections 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.185+0000 7f07a887e640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f07900b52e0 0x7f07900b77a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.185+0000 7f07a887e640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07a410d100 0x7f07a41023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.185+0000 7f07a887e640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07a41045c0 0x7f07a4101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.185+0000 7f07a887e640 1 --2- 172.21.15.33:0/3919233519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f07a4103ca0 0x7f07a4101950 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:41.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.185+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 >> 172.21.15.33:0/3919233519 conn(0x7f07a40fd700 msgr2=0x7f07a410f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:41.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.185+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 shutdown_connections 2024-09-10T21:18:41.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:41.185+0000 7f07a887e640 1 -- 172.21.15.33:0/3919233519 wait complete. 2024-09-10T21:18:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:41 smithi104 ceph-mon[35325]: pgmap v787: 97 pgs: 97 active+clean; 868 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T21:18:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:41 smithi174 ceph-mon[35635]: pgmap v787: 97 pgs: 97 active+clean; 868 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T21:18:41.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:41 smithi033 ceph-mon[30860]: pgmap v787: 97 pgs: 97 active+clean; 868 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T21:18:42.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 87825 48.13 MB/sec warmup 102 sec latency 117.327 ms 2024-09-10T21:18:42.932 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:18:43.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 88594 48.07 MB/sec warmup 103 sec latency 132.239 ms 2024-09-10T21:18:43.233 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:43.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:43 smithi033 ceph-mon[30860]: pgmap v788: 97 pgs: 97 active+clean; 871 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:18:43.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:43 smithi104 ceph-mon[35325]: pgmap v788: 97 pgs: 97 active+clean; 871 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:18:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:43 smithi174 ceph-mon[35635]: pgmap v788: 97 pgs: 97 active+clean; 871 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:18:44.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 89386 47.94 MB/sec warmup 104 sec latency 171.060 ms 2024-09-10T21:18:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 -- 172.21.15.33:0/3765011248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac10d100 msgr2=0x7fbeac10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 --2- 172.21.15.33:0/3765011248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac10d100 0x7fbeac10f560 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009ca0 tx=0x7fbea402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 -- 172.21.15.33:0/3765011248 shutdown_connections 2024-09-10T21:18:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 --2- 172.21.15.33:0/3765011248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac10d100 0x7fbeac10f560 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 --2- 172.21.15.33:0/3765011248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeac1045c0 0x7fbeac10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 --2- 172.21.15.33:0/3765011248 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac103ca0 0x7fbeac104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 -- 172.21.15.33:0/3765011248 >> 172.21.15.33:0/3765011248 conn(0x7fbeac0fd700 msgr2=0x7fbeac0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 -- 172.21.15.33:0/3765011248 shutdown_connections 2024-09-10T21:18:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.360+0000 7fbeb301b640 1 -- 172.21.15.33:0/3765011248 wait complete. 2024-09-10T21:18:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 Processor -- start 2024-09-10T21:18:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 -- start start 2024-09-10T21:18:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeac103ca0 0x7fbeac101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac1045c0 0x7fbeac101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac10d100 0x7fbeac1022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac112d70 con 0x7fbeac1045c0 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac112bf0 con 0x7fbeac10d100 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.361+0000 7fbeb301b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac112ef0 con 0x7fbeac103ca0 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac10d100 0x7fbeac1022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac10d100 0x7fbeac1022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38956/0 (socket says 172.21.15.33:38956) 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 -- 172.21.15.33:0/1768940038 learned_addr learned my addr 172.21.15.33:0/1768940038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeabfff640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac1045c0 0x7fbeac101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 -- 172.21.15.33:0/1768940038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeac103ca0 msgr2=0x7fbeac101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeac103ca0 0x7fbeac101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 -- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac1045c0 msgr2=0x7fbeac101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac1045c0 0x7fbeac101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.362+0000 7fbeb15b8640 1 -- 172.21.15.33:0/1768940038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac073b90 con 0x7fbeac10d100 2024-09-10T21:18:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.363+0000 7fbeb15b8640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac10d100 0x7fbeac1022b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005cc0 tx=0x7fbea4031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.363+0000 7fbea9ffb640 1 -- 172.21.15.33:0/1768940038 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003b70 con 0x7fbeac10d100 2024-09-10T21:18:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.363+0000 7fbea9ffb640 1 -- 172.21.15.33:0/1768940038 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea40318f0 con 0x7fbeac10d100 2024-09-10T21:18:44.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.363+0000 7fbea9ffb640 1 -- 172.21.15.33:0/1768940038 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea402f950 con 0x7fbeac10d100 2024-09-10T21:18:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.363+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac073e20 con 0x7fbeac10d100 2024-09-10T21:18:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.363+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac0743b0 con 0x7fbeac10d100 2024-09-10T21:18:44.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.366+0000 7fbeb53f0740 1 Processor -- start 2024-09-10T21:18:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.366+0000 7fbea9ffb640 1 -- 172.21.15.33:0/1768940038 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbea402fb50 con 0x7fbeac10d100 2024-09-10T21:18:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.366+0000 7fbeb53f0740 1 -- start start 2024-09-10T21:18:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394dad80 0x5600394dd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560039444ea0 con 0x5600394cedf0 2024-09-10T21:18:44.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5600394342d0 con 0x5600394ce4d0 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600394380e0 con 0x5600394dad80 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38960/0 (socket says 172.21.15.33:38960) 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeabfff640 1 -- 172.21.15.33:0/1353871456 learned_addr learned my addr 172.21.15.33:0/1353871456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeabfff640 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394dad80 0x5600394dd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbea9ffb640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe9c0b5360 0x7fbe9c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb0db7640 1 -- 172.21.15.33:0/1353871456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394dad80 msgr2=0x5600394dd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394dad80 0x5600394dd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb0db7640 1 -- 172.21.15.33:0/1353871456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 msgr2=0x5600394ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeb0db7640 1 -- 172.21.15.33:0/1353871456 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560039192650 con 0x5600394cedf0 2024-09-10T21:18:44.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.367+0000 7fbeabfff640 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.368+0000 7fbeb15b8640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe9c0b5360 0x7fbe9c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.368+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394da840 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009b40 tx=0x7fbea000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e946539c6e1c2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.368+0000 7fbea9ffb640 1 -- 172.21.15.33:0/1768940038 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbea4106d00 con 0x7fbeac10d100 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.368+0000 7fbe96ffd640 1 -- 172.21.15.33:0/1353871456 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea000ed80 con 0x5600394cedf0 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.368+0000 7fbe96ffd640 1 -- 172.21.15.33:0/1353871456 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea0005020 con 0x5600394cedf0 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.368+0000 7fbe96ffd640 1 -- 172.21.15.33:0/1353871456 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0004a00 con 0x5600394cedf0 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb53f0740 1 -- 172.21.15.33:0/1353871456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 msgr2=0x5600394da840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb15b8640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe9c0b5360 0x7fbe9c0b7820 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbea40cd450 tx=0x7fbea40047b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394da840 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbea0009b40 tx=0x7fbea000e350 comp rx=0 tx=0).stop 2024-09-10T21:18:44.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb53f0740 1 -- 172.21.15.33:0/1353871456 shutdown_connections 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394dad80 0x5600394dd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394da840 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/1353871456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.369+0000 7fbeb53f0740 1 -- 172.21.15.33:0/1353871456 >> 172.21.15.33:0/1353871456 conn(0x5600394cc300 msgr2=0x5600394cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.370+0000 7fbeb53f0740 1 -- 172.21.15.33:0/1353871456 shutdown_connections 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.370+0000 7fbeb53f0740 1 -- 172.21.15.33:0/1353871456 wait complete. 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.370+0000 7fbeb53f0740 1 -- start start 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.370+0000 7fbeb53f0740 1 Processor -- start 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394ce4d0 0x5600394da640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394d7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394d8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560039295700 con 0x5600394ce4d0 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5600393cf160 con 0x5600394dad80 2024-09-10T21:18:44.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600394dda00 con 0x5600394cedf0 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb53f0740 10 client.0 _ll_register_callbacks cb 0x5600393e89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394ce4d0 0x5600394da640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394ce4d0 0x5600394da640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41372/0 (socket says 172.21.15.33:41372) 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 -- 172.21.15.33:0/2502861118 learned_addr learned my addr 172.21.15.33:0/2502861118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394d8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394d7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 -- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 msgr2=0x5600394d7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394d7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 -- 172.21.15.33:0/2502861118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394ce4d0 msgr2=0x5600394da640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394ce4d0 0x5600394da640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.371+0000 7fbeabfff640 1 -- 172.21.15.33:0/2502861118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600393fcc90 con 0x5600394dad80 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbeabfff640 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394d8070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe98011e90 tx=0x7fbe98009820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=119f39af928bcef5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbe8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbe8bfff640 1 -- 172.21.15.33:0/2502861118 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98011050 con 0x5600394dad80 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbe8bfff640 1 -- 172.21.15.33:0/2502861118 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbe980111f0 con 0x5600394dad80 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbea0005020 con 0x5600394dad80 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbe8bfff640 1 -- 172.21.15.33:0/2502861118 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbe98022420 con 0x5600394dad80 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbeb53f0740 10 client.27521 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.372+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5600394e9390 con 0x5600394dad80 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbe8bfff640 1 -- 172.21.15.33:0/2502861118 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea0005020 con 0x5600394dad80 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbe8bfff640 1 -- 172.21.15.33:0/2502861118 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fbe9802c400 con 0x5600394dad80 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbeb53f0740 10 client.27521 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbeb53f0740 10 client.27521 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbeb53f0740 20 client.27521 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x5600394ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbeb53f0740 4 client.27521 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:44.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560039446df0 con 0x5600394ea860 2024-09-10T21:18:44.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.373+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x5600394ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.374+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x5600394ecc70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92eae0d228f2d4f2 server_cookie=fe676fae78106b08 in_seq=0 out_seq=0 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.374+0000 7fbe8bfff640 10 client.27521 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.375+0000 7fbe8bfff640 1 -- 172.21.15.33:0/2502861118 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560039446df0 con 0x5600394ea860 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.375+0000 7fbe8bfff640 10 client.27521 handle_command_reply: tid=0 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 client.27521 shutdown 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 msgr2=0x5600394d8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394d8070 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbe98011e90 tx=0x7fbe98009820 comp rx=0 tx=0).stop 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 shutdown_connections 2024-09-10T21:18:44.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394d8070 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x5600394ecc70 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbe8bfff640 0 client.27521 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394d7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2502861118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394ce4d0 0x5600394da640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 >> 172.21.15.33:0/2502861118 conn(0x5600394cc300 msgr2=0x5600394d1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 shutdown_connections 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2502861118 wait complete. 2024-09-10T21:18:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.376+0000 7fbeb53f0740 20 client.27521 trim_cache size 0 max 16384 2024-09-10T21:18:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 Processor -- start 2024-09-10T21:18:44.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 -- start start 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394ce4d0 0x5600394e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394d97d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394dc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1bd1d0 con 0x5600394cedf0 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1bef20 con 0x5600394dad80 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.393+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1bd8d0 con 0x5600394ce4d0 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394d97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394d97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41388/0 (socket says 172.21.15.33:41388) 2024-09-10T21:18:44.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 -- 172.21.15.33:0/2596297244 learned_addr learned my addr 172.21.15.33:0/2596297244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeabfff640 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394dc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeabfff640 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394ce4d0 0x5600394e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 -- 172.21.15.33:0/2596297244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394ce4d0 msgr2=0x5600394e6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394ce4d0 0x5600394e6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 -- 172.21.15.33:0/2596297244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 msgr2=0x5600394dc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394dc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 -- 172.21.15.33:0/2596297244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600394cec20 con 0x5600394cedf0 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.394+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394d97d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x5600394cd1e0 tx=0x7fbea00093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a21b518ad6b3538 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbe967fc640 1 -- 172.21.15.33:0/2596297244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0029070 con 0x5600394cedf0 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbe967fc640 1 -- 172.21.15.33:0/2596297244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbea00097c0 con 0x5600394cedf0 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbe967fc640 1 -- 172.21.15.33:0/2596297244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbea0004b20 con 0x5600394cedf0 2024-09-10T21:18:44.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2596297244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 msgr2=0x5600394d97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394d97d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x5600394cd1e0 tx=0x7fbea00093b0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2596297244 shutdown_connections 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394dad80 0x5600394dc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.395+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394cedf0 0x5600394d97d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.396+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/2596297244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394ce4d0 0x5600394e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.396+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2596297244 >> 172.21.15.33:0/2596297244 conn(0x5600394cc300 msgr2=0x5600394d90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.396+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2596297244 shutdown_connections 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.396+0000 7fbeb53f0740 1 -- 172.21.15.33:0/2596297244 wait complete. 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.396+0000 7fbeb53f0740 1 -- start start 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.396+0000 7fbeb53f0740 1 Processor -- start 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394e3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb53f0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394dad80 0x5600394e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1be7c0 con 0x5600394dad80 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1be030 con 0x5600394ce4d0 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb53f0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac1bb190 con 0x5600394cedf0 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb53f0740 10 client.0 _ll_register_callbacks cb 0x7fbe98009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb0db7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394dad80 0x5600394e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb0db7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394e0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40812/0 (socket says 172.21.15.33:40812) 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeb0db7640 1 -- 172.21.15.33:0/499541142 learned_addr learned my addr 172.21.15.33:0/499541142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeabfff640 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394e3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeabfff640 1 -- 172.21.15.33:0/499541142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 msgr2=0x5600394e0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeabfff640 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394e0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeabfff640 1 -- 172.21.15.33:0/499541142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 msgr2=0x5600394e3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeabfff640 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394e3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.397+0000 7fbeabfff640 1 -- 172.21.15.33:0/499541142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600394e1c20 con 0x5600394dad80 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbeabfff640 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394dad80 0x5600394e13f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fbe98011c20 tx=0x7fbe98009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864bde9724d7f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbe8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbe8bfff640 1 -- 172.21.15.33:0/499541142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98032040 con 0x5600394dad80 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbe8bfff640 1 -- 172.21.15.33:0/499541142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbe9802cbd0 con 0x5600394dad80 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5600394428a0 con 0x5600394dad80 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbe8bfff640 1 -- 172.21.15.33:0/499541142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fbe9803c050 con 0x5600394dad80 2024-09-10T21:18:44.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbeb53f0740 10 client.18108 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.398+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x560039558710 con 0x5600394dad80 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbe8bfff640 1 -- 172.21.15.33:0/499541142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5600394428a0 con 0x5600394dad80 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbe8bfff640 1 -- 172.21.15.33:0/499541142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fbe98011130 con 0x5600394dad80 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbeb53f0740 10 client.18108 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbeb53f0740 10 client.18108 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbeb53f0740 20 client.18108 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x56003955af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbeb53f0740 4 client.18108 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56003943fcd0 con 0x5600394ea860 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.399+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x56003955af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.400+0000 7fbeb0db7640 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x56003955af90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53686599082c0561 server_cookie=fea142c09fc9e61d in_seq=0 out_seq=0 2024-09-10T21:18:44.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.400+0000 7fbe8bfff640 10 client.18108 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:18:44.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbe8bfff640 1 -- 172.21.15.33:0/499541142 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56003943fcd0 con 0x5600394ea860 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbe8bfff640 10 client.18108 handle_command_reply: tid=0 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 client.18108 shutdown 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394dad80 msgr2=0x5600394e13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394dad80 0x5600394e13f0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fbe98011c20 tx=0x7fbe98009820 comp rx=0 tx=0).stop 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 shutdown_connections 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5600394dad80 0x5600394e13f0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5600394ea860 0x56003955af90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600394cedf0 0x5600394e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 --2- 172.21.15.33:0/499541142 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5600394ce4d0 0x5600394e3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbe8bfff640 0 client.18108 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 >> 172.21.15.33:0/499541142 conn(0x5600394cc300 msgr2=0x5600394d98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.404+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 shutdown_connections 2024-09-10T21:18:44.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb53f0740 1 -- 172.21.15.33:0/499541142 wait complete. 2024-09-10T21:18:44.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb53f0740 20 client.18108 trim_cache size 0 max 16384 2024-09-10T21:18:44.410 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:18:44.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe9c0b5360 msgr2=0x7fbe9c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe9c0b5360 0x7fbe9c0b7820 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fbea40cd450 tx=0x7fbea40047b0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac10d100 msgr2=0x7fbeac1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac10d100 0x7fbeac1022b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fbea4005cc0 tx=0x7fbea4031700 comp rx=0 tx=0).stop 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 shutdown_connections 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbe9c0b5360 0x7fbe9c0b7820 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbeac10d100 0x7fbeac1022b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbeac1045c0 0x7fbeac101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 --2- 172.21.15.33:0/1768940038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbeac103ca0 0x7fbeac101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.405+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 >> 172.21.15.33:0/1768940038 conn(0x7fbeac0fd700 msgr2=0x7fbeac0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.406+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 shutdown_connections 2024-09-10T21:18:44.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:44.406+0000 7fbeb301b640 1 -- 172.21.15.33:0/1768940038 wait complete. 2024-09-10T21:18:45.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 90446 48.08 MB/sec warmup 105 sec latency 76.054 ms 2024-09-10T21:18:45.134 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:18:45.502 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:45.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:45 smithi104 ceph-mon[35325]: pgmap v789: 97 pgs: 97 active+clean; 880 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 972 op/s 2024-09-10T21:18:45.812 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:45 smithi033 ceph-mon[30860]: pgmap v789: 97 pgs: 97 active+clean; 880 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 972 op/s 2024-09-10T21:18:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:45 smithi174 ceph-mon[35635]: pgmap v789: 97 pgs: 97 active+clean; 880 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 972 op/s 2024-09-10T21:18:46.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 91153 48.03 MB/sec warmup 106 sec latency 259.414 ms 2024-09-10T21:18:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.326+0000 7f22d7330640 1 -- 172.21.15.33:0/402072907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0109150 msgr2=0x7f22d010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.326+0000 7f22d7330640 1 --2- 172.21.15.33:0/402072907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0109150 0x7f22d010f950 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009ca0 tx=0x7f22c802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.326+0000 7f22d7330640 1 -- 172.21.15.33:0/402072907 shutdown_connections 2024-09-10T21:18:46.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.326+0000 7f22d7330640 1 --2- 172.21.15.33:0/402072907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0109150 0x7f22d010f950 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.326+0000 7f22d7330640 1 --2- 172.21.15.33:0/402072907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22d01087d0 0x7f22d0108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.326+0000 7f22d7330640 1 --2- 172.21.15.33:0/402072907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22d0102dc0 0x7f22d01031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.326+0000 7f22d7330640 1 -- 172.21.15.33:0/402072907 >> 172.21.15.33:0/402072907 conn(0x7f22d00fe870 msgr2=0x7f22d0100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.327+0000 7f22d7330640 1 -- 172.21.15.33:0/402072907 shutdown_connections 2024-09-10T21:18:46.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.327+0000 7f22d7330640 1 -- 172.21.15.33:0/402072907 wait complete. 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 Processor -- start 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 -- start start 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0102dc0 0x7f22d0076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22d01087d0 0x7f22d0109940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22d0109150 0x7f22d0109e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22d006d430 con 0x7f22d01087d0 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22d006d2b0 con 0x7f22d0109150 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d7330640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22d006d5b0 con 0x7f22d0102dc0 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d50cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0102dc0 0x7f22d0076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d50cc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0102dc0 0x7f22d0076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40816/0 (socket says 172.21.15.33:40816) 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.328+0000 7f22d50cc640 1 -- 172.21.15.33:0/1468928883 learned_addr learned my addr 172.21.15.33:0/1468928883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d48cb640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22d01087d0 0x7f22d0109940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d58cd640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22d0109150 0x7f22d0109e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d50cc640 1 -- 172.21.15.33:0/1468928883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22d0109150 msgr2=0x7f22d0109e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d50cc640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22d0109150 0x7f22d0109e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d50cc640 1 -- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22d01087d0 msgr2=0x7f22d0109940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d50cc640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22d01087d0 0x7f22d0109940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d50cc640 1 -- 172.21.15.33:0/1468928883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22d010d990 con 0x7f22d0102dc0 2024-09-10T21:18:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d48cb640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22d01087d0 0x7f22d0109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.329+0000 7f22d58cd640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22d0109150 0x7f22d0109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:46.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.330+0000 7f22d50cc640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0102dc0 0x7f22d0076ce0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00e9f0 tx=0x7f22cc00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.330+0000 7f22c67fc640 1 -- 172.21.15.33:0/1468928883 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc00cd80 con 0x7f22d0102dc0 2024-09-10T21:18:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.330+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22d010a420 con 0x7f22d0102dc0 2024-09-10T21:18:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.330+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22d01b4c30 con 0x7f22d0102dc0 2024-09-10T21:18:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.330+0000 7f22c67fc640 1 -- 172.21.15.33:0/1468928883 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22cc0054d0 con 0x7f22d0102dc0 2024-09-10T21:18:46.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.331+0000 7f22c67fc640 1 -- 172.21.15.33:0/1468928883 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22cc010690 con 0x7f22d0102dc0 2024-09-10T21:18:46.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.335+0000 7f22d9705740 1 Processor -- start 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.336+0000 7f22d9705740 1 -- start start 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d9705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a5d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d9705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5d9a10 0x563b6a5e5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d9705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5e59a0 0x563b6a5e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d9705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563b6a54fea0 con 0x563b6a5e59a0 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d9705740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563b6a53f2d0 con 0x563b6a5d90f0 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d9705740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b6a5430e0 con 0x563b6a5d9a10 2024-09-10T21:18:46.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d58cd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a5d94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d58cd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a5d94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39030/0 (socket says 172.21.15.33:39030) 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.337+0000 7f22d58cd640 1 -- 172.21.15.33:0/3244930352 learned_addr learned my addr 172.21.15.33:0/3244930352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.338+0000 7f22d48cb640 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5e59a0 0x563b6a5e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.338+0000 7f22d58cd640 1 -- 172.21.15.33:0/3244930352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5d9a10 msgr2=0x563b6a5e5460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.338+0000 7f22d58cd640 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5d9a10 0x563b6a5e5460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.338+0000 7f22d58cd640 1 -- 172.21.15.33:0/3244930352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5e59a0 msgr2=0x563b6a5e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.338+0000 7f22d58cd640 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5e59a0 0x563b6a5e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.338+0000 7f22d58cd640 1 -- 172.21.15.33:0/3244930352 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6a29d650 con 0x563b6a5d90f0 2024-09-10T21:18:46.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.338+0000 7f22c67fc640 1 -- 172.21.15.33:0/1468928883 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f22cc0046c0 con 0x7f22d0102dc0 2024-09-10T21:18:46.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.339+0000 7f22d58cd640 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a5d94d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009890 tx=0x7f22c80098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=257fde0745946cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.340+0000 7f22c67fc640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22bc0b5420 0x7f22bc0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.340+0000 7f22bb7fe640 1 -- 172.21.15.33:0/3244930352 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8031cf0 con 0x563b6a5d90f0 2024-09-10T21:18:46.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.340+0000 7f22d50cc640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22bc0b5420 0x7f22bc0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.340+0000 7f22bb7fe640 1 -- 172.21.15.33:0/3244930352 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c8036070 con 0x563b6a5d90f0 2024-09-10T21:18:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.340+0000 7f22c67fc640 1 -- 172.21.15.33:0/1468928883 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f22cc0e5f60 con 0x7f22d0102dc0 2024-09-10T21:18:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.340+0000 7f22bb7fe640 1 -- 172.21.15.33:0/3244930352 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c80042a0 con 0x563b6a5d90f0 2024-09-10T21:18:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.341+0000 7f22d9705740 1 -- 172.21.15.33:0/3244930352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 msgr2=0x563b6a5d94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.341+0000 7f22d9705740 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a5d94d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009890 tx=0x7f22c80098c0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.341+0000 7f22d50cc640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22bc0b5420 0x7f22bc0b78e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0a8cb0 tx=0x7f22cc0dcb50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.341+0000 7f22d9705740 1 -- 172.21.15.33:0/3244930352 shutdown_connections 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.341+0000 7f22d9705740 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5e59a0 0x563b6a5e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.341+0000 7f22d9705740 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5d9a10 0x563b6a5e5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.341+0000 7f22d9705740 1 --2- 172.21.15.33:0/3244930352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a5d94d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.342+0000 7f22d9705740 1 -- 172.21.15.33:0/3244930352 >> 172.21.15.33:0/3244930352 conn(0x563b6a5d6f20 msgr2=0x563b6a5d80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.342+0000 7f22d9705740 1 -- 172.21.15.33:0/3244930352 shutdown_connections 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.342+0000 7f22d9705740 1 -- 172.21.15.33:0/3244930352 wait complete. 2024-09-10T21:18:46.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.342+0000 7f22d9705740 1 -- start start 2024-09-10T21:18:46.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 1 Processor -- start 2024-09-10T21:18:46.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5e51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5e2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563b6a3a0700 con 0x563b6a5d90f0 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x563b6a4da160 con 0x563b6a5d9a10 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b6a5f95d0 con 0x563b6a5e59a0 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.343+0000 7f22d9705740 10 client.0 _ll_register_callbacks cb 0x563b6a4f39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d48cb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5e2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d48cb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5e2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39044/0 (socket says 172.21.15.33:39044) 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d48cb640 1 -- 172.21.15.33:0/512022987 learned_addr learned my addr 172.21.15.33:0/512022987 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d58cd640 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5e51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d58cd640 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d58cd640 1 -- 172.21.15.33:0/512022987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 msgr2=0x563b6a5e2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d58cd640 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d58cd640 1 -- 172.21.15.33:0/512022987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 msgr2=0x563b6a5e2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d58cd640 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5e2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.344+0000 7f22d58cd640 1 -- 172.21.15.33:0/512022987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6a507c90 con 0x563b6a5d90f0 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.345+0000 7f22b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.345+0000 7f22d58cd640 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5e51f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f22c802f4d0 tx=0x7f22c8003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954bbbd8e906bc69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.345+0000 7f22b8ff9640 1 -- 172.21.15.33:0/512022987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8058060 con 0x563b6a5d90f0 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.345+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c8036070 con 0x563b6a5d90f0 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.345+0000 7f22b8ff9640 1 -- 172.21.15.33:0/512022987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f22c8031450 con 0x563b6a5d90f0 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.346+0000 7f22b8ff9640 1 -- 172.21.15.33:0/512022987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f22c8058060 con 0x563b6a5d90f0 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.346+0000 7f22d9705740 10 client.18117 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:46.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.346+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563b6a604ef0 con 0x563b6a5d90f0 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.346+0000 7f22b8ff9640 1 -- 172.21.15.33:0/512022987 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c8036070 con 0x563b6a5d90f0 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.346+0000 7f22b8ff9640 1 -- 172.21.15.33:0/512022987 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f22c802fc80 con 0x563b6a5d90f0 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.347+0000 7f22d9705740 10 client.18117 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.347+0000 7f22d9705740 10 client.18117 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.347+0000 7f22d9705740 20 client.18117 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.347+0000 7f22d9705740 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a606240 0x563b6a608630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.347+0000 7f22d9705740 4 client.18117 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.347+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b6a551df0 con 0x563b6a606240 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.347+0000 7f22d48cb640 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a606240 0x563b6a608630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.348+0000 7f22b8ff9640 10 client.18117 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.348+0000 7f22d48cb640 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a606240 0x563b6a608630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5d43eaf636df7505 server_cookie=3843c5a0f1b9408d in_seq=0 out_seq=0 2024-09-10T21:18:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.349+0000 7f22b8ff9640 1 -- 172.21.15.33:0/512022987 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b6a551df0 con 0x563b6a606240 2024-09-10T21:18:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.349+0000 7f22b8ff9640 10 client.18117 handle_command_reply: tid=0 2024-09-10T21:18:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.349+0000 7f22d9705740 1 client.18117 shutdown 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.349+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 msgr2=0x563b6a5e51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.349+0000 7f22d9705740 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5e51f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f22c802f4d0 tx=0x7f22c8003e50 comp rx=0 tx=0).stop 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 shutdown_connections 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22d9705740 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22d9705740 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5e2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22d9705740 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a606240 0x563b6a608630 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22d9705740 1 --2- 172.21.15.33:0/512022987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5e51f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 >> 172.21.15.33:0/512022987 conn(0x563b6a5d6f20 msgr2=0x563b6a5dca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22b8ff9640 0 client.18117 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:18:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.350+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 shutdown_connections 2024-09-10T21:18:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.351+0000 7f22d9705740 1 -- 172.21.15.33:0/512022987 wait complete. 2024-09-10T21:18:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.352+0000 7f22d9705740 20 client.18117 trim_cache size 0 max 16384 2024-09-10T21:18:46.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.378+0000 7f22d9705740 1 Processor -- start 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d9705740 1 -- start start 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d9705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d9705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5d8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d9705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d9705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22b400e790 con 0x563b6a5d90f0 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d9705740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22b40104e0 con 0x563b6a5d9a10 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d9705740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22b400ee90 con 0x563b6a5e59a0 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d58cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d58cd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5d7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41446/0 (socket says 172.21.15.33:41446) 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.379+0000 7f22d58cd640 1 -- 172.21.15.33:0/1041365937 learned_addr learned my addr 172.21.15.33:0/1041365937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d58cd640 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d48cb640 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5d8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d58cd640 1 -- 172.21.15.33:0/1041365937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 msgr2=0x563b6a5d8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d58cd640 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5d8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d58cd640 1 -- 172.21.15.33:0/1041365937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 msgr2=0x563b6a5d7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d58cd640 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5d7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d58cd640 1 -- 172.21.15.33:0/1041365937 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b6a5e5560 con 0x563b6a5e59a0 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.380+0000 7f22d58cd640 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e0ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009a60 tx=0x7f22c8009560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4507ad52e1ea9ec0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.381+0000 7f22baffd640 1 -- 172.21.15.33:0/1041365937 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8031e60 con 0x563b6a5e59a0 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.381+0000 7f22baffd640 1 -- 172.21.15.33:0/1041365937 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c8036040 con 0x563b6a5e59a0 2024-09-10T21:18:46.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.381+0000 7f22baffd640 1 -- 172.21.15.33:0/1041365937 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8058060 con 0x563b6a5e59a0 2024-09-10T21:18:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.382+0000 7f22d9705740 1 -- 172.21.15.33:0/1041365937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 msgr2=0x563b6a5e0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.382+0000 7f22d9705740 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e0ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f22c8009a60 tx=0x7f22c8009560 comp rx=0 tx=0).stop 2024-09-10T21:18:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.383+0000 7f22d9705740 1 -- 172.21.15.33:0/1041365937 shutdown_connections 2024-09-10T21:18:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.383+0000 7f22d9705740 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a5e0ec0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.383+0000 7f22d9705740 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d9a10 0x563b6a5d8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.383+0000 7f22d9705740 1 --2- 172.21.15.33:0/1041365937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d90f0 0x563b6a5d7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.383+0000 7f22d9705740 1 -- 172.21.15.33:0/1041365937 >> 172.21.15.33:0/1041365937 conn(0x563b6a5d6f20 msgr2=0x563b6a5e23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.383+0000 7f22d9705740 1 -- 172.21.15.33:0/1041365937 shutdown_connections 2024-09-10T21:18:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.383+0000 7f22d9705740 1 -- 172.21.15.33:0/1041365937 wait complete. 2024-09-10T21:18:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.384+0000 7f22d9705740 1 -- start start 2024-09-10T21:18:46.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.384+0000 7f22d9705740 1 Processor -- start 2024-09-10T21:18:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.385+0000 7f22d9705740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a609ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.385+0000 7f22d9705740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d9a10 0x563b6a60a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.385+0000 7f22d9705740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a60a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.385+0000 7f22d9705740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f22b400fd80 con 0x563b6a5d9a10 2024-09-10T21:18:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.385+0000 7f22d9705740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f22b400f5f0 con 0x563b6a5d90f0 2024-09-10T21:18:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.385+0000 7f22d9705740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22b400c710 con 0x563b6a5e59a0 2024-09-10T21:18:46.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.385+0000 7f22d9705740 10 client.0 _ll_register_callbacks cb 0x7f22c0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a609ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a609ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39056/0 (socket says 172.21.15.33:39056) 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 -- 172.21.15.33:0/3426782446 learned_addr learned my addr 172.21.15.33:0/3426782446 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a60a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d48cb640 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d9a10 0x563b6a60a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 -- 172.21.15.33:0/3426782446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 msgr2=0x563b6a60a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a60a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 -- 172.21.15.33:0/3426782446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d9a10 msgr2=0x563b6a60a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d9a10 0x563b6a60a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 -- 172.21.15.33:0/3426782446 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b6a671dd0 con 0x563b6a5d90f0 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22d58cd640 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a609ac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22c80040d0 tx=0x7f22c8003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54b1bf44f042f66b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:46.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.386+0000 7f22b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.387+0000 7f22b8ff9640 1 -- 172.21.15.33:0/3426782446 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22c8031e60 con 0x563b6a5d90f0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.387+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22c8036040 con 0x563b6a5d90f0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.387+0000 7f22b8ff9640 1 -- 172.21.15.33:0/3426782446 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f22c8031450 con 0x563b6a5d90f0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.387+0000 7f22b8ff9640 1 -- 172.21.15.33:0/3426782446 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f22c8058040 con 0x563b6a5d90f0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.387+0000 7f22d9705740 10 client.27542 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.387+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x563b6a6730d0 con 0x563b6a5d90f0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.387+0000 7f22b8ff9640 1 -- 172.21.15.33:0/3426782446 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22c8036040 con 0x563b6a5d90f0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.388+0000 7f22b8ff9640 1 -- 172.21.15.33:0/3426782446 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f22c803a8b0 con 0x563b6a5d90f0 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.388+0000 7f22d9705740 10 client.27542 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.388+0000 7f22d9705740 10 client.27542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.388+0000 7f22d9705740 20 client.27542 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.388+0000 7f22d9705740 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a6738b0 0x563b6a675cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.388+0000 7f22d9705740 4 client.27542 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.388+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563b6a54d8a0 con 0x563b6a6738b0 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.389+0000 7f22d48cb640 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a6738b0 0x563b6a675cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.389+0000 7f22b8ff9640 10 client.27542 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.389+0000 7f22d48cb640 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a6738b0 0x563b6a675cd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de8d67e3b10d28ae server_cookie=eb8cef63b540e103 in_seq=0 out_seq=0 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22b8ff9640 1 -- 172.21.15.33:0/3426782446 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563b6a54d8a0 con 0x563b6a6738b0 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22b8ff9640 10 client.27542 handle_command_reply: tid=0 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 client.27542 shutdown 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 msgr2=0x563b6a609ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a609ac0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f22c80040d0 tx=0x7f22c8003c20 comp rx=0 tx=0).stop 2024-09-10T21:18:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 shutdown_connections 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b6a5e59a0 0x563b6a60a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x563b6a6738b0 0x563b6a675cd0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x563b6a5d9a10 0x563b6a60a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 --2- 172.21.15.33:0/3426782446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x563b6a5d90f0 0x563b6a609ac0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22b8ff9640 0 client.27542 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.390+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 >> 172.21.15.33:0/3426782446 conn(0x563b6a5d6f20 msgr2=0x563b6a5dd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.391+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 shutdown_connections 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.391+0000 7f22d9705740 1 -- 172.21.15.33:0/3426782446 wait complete. 2024-09-10T21:18:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.391+0000 7f22d9705740 20 client.27542 trim_cache size 0 max 16384 2024-09-10T21:18:46.398 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:18:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22bc0b5420 msgr2=0x7f22bc0b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22bc0b5420 0x7f22bc0b78e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0a8cb0 tx=0x7f22cc0dcb50 comp rx=0 tx=0).stop 2024-09-10T21:18:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0102dc0 msgr2=0x7f22d0076ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:46.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0102dc0 0x7f22d0076ce0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00e9f0 tx=0x7f22cc00edf0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 shutdown_connections 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f22bc0b5420 0x7f22bc0b78e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f22d0109150 0x7f22d0109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f22d01087d0 0x7f22d0109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 --2- 172.21.15.33:0/1468928883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f22d0102dc0 0x7f22d0076ce0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 >> 172.21.15.33:0/1468928883 conn(0x7f22d00fe870 msgr2=0x7f22d00fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.392+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 shutdown_connections 2024-09-10T21:18:46.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:46.394+0000 7f22d7330640 1 -- 172.21.15.33:0/1468928883 wait complete. 2024-09-10T21:18:47.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 91849 47.92 MB/sec warmup 107 sec latency 174.081 ms 2024-09-10T21:18:47.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:47 smithi033 ceph-mon[30860]: pgmap v790: 97 pgs: 97 active+clean; 886 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-09-10T21:18:47.726 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:18:47.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:47 smithi104 ceph-mon[35325]: pgmap v790: 97 pgs: 97 active+clean; 886 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-09-10T21:18:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:47 smithi174 ceph-mon[35635]: pgmap v790: 97 pgs: 97 active+clean; 886 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-09-10T21:18:48.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 92606 47.81 MB/sec warmup 108 sec latency 127.518 ms 2024-09-10T21:18:48.094 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 -- 172.21.15.33:0/2625051100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c102de0 msgr2=0x7fa37c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 --2- 172.21.15.33:0/2625051100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c102de0 0x7fa37c1031c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa374009ca0 tx=0x7fa37402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 -- 172.21.15.33:0/2625051100 shutdown_connections 2024-09-10T21:18:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 --2- 172.21.15.33:0/2625051100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa37c109170 0x7fa37c10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 --2- 172.21.15.33:0/2625051100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa37c1087f0 0x7fa37c108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 --2- 172.21.15.33:0/2625051100 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c102de0 0x7fa37c1031c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 -- 172.21.15.33:0/2625051100 >> 172.21.15.33:0/2625051100 conn(0x7fa37c0fe830 msgr2=0x7fa37c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 -- 172.21.15.33:0/2625051100 shutdown_connections 2024-09-10T21:18:48.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.811+0000 7fa38285d640 1 -- 172.21.15.33:0/2625051100 wait complete. 2024-09-10T21:18:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.812+0000 7fa38285d640 1 Processor -- start 2024-09-10T21:18:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.812+0000 7fa38285d640 1 -- start start 2024-09-10T21:18:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.812+0000 7fa38285d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa37c102de0 0x7fa37c199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa38285d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c1087f0 0x7fa37c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa38285d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa37c109170 0x7fa37c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa38285d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c112e80 con 0x7fa37c109170 2024-09-10T21:18:48.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa38285d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c112d00 con 0x7fa37c1087f0 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa38285d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c113000 con 0x7fa37c102de0 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa37b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c1087f0 0x7fa37c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa380dfa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa37c109170 0x7fa37c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa37b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c1087f0 0x7fa37c199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39074/0 (socket says 172.21.15.33:39074) 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa37b7fe640 1 -- 172.21.15.33:0/1334161607 learned_addr learned my addr 172.21.15.33:0/1334161607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa380dfa640 1 -- 172.21.15.33:0/1334161607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa37c102de0 msgr2=0x7fa37c199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa380dfa640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa37c102de0 0x7fa37c199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa380dfa640 1 -- 172.21.15.33:0/1334161607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c1087f0 msgr2=0x7fa37c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa380dfa640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c1087f0 0x7fa37c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.813+0000 7fa380dfa640 1 -- 172.21.15.33:0/1334161607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa37c19f3c0 con 0x7fa37c109170 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.814+0000 7fa380dfa640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa37c109170 0x7fa37c19ebf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00b410 tx=0x7fa36c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.814+0000 7fa3797fa640 1 -- 172.21.15.33:0/1334161607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c015020 con 0x7fa37c109170 2024-09-10T21:18:48.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.814+0000 7fa3797fa640 1 -- 172.21.15.33:0/1334161607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa36c011ce0 con 0x7fa37c109170 2024-09-10T21:18:48.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.814+0000 7fa3797fa640 1 -- 172.21.15.33:0/1334161607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c010da0 con 0x7fa37c109170 2024-09-10T21:18:48.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.814+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa37c19f6b0 con 0x7fa37c109170 2024-09-10T21:18:48.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.814+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa37c1a5df0 con 0x7fa37c109170 2024-09-10T21:18:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.818+0000 7fa384c32740 1 Processor -- start 2024-09-10T21:18:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.818+0000 7fa3797fa640 1 -- 172.21.15.33:0/1334161607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa36c011830 con 0x7fa37c109170 2024-09-10T21:18:48.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa384c32740 1 -- start start 2024-09-10T21:18:48.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa384c32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348076f50 0x55c348077330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa384c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348086cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa384c32740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c348090be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa384c32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c347fefea0 con 0x55c348086870 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa384c32740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c347fdf2d0 con 0x55c3480871f0 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa384c32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c347fe30e0 con 0x55c348076f50 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348086cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348086cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41476/0 (socket says 172.21.15.33:41476) 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa37b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c348090be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa37bfff640 1 -- 172.21.15.33:0/758712468 learned_addr learned my addr 172.21.15.33:0/758712468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.819+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348076f50 0x55c348077330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37b7fe640 1 -- 172.21.15.33:0/758712468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348076f50 msgr2=0x55c348077330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:48.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348076f50 0x55c348077330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37b7fe640 1 -- 172.21.15.33:0/758712468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 msgr2=0x55c348086cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348086cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37b7fe640 1 -- 172.21.15.33:0/758712468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c347d3d650 con 0x55c3480871f0 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37bfff640 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348086cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa3797fa640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3640b54b0 0x7fa3640b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37bfff640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3640b54b0 0x7fa3640b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa3797fa640 1 -- 172.21.15.33:0/1334161607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa36c0e54f0 con 0x7fa37c109170 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.820+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c348090be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa370009620 tx=0x7fa370009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152b91f76a4a2f69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.821+0000 7fa3627fc640 1 -- 172.21.15.33:0/758712468 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370004030 con 0x55c3480871f0 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.821+0000 7fa3627fc640 1 -- 172.21.15.33:0/758712468 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3700054d0 con 0x55c3480871f0 2024-09-10T21:18:48.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.821+0000 7fa3627fc640 1 -- 172.21.15.33:0/758712468 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370004b20 con 0x55c3480871f0 2024-09-10T21:18:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.821+0000 7fa384c32740 1 -- 172.21.15.33:0/758712468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 msgr2=0x55c348090be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.821+0000 7fa384c32740 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c348090be0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa370009620 tx=0x7fa370009530 comp rx=0 tx=0).stop 2024-09-10T21:18:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.821+0000 7fa37bfff640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3640b54b0 0x7fa3640b7970 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa374031040 tx=0x7fa374004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 -- 172.21.15.33:0/758712468 shutdown_connections 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c348090be0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348086cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 --2- 172.21.15.33:0/758712468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348076f50 0x55c348077330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 -- 172.21.15.33:0/758712468 >> 172.21.15.33:0/758712468 conn(0x55c348079470 msgr2=0x55c348079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 -- 172.21.15.33:0/758712468 shutdown_connections 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 -- 172.21.15.33:0/758712468 wait complete. 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 -- start start 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.822+0000 7fa384c32740 1 Processor -- start 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa384c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348076f50 0x55c348090ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa384c32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34808df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348076f50 0x55c348090ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa384c32740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c34808e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa384c32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c347e40700 con 0x55c348076f50 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa384c32740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c347f7a160 con 0x55c3480871f0 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa384c32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3480912d0 con 0x55c348086870 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348076f50 0x55c348090ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41478/0 (socket says 172.21.15.33:41478) 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 -- 172.21.15.33:0/1516315082 learned_addr learned my addr 172.21.15.33:0/1516315082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa384c32740 10 client.0 _ll_register_callbacks cb 0x55c347f939e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34808df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 -- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 msgr2=0x55c34808df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34808df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 -- 172.21.15.33:0/1516315082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 msgr2=0x55c34808e5c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c34808e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.823+0000 7fa37bfff640 1 -- 172.21.15.33:0/1516315082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c347fa7c90 con 0x55c348076f50 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.824+0000 7fa37bfff640 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348076f50 0x55c348090ac0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa374009dc0 tx=0x7fa3740095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850a4c99ba2f04e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.824+0000 7fa35b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:48.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.825+0000 7fa35b7fe640 1 -- 172.21.15.33:0/1516315082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374004450 con 0x55c348076f50 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.825+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3700054d0 con 0x55c348076f50 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.825+0000 7fa35b7fe640 1 -- 172.21.15.33:0/1516315082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa3740045f0 con 0x55c348076f50 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.825+0000 7fa35b7fe640 1 -- 172.21.15.33:0/1516315082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fa374003e70 con 0x55c348076f50 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.825+0000 7fa384c32740 10 client.18135 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.825+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55c34809cc50 con 0x55c348076f50 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.826+0000 7fa35b7fe640 1 -- 172.21.15.33:0/1516315082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa374036040 con 0x55c348076f50 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.826+0000 7fa35b7fe640 1 -- 172.21.15.33:0/1516315082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fa37404e070 con 0x55c348076f50 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.826+0000 7fa384c32740 10 client.18135 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.826+0000 7fa384c32740 10 client.18135 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.826+0000 7fa384c32740 20 client.18135 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.826+0000 7fa384c32740 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34809e130 0x55c3480a05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.826+0000 7fa384c32740 4 client.18135 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.827+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa3700054d0 con 0x55c34809e130 2024-09-10T21:18:48.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.827+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34809e130 0x55c3480a05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.827+0000 7fa35b7fe640 10 client.18135 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:18:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.827+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34809e130 0x55c3480a05b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6048699062f4dea1 server_cookie=ddaba56f69495499 in_seq=0 out_seq=0 2024-09-10T21:18:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.829+0000 7fa35b7fe640 1 -- 172.21.15.33:0/1516315082 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fa3700054d0 con 0x55c34809e130 2024-09-10T21:18:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.829+0000 7fa35b7fe640 10 client.18135 handle_command_reply: tid=0 2024-09-10T21:18:48.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.829+0000 7fa384c32740 1 client.18135 shutdown 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.829+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348076f50 msgr2=0x55c348090ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.829+0000 7fa384c32740 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348076f50 0x55c348090ac0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa374009dc0 tx=0x7fa3740095c0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 shutdown_connections 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3480871f0 0x55c34808e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34808df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34809e130 0x55c3480a05b0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 --2- 172.21.15.33:0/1516315082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348076f50 0x55c348090ac0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 >> 172.21.15.33:0/1516315082 conn(0x55c348079470 msgr2=0x55c348089180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa35b7fe640 0 client.18135 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 shutdown_connections 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.830+0000 7fa384c32740 1 -- 172.21.15.33:0/1516315082 wait complete. 2024-09-10T21:18:48.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.831+0000 7fa384c32740 20 client.18135 trim_cache size 0 max 16384 2024-09-10T21:18:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.849+0000 7fa384c32740 1 Processor -- start 2024-09-10T21:18:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.849+0000 7fa384c32740 1 -- start start 2024-09-10T21:18:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.849+0000 7fa384c32740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c34808baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.849+0000 7fa384c32740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34809a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.849+0000 7fa384c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3480871f0 0x55c348072480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa384c32740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1ac430 con 0x55c3480871f0 2024-09-10T21:18:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c34808baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa384c32740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1ae180 con 0x55c348076f50 2024-09-10T21:18:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa384c32740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1acb30 con 0x55c348086870 2024-09-10T21:18:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c34808baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39100/0 (socket says 172.21.15.33:39100) 2024-09-10T21:18:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 -- 172.21.15.33:0/3478302151 learned_addr learned my addr 172.21.15.33:0/3478302151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34809a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa380dfa640 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3480871f0 0x55c348072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 -- 172.21.15.33:0/3478302151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 msgr2=0x55c34809a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34809a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 -- 172.21.15.33:0/3478302151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3480871f0 msgr2=0x55c348072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3480871f0 0x55c348072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 -- 172.21.15.33:0/3478302151 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c348090e30 con 0x55c348076f50 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.850+0000 7fa37bfff640 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c34808baf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa374031590 tx=0x7fa3740304f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a18ed492ca244b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.851+0000 7fa361ffb640 1 -- 172.21.15.33:0/3478302151 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa374031e20 con 0x55c348076f50 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.851+0000 7fa361ffb640 1 -- 172.21.15.33:0/3478302151 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa374036070 con 0x55c348076f50 2024-09-10T21:18:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.851+0000 7fa361ffb640 1 -- 172.21.15.33:0/3478302151 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37405b6c0 con 0x55c348076f50 2024-09-10T21:18:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 -- 172.21.15.33:0/3478302151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 msgr2=0x55c34808baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c34808baf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa374031590 tx=0x7fa3740304f0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 -- 172.21.15.33:0/3478302151 shutdown_connections 2024-09-10T21:18:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3480871f0 0x55c348072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c348086870 0x55c34809a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 --2- 172.21.15.33:0/3478302151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c34808baf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 -- 172.21.15.33:0/3478302151 >> 172.21.15.33:0/3478302151 conn(0x55c348079470 msgr2=0x55c3480a0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 -- 172.21.15.33:0/3478302151 shutdown_connections 2024-09-10T21:18:48.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.852+0000 7fa384c32740 1 -- 172.21.15.33:0/3478302151 wait complete. 2024-09-10T21:18:48.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.853+0000 7fa384c32740 1 -- start start 2024-09-10T21:18:48.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.853+0000 7fa384c32740 1 Processor -- start 2024-09-10T21:18:48.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.854+0000 7fa384c32740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c348088a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.854+0000 7fa384c32740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348088f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.854+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c348088a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.854+0000 7fa37bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c348088a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39110/0 (socket says 172.21.15.33:39110) 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.854+0000 7fa37bfff640 1 -- 172.21.15.33:0/925067782 learned_addr learned my addr 172.21.15.33:0/925067782 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.854+0000 7fa384c32740 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3480871f0 0x55c348089490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.855+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1ada20 con 0x55c348086870 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.855+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1ad290 con 0x55c348076f50 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.855+0000 7fa380dfa640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3480871f0 0x55c348089490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.855+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa37c1aa3f0 con 0x55c3480871f0 2024-09-10T21:18:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.855+0000 7fa384c32740 10 client.0 _ll_register_callbacks cb 0x7fa374009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:48.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.855+0000 7fa37bfff640 1 -- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3480871f0 msgr2=0x55c348089490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa37bfff640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3480871f0 0x55c348089490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa37bfff640 1 -- 172.21.15.33:0/925067782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 msgr2=0x55c348088f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348088f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa37bfff640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348088f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa37bfff640 1 -- 172.21.15.33:0/925067782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c348095d10 con 0x55c348076f50 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa380dfa640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3480871f0 0x55c348089490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa37bfff640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c348088a10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa374030580 tx=0x7fa374004df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9382aa9401116da2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa35b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348088f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa374036070 con 0x55c348076f50 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa35b7fe640 1 -- 172.21.15.33:0/925067782 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37404a040 con 0x55c348076f50 2024-09-10T21:18:48.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.856+0000 7fa35b7fe640 1 -- 172.21.15.33:0/925067782 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa374004660 con 0x55c348076f50 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa35b7fe640 1 -- 172.21.15.33:0/925067782 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fa37405b460 con 0x55c348076f50 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 10 client.27557 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=69}) -- 0x55c34810b380 con 0x55c348076f50 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa35b7fe640 1 -- 172.21.15.33:0/925067782 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7fa37404e040 con 0x55c348076f50 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 10 client.27557 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 10 client.27557 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 20 client.27557 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34810ba70 0x55c34810de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 4 client.27557 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.857+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c347ff1df0 con 0x55c34810ba70 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.858+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34810ba70 0x55c34810de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:48.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.858+0000 7fa37b7fe640 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34810ba70 0x55c34810de60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbf855d369b7de3b server_cookie=176fd89582297568 in_seq=0 out_seq=0 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.858+0000 7fa35b7fe640 10 client.27557 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa35b7fe640 1 -- 172.21.15.33:0/925067782 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c347ff1df0 con 0x55c34810ba70 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa35b7fe640 10 client.27557 handle_command_reply: tid=0 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 client.27557 shutdown 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 msgr2=0x55c348088a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c348088a10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa374030580 tx=0x7fa374004df0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 shutdown_connections 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3480871f0 0x55c348089490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c34810ba70 0x55c34810de60 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c348086870 0x55c348088f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 --2- 172.21.15.33:0/925067782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c348076f50 0x55c348088a10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa35b7fe640 0 client.27557 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:18:48.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.859+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 >> 172.21.15.33:0/925067782 conn(0x55c348079470 msgr2=0x55c348075c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.860+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 shutdown_connections 2024-09-10T21:18:48.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.860+0000 7fa384c32740 1 -- 172.21.15.33:0/925067782 wait complete. 2024-09-10T21:18:48.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.860+0000 7fa384c32740 20 client.27557 trim_cache size 0 max 16384 2024-09-10T21:18:48.868 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.861+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3640b54b0 msgr2=0x7fa3640b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.861+0000 7fa38285d640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3640b54b0 0x7fa3640b7970 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa374031040 tx=0x7fa374004a40 comp rx=0 tx=0).stop 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.861+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa37c109170 msgr2=0x7fa37c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.861+0000 7fa38285d640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa37c109170 0x7fa37c19ebf0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa36c00b410 tx=0x7fa36c00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 shutdown_connections 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa3640b54b0 0x7fa3640b7970 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa37c109170 0x7fa37c19ebf0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa37c1087f0 0x7fa37c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 --2- 172.21.15.33:0/1334161607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa37c102de0 0x7fa37c199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 >> 172.21.15.33:0/1334161607 conn(0x7fa37c0fe830 msgr2=0x7fa37c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:48.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 shutdown_connections 2024-09-10T21:18:48.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:48.862+0000 7fa38285d640 1 -- 172.21.15.33:0/1334161607 wait complete. 2024-09-10T21:18:49.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 93486 47.80 MB/sec warmup 109 sec latency 137.300 ms 2024-09-10T21:18:49.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:49 smithi104 ceph-mon[35325]: pgmap v791: 97 pgs: 97 active+clean; 894 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-09-10T21:18:49.857 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:49 smithi033 ceph-mon[30860]: pgmap v791: 97 pgs: 97 active+clean; 894 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-09-10T21:18:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:49 smithi174 ceph-mon[35635]: pgmap v791: 97 pgs: 97 active+clean; 894 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.00k op/s 2024-09-10T21:18:50.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 94596 47.99 MB/sec warmup 110 sec latency 57.673 ms 2024-09-10T21:18:50.196 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:18:50.559 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:51.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 95090 47.81 MB/sec warmup 111 sec latency 183.766 ms 2024-09-10T21:18:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.253+0000 7ff92b3f4640 1 -- 172.21.15.33:0/4058078345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff924101c90 msgr2=0x7ff924102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.253+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/4058078345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff924101c90 0x7ff924102070 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7ff920009900 tx=0x7ff92002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.253+0000 7ff92b3f4640 1 -- 172.21.15.33:0/4058078345 shutdown_connections 2024-09-10T21:18:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.253+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/4058078345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924108020 0x7ff92410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.253+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/4058078345 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9241076a0 0x7ff924107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.253+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/4058078345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff924101c90 0x7ff924102070 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.253+0000 7ff92b3f4640 1 -- 172.21.15.33:0/4058078345 >> 172.21.15.33:0/4058078345 conn(0x7ff9240fd700 msgr2=0x7ff9240ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.254+0000 7ff92b3f4640 1 -- 172.21.15.33:0/4058078345 shutdown_connections 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.254+0000 7ff92b3f4640 1 -- 172.21.15.33:0/4058078345 wait complete. 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.254+0000 7ff92b3f4640 1 Processor -- start 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92b3f4640 1 -- start start 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92b3f4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924101c90 0x7ff924108910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92b3f4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241076a0 0x7ff924108e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92b3f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924108020 0x7ff924109390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92b3f4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff924111cf0 con 0x7ff9241076a0 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92b3f4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff924111b70 con 0x7ff924101c90 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92b3f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff924111e70 con 0x7ff924108020 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92898f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241076a0 0x7ff924108e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92898f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241076a0 0x7ff924108e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37938/0 (socket says 172.21.15.33:37938) 2024-09-10T21:18:51.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff92898f640 1 -- 172.21.15.33:0/1307938725 learned_addr learned my addr 172.21.15.33:0/1307938725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff929190640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924101c90 0x7ff924108910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.255+0000 7ff929991640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924108020 0x7ff924109390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff929190640 1 -- 172.21.15.33:0/1307938725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924108020 msgr2=0x7ff924109390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff929190640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924108020 0x7ff924109390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff929190640 1 -- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241076a0 msgr2=0x7ff924108e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff929190640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241076a0 0x7ff924108e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff929190640 1 -- 172.21.15.33:0/1307938725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9241a76e0 con 0x7ff924101c90 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff929190640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924101c90 0x7ff924108910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff92002a9e0 tx=0x7ff920003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff91a7fc640 1 -- 172.21.15.33:0/1307938725 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92003f070 con 0x7ff924101c90 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff924198fd0 con 0x7ff924101c90 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff91a7fc640 1 -- 172.21.15.33:0/1307938725 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff920031b50 con 0x7ff924101c90 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff91a7fc640 1 -- 172.21.15.33:0/1307938725 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff920035030 con 0x7ff924101c90 2024-09-10T21:18:51.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.256+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9241994d0 con 0x7ff924101c90 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.260+0000 7ff91a7fc640 1 -- 172.21.15.33:0/1307938725 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff92004c060 con 0x7ff924101c90 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 Processor -- start 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 -- start start 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346bf9a0 0x5636346c1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x563634629ea0 con 0x5636346b30f0 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636346192d0 con 0x5636346bf9a0 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56363461d0e0 con 0x5636346b3a10 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff91a7fc640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9100b5360 0x7ff9100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff929991640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff929190640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92898f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346bf9a0 0x5636346c1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff929190640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346bf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:56142/0 (socket says 172.21.15.33:56142) 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff929190640 1 -- 172.21.15.33:0/2888505521 learned_addr learned my addr 172.21.15.33:0/2888505521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.261+0000 7ff92898f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346bf9a0 0x5636346c1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51702/0 (socket says 172.21.15.33:51702) 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9100b5360 0x7ff9100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff91a7fc640 1 -- 172.21.15.33:0/1307938725 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff920105dd0 con 0x7ff924101c90 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 -- 172.21.15.33:0/2888505521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 msgr2=0x5636346bf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346bf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 -- 172.21.15.33:0/2888505521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346bf9a0 msgr2=0x5636346c1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346bf9a0 0x5636346c1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 -- 172.21.15.33:0/2888505521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563634377650 con 0x5636346b30f0 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929190640 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346bf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:51.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346b34d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005fd0 tx=0x7ff91c014a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda1b70151681ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff929991640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9100b5360 0x7ff9100b7820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff91c012fd0 tx=0x7ff91c018000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.262+0000 7ff90f7fe640 1 -- 172.21.15.33:0/2888505521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c014490 con 0x5636346b30f0 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.263+0000 7ff90f7fe640 1 -- 172.21.15.33:0/2888505521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff91c012d00 con 0x5636346b30f0 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.263+0000 7ff90f7fe640 1 -- 172.21.15.33:0/2888505521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c005770 con 0x5636346b30f0 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2888505521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 msgr2=0x5636346b34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346b34d0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005fd0 tx=0x7ff91c014a10 comp rx=0 tx=0).stop 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2888505521 shutdown_connections 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346bf9a0 0x5636346c1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346bf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2888505521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346b34d0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2888505521 >> 172.21.15.33:0/2888505521 conn(0x5636346b0f20 msgr2=0x5636346b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2888505521 shutdown_connections 2024-09-10T21:18:51.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2888505521 wait complete. 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.264+0000 7ff92d7c9740 1 -- start start 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.265+0000 7ff92d7c9740 1 Processor -- start 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.265+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.265+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.265+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346bf9a0 0x5636346bcdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56363447a700 con 0x5636346b30f0 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5636345b4160 con 0x5636346b3a10 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5636346c25b0 con 0x5636346bf9a0 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff92898f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346bf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37958/0 (socket says 172.21.15.33:37958) 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 -- 172.21.15.33:0/2794486318 learned_addr learned my addr 172.21.15.33:0/2794486318 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff92d7c9740 10 client.0 _ll_register_callbacks cb 0x5636345cd9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929991640 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346bf9a0 0x5636346bcdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 -- 172.21.15.33:0/2794486318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346bf9a0 msgr2=0x5636346bcdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346bf9a0 0x5636346bcdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 -- 172.21.15.33:0/2794486318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 msgr2=0x5636346bc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346bc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 -- 172.21.15.33:0/2794486318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636345e1c90 con 0x5636346b30f0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff929190640 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346bf2e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff920006010 tx=0x7ff9200f3610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493d4a2108a7a243 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.266+0000 7ff90cff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff90cff9640 1 -- 172.21.15.33:0/2794486318 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9200f3060 con 0x5636346b30f0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff90cff9640 1 -- 172.21.15.33:0/2794486318 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff9200cc8a0 con 0x5636346b30f0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff91c012d00 con 0x5636346b30f0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff90cff9640 1 -- 172.21.15.33:0/2794486318 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff920049070 con 0x5636346b30f0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 10 client.18150 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x5636346cdf10 con 0x5636346b30f0 2024-09-10T21:18:51.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff90cff9640 1 -- 172.21.15.33:0/2794486318 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff91c012d00 con 0x5636346b30f0 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff90cff9640 1 -- 172.21.15.33:0/2794486318 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7ff9201021a0 con 0x5636346b30f0 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 10 client.18150 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 10 client.18150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 20 client.18150 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5636346cf300 0x5636346d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 4 client.18150 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.267+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56363462bdf0 con 0x5636346cf300 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.268+0000 7ff92898f640 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5636346cf300 0x5636346d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.268+0000 7ff92898f640 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5636346cf300 0x5636346d1740 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5476c62131bc165 server_cookie=8af1769fb98242ab in_seq=0 out_seq=0 2024-09-10T21:18:51.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.268+0000 7ff90cff9640 10 client.18150 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff90cff9640 1 -- 172.21.15.33:0/2794486318 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56363462bdf0 con 0x5636346cf300 2024-09-10T21:18:51.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff90cff9640 10 client.18150 handle_command_reply: tid=0 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 client.18150 shutdown 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 msgr2=0x5636346bf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346bf2e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff920006010 tx=0x7ff9200f3610 comp rx=0 tx=0).stop 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 shutdown_connections 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5636346cf300 0x5636346d1740 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346bf9a0 0x5636346bcdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/2794486318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346b30f0 0x5636346bf2e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 >> 172.21.15.33:0/2794486318 conn(0x5636346b0f20 msgr2=0x5636346b6a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.269+0000 7ff90cff9640 0 client.18150 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.270+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 shutdown_connections 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.270+0000 7ff92d7c9740 1 -- 172.21.15.33:0/2794486318 wait complete. 2024-09-10T21:18:51.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.270+0000 7ff92d7c9740 20 client.18150 trim_cache size 0 max 16384 2024-09-10T21:18:51.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 Processor -- start 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 -- start start 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b30f0 0x5636346cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346be470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241ac1f0 con 0x5636346bf9a0 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241adf40 con 0x5636346b30f0 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241ac8f0 con 0x5636346b3a10 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37968/0 (socket says 172.21.15.33:37968) 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929190640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b30f0 0x5636346cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 -- 172.21.15.33:0/1927568343 learned_addr learned my addr 172.21.15.33:0/1927568343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff92898f640 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346be470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 -- 172.21.15.33:0/1927568343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 msgr2=0x5636346be470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346be470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 -- 172.21.15.33:0/1927568343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b30f0 msgr2=0x5636346cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b30f0 0x5636346cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.286+0000 7ff929991640 1 -- 172.21.15.33:0/1927568343 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636346d1dc0 con 0x5636346bf9a0 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.287+0000 7ff929991640 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c0d90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005c40 tx=0x7ff91c014c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a65200400bb37e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.287+0000 7ff90effd640 1 -- 172.21.15.33:0/1927568343 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c039070 con 0x5636346bf9a0 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.287+0000 7ff90effd640 1 -- 172.21.15.33:0/1927568343 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff91c031040 con 0x5636346bf9a0 2024-09-10T21:18:51.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.287+0000 7ff90effd640 1 -- 172.21.15.33:0/1927568343 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c030050 con 0x5636346bf9a0 2024-09-10T21:18:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1927568343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 msgr2=0x5636346c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c0d90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7ff91c005c40 tx=0x7ff91c014c60 comp rx=0 tx=0).stop 2024-09-10T21:18:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1927568343 shutdown_connections 2024-09-10T21:18:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c0d90 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b3a10 0x5636346be470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1927568343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b30f0 0x5636346cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1927568343 >> 172.21.15.33:0/1927568343 conn(0x5636346b0f20 msgr2=0x5636346bdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1927568343 shutdown_connections 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1927568343 wait complete. 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 -- start start 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.288+0000 7ff92d7c9740 1 Processor -- start 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b30f0 0x5636346c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92d7c9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241ad7e0 con 0x5636346bf9a0 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241ad050 con 0x5636346b3a10 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92d7c9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9241aa1f0 con 0x5636346b30f0 2024-09-10T21:18:51.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92d7c9740 10 client.0 _ll_register_callbacks cb 0x7ff920031f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92898f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff929190640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b30f0 0x5636346c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff929991640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92898f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346c3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51730/0 (socket says 172.21.15.33:51730) 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92898f640 1 -- 172.21.15.33:0/1312907084 learned_addr learned my addr 172.21.15.33:0/1312907084 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92898f640 1 -- 172.21.15.33:0/1312907084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b30f0 msgr2=0x5636346c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92898f640 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b30f0 0x5636346c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92898f640 1 -- 172.21.15.33:0/1312907084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 msgr2=0x5636346c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.289+0000 7ff92898f640 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff92898f640 1 -- 172.21.15.33:0/1312907084 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636346c46d0 con 0x5636346b3a10 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff929190640 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b30f0 0x5636346c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:51.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff92898f640 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346c3960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5636346d1ff0 tx=0x7ff914010ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38f760270de7ba9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff90cff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff90cff9640 1 -- 172.21.15.33:0/1312907084 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff914004230 con 0x5636346b3a10 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff90cff9640 1 -- 172.21.15.33:0/1312907084 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff9140043d0 con 0x5636346b3a10 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff91c031040 con 0x5636346b3a10 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff90cff9640 1 -- 172.21.15.33:0/1312907084 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7ff91400f7d0 con 0x5636346b3a10 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff92d7c9740 10 client.27575 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x56363473d2d0 con 0x5636346b3a10 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.290+0000 7ff90cff9640 1 -- 172.21.15.33:0/1312907084 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff91c031040 con 0x5636346b3a10 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff90cff9640 1 -- 172.21.15.33:0/1312907084 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7ff914003c30 con 0x5636346b3a10 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff92d7c9740 10 client.27575 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff92d7c9740 10 client.27575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff92d7c9740 20 client.27575 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:51.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56363473db10 0x56363473ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff92d7c9740 4 client.27575 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:18:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5636346278a0 con 0x56363473db10 2024-09-10T21:18:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff929190640 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56363473db10 0x56363473ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.291+0000 7ff90cff9640 10 client.27575 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:51.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.292+0000 7ff929190640 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56363473db10 0x56363473ff30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ea0275a8e70bb70 server_cookie=d9e48df6f0b9de64 in_seq=0 out_seq=0 2024-09-10T21:18:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff90cff9640 1 -- 172.21.15.33:0/1312907084 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5636346278a0 con 0x56363473db10 2024-09-10T21:18:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff90cff9640 10 client.27575 handle_command_reply: tid=0 2024-09-10T21:18:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 client.27575 shutdown 2024-09-10T21:18:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 msgr2=0x5636346c3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346c3960 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x5636346d1ff0 tx=0x7ff914010ad0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 shutdown_connections 2024-09-10T21:18:51.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56363473db10 0x56363473ff30 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5636346bf9a0 0x5636346c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5636346b3a10 0x5636346c3960 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff90cff9640 0 client.27575 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:51.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 --2- 172.21.15.33:0/1312907084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5636346b30f0 0x5636346c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 >> 172.21.15.33:0/1312907084 conn(0x5636346b0f20 msgr2=0x5636346be580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:51.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.293+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 shutdown_connections 2024-09-10T21:18:51.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92d7c9740 1 -- 172.21.15.33:0/1312907084 wait complete. 2024-09-10T21:18:51.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92d7c9740 20 client.27575 trim_cache size 0 max 16384 2024-09-10T21:18:51.304 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9100b5360 msgr2=0x7ff9100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9100b5360 0x7ff9100b7820 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff91c012fd0 tx=0x7ff91c018000 comp rx=0 tx=0).stop 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924101c90 msgr2=0x7ff924108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924101c90 0x7ff924108910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff92002a9e0 tx=0x7ff920003b90 comp rx=0 tx=0).stop 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 shutdown_connections 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9100b5360 0x7ff9100b7820 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff924108020 0x7ff924109390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9241076a0 0x7ff924108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 --2- 172.21.15.33:0/1307938725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff924101c90 0x7ff924108910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.294+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 >> 172.21.15.33:0/1307938725 conn(0x7ff9240fd700 msgr2=0x7ff92406a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:51.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.295+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 shutdown_connections 2024-09-10T21:18:51.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:51.295+0000 7ff92b3f4640 1 -- 172.21.15.33:0/1307938725 wait complete. 2024-09-10T21:18:51.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:51 smithi104 ceph-mon[35325]: pgmap v792: 97 pgs: 97 active+clean; 901 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:18:51.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:51 smithi174 ceph-mon[35635]: pgmap v792: 97 pgs: 97 active+clean; 901 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:18:51.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:51 smithi033 ceph-mon[30860]: pgmap v792: 97 pgs: 97 active+clean; 901 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:18:52.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 95904 47.66 MB/sec warmup 112 sec latency 60.200 ms 2024-09-10T21:18:53.035 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:18:53.035 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:18:53.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 97127 47.87 MB/sec warmup 113 sec latency 71.996 ms 2024-09-10T21:18:53.376 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:53.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:53 smithi104 ceph-mon[35325]: pgmap v793: 97 pgs: 97 active+clean; 902 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 878 op/s 2024-09-10T21:18:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:53 smithi174 ceph-mon[35635]: pgmap v793: 97 pgs: 97 active+clean; 902 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 878 op/s 2024-09-10T21:18:53.897 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:53 smithi033 ceph-mon[30860]: pgmap v793: 97 pgs: 97 active+clean; 902 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 878 op/s 2024-09-10T21:18:54.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 97898 47.82 MB/sec warmup 114 sec latency 179.173 ms 2024-09-10T21:18:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.264+0000 7f14c1b71640 1 -- 172.21.15.33:0/1841335468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 msgr2=0x7f14bc1031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.264+0000 7f14c1b71640 1 --2- 172.21.15.33:0/1841335468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc1031a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009900 tx=0x7f14ac02a410 comp rx=0 tx=0).stop 2024-09-10T21:18:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.264+0000 7f14c1b71640 1 -- 172.21.15.33:0/1841335468 shutdown_connections 2024-09-10T21:18:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.264+0000 7f14c1b71640 1 --2- 172.21.15.33:0/1841335468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc109150 0x7f14bc10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.264+0000 7f14c1b71640 1 --2- 172.21.15.33:0/1841335468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14bc1087d0 0x7f14bc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.264+0000 7f14c1b71640 1 --2- 172.21.15.33:0/1841335468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc1031a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.264+0000 7f14c1b71640 1 -- 172.21.15.33:0/1841335468 >> 172.21.15.33:0/1841335468 conn(0x7f14bc0fe870 msgr2=0x7f14bc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.265+0000 7f14c1b71640 1 -- 172.21.15.33:0/1841335468 shutdown_connections 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.265+0000 7f14c1b71640 1 -- 172.21.15.33:0/1841335468 wait complete. 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.265+0000 7f14c1b71640 1 Processor -- start 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.265+0000 7f14c1b71640 1 -- start start 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14c1b71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14c1b71640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14bc1087d0 0x7f14bc0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14c1b71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc109150 0x7f14bc075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14c1b71640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc112dd0 con 0x7f14bc102dc0 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14c1b71640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc112c50 con 0x7f14bc1087d0 2024-09-10T21:18:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14c1b71640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc112f50 con 0x7f14bc109150 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc0754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38004/0 (socket says 172.21.15.33:38004) 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14bb7fe640 1 -- 172.21.15.33:0/2792730176 learned_addr learned my addr 172.21.15.33:0/2792730176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14baffd640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14bc1087d0 0x7f14bc0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc109150 0x7f14bc075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14baffd640 1 -- 172.21.15.33:0/2792730176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc109150 msgr2=0x7f14bc075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14baffd640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc109150 0x7f14bc075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14baffd640 1 -- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 msgr2=0x7f14bc0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14baffd640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14baffd640 1 -- 172.21.15.33:0/2792730176 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc1a76a0 con 0x7f14bc1087d0 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.266+0000 7f14bb7fe640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.267+0000 7f14baffd640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14bc1087d0 0x7f14bc0759e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e9f0 tx=0x7f14b000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.267+0000 7f14b8ff9640 1 -- 172.21.15.33:0/2792730176 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b000ccc0 con 0x7f14bc1087d0 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.267+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc1a7990 con 0x7f14bc1087d0 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.267+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc1a7f20 con 0x7f14bc1087d0 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.267+0000 7f14b8ff9640 1 -- 172.21.15.33:0/2792730176 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b00108e0 con 0x7f14bc1087d0 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.267+0000 7f14b8ff9640 1 -- 172.21.15.33:0/2792730176 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0018750 con 0x7f14bc1087d0 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.270+0000 7f14b8ff9640 1 -- 172.21.15.33:0/2792730176 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f14b0010430 con 0x7f14bc1087d0 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.270+0000 7f14c3f46740 1 Processor -- start 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.270+0000 7f14c3f46740 1 -- start start 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.271+0000 7f14b8ff9640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f14a40b5360 0x7f14a40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14b8ff9640 1 -- 172.21.15.33:0/2792730176 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f14b00e5f60 con 0x7f14bc1087d0 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b630182c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63024250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b63026b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14c3f46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b62f8ee60 con 0x561b63024790 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14c3f46740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561b62f7e290 con 0x561b63017ee0 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14c3f46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b62f820a0 con 0x561b63018800 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f14a40b5360 0x7f14a40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14bb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b630182c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14bb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b630182c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51760/0 (socket says 172.21.15.33:51760) 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14bb7fe640 1 -- 172.21.15.33:0/1604368220 learned_addr learned my addr 172.21.15.33:0/1604368220 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14bbfff640 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b63026b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.272+0000 7f14baffd640 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63024250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14baffd640 1 -- 172.21.15.33:0/1604368220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 msgr2=0x561b630182c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14baffd640 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b630182c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14baffd640 1 -- 172.21.15.33:0/1604368220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 msgr2=0x561b63026b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14baffd640 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b63026b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14baffd640 1 -- 172.21.15.33:0/1604368220 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b62cdc930 con 0x561b63018800 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14baffd640 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63024250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f14b00ac720 tx=0x7f14b00ac000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=565ad654f1f0d2bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f14a40b5360 0x7f14a40b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f14bc077560 tx=0x7f14b4013b60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14a1ffb640 1 -- 172.21.15.33:0/1604368220 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00b5c70 con 0x561b63018800 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14a1ffb640 1 -- 172.21.15.33:0/1604368220 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b0010d40 con 0x561b63018800 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14a1ffb640 1 -- 172.21.15.33:0/1604368220 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00ab460 con 0x561b63018800 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.273+0000 7f14bb7fe640 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b630182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.274+0000 7f14c3f46740 1 -- 172.21.15.33:0/1604368220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 msgr2=0x561b63024250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.274+0000 7f14c3f46740 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63024250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f14b00ac720 tx=0x7f14b00ac000 comp rx=0 tx=0).stop 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.274+0000 7f14c3f46740 1 -- 172.21.15.33:0/1604368220 shutdown_connections 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.274+0000 7f14c3f46740 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b63026b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.274+0000 7f14c3f46740 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63024250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.274+0000 7f14c3f46740 1 --2- 172.21.15.33:0/1604368220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b630182c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.274+0000 7f14c3f46740 1 -- 172.21.15.33:0/1604368220 >> 172.21.15.33:0/1604368220 conn(0x561b63015d10 msgr2=0x561b63016eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.275+0000 7f14c3f46740 1 -- 172.21.15.33:0/1604368220 shutdown_connections 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.275+0000 7f14c3f46740 1 -- 172.21.15.33:0/1604368220 wait complete. 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.275+0000 7f14c3f46740 1 -- start start 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.275+0000 7f14c3f46740 1 Processor -- start 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 0x561b630217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63021d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63024790 0x561b63022270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14c3f46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b62ddf890 con 0x561b63017ee0 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14c3f46740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561b62f18d40 con 0x561b63024790 2024-09-10T21:18:54.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14c3f46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b630273e0 con 0x561b63018800 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14c3f46740 10 client.0 _ll_register_callbacks cb 0x561b62f7cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63024790 0x561b63022270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63024790 0x561b63022270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51768/0 (socket says 172.21.15.33:51768) 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14bbfff640 1 -- 172.21.15.33:0/2607254410 learned_addr learned my addr 172.21.15.33:0/2607254410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.276+0000 7f14baffd640 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63021d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f14bbfff640 1 -- 172.21.15.33:0/2607254410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 msgr2=0x561b63021d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63021d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f14bbfff640 1 -- 172.21.15.33:0/2607254410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 msgr2=0x561b630217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 0x561b630217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f14bbfff640 1 -- 172.21.15.33:0/2607254410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b630308a0 con 0x561b63024790 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63024790 0x561b63022270 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f14b400fe10 tx=0x7f14b4004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efed13982b923a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f149affd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:54.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.277+0000 7f149affd640 1 -- 172.21.15.33:0/2607254410 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4004760 con 0x561b63024790 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f149affd640 1 -- 172.21.15.33:0/2607254410 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f14b4004b20 con 0x561b63024790 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b0010d40 con 0x561b63024790 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f149affd640 1 -- 172.21.15.33:0/2607254410 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f14b4004d20 con 0x561b63024790 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 10 client.27587 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x561b63033230 con 0x561b63024790 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f149affd640 1 -- 172.21.15.33:0/2607254410 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b0010d40 con 0x561b63024790 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f149affd640 1 -- 172.21.15.33:0/2607254410 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f14b40104c0 con 0x561b63024790 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 10 client.27587 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 10 client.27587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 20 client.27587 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b63034740 0x561b63036b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 4 client.27587 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:54.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.278+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b62f90db0 con 0x561b63034740 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.279+0000 7f14bb7fe640 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b63034740 0x561b63036b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.280+0000 7f149affd640 10 client.27587 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.280+0000 7f14bb7fe640 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b63034740 0x561b63036b80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1deb0e33ee37175f server_cookie=58e14b447aa53111 in_seq=0 out_seq=0 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.280+0000 7f149affd640 1 -- 172.21.15.33:0/2607254410 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b62f90db0 con 0x561b63034740 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.280+0000 7f149affd640 10 client.27587 handle_command_reply: tid=0 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.280+0000 7f14c3f46740 1 client.27587 shutdown 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63024790 msgr2=0x561b63022270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63024790 0x561b63022270 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f14b400fe10 tx=0x7f14b4004ab0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 shutdown_connections 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b63034740 0x561b63036b80 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63024790 0x561b63022270 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63021d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2607254410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 0x561b630217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 >> 172.21.15.33:0/2607254410 conn(0x561b63015d10 msgr2=0x561b63014f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f149affd640 0 client.27587 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.281+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 shutdown_connections 2024-09-10T21:18:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.282+0000 7f14c3f46740 1 -- 172.21.15.33:0/2607254410 wait complete. 2024-09-10T21:18:54.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.282+0000 7f14c3f46740 20 client.27587 trim_cache size 0 max 16384 2024-09-10T21:18:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 Processor -- start 2024-09-10T21:18:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 -- start start 2024-09-10T21:18:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b63021cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63022230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b6301ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1ac380 con 0x561b63024790 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1ae0d0 con 0x561b63017ee0 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14c3f46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1aca80 con 0x561b63018800 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b6301ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14bb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b63021cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b6301ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38042/0 (socket says 172.21.15.33:38042) 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14bbfff640 1 -- 172.21.15.33:0/2053892107 learned_addr learned my addr 172.21.15.33:0/2053892107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.304+0000 7f14bb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b63021cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51780/0 (socket says 172.21.15.33:51780) 2024-09-10T21:18:54.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14baffd640 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63022230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14bbfff640 1 -- 172.21.15.33:0/2053892107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 msgr2=0x561b63022230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63022230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14bbfff640 1 -- 172.21.15.33:0/2053892107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 msgr2=0x561b63021cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b63021cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14bbfff640 1 -- 172.21.15.33:0/2053892107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b63011640 con 0x561b63024790 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14bbfff640 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b6301ea00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f14b400fde0 tx=0x7f14b4004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8252784bdf94586 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14a17fa640 1 -- 172.21.15.33:0/2053892107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b400fa10 con 0x561b63024790 2024-09-10T21:18:54.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14a17fa640 1 -- 172.21.15.33:0/2053892107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4003e00 con 0x561b63024790 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.305+0000 7f14a17fa640 1 -- 172.21.15.33:0/2053892107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4010430 con 0x561b63024790 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 -- 172.21.15.33:0/2053892107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 msgr2=0x561b6301ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b6301ea00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f14b400fde0 tx=0x7f14b4004420 comp rx=0 tx=0).stop 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 -- 172.21.15.33:0/2053892107 shutdown_connections 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63024790 0x561b6301ea00 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63018800 0x561b63022230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 --2- 172.21.15.33:0/2053892107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63017ee0 0x561b63021cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 -- 172.21.15.33:0/2053892107 >> 172.21.15.33:0/2053892107 conn(0x561b63015d10 msgr2=0x561b63023640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 -- 172.21.15.33:0/2053892107 shutdown_connections 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 -- 172.21.15.33:0/2053892107 wait complete. 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.306+0000 7f14c3f46740 1 -- start start 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 1 Processor -- start 2024-09-10T21:18:54.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 0x561b63029040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63018800 0x561b630295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63024790 0x561b6309a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1ad970 con 0x561b63017ee0 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1ad1e0 con 0x561b63018800 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1aa340 con 0x561b63024790 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14c3f46740 10 client.0 _ll_register_callbacks cb 0x7f14b4012050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14baffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63018800 0x561b630295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14baffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63018800 0x561b630295e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51788/0 (socket says 172.21.15.33:51788) 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14baffd640 1 -- 172.21.15.33:0/4239848711 learned_addr learned my addr 172.21.15.33:0/4239848711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14bb7fe640 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 0x561b63029040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.307+0000 7f14bbfff640 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63024790 0x561b6309a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14baffd640 1 -- 172.21.15.33:0/4239848711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63024790 msgr2=0x561b6309a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14baffd640 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63024790 0x561b6309a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14baffd640 1 -- 172.21.15.33:0/4239848711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 msgr2=0x561b63029040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14baffd640 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 0x561b63029040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14baffd640 1 -- 172.21.15.33:0/4239848711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b6309ae30 con 0x561b63018800 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14baffd640 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63018800 0x561b630295e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f14b00ac490 tx=0x7f14b00a8650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=705e2a1a2ac53505 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:54.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f149affd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f149affd640 1 -- 172.21.15.33:0/4239848711 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00abbf0 con 0x561b63018800 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f149affd640 1 -- 172.21.15.33:0/4239848711 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f14b00a80e0 con 0x561b63018800 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b4003e00 con 0x561b63018800 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f149affd640 1 -- 172.21.15.33:0/4239848711 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f14b00b5c70 con 0x561b63018800 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14c3f46740 10 client.27593 fetch_fsmap learned FSMap version 69 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x561b630a2240 con 0x561b63018800 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.308+0000 7f149affd640 1 -- 172.21.15.33:0/4239848711 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b00c7420 con 0x561b63018800 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f149affd640 1 -- 172.21.15.33:0/4239848711 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 69) ==== 8422+0+0 (secure 0 0 0) 0x7f14b00cfa30 con 0x561b63018800 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f14c3f46740 10 client.27593 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f14c3f46740 10 client.27593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f14c3f46740 20 client.27593 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f14c3f46740 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b630a2630 0x561b630a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f14c3f46740 4 client.27593 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f14b4003e00 con 0x561b630a2630 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.309+0000 7f14bb7fe640 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b630a2630 0x561b630a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.310+0000 7f14bb7fe640 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b630a2630 0x561b630a4a20 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af48a00c54e76a1a server_cookie=781a2f539dcb2c13 in_seq=0 out_seq=0 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.310+0000 7f149affd640 10 client.27593 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f149affd640 1 -- 172.21.15.33:0/4239848711 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f14b4003e00 con 0x561b630a2630 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f149affd640 10 client.27593 handle_command_reply: tid=0 2024-09-10T21:18:54.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 client.27593 shutdown 2024-09-10T21:18:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63018800 msgr2=0x561b630295e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63018800 0x561b630295e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f14b00ac490 tx=0x7f14b00a8650 comp rx=0 tx=0).stop 2024-09-10T21:18:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 shutdown_connections 2024-09-10T21:18:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561b630a2630 0x561b630a4a20 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f149affd640 0 client.27593 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b63024790 0x561b6309a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b63018800 0x561b630295e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 --2- 172.21.15.33:0/4239848711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b63017ee0 0x561b63029040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.314+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 >> 172.21.15.33:0/4239848711 conn(0x561b63015d10 msgr2=0x561b63022e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.315+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 shutdown_connections 2024-09-10T21:18:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.315+0000 7f14c3f46740 1 -- 172.21.15.33:0/4239848711 wait complete. 2024-09-10T21:18:54.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.315+0000 7f14c3f46740 20 client.27593 trim_cache size 0 max 16384 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "d1ed5978-a0e0-405b-a593-858fe0b8c98d", 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f14a40b5360 msgr2=0x7f14a40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f14a40b5360 0x7f14a40b7820 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f14bc077560 tx=0x7f14b4013b60 comp rx=0 tx=0).stop 2024-09-10T21:18:54.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14bc1087d0 msgr2=0x7f14bc0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14bc1087d0 0x7f14bc0759e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f14b000e9f0 tx=0x7f14b000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 shutdown_connections 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f14a40b5360 0x7f14a40b7820 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14bc109150 0x7f14bc075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f14bc1087d0 0x7f14bc0759e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 --2- 172.21.15.33:0/2792730176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14bc102dc0 0x7f14bc0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.316+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 >> 172.21.15.33:0/2792730176 conn(0x7f14bc0fe870 msgr2=0x7f14bc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.317+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 shutdown_connections 2024-09-10T21:18:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:54.317+0000 7f14c1b71640 1 -- 172.21.15.33:0/2792730176 wait complete. 2024-09-10T21:18:54.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:54 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:54.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:54 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:54.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:54 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:54 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:54.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:54 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:54.901 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:54 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:55.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 98468 47.68 MB/sec warmup 115 sec latency 295.743 ms 2024-09-10T21:18:55.302 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:18:55.671 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:18:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:18:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:55.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: pgmap v794: 97 pgs: 97 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 986 op/s 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.813 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:55 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:18:55.820 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: pgmap v794: 97 pgs: 97 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 986 op/s 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.821 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:55 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: pgmap v794: 97 pgs: 97 active+clean; 911 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 986 op/s 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:18:55.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:18:55.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:55.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:55 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:18:56.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 99201 47.62 MB/sec warmup 116 sec latency 220.858 ms 2024-09-10T21:18:56.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 -- 172.21.15.33:0/2996468035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 msgr2=0x7f2b1c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/2996468035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c104080 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009900 tx=0x7f2b0c02a410 comp rx=0 tx=0).stop 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 -- 172.21.15.33:0/2996468035 shutdown_connections 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/2996468035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c10d100 0x7f2b1c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/2996468035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c1045c0 0x7f2b1c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/2996468035 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c104080 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 -- 172.21.15.33:0/2996468035 >> 172.21.15.33:0/2996468035 conn(0x7f2b1c0fd700 msgr2=0x7f2b1c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 -- 172.21.15.33:0/2996468035 shutdown_connections 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.233+0000 7f2b21d9d640 1 -- 172.21.15.33:0/2996468035 wait complete. 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.234+0000 7f2b21d9d640 1 Processor -- start 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.234+0000 7f2b21d9d640 1 -- start start 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b21d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b21d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c1045c0 0x7f2b1c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b21d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c10d100 0x7f2b1c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b21d9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c112dc0 con 0x7f2b1c103ca0 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b21d9d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c112c40 con 0x7f2b1c10d100 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b21d9d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c112f40 con 0x7f2b1c1045c0 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c101950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38082/0 (socket says 172.21.15.33:38082) 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1645997123 learned_addr learned my addr 172.21.15.33:0/1645997123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b13fff640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c1045c0 0x7f2b1c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.235+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c10d100 0x7f2b1c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1645997123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c1045c0 msgr2=0x7f2b1c101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c1045c0 0x7f2b1c101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1645997123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c10d100 msgr2=0x7f2b1c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c10d100 0x7f2b1c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1645997123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c1a76d0 con 0x7f2b1c103ca0 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c10d100 0x7f2b1c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c101950 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005ab0 tx=0x7f2b0c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b197fa640 1 -- 172.21.15.33:0/1645997123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c03f070 con 0x7f2b1c103ca0 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c1a7960 con 0x7f2b1c103ca0 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b197fa640 1 -- 172.21.15.33:0/1645997123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0c031d60 con 0x7f2b1c103ca0 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b197fa640 1 -- 172.21.15.33:0/1645997123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c035030 con 0x7f2b1c103ca0 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.236+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c1a7e90 con 0x7f2b1c103ca0 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.239+0000 7f2b24172740 1 Processor -- start 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.239+0000 7f2b24172740 1 -- start start 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b24172740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd940860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b24172740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd94c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b24172740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd94f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b24172740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5563fd8b6e50 con 0x5563fd940480 2024-09-10T21:18:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b24172740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5563fd8a6280 con 0x5563fd94cd30 2024-09-10T21:18:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b24172740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563fd8aa090 con 0x5563fd940da0 2024-09-10T21:18:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd940860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b13fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd94f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd940860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38088/0 (socket says 172.21.15.33:38088) 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 -- 172.21.15.33:0/2788497997 learned_addr learned my addr 172.21.15.33:0/2788497997 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b197fa640 1 -- 172.21.15.33:0/1645997123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2b0c04c060 con 0x7f2b1c103ca0 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 -- 172.21.15.33:0/2788497997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 msgr2=0x5563fd94c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/2788497997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd94c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 -- 172.21.15.33:0/2788497997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 msgr2=0x5563fd94f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/2788497997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd94f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.240+0000 7f2b1bfff640 1 -- 172.21.15.33:0/2788497997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563fd604650 con 0x5563fd940480 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.241+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/2788497997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd940860 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2b140117a0 tx=0x7f2b14011c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41f483f930626ce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.241+0000 7f2b11ffb640 1 -- 172.21.15.33:0/2788497997 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14011220 con 0x5563fd940480 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.241+0000 7f2b11ffb640 1 -- 172.21.15.33:0/2788497997 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b1400fa60 con 0x5563fd940480 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.241+0000 7f2b11ffb640 1 -- 172.21.15.33:0/2788497997 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14005640 con 0x5563fd940480 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b197fa640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b080b5360 0x7f2b080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 -- 172.21.15.33:0/2788497997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 msgr2=0x5563fd940860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b080b5360 0x7f2b080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 --2- 172.21.15.33:0/2788497997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd940860 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2b140117a0 tx=0x7f2b14011c60 comp rx=0 tx=0).stop 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 -- 172.21.15.33:0/2788497997 shutdown_connections 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 --2- 172.21.15.33:0/2788497997 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd94f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 --2- 172.21.15.33:0/2788497997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd94c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 --2- 172.21.15.33:0/2788497997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd940860 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 -- 172.21.15.33:0/2788497997 >> 172.21.15.33:0/2788497997 conn(0x5563fd93e2b0 msgr2=0x5563fd93f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 -- 172.21.15.33:0/2788497997 shutdown_connections 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b24172740 1 -- 172.21.15.33:0/2788497997 wait complete. 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.242+0000 7f2b197fa640 1 -- 172.21.15.33:0/1645997123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2b0c105ff0 con 0x7f2b1c103ca0 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b080b5360 0x7f2b080b7820 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c031d30 tx=0x7f2b0c099c80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 -- start start 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 Processor -- start 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd94c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd9499f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd949f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5563fd707700 con 0x5563fd940480 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5563fd841110 con 0x5563fd94cd30 2024-09-10T21:18:56.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5563fd94f9b0 con 0x5563fd940da0 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b24172740 10 client.0 _ll_register_callbacks cb 0x5563fd85a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd94c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd94c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38094/0 (socket says 172.21.15.33:38094) 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.243+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1702429750 learned_addr learned my addr 172.21.15.33:0/1702429750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b13fff640 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd9499f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1702429750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 msgr2=0x5563fd9499f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd9499f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1702429750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 msgr2=0x5563fd949f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd949f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1702429750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563fd86ec40 con 0x5563fd940480 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b13fff640 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd9499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd94c5a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400a010 tx=0x7f2b14004e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3ce50952d07d0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2afaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.244+0000 7f2afaffd640 1 -- 172.21.15.33:0/1702429750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14028020 con 0x5563fd940480 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2afaffd640 1 -- 172.21.15.33:0/1702429750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2b140053a0 con 0x5563fd940480 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b1400fa60 con 0x5563fd940480 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2afaffd640 1 -- 172.21.15.33:0/1702429750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f2b1401d070 con 0x5563fd940480 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2b24172740 10 client.18183 fetch_fsmap learned FSMap version 70 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5563fd95b2c0 con 0x5563fd940480 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2afaffd640 1 -- 172.21.15.33:0/1702429750 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b1400fa60 con 0x5563fd940480 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2afaffd640 1 -- 172.21.15.33:0/1702429750 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 70) ==== 8418+0+0 (secure 0 0 0) 0x7f2b14024b50 con 0x5563fd940480 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2b24172740 10 client.18183 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.245+0000 7f2b24172740 10 client.18183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.246+0000 7f2b24172740 20 client.18183 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.246+0000 7f2b24172740 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd95eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.246+0000 7f2b24172740 4 client.18183 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.246+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563fd8b8da0 con 0x5563fd95c6c0 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.246+0000 7f2b13fff640 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd95eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.246+0000 7f2b13fff640 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd95eae0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a146fcb9f1daacbd server_cookie=1739e304d5e787ec in_seq=0 out_seq=0 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.246+0000 7f2afaffd640 10 client.18183 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.250+0000 7f2afaffd640 1 -- 172.21.15.33:0/1702429750 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5563fd8b8da0 con 0x5563fd95c6c0 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.250+0000 7f2afaffd640 10 client.18183 handle_command_reply: tid=0 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.250+0000 7f2b24172740 1 client.18183 shutdown 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.251+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 msgr2=0x5563fd94c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.251+0000 7f2b24172740 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd94c5a0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400a010 tx=0x7f2b14004e00 comp rx=0 tx=0).stop 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 shutdown_connections 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd95eae0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd949f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2afaffd640 0 client.18183 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:56.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd9499f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 --2- 172.21.15.33:0/1702429750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd94c5a0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 >> 172.21.15.33:0/1702429750 conn(0x5563fd93e2b0 msgr2=0x5563fd943e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 shutdown_connections 2024-09-10T21:18:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 1 -- 172.21.15.33:0/1702429750 wait complete. 2024-09-10T21:18:56.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.252+0000 7f2b24172740 20 client.18183 trim_cache size 0 max 16384 2024-09-10T21:18:56.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 Processor -- start 2024-09-10T21:18:56.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 -- start start 2024-09-10T21:18:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd940480 0x5563fd958c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940da0 0x5563fd93fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd94cd30 0x5563fd93dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b0600 con 0x5563fd940da0 2024-09-10T21:18:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b2320 con 0x5563fd940480 2024-09-10T21:18:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.275+0000 7f2b24172740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b0cd0 con 0x5563fd94cd30 2024-09-10T21:18:56.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd940480 0x5563fd958c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b13fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940da0 0x5563fd93fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd940480 0x5563fd958c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51818/0 (socket says 172.21.15.33:51818) 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b1bfff640 1 -- 172.21.15.33:0/396608788 learned_addr learned my addr 172.21.15.33:0/396608788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd94cd30 0x5563fd93dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b13fff640 1 -- 172.21.15.33:0/396608788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd94cd30 msgr2=0x5563fd93dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b13fff640 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd94cd30 0x5563fd93dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b13fff640 1 -- 172.21.15.33:0/396608788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd940480 msgr2=0x5563fd958c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b13fff640 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd940480 0x5563fd958c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b13fff640 1 -- 172.21.15.33:0/396608788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563fd939940 con 0x5563fd940da0 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b13fff640 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940da0 0x5563fd93fa10 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x5563fd95f330 tx=0x7f2b04014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6ffd4ac7006dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b117fa640 1 -- 172.21.15.33:0/396608788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b040140d0 con 0x5563fd940da0 2024-09-10T21:18:56.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b117fa640 1 -- 172.21.15.33:0/396608788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b04014270 con 0x5563fd940da0 2024-09-10T21:18:56.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.276+0000 7f2b117fa640 1 -- 172.21.15.33:0/396608788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04012450 con 0x5563fd940da0 2024-09-10T21:18:56.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 -- 172.21.15.33:0/396608788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940da0 msgr2=0x5563fd93fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940da0 0x5563fd93fa10 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x5563fd95f330 tx=0x7f2b04014840 comp rx=0 tx=0).stop 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 -- 172.21.15.33:0/396608788 shutdown_connections 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd94cd30 0x5563fd93dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940da0 0x5563fd93fa10 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 --2- 172.21.15.33:0/396608788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd940480 0x5563fd958c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 -- 172.21.15.33:0/396608788 >> 172.21.15.33:0/396608788 conn(0x5563fd93e2b0 msgr2=0x5563fd93f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 -- 172.21.15.33:0/396608788 shutdown_connections 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.277+0000 7f2b24172740 1 -- 172.21.15.33:0/396608788 wait complete. 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 -- start start 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 Processor -- start 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd950750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd950cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd9c1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b1bc0 con 0x5563fd940480 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1b1430 con 0x5563fd94cd30 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c1ae5c0 con 0x5563fd940da0 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.278+0000 7f2b24172740 10 client.0 _ll_register_callbacks cb 0x7f2b140116a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd950750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd950750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38106/0 (socket says 172.21.15.33:38106) 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 -- 172.21.15.33:0/3812439882 learned_addr learned my addr 172.21.15.33:0/3812439882 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd9c1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b13fff640 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd950cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 -- 172.21.15.33:0/3812439882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 msgr2=0x5563fd950cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd950cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 -- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 msgr2=0x5563fd9c1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd9c1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 -- 172.21.15.33:0/3812439882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563fd9c1d90 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd950750 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2b14006010 tx=0x7f2b1400ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661c5d5187f55f0b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2afaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2afaffd640 1 -- 172.21.15.33:0/3812439882 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b14028020 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.279+0000 7f2afaffd640 1 -- 172.21.15.33:0/3812439882 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2b14010970 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b04014270 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2afaffd640 1 -- 172.21.15.33:0/3812439882 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f2b14010b70 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 10 client.18189 fetch_fsmap learned FSMap version 70 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) -- 0x5563fd9c9220 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2afaffd640 1 -- 172.21.15.33:0/3812439882 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b04014270 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2afaffd640 1 -- 172.21.15.33:0/3812439882 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 70) ==== 8418+0+0 (secure 0 0 0) 0x7f2b14024480 con 0x5563fd940480 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 10 client.18189 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 10 client.18189 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 20 client.18189 populate_metadata read hostname 'smithi033' 2024-09-10T21:18:56.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd9cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:18:56.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.280+0000 7f2b24172740 4 client.18189 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:18:56.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.281+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563fd8b4850 con 0x5563fd95c6c0 2024-09-10T21:18:56.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.281+0000 7f2b13fff640 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd9cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:18:56.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.281+0000 7f2afaffd640 10 client.18189 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:56.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.281+0000 7f2b13fff640 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd9cb950 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5d7802d37051b1 server_cookie=bf146007a90aab81 in_seq=0 out_seq=0 2024-09-10T21:18:56.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2afaffd640 1 -- 172.21.15.33:0/3812439882 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5966 (crc 0 0 0) 0x5563fd8b4850 con 0x5563fd95c6c0 2024-09-10T21:18:56.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2afaffd640 10 client.18189 handle_command_reply: tid=0 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 client.18189 shutdown 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 msgr2=0x5563fd950750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd950750 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f2b14006010 tx=0x7f2b1400ff40 comp rx=0 tx=0).stop 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 shutdown_connections 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5563fd95c6c0 0x5563fd9cb950 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2afaffd640 0 client.18189 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5563fd94cd30 0x5563fd9c1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563fd940da0 0x5563fd950cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 --2- 172.21.15.33:0/3812439882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5563fd940480 0x5563fd950750 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 >> 172.21.15.33:0/3812439882 conn(0x5563fd93e2b0 msgr2=0x5563fd94b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 shutdown_connections 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.282+0000 7f2b24172740 1 -- 172.21.15.33:0/3812439882 wait complete. 2024-09-10T21:18:56.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b24172740 20 client.18189 trim_cache size 0 max 16384 2024-09-10T21:18:56.292 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:18:56.292 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:18:56.292 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:18:56.292 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:18:56.292 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "d1ed5978-a0e0-405b-a593-858fe0b8c98d", 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.295 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:18:56.297 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.298 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:18:56.302 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:56.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:18:56.306 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:18:56.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b080b5360 msgr2=0x7f2b080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b080b5360 0x7f2b080b7820 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c031d30 tx=0x7f2b0c099c80 comp rx=0 tx=0).stop 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 msgr2=0x7f2b1c101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c101950 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005ab0 tx=0x7f2b0c003c80 comp rx=0 tx=0).stop 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 shutdown_connections 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b080b5360 0x7f2b080b7820 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c10d100 0x7f2b1c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c1045c0 0x7f2b1c101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 --2- 172.21.15.33:0/1645997123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c103ca0 0x7f2b1c101950 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.283+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 >> 172.21.15.33:0/1645997123 conn(0x7f2b1c0fd700 msgr2=0x7f2b1c10f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.284+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 shutdown_connections 2024-09-10T21:18:56.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:18:56.284+0000 7f2b21d9d640 1 -- 172.21.15.33:0/1645997123 wait complete. 2024-09-10T21:18:57.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 99959 47.59 MB/sec warmup 117 sec latency 152.094 ms 2024-09-10T21:18:57.378 INFO:tasks.cephfs.filesystem:scrub status for tag:d1ed5978-a0e0-405b-a593-858fe0b8c98d - {'path': '/', 'tag': 'd1ed5978-a0e0-405b-a593-858fe0b8c98d', 'options': 'recursive,force'} 2024-09-10T21:18:57.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:57 smithi033 ceph-mon[30860]: pgmap v795: 97 pgs: 97 active+clean; 919 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 895 op/s 2024-09-10T21:18:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:57 smithi174 ceph-mon[35635]: pgmap v795: 97 pgs: 97 active+clean; 919 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 895 op/s 2024-09-10T21:18:58.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 100808 47.50 MB/sec warmup 118 sec latency 65.075 ms 2024-09-10T21:18:58.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:57 smithi104 ceph-mon[35325]: pgmap v795: 97 pgs: 97 active+clean; 919 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 895 op/s 2024-09-10T21:18:59.053 INFO:tasks.workunit.client.0.smithi033.stdout: 10 101944 47.59 MB/sec warmup 119 sec latency 85.885 ms 2024-09-10T21:18:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:18:59 smithi174 ceph-mon[35635]: pgmap v796: 97 pgs: 97 active+clean; 925 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 995 op/s 2024-09-10T21:18:59.928 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:18:59 smithi033 ceph-mon[30860]: pgmap v796: 97 pgs: 97 active+clean; 925 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 995 op/s 2024-09-10T21:19:00.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:18:59 smithi104 ceph-mon[35325]: pgmap v796: 97 pgs: 97 active+clean; 925 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 995 op/s 2024-09-10T21:19:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:00 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:00.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:00 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:01.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 103674 25.56 MB/sec execute 1 sec latency 228.076 ms 2024-09-10T21:19:01.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:00 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:01 smithi174 ceph-mon[35635]: pgmap v797: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 942 op/s 2024-09-10T21:19:01.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:01 smithi033 ceph-mon[30860]: pgmap v797: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 942 op/s 2024-09-10T21:19:02.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 104440 32.93 MB/sec execute 2 sec latency 111.768 ms 2024-09-10T21:19:02.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:01 smithi104 ceph-mon[35325]: pgmap v797: 97 pgs: 97 active+clean; 931 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 942 op/s 2024-09-10T21:19:03.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 105310 37.30 MB/sec execute 3 sec latency 74.694 ms 2024-09-10T21:19:03.864 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:03 smithi104 ceph-mon[35325]: pgmap v798: 97 pgs: 97 active+clean; 936 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:19:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:03 smithi174 ceph-mon[35635]: pgmap v798: 97 pgs: 97 active+clean; 936 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:19:03.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:03 smithi033 ceph-mon[30860]: pgmap v798: 97 pgs: 97 active+clean; 936 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:19:04.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 106116 39.63 MB/sec execute 4 sec latency 224.064 ms 2024-09-10T21:19:05.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 107037 39.70 MB/sec execute 5 sec latency 55.262 ms 2024-09-10T21:19:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:05 smithi174 ceph-mon[35635]: pgmap v799: 97 pgs: 97 active+clean; 945 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-10T21:19:05.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:05 smithi033 ceph-mon[30860]: pgmap v799: 97 pgs: 97 active+clean; 945 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-10T21:19:06.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 107846 41.89 MB/sec execute 6 sec latency 175.554 ms 2024-09-10T21:19:06.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:05 smithi104 ceph-mon[35325]: pgmap v799: 97 pgs: 97 active+clean; 945 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 1.01k op/s 2024-09-10T21:19:06.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:06.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:06.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:06.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:06.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:06.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:07.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 108402 40.83 MB/sec execute 7 sec latency 272.757 ms 2024-09-10T21:19:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:07.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:07 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:07 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:07 smithi174 ceph-mon[35635]: pgmap v800: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 855 op/s 2024-09-10T21:19:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:07 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:07 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:07.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:07 smithi033 ceph-mon[30860]: pgmap v800: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 855 op/s 2024-09-10T21:19:08.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 109092 40.19 MB/sec execute 8 sec latency 182.441 ms 2024-09-10T21:19:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:07 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:07 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:08.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:07 smithi104 ceph-mon[35325]: pgmap v800: 97 pgs: 97 active+clean; 947 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 855 op/s 2024-09-10T21:19:09.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 109584 38.09 MB/sec execute 9 sec latency 419.578 ms 2024-09-10T21:19:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:09 smithi174 ceph-mon[35635]: pgmap v801: 97 pgs: 97 active+clean; 957 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-10T21:19:09.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:09 smithi033 ceph-mon[30860]: pgmap v801: 97 pgs: 97 active+clean; 957 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-10T21:19:10.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 110041 36.63 MB/sec execute 10 sec latency 1007.832 ms 2024-09-10T21:19:10.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:09 smithi104 ceph-mon[35325]: pgmap v801: 97 pgs: 97 active+clean; 957 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 943 op/s 2024-09-10T21:19:11.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 110958 38.35 MB/sec execute 11 sec latency 66.366 ms 2024-09-10T21:19:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:11 smithi174 ceph-mon[35635]: pgmap v802: 97 pgs: 97 active+clean; 959 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 871 op/s 2024-09-10T21:19:11.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:11 smithi033 ceph-mon[30860]: pgmap v802: 97 pgs: 97 active+clean; 959 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 871 op/s 2024-09-10T21:19:12.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 111522 37.10 MB/sec execute 12 sec latency 292.769 ms 2024-09-10T21:19:12.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:11 smithi104 ceph-mon[35325]: pgmap v802: 97 pgs: 97 active+clean; 959 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 871 op/s 2024-09-10T21:19:13.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 112453 38.26 MB/sec execute 13 sec latency 93.862 ms 2024-09-10T21:19:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:13 smithi174 ceph-mon[35635]: pgmap v803: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-10T21:19:13.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:13 smithi033 ceph-mon[30860]: pgmap v803: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-10T21:19:14.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 113250 38.80 MB/sec execute 14 sec latency 190.895 ms 2024-09-10T21:19:14.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:13 smithi104 ceph-mon[35325]: pgmap v803: 97 pgs: 97 active+clean; 961 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-10T21:19:15.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 114185 39.75 MB/sec execute 15 sec latency 63.462 ms 2024-09-10T21:19:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:15 smithi174 ceph-mon[35635]: pgmap v804: 97 pgs: 97 active+clean; 965 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 901 op/s 2024-09-10T21:19:15.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:15 smithi033 ceph-mon[30860]: pgmap v804: 97 pgs: 97 active+clean; 965 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 901 op/s 2024-09-10T21:19:16.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 115081 40.19 MB/sec execute 16 sec latency 61.717 ms 2024-09-10T21:19:16.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:15 smithi104 ceph-mon[35325]: pgmap v804: 97 pgs: 97 active+clean; 965 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 901 op/s 2024-09-10T21:19:17.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 115704 39.90 MB/sec execute 17 sec latency 264.900 ms 2024-09-10T21:19:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:17 smithi174 ceph-mon[35635]: pgmap v805: 97 pgs: 97 active+clean; 966 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 770 op/s 2024-09-10T21:19:17.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:17 smithi033 ceph-mon[30860]: pgmap v805: 97 pgs: 97 active+clean; 966 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 770 op/s 2024-09-10T21:19:18.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 116450 39.31 MB/sec execute 18 sec latency 161.591 ms 2024-09-10T21:19:18.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:17 smithi104 ceph-mon[35325]: pgmap v805: 97 pgs: 97 active+clean; 966 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 770 op/s 2024-09-10T21:19:19.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 117480 40.35 MB/sec execute 19 sec latency 184.602 ms 2024-09-10T21:19:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:19 smithi174 ceph-mon[35635]: pgmap v806: 97 pgs: 97 active+clean; 981 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-10T21:19:19.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:19 smithi033 ceph-mon[30860]: pgmap v806: 97 pgs: 97 active+clean; 981 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-10T21:19:20.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 118620 41.98 MB/sec execute 20 sec latency 57.534 ms 2024-09-10T21:19:20.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:19 smithi104 ceph-mon[35325]: pgmap v806: 97 pgs: 97 active+clean; 981 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-10T21:19:21.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 119329 41.56 MB/sec execute 21 sec latency 59.190 ms 2024-09-10T21:19:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:21 smithi174 ceph-mon[35635]: pgmap v807: 97 pgs: 97 active+clean; 988 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-10T21:19:21.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:21 smithi033 ceph-mon[30860]: pgmap v807: 97 pgs: 97 active+clean; 988 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-10T21:19:22.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 120047 41.07 MB/sec execute 22 sec latency 257.773 ms 2024-09-10T21:19:22.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:21 smithi104 ceph-mon[35325]: pgmap v807: 97 pgs: 97 active+clean; 988 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 915 op/s 2024-09-10T21:19:23.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 120946 41.34 MB/sec execute 23 sec latency 106.602 ms 2024-09-10T21:19:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:23 smithi174 ceph-mon[35635]: pgmap v808: 97 pgs: 97 active+clean; 980 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 884 op/s 2024-09-10T21:19:23.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:23 smithi033 ceph-mon[30860]: pgmap v808: 97 pgs: 97 active+clean; 980 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 884 op/s 2024-09-10T21:19:24.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 121961 41.87 MB/sec execute 24 sec latency 83.145 ms 2024-09-10T21:19:24.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:23 smithi104 ceph-mon[35325]: pgmap v808: 97 pgs: 97 active+clean; 980 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 884 op/s 2024-09-10T21:19:25.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 122866 42.40 MB/sec execute 25 sec latency 182.675 ms 2024-09-10T21:19:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:25 smithi174 ceph-mon[35635]: pgmap v809: 97 pgs: 97 active+clean; 977 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 952 op/s 2024-09-10T21:19:25.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:25 smithi033 ceph-mon[30860]: pgmap v809: 97 pgs: 97 active+clean; 977 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 952 op/s 2024-09-10T21:19:26.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 123751 42.87 MB/sec execute 26 sec latency 184.539 ms 2024-09-10T21:19:26.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:25 smithi104 ceph-mon[35325]: pgmap v809: 97 pgs: 97 active+clean; 977 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 952 op/s 2024-09-10T21:19:27.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 124198 42.14 MB/sec execute 27 sec latency 281.726 ms 2024-09-10T21:19:27.379 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:19:27.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:27 smithi033 ceph-mon[30860]: pgmap v810: 97 pgs: 97 active+clean; 982 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 893 op/s 2024-09-10T21:19:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:27 smithi174 ceph-mon[35635]: pgmap v810: 97 pgs: 97 active+clean; 982 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 893 op/s 2024-09-10T21:19:28.054 INFO:tasks.workunit.client.0.smithi033.stdout: 10 124883 41.50 MB/sec execute 28 sec latency 108.624 ms 2024-09-10T21:19:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:27 smithi104 ceph-mon[35325]: pgmap v810: 97 pgs: 97 active+clean; 982 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 893 op/s 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.258+0000 7f54d7128640 1 -- 172.21.15.33:0/3856393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d010f450 msgr2=0x7f54d0111840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.258+0000 7f54d7128640 1 --2- 172.21.15.33:0/3856393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d010f450 0x7f54d0111840 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009ca0 tx=0x7f54c802a950 comp rx=0 tx=0).stop 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 -- 172.21.15.33:0/3856393701 shutdown_connections 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 --2- 172.21.15.33:0/3856393701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d010f450 0x7f54d0111840 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 --2- 172.21.15.33:0/3856393701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54d01021c0 0x7f54d010ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 --2- 172.21.15.33:0/3856393701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d01018a0 0x7f54d0101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 -- 172.21.15.33:0/3856393701 >> 172.21.15.33:0/3856393701 conn(0x7f54d00fd6f0 msgr2=0x7f54d00ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 -- 172.21.15.33:0/3856393701 shutdown_connections 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 -- 172.21.15.33:0/3856393701 wait complete. 2024-09-10T21:19:28.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 Processor -- start 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.259+0000 7f54d7128640 1 -- start start 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d7128640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d01018a0 0x7f54d019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d7128640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54d01021c0 0x7f54d019b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d7128640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d010f450 0x7f54d01a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d01018a0 0x7f54d019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d01018a0 0x7f54d019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35144/0 (socket says 172.21.15.33:35144) 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 -- 172.21.15.33:0/4053885986 learned_addr learned my addr 172.21.15.33:0/4053885986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:28.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0114f50 con 0x7f54d01018a0 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54d0114dd0 con 0x7f54d01021c0 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54cffff640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54d01021c0 0x7f54d019b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01150d0 con 0x7f54d010f450 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d56c5640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d010f450 0x7f54d01a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 -- 172.21.15.33:0/4053885986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d010f450 msgr2=0x7f54d01a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d010f450 0x7f54d01a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 -- 172.21.15.33:0/4053885986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54d01021c0 msgr2=0x7f54d019b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54d01021c0 0x7f54d019b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.260+0000 7f54d4ec4640 1 -- 172.21.15.33:0/4053885986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d01a14d0 con 0x7f54d01018a0 2024-09-10T21:19:28.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.261+0000 7f54d4ec4640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d01018a0 0x7f54d019b420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f54c400e9f0 tx=0x7f54c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.261+0000 7f54cdffb640 1 -- 172.21.15.33:0/4053885986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c400cd80 con 0x7f54d01018a0 2024-09-10T21:19:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.261+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d01a17c0 con 0x7f54d01018a0 2024-09-10T21:19:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.261+0000 7f54cdffb640 1 -- 172.21.15.33:0/4053885986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c40108e0 con 0x7f54d01018a0 2024-09-10T21:19:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.261+0000 7f54cdffb640 1 -- 172.21.15.33:0/4053885986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4018980 con 0x7f54d01018a0 2024-09-10T21:19:28.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.261+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d01a8050 con 0x7f54d01018a0 2024-09-10T21:19:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 Processor -- start 2024-09-10T21:19:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 -- start start 2024-09-10T21:19:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54cdffb640 1 -- 172.21.15.33:0/4053885986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f54c4010430 con 0x7f54d01018a0 2024-09-10T21:19:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a2dda0 0x5643a3a397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a39d30 0x5643a3a3c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5643a39a3e50 con 0x5643a3a2d480 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5643a3993280 con 0x5643a3a2dda0 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d94fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643a3997090 con 0x5643a3a39d30 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a2d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35150/0 (socket says 172.21.15.33:35150) 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.265+0000 7f54d56c5640 1 -- 172.21.15.33:0/3722778630 learned_addr learned my addr 172.21.15.33:0/3722778630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54cffff640 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a39d30 0x5643a3a3c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d4ec4640 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a2dda0 0x5643a3a397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d56c5640 1 -- 172.21.15.33:0/3722778630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a39d30 msgr2=0x5643a3a3c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d56c5640 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a39d30 0x5643a3a3c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d56c5640 1 -- 172.21.15.33:0/3722778630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a2dda0 msgr2=0x5643a3a397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d56c5640 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a2dda0 0x5643a3a397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d56c5640 1 -- 172.21.15.33:0/3722778630 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a36f1650 con 0x5643a3a2d480 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54cffff640 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a39d30 0x5643a3a3c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d4ec4640 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a2dda0 0x5643a3a397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54d56c5640 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a2d860 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f54c8035000 tx=0x7f54c8009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84af4eca9000b9ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.266+0000 7f54cdffb640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54c00b52e0 0x7f54c00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.267+0000 7f54baffd640 1 -- 172.21.15.33:0/3722778630 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8034040 con 0x5643a3a2d480 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.267+0000 7f54baffd640 1 -- 172.21.15.33:0/3722778630 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c8003e20 con 0x5643a3a2d480 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.267+0000 7f54cdffb640 1 -- 172.21.15.33:0/4053885986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f54c40e5fd0 con 0x7f54d01018a0 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.267+0000 7f54baffd640 1 -- 172.21.15.33:0/3722778630 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80042a0 con 0x5643a3a2d480 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.267+0000 7f54d4ec4640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54c00b52e0 0x7f54c00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 -- 172.21.15.33:0/3722778630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 msgr2=0x5643a3a2d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d4ec4640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54c00b52e0 0x7f54c00b77a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f54c40ac490 tx=0x7f54c4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a2d860 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f54c8035000 tx=0x7f54c8009a20 comp rx=0 tx=0).stop 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 -- 172.21.15.33:0/3722778630 shutdown_connections 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a39d30 0x5643a3a3c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a2dda0 0x5643a3a397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3722778630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a2d860 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 -- 172.21.15.33:0/3722778630 >> 172.21.15.33:0/3722778630 conn(0x5643a3a2b2b0 msgr2=0x5643a3a2c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 -- 172.21.15.33:0/3722778630 shutdown_connections 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 -- 172.21.15.33:0/3722778630 wait complete. 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 -- start start 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.268+0000 7f54d94fd740 1 Processor -- start 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2d480 0x5643a3a395e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2dda0 0x5643a3a36a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a36ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d94fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5643a37f4700 con 0x5643a3a2dda0 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d94fd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5643a392e110 con 0x5643a3a39d30 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d94fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643a3a3c950 con 0x5643a3a2d480 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d94fd740 10 client.0 _ll_register_callbacks cb 0x5643a3947990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2d480 0x5643a3a395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2d480 0x5643a3a395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55326/0 (socket says 172.21.15.33:55326) 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2dda0 0x5643a3a36a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d56c5640 1 -- 172.21.15.33:0/342551783 learned_addr learned my addr 172.21.15.33:0/342551783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:28.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54d56c5640 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a36ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54cffff640 1 -- 172.21.15.33:0/342551783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2d480 msgr2=0x5643a3a395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54cffff640 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2d480 0x5643a3a395e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54cffff640 1 -- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 msgr2=0x5643a3a36ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54cffff640 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a36ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.269+0000 7f54cffff640 1 -- 172.21.15.33:0/342551783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a395bc40 con 0x5643a3a2dda0 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54cffff640 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2dda0 0x5643a3a36a30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0126d0 tx=0x7f54bc012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6519f1127fddd6cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54affff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54affff640 1 -- 172.21.15.33:0/342551783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54bc004050 con 0x5643a3a2dda0 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54affff640 1 -- 172.21.15.33:0/342551783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f54bc0041f0 con 0x5643a3a2dda0 2024-09-10T21:19:28.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c8003e20 con 0x5643a3a2dda0 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54affff640 1 -- 172.21.15.33:0/342551783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f54bc0043f0 con 0x5643a3a2dda0 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54d94fd740 10 client.18201 fetch_fsmap learned FSMap version 71 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5643a3a48270 con 0x5643a3a2dda0 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.270+0000 7f54affff640 1 -- 172.21.15.33:0/342551783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c8003e20 con 0x5643a3a2dda0 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54affff640 1 -- 172.21.15.33:0/342551783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 71) ==== 8414+0+0 (secure 0 0 0) 0x7f54bc003bc0 con 0x5643a3a2dda0 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d94fd740 10 client.18201 fetch_fsmap finished waiting for FSMap version 71 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d94fd740 10 client.18201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d94fd740 20 client.18201 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d94fd740 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3a4bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d94fd740 4 client.18201 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643a39a5da0 con 0x5643a3a49820 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d56c5640 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3a4bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54affff640 10 client.18201 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:28.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.271+0000 7f54d56c5640 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3a4bc40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff2712d6bffb422 server_cookie=a1c2f35984e3d77d in_seq=0 out_seq=0 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.276+0000 7f54affff640 1 -- 172.21.15.33:0/342551783 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643a39a5da0 con 0x5643a3a49820 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.276+0000 7f54affff640 10 client.18201 handle_command_reply: tid=0 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.276+0000 7f54d94fd740 1 client.18201 shutdown 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.276+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2dda0 msgr2=0x5643a3a36a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.276+0000 7f54d94fd740 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2dda0 0x5643a3a36a30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0126d0 tx=0x7f54bc012b90 comp rx=0 tx=0).stop 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.276+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 shutdown_connections 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3a4bc40 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a36ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2dda0 0x5643a3a36a30 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54affff640 0 client.18201 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 1 --2- 172.21.15.33:0/342551783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2d480 0x5643a3a395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 >> 172.21.15.33:0/342551783 conn(0x5643a3a2b2b0 msgr2=0x5643a3a30e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 shutdown_connections 2024-09-10T21:19:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 1 -- 172.21.15.33:0/342551783 wait complete. 2024-09-10T21:19:28.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.277+0000 7f54d94fd740 20 client.18201 trim_cache size 0 max 16384 2024-09-10T21:19:28.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 Processor -- start 2024-09-10T21:19:28.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 -- start start 2024-09-10T21:19:28.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a45be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a3b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b0640 con 0x5643a3a2d480 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b2330 con 0x5643a3a39d30 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d94fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b0d10 con 0x5643a3a2dda0 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a45be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a45be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35174/0 (socket says 172.21.15.33:35174) 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d56c5640 1 -- 172.21.15.33:0/3926815968 learned_addr learned my addr 172.21.15.33:0/3926815968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54cffff640 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a3b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d56c5640 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d56c5640 1 -- 172.21.15.33:0/3926815968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 msgr2=0x5643a3a3b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d56c5640 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a3b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.305+0000 7f54d56c5640 1 -- 172.21.15.33:0/3926815968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 msgr2=0x5643a3a3b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.306+0000 7f54d56c5640 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.306+0000 7f54d56c5640 1 -- 172.21.15.33:0/3926815968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a3a317a0 con 0x5643a3a2d480 2024-09-10T21:19:28.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.306+0000 7f54d56c5640 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a45be0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f54c8000f80 tx=0x7f54c8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31609f66377ffb6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.306+0000 7f54ba7fc640 1 -- 172.21.15.33:0/3926815968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c802f870 con 0x5643a3a2d480 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.306+0000 7f54ba7fc640 1 -- 172.21.15.33:0/3926815968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c8003e20 con 0x5643a3a2d480 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.306+0000 7f54ba7fc640 1 -- 172.21.15.33:0/3926815968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c80047c0 con 0x5643a3a2d480 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.307+0000 7f54d94fd740 1 -- 172.21.15.33:0/3926815968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 msgr2=0x5643a3a45be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.307+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a45be0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f54c8000f80 tx=0x7f54c8005fd0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.308+0000 7f54d94fd740 1 -- 172.21.15.33:0/3926815968 shutdown_connections 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.308+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.308+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a3b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.308+0000 7f54d94fd740 1 --2- 172.21.15.33:0/3926815968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a45be0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.308+0000 7f54d94fd740 1 -- 172.21.15.33:0/3926815968 >> 172.21.15.33:0/3926815968 conn(0x5643a3a2b2b0 msgr2=0x5643a3a31ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:28.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.308+0000 7f54d94fd740 1 -- 172.21.15.33:0/3926815968 shutdown_connections 2024-09-10T21:19:28.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.308+0000 7f54d94fd740 1 -- 172.21.15.33:0/3926815968 wait complete. 2024-09-10T21:19:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.309+0000 7f54d94fd740 1 -- start start 2024-09-10T21:19:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.309+0000 7f54d94fd740 1 Processor -- start 2024-09-10T21:19:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a41c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a42190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d94fd740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d94fd740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b1bd0 con 0x5643a3a2d480 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d94fd740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01b1440 con 0x5643a3a39d30 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d94fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54d01ae600 con 0x5643a3a2dda0 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d94fd740 10 client.0 _ll_register_callbacks cb 0x7f54bc009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d56c5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54460/0 (socket says 172.21.15.33:54460) 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d56c5640 1 -- 172.21.15.33:0/2935956880 learned_addr learned my addr 172.21.15.33:0/2935956880 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54cffff640 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a42190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.310+0000 7f54d56c5640 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a41c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54d56c5640 1 -- 172.21.15.33:0/2935956880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 msgr2=0x5643a3a42190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54d56c5640 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a42190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54d56c5640 1 -- 172.21.15.33:0/2935956880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 msgr2=0x5643a3a41c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54d56c5640 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a41c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54d56c5640 1 -- 172.21.15.33:0/2935956880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a3a3d140 con 0x5643a3a39d30 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54d56c5640 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3cc00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54c8003a60 tx=0x7f54c8046040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a674063e7cbe817f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54affff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54affff640 1 -- 172.21.15.33:0/2935956880 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8034070 con 0x5643a3a39d30 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.311+0000 7f54affff640 1 -- 172.21.15.33:0/2935956880 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f54c8004d40 con 0x5643a3a39d30 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.312+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54c8003e20 con 0x5643a3a39d30 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.312+0000 7f54affff640 1 -- 172.21.15.33:0/2935956880 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f54c8049040 con 0x5643a3a39d30 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.312+0000 7f54d94fd740 10 client.27623 fetch_fsmap learned FSMap version 71 2024-09-10T21:19:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.312+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) -- 0x5643a3ab6f00 con 0x5643a3a39d30 2024-09-10T21:19:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.312+0000 7f54affff640 1 -- 172.21.15.33:0/2935956880 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54c8003e20 con 0x5643a3a39d30 2024-09-10T21:19:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.312+0000 7f54affff640 1 -- 172.21.15.33:0/2935956880 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 71) ==== 8414+0+0 (secure 0 0 0) 0x7f54c804f040 con 0x5643a3a39d30 2024-09-10T21:19:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.312+0000 7f54d94fd740 10 client.27623 fetch_fsmap finished waiting for FSMap version 71 2024-09-10T21:19:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.313+0000 7f54d94fd740 10 client.27623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:28.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.313+0000 7f54d94fd740 20 client.27623 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.313+0000 7f54d94fd740 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3ab90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.313+0000 7f54d94fd740 4 client.27623 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:19:28.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.314+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643a39a1850 con 0x5643a3a49820 2024-09-10T21:19:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.314+0000 7f54cffff640 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3ab90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.315+0000 7f54cffff640 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3ab90e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=921f63b9cd1d6332 server_cookie=5bd3f1ee00121d96 in_seq=0 out_seq=0 2024-09-10T21:19:28.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.315+0000 7f54affff640 10 client.27623 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.315+0000 7f54affff640 1 -- 172.21.15.33:0/2935956880 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5770 (crc 0 0 0) 0x5643a39a1850 con 0x5643a3a49820 2024-09-10T21:19:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54affff640 10 client.27623 handle_command_reply: tid=0 2024-09-10T21:19:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 client.27623 shutdown 2024-09-10T21:19:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 msgr2=0x5643a3a3cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3cc00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f54c8003a60 tx=0x7f54c8046040 comp rx=0 tx=0).stop 2024-09-10T21:19:28.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 shutdown_connections 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5643a3a49820 0x5643a3ab90e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5643a3a39d30 0x5643a3a3cc00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643a3a2dda0 0x5643a3a42190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 --2- 172.21.15.33:0/2935956880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5643a3a2d480 0x5643a3a41c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 >> 172.21.15.33:0/2935956880 conn(0x5643a3a2b2b0 msgr2=0x5643a3a29bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54affff640 0 client.27623 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 shutdown_connections 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.316+0000 7f54d94fd740 1 -- 172.21.15.33:0/2935956880 wait complete. 2024-09-10T21:19:28.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.317+0000 7f54d94fd740 20 client.27623 trim_cache size 0 max 16384 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.323 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:19:28.324 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.325 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:19:28.326 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.327 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.328 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.329 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.330 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:19:28.331 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.332 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.333 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.334 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:19:28.335 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:19:28.336 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:19:28.336 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:19:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.318+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54c00b52e0 msgr2=0x7f54c00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.318+0000 7f54d7128640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54c00b52e0 0x7f54c00b77a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f54c40ac490 tx=0x7f54c4000bc0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.318+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d01018a0 msgr2=0x7f54d019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.318+0000 7f54d7128640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d01018a0 0x7f54d019b420 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f54c400e9f0 tx=0x7f54c400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 shutdown_connections 2024-09-10T21:19:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54c00b52e0 0x7f54c00b77a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54d010f450 0x7f54d01a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54d01021c0 0x7f54d019b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 --2- 172.21.15.33:0/4053885986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54d01018a0 0x7f54d019b420 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 >> 172.21.15.33:0/4053885986 conn(0x7f54d00fd6f0 msgr2=0x7f54d01116f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 shutdown_connections 2024-09-10T21:19:28.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:28.319+0000 7f54d7128640 1 -- 172.21.15.33:0/4053885986 wait complete. 2024-09-10T21:19:29.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 125869 42.05 MB/sec execute 29 sec latency 80.788 ms 2024-09-10T21:19:29.516 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:19:29.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:19:29.800 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:29 smithi174 ceph-mon[35635]: pgmap v811: 97 pgs: 97 active+clean; 986 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:19:29.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:29 smithi033 ceph-mon[30860]: pgmap v811: 97 pgs: 97 active+clean; 986 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:19:30.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 126757 42.28 MB/sec execute 30 sec latency 164.611 ms 2024-09-10T21:19:30.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:29 smithi104 ceph-mon[35325]: pgmap v811: 97 pgs: 97 active+clean; 986 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:19:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 -- 172.21.15.33:0/1707885351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c103ca0 msgr2=0x7f092c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 --2- 172.21.15.33:0/1707885351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c103ca0 0x7f092c104080 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f091c009900 tx=0x7f091c02a410 comp rx=0 tx=0).stop 2024-09-10T21:19:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 -- 172.21.15.33:0/1707885351 shutdown_connections 2024-09-10T21:19:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 --2- 172.21.15.33:0/1707885351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092c10d100 0x7f092c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 --2- 172.21.15.33:0/1707885351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092c1045c0 0x7f092c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 --2- 172.21.15.33:0/1707885351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c103ca0 0x7f092c104080 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 -- 172.21.15.33:0/1707885351 >> 172.21.15.33:0/1707885351 conn(0x7f092c0fd700 msgr2=0x7f092c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 -- 172.21.15.33:0/1707885351 shutdown_connections 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.331+0000 7f0930a46640 1 -- 172.21.15.33:0/1707885351 wait complete. 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.332+0000 7f0930a46640 1 Processor -- start 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0930a46640 1 -- start start 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0930a46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092c103ca0 0x7f092c10bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0930a46640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c1045c0 0x7f092c10c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0930a46640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092c10d100 0x7f092c1a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0930a46640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f092c112dd0 con 0x7f092c1045c0 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0930a46640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f092c112c50 con 0x7f092c10d100 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0930a46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f092c112f50 con 0x7f092c103ca0 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0929d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c1045c0 0x7f092c10c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f092ad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092c10d100 0x7f092c1a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0929d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c1045c0 0x7f092c10c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35356/0 (socket says 172.21.15.33:35356) 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.333+0000 7f0929d9b640 1 -- 172.21.15.33:0/3442145435 learned_addr learned my addr 172.21.15.33:0/3442145435 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f092a59c640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092c103ca0 0x7f092c10bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f0929d9b640 1 -- 172.21.15.33:0/3442145435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092c103ca0 msgr2=0x7f092c10bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f0929d9b640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092c103ca0 0x7f092c10bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f0929d9b640 1 -- 172.21.15.33:0/3442145435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092c10d100 msgr2=0x7f092c1a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f0929d9b640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092c10d100 0x7f092c1a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f0929d9b640 1 -- 172.21.15.33:0/3442145435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f092c1a2770 con 0x7f092c1045c0 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f092ad9d640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092c10d100 0x7f092c1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f0929d9b640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c1045c0 0x7f092c10c230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f092000b3c0 tx=0x7f092000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.334+0000 7f091b7fe640 1 -- 172.21.15.33:0/3442145435 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920011830 con 0x7f092c1045c0 2024-09-10T21:19:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.335+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f092c1a2a60 con 0x7f092c1045c0 2024-09-10T21:19:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.335+0000 7f091b7fe640 1 -- 172.21.15.33:0/3442145435 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0920016070 con 0x7f092c1045c0 2024-09-10T21:19:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.335+0000 7f091b7fe640 1 -- 172.21.15.33:0/3442145435 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0920011830 con 0x7f092c1045c0 2024-09-10T21:19:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.335+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092c1ac3e0 con 0x7f092c1045c0 2024-09-10T21:19:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.337+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f092c104080 con 0x7f092c1045c0 2024-09-10T21:19:30.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.339+0000 7f091b7fe640 1 -- 172.21.15.33:0/3442145435 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0920010c00 con 0x7f092c1045c0 2024-09-10T21:19:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.340+0000 7f091b7fe640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09140b54e0 0x7f09140b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.340+0000 7f092a59c640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09140b54e0 0x7f09140b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.340+0000 7f091b7fe640 1 -- 172.21.15.33:0/3442145435 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f09200e4e10 con 0x7f092c1045c0 2024-09-10T21:19:30.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.341+0000 7f092a59c640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09140b54e0 0x7f09140b79a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f091c031440 tx=0x7f091c0095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:30.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.346+0000 7f091b7fe640 1 -- 172.21.15.33:0/3442145435 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f09200abcd0 con 0x7f092c1045c0 2024-09-10T21:19:30.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.619+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f092c1ac730 con 0x7f092c1045c0 2024-09-10T21:19:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.620+0000 7f091b7fe640 1 -- 172.21.15.33:0/3442145435 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+11995 (secure 0 0 0) 0x7f09200ab5d0 con 0x7f092c1045c0 2024-09-10T21:19:30.624 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:19:30.624 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 71 2024-09-10T21:19:30.625 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":71,"btime":"2024-09-10T21:18:59:561841+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":71,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:18:59.469840+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:19:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09140b54e0 msgr2=0x7f09140b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09140b54e0 0x7f09140b79a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f091c031440 tx=0x7f091c0095f0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c1045c0 msgr2=0x7f092c10c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c1045c0 0x7f092c10c230 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f092000b3c0 tx=0x7f092000b880 comp rx=0 tx=0).stop 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 shutdown_connections 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f09140b54e0 0x7f09140b79a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f092c10d100 0x7f092c1a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f092c1045c0 0x7f092c10c230 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 --2- 172.21.15.33:0/3442145435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f092c103ca0 0x7f092c10bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 >> 172.21.15.33:0/3442145435 conn(0x7f092c0fd700 msgr2=0x7f092c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:30.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 shutdown_connections 2024-09-10T21:19:30.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:30.623+0000 7f0930a46640 1 -- 172.21.15.33:0/3442145435 wait complete. 2024-09-10T21:19:30.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:30 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3442145435' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:19:30.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:30 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3442145435' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:19:31.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 127511 42.27 MB/sec execute 31 sec latency 62.302 ms 2024-09-10T21:19:31.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:30 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3442145435' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:19:31.320 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:19:31.740 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:31.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:31 smithi033 ceph-mon[30860]: pgmap v812: 97 pgs: 97 active+clean; 990 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 904 op/s 2024-09-10T21:19:31.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:31 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:19:32.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 127923 41.56 MB/sec execute 32 sec latency 297.331 ms 2024-09-10T21:19:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:31 smithi104 ceph-mon[35325]: pgmap v812: 97 pgs: 97 active+clean; 990 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 904 op/s 2024-09-10T21:19:32.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:31 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:19:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:31 smithi174 ceph-mon[35635]: pgmap v812: 97 pgs: 97 active+clean; 990 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 904 op/s 2024-09-10T21:19:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:31 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.299+0000 7f86be011640 1 -- 172.21.15.33:0/2699883851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8108020 msgr2=0x7f86b810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.299+0000 7f86be011640 1 --2- 172.21.15.33:0/2699883851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8108020 0x7f86b810e820 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009ca0 tx=0x7f86b002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 -- 172.21.15.33:0/2699883851 shutdown_connections 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 --2- 172.21.15.33:0/2699883851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8108020 0x7f86b810e820 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 --2- 172.21.15.33:0/2699883851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b81076a0 0x7f86b8107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 --2- 172.21.15.33:0/2699883851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b8101c90 0x7f86b8102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 -- 172.21.15.33:0/2699883851 >> 172.21.15.33:0/2699883851 conn(0x7f86b80fd700 msgr2=0x7f86b80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 -- 172.21.15.33:0/2699883851 shutdown_connections 2024-09-10T21:19:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 -- 172.21.15.33:0/2699883851 wait complete. 2024-09-10T21:19:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 Processor -- start 2024-09-10T21:19:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.300+0000 7f86be011640 1 -- start start 2024-09-10T21:19:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86be011640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8101c90 0x7f86b819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86be011640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b81076a0 0x7f86b819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86be011640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8108020 0x7f86b81a0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86be011640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8112de0 con 0x7f86b81076a0 2024-09-10T21:19:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86be011640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8112c60 con 0x7f86b8101c90 2024-09-10T21:19:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86be011640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86b8112f60 con 0x7f86b8108020 2024-09-10T21:19:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b81076a0 0x7f86b819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8101c90 0x7f86b819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b81076a0 0x7f86b819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35368/0 (socket says 172.21.15.33:35368) 2024-09-10T21:19:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 -- 172.21.15.33:0/170036698 learned_addr learned my addr 172.21.15.33:0/170036698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86b7fff640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8108020 0x7f86b81a0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 -- 172.21.15.33:0/170036698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8108020 msgr2=0x7f86b81a0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8108020 0x7f86b81a0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 -- 172.21.15.33:0/170036698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8101c90 msgr2=0x7f86b819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8101c90 0x7f86b819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.301+0000 7f86affff640 1 -- 172.21.15.33:0/170036698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b81a1500 con 0x7f86b81076a0 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.302+0000 7f86affff640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b81076a0 0x7f86b819b990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f86a000e9f0 tx=0x7f86a000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.302+0000 7f86b57fa640 1 -- 172.21.15.33:0/170036698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a000cd80 con 0x7f86b81076a0 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.302+0000 7f86b57fa640 1 -- 172.21.15.33:0/170036698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a00054d0 con 0x7f86b81076a0 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.302+0000 7f86b57fa640 1 -- 172.21.15.33:0/170036698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a0010690 con 0x7f86b81076a0 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.302+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b81a17f0 con 0x7f86b81076a0 2024-09-10T21:19:32.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.302+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b81a7eb0 con 0x7f86b81076a0 2024-09-10T21:19:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.305+0000 7f86b57fa640 1 -- 172.21.15.33:0/170036698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f86a0004e90 con 0x7f86b81076a0 2024-09-10T21:19:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.305+0000 7f86c03e6740 1 Processor -- start 2024-09-10T21:19:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.305+0000 7f86c03e6740 1 -- start start 2024-09-10T21:19:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x56454058c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56454058cdf0 0x5645405a1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645405a1590 0x5645405a3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86c03e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564540502ea0 con 0x5645405a1590 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86c03e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5645404f22d0 con 0x56454058c4d0 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86c03e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645404f60e0 con 0x56454058cdf0 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56454058cdf0 0x5645405a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56454058cdf0 0x5645405a1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46306/0 (socket says 172.21.15.33:46306) 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86b77fe640 1 -- 172.21.15.33:0/2845470696 learned_addr learned my addr 172.21.15.33:0/2845470696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86b7fff640 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x56454058c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.306+0000 7f86affff640 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645405a1590 0x5645405a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86b77fe640 1 -- 172.21.15.33:0/2845470696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 msgr2=0x56454058c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86b77fe640 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x56454058c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86b77fe640 1 -- 172.21.15.33:0/2845470696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645405a1590 msgr2=0x5645405a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86b77fe640 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645405a1590 0x5645405a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86b77fe640 1 -- 172.21.15.33:0/2845470696 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564540250650 con 0x56454058cdf0 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86affff640 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645405a1590 0x5645405a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86b57fa640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86a40b52e0 0x7f86a40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.307+0000 7f86b7fff640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86a40b52e0 0x7f86a40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.308+0000 7f86b57fa640 1 -- 172.21.15.33:0/170036698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86a00e5fe0 con 0x7f86b81076a0 2024-09-10T21:19:32.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.308+0000 7f86b7fff640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86a40b52e0 0x7f86a40b77a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f86b0006010 tx=0x7f86b00099a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.309+0000 7f86b77fe640 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56454058cdf0 0x5645405a1050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009d20 tx=0x7f86a8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0052475e3286263 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.310+0000 7f86adffb640 1 -- 172.21.15.33:0/2845470696 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a801e070 con 0x56454058cdf0 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.310+0000 7f86adffb640 1 -- 172.21.15.33:0/2845470696 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a8019ae0 con 0x56454058cdf0 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.310+0000 7f86adffb640 1 -- 172.21.15.33:0/2845470696 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a800e9e0 con 0x56454058cdf0 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 -- 172.21.15.33:0/2845470696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56454058cdf0 msgr2=0x5645405a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56454058cdf0 0x5645405a1050 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f86a8009d20 tx=0x7f86a8011810 comp rx=0 tx=0).stop 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 -- 172.21.15.33:0/2845470696 shutdown_connections 2024-09-10T21:19:32.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5645405a1590 0x5645405a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56454058cdf0 0x5645405a1050 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 --2- 172.21.15.33:0/2845470696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x56454058c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 -- 172.21.15.33:0/2845470696 >> 172.21.15.33:0/2845470696 conn(0x56454058a300 msgr2=0x56454058b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 -- 172.21.15.33:0/2845470696 shutdown_connections 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 -- 172.21.15.33:0/2845470696 wait complete. 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 -- start start 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.311+0000 7f86c03e6740 1 Processor -- start 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454059e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x56454059e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86c03e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564540353700 con 0x56454058cdf0 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86c03e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56454048d160 con 0x56454058c4d0 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86c03e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5645405a4240 con 0x5645405a1590 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86c03e6740 10 client.0 _ll_register_callbacks cb 0x5645404a69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51170/0 (socket says 172.21.15.33:51170) 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86b77fe640 1 -- 172.21.15.33:0/3679768234 learned_addr learned my addr 172.21.15.33:0/3679768234 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86affff640 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454059e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86b7fff640 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x56454059e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86affff640 1 -- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 msgr2=0x56454059e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86affff640 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x56454059e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86affff640 1 -- 172.21.15.33:0/3679768234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 msgr2=0x5645405a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86affff640 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86affff640 1 -- 172.21.15.33:0/3679768234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645404bac90 con 0x56454058cdf0 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f86affff640 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454059e2a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x56454058c480 tx=0x7f86a00e9840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2232a3abcc9ce18 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.312+0000 7f8696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f8696ffd640 1 -- 172.21.15.33:0/3679768234 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a00abf10 con 0x56454058cdf0 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a8019ae0 con 0x56454058cdf0 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f8696ffd640 1 -- 172.21.15.33:0/3679768234 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86a00b58a0 con 0x56454058cdf0 2024-09-10T21:19:32.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f8696ffd640 1 -- 172.21.15.33:0/3679768234 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f86a00ac2b0 con 0x56454058cdf0 2024-09-10T21:19:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f86c03e6740 10 client.18225 fetch_fsmap learned FSMap version 72 2024-09-10T21:19:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x5645405afbf0 con 0x56454058cdf0 2024-09-10T21:19:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f8696ffd640 1 -- 172.21.15.33:0/3679768234 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a8019ae0 con 0x56454058cdf0 2024-09-10T21:19:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f8696ffd640 1 -- 172.21.15.33:0/3679768234 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 72) ==== 8410+0+0 (secure 0 0 0) 0x7f86a00a80f0 con 0x56454058cdf0 2024-09-10T21:19:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f86c03e6740 10 client.18225 fetch_fsmap finished waiting for FSMap version 72 2024-09-10T21:19:32.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f86c03e6740 10 client.18225 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:19:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.313+0000 7f86c03e6740 20 client.18225 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.314+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x5645405b3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.314+0000 7f86c03e6740 4 client.18225 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.314+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564540504df0 con 0x5645405b1470 2024-09-10T21:19:32.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.314+0000 7f86b77fe640 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x5645405b3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.315+0000 7f8696ffd640 10 client.18225 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:19:32.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.315+0000 7f86b77fe640 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x5645405b3910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b610e177c4445ca2 server_cookie=91b1cb0d184eb9bf in_seq=0 out_seq=0 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.316+0000 7f8696ffd640 1 -- 172.21.15.33:0/3679768234 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564540504df0 con 0x5645405b1470 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.316+0000 7f8696ffd640 10 client.18225 handle_command_reply: tid=0 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.316+0000 7f86c03e6740 1 client.18225 shutdown 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 msgr2=0x56454059e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454059e2a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x56454058c480 tx=0x7f86a00e9840 comp rx=0 tx=0).stop 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 shutdown_connections 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x5645405b3910 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f8696ffd640 0 client.18225 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x56454059e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454059e2a0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3679768234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.317+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 >> 172.21.15.33:0/3679768234 conn(0x56454058a300 msgr2=0x56454058fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.318+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 shutdown_connections 2024-09-10T21:19:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.318+0000 7f86c03e6740 1 -- 172.21.15.33:0/3679768234 wait complete. 2024-09-10T21:19:32.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.318+0000 7f86c03e6740 20 client.18225 trim_cache size 0 max 16384 2024-09-10T21:19:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 Processor -- start 2024-09-10T21:19:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 -- start start 2024-09-10T21:19:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454058b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405b9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81ac1d0 con 0x56454058cdf0 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81ade90 con 0x56454058c4d0 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.339+0000 7f86c03e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81ac8a0 con 0x5645405a1590 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405ad4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51174/0 (socket says 172.21.15.33:51174) 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86b77fe640 1 -- 172.21.15.33:0/69142830 learned_addr learned my addr 172.21.15.33:0/69142830 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454058b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86b7fff640 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405b9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86affff640 1 -- 172.21.15.33:0/69142830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 msgr2=0x5645405b9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86affff640 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405b9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86affff640 1 -- 172.21.15.33:0/69142830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 msgr2=0x5645405ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86affff640 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86affff640 1 -- 172.21.15.33:0/69142830 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56454058a0f0 con 0x56454058cdf0 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.340+0000 7f86affff640 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454058b7b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f86a00ac730 tx=0x7f86a00b11b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d547cf2e65ca1a34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.341+0000 7f86ad7fa640 1 -- 172.21.15.33:0/69142830 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a00abf10 con 0x56454058cdf0 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.341+0000 7f86ad7fa640 1 -- 172.21.15.33:0/69142830 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a00cf040 con 0x56454058cdf0 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.341+0000 7f86ad7fa640 1 -- 172.21.15.33:0/69142830 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a00ac570 con 0x56454058cdf0 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.341+0000 7f86c03e6740 1 -- 172.21.15.33:0/69142830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 msgr2=0x56454058b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.341+0000 7f86c03e6740 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454058b7b0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f86a00ac730 tx=0x7f86a00b11b0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 -- 172.21.15.33:0/69142830 shutdown_connections 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405b9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x56454058b7b0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 --2- 172.21.15.33:0/69142830 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 -- 172.21.15.33:0/69142830 >> 172.21.15.33:0/69142830 conn(0x56454058a300 msgr2=0x56454058b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 -- 172.21.15.33:0/69142830 shutdown_connections 2024-09-10T21:19:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 -- 172.21.15.33:0/69142830 wait complete. 2024-09-10T21:19:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 -- start start 2024-09-10T21:19:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.342+0000 7f86c03e6740 1 Processor -- start 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x5645405a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86c03e6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405a6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86c03e6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81ad760 con 0x56454058cdf0 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86c03e6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81acfd0 con 0x56454058c4d0 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86c03e6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86b81aa190 con 0x5645405a1590 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86c03e6740 10 client.0 _ll_register_callbacks cb 0x7f86a000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51186/0 (socket says 172.21.15.33:51186) 2024-09-10T21:19:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86affff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x5645405a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 -- 172.21.15.33:0/3257211343 learned_addr learned my addr 172.21.15.33:0/3257211343 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b7fff640 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 -- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 msgr2=0x5645405a6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405a6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 -- 172.21.15.33:0/3257211343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 msgr2=0x5645405a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x5645405a9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 -- 172.21.15.33:0/3257211343 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645405a6d30 con 0x56454058c4d0 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b7fff640 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.343+0000 7f86b77fe640 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a9710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f86a8011d10 tx=0x7f86a8005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2621be4e4a241a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.344+0000 7f8696ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.344+0000 7f8696ffd640 1 -- 172.21.15.33:0/3257211343 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a8030070 con 0x56454058c4d0 2024-09-10T21:19:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.344+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a00cf040 con 0x56454058c4d0 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.344+0000 7f8696ffd640 1 -- 172.21.15.33:0/3257211343 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f86a8005240 con 0x56454058c4d0 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.344+0000 7f8696ffd640 1 -- 172.21.15.33:0/3257211343 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f86a801e040 con 0x56454058c4d0 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.344+0000 7f86c03e6740 10 client.27647 fetch_fsmap learned FSMap version 72 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.344+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56454061ef30 con 0x56454058c4d0 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f8696ffd640 1 -- 172.21.15.33:0/3257211343 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86a00cf040 con 0x56454058c4d0 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f8696ffd640 1 -- 172.21.15.33:0/3257211343 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 72) ==== 8410+0+0 (secure 0 0 0) 0x7f86a00051b0 con 0x56454058c4d0 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f86c03e6740 10 client.27647 fetch_fsmap finished waiting for FSMap version 72 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f86c03e6740 10 client.27647 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f86c03e6740 20 client.27647 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x564540621590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f86c03e6740 4 client.27647 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:19:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.345+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645405008a0 con 0x5645405b1470 2024-09-10T21:19:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.347+0000 7f86affff640 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x564540621590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.349+0000 7f8696ffd640 10 client.27647 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:19:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.349+0000 7f86affff640 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x564540621590 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f273dc715f4f9824 server_cookie=3b0620363241f319 in_seq=0 out_seq=0 2024-09-10T21:19:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.349+0000 7f8696ffd640 1 -- 172.21.15.33:0/3257211343 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645405008a0 con 0x5645405b1470 2024-09-10T21:19:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.349+0000 7f8696ffd640 10 client.27647 handle_command_reply: tid=0 2024-09-10T21:19:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.349+0000 7f86c03e6740 1 client.27647 shutdown 2024-09-10T21:19:32.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 msgr2=0x5645405a9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a9710 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f86a8011d10 tx=0x7f86a8005b40 comp rx=0 tx=0).stop 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 shutdown_connections 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5645405b1470 0x564540621590 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5645405a1590 0x5645405a6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56454058cdf0 0x5645405a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f8696ffd640 0 client.27647 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 --2- 172.21.15.33:0/3257211343 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56454058c4d0 0x5645405a9710 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 >> 172.21.15.33:0/3257211343 conn(0x56454058a300 msgr2=0x5645405a00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 shutdown_connections 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 1 -- 172.21.15.33:0/3257211343 wait complete. 2024-09-10T21:19:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.350+0000 7f86c03e6740 20 client.27647 trim_cache size 0 max 16384 2024-09-10T21:19:32.359 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:19:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.351+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86a40b52e0 msgr2=0x7f86a40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.351+0000 7f86be011640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86a40b52e0 0x7f86a40b77a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f86b0006010 tx=0x7f86b00099a0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.351+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b81076a0 msgr2=0x7f86b819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.351+0000 7f86be011640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b81076a0 0x7f86b819b990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f86a000e9f0 tx=0x7f86a000edf0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 shutdown_connections 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f86a40b52e0 0x7f86a40b77a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86b8108020 0x7f86b81a0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86b81076a0 0x7f86b819b990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 --2- 172.21.15.33:0/170036698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86b8101c90 0x7f86b819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 >> 172.21.15.33:0/170036698 conn(0x7f86b80fd700 msgr2=0x7f86b8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 shutdown_connections 2024-09-10T21:19:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:32.352+0000 7f86be011640 1 -- 172.21.15.33:0/170036698 wait complete. 2024-09-10T21:19:32.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:32 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:33.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 128720 41.72 MB/sec execute 33 sec latency 291.711 ms 2024-09-10T21:19:33.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:32 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:32 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:33 smithi033 ceph-mon[30860]: pgmap v813: 97 pgs: 97 active+clean; 995 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 879 op/s 2024-09-10T21:19:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:19:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:19:33.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:19:33.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:33 smithi104 ceph-mon[35325]: pgmap v813: 97 pgs: 97 active+clean; 995 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 879 op/s 2024-09-10T21:19:33.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:19:33.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:19:33.963 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:19:34.008 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:19:34.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 129615 41.89 MB/sec execute 34 sec latency 63.782 ms 2024-09-10T21:19:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:33 smithi174 ceph-mon[35635]: pgmap v813: 97 pgs: 97 active+clean; 995 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 879 op/s 2024-09-10T21:19:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:19:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:19:34.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:19:34.317 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:34.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 -- 172.21.15.33:0/7279481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384108020 msgr2=0x7fc38410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/7279481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384108020 0x7fc38410e820 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009ca0 tx=0x7fc37c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 -- 172.21.15.33:0/7279481 shutdown_connections 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/7279481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384108020 0x7fc38410e820 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/7279481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3841076a0 0x7fc384107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/7279481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc384101c90 0x7fc384102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 -- 172.21.15.33:0/7279481 >> 172.21.15.33:0/7279481 conn(0x7fc3840fd700 msgr2=0x7fc3840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.852+0000 7fc38a5b5640 1 -- 172.21.15.33:0/7279481 shutdown_connections 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.853+0000 7fc38a5b5640 1 -- 172.21.15.33:0/7279481 wait complete. 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.853+0000 7fc38a5b5640 1 Processor -- start 2024-09-10T21:19:34.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.853+0000 7fc38a5b5640 1 -- start start 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc38a5b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384101c90 0x7fc3841992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc38a5b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3841076a0 0x7fc384199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc38a5b5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc384108020 0x7fc38419ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc38a5b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc384111ca0 con 0x7fc384101c90 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc38a5b5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc384111b20 con 0x7fc384108020 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc38a5b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc384111e20 con 0x7fc3841076a0 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc383fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384101c90 0x7fc3841992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc383fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384101c90 0x7fc3841992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35418/0 (socket says 172.21.15.33:35418) 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc383fff640 1 -- 172.21.15.33:0/2268151383 learned_addr learned my addr 172.21.15.33:0/2268151383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.854+0000 7fc3837fe640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3841076a0 0x7fc384199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc383fff640 1 -- 172.21.15.33:0/2268151383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3841076a0 msgr2=0x7fc384199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc383fff640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3841076a0 0x7fc384199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc383fff640 1 -- 172.21.15.33:0/2268151383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc384108020 msgr2=0x7fc38419ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc383fff640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc384108020 0x7fc38419ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc383fff640 1 -- 172.21.15.33:0/2268151383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc38419f380 con 0x7fc384101c90 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc383fff640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384101c90 0x7fc3841992d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e9f0 tx=0x7fc37400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc3817fa640 1 -- 172.21.15.33:0/2268151383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37400cd80 con 0x7fc384101c90 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc38419f670 con 0x7fc384101c90 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc3817fa640 1 -- 172.21.15.33:0/2268151383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3740108e0 con 0x7fc384101c90 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.855+0000 7fc3817fa640 1 -- 172.21.15.33:0/2268151383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc374018980 con 0x7fc384101c90 2024-09-10T21:19:34.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.856+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3840fdf40 con 0x7fc384101c90 2024-09-10T21:19:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 Processor -- start 2024-09-10T21:19:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 -- start start 2024-09-10T21:19:34.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766fa0f0 0x5644766fa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766faa10 0x564476706460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476708d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564476670ea0 con 0x5644766fa0f0 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5644766602d0 con 0x5644766faa10 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc38c98a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5644766640e0 con 0x5644767069a0 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc3817fa640 1 -- 172.21.15.33:0/2268151383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc374010430 con 0x7fc384101c90 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476708d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc388b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766fa0f0 0x5644766fa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.859+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476708d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46364/0 (socket says 172.21.15.33:46364) 2024-09-10T21:19:34.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc3837fe640 1 -- 172.21.15.33:0/1121709138 learned_addr learned my addr 172.21.15.33:0/1121709138 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc383fff640 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766faa10 0x564476706460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc388b52640 1 -- 172.21.15.33:0/1121709138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 msgr2=0x564476708d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc388b52640 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476708d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc388b52640 1 -- 172.21.15.33:0/1121709138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766faa10 msgr2=0x564476706460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc388b52640 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766faa10 0x564476706460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc388b52640 1 -- 172.21.15.33:0/1121709138 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644763be650 con 0x5644766fa0f0 2024-09-10T21:19:34.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.860+0000 7fc388b52640 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766fa0f0 0x5644766fa4d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc37c02ade0 tx=0x7fc37c0093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd4b581246089cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.861+0000 7fc36a7fc640 1 -- 172.21.15.33:0/1121709138 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c031cf0 con 0x5644766fa0f0 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.861+0000 7fc36a7fc640 1 -- 172.21.15.33:0/1121709138 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc37c03f070 con 0x5644766fa0f0 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.861+0000 7fc36a7fc640 1 -- 172.21.15.33:0/1121709138 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c004040 con 0x5644766fa0f0 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.861+0000 7fc3817fa640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc36c0b5360 0x7fc36c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.861+0000 7fc3817fa640 1 -- 172.21.15.33:0/2268151383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc3740e5ee0 con 0x7fc384101c90 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.862+0000 7fc383fff640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc36c0b5360 0x7fc36c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.862+0000 7fc38c98a740 1 -- 172.21.15.33:0/1121709138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766fa0f0 msgr2=0x5644766fa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.862+0000 7fc38c98a740 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766fa0f0 0x5644766fa4d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc37c02ade0 tx=0x7fc37c0093c0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.862+0000 7fc383fff640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc36c0b5360 0x7fc36c0b7820 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc3740ac3a0 tx=0x7fc374000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 -- 172.21.15.33:0/1121709138 shutdown_connections 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476708d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766faa10 0x564476706460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 --2- 172.21.15.33:0/1121709138 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766fa0f0 0x5644766fa4d0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 -- 172.21.15.33:0/1121709138 >> 172.21.15.33:0/1121709138 conn(0x5644766f7f20 msgr2=0x5644766f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 -- 172.21.15.33:0/1121709138 shutdown_connections 2024-09-10T21:19:34.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 -- 172.21.15.33:0/1121709138 wait complete. 2024-09-10T21:19:34.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.863+0000 7fc38c98a740 1 -- start start 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.864+0000 7fc38c98a740 1 Processor -- start 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766fa0f0 0x5644767062f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x564476703740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644767069a0 0x564476703da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc388b52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766fa0f0 0x5644767062f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc38c98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5644764c1700 con 0x5644766faa10 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc38c98a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5644765fb160 con 0x5644767069a0 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc38c98a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564476711dd0 con 0x5644766fa0f0 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc388b52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766fa0f0 0x5644767062f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46372/0 (socket says 172.21.15.33:46372) 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc388b52640 1 -- 172.21.15.33:0/774784241 learned_addr learned my addr 172.21.15.33:0/774784241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:34.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc38c98a740 10 client.0 _ll_register_callbacks cb 0x5644766149e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.865+0000 7fc3837fe640 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x564476703740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.866+0000 7fc388b52640 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644767069a0 0x564476703da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.866+0000 7fc388b52640 1 -- 172.21.15.33:0/774784241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644767069a0 msgr2=0x564476703da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.866+0000 7fc388b52640 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644767069a0 0x564476703da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.866+0000 7fc388b52640 1 -- 172.21.15.33:0/774784241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 msgr2=0x564476703740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.866+0000 7fc388b52640 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x564476703740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.866+0000 7fc388b52640 1 -- 172.21.15.33:0/774784241 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564476628c90 con 0x5644766fa0f0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.867+0000 7fc388b52640 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766fa0f0 0x5644767062f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc37c003fb0 tx=0x7fc37c009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25031604c7295410 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.867+0000 7fc3637fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.867+0000 7fc3637fe640 1 -- 172.21.15.33:0/774784241 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c04f030 con 0x5644766fa0f0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.867+0000 7fc3637fe640 1 -- 172.21.15.33:0/774784241 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc37c031890 con 0x5644766fa0f0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.867+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc37c03f070 con 0x5644766fa0f0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.867+0000 7fc3637fe640 1 -- 172.21.15.33:0/774784241 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc37c005bd0 con 0x5644766fa0f0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.868+0000 7fc38c98a740 10 client.27565 fetch_fsmap learned FSMap version 72 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.868+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56447671d770 con 0x5644766fa0f0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.868+0000 7fc3637fe640 1 -- 172.21.15.33:0/774784241 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc37c03f070 con 0x5644766fa0f0 2024-09-10T21:19:34.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.868+0000 7fc3637fe640 1 -- 172.21.15.33:0/774784241 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 72) ==== 8410+0+0 (secure 0 0 0) 0x7fc37c02f8d0 con 0x5644766fa0f0 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.868+0000 7fc38c98a740 10 client.27565 fetch_fsmap finished waiting for FSMap version 72 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.868+0000 7fc38c98a740 10 client.27565 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.868+0000 7fc38c98a740 20 client.27565 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.869+0000 7fc38c98a740 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447671eac0 0x564476720eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.869+0000 7fc38c98a740 4 client.27565 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.869+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564476672df0 con 0x56447671eac0 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.869+0000 7fc3837fe640 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447671eac0 0x564476720eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.869+0000 7fc3837fe640 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447671eac0 0x564476720eb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f85828a6e64fd85 server_cookie=9bdaccba0404c72d in_seq=0 out_seq=0 2024-09-10T21:19:34.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.869+0000 7fc3637fe640 10 client.27565 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:19:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.871+0000 7fc3637fe640 1 -- 172.21.15.33:0/774784241 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564476672df0 con 0x56447671eac0 2024-09-10T21:19:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.871+0000 7fc3637fe640 10 client.27565 handle_command_reply: tid=0 2024-09-10T21:19:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.871+0000 7fc38c98a740 1 client.27565 shutdown 2024-09-10T21:19:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.871+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766fa0f0 msgr2=0x5644767062f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.871+0000 7fc38c98a740 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766fa0f0 0x5644767062f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc37c003fb0 tx=0x7fc37c009960 comp rx=0 tx=0).stop 2024-09-10T21:19:34.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 shutdown_connections 2024-09-10T21:19:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644767069a0 0x564476703da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x564476703740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447671eac0 0x564476720eb0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 --2- 172.21.15.33:0/774784241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766fa0f0 0x5644767062f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc3637fe640 0 client.27565 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:19:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 >> 172.21.15.33:0/774784241 conn(0x5644766f7f20 msgr2=0x5644766fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:34.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 shutdown_connections 2024-09-10T21:19:34.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 1 -- 172.21.15.33:0/774784241 wait complete. 2024-09-10T21:19:34.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.872+0000 7fc38c98a740 20 client.27565 trim_cache size 0 max 16384 2024-09-10T21:19:34.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 Processor -- start 2024-09-10T21:19:34.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 -- start start 2024-09-10T21:19:34.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x56447671b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766faa10 0x5644766f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644767069a0 0x564476705480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3841ac2f0 con 0x5644767069a0 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3841adfb0 con 0x5644766fa0f0 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc38c98a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3841ac9c0 con 0x5644766faa10 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc388b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644767069a0 0x564476705480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc388b52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644767069a0 0x564476705480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35428/0 (socket says 172.21.15.33:35428) 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc388b52640 1 -- 172.21.15.33:0/3213464581 learned_addr learned my addr 172.21.15.33:0/3213464581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.889+0000 7fc388b52640 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x56447671b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc388b52640 1 -- 172.21.15.33:0/3213464581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766faa10 msgr2=0x5644766f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc388b52640 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766faa10 0x5644766f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc388b52640 1 -- 172.21.15.33:0/3213464581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 msgr2=0x56447671b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc388b52640 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x56447671b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc388b52640 1 -- 172.21.15.33:0/3213464581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644766fa840 con 0x5644767069a0 2024-09-10T21:19:34.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc388b52640 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644767069a0 0x564476705480 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009620 tx=0x7fc37c003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dafd65d2f85dd91c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:34.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc369ffb640 1 -- 172.21.15.33:0/3213464581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c031e60 con 0x5644767069a0 2024-09-10T21:19:34.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc369ffb640 1 -- 172.21.15.33:0/3213464581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc37c03f040 con 0x5644767069a0 2024-09-10T21:19:34.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.890+0000 7fc369ffb640 1 -- 172.21.15.33:0/3213464581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c030aa0 con 0x5644767069a0 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 -- 172.21.15.33:0/3213464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644767069a0 msgr2=0x564476705480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644767069a0 0x564476705480 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009620 tx=0x7fc37c003ee0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 -- 172.21.15.33:0/3213464581 shutdown_connections 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644767069a0 0x564476705480 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644766faa10 0x5644766f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3213464581 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x56447671b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 -- 172.21.15.33:0/3213464581 >> 172.21.15.33:0/3213464581 conn(0x5644766f7f20 msgr2=0x564476704d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 -- 172.21.15.33:0/3213464581 shutdown_connections 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 -- 172.21.15.33:0/3213464581 wait complete. 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.891+0000 7fc38c98a740 1 -- start start 2024-09-10T21:19:34.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 1 Processor -- start 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x564476717070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x5644767175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476712020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc3841ad880 con 0x5644766faa10 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc3841ad0f0 con 0x5644766fa0f0 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3841aa2b0 con 0x5644767069a0 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc38c98a740 10 client.0 _ll_register_callbacks cb 0x7fc378005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x5644767175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc388b52640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x564476717070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc388b52640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x564476717070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51228/0 (socket says 172.21.15.33:51228) 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc388b52640 1 -- 172.21.15.33:0/3864601266 learned_addr learned my addr 172.21.15.33:0/3864601266 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc3837fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x5644767175b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35434/0 (socket says 172.21.15.33:35434) 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc388b52640 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476712020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc3837fe640 1 -- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 msgr2=0x564476712020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc3837fe640 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476712020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc3837fe640 1 -- 172.21.15.33:0/3864601266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 msgr2=0x564476717070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc3837fe640 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x564476717070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.892+0000 7fc3837fe640 1 -- 172.21.15.33:0/3864601266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564476712560 con 0x5644766faa10 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc3837fe640 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x5644767175b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x5644766f8b40 tx=0x7fc378004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba13f427ffec27f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc3637fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc3637fe640 1 -- 172.21.15.33:0/3864601266 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc378003e50 con 0x5644766faa10 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc3637fe640 1 -- 172.21.15.33:0/3864601266 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc378010430 con 0x5644766faa10 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc37c03f040 con 0x5644766faa10 2024-09-10T21:19:34.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc3637fe640 1 -- 172.21.15.33:0/3864601266 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7fc378010630 con 0x5644766faa10 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc38c98a740 10 client.18249 fetch_fsmap learned FSMap version 72 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.893+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x56447678bef0 con 0x5644766faa10 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc3637fe640 1 -- 172.21.15.33:0/3864601266 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc37c03f040 con 0x5644766faa10 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc3637fe640 1 -- 172.21.15.33:0/3864601266 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 72) ==== 8410+0+0 (secure 0 0 0) 0x7fc37802f8e0 con 0x5644766faa10 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc38c98a740 10 client.18249 fetch_fsmap finished waiting for FSMap version 72 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc38c98a740 10 client.18249 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc38c98a740 20 client.18249 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447678c760 0x56447678eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc38c98a740 4 client.18249 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56447666e8a0 con 0x56447678c760 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.894+0000 7fc388b52640 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447678c760 0x56447678eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc3637fe640 10 client.18249 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:19:34.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc388b52640 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447678c760 0x56447678eb80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e6ce9dd9f283ede7 server_cookie=bf411d719ae6aa53 in_seq=0 out_seq=0 2024-09-10T21:19:34.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc3637fe640 1 -- 172.21.15.33:0/3864601266 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56447666e8a0 con 0x56447678c760 2024-09-10T21:19:34.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc3637fe640 10 client.18249 handle_command_reply: tid=0 2024-09-10T21:19:34.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc38c98a740 1 client.18249 shutdown 2024-09-10T21:19:34.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 msgr2=0x5644767175b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x5644767175b0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x5644766f8b40 tx=0x7fc378004410 comp rx=0 tx=0).stop 2024-09-10T21:19:34.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 shutdown_connections 2024-09-10T21:19:34.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.895+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x56447678c760 0x56447678eb80 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5644767069a0 0x564476712020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc3637fe640 0 client.18249 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5644766faa10 0x5644767175b0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc38c98a740 1 --2- 172.21.15.33:0/3864601266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5644766fa0f0 0x564476717070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 >> 172.21.15.33:0/3864601266 conn(0x5644766f7f20 msgr2=0x564476705590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 shutdown_connections 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc38c98a740 1 -- 172.21.15.33:0/3864601266 wait complete. 2024-09-10T21:19:34.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.896+0000 7fc38c98a740 20 client.18249 trim_cache size 0 max 16384 2024-09-10T21:19:34.905 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc36c0b5360 msgr2=0x7fc36c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc36c0b5360 0x7fc36c0b7820 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc3740ac3a0 tx=0x7fc374000bc0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384101c90 msgr2=0x7fc3841992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384101c90 0x7fc3841992d0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e9f0 tx=0x7fc37400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 shutdown_connections 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc36c0b5360 0x7fc36c0b7820 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc384108020 0x7fc38419ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc3841076a0 0x7fc384199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 --2- 172.21.15.33:0/2268151383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc384101c90 0x7fc3841992d0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:34.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.897+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 >> 172.21.15.33:0/2268151383 conn(0x7fc3840fd700 msgr2=0x7fc384104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:34.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.898+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 shutdown_connections 2024-09-10T21:19:34.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:34.898+0000 7fc38a5b5640 1 -- 172.21.15.33:0/2268151383 wait complete. 2024-09-10T21:19:35.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 130440 41.88 MB/sec execute 35 sec latency 76.830 ms 2024-09-10T21:19:35.670 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:19:35.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:35 smithi033 ceph-mon[30860]: pgmap v814: 97 pgs: 97 active+clean; 1000 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 945 op/s 2024-09-10T21:19:35.983 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:36.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 131438 42.32 MB/sec execute 36 sec latency 150.141 ms 2024-09-10T21:19:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:35 smithi104 ceph-mon[35325]: pgmap v814: 97 pgs: 97 active+clean; 1000 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 945 op/s 2024-09-10T21:19:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:35 smithi174 ceph-mon[35635]: pgmap v814: 97 pgs: 97 active+clean; 1000 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 945 op/s 2024-09-10T21:19:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 -- 172.21.15.33:0/3174489785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444010d150 msgr2=0x7f444010f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 --2- 172.21.15.33:0/3174489785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444010d150 0x7f444010f580 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f4438009ca0 tx=0x7f443802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 -- 172.21.15.33:0/3174489785 shutdown_connections 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 --2- 172.21.15.33:0/3174489785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f444010d150 0x7f444010f580 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 --2- 172.21.15.33:0/3174489785 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44401045a0 0x7f444010cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 --2- 172.21.15.33:0/3174489785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4440103ba0 0x7f4440103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 -- 172.21.15.33:0/3174489785 >> 172.21.15.33:0/3174489785 conn(0x7f44400fd770 msgr2=0x7f44400ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 -- 172.21.15.33:0/3174489785 shutdown_connections 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.526+0000 7f444662b640 1 -- 172.21.15.33:0/3174489785 wait complete. 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.527+0000 7f444662b640 1 Processor -- start 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.527+0000 7f444662b640 1 -- start start 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.527+0000 7f444662b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4440103ba0 0x7f4440199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.527+0000 7f444662b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44401045a0 0x7f4440199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f444662b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444010d150 0x7f444019eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f444662b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4440112d90 con 0x7f44401045a0 2024-09-10T21:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f444662b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4440112c10 con 0x7f4440103ba0 2024-09-10T21:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f444662b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4440112f10 con 0x7f444010d150 2024-09-10T21:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f443ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4440103ba0 0x7f4440199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f4444bc8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444010d150 0x7f444019eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f443ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4440103ba0 0x7f4440199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51236/0 (socket says 172.21.15.33:51236) 2024-09-10T21:19:36.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f443ffff640 1 -- 172.21.15.33:0/270784044 learned_addr learned my addr 172.21.15.33:0/270784044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f443f7fe640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44401045a0 0x7f4440199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f4444bc8640 1 -- 172.21.15.33:0/270784044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4440103ba0 msgr2=0x7f4440199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f4444bc8640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4440103ba0 0x7f4440199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f4444bc8640 1 -- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44401045a0 msgr2=0x7f4440199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f4444bc8640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44401045a0 0x7f4440199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.528+0000 7f4444bc8640 1 -- 172.21.15.33:0/270784044 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444019f2f0 con 0x7f444010d150 2024-09-10T21:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.529+0000 7f4444bc8640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444010d150 0x7f444019eb20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f443802ade0 tx=0x7f4438031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.529+0000 7f443d7fa640 1 -- 172.21.15.33:0/270784044 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438003b70 con 0x7f444010d150 2024-09-10T21:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.529+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444019f580 con 0x7f444010d150 2024-09-10T21:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.529+0000 7f443d7fa640 1 -- 172.21.15.33:0/270784044 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4438031910 con 0x7f444010d150 2024-09-10T21:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.529+0000 7f443d7fa640 1 -- 172.21.15.33:0/270784044 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443802f950 con 0x7f444010d150 2024-09-10T21:19:36.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.529+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44401a5ca0 con 0x7f444010d150 2024-09-10T21:19:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f443d7fa640 1 -- 172.21.15.33:0/270784044 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f443802faf0 con 0x7f444010d150 2024-09-10T21:19:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 Processor -- start 2024-09-10T21:19:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 -- start start 2024-09-10T21:19:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07ab38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3e20 0x55bd07abf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd07a25ea0 con 0x55bd07ab3e20 2024-09-10T21:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bd07a152d0 con 0x55bd07abfdb0 2024-09-10T21:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.532+0000 7f4448a00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bd07a190e0 con 0x55bd07ab3500 2024-09-10T21:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07ab38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07ab38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46430/0 (socket says 172.21.15.33:46430) 2024-09-10T21:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 -- 172.21.15.33:0/4054182824 learned_addr learned my addr 172.21.15.33:0/4054182824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443ffff640 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3e20 0x55bd07abf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 -- 172.21.15.33:0/4054182824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 msgr2=0x55bd07ac21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 -- 172.21.15.33:0/4054182824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3e20 msgr2=0x55bd07abf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3e20 0x55bd07abf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 -- 172.21.15.33:0/4054182824 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd07773650 con 0x55bd07ab3500 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443d7fa640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f44280b52e0 0x7f44280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443f7fe640 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07ab38e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4434012780 tx=0x7f44340096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3755c394e1e08421 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:36.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.533+0000 7f443ffff640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f44280b52e0 0x7f44280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.534+0000 7f443d7fa640 1 -- 172.21.15.33:0/270784044 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4438106d20 con 0x7f444010d150 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.534+0000 7f44267fc640 1 -- 172.21.15.33:0/4054182824 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434012d00 con 0x55bd07ab3500 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.534+0000 7f44267fc640 1 -- 172.21.15.33:0/4054182824 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44340107d0 con 0x55bd07ab3500 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.534+0000 7f44267fc640 1 -- 172.21.15.33:0/4054182824 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4434003850 con 0x55bd07ab3500 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.534+0000 7f4448a00740 1 -- 172.21.15.33:0/4054182824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 msgr2=0x55bd07ab38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.534+0000 7f443ffff640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f44280b52e0 0x7f44280b77a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55bd07ab2210 tx=0x7f4430005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.534+0000 7f4448a00740 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07ab38e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4434012780 tx=0x7f44340096e0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 -- 172.21.15.33:0/4054182824 shutdown_connections 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3e20 0x55bd07abf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 --2- 172.21.15.33:0/4054182824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07ab38e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 -- 172.21.15.33:0/4054182824 >> 172.21.15.33:0/4054182824 conn(0x55bd07ab1330 msgr2=0x55bd07ab24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 -- 172.21.15.33:0/4054182824 shutdown_connections 2024-09-10T21:19:36.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 -- 172.21.15.33:0/4054182824 wait complete. 2024-09-10T21:19:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 -- start start 2024-09-10T21:19:36.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.535+0000 7f4448a00740 1 Processor -- start 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4448a00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07abf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4448a00740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3e20 0x55bd07abcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4448a00740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07abd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4448a00740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bd07876700 con 0x55bd07abfdb0 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4448a00740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bd079b0160 con 0x55bd07ab3e20 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4448a00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bd07ac2a30 con 0x55bd07ab3500 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4448a00740 10 client.0 _ll_register_callbacks cb 0x55bd079c99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f443ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07abf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f443ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07abf740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46436/0 (socket says 172.21.15.33:46436) 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f443ffff640 1 -- 172.21.15.33:0/2026512469 learned_addr learned my addr 172.21.15.33:0/2026512469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:36.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4444bc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07abd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f443f7fe640 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3e20 0x55bd07abcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4444bc8640 1 -- 172.21.15.33:0/2026512469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 msgr2=0x55bd07abf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4444bc8640 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07abf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4444bc8640 1 -- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3e20 msgr2=0x55bd07abcb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4444bc8640 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3e20 0x55bd07abcb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f4444bc8640 1 -- 172.21.15.33:0/2026512469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd079ddc90 con 0x55bd07abfdb0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.536+0000 7f443ffff640 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07abf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f4444bc8640 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07abd260 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f44380ca710 tx=0x7f44380ccdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e281b147d9c575 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f441f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f441f7fe640 1 -- 172.21.15.33:0/2026512469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44380cdba0 con 0x55bd07abfdb0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f441f7fe640 1 -- 172.21.15.33:0/2026512469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f44380cd700 con 0x55bd07abfdb0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44340107d0 con 0x55bd07abfdb0 2024-09-10T21:19:36.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f441f7fe640 1 -- 172.21.15.33:0/2026512469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f4438003d10 con 0x55bd07abfdb0 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f4448a00740 10 client.18261 fetch_fsmap learned FSMap version 72 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55bd07aac870 con 0x55bd07abfdb0 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f441f7fe640 1 -- 172.21.15.33:0/2026512469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44340107d0 con 0x55bd07abfdb0 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.537+0000 7f441f7fe640 1 -- 172.21.15.33:0/2026512469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 72) ==== 8410+0+0 (secure 0 0 0) 0x7f44380d65a0 con 0x55bd07abfdb0 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f4448a00740 10 client.18261 fetch_fsmap finished waiting for FSMap version 72 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f4448a00740 10 client.18261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f4448a00740 20 client.18261 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f4448a00740 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07aaddd0 0x55bd07ab0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f4448a00740 4 client.18261 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd07a27df0 con 0x55bd07aaddd0 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f443ffff640 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07aaddd0 0x55bd07ab0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f443ffff640 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07aaddd0 0x55bd07ab0230 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14163e20b0392ecc server_cookie=59d8142d8aca8068 in_seq=0 out_seq=0 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.538+0000 7f441f7fe640 10 client.18261 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:19:36.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.539+0000 7f441f7fe640 1 -- 172.21.15.33:0/2026512469 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bd07a27df0 con 0x55bd07aaddd0 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.539+0000 7f441f7fe640 10 client.18261 handle_command_reply: tid=0 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.539+0000 7f4448a00740 1 client.18261 shutdown 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 msgr2=0x55bd07abd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07abd260 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f44380ca710 tx=0x7f44380ccdf0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 shutdown_connections 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07aaddd0 0x55bd07ab0230 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07abd260 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3e20 0x55bd07abcb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 --2- 172.21.15.33:0/2026512469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3500 0x55bd07abf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f441f7fe640 0 client.18261 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 >> 172.21.15.33:0/2026512469 conn(0x55bd07ab1330 msgr2=0x55bd07ab6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 shutdown_connections 2024-09-10T21:19:36.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 1 -- 172.21.15.33:0/2026512469 wait complete. 2024-09-10T21:19:36.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.540+0000 7f4448a00740 20 client.18261 trim_cache size 0 max 16384 2024-09-10T21:19:36.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 Processor -- start 2024-09-10T21:19:36.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 -- start start 2024-09-10T21:19:36.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3500 0x55bd07acbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07aa80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44401ac160 con 0x55bd07ab3500 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44401ade20 con 0x55bd07abfdb0 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4448a00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44401ac830 con 0x55bd07ab3e20 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3500 0x55bd07acbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3500 0x55bd07acbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35488/0 (socket says 172.21.15.33:35488) 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 -- 172.21.15.33:0/1057684617 learned_addr learned my addr 172.21.15.33:0/1057684617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443f7fe640 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07aa80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f4444bc8640 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 -- 172.21.15.33:0/1057684617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 msgr2=0x55bd07aa80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07aa80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 -- 172.21.15.33:0/1057684617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 msgr2=0x55bd07ac11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.555+0000 7f443ffff640 1 -- 172.21.15.33:0/1057684617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd07abcfc0 con 0x55bd07ab3500 2024-09-10T21:19:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.556+0000 7f443ffff640 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3500 0x55bd07acbcd0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x55bd07ab0a70 tx=0x7f443001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18db35d75beb31f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.556+0000 7f4425ffb640 1 -- 172.21.15.33:0/1057684617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f443002c580 con 0x55bd07ab3500 2024-09-10T21:19:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.556+0000 7f4425ffb640 1 -- 172.21.15.33:0/1057684617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4430038070 con 0x55bd07ab3500 2024-09-10T21:19:36.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.556+0000 7f4425ffb640 1 -- 172.21.15.33:0/1057684617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44300339d0 con 0x55bd07ab3500 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 -- 172.21.15.33:0/1057684617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3500 msgr2=0x55bd07acbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3500 0x55bd07acbcd0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x55bd07ab0a70 tx=0x7f443001b000 comp rx=0 tx=0).stop 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 -- 172.21.15.33:0/1057684617 shutdown_connections 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07abfdb0 0x55bd07ac11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07aa80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 --2- 172.21.15.33:0/1057684617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07ab3500 0x55bd07acbcd0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 -- 172.21.15.33:0/1057684617 >> 172.21.15.33:0/1057684617 conn(0x55bd07ab1330 msgr2=0x55bd07ab7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 -- 172.21.15.33:0/1057684617 shutdown_connections 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 -- 172.21.15.33:0/1057684617 wait complete. 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 -- start start 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.557+0000 7f4448a00740 1 Processor -- start 2024-09-10T21:19:36.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4448a00740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3500 0x55bd07ad3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4448a00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07acf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4448a00740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07acf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4448a00740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f44401ad6f0 con 0x55bd07abfdb0 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4448a00740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f44401acf60 con 0x55bd07ab3500 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4448a00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44401aa140 con 0x55bd07ab3e20 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4448a00740 10 client.0 _ll_register_callbacks cb 0x7f44380c9340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07acf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f443ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3500 0x55bd07ad3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07acf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35500/0 (socket says 172.21.15.33:35500) 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 -- 172.21.15.33:0/2642566269 learned_addr learned my addr 172.21.15.33:0/2642566269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f443f7fe640 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07acf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 -- 172.21.15.33:0/2642566269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 msgr2=0x55bd07acf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07acf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 -- 172.21.15.33:0/2642566269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3500 msgr2=0x55bd07ad3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3500 0x55bd07ad3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 -- 172.21.15.33:0/2642566269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd07b37630 con 0x55bd07abfdb0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f4444bc8640 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07acf820 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f44380c9a70 tx=0x7f44380ccf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=563ab907f9a9f35c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.558+0000 7f441f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f441f7fe640 1 -- 172.21.15.33:0/2642566269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44380d6a30 con 0x55bd07abfdb0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f441f7fe640 1 -- 172.21.15.33:0/2642566269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f44380f4370 con 0x55bd07abfdb0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4430038070 con 0x55bd07abfdb0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f441f7fe640 1 -- 172.21.15.33:0/2642566269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f44380c9af0 con 0x55bd07abfdb0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 10 client.18267 fetch_fsmap learned FSMap version 72 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55bd07b387a0 con 0x55bd07abfdb0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f441f7fe640 1 -- 172.21.15.33:0/2642566269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4430038070 con 0x55bd07abfdb0 2024-09-10T21:19:36.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f441f7fe640 1 -- 172.21.15.33:0/2642566269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 72) ==== 8410+0+0 (secure 0 0 0) 0x7f44380cc700 con 0x55bd07abfdb0 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 10 client.18267 fetch_fsmap finished waiting for FSMap version 72 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 10 client.18267 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 20 client.18267 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07b38e40 0x55bd07b3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 4 client.18267 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bd07a238a0 con 0x55bd07b38e40 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.559+0000 7f443ffff640 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07b38e40 0x55bd07b3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.560+0000 7f443ffff640 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07b38e40 0x55bd07b3b260 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f9d49465ff80d06 server_cookie=d79081a0421e0780 in_seq=0 out_seq=0 2024-09-10T21:19:36.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.560+0000 7f441f7fe640 10 client.18267 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:19:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.562+0000 7f441f7fe640 1 -- 172.21.15.33:0/2642566269 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bd07a238a0 con 0x55bd07b38e40 2024-09-10T21:19:36.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.562+0000 7f441f7fe640 10 client.18267 handle_command_reply: tid=0 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.562+0000 7f4448a00740 1 client.18267 shutdown 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 msgr2=0x55bd07acf820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07acf820 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f44380c9a70 tx=0x7f44380ccf20 comp rx=0 tx=0).stop 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 shutdown_connections 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55bd07b38e40 0x55bd07b3b260 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bd07abfdb0 0x55bd07acf820 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bd07ab3e20 0x55bd07acf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 --2- 172.21.15.33:0/2642566269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bd07ab3500 0x55bd07ad3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 >> 172.21.15.33:0/2642566269 conn(0x55bd07ab1330 msgr2=0x55bd07ab7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f441f7fe640 0 client.18267 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 shutdown_connections 2024-09-10T21:19:36.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 1 -- 172.21.15.33:0/2642566269 wait complete. 2024-09-10T21:19:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.563+0000 7f4448a00740 20 client.18267 trim_cache size 0 max 16384 2024-09-10T21:19:36.570 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:19:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f44280b52e0 msgr2=0x7f44280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f44280b52e0 0x7f44280b77a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55bd07ab2210 tx=0x7f4430005f90 comp rx=0 tx=0).stop 2024-09-10T21:19:36.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444010d150 msgr2=0x7f444019eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444010d150 0x7f444019eb20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f443802ade0 tx=0x7f4438031660 comp rx=0 tx=0).stop 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 shutdown_connections 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f44280b52e0 0x7f44280b77a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f444010d150 0x7f444019eb20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44401045a0 0x7f4440199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 --2- 172.21.15.33:0/270784044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4440103ba0 0x7f4440199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 >> 172.21.15.33:0/270784044 conn(0x7f44400fd770 msgr2=0x7f444010f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 shutdown_connections 2024-09-10T21:19:36.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:36.564+0000 7f444662b640 1 -- 172.21.15.33:0/270784044 wait complete. 2024-09-10T21:19:37.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 131903 41.75 MB/sec execute 37 sec latency 239.383 ms 2024-09-10T21:19:37.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:19:37.731 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:37.983 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:37 smithi033 ceph-mon[30860]: pgmap v815: 97 pgs: 97 active+clean; 1002 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 827 op/s 2024-09-10T21:19:38.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 132548 41.58 MB/sec execute 38 sec latency 192.324 ms 2024-09-10T21:19:38.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:37 smithi104 ceph-mon[35325]: pgmap v815: 97 pgs: 97 active+clean; 1002 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 827 op/s 2024-09-10T21:19:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:37 smithi174 ceph-mon[35635]: pgmap v815: 97 pgs: 97 active+clean; 1002 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 827 op/s 2024-09-10T21:19:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 -- 172.21.15.33:0/365862456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 msgr2=0x7f9e8010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 --2- 172.21.15.33:0/365862456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 0x7f9e8010e820 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009ca0 tx=0x7f9e7802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 -- 172.21.15.33:0/365862456 shutdown_connections 2024-09-10T21:19:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 --2- 172.21.15.33:0/365862456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 0x7f9e8010e820 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 --2- 172.21.15.33:0/365862456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e801076a0 0x7f9e80107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 --2- 172.21.15.33:0/365862456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 0x7f9e80102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 -- 172.21.15.33:0/365862456 >> 172.21.15.33:0/365862456 conn(0x7f9e800fd700 msgr2=0x7f9e800ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 -- 172.21.15.33:0/365862456 shutdown_connections 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.214+0000 7f9e884a7640 1 -- 172.21.15.33:0/365862456 wait complete. 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 Processor -- start 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 -- start start 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 0x7f9e80199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e801076a0 0x7f9e80199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 0x7f9e8019eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80111cf0 con 0x7f9e80108020 2024-09-10T21:19:38.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80111b70 con 0x7f9e80101c90 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e884a7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e80111e70 con 0x7f9e801076a0 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e86243640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 0x7f9e80199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e86a44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 0x7f9e8019eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e86243640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 0x7f9e80199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51290/0 (socket says 172.21.15.33:51290) 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e86243640 1 -- 172.21.15.33:0/2982624902 learned_addr learned my addr 172.21.15.33:0/2982624902 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.215+0000 7f9e85a42640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e801076a0 0x7f9e80199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e86243640 1 -- 172.21.15.33:0/2982624902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e801076a0 msgr2=0x7f9e80199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e86243640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e801076a0 0x7f9e80199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e86243640 1 -- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 msgr2=0x7f9e8019eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e86243640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 0x7f9e8019eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e86243640 1 -- 172.21.15.33:0/2982624902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8019f2e0 con 0x7f9e80101c90 2024-09-10T21:19:38.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e86243640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 0x7f9e80199230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00ea60 tx=0x7f9e7c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e777fe640 1 -- 172.21.15.33:0/2982624902 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00cde0 con 0x7f9e80101c90 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8019f5d0 con 0x7f9e80101c90 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e777fe640 1 -- 172.21.15.33:0/2982624902 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c0035f0 con 0x7f9e80101c90 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e777fe640 1 -- 172.21.15.33:0/2982624902 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c010690 con 0x7f9e80101c90 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.216+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e801a5e60 con 0x7f9e80101c90 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 Processor -- start 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 -- start start 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451cdd80 0x557b451d0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b45137ea0 con 0x557b451c14d0 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557b451272d0 con 0x557b451c1df0 2024-09-10T21:19:38.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b4512b0e0 con 0x557b451cdd80 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e777fe640 1 -- 172.21.15.33:0/2982624902 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9e7c004000 con 0x7f9e80101c90 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.220+0000 7f9e86a44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35540/0 (socket says 172.21.15.33:35540) 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 -- 172.21.15.33:0/4267380314 learned_addr learned my addr 172.21.15.33:0/4267380314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e85a42640 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451cdd80 0x557b451d0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86243640 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 -- 172.21.15.33:0/4267380314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451cdd80 msgr2=0x557b451d0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451cdd80 0x557b451d0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 -- 172.21.15.33:0/4267380314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 msgr2=0x557b451cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 -- 172.21.15.33:0/4267380314 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b44e85650 con 0x557b451c14d0 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86243640 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:38.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e86a44640 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c18b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009ca0 tx=0x7f9e7802f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ce4f4d3e37f000 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e777fe640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e6c0b52e0 0x7f9e6c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.221+0000 7f9e74ff9640 1 -- 172.21.15.33:0/4267380314 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78031db0 con 0x557b451c14d0 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.222+0000 7f9e74ff9640 1 -- 172.21.15.33:0/4267380314 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e78036030 con 0x557b451c14d0 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.222+0000 7f9e74ff9640 1 -- 172.21.15.33:0/4267380314 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780042a0 con 0x557b451c14d0 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.222+0000 7f9e86243640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e6c0b52e0 0x7f9e6c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.222+0000 7f9e777fe640 1 -- 172.21.15.33:0/2982624902 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9e7c0e5e80 con 0x7f9e80101c90 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.222+0000 7f9e8a87c740 1 -- 172.21.15.33:0/4267380314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 msgr2=0x557b451c18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.222+0000 7f9e86243640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e6c0b52e0 0x7f9e6c0b77a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c096c20 tx=0x7f9e7c00ef90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.222+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c18b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009ca0 tx=0x7f9e7802f3c0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- 172.21.15.33:0/4267380314 shutdown_connections 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451cdd80 0x557b451d0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/4267380314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c18b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- 172.21.15.33:0/4267380314 >> 172.21.15.33:0/4267380314 conn(0x557b451bf300 msgr2=0x557b451c04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- 172.21.15.33:0/4267380314 shutdown_connections 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- 172.21.15.33:0/4267380314 wait complete. 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- start start 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 Processor -- start 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c1df0 0x557b451cab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451cb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557b44f88700 con 0x557b451c1df0 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557b450c2160 con 0x557b451cdd80 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.223+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b451d4c30 con 0x557b451c14d0 2024-09-10T21:19:38.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e8a87c740 10 client.0 _ll_register_callbacks cb 0x557b450db9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c1df0 0x557b451cab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c1df0 0x557b451cab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35546/0 (socket says 172.21.15.33:35546) 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 -- 172.21.15.33:0/2241692663 learned_addr learned my addr 172.21.15.33:0/2241692663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 -- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 msgr2=0x557b451cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 -- 172.21.15.33:0/2241692663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 msgr2=0x557b451cb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451cb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 -- 172.21.15.33:0/2241692663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b450efc90 con 0x557b451c1df0 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:19:38.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e85a42640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c1df0 0x557b451cab00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000bad0 tx=0x7f9e7000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f3020f524cef8f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2241692663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e70003e20 con 0x557b451c1df0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2241692663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9e70011440 con 0x557b451c1df0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.224+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e78036030 con 0x557b451c1df0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2241692663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f9e70011640 con 0x557b451c1df0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 10 client.18279 fetch_fsmap learned FSMap version 73 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x557b451e0600 con 0x557b451c1df0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2241692663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e78036030 con 0x557b451c1df0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2241692663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 73) ==== 8406+0+0 (secure 0 0 0) 0x7f9e7001b7f0 con 0x557b451c1df0 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 10 client.18279 fetch_fsmap finished waiting for FSMap version 73 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 10 client.18279 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 20 client.18279 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:38.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b451e1b80 0x557b451e3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 4 client.18279 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b45139df0 con 0x557b451e1b80 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.225+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b451e1b80 0x557b451e3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.226+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b451e1b80 0x557b451e3fa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39f8d659979e8122 server_cookie=6842e3215d913dba in_seq=0 out_seq=0 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.226+0000 7f9e69ffb640 10 client.18279 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.226+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2241692663 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557b45139df0 con 0x557b451e1b80 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.226+0000 7f9e69ffb640 10 client.18279 handle_command_reply: tid=0 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.226+0000 7f9e8a87c740 1 client.18279 shutdown 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c1df0 msgr2=0x557b451cab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c1df0 0x557b451cab00 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000bad0 tx=0x7f9e7000bf90 comp rx=0 tx=0).stop 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 shutdown_connections 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451cb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b451e1b80 0x557b451e3fa0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c1df0 0x557b451cab00 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2241692663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e69ffb640 0 client.18279 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 >> 172.21.15.33:0/2241692663 conn(0x557b451bf300 msgr2=0x557b451c4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 shutdown_connections 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2241692663 wait complete. 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.227+0000 7f9e8a87c740 20 client.18279 trim_cache size 0 max 16384 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 Processor -- start 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 -- start start 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c1df0 0x557b451dde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801ac1e0 con 0x557b451c14d0 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801ade70 con 0x557b451cdd80 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801ac8b0 con 0x557b451c1df0 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e86a44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e86a44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35552/0 (socket says 172.21.15.33:35552) 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e86a44640 1 -- 172.21.15.33:0/3337784890 learned_addr learned my addr 172.21.15.33:0/3337784890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e85a42640 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c1df0 0x557b451dde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.242+0000 7f9e86a44640 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e86a44640 1 -- 172.21.15.33:0/3337784890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c1df0 msgr2=0x557b451dde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e86a44640 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c1df0 0x557b451dde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e86a44640 1 -- 172.21.15.33:0/3337784890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 msgr2=0x557b451ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e86a44640 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e86a44640 1 -- 172.21.15.33:0/3337784890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b451e4620 con 0x557b451c14d0 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e86a44640 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c0b40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9e780096c0 tx=0x7f9e7802f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c11156ddaf3647 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e6bfff640 1 -- 172.21.15.33:0/3337784890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78003c10 con 0x557b451c14d0 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e6bfff640 1 -- 172.21.15.33:0/3337784890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e78036070 con 0x557b451c14d0 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.243+0000 7f9e6bfff640 1 -- 172.21.15.33:0/3337784890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78004260 con 0x557b451c14d0 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 -- 172.21.15.33:0/3337784890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 msgr2=0x557b451c0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c0b40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f9e780096c0 tx=0x7f9e7802f3c0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 -- 172.21.15.33:0/3337784890 shutdown_connections 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451cdd80 0x557b451ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c1df0 0x557b451dde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/3337784890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451c14d0 0x557b451c0b40 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 -- 172.21.15.33:0/3337784890 >> 172.21.15.33:0/3337784890 conn(0x557b451bf300 msgr2=0x557b451c0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:38.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 -- 172.21.15.33:0/3337784890 shutdown_connections 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 -- 172.21.15.33:0/3337784890 wait complete. 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 -- start start 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.244+0000 7f9e8a87c740 1 Processor -- start 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451d9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451da410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e8a87c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451cdd80 0x557b451d4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801ad740 con 0x557b451cdd80 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801acfe0 con 0x557b451c1df0 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e8a87c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e801aa1a0 con 0x557b451c14d0 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e8a87c740 10 client.0 _ll_register_callbacks cb 0x7f9e70004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e85a42640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451da410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e85a42640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451da410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51328/0 (socket says 172.21.15.33:51328) 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e85a42640 1 -- 172.21.15.33:0/2585016800 learned_addr learned my addr 172.21.15.33:0/2585016800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451cdd80 0x557b451d4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451d9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e86a44640 1 -- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 msgr2=0x557b451d9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451d9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e86a44640 1 -- 172.21.15.33:0/2585016800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 msgr2=0x557b451da410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451da410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.245+0000 7f9e86a44640 1 -- 172.21.15.33:0/2585016800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b451dab00 con 0x557b451cdd80 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e86a44640 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451cdd80 0x557b451d4e80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009bf0 tx=0x7f9e78031e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebe965baa094e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e69ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2585016800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7805f040 con 0x557b451cdd80 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2585016800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9e78031450 con 0x557b451cdd80 2024-09-10T21:19:38.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e78036070 con 0x557b451cdd80 2024-09-10T21:19:38.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2585016800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f9e78031650 con 0x557b451cdd80 2024-09-10T21:19:38.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 10 client.18285 fetch_fsmap learned FSMap version 73 2024-09-10T21:19:38.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x557b4524ef50 con 0x557b451cdd80 2024-09-10T21:19:38.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2585016800 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e78036070 con 0x557b451cdd80 2024-09-10T21:19:38.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2585016800 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 73) ==== 8406+0+0 (secure 0 0 0) 0x7f9e78043070 con 0x557b451cdd80 2024-09-10T21:19:38.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 10 client.18285 fetch_fsmap finished waiting for FSMap version 73 2024-09-10T21:19:38.774 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 10 client.18285 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 20 client.18285 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b4524f310 0x557b45251730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 4 client.18285 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.246+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557b451358a0 con 0x557b4524f310 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.247+0000 7f9e85a42640 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b4524f310 0x557b45251730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.247+0000 7f9e85a42640 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b4524f310 0x557b45251730 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24171284bd25d01e server_cookie=61ad4043e3f8ca72 in_seq=0 out_seq=0 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.247+0000 7f9e69ffb640 10 client.18285 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e69ffb640 1 -- 172.21.15.33:0/2585016800 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557b451358a0 con 0x557b4524f310 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e69ffb640 10 client.18285 handle_command_reply: tid=0 2024-09-10T21:19:38.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 client.18285 shutdown 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451cdd80 msgr2=0x557b451d4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451cdd80 0x557b451d4e80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009bf0 tx=0x7f9e78031e10 comp rx=0 tx=0).stop 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 shutdown_connections 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557b4524f310 0x557b45251730 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557b451cdd80 0x557b451d4e80 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557b451c1df0 0x557b451da410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 --2- 172.21.15.33:0/2585016800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b451c14d0 0x557b451d9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e69ffb640 0 client.18285 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 >> 172.21.15.33:0/2585016800 conn(0x557b451bf300 msgr2=0x557b451cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 shutdown_connections 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 1 -- 172.21.15.33:0/2585016800 wait complete. 2024-09-10T21:19:38.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.249+0000 7f9e8a87c740 20 client.18285 trim_cache size 0 max 16384 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e6c0b52e0 msgr2=0x7f9e6c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e6c0b52e0 0x7f9e6c0b77a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c096c20 tx=0x7f9e7c00ef90 comp rx=0 tx=0).stop 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 msgr2=0x7f9e80199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 0x7f9e80199230 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00ea60 tx=0x7f9e7c00ef20 comp rx=0 tx=0).stop 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 shutdown_connections 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9e6c0b52e0 0x7f9e6c0b77a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e80108020 0x7f9e8019eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e801076a0 0x7f9e80199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 --2- 172.21.15.33:0/2982624902 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9e80101c90 0x7f9e80199230 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 >> 172.21.15.33:0/2982624902 conn(0x7f9e800fd700 msgr2=0x7f9e80105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 shutdown_connections 2024-09-10T21:19:38.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:38.250+0000 7f9e884a7640 1 -- 172.21.15.33:0/2982624902 wait complete. 2024-09-10T21:19:38.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:38 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:39.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 133457 41.90 MB/sec execute 39 sec latency 259.868 ms 2024-09-10T21:19:39.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:38 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:39.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:38 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:39.793 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:19:40.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 134397 42.14 MB/sec execute 40 sec latency 101.366 ms 2024-09-10T21:19:40.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:39 smithi104 ceph-mon[35325]: pgmap v816: 97 pgs: 97 active+clean; 1009 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 978 op/s 2024-09-10T21:19:40.125 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:39 smithi174 ceph-mon[35635]: pgmap v816: 97 pgs: 97 active+clean; 1009 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 978 op/s 2024-09-10T21:19:40.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:39 smithi033 ceph-mon[30860]: pgmap v816: 97 pgs: 97 active+clean; 1009 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 978 op/s 2024-09-10T21:19:40.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 -- 172.21.15.33:0/597690598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f41045c0 msgr2=0x7f46f410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 --2- 172.21.15.33:0/597690598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f41045c0 0x7f46f410cab0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009900 tx=0x7f46e402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 -- 172.21.15.33:0/597690598 shutdown_connections 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 --2- 172.21.15.33:0/597690598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46f410d100 0x7f46f410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 --2- 172.21.15.33:0/597690598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f41045c0 0x7f46f410cab0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 --2- 172.21.15.33:0/597690598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46f4103ca0 0x7f46f4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 -- 172.21.15.33:0/597690598 >> 172.21.15.33:0/597690598 conn(0x7f46f40fd700 msgr2=0x7f46f40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 -- 172.21.15.33:0/597690598 shutdown_connections 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.702+0000 7f46fb846640 1 -- 172.21.15.33:0/597690598 wait complete. 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.703+0000 7f46fb846640 1 Processor -- start 2024-09-10T21:19:40.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.703+0000 7f46fb846640 1 -- start start 2024-09-10T21:19:40.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.703+0000 7f46fb846640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4103ca0 0x7f46f41992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.703+0000 7f46fb846640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46f41045c0 0x7f46f4199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.703+0000 7f46fb846640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46f410d100 0x7f46f419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46fb846640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4112d70 con 0x7f46f4103ca0 2024-09-10T21:19:40.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46fb846640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4112bf0 con 0x7f46f410d100 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46fb846640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46f4112ef0 con 0x7f46f41045c0 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f95e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4103ca0 0x7f46f41992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f9de3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46f410d100 0x7f46f419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f95e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4103ca0 0x7f46f41992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60528/0 (socket says 172.21.15.33:60528) 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f95e2640 1 -- 172.21.15.33:0/2403306431 learned_addr learned my addr 172.21.15.33:0/2403306431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f8de1640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46f41045c0 0x7f46f4199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f9de3640 1 -- 172.21.15.33:0/2403306431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46f41045c0 msgr2=0x7f46f4199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f9de3640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46f41045c0 0x7f46f4199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f9de3640 1 -- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4103ca0 msgr2=0x7f46f41992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f9de3640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4103ca0 0x7f46f41992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f9de3640 1 -- 172.21.15.33:0/2403306431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46f419f390 con 0x7f46f410d100 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f95e2640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4103ca0 0x7f46f41992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.704+0000 7f46f9de3640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46f410d100 0x7f46f419ebc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009fd0 tx=0x7f46ec00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.705+0000 7f46ea7fc640 1 -- 172.21.15.33:0/2403306431 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec00ce20 con 0x7f46f410d100 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.705+0000 7f46ea7fc640 1 -- 172.21.15.33:0/2403306431 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46ec005020 con 0x7f46f410d100 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.705+0000 7f46ea7fc640 1 -- 172.21.15.33:0/2403306431 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec010690 con 0x7f46f410d100 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.705+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46f419f680 con 0x7f46f410d100 2024-09-10T21:19:40.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.705+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46f4101830 con 0x7f46f410d100 2024-09-10T21:19:40.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 Processor -- start 2024-09-10T21:19:40.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46ea7fc640 1 -- 172.21.15.33:0/2403306431 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f46ec0054d0 con 0x7f46f410d100 2024-09-10T21:19:40.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 -- start start 2024-09-10T21:19:40.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e362ff68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e363002840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e363005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e362f6cea0 con 0x55e362ff6df0 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e362f5c2d0 con 0x55e362ff64d0 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.709+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e362f600e0 con 0x55e363002d80 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f8de1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e362ff68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f95e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e363002840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f8de1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e362ff68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:32786/0 (socket says 172.21.15.33:32786) 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f8de1640 1 -- 172.21.15.33:0/492286879 learned_addr learned my addr 172.21.15.33:0/492286879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f8de1640 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e363005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46ea7fc640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46e00b5360 0x7f46e00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f95e2640 1 -- 172.21.15.33:0/492286879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 msgr2=0x55e363005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f95e2640 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e363005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f95e2640 1 -- 172.21.15.33:0/492286879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 msgr2=0x55e362ff68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f95e2640 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e362ff68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f95e2640 1 -- 172.21.15.33:0/492286879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e362cba650 con 0x55e362ff6df0 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.710+0000 7f46f9de3640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46e00b5360 0x7f46e00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.711+0000 7f46ea7fc640 1 -- 172.21.15.33:0/2403306431 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f46ec0e6290 con 0x7f46f410d100 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.711+0000 7f46f95e2640 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e363002840 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f46f00058c0 tx=0x7f46f00109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6914b9387ac727dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.711+0000 7f46df7fe640 1 -- 172.21.15.33:0/492286879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0011820 con 0x55e362ff6df0 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.711+0000 7f46df7fe640 1 -- 172.21.15.33:0/492286879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f000d340 con 0x55e362ff6df0 2024-09-10T21:19:40.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.711+0000 7f46df7fe640 1 -- 172.21.15.33:0/492286879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0019570 con 0x55e362ff6df0 2024-09-10T21:19:40.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 -- 172.21.15.33:0/492286879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 msgr2=0x55e363002840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e363002840 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f46f00058c0 tx=0x7f46f00109e0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46f9de3640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46e00b5360 0x7f46e00b7820 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0ac9e0 tx=0x7f46ec071660 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:40.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 -- 172.21.15.33:0/492286879 shutdown_connections 2024-09-10T21:19:40.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e363005170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e363002840 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/492286879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e362ff68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 -- 172.21.15.33:0/492286879 >> 172.21.15.33:0/492286879 conn(0x55e362ff4300 msgr2=0x55e362ff54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 -- 172.21.15.33:0/492286879 shutdown_connections 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 -- 172.21.15.33:0/492286879 wait complete. 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 -- start start 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.712+0000 7f46fdc1b740 1 Processor -- start 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e363002670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff6df0 0x55e362fffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e363002d80 0x55e3630000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e362dbd700 con 0x55e363002d80 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e362ef7160 con 0x55e362ff6df0 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e363005a00 con 0x55e362ff64d0 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46fdc1b740 10 client.0 _ll_register_callbacks cb 0x55e362f109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f8de1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e363002d80 0x55e3630000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff6df0 0x55e362fffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff6df0 0x55e362fffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:32790/0 (socket says 172.21.15.33:32790) 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f8de1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e363002d80 0x55e3630000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60546/0 (socket says 172.21.15.33:60546) 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 -- 172.21.15.33:0/2712549132 learned_addr learned my addr 172.21.15.33:0/2712549132 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f8de1640 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e363002670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 -- 172.21.15.33:0/2712549132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 msgr2=0x55e363002670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e363002670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 -- 172.21.15.33:0/2712549132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e363002d80 msgr2=0x55e3630000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e363002d80 0x55e3630000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f95e2640 1 -- 172.21.15.33:0/2712549132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e362f24c90 con 0x55e362ff6df0 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.713+0000 7f46f8de1640 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e363002d80 0x55e3630000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.714+0000 7f46f95e2640 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff6df0 0x55e362fffac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f46f000dc80 tx=0x7f46f000d720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=647401b1e6680117 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:40.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.714+0000 7f46dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.714+0000 7f46dcff9640 1 -- 172.21.15.33:0/2712549132 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f002a060 con 0x55e362ff6df0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.714+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f000d340 con 0x55e362ff6df0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.714+0000 7f46dcff9640 1 -- 172.21.15.33:0/2712549132 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f46f0005250 con 0x55e362ff6df0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.714+0000 7f46dcff9640 1 -- 172.21.15.33:0/2712549132 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f46f000bd20 con 0x55e362ff6df0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.714+0000 7f46fdc1b740 10 client.27710 fetch_fsmap learned FSMap version 73 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.715+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55e363011360 con 0x55e362ff6df0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.715+0000 7f46dcff9640 1 -- 172.21.15.33:0/2712549132 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46f000d890 con 0x55e362ff6df0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46dcff9640 1 -- 172.21.15.33:0/2712549132 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 73) ==== 8406+0+0 (secure 0 0 0) 0x7f46f002f070 con 0x55e362ff6df0 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46fdc1b740 10 client.27710 fetch_fsmap finished waiting for FSMap version 73 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46fdc1b740 10 client.27710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46fdc1b740 20 client.27710 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:40.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e363012770 0x55e363014b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46fdc1b740 4 client.27710 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:40.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f46f000d340 con 0x55e363012770 2024-09-10T21:19:40.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.716+0000 7f46f8de1640 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e363012770 0x55e363014b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.717+0000 7f46f8de1640 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e363012770 0x55e363014b60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3fcf519f88ded54 server_cookie=35280ee75d357f3e in_seq=0 out_seq=0 2024-09-10T21:19:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.717+0000 7f46dcff9640 10 client.27710 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.717+0000 7f46dcff9640 1 -- 172.21.15.33:0/2712549132 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f46f000d340 con 0x55e363012770 2024-09-10T21:19:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46dcff9640 10 client.27710 handle_command_reply: tid=0 2024-09-10T21:19:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 client.27710 shutdown 2024-09-10T21:19:40.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff6df0 msgr2=0x55e362fffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff6df0 0x55e362fffac0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f46f000dc80 tx=0x7f46f000d720 comp rx=0 tx=0).stop 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 shutdown_connections 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e363012770 0x55e363014b60 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e363002d80 0x55e3630000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff6df0 0x55e362fffac0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/2712549132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e363002670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 >> 172.21.15.33:0/2712549132 conn(0x55e362ff4300 msgr2=0x55e362ff9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.718+0000 7f46dcff9640 0 client.27710 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.719+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 shutdown_connections 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.719+0000 7f46fdc1b740 1 -- 172.21.15.33:0/2712549132 wait complete. 2024-09-10T21:19:40.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.719+0000 7f46fdc1b740 20 client.27710 trim_cache size 0 max 16384 2024-09-10T21:19:40.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 Processor -- start 2024-09-10T21:19:40.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 -- start start 2024-09-10T21:19:40.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e36300ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e362ff3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e363002d80 0x55e363004170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b05d0 con 0x55e362ff6df0 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b2260 con 0x55e363002d80 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b0ca0 con 0x55e362ff64d0 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46f95e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e362ff3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46f95e2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e362ff3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60554/0 (socket says 172.21.15.33:60554) 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46f8de1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e363002d80 0x55e363004170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46f95e2640 1 -- 172.21.15.33:0/593227778 learned_addr learned my addr 172.21.15.33:0/593227778 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.734+0000 7f46f8de1640 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e36300ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46f8de1640 1 -- 172.21.15.33:0/593227778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 msgr2=0x55e36300ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46f8de1640 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e36300ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46f8de1640 1 -- 172.21.15.33:0/593227778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 msgr2=0x55e362ff3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46f8de1640 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e362ff3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46f8de1640 1 -- 172.21.15.33:0/593227778 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e362ff4d80 con 0x55e363002d80 2024-09-10T21:19:40.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46f95e2640 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e362ff3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:40.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46f8de1640 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e363002d80 0x55e363004170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006010 tx=0x7f46e4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850883404fa20480 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:40.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.735+0000 7f46deffd640 1 -- 172.21.15.33:0/593227778 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4035040 con 0x55e363002d80 2024-09-10T21:19:40.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.736+0000 7f46deffd640 1 -- 172.21.15.33:0/593227778 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e4005500 con 0x55e363002d80 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.736+0000 7f46deffd640 1 -- 172.21.15.33:0/593227778 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40035f0 con 0x55e363002d80 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.736+0000 7f46fdc1b740 1 -- 172.21.15.33:0/593227778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e363002d80 msgr2=0x55e363004170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.736+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e363002d80 0x55e363004170 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f46e4006010 tx=0x7f46e4009260 comp rx=0 tx=0).stop 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 -- 172.21.15.33:0/593227778 shutdown_connections 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e363002d80 0x55e363004170 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e362ff3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/593227778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e362ff64d0 0x55e36300ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 -- 172.21.15.33:0/593227778 >> 172.21.15.33:0/593227778 conn(0x55e362ff4300 msgr2=0x55e362ff2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 -- 172.21.15.33:0/593227778 shutdown_connections 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 -- 172.21.15.33:0/593227778 wait complete. 2024-09-10T21:19:40.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 -- start start 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.737+0000 7f46fdc1b740 1 Processor -- start 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e36307c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e36307c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46fdc1b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e36307ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b1b30 con 0x55e362ff6df0 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41b13d0 con 0x55e362ff64d0 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46fdc1b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46f41ae590 con 0x55e363002d80 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46fdc1b740 10 client.0 _ll_register_callbacks cb 0x7f46ec0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e36307c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e36307c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:32814/0 (socket says 172.21.15.33:32814) 2024-09-10T21:19:40.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 -- 172.21.15.33:0/1615464189 learned_addr learned my addr 172.21.15.33:0/1615464189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f95e2640 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e36307c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e36307ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 -- 172.21.15.33:0/1615464189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 msgr2=0x55e36307ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e36307ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 -- 172.21.15.33:0/1615464189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 msgr2=0x55e36307c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e36307c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.738+0000 7f46f8de1640 1 -- 172.21.15.33:0/1615464189 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e363079460 con 0x55e362ff64d0 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46f8de1640 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e36307c390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46e4032b10 tx=0x7f46e4004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14fc25d9186ca98 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:19:40.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46dcff9640 1 -- 172.21.15.33:0/1615464189 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4035040 con 0x55e362ff64d0 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46dcff9640 1 -- 172.21.15.33:0/1615464189 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f46e4004770 con 0x55e362ff64d0 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e4005500 con 0x55e362ff64d0 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46dcff9640 1 -- 172.21.15.33:0/1615464189 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) ==== 24+0+0 (secure 0 0 0) 0x7f46e4004310 con 0x55e362ff64d0 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46fdc1b740 10 client.27716 fetch_fsmap learned FSMap version 73 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) -- 0x55e36300a0e0 con 0x55e362ff64d0 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46dcff9640 1 -- 172.21.15.33:0/1615464189 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e4005500 con 0x55e362ff64d0 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.739+0000 7f46dcff9640 1 -- 172.21.15.33:0/1615464189 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 73) ==== 8406+0+0 (secure 0 0 0) 0x7f46e4053030 con 0x55e362ff64d0 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46fdc1b740 10 client.27716 fetch_fsmap finished waiting for FSMap version 73 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46fdc1b740 10 client.27716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46fdc1b740 20 client.27716 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e36300a8f0 0x55e36300ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46fdc1b740 4 client.27716 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:19:40.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e362f6edf0 con 0x55e36300a8f0 2024-09-10T21:19:40.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46f95e2640 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e36300a8f0 0x55e36300ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46dcff9640 10 client.27716 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46f95e2640 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e36300a8f0 0x55e36300ad00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c548928d2aa72c server_cookie=b956ab4aecb7affb in_seq=0 out_seq=0 2024-09-10T21:19:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46dcff9640 1 -- 172.21.15.33:0/1615464189 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e362f6edf0 con 0x55e36300a8f0 2024-09-10T21:19:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46dcff9640 10 client.27716 handle_command_reply: tid=0 2024-09-10T21:19:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.740+0000 7f46fdc1b740 1 client.27716 shutdown 2024-09-10T21:19:40.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 msgr2=0x55e36307c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e36307c390 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f46e4032b10 tx=0x7f46e4004a90 comp rx=0 tx=0).stop 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 shutdown_connections 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e36300a8f0 0x55e36300ad00 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e363002d80 0x55e36307ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e362ff6df0 0x55e36307c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46dcff9640 0 client.27716 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 --2- 172.21.15.33:0/1615464189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e362ff64d0 0x55e36307c390 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 >> 172.21.15.33:0/1615464189 conn(0x55e362ff4300 msgr2=0x55e362ff3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 shutdown_connections 2024-09-10T21:19:40.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 1 -- 172.21.15.33:0/1615464189 wait complete. 2024-09-10T21:19:40.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.741+0000 7f46fdc1b740 20 client.27716 trim_cache size 0 max 16384 2024-09-10T21:19:40.751 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46e00b5360 msgr2=0x7f46e00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46e00b5360 0x7f46e00b7820 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0ac9e0 tx=0x7f46ec071660 comp rx=0 tx=0).stop 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46f410d100 msgr2=0x7f46f419ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46f410d100 0x7f46f419ebc0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f46ec009fd0 tx=0x7f46ec00ef10 comp rx=0 tx=0).stop 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 shutdown_connections 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f46e00b5360 0x7f46e00b7820 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f46f410d100 0x7f46f419ebc0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46f41045c0 0x7f46f4199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 --2- 172.21.15.33:0/2403306431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4103ca0 0x7f46f41992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:40.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 >> 172.21.15.33:0/2403306431 conn(0x7f46f40fd700 msgr2=0x7f46f40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:40.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 shutdown_connections 2024-09-10T21:19:40.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:40.742+0000 7f46fb846640 1 -- 172.21.15.33:0/2403306431 wait complete. 2024-09-10T21:19:41.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 135202 42.02 MB/sec execute 41 sec latency 114.370 ms 2024-09-10T21:19:42.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 136027 42.12 MB/sec execute 42 sec latency 221.277 ms 2024-09-10T21:19:42.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:41 smithi104 ceph-mon[35325]: pgmap v817: 97 pgs: 97 active+clean; 1011 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 884 op/s 2024-09-10T21:19:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:41 smithi174 ceph-mon[35635]: pgmap v817: 97 pgs: 97 active+clean; 1011 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 884 op/s 2024-09-10T21:19:42.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:41 smithi033 ceph-mon[30860]: pgmap v817: 97 pgs: 97 active+clean; 1011 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 884 op/s 2024-09-10T21:19:42.937 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:19:42.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:19:42.966 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:42 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:43.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 136786 42.02 MB/sec execute 43 sec latency 120.575 ms 2024-09-10T21:19:43.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:42 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:43.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:42 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:43.263 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:43.925 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:19:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:19:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "6023db06-103a-4538-877a-f434e14bd948", 2024-09-10T21:19:43.925 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:19:43.925 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:19:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 -- 172.21.15.33:0/3153956038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc1087f0 msgr2=0x7fd7fc108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 --2- 172.21.15.33:0/3153956038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc108c30 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009900 tx=0x7fd7e802a410 comp rx=0 tx=0).stop 2024-09-10T21:19:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 -- 172.21.15.33:0/3153956038 shutdown_connections 2024-09-10T21:19:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 --2- 172.21.15.33:0/3153956038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7fc109170 0x7fd7fc10f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 --2- 172.21.15.33:0/3153956038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc108c30 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 --2- 172.21.15.33:0/3153956038 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc102de0 0x7fd7fc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 -- 172.21.15.33:0/3153956038 >> 172.21.15.33:0/3153956038 conn(0x7fd7fc0fe830 msgr2=0x7fd7fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 -- 172.21.15.33:0/3153956038 shutdown_connections 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.857+0000 7fd802ea5640 1 -- 172.21.15.33:0/3153956038 wait complete. 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 Processor -- start 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 -- start start 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7fc102de0 0x7fd7fc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc109170 0x7fd7fc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc112dc0 con 0x7fd7fc109170 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc112c40 con 0x7fd7fc1087f0 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd802ea5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc112f40 con 0x7fd7fc102de0 2024-09-10T21:19:43.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:32836/0 (socket says 172.21.15.33:32836) 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd7fbfff640 1 -- 172.21.15.33:0/1655172627 learned_addr learned my addr 172.21.15.33:0/1655172627 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd800c41640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7fc102de0 0x7fd7fc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.858+0000 7fd801442640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc109170 0x7fd7fc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.859+0000 7fd7fbfff640 1 -- 172.21.15.33:0/1655172627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7fc102de0 msgr2=0x7fd7fc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.859+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7fc102de0 0x7fd7fc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.859+0000 7fd7fbfff640 1 -- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc109170 msgr2=0x7fd7fc19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.859+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc109170 0x7fd7fc19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.859+0000 7fd7fbfff640 1 -- 172.21.15.33:0/1655172627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7fc19f340 con 0x7fd7fc1087f0 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.859+0000 7fd801442640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc109170 0x7fd7fc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.859+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc1997d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005940 tx=0x7fd7e80040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.860+0000 7fd7f9ffb640 1 -- 172.21.15.33:0/1655172627 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e803f030 con 0x7fd7fc1087f0 2024-09-10T21:19:43.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.860+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7fc19f5d0 con 0x7fd7fc1087f0 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.860+0000 7fd7f9ffb640 1 -- 172.21.15.33:0/1655172627 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e80044c0 con 0x7fd7fc1087f0 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.860+0000 7fd7f9ffb640 1 -- 172.21.15.33:0/1655172627 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8035070 con 0x7fd7fc1087f0 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.860+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7fc075380 con 0x7fd7fc1087f0 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd7f9ffb640 1 -- 172.21.15.33:0/1655172627 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd7e803ba20 con 0x7fd7fc1087f0 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 Processor -- start 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 -- start start 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d05fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0608250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5619d0572e60 con 0x5619d05fc800 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5619d0562290 con 0x5619d0608790 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd80527a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5619d05660a0 con 0x5619d05fbee0 2024-09-10T21:19:43.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd800c41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0608250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd800c41640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0608250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60604/0 (socket says 172.21.15.33:60604) 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd800c41640 1 -- 172.21.15.33:0/1069396473 learned_addr learned my addr 172.21.15.33:0/1069396473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.863+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd800c41640 1 -- 172.21.15.33:0/1069396473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 msgr2=0x5619d05fc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd800c41640 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d05fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd800c41640 1 -- 172.21.15.33:0/1069396473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 msgr2=0x5619d060ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd800c41640 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd800c41640 1 -- 172.21.15.33:0/1069396473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d02c0930 con 0x5619d05fc800 2024-09-10T21:19:43.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd7f9ffb640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7ec0b52e0 0x7fd7ec0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd801442640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7ec0b52e0 0x7fd7ec0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd7f9ffb640 1 -- 172.21.15.33:0/1655172627 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd7e8106250 con 0x7fd7fc1087f0 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd800c41640 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0608250 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009cb0 tx=0x7fd7f00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64f808179bd80d1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd7e6ffd640 1 -- 172.21.15.33:0/1069396473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f001b470 con 0x5619d05fc800 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd7e6ffd640 1 -- 172.21.15.33:0/1069396473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7f0016070 con 0x5619d05fc800 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.864+0000 7fd7e6ffd640 1 -- 172.21.15.33:0/1069396473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0012960 con 0x5619d05fc800 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 -- 172.21.15.33:0/1069396473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 msgr2=0x5619d0608250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0608250 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0009cb0 tx=0x7fd7f00117b0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd801442640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7ec0b52e0 0x7fd7ec0b77a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005fd0 tx=0x7fd7f4012040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 -- 172.21.15.33:0/1069396473 shutdown_connections 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0608250 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 --2- 172.21.15.33:0/1069396473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d05fc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 -- 172.21.15.33:0/1069396473 >> 172.21.15.33:0/1069396473 conn(0x5619d05f9d10 msgr2=0x5619d05faeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 -- 172.21.15.33:0/1069396473 shutdown_connections 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.865+0000 7fd80527a740 1 -- 172.21.15.33:0/1069396473 wait complete. 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.866+0000 7fd80527a740 1 -- start start 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.866+0000 7fd80527a740 1 Processor -- start 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.866+0000 7fd80527a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fbee0 0x5619d0605730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.866+0000 7fd80527a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0605c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.866+0000 7fd80527a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d0608790 0x5619d06061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.866+0000 7fd80527a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5619d03c3890 con 0x5619d05fc800 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd80527a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5619d04fcd40 con 0x5619d05fbee0 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd80527a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5619d060b440 con 0x5619d0608790 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd80527a740 10 client.0 _ll_register_callbacks cb 0x5619d0560c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0605c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd800c41640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fbee0 0x5619d0605730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0605c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60612/0 (socket says 172.21.15.33:60612) 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 -- 172.21.15.33:0/2199423782 learned_addr learned my addr 172.21.15.33:0/2199423782 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 -- 172.21.15.33:0/2199423782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d0608790 msgr2=0x5619d06061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d0608790 0x5619d06061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 -- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fbee0 msgr2=0x5619d0605730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fbee0 0x5619d0605730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 -- 172.21.15.33:0/2199423782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d06147f0 con 0x5619d05fc800 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd800c41640 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fbee0 0x5619d0605730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0605c70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80c8fe0 tx=0x7fd7e802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc21390a01e40a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7dbfff640 1 -- 172.21.15.33:0/2199423782 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8049040 con 0x5619d05fc800 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd7dbfff640 1 -- 172.21.15.33:0/2199423782 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd7e80ccc70 con 0x5619d05fc800 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.867+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f0016070 con 0x5619d05fc800 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd7dbfff640 1 -- 172.21.15.33:0/2199423782 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fd7e80c88b0 con 0x5619d05fc800 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 10 client.18315 fetch_fsmap learned FSMap version 74 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5619d06171a0 con 0x5619d05fc800 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd7dbfff640 1 -- 172.21.15.33:0/2199423782 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7f0016070 con 0x5619d05fc800 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd7dbfff640 1 -- 172.21.15.33:0/2199423782 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 74) ==== 8402+0+0 (secure 0 0 0) 0x7fd7e80d1080 con 0x5619d05fc800 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 10 client.18315 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 10 client.18315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 20 client.18315 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d06186f0 0x5619d061ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 4 client.18315 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:43.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619d0574db0 con 0x5619d06186f0 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.868+0000 7fd800c41640 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d06186f0 0x5619d061ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.869+0000 7fd7dbfff640 10 client.18315 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.869+0000 7fd800c41640 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d06186f0 0x5619d061ab90 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=662e44db369da95c server_cookie=b7143e9d47e77b03 in_seq=0 out_seq=0 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.870+0000 7fd7dbfff640 1 -- 172.21.15.33:0/2199423782 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5619d0574db0 con 0x5619d06186f0 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.870+0000 7fd7dbfff640 10 client.18315 handle_command_reply: tid=0 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.870+0000 7fd80527a740 1 client.18315 shutdown 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 msgr2=0x5619d0605c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0605c70 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80c8fe0 tx=0x7fd7e802a410 comp rx=0 tx=0).stop 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 shutdown_connections 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d06186f0 0x5619d061ab90 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d0608790 0x5619d06061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0605c70 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 --2- 172.21.15.33:0/2199423782 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fbee0 0x5619d0605730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd7dbfff640 0 client.18315 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 >> 172.21.15.33:0/2199423782 conn(0x5619d05f9d10 msgr2=0x5619d05f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 shutdown_connections 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 1 -- 172.21.15.33:0/2199423782 wait complete. 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.871+0000 7fd80527a740 20 client.18315 trim_cache size 0 max 16384 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 Processor -- start 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 -- start start 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d05fb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fc800 0x5619d0607470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d0608790 0x5619d0602960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1ac240 con 0x5619d0608790 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1adea0 con 0x5619d05fc800 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd80527a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1ac910 con 0x5619d05fbee0 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d0608790 0x5619d0602960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d0608790 0x5619d0602960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60618/0 (socket says 172.21.15.33:60618) 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 -- 172.21.15.33:0/2474526655 learned_addr learned my addr 172.21.15.33:0/2474526655 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fc800 0x5619d0607470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd800c41640 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d05fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 -- 172.21.15.33:0/2474526655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 msgr2=0x5619d05fb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d05fb290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 -- 172.21.15.33:0/2474526655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fc800 msgr2=0x5619d0607470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fc800 0x5619d0607470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd801442640 1 -- 172.21.15.33:0/2474526655 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d05fa790 con 0x5619d0608790 2024-09-10T21:19:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.894+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fc800 0x5619d0607470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.895+0000 7fd801442640 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d0608790 0x5619d0602960 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004a80 tx=0x7fd7f4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74435d6c14812dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.895+0000 7fd7e67fc640 1 -- 172.21.15.33:0/2474526655 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f402c070 con 0x5619d0608790 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.895+0000 7fd7e67fc640 1 -- 172.21.15.33:0/2474526655 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7f4023040 con 0x5619d0608790 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.895+0000 7fd7e67fc640 1 -- 172.21.15.33:0/2474526655 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4027cd0 con 0x5619d0608790 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 -- 172.21.15.33:0/2474526655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d0608790 msgr2=0x5619d0602960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d0608790 0x5619d0602960 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4004a80 tx=0x7fd7f4009500 comp rx=0 tx=0).stop 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 -- 172.21.15.33:0/2474526655 shutdown_connections 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d0608790 0x5619d0602960 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d05fc800 0x5619d0607470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 --2- 172.21.15.33:0/2474526655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d05fb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 -- 172.21.15.33:0/2474526655 >> 172.21.15.33:0/2474526655 conn(0x5619d05f9d10 msgr2=0x5619d0606d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 -- 172.21.15.33:0/2474526655 shutdown_connections 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 -- 172.21.15.33:0/2474526655 wait complete. 2024-09-10T21:19:43.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.896+0000 7fd80527a740 1 -- start start 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 1 Processor -- start 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d06108b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0610e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1ad7a0 con 0x5619d05fc800 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1ad040 con 0x5619d0608790 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd7fc1aa200 con 0x5619d05fbee0 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd80527a740 10 client.0 _ll_register_callbacks cb 0x7fd7e81099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0610e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd7fbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0610e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60624/0 (socket says 172.21.15.33:60624) 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd7fbfff640 1 -- 172.21.15.33:0/353597925 learned_addr learned my addr 172.21.15.33:0/353597925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd800c41640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d06108b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.897+0000 7fd801442640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7fbfff640 1 -- 172.21.15.33:0/353597925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 msgr2=0x5619d06108b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d06108b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7fbfff640 1 -- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 msgr2=0x5619d060d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7fbfff640 1 -- 172.21.15.33:0/353597925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d060ded0 con 0x5619d05fc800 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd801442640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7fbfff640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0610e50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80cca60 tx=0x7fd7e80c8f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840f92b22493d075 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7dbfff640 1 -- 172.21.15.33:0/353597925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8049070 con 0x5619d05fc800 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd7dbfff640 1 -- 172.21.15.33:0/353597925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd7e80ccc70 con 0x5619d05fc800 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.898+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7f4023040 con 0x5619d05fc800 2024-09-10T21:19:43.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd7dbfff640 1 -- 172.21.15.33:0/353597925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fd7e80cc180 con 0x5619d05fc800 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 10 client.18321 fetch_fsmap learned FSMap version 74 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5619d060ded0 con 0x5619d05fc800 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd7dbfff640 1 -- 172.21.15.33:0/353597925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7e80d1520 con 0x5619d05fc800 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd7dbfff640 1 -- 172.21.15.33:0/353597925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 74) ==== 8402+0+0 (secure 0 0 0) 0x7fd7e807c1a0 con 0x5619d05fc800 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 10 client.18321 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 10 client.18321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 20 client.18321 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d0686610 0x5619d0688a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 4 client.18321 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fd7f4023040 con 0x5619d0686610 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.899+0000 7fd800c41640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d0686610 0x5619d0688a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.900+0000 7fd7dbfff640 10 client.18321 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.900+0000 7fd800c41640 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d0686610 0x5619d0688a30 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96a684eac229d80 server_cookie=f5809dfac808e984 in_seq=0 out_seq=0 2024-09-10T21:19:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd7dbfff640 1 -- 172.21.15.33:0/353597925 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fd7f4023040 con 0x5619d0686610 2024-09-10T21:19:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd7dbfff640 10 client.18321 handle_command_reply: tid=0 2024-09-10T21:19:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 client.18321 shutdown 2024-09-10T21:19:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 msgr2=0x5619d0610e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0610e50 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fd7e80cca60 tx=0x7fd7e80c8f10 comp rx=0 tx=0).stop 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 shutdown_connections 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5619d0686610 0x5619d0688a30 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5619d0608790 0x5619d060d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5619d05fc800 0x5619d0610e50 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 --2- 172.21.15.33:0/353597925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d05fbee0 0x5619d06108b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 >> 172.21.15.33:0/353597925 conn(0x5619d05f9d10 msgr2=0x5619d0603bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.905+0000 7fd7dbfff640 0 client.18321 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.906+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 shutdown_connections 2024-09-10T21:19:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.906+0000 7fd80527a740 1 -- 172.21.15.33:0/353597925 wait complete. 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.906+0000 7fd80527a740 20 client.18321 trim_cache size 0 max 16384 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.906+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7ec0b52e0 msgr2=0x7fd7ec0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.906+0000 7fd802ea5640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7ec0b52e0 0x7fd7ec0b77a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4005fd0 tx=0x7fd7f4012040 comp rx=0 tx=0).stop 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.906+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc1087f0 msgr2=0x7fd7fc1997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.906+0000 7fd802ea5640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc1997d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8005940 tx=0x7fd7e80040c0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 shutdown_connections 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd7ec0b52e0 0x7fd7ec0b77a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd7fc109170 0x7fd7fc19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7fc1087f0 0x7fd7fc1997d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 --2- 172.21.15.33:0/1655172627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd7fc102de0 0x7fd7fc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 >> 172.21.15.33:0/1655172627 conn(0x7fd7fc0fe830 msgr2=0x7fd7fc105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 shutdown_connections 2024-09-10T21:19:43.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:43.907+0000 7fd802ea5640 1 -- 172.21.15.33:0/1655172627 wait complete. 2024-09-10T21:19:44.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 137656 42.36 MB/sec execute 44 sec latency 124.771 ms 2024-09-10T21:19:44.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:43 smithi104 ceph-mon[35325]: pgmap v818: 97 pgs: 97 active+clean; 1014 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 806 op/s 2024-09-10T21:19:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:43 smithi174 ceph-mon[35635]: pgmap v818: 97 pgs: 97 active+clean; 1014 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 806 op/s 2024-09-10T21:19:44.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:43 smithi033 ceph-mon[30860]: pgmap v818: 97 pgs: 97 active+clean; 1014 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 806 op/s 2024-09-10T21:19:44.782 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:19:45.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 138311 42.12 MB/sec execute 45 sec latency 210.938 ms 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:44 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:44 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:44 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:45.183 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:19:46.045 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.046 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:45 smithi033 ceph-mon[30860]: pgmap v819: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 896 op/s 2024-09-10T21:19:46.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 139142 42.18 MB/sec execute 46 sec latency 106.675 ms 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:45 smithi104 ceph-mon[35325]: pgmap v819: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 896 op/s 2024-09-10T21:19:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:19:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:45 smithi174 ceph-mon[35635]: pgmap v819: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 896 op/s 2024-09-10T21:19:46.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 -- 172.21.15.33:0/2107546007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 msgr2=0x7fa14810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 --2- 172.21.15.33:0/2107546007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa14810f560 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fa140009900 tx=0x7fa14002a410 comp rx=0 tx=0).stop 2024-09-10T21:19:46.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 -- 172.21.15.33:0/2107546007 shutdown_connections 2024-09-10T21:19:46.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 --2- 172.21.15.33:0/2107546007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa14810f560 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 --2- 172.21.15.33:0/2107546007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1481045c0 0x7fa14810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 --2- 172.21.15.33:0/2107546007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa148103ca0 0x7fa148104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 -- 172.21.15.33:0/2107546007 >> 172.21.15.33:0/2107546007 conn(0x7fa1480fd700 msgr2=0x7fa1480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 -- 172.21.15.33:0/2107546007 shutdown_connections 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.156+0000 7fa14db56640 1 -- 172.21.15.33:0/2107546007 wait complete. 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.157+0000 7fa14db56640 1 Processor -- start 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.157+0000 7fa14db56640 1 -- start start 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa14db56640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa148103ca0 0x7fa14819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa14db56640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1481045c0 0x7fa14819b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa14db56640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa1481a0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa14db56640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481025d0 con 0x7fa14810d100 2024-09-10T21:19:46.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa14db56640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa148102450 con 0x7fa148103ca0 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa14db56640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa148102750 con 0x7fa1481045c0 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa1481a0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa1481a0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60646/0 (socket says 172.21.15.33:60646) 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.158+0000 7fa147fff640 1 -- 172.21.15.33:0/1656611833 learned_addr learned my addr 172.21.15.33:0/1656611833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.159+0000 7fa1477fe640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa148103ca0 0x7fa14819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.159+0000 7fa146ffd640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1481045c0 0x7fa14819b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.159+0000 7fa147fff640 1 -- 172.21.15.33:0/1656611833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1481045c0 msgr2=0x7fa14819b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.159+0000 7fa147fff640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1481045c0 0x7fa14819b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.159+0000 7fa147fff640 1 -- 172.21.15.33:0/1656611833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa148103ca0 msgr2=0x7fa14819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.159+0000 7fa147fff640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa148103ca0 0x7fa14819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.159+0000 7fa147fff640 1 -- 172.21.15.33:0/1656611833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1481a1540 con 0x7fa14810d100 2024-09-10T21:19:46.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.160+0000 7fa147fff640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa1481a0d70 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa14002a930 tx=0x7fa140003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:46.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.160+0000 7fa144ff9640 1 -- 172.21.15.33:0/1656611833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140035030 con 0x7fa14810d100 2024-09-10T21:19:46.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.160+0000 7fa144ff9640 1 -- 172.21.15.33:0/1656611833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa140045070 con 0x7fa14810d100 2024-09-10T21:19:46.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.160+0000 7fa144ff9640 1 -- 172.21.15.33:0/1656611833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14004a760 con 0x7fa14810d100 2024-09-10T21:19:46.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.160+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1481a17d0 con 0x7fa14810d100 2024-09-10T21:19:46.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.160+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1481a1ba0 con 0x7fa14810d100 2024-09-10T21:19:46.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.164+0000 7fa14ff2b740 1 Processor -- start 2024-09-10T21:19:46.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.164+0000 7fa144ff9640 1 -- 172.21.15.33:0/1656611833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa140052030 con 0x7fa14810d100 2024-09-10T21:19:46.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.165+0000 7fa14ff2b740 1 -- start start 2024-09-10T21:19:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa144ff9640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1300b5430 0x7fa1300b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6da9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9da0 0x55b0b6db57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0b6d1fe50 con 0x55b0b6db5d30 2024-09-10T21:19:46.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b0b6d0f280 con 0x55b0b6da9480 2024-09-10T21:19:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b0b6d13090 con 0x55b0b6da9da0 2024-09-10T21:19:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa146ffd640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1300b5430 0x7fa1300b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa144ff9640 1 -- 172.21.15.33:0/1656611833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa14010e470 con 0x7fa14810d100 2024-09-10T21:19:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa1477fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6da9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9da0 0x55b0b6db57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa1477fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6da9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:32900/0 (socket says 172.21.15.33:32900) 2024-09-10T21:19:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa1477fe640 1 -- 172.21.15.33:0/1388980198 learned_addr learned my addr 172.21.15.33:0/1388980198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa147fff640 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.166+0000 7fa146ffd640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1300b5430 0x7fa1300b78f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa14810d8b0 tx=0x7fa13c0138e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa147fff640 1 -- 172.21.15.33:0/1388980198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9da0 msgr2=0x55b0b6db57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa147fff640 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9da0 0x55b0b6db57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa147fff640 1 -- 172.21.15.33:0/1388980198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 msgr2=0x55b0b6da9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa147fff640 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6da9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa147fff640 1 -- 172.21.15.33:0/1388980198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b6a6d650 con 0x55b0b6db5d30 2024-09-10T21:19:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa147fff640 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db8120 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa1400d1ea0 tx=0x7fa140009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8702365628740ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa12dffb640 1 -- 172.21.15.33:0/1388980198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400d4740 con 0x55b0b6db5d30 2024-09-10T21:19:46.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa12dffb640 1 -- 172.21.15.33:0/1388980198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1400d5330 con 0x55b0b6db5d30 2024-09-10T21:19:46.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.167+0000 7fa12dffb640 1 -- 172.21.15.33:0/1388980198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14010e0e0 con 0x55b0b6db5d30 2024-09-10T21:19:46.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1388980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 msgr2=0x55b0b6db8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db8120 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fa1400d1ea0 tx=0x7fa140009260 comp rx=0 tx=0).stop 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1388980198 shutdown_connections 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db8120 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9da0 0x55b0b6db57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1388980198 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6da9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1388980198 >> 172.21.15.33:0/1388980198 conn(0x55b0b6da72b0 msgr2=0x55b0b6da8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1388980198 shutdown_connections 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.168+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1388980198 wait complete. 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 -- start start 2024-09-10T21:19:46.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 Processor -- start 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6db5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6db2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 0x55b0b6db3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b0b6b70700 con 0x55b0b6da9da0 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b0b6caa110 con 0x55b0b6da9480 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b0b6db8770 con 0x55b0b6db5d30 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6db2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa14ff2b740 10 client.0 _ll_register_callbacks cb 0x55b0b6cc3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6db2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60662/0 (socket says 172.21.15.33:60662) 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa146ffd640 1 -- 172.21.15.33:0/3122090189 learned_addr learned my addr 172.21.15.33:0/3122090189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.169+0000 7fa1477fe640 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6db5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa147fff640 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 0x55b0b6db3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa146ffd640 1 -- 172.21.15.33:0/3122090189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 msgr2=0x55b0b6db3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa146ffd640 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 0x55b0b6db3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa146ffd640 1 -- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 msgr2=0x55b0b6db5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa146ffd640 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6db5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa146ffd640 1 -- 172.21.15.33:0/3122090189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b6cd7c40 con 0x55b0b6da9da0 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa146ffd640 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6db2b70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098b0 tx=0x7fa13c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a4000f8b704de8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:46.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa126ffd640 1 -- 172.21.15.33:0/3122090189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c02d060 con 0x55b0b6da9da0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.170+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1400d5330 con 0x55b0b6da9da0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa126ffd640 1 -- 172.21.15.33:0/3122090189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa13c020cb0 con 0x55b0b6da9da0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa126ffd640 1 -- 172.21.15.33:0/3122090189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa13c028420 con 0x55b0b6da9da0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 10 client.18333 fetch_fsmap learned FSMap version 74 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55b0b6dc4130 con 0x55b0b6da9da0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa126ffd640 1 -- 172.21.15.33:0/3122090189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1400d5330 con 0x55b0b6da9da0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa126ffd640 1 -- 172.21.15.33:0/3122090189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 74) ==== 8402+0+0 (secure 0 0 0) 0x7fa13c032070 con 0x55b0b6da9da0 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 10 client.18333 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T21:19:46.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 10 client.18333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 20 client.18333 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6dc55b0 0x55b0b6dc79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 4 client.18333 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.171+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0b6d21da0 con 0x55b0b6dc55b0 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.172+0000 7fa1477fe640 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6dc55b0 0x55b0b6dc79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.172+0000 7fa1477fe640 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6dc55b0 0x55b0b6dc79f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3abe64b6692a4b server_cookie=7f26c6e218fd2fa5 in_seq=0 out_seq=0 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.172+0000 7fa126ffd640 10 client.18333 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa126ffd640 1 -- 172.21.15.33:0/3122090189 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b0b6d21da0 con 0x55b0b6dc55b0 2024-09-10T21:19:46.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa126ffd640 10 client.18333 handle_command_reply: tid=0 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 client.18333 shutdown 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 msgr2=0x55b0b6db2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6db2b70 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098b0 tx=0x7fa13c004730 comp rx=0 tx=0).stop 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 shutdown_connections 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6dc55b0 0x55b0b6dc79f0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 0x55b0b6db3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6db2b70 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/3122090189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6db5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 >> 172.21.15.33:0/3122090189 conn(0x55b0b6da72b0 msgr2=0x55b0b6dace50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:46.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa126ffd640 0 client.18333 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:46.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.173+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 shutdown_connections 2024-09-10T21:19:46.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.174+0000 7fa14ff2b740 1 -- 172.21.15.33:0/3122090189 wait complete. 2024-09-10T21:19:46.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.174+0000 7fa14ff2b740 20 client.18333 trim_cache size 0 max 16384 2024-09-10T21:19:46.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 Processor -- start 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 -- start start 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9480 0x55b0b6da6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9da0 0x55b0b6dc1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481ac130 con 0x55b0b6db5d30 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481add90 con 0x55b0b6da9da0 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481ac800 con 0x55b0b6da9480 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa147fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60664/0 (socket says 172.21.15.33:60664) 2024-09-10T21:19:46.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.202+0000 7fa147fff640 1 -- 172.21.15.33:0/1963833461 learned_addr learned my addr 172.21.15.33:0/1963833461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa146ffd640 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9da0 0x55b0b6dc1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa147fff640 1 -- 172.21.15.33:0/1963833461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9480 msgr2=0x55b0b6da6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa147fff640 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9480 0x55b0b6da6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa147fff640 1 -- 172.21.15.33:0/1963833461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9da0 msgr2=0x55b0b6dc1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa147fff640 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9da0 0x55b0b6dc1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa147fff640 1 -- 172.21.15.33:0/1963833461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0b6db3e40 con 0x55b0b6db5d30 2024-09-10T21:19:46.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa147fff640 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db7120 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa140005ab0 tx=0x7fa140005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4139247b983107b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa12d7fa640 1 -- 172.21.15.33:0/1963833461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400d4740 con 0x55b0b6db5d30 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa12d7fa640 1 -- 172.21.15.33:0/1963833461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1400d5330 con 0x55b0b6db5d30 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.203+0000 7fa12d7fa640 1 -- 172.21.15.33:0/1963833461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1400d1020 con 0x55b0b6db5d30 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.204+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1963833461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 msgr2=0x55b0b6db7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.204+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db7120 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fa140005ab0 tx=0x7fa140005c00 comp rx=0 tx=0).stop 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.204+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1963833461 shutdown_connections 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.204+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6db5d30 0x55b0b6db7120 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.204+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9da0 0x55b0b6dc1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.204+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/1963833461 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6da9480 0x55b0b6da6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.204+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1963833461 >> 172.21.15.33:0/1963833461 conn(0x55b0b6da72b0 msgr2=0x55b0b6dc0950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.205+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1963833461 shutdown_connections 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.205+0000 7fa14ff2b740 1 -- 172.21.15.33:0/1963833461 wait complete. 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.205+0000 7fa14ff2b740 1 -- start start 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 1 Processor -- start 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6db0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6dbd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 0x55b0b6dbdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481ad690 con 0x55b0b6da9da0 2024-09-10T21:19:46.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481acf30 con 0x55b0b6da9480 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1481aa110 con 0x55b0b6db5d30 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa14ff2b740 10 client.0 _ll_register_callbacks cb 0x7fa13c009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6dbd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa146ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6dbd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60676/0 (socket says 172.21.15.33:60676) 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.206+0000 7fa146ffd640 1 -- 172.21.15.33:0/335538420 learned_addr learned my addr 172.21.15.33:0/335538420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.207+0000 7fa146ffd640 1 -- 172.21.15.33:0/335538420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 msgr2=0x55b0b6dbdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.207+0000 7fa146ffd640 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 0x55b0b6dbdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.207+0000 7fa146ffd640 1 -- 172.21.15.33:0/335538420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 msgr2=0x55b0b6db0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.207+0000 7fa146ffd640 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6db0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.207+0000 7fa146ffd640 1 -- 172.21.15.33:0/335538420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0b6daed90 con 0x55b0b6da9da0 2024-09-10T21:19:46.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.207+0000 7fa146ffd640 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6dbd9c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa13c010430 tx=0x7fa13c0109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df2de4e0ff7e592 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:19:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.207+0000 7fa126ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:19:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.208+0000 7fa126ffd640 1 -- 172.21.15.33:0/335538420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa13c038030 con 0x55b0b6da9da0 2024-09-10T21:19:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.208+0000 7fa126ffd640 1 -- 172.21.15.33:0/335538420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa13c010d50 con 0x55b0b6da9da0 2024-09-10T21:19:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.208+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1400d5330 con 0x55b0b6da9da0 2024-09-10T21:19:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.208+0000 7fa126ffd640 1 -- 172.21.15.33:0/335538420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa13c02d040 con 0x55b0b6da9da0 2024-09-10T21:19:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.208+0000 7fa14ff2b740 10 client.18339 fetch_fsmap learned FSMap version 74 2024-09-10T21:19:46.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.208+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55b0b6e336a0 con 0x55b0b6da9da0 2024-09-10T21:19:46.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa126ffd640 1 -- 172.21.15.33:0/335538420 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa13c024030 con 0x55b0b6da9da0 2024-09-10T21:19:46.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa126ffd640 1 -- 172.21.15.33:0/335538420 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 74) ==== 8402+0+0 (secure 0 0 0) 0x7fa13c045050 con 0x55b0b6da9da0 2024-09-10T21:19:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa14ff2b740 10 client.18339 fetch_fsmap finished waiting for FSMap version 74 2024-09-10T21:19:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa14ff2b740 10 client.18339 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:19:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa14ff2b740 20 client.18339 populate_metadata read hostname 'smithi033' 2024-09-10T21:19:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6e33df0 0x55b0b6e36210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:19:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa14ff2b740 4 client.18339 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:19:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa1400d5330 con 0x55b0b6e33df0 2024-09-10T21:19:46.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.209+0000 7fa1477fe640 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6e33df0 0x55b0b6e36210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:19:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa1477fe640 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6e33df0 0x55b0b6e36210 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef5ea98aa222a94 server_cookie=8346fe6946b7978d in_seq=0 out_seq=0 2024-09-10T21:19:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa126ffd640 10 client.18339 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa126ffd640 1 -- 172.21.15.33:0/335538420 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6250 (crc 0 0 0) 0x7fa1400d5330 con 0x55b0b6e33df0 2024-09-10T21:19:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa126ffd640 10 client.18339 handle_command_reply: tid=0 2024-09-10T21:19:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa14ff2b740 1 client.18339 shutdown 2024-09-10T21:19:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 msgr2=0x55b0b6dbd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6dbd9c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fa13c010430 tx=0x7fa13c0109d0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 shutdown_connections 2024-09-10T21:19:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.210+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b0b6e33df0 0x55b0b6e36210 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa126ffd640 0 client.18339 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:19:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0b6db5d30 0x55b0b6dbdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b0b6da9da0 0x55b0b6dbd9c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14ff2b740 1 --2- 172.21.15.33:0/335538420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b0b6da9480 0x55b0b6db0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 >> 172.21.15.33:0/335538420 conn(0x55b0b6da72b0 msgr2=0x55b0b6dc1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 shutdown_connections 2024-09-10T21:19:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14ff2b740 1 -- 172.21.15.33:0/335538420 wait complete. 2024-09-10T21:19:46.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14ff2b740 20 client.18339 trim_cache size 0 max 16384 2024-09-10T21:19:46.223 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:19:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:19:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:19:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:19:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "6023db06-103a-4538-877a-f434e14bd948", 2024-09-10T21:19:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:19:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:19:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:19:46.224 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:19:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:19:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.225 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:19:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:19:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.226 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.227 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:19:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:19:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.228 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:19:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:19:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.229 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.230 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:19:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:19:46.231 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:19:46.232 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:19:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.233 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:19:46.234 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.235 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:19:46.236 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:19:46.237 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:19:46.238 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.239 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.240 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.241 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:19:46.242 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:19:46.243 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1300b5430 msgr2=0x7fa1300b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14db56640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1300b5430 0x7fa1300b78f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa14810d8b0 tx=0x7fa13c0138e0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 msgr2=0x7fa1481a0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.211+0000 7fa14db56640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa1481a0d70 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fa14002a930 tx=0x7fa140003b70 comp rx=0 tx=0).stop 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 shutdown_connections 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa1300b5430 0x7fa1300b78f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa14810d100 0x7fa1481a0d70 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1481045c0 0x7fa14819b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 --2- 172.21.15.33:0/1656611833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa148103ca0 0x7fa14819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:19:46.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 >> 172.21.15.33:0/1656611833 conn(0x7fa1480fd700 msgr2=0x7fa1480ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:19:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 shutdown_connections 2024-09-10T21:19:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:19:46.212+0000 7fa14db56640 1 -- 172.21.15.33:0/1656611833 wait complete. 2024-09-10T21:19:47.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 139900 41.93 MB/sec execute 47 sec latency 213.386 ms 2024-09-10T21:19:47.721 INFO:tasks.cephfs.filesystem:scrub status for tag:6023db06-103a-4538-877a-f434e14bd948 - {'path': '/', 'tag': '6023db06-103a-4538-877a-f434e14bd948', 'options': 'recursive,force'} 2024-09-10T21:19:48.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 140682 41.99 MB/sec execute 48 sec latency 263.186 ms 2024-09-10T21:19:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:47 smithi104 ceph-mon[35325]: pgmap v820: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-09-10T21:19:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:47 smithi174 ceph-mon[35635]: pgmap v820: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-09-10T21:19:48.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:47 smithi033 ceph-mon[30860]: pgmap v820: 97 pgs: 97 active+clean; 1.0 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 859 op/s 2024-09-10T21:19:49.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 141583 42.34 MB/sec execute 49 sec latency 61.718 ms 2024-09-10T21:19:49.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:48 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:49.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:48 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:49.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:48 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:50.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 142289 42.17 MB/sec execute 50 sec latency 76.069 ms 2024-09-10T21:19:50.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:49 smithi104 ceph-mon[35325]: pgmap v821: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 962 op/s 2024-09-10T21:19:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:49 smithi174 ceph-mon[35635]: pgmap v821: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 962 op/s 2024-09-10T21:19:50.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:49 smithi033 ceph-mon[30860]: pgmap v821: 97 pgs: 97 active+clean; 1.0 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 962 op/s 2024-09-10T21:19:51.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 143191 42.19 MB/sec execute 51 sec latency 96.905 ms 2024-09-10T21:19:51.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:50 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:51.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:50 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:51.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:50 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:52.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 144155 42.52 MB/sec execute 52 sec latency 117.634 ms 2024-09-10T21:19:52.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:51 smithi104 ceph-mon[35325]: pgmap v822: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 874 op/s 2024-09-10T21:19:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:51 smithi174 ceph-mon[35635]: pgmap v822: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 874 op/s 2024-09-10T21:19:52.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:51 smithi033 ceph-mon[30860]: pgmap v822: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 874 op/s 2024-09-10T21:19:53.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 144596 42.18 MB/sec execute 53 sec latency 459.143 ms 2024-09-10T21:19:53.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:52 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:52 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:53.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:52 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:19:54.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 145370 42.02 MB/sec execute 54 sec latency 62.254 ms 2024-09-10T21:19:54.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:53 smithi104 ceph-mon[35325]: pgmap v823: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 850 op/s 2024-09-10T21:19:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:53 smithi174 ceph-mon[35635]: pgmap v823: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 850 op/s 2024-09-10T21:19:54.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:53 smithi033 ceph-mon[30860]: pgmap v823: 97 pgs: 97 active+clean; 1.0 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 850 op/s 2024-09-10T21:19:55.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 146405 42.27 MB/sec execute 55 sec latency 62.981 ms 2024-09-10T21:19:56.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 147376 42.68 MB/sec execute 56 sec latency 91.620 ms 2024-09-10T21:19:56.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:55 smithi104 ceph-mon[35325]: pgmap v824: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:19:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:55 smithi174 ceph-mon[35635]: pgmap v824: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:19:56.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:55 smithi033 ceph-mon[30860]: pgmap v824: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:19:57.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 148014 42.46 MB/sec execute 57 sec latency 169.846 ms 2024-09-10T21:19:57.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:57.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:57.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:57.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:57.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:57.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:57.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:57.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:58.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 148561 42.12 MB/sec execute 58 sec latency 384.581 ms 2024-09-10T21:19:58.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:57 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:19:58.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:57 smithi104 ceph-mon[35325]: pgmap v825: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:19:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:57 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:19:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:57 smithi174 ceph-mon[35635]: pgmap v825: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:19:58.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:57 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:19:58.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:57 smithi033 ceph-mon[30860]: pgmap v825: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:19:59.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 149565 42.21 MB/sec execute 59 sec latency 102.061 ms 2024-09-10T21:20:00.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 150582 42.65 MB/sec execute 60 sec latency 142.635 ms 2024-09-10T21:20:00.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:19:59 smithi104 ceph-mon[35325]: pgmap v826: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-10T21:20:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:19:59 smithi174 ceph-mon[35635]: pgmap v826: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-10T21:20:00.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:19:59 smithi033 ceph-mon[30860]: pgmap v826: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 948 op/s 2024-09-10T21:20:01.045 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:00 smithi174 ceph-mon[35635]: overall HEALTH_OK 2024-09-10T21:20:01.055 INFO:tasks.workunit.client.0.smithi033.stdout: 10 151439 42.65 MB/sec execute 61 sec latency 65.701 ms 2024-09-10T21:20:01.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:00 smithi104 ceph-mon[35325]: overall HEALTH_OK 2024-09-10T21:20:01.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:00 smithi033 ceph-mon[30860]: overall HEALTH_OK 2024-09-10T21:20:02.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 152222 42.64 MB/sec execute 62 sec latency 99.348 ms 2024-09-10T21:20:02.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:01 smithi104 ceph-mon[35325]: pgmap v827: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 935 op/s 2024-09-10T21:20:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:01 smithi174 ceph-mon[35635]: pgmap v827: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 935 op/s 2024-09-10T21:20:02.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:01 smithi033 ceph-mon[30860]: pgmap v827: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 935 op/s 2024-09-10T21:20:03.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 152701 42.40 MB/sec execute 63 sec latency 396.586 ms 2024-09-10T21:20:04.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 153539 42.24 MB/sec execute 64 sec latency 65.106 ms 2024-09-10T21:20:04.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:03 smithi104 ceph-mon[35325]: pgmap v828: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 869 op/s 2024-09-10T21:20:04.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:03 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:20:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:03 smithi174 ceph-mon[35635]: pgmap v828: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 869 op/s 2024-09-10T21:20:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:03 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:20:04.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:03 smithi033 ceph-mon[30860]: pgmap v828: 97 pgs: 97 active+clean; 1.0 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 869 op/s 2024-09-10T21:20:04.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:03 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:20:05.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 154718 42.68 MB/sec execute 65 sec latency 58.799 ms 2024-09-10T21:20:06.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 155623 42.93 MB/sec execute 66 sec latency 54.885 ms 2024-09-10T21:20:06.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:05 smithi104 ceph-mon[35325]: pgmap v829: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 978 op/s 2024-09-10T21:20:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:05 smithi174 ceph-mon[35635]: pgmap v829: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 978 op/s 2024-09-10T21:20:06.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:05 smithi033 ceph-mon[30860]: pgmap v829: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 978 op/s 2024-09-10T21:20:07.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 156257 42.77 MB/sec execute 67 sec latency 214.543 ms 2024-09-10T21:20:08.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 156810 42.45 MB/sec execute 68 sec latency 318.128 ms 2024-09-10T21:20:08.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:07 smithi104 ceph-mon[35325]: pgmap v830: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 817 op/s 2024-09-10T21:20:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:07 smithi174 ceph-mon[35635]: pgmap v830: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 817 op/s 2024-09-10T21:20:08.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:07 smithi033 ceph-mon[30860]: pgmap v830: 97 pgs: 97 active+clean; 1.0 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 817 op/s 2024-09-10T21:20:09.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 157685 42.59 MB/sec execute 69 sec latency 187.961 ms 2024-09-10T21:20:10.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 158512 42.61 MB/sec execute 70 sec latency 264.023 ms 2024-09-10T21:20:10.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:09 smithi104 ceph-mon[35325]: pgmap v831: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 965 op/s 2024-09-10T21:20:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:09 smithi174 ceph-mon[35635]: pgmap v831: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 965 op/s 2024-09-10T21:20:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:09 smithi033 ceph-mon[30860]: pgmap v831: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 965 op/s 2024-09-10T21:20:11.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 159463 42.60 MB/sec execute 71 sec latency 54.629 ms 2024-09-10T21:20:12.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 160668 43.08 MB/sec execute 72 sec latency 55.160 ms 2024-09-10T21:20:12.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:11 smithi104 ceph-mon[35325]: pgmap v832: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-10T21:20:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:11 smithi174 ceph-mon[35635]: pgmap v832: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-10T21:20:12.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:11 smithi033 ceph-mon[30860]: pgmap v832: 97 pgs: 97 active+clean; 1.1 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 936 op/s 2024-09-10T21:20:13.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 161227 42.90 MB/sec execute 73 sec latency 300.586 ms 2024-09-10T21:20:14.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 161955 42.83 MB/sec execute 74 sec latency 77.748 ms 2024-09-10T21:20:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:13 smithi174 ceph-mon[35635]: pgmap v833: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 848 op/s 2024-09-10T21:20:14.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:13 smithi033 ceph-mon[30860]: pgmap v833: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 848 op/s 2024-09-10T21:20:14.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:13 smithi104 ceph-mon[35325]: pgmap v833: 97 pgs: 97 active+clean; 1.1 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 848 op/s 2024-09-10T21:20:15.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 162918 42.82 MB/sec execute 75 sec latency 45.714 ms 2024-09-10T21:20:16.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 164042 43.20 MB/sec execute 76 sec latency 59.435 ms 2024-09-10T21:20:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:15 smithi174 ceph-mon[35635]: pgmap v834: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 938 op/s 2024-09-10T21:20:16.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:15 smithi033 ceph-mon[30860]: pgmap v834: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 938 op/s 2024-09-10T21:20:16.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:15 smithi104 ceph-mon[35325]: pgmap v834: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 938 op/s 2024-09-10T21:20:17.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 164856 43.17 MB/sec execute 77 sec latency 71.606 ms 2024-09-10T21:20:17.726 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:20:18.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 165444 43.02 MB/sec execute 78 sec latency 332.705 ms 2024-09-10T21:20:18.080 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:17 smithi174 ceph-mon[35635]: pgmap v835: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 841 op/s 2024-09-10T21:20:18.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:17 smithi033 ceph-mon[30860]: pgmap v835: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 841 op/s 2024-09-10T21:20:18.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:17 smithi104 ceph-mon[35325]: pgmap v835: 97 pgs: 97 active+clean; 1.1 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 841 op/s 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.793+0000 7f566f518640 1 -- 172.21.15.33:0/2717450301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 msgr2=0x7f566811a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.793+0000 7f566f518640 1 --2- 172.21.15.33:0/2717450301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 0x7f566811a1f0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a480 comp rx=0 tx=0).stop 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.793+0000 7f566f518640 1 -- 172.21.15.33:0/2717450301 shutdown_connections 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.793+0000 7f566f518640 1 --2- 172.21.15.33:0/2717450301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 0x7f566811a1f0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.793+0000 7f566f518640 1 --2- 172.21.15.33:0/2717450301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f566810aa70 0x7f56681178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.793+0000 7f566f518640 1 --2- 172.21.15.33:0/2717450301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566810a150 0x7f566810a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.793+0000 7f566f518640 1 -- 172.21.15.33:0/2717450301 >> 172.21.15.33:0/2717450301 conn(0x7f566806e2e0 msgr2=0x7f566806e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:18.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.794+0000 7f566f518640 1 -- 172.21.15.33:0/2717450301 shutdown_connections 2024-09-10T21:20:18.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.795+0000 7f566f518640 1 -- 172.21.15.33:0/2717450301 wait complete. 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 Processor -- start 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 -- start start 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f566810a150 0x7f5668117570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566810aa70 0x7f56681147c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 0x7f5668114d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f566811d7a0 con 0x7f5668117e00 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f566811d620 con 0x7f566810a150 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.796+0000 7f566f518640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f566811d920 con 0x7f566810aa70 2024-09-10T21:20:18.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f566810a150 0x7f5668117570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f566810a150 0x7f5668117570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:60238/0 (socket says 172.21.15.33:60238) 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566d2b4640 1 -- 172.21.15.33:0/410711504 learned_addr learned my addr 172.21.15.33:0/410711504 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566cab3640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566810aa70 0x7f56681147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566dab5640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 0x7f5668114d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566cab3640 1 -- 172.21.15.33:0/410711504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f566810a150 msgr2=0x7f5668117570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566cab3640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f566810a150 0x7f5668117570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566cab3640 1 -- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 msgr2=0x7f5668114d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566cab3640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 0x7f5668114d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.797+0000 7f566cab3640 1 -- 172.21.15.33:0/410711504 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5668115500 con 0x7f566810aa70 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.798+0000 7f566cab3640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566810aa70 0x7f56681147c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f565800d6d0 tx=0x7f565800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.798+0000 7f565e7fc640 1 -- 172.21.15.33:0/410711504 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658017070 con 0x7f566810aa70 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.798+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56681a2cd0 con 0x7f566810aa70 2024-09-10T21:20:18.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.798+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56681a3260 con 0x7f566810aa70 2024-09-10T21:20:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.798+0000 7f565e7fc640 1 -- 172.21.15.33:0/410711504 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5658009820 con 0x7f566810aa70 2024-09-10T21:20:18.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.798+0000 7f565e7fc640 1 -- 172.21.15.33:0/410711504 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658020060 con 0x7f566810aa70 2024-09-10T21:20:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.802+0000 7f56718ed740 1 Processor -- start 2024-09-10T21:20:18.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.802+0000 7f56718ed740 1 -- start start 2024-09-10T21:20:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.802+0000 7f56718ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e17480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f56718ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e23410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f56718ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 0x55d711e25d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f56718ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d711d8de50 con 0x55d711e179c0 2024-09-10T21:20:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f56718ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d711d7d280 con 0x55d711e23950 2024-09-10T21:20:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f56718ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d711d81090 con 0x55d711e170a0 2024-09-10T21:20:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e23410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e23410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52222/0 (socket says 172.21.15.33:52222) 2024-09-10T21:20:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f566d2b4640 1 -- 172.21.15.33:0/777942590 learned_addr learned my addr 172.21.15.33:0/777942590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:18.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.803+0000 7f566dab5640 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e17480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.804+0000 7f566d2b4640 1 -- 172.21.15.33:0/777942590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 msgr2=0x55d711e17480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:18.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.804+0000 7f566d2b4640 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e17480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.804+0000 7f566d2b4640 1 -- 172.21.15.33:0/777942590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 msgr2=0x55d711e25d40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:20:18.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.804+0000 7f566d2b4640 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 0x55d711e25d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.804+0000 7f566d2b4640 1 -- 172.21.15.33:0/777942590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d711adb650 con 0x55d711e179c0 2024-09-10T21:20:18.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.804+0000 7f565e7fc640 1 -- 172.21.15.33:0/410711504 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5658004b60 con 0x7f566810aa70 2024-09-10T21:20:18.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.804+0000 7f566d2b4640 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e23410 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5660011490 tx=0x7f5660011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607ae31fc5c070d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.805+0000 7f566dab5640 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e17480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.805+0000 7f56537fe640 1 -- 172.21.15.33:0/777942590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660012770 con 0x55d711e179c0 2024-09-10T21:20:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.805+0000 7f56537fe640 1 -- 172.21.15.33:0/777942590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5660012dc0 con 0x55d711e179c0 2024-09-10T21:20:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.805+0000 7f56537fe640 1 -- 172.21.15.33:0/777942590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56600193a0 con 0x55d711e179c0 2024-09-10T21:20:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.805+0000 7f565e7fc640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56540b53e0 0x7f56540b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.805+0000 7f565e7fc640 1 -- 172.21.15.33:0/410711504 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f56580e5330 con 0x7f566810aa70 2024-09-10T21:20:18.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f566dab5640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56540b53e0 0x7f56540b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f56718ed740 1 -- 172.21.15.33:0/777942590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 msgr2=0x55d711e23410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f56718ed740 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e23410 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5660011490 tx=0x7f5660011950 comp rx=0 tx=0).stop 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f56718ed740 1 -- 172.21.15.33:0/777942590 shutdown_connections 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f56718ed740 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 0x55d711e25d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f56718ed740 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e23410 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f56718ed740 1 --2- 172.21.15.33:0/777942590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e17480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.806+0000 7f56718ed740 1 -- 172.21.15.33:0/777942590 >> 172.21.15.33:0/777942590 conn(0x55d711e14ed0 msgr2=0x55d711e16070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.807+0000 7f56718ed740 1 -- 172.21.15.33:0/777942590 shutdown_connections 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.807+0000 7f56718ed740 1 -- 172.21.15.33:0/777942590 wait complete. 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.807+0000 7f566dab5640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56540b53e0 0x7f56540b78a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:18.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.807+0000 7f56718ed740 1 -- start start 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.807+0000 7f56718ed740 1 Processor -- start 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f56718ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e23210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f56718ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e20660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f56718ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 0x55d711e20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f56718ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d711bde700 con 0x55d711e179c0 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f56718ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d711d18110 con 0x55d711e23950 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f56718ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d711e265d0 con 0x55d711e170a0 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f56718ed740 10 client.0 _ll_register_callbacks cb 0x55d711d31990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e23210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e23210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47530/0 (socket says 172.21.15.33:47530) 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f566d2b4640 1 -- 172.21.15.33:0/3887236204 learned_addr learned my addr 172.21.15.33:0/3887236204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f566dab5640 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 0x55d711e20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.808+0000 7f566cab3640 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e20660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.809+0000 7f566cab3640 1 -- 172.21.15.33:0/3887236204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 msgr2=0x55d711e23210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.809+0000 7f566cab3640 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e23210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.809+0000 7f566cab3640 1 -- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 msgr2=0x55d711e20c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.809+0000 7f566cab3640 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 0x55d711e20c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.809+0000 7f566cab3640 1 -- 172.21.15.33:0/3887236204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d711d45c40 con 0x55d711e179c0 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.809+0000 7f566cab3640 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e20660 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f56580ab7b0 tx=0x7f56580ab9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c872f18cc479d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.810+0000 7f5650ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.810+0000 7f5650ff9640 1 -- 172.21.15.33:0/3887236204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56580ab5c0 con 0x55d711e179c0 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.810+0000 7f5650ff9640 1 -- 172.21.15.33:0/3887236204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f56580b0170 con 0x55d711e179c0 2024-09-10T21:20:18.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.810+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5660012dc0 con 0x55d711e179c0 2024-09-10T21:20:18.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.810+0000 7f5650ff9640 1 -- 172.21.15.33:0/3887236204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f56580b0370 con 0x55d711e179c0 2024-09-10T21:20:18.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.811+0000 7f56718ed740 10 client.18351 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:18.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.811+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55d711e31fc0 con 0x55d711e179c0 2024-09-10T21:20:18.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f5650ff9640 1 -- 172.21.15.33:0/3887236204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5660012dc0 con 0x55d711e179c0 2024-09-10T21:20:18.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f5650ff9640 1 -- 172.21.15.33:0/3887236204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f56580aa780 con 0x55d711e179c0 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f56718ed740 10 client.18351 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f56718ed740 10 client.18351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f56718ed740 20 client.18351 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f56718ed740 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711e339c0 0x55d711e35de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f56718ed740 4 client.18351 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.812+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d711d8fda0 con 0x55d711e339c0 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.813+0000 7f566d2b4640 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711e339c0 0x55d711e35de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.814+0000 7f566d2b4640 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711e339c0 0x55d711e35de0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bad67489bfb48ae server_cookie=74a1f722184995b5 in_seq=0 out_seq=0 2024-09-10T21:20:18.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.814+0000 7f5650ff9640 10 client.18351 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:18.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.816+0000 7f5650ff9640 1 -- 172.21.15.33:0/3887236204 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d711d8fda0 con 0x55d711e339c0 2024-09-10T21:20:18.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.816+0000 7f5650ff9640 10 client.18351 handle_command_reply: tid=0 2024-09-10T21:20:18.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.816+0000 7f56718ed740 1 client.18351 shutdown 2024-09-10T21:20:18.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 msgr2=0x55d711e20660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e20660 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f56580ab7b0 tx=0x7f56580ab9b0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 shutdown_connections 2024-09-10T21:20:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711e339c0 0x55d711e35de0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e23950 0x55d711e20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e20660 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 --2- 172.21.15.33:0/3887236204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e170a0 0x55d711e23210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 >> 172.21.15.33:0/3887236204 conn(0x55d711e14ed0 msgr2=0x55d711e1aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:18.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f5650ff9640 0 client.18351 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 shutdown_connections 2024-09-10T21:20:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.817+0000 7f56718ed740 1 -- 172.21.15.33:0/3887236204 wait complete. 2024-09-10T21:20:18.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.818+0000 7f56718ed740 20 client.18351 trim_cache size 0 max 16384 2024-09-10T21:20:18.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.844+0000 7f56718ed740 1 Processor -- start 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f56718ed740 1 -- start start 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f56718ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e170a0 0x55d711e0fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f56718ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e179c0 0x55d711e15d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f56718ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e162b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f56718ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f564c00e4f0 con 0x55d711e170a0 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f56718ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f564c010030 con 0x55d711e179c0 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f56718ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f564c00ebc0 con 0x55d711e23950 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f566dab5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f566dab5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47540/0 (socket says 172.21.15.33:47540) 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f566dab5640 1 -- 172.21.15.33:0/668961703 learned_addr learned my addr 172.21.15.33:0/668961703 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:18.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.845+0000 7f566d2b4640 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e170a0 0x55d711e0fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.846+0000 7f566dab5640 1 -- 172.21.15.33:0/668961703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e179c0 msgr2=0x55d711e15d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:20:18.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.846+0000 7f566dab5640 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e179c0 0x55d711e15d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.846+0000 7f566dab5640 1 -- 172.21.15.33:0/668961703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e170a0 msgr2=0x55d711e0fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:18.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.846+0000 7f566dab5640 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e170a0 0x55d711e0fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.846+0000 7f566dab5640 1 -- 172.21.15.33:0/668961703 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d711e10570 con 0x55d711e23950 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.847+0000 7f566dab5640 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e162b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566403d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb2be46810965a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.847+0000 7f5652ffd640 1 -- 172.21.15.33:0/668961703 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664035050 con 0x55d711e23950 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.847+0000 7f5652ffd640 1 -- 172.21.15.33:0/668961703 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566404d070 con 0x55d711e23950 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.847+0000 7f5652ffd640 1 -- 172.21.15.33:0/668961703 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664052470 con 0x55d711e23950 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.848+0000 7f56718ed740 1 -- 172.21.15.33:0/668961703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 msgr2=0x55d711e162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.848+0000 7f56718ed740 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e162b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566403d000 comp rx=0 tx=0).stop 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.848+0000 7f56718ed740 1 -- 172.21.15.33:0/668961703 shutdown_connections 2024-09-10T21:20:18.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.848+0000 7f56718ed740 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e162b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.849+0000 7f56718ed740 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e179c0 0x55d711e15d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.849+0000 7f56718ed740 1 --2- 172.21.15.33:0/668961703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e170a0 0x55d711e0fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.849+0000 7f56718ed740 1 -- 172.21.15.33:0/668961703 >> 172.21.15.33:0/668961703 conn(0x55d711e14ed0 msgr2=0x55d711e1b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:18.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.849+0000 7f56718ed740 1 -- 172.21.15.33:0/668961703 shutdown_connections 2024-09-10T21:20:18.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.849+0000 7f56718ed740 1 -- 172.21.15.33:0/668961703 wait complete. 2024-09-10T21:20:18.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.849+0000 7f56718ed740 1 -- start start 2024-09-10T21:20:18.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.849+0000 7f56718ed740 1 Processor -- start 2024-09-10T21:20:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f56718ed740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e170a0 0x55d711e2bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f56718ed740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e2c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f56718ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e1a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f56718ed740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f564c00f960 con 0x55d711e179c0 2024-09-10T21:20:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f56718ed740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f564c00f290 con 0x55d711e170a0 2024-09-10T21:20:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f56718ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f564c00c4b0 con 0x55d711e23950 2024-09-10T21:20:18.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f56718ed740 10 client.0 _ll_register_callbacks cb 0x7f56580057f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e170a0 0x55d711e2bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e170a0 0x55d711e2bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:60274/0 (socket says 172.21.15.33:60274) 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 -- 172.21.15.33:0/3431294892 learned_addr learned my addr 172.21.15.33:0/3431294892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566cab3640 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e2c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 -- 172.21.15.33:0/3431294892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 msgr2=0x55d711e1a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e1a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 -- 172.21.15.33:0/3431294892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 msgr2=0x55d711e2c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e2c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.850+0000 7f566d2b4640 1 -- 172.21.15.33:0/3431294892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d711e177f0 con 0x55d711e170a0 2024-09-10T21:20:18.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.851+0000 7f566d2b4640 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e170a0 0x55d711e2bd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5660011490 tx=0x7f566001a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abbf1f4172404acb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:18.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.851+0000 7f5650ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:18.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.851+0000 7f5650ff9640 1 -- 172.21.15.33:0/3431294892 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660003b70 con 0x55d711e170a0 2024-09-10T21:20:18.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.851+0000 7f5650ff9640 1 -- 172.21.15.33:0/3431294892 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5660003d10 con 0x55d711e170a0 2024-09-10T21:20:18.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.852+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566404d070 con 0x55d711e170a0 2024-09-10T21:20:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.853+0000 7f5650ff9640 1 -- 172.21.15.33:0/3431294892 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f5660012570 con 0x55d711e170a0 2024-09-10T21:20:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.853+0000 7f56718ed740 10 client.27761 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.853+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55d711e2a450 con 0x55d711e170a0 2024-09-10T21:20:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f5650ff9640 1 -- 172.21.15.33:0/3431294892 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5660012bc0 con 0x55d711e170a0 2024-09-10T21:20:18.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f5650ff9640 1 -- 172.21.15.33:0/3431294892 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f566000f040 con 0x55d711e170a0 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f56718ed740 10 client.27761 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f56718ed740 10 client.27761 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f56718ed740 20 client.27761 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f56718ed740 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711ea08c0 0x55d711ea2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f56718ed740 4 client.27761 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.854+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f566404d070 con 0x55d711ea08c0 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.855+0000 7f566cab3640 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711ea08c0 0x55d711ea2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.856+0000 7f566cab3640 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711ea08c0 0x55d711ea2cb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e47d7d6e44e8c3 server_cookie=e28f5a5458acd2cf in_seq=0 out_seq=0 2024-09-10T21:20:18.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.856+0000 7f5650ff9640 10 client.27761 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.858+0000 7f5650ff9640 1 -- 172.21.15.33:0/3431294892 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6054 (crc 0 0 0) 0x7f566404d070 con 0x55d711ea08c0 2024-09-10T21:20:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.858+0000 7f5650ff9640 10 client.27761 handle_command_reply: tid=0 2024-09-10T21:20:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.858+0000 7f56718ed740 1 client.27761 shutdown 2024-09-10T21:20:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.858+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e170a0 msgr2=0x55d711e2bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.858+0000 7f56718ed740 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e170a0 0x55d711e2bd60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5660011490 tx=0x7f566001a620 comp rx=0 tx=0).stop 2024-09-10T21:20:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 shutdown_connections 2024-09-10T21:20:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d711ea08c0 0x55d711ea2cb0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d711e23950 0x55d711e1a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f5650ff9640 0 client.27761 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d711e179c0 0x55d711e2c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 --2- 172.21.15.33:0/3431294892 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d711e170a0 0x55d711e2bd60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 >> 172.21.15.33:0/3431294892 conn(0x55d711e14ed0 msgr2=0x55d711e15bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 shutdown_connections 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.860+0000 7f56718ed740 1 -- 172.21.15.33:0/3431294892 wait complete. 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.861+0000 7f56718ed740 20 client.27761 trim_cache size 0 max 16384 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:20:18.866 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.867 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:20:18.869 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.870 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.871 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.872 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.873 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:20:18.874 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:20:18.876 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:20:18.878 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:20:18.880 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:20:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.862+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56540b53e0 msgr2=0x7f56540b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.862+0000 7f566f518640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56540b53e0 0x7f56540b78a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f5664006010 tx=0x7f5664005b50 comp rx=0 tx=0).stop 2024-09-10T21:20:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.862+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566810aa70 msgr2=0x7f56681147c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.862+0000 7f566f518640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566810aa70 0x7f56681147c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f565800d6d0 tx=0x7f565800db90 comp rx=0 tx=0).stop 2024-09-10T21:20:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.864+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 shutdown_connections 2024-09-10T21:20:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.864+0000 7f566f518640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56540b53e0 0x7f56540b78a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.864+0000 7f566f518640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668117e00 0x7f5668114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.864+0000 7f566f518640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566810aa70 0x7f56681147c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.864+0000 7f566f518640 1 --2- 172.21.15.33:0/410711504 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f566810a150 0x7f5668117570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.864+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 >> 172.21.15.33:0/410711504 conn(0x7f566806e2e0 msgr2=0x7f56681181e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.868+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 shutdown_connections 2024-09-10T21:20:18.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:18.868+0000 7f566f518640 1 -- 172.21.15.33:0/410711504 wait complete. 2024-09-10T21:20:19.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 166173 42.97 MB/sec execute 79 sec latency 85.299 ms 2024-09-10T21:20:19.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:19 smithi104 ceph-mon[35325]: pgmap v836: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-10T21:20:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:19 smithi174 ceph-mon[35635]: pgmap v836: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-10T21:20:19.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:19 smithi033 ceph-mon[30860]: pgmap v836: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2024-09-10T21:20:20.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 167001 43.04 MB/sec execute 80 sec latency 76.681 ms 2024-09-10T21:20:20.129 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:20:20.129 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:20:20.447 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:21.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 -- 172.21.15.33:0/2664866692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1810d0e0 msgr2=0x7f6c1810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:21.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/2664866692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1810d0e0 0x7f6c1810f550 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009ca0 tx=0x7f6c1002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 -- 172.21.15.33:0/2664866692 shutdown_connections 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/2664866692 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c1810d0e0 0x7f6c1810f550 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/2664866692 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c18104530 0x7f6c1810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/2664866692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c18103b30 0x7f6c18103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 -- 172.21.15.33:0/2664866692 >> 172.21.15.33:0/2664866692 conn(0x7f6c180fd700 msgr2=0x7f6c180ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 -- 172.21.15.33:0/2664866692 shutdown_connections 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.988+0000 7f6c1cb26640 1 -- 172.21.15.33:0/2664866692 wait complete. 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 Processor -- start 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 -- start start 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c18103b30 0x7f6c181019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c18104530 0x7f6c18101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1810d0e0 0x7f6c18102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:21.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18112db0 con 0x7f6c18104530 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18112c30 con 0x7f6c18103b30 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1cb26640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c18112f30 con 0x7f6c1810d0e0 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c15d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c18104530 0x7f6c18101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c15d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c18104530 0x7f6c18101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57254/0 (socket says 172.21.15.33:57254) 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c15d9b640 1 -- 172.21.15.33:0/222938510 learned_addr learned my addr 172.21.15.33:0/222938510 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.989+0000 7f6c1659c640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c18103b30 0x7f6c181019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c16d9d640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1810d0e0 0x7f6c18102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c15d9b640 1 -- 172.21.15.33:0/222938510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1810d0e0 msgr2=0x7f6c18102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c15d9b640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1810d0e0 0x7f6c18102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c15d9b640 1 -- 172.21.15.33:0/222938510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c18103b30 msgr2=0x7f6c181019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:21.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c15d9b640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c18103b30 0x7f6c181019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c15d9b640 1 -- 172.21.15.33:0/222938510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c181a7620 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c15d9b640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c18104530 0x7f6c18101ef0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00e9f0 tx=0x7f6c0c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c077fe640 1 -- 172.21.15.33:0/222938510 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c00cd80 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c181a7910 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c077fe640 1 -- 172.21.15.33:0/222938510 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c0c0108e0 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c077fe640 1 -- 172.21.15.33:0/222938510 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c018980 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.990+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c181a7ea0 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.992+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c18103f10 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.993+0000 7f6c077fe640 1 -- 172.21.15.33:0/222938510 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6c0c010430 con 0x7f6c18104530 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.994+0000 7f6c077fe640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6c000b5360 0x7f6c000b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.995+0000 7f6c1659c640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6c000b5360 0x7f6c000b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:21.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.995+0000 7f6c077fe640 1 -- 172.21.15.33:0/222938510 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6c0c0e5f00 con 0x7f6c18104530 2024-09-10T21:20:21.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.995+0000 7f6c1659c640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6c000b5360 0x7f6c000b7820 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005d00 tx=0x7f6c0800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:21.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:20.999+0000 7f6c077fe640 1 -- 172.21.15.33:0/222938510 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6c0c0acd80 con 0x7f6c18104530 2024-09-10T21:20:21.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 167730 42.95 MB/sec execute 81 sec latency 80.829 ms 2024-09-10T21:20:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.273+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c181a8230 con 0x7f6c18104530 2024-09-10T21:20:21.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.274+0000 7f6c077fe640 1 -- 172.21.15.33:0/222938510 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+11980 (secure 0 0 0) 0x7f6c0c0ac680 con 0x7f6c18104530 2024-09-10T21:20:21.278 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:20:21.278 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":78,"btime":"2024-09-10T21:20:02:787680+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":78,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:20:02.209617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:20:21.278 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 78 2024-09-10T21:20:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.276+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6c000b5360 msgr2=0x7f6c000b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.276+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6c000b5360 0x7f6c000b7820 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f6c08005d00 tx=0x7f6c0800b410 comp rx=0 tx=0).stop 2024-09-10T21:20:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.276+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c18104530 msgr2=0x7f6c18101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.276+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c18104530 0x7f6c18101ef0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00e9f0 tx=0x7f6c0c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.276+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 shutdown_connections 2024-09-10T21:20:21.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.277+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6c000b5360 0x7f6c000b7820 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.277+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c1810d0e0 0x7f6c18102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.277+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c18104530 0x7f6c18101ef0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.277+0000 7f6c1cb26640 1 --2- 172.21.15.33:0/222938510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6c18103b30 0x7f6c181019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.277+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 >> 172.21.15.33:0/222938510 conn(0x7f6c180fd700 msgr2=0x7f6c180fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.277+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 shutdown_connections 2024-09-10T21:20:21.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:21.277+0000 7f6c1cb26640 1 -- 172.21.15.33:0/222938510 wait complete. 2024-09-10T21:20:21.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:21 smithi033 ceph-mon[30860]: pgmap v837: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 894 op/s 2024-09-10T21:20:21.610 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:21 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/222938510' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:20:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:21 smithi104 ceph-mon[35325]: pgmap v837: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 894 op/s 2024-09-10T21:20:21.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:21 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/222938510' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:20:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:21 smithi174 ceph-mon[35635]: pgmap v837: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 894 op/s 2024-09-10T21:20:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:21 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/222938510' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:20:22.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 168518 42.92 MB/sec execute 82 sec latency 87.790 ms 2024-09-10T21:20:22.080 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:20:22.383 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:22.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 -- 172.21.15.33:0/951590076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3054109150 msgr2=0x7f305410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 --2- 172.21.15.33:0/951590076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3054109150 0x7f305410f950 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f304c009ca0 tx=0x7f304c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 -- 172.21.15.33:0/951590076 shutdown_connections 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 --2- 172.21.15.33:0/951590076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3054109150 0x7f305410f950 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 --2- 172.21.15.33:0/951590076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30541087d0 0x7f3054108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 --2- 172.21.15.33:0/951590076 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054102dc0 0x7f30541031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 -- 172.21.15.33:0/951590076 >> 172.21.15.33:0/951590076 conn(0x7f30540fe830 msgr2=0x7f3054100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 -- 172.21.15.33:0/951590076 shutdown_connections 2024-09-10T21:20:22.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.935+0000 7f305afa4640 1 -- 172.21.15.33:0/951590076 wait complete. 2024-09-10T21:20:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 Processor -- start 2024-09-10T21:20:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 -- start start 2024-09-10T21:20:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3054102dc0 0x7f30540754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30541087d0 0x7f30540759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054109150 0x7f3054075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3054112e20 con 0x7f30541087d0 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3054112ca0 con 0x7f3054109150 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f305afa4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3054112fa0 con 0x7f3054102dc0 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f3059541640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054109150 0x7f3054075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f3059541640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054109150 0x7f3054075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50944/0 (socket says 172.21.15.33:50944) 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f3053fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30541087d0 0x7f30540759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f3059541640 1 -- 172.21.15.33:0/2519273751 learned_addr learned my addr 172.21.15.33:0/2519273751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.936+0000 7f3058d40640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3054102dc0 0x7f30540754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3059541640 1 -- 172.21.15.33:0/2519273751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3054102dc0 msgr2=0x7f30540754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3059541640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3054102dc0 0x7f30540754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3059541640 1 -- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30541087d0 msgr2=0x7f30540759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3059541640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30541087d0 0x7f30540759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3059541640 1 -- 172.21.15.33:0/2519273751 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30541a7670 con 0x7f3054109150 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3059541640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054109150 0x7f3054075f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f304c009430 tx=0x7f304c0319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3051ffb640 1 -- 172.21.15.33:0/2519273751 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c003c30 con 0x7f3054109150 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3051ffb640 1 -- 172.21.15.33:0/2519273751 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f304c003dd0 con 0x7f3054109150 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30541a7900 con 0x7f3054109150 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.937+0000 7f3051ffb640 1 -- 172.21.15.33:0/2519273751 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c02faa0 con 0x7f3054109150 2024-09-10T21:20:22.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.938+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30541a7e90 con 0x7f3054109150 2024-09-10T21:20:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.940+0000 7f305d379740 1 Processor -- start 2024-09-10T21:20:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.940+0000 7f3051ffb640 1 -- 172.21.15.33:0/2519273751 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f304c02fca0 con 0x7f3054109150 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f305d379740 1 -- start start 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f305d379740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f305d379740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c94840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f305d379740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f305d379740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653e8bfeea0 con 0x5653e8c884d0 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f305d379740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5653e8bee2d0 con 0x5653e8c94d80 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f305d379740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653e8bf20e0 con 0x5653e8c88df0 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f3053fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f3053fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c888b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57282/0 (socket says 172.21.15.33:57282) 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f3053fff640 1 -- 172.21.15.33:0/1883686921 learned_addr learned my addr 172.21.15.33:0/1883686921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f3058d40640 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c94840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.941+0000 7f3053fff640 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3053fff640 1 -- 172.21.15.33:0/1883686921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 msgr2=0x5653e8c94840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3053fff640 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c94840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3053fff640 1 -- 172.21.15.33:0/1883686921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 msgr2=0x5653e8c888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3053fff640 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c888b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3053fff640 1 -- 172.21.15.33:0/1883686921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e894c650 con 0x5653e8c94d80 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3051ffb640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f30440b5360 0x7f30440b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3053fff640 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3040012720 tx=0x7f304000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1364f2f7c383793c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3058d40640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f30440b5360 0x7f30440b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f303effd640 1 -- 172.21.15.33:0/1883686921 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30400096a0 con 0x5653e8c94d80 2024-09-10T21:20:22.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f3051ffb640 1 -- 172.21.15.33:0/2519273751 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f304c106f00 con 0x7f3054109150 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f303effd640 1 -- 172.21.15.33:0/1883686921 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3040009cf0 con 0x5653e8c94d80 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.942+0000 7f303effd640 1 -- 172.21.15.33:0/1883686921 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040010c30 con 0x5653e8c94d80 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 -- 172.21.15.33:0/1883686921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 msgr2=0x5653e8c97170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3040012720 tx=0x7f304000b390 comp rx=0 tx=0).stop 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f3058d40640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f30440b5360 0x7f30440b7820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x5653e8c871e0 tx=0x7f304800d580 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 -- 172.21.15.33:0/1883686921 shutdown_connections 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97170 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c94840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 --2- 172.21.15.33:0/1883686921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 -- 172.21.15.33:0/1883686921 >> 172.21.15.33:0/1883686921 conn(0x5653e8c86300 msgr2=0x5653e8c874a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 -- 172.21.15.33:0/1883686921 shutdown_connections 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 -- 172.21.15.33:0/1883686921 wait complete. 2024-09-10T21:20:22.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.943+0000 7f305d379740 1 -- start start 2024-09-10T21:20:22.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.944+0000 7f305d379740 1 Processor -- start 2024-09-10T21:20:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.944+0000 7f305d379740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c884d0 0x5653e8c94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.944+0000 7f305d379740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c91ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.944+0000 7f305d379740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c94d80 0x5653e8c920e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f305d379740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653e8a4f700 con 0x5653e8c94d80 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f305d379740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5653e8b89160 con 0x5653e8c88df0 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f305d379740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653e8c97a00 con 0x5653e8c884d0 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3058d40640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c884d0 0x5653e8c94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3053fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c91ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3058d40640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c884d0 0x5653e8c94680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36182/0 (socket says 172.21.15.33:36182) 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3053fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c91ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50962/0 (socket says 172.21.15.33:50962) 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3058d40640 1 -- 172.21.15.33:0/645795329 learned_addr learned my addr 172.21.15.33:0/645795329 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f305d379740 10 client.0 _ll_register_callbacks cb 0x5653e8ba29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3059541640 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c94d80 0x5653e8c920e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3053fff640 1 -- 172.21.15.33:0/645795329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c884d0 msgr2=0x5653e8c94680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3053fff640 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c884d0 0x5653e8c94680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3053fff640 1 -- 172.21.15.33:0/645795329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c94d80 msgr2=0x5653e8c920e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3053fff640 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c94d80 0x5653e8c920e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.945+0000 7f3053fff640 1 -- 172.21.15.33:0/645795329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653e8bb6c90 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f3053fff640 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c91ad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3040012720 tx=0x7f3040009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41d78e29d60f9ea1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f3033fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f3033fff640 1 -- 172.21.15.33:0/645795329 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3040016070 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f3033fff640 1 -- 172.21.15.33:0/645795329 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3040012030 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3040009cf0 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f3033fff640 1 -- 172.21.15.33:0/645795329 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f304002e050 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f305d379740 10 client.27779 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5653e8ca3360 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.946+0000 7f3033fff640 1 -- 172.21.15.33:0/645795329 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3040009cf0 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f3033fff640 1 -- 172.21.15.33:0/645795329 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f3040026020 con 0x5653e8c88df0 2024-09-10T21:20:22.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f305d379740 10 client.27779 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f305d379740 10 client.27779 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:20:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f305d379740 20 client.27779 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f305d379740 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8ca6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f305d379740 4 client.27779 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653e8c00df0 con 0x5653e8ca46b0 2024-09-10T21:20:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.947+0000 7f3058d40640 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8ca6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.949+0000 7f3058d40640 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8ca6aa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3d69b764d99596a server_cookie=aa1c63a6b335bd4f in_seq=0 out_seq=0 2024-09-10T21:20:22.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.949+0000 7f3033fff640 10 client.27779 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.953+0000 7f3033fff640 1 -- 172.21.15.33:0/645795329 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5653e8c00df0 con 0x5653e8ca46b0 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.953+0000 7f3033fff640 10 client.27779 handle_command_reply: tid=0 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.953+0000 7f305d379740 1 client.27779 shutdown 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 msgr2=0x5653e8c91ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c91ad0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3040012720 tx=0x7f3040009f10 comp rx=0 tx=0).stop 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 shutdown_connections 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8ca6aa0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c94d80 0x5653e8c920e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f3033fff640 0 client.27779 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c91ad0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 --2- 172.21.15.33:0/645795329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c884d0 0x5653e8c94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 >> 172.21.15.33:0/645795329 conn(0x5653e8c86300 msgr2=0x5653e8c8be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:22.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 shutdown_connections 2024-09-10T21:20:22.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 1 -- 172.21.15.33:0/645795329 wait complete. 2024-09-10T21:20:22.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.954+0000 7f305d379740 20 client.27779 trim_cache size 0 max 16384 2024-09-10T21:20:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 Processor -- start 2024-09-10T21:20:22.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 -- start start 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8ca0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c8bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c94d80 0x5653e8c8c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30541b05f0 con 0x5653e8c884d0 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30541b2130 con 0x5653e8c88df0 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f305d379740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30541b0cc0 con 0x5653e8c94d80 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f3058d40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8ca0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f3058d40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8ca0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57300/0 (socket says 172.21.15.33:57300) 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.970+0000 7f3058d40640 1 -- 172.21.15.33:0/1083562954 learned_addr learned my addr 172.21.15.33:0/1083562954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3053fff640 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c8bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3059541640 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c94d80 0x5653e8c8c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3058d40640 1 -- 172.21.15.33:0/1083562954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c94d80 msgr2=0x5653e8c8c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3058d40640 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c94d80 0x5653e8c8c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3058d40640 1 -- 172.21.15.33:0/1083562954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 msgr2=0x5653e8c8bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3058d40640 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c8bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3058d40640 1 -- 172.21.15.33:0/1083562954 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e8c86d80 con 0x5653e8c884d0 2024-09-10T21:20:22.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.971+0000 7f3058d40640 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8ca0c60 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3048005fd0 tx=0x7f3048015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24f5c21a71990de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.972+0000 7f303e7fc640 1 -- 172.21.15.33:0/1083562954 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480042f0 con 0x5653e8c884d0 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.972+0000 7f303e7fc640 1 -- 172.21.15.33:0/1083562954 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3048004490 con 0x5653e8c884d0 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.972+0000 7f303e7fc640 1 -- 172.21.15.33:0/1083562954 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30480299c0 con 0x5653e8c884d0 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 -- 172.21.15.33:0/1083562954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 msgr2=0x5653e8ca0c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8ca0c60 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f3048005fd0 tx=0x7f3048015000 comp rx=0 tx=0).stop 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 -- 172.21.15.33:0/1083562954 shutdown_connections 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c94d80 0x5653e8c8c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c88df0 0x5653e8c8bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 --2- 172.21.15.33:0/1083562954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8ca0c60 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 -- 172.21.15.33:0/1083562954 >> 172.21.15.33:0/1083562954 conn(0x5653e8c86300 msgr2=0x5653e8c8b530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 -- 172.21.15.33:0/1083562954 shutdown_connections 2024-09-10T21:20:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.973+0000 7f305d379740 1 -- 172.21.15.33:0/1083562954 wait complete. 2024-09-10T21:20:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 -- start start 2024-09-10T21:20:22.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 Processor -- start 2024-09-10T21:20:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c9d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c9d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f30541b1a60 con 0x5653e8c884d0 2024-09-10T21:20:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f30541b1390 con 0x5653e8c94d80 2024-09-10T21:20:22.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30541ae5b0 con 0x5653e8c88df0 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f305d379740 10 client.0 _ll_register_callbacks cb 0x7f304000ba60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f3058d40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c9d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f3058d40640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c9d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57302/0 (socket says 172.21.15.33:57302) 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.974+0000 7f3058d40640 1 -- 172.21.15.33:0/3669955069 learned_addr learned my addr 172.21.15.33:0/3669955069 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3059541640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3053fff640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c9d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3058d40640 1 -- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 msgr2=0x5653e8c9d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3058d40640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c9d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3058d40640 1 -- 172.21.15.33:0/3669955069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 msgr2=0x5653e8c97eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3058d40640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3058d40640 1 -- 172.21.15.33:0/3669955069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653e8c986e0 con 0x5653e8c884d0 2024-09-10T21:20:22.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3053fff640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c9d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3059541640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3058d40640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c9d180 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4718a82e2b3176e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3033fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3033fff640 1 -- 172.21.15.33:0/3669955069 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048029640 con 0x5653e8c884d0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3048004490 con 0x5653e8c884d0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.975+0000 7f3033fff640 1 -- 172.21.15.33:0/3669955069 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f30480297e0 con 0x5653e8c884d0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f3033fff640 1 -- 172.21.15.33:0/3669955069 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f30480299e0 con 0x5653e8c884d0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 10 client.18381 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5653e8d122d0 con 0x5653e8c884d0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f3033fff640 1 -- 172.21.15.33:0/3669955069 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3048004490 con 0x5653e8c884d0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f3033fff640 1 -- 172.21.15.33:0/3669955069 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f304803d420 con 0x5653e8c884d0 2024-09-10T21:20:22.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 10 client.18381 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 10 client.18381 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 20 client.18381 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8d14950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 4 client.18381 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.976+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653e8bfc8a0 con 0x5653e8ca46b0 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.978+0000 7f3053fff640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8d14950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.978+0000 7f3053fff640 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8d14950 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4e7cab347bdedad server_cookie=b471ee8ef604e89e in_seq=0 out_seq=0 2024-09-10T21:20:22.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.978+0000 7f3033fff640 10 client.18381 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.980+0000 7f3033fff640 1 -- 172.21.15.33:0/3669955069 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653e8bfc8a0 con 0x5653e8ca46b0 2024-09-10T21:20:22.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.980+0000 7f3033fff640 10 client.18381 handle_command_reply: tid=0 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.980+0000 7f305d379740 1 client.18381 shutdown 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 msgr2=0x5653e8c9d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c9d180 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f3048000c00 tx=0x7f3048005970 comp rx=0 tx=0).stop 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 shutdown_connections 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5653e8ca46b0 0x5653e8d14950 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f3033fff640 0 client.18381 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653e8c94d80 0x5653e8c97eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653e8c88df0 0x5653e8c9d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 --2- 172.21.15.33:0/3669955069 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653e8c884d0 0x5653e8c9d180 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 >> 172.21.15.33:0/3669955069 conn(0x5653e8c86300 msgr2=0x5653e8c8bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 shutdown_connections 2024-09-10T21:20:22.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 1 -- 172.21.15.33:0/3669955069 wait complete. 2024-09-10T21:20:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.981+0000 7f305d379740 20 client.18381 trim_cache size 0 max 16384 2024-09-10T21:20:22.986 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:20:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f30440b5360 msgr2=0x7f30440b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f30440b5360 0x7f30440b7820 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x5653e8c871e0 tx=0x7f304800d580 comp rx=0 tx=0).stop 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054109150 msgr2=0x7f3054075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054109150 0x7f3054075f20 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f304c009430 tx=0x7f304c0319f0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 shutdown_connections 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f30440b5360 0x7f30440b7820 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3054109150 0x7f3054075f20 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30541087d0 0x7f30540759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 --2- 172.21.15.33:0/2519273751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3054102dc0 0x7f30540754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 >> 172.21.15.33:0/2519273751 conn(0x7f30540fe830 msgr2=0x7f30540ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 shutdown_connections 2024-09-10T21:20:22.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:22.982+0000 7f305afa4640 1 -- 172.21.15.33:0/2519273751 wait complete. 2024-09-10T21:20:23.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 169064 42.73 MB/sec execute 83 sec latency 394.640 ms 2024-09-10T21:20:23.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:23 smithi033 ceph-mon[30860]: pgmap v838: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-10T21:20:23.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:23 smithi104 ceph-mon[35325]: pgmap v838: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-10T21:20:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:23 smithi174 ceph-mon[35635]: pgmap v838: 97 pgs: 97 active+clean; 1.1 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 905 op/s 2024-09-10T21:20:24.056 INFO:tasks.workunit.client.0.smithi033.stdout: 10 169828 42.81 MB/sec execute 84 sec latency 103.414 ms 2024-09-10T21:20:24.889 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:20:25.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 170563 42.76 MB/sec execute 85 sec latency 82.093 ms 2024-09-10T21:20:25.207 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:25.586 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:25 smithi033 ceph-mon[30860]: pgmap v839: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 954 op/s 2024-09-10T21:20:25.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:25 smithi104 ceph-mon[35325]: pgmap v839: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 954 op/s 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.853+0000 7f2516a05640 1 -- 172.21.15.33:0/3678150619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2510108020 msgr2=0x7f251010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.853+0000 7f2516a05640 1 --2- 172.21.15.33:0/3678150619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2510108020 0x7f251010e820 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f24f800d960 tx=0x7f24f802c2b0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.854+0000 7f2516a05640 1 -- 172.21.15.33:0/3678150619 shutdown_connections 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.854+0000 7f2516a05640 1 --2- 172.21.15.33:0/3678150619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2510108020 0x7f251010e820 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.854+0000 7f2516a05640 1 --2- 172.21.15.33:0/3678150619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25101076a0 0x7f2510107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.854+0000 7f2516a05640 1 --2- 172.21.15.33:0/3678150619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 0x7f2510102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.854+0000 7f2516a05640 1 -- 172.21.15.33:0/3678150619 >> 172.21.15.33:0/3678150619 conn(0x7f25100fd700 msgr2=0x7f25100ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:25.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.854+0000 7f2516a05640 1 -- 172.21.15.33:0/3678150619 shutdown_connections 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.854+0000 7f2516a05640 1 -- 172.21.15.33:0/3678150619 wait complete. 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 Processor -- start 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 -- start start 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 0x7f251019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25101076a0 0x7f251019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2510108020 0x7f25101a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2510110be0 con 0x7f2510101c90 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2510110a60 con 0x7f2510108020 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f2516a05640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2510110d60 con 0x7f25101076a0 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.855+0000 7f250ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 0x7f251019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25101076a0 0x7f251019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 0x7f251019b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57320/0 (socket says 172.21.15.33:57320) 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250ffff640 1 -- 172.21.15.33:0/2662229809 learned_addr learned my addr 172.21.15.33:0/2662229809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f2514fa2640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2510108020 0x7f25101a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250ffff640 1 -- 172.21.15.33:0/2662229809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25101076a0 msgr2=0x7f251019b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250ffff640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25101076a0 0x7f251019b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250ffff640 1 -- 172.21.15.33:0/2662229809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2510108020 msgr2=0x7f25101a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250ffff640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2510108020 0x7f25101a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.856+0000 7f250ffff640 1 -- 172.21.15.33:0/2662229809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25101a14b0 con 0x7f2510101c90 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.857+0000 7f250ffff640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 0x7f251019b400 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f250800e9c0 tx=0x7f250800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:25.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.857+0000 7f250d7fa640 1 -- 172.21.15.33:0/2662229809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250800cde0 con 0x7f2510101c90 2024-09-10T21:20:25.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.858+0000 7f250d7fa640 1 -- 172.21.15.33:0/2662229809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25080035f0 con 0x7f2510101c90 2024-09-10T21:20:25.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.858+0000 7f250d7fa640 1 -- 172.21.15.33:0/2662229809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508010690 con 0x7f2510101c90 2024-09-10T21:20:25.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.858+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25101a17a0 con 0x7f2510101c90 2024-09-10T21:20:25.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.858+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25101a8030 con 0x7f2510101c90 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 Processor -- start 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 -- start start 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad67b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad687840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 0x55faad68a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55faad5f1ea0 con 0x55faad67bdf0 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55faad5e12d0 con 0x55faad67b4d0 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2518dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55faad5e50e0 con 0x55faad687d80 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad67b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad67b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51012/0 (socket says 172.21.15.33:51012) 2024-09-10T21:20:25.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.862+0000 7f2514fa2640 1 -- 172.21.15.33:0/2607939160 learned_addr learned my addr 172.21.15.33:0/2607939160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.863+0000 7f2514fa2640 1 -- 172.21.15.33:0/2607939160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 msgr2=0x55faad68a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.863+0000 7f2514fa2640 1 --2- 172.21.15.33:0/2607939160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 0x55faad68a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.863+0000 7f2514fa2640 1 -- 172.21.15.33:0/2607939160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 msgr2=0x55faad687840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:20:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.863+0000 7f2514fa2640 1 --2- 172.21.15.33:0/2607939160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad687840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.863+0000 7f2514fa2640 1 -- 172.21.15.33:0/2607939160 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faad33f650 con 0x55faad67b4d0 2024-09-10T21:20:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.864+0000 7f2514fa2640 1 --2- 172.21.15.33:0/2607939160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad67b8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f24f8036000 tx=0x7f24f800d350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735518616214af3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.864+0000 7f250d7fa640 1 -- 172.21.15.33:0/2662229809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2508004000 con 0x7f2510101c90 2024-09-10T21:20:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.864+0000 7f25067fc640 1 -- 172.21.15.33:0/2607939160 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8039040 con 0x55faad67b4d0 2024-09-10T21:20:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.864+0000 7f25067fc640 1 -- 172.21.15.33:0/2607939160 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f8032900 con 0x55faad67b4d0 2024-09-10T21:20:25.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.864+0000 7f25067fc640 1 -- 172.21.15.33:0/2607939160 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8039040 con 0x55faad67b4d0 2024-09-10T21:20:25.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f250d7fa640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24fc0b5360 0x7f24fc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 -- 172.21.15.33:0/2607939160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 msgr2=0x55faad67b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 --2- 172.21.15.33:0/2607939160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad67b8b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f24f8036000 tx=0x7f24f800d350 comp rx=0 tx=0).stop 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 -- 172.21.15.33:0/2607939160 shutdown_connections 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 --2- 172.21.15.33:0/2607939160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 0x55faad68a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 --2- 172.21.15.33:0/2607939160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad687840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 --2- 172.21.15.33:0/2607939160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad67b8b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 -- 172.21.15.33:0/2607939160 >> 172.21.15.33:0/2607939160 conn(0x55faad679300 msgr2=0x55faad67a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f250ffff640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24fc0b5360 0x7f24fc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 -- 172.21.15.33:0/2607939160 shutdown_connections 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f2518dda740 1 -- 172.21.15.33:0/2607939160 wait complete. 2024-09-10T21:20:25.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.865+0000 7f250d7fa640 1 -- 172.21.15.33:0/2662229809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f25080e5fe0 con 0x7f2510101c90 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 -- start start 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f250ffff640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24fc0b5360 0x7f24fc0b7820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2508078960 tx=0x7f2508079c30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 Processor -- start 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67b4d0 0x55faad6875c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67bdf0 0x55faad684a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad684f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55faad442700 con 0x55faad687d80 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55faad57c160 con 0x55faad67bdf0 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55faad68a9a0 con 0x55faad67b4d0 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.866+0000 7f2518dda740 10 client.0 _ll_register_callbacks cb 0x55faad5959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67b4d0 0x55faad6875c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f250f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67bdf0 0x55faad684a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67b4d0 0x55faad6875c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36224/0 (socket says 172.21.15.33:36224) 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 -- 172.21.15.33:0/1614421439 learned_addr learned my addr 172.21.15.33:0/1614421439 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad684f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 -- 172.21.15.33:0/1614421439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67b4d0 msgr2=0x55faad6875c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67b4d0 0x55faad6875c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 -- 172.21.15.33:0/1614421439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67bdf0 msgr2=0x55faad684a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.867+0000 7f2514fa2640 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67bdf0 0x55faad684a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f2514fa2640 1 -- 172.21.15.33:0/1614421439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faad5a9c90 con 0x55faad687d80 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f250f7fe640 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67bdf0 0x55faad684a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f2514fa2640 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad684f70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f24f802c7d0 tx=0x7f24f802c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0408aca99d7496 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f24ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f24ef7fe640 1 -- 172.21.15.33:0/1614421439 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8051060 con 0x55faad687d80 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f8032900 con 0x55faad687d80 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f24ef7fe640 1 -- 172.21.15.33:0/1614421439 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f24f8034710 con 0x55faad687d80 2024-09-10T21:20:25.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f24ef7fe640 1 -- 172.21.15.33:0/1614421439 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f24f8039040 con 0x55faad687d80 2024-09-10T21:20:25.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f2518dda740 10 client.18390 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:25.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.868+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55faad696260 con 0x55faad687d80 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f24ef7fe640 1 -- 172.21.15.33:0/1614421439 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f8032900 con 0x55faad687d80 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f24ef7fe640 1 -- 172.21.15.33:0/1614421439 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f24f8056070 con 0x55faad687d80 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f2518dda740 10 client.18390 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f2518dda740 10 client.18390 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f2518dda740 20 client.18390 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f2518dda740 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad6975b0 0x55faad6999a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f2518dda740 4 client.18390 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faad5f3df0 con 0x55faad6975b0 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.869+0000 7f250f7fe640 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad6975b0 0x55faad6999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.870+0000 7f250f7fe640 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad6975b0 0x55faad6999a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f2fb7571e9cd1454 server_cookie=fdda8873b502a40f in_seq=0 out_seq=0 2024-09-10T21:20:25.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.870+0000 7f24ef7fe640 10 client.18390 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:20:25.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f24ef7fe640 1 -- 172.21.15.33:0/1614421439 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55faad5f3df0 con 0x55faad6975b0 2024-09-10T21:20:25.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f24ef7fe640 10 client.18390 handle_command_reply: tid=0 2024-09-10T21:20:25.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 client.18390 shutdown 2024-09-10T21:20:25.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 msgr2=0x55faad684f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad684f70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f24f802c7d0 tx=0x7f24f802c2b0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 shutdown_connections 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad684f70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67bdf0 0x55faad684a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad6975b0 0x55faad6999a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 --2- 172.21.15.33:0/1614421439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67b4d0 0x55faad6875c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 >> 172.21.15.33:0/1614421439 conn(0x55faad679300 msgr2=0x55faad67eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f24ef7fe640 0 client.18390 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 shutdown_connections 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.875+0000 7f2518dda740 1 -- 172.21.15.33:0/1614421439 wait complete. 2024-09-10T21:20:25.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.876+0000 7f2518dda740 20 client.18390 trim_cache size 0 max 16384 2024-09-10T21:20:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:25 smithi174 ceph-mon[35635]: pgmap v839: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 954 op/s 2024-09-10T21:20:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 Processor -- start 2024-09-10T21:20:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 -- start start 2024-09-10T21:20:25.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad686750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67bdf0 0x55faad681c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad6821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b0630 con 0x55faad687d80 2024-09-10T21:20:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b2170 con 0x55faad67b4d0 2024-09-10T21:20:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.890+0000 7f2518dda740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b0d00 con 0x55faad67bdf0 2024-09-10T21:20:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad686750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f250f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67bdf0 0x55faad681c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f250f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67bdf0 0x55faad681c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36234/0 (socket says 172.21.15.33:36234) 2024-09-10T21:20:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f250f7fe640 1 -- 172.21.15.33:0/3470908129 learned_addr learned my addr 172.21.15.33:0/3470908129 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad686750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51024/0 (socket says 172.21.15.33:51024) 2024-09-10T21:20:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad6821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 -- 172.21.15.33:0/3470908129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67bdf0 msgr2=0x55faad681c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67bdf0 0x55faad681c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 -- 172.21.15.33:0/3470908129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 msgr2=0x55faad6821a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad6821a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.891+0000 7f2514fa2640 1 -- 172.21.15.33:0/3470908129 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faad679fe0 con 0x55faad67b4d0 2024-09-10T21:20:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.892+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad686750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f24f8044350 tx=0x7f24f8005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19947241db6128f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.892+0000 7f2505ffb640 1 -- 172.21.15.33:0/3470908129 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f805c070 con 0x55faad67b4d0 2024-09-10T21:20:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.892+0000 7f2505ffb640 1 -- 172.21.15.33:0/3470908129 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f8032900 con 0x55faad67b4d0 2024-09-10T21:20:25.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.892+0000 7f2505ffb640 1 -- 172.21.15.33:0/3470908129 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8051070 con 0x55faad67b4d0 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 -- 172.21.15.33:0/3470908129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 msgr2=0x55faad686750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad686750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f24f8044350 tx=0x7f24f8005b20 comp rx=0 tx=0).stop 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 -- 172.21.15.33:0/3470908129 shutdown_connections 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad687d80 0x55faad6821a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad67bdf0 0x55faad681c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 --2- 172.21.15.33:0/3470908129 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad686750 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 -- 172.21.15.33:0/3470908129 >> 172.21.15.33:0/3470908129 conn(0x55faad679300 msgr2=0x55faad686040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 -- 172.21.15.33:0/3470908129 shutdown_connections 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.893+0000 7f2518dda740 1 -- 172.21.15.33:0/3470908129 wait complete. 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.894+0000 7f2518dda740 1 -- start start 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.894+0000 7f2518dda740 1 Processor -- start 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2518dda740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad68fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2518dda740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad690170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2518dda740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 0x55faad68cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad68fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2518dda740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b1aa0 con 0x55faad67bdf0 2024-09-10T21:20:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2518dda740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f25101b13d0 con 0x55faad67b4d0 2024-09-10T21:20:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2514fa2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad68fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51026/0 (socket says 172.21.15.33:51026) 2024-09-10T21:20:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f250f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad690170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2514fa2640 1 -- 172.21.15.33:0/3411864087 learned_addr learned my addr 172.21.15.33:0/3411864087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25101ae5f0 con 0x55faad687d80 2024-09-10T21:20:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2518dda740 10 client.0 _ll_register_callbacks cb 0x7f25000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.895+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 0x55faad68cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2514fa2640 1 -- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 msgr2=0x55faad68cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 0x55faad68cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2514fa2640 1 -- 172.21.15.33:0/3411864087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 msgr2=0x55faad690170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:25.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad690170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2514fa2640 1 -- 172.21.15.33:0/3411864087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faad6907b0 con 0x55faad67b4d0 2024-09-10T21:20:25.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f250f7fe640 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad690170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:25.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2514fa2640 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad68fc30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f24f8006010 tx=0x7f24f8048f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4da1318a6cf387c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:25.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f24ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:25.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f24ef7fe640 1 -- 172.21.15.33:0/3411864087 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f805c070 con 0x55faad67b4d0 2024-09-10T21:20:25.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f24ef7fe640 1 -- 172.21.15.33:0/3411864087 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f24f80475e0 con 0x55faad67b4d0 2024-09-10T21:20:25.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24f8032900 con 0x55faad67b4d0 2024-09-10T21:20:25.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f24ef7fe640 1 -- 172.21.15.33:0/3411864087 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f24f8051030 con 0x55faad67b4d0 2024-09-10T21:20:25.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2518dda740 10 client.27803 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:25.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55faad704620 con 0x55faad67b4d0 2024-09-10T21:20:25.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.896+0000 7f24ef7fe640 1 -- 172.21.15.33:0/3411864087 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24f8032900 con 0x55faad67b4d0 2024-09-10T21:20:25.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.897+0000 7f24ef7fe640 1 -- 172.21.15.33:0/3411864087 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f24f8056070 con 0x55faad67b4d0 2024-09-10T21:20:25.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.897+0000 7f2518dda740 10 client.27803 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:25.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.897+0000 7f2518dda740 10 client.27803 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:20:25.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.897+0000 7f2518dda740 20 client.27803 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:25.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.897+0000 7f2518dda740 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad704cc0 0x55faad7070e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:25.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.897+0000 7f2518dda740 4 client.27803 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:20:25.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.897+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faad5ef8a0 con 0x55faad704cc0 2024-09-10T21:20:25.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.901+0000 7f250f7fe640 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad704cc0 0x55faad7070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:25.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.903+0000 7f250f7fe640 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad704cc0 0x55faad7070e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cddc4cc33cf95d1 server_cookie=2099fc9438c36385 in_seq=0 out_seq=0 2024-09-10T21:20:25.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.903+0000 7f24ef7fe640 10 client.27803 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f24ef7fe640 1 -- 172.21.15.33:0/3411864087 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faad5ef8a0 con 0x55faad704cc0 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f24ef7fe640 10 client.27803 handle_command_reply: tid=0 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 client.27803 shutdown 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 msgr2=0x55faad68fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad68fc30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f24f8006010 tx=0x7f24f8048f90 comp rx=0 tx=0).stop 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 shutdown_connections 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55faad704cc0 0x55faad7070e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.911 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:20:25.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f24ef7fe640 0 client.27803 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:20:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55faad687d80 0x55faad68cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55faad67bdf0 0x55faad690170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 --2- 172.21.15.33:0/3411864087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55faad67b4d0 0x55faad68fc30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 >> 172.21.15.33:0/3411864087 conn(0x55faad679300 msgr2=0x55faad686860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:25.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 shutdown_connections 2024-09-10T21:20:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 1 -- 172.21.15.33:0/3411864087 wait complete. 2024-09-10T21:20:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.905+0000 7f2518dda740 20 client.27803 trim_cache size 0 max 16384 2024-09-10T21:20:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24fc0b5360 msgr2=0x7f24fc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24fc0b5360 0x7f24fc0b7820 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f2508078960 tx=0x7f2508079c30 comp rx=0 tx=0).stop 2024-09-10T21:20:25.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 msgr2=0x7f251019b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 0x7f251019b400 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f250800e9c0 tx=0x7f250800ee80 comp rx=0 tx=0).stop 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 shutdown_connections 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f24fc0b5360 0x7f24fc0b7820 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2510108020 0x7f25101a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25101076a0 0x7f251019b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 --2- 172.21.15.33:0/2662229809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2510101c90 0x7f251019b400 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 >> 172.21.15.33:0/2662229809 conn(0x7f25100fd700 msgr2=0x7f251010cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 shutdown_connections 2024-09-10T21:20:25.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:25.906+0000 7f2516a05640 1 -- 172.21.15.33:0/2662229809 wait complete. 2024-09-10T21:20:26.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 171306 42.65 MB/sec execute 86 sec latency 105.979 ms 2024-09-10T21:20:26.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:20:27.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 172189 42.71 MB/sec execute 87 sec latency 100.379 ms 2024-09-10T21:20:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:27.529 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:27 smithi033 ceph-mon[30860]: pgmap v840: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 801 op/s 2024-09-10T21:20:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/3285947087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e781045c0 msgr2=0x7f6e7810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/3285947087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e781045c0 0x7f6e7810cab0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009900 tx=0x7f6e6802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/3285947087 shutdown_connections 2024-09-10T21:20:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/3285947087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e7810d100 0x7f6e7810f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/3285947087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e781045c0 0x7f6e7810cab0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/3285947087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e78103ca0 0x7f6e78104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/3285947087 >> 172.21.15.33:0/3285947087 conn(0x7f6e780fd700 msgr2=0x7f6e780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:27.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.642+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/3285947087 shutdown_connections 2024-09-10T21:20:27.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.643+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/3285947087 wait complete. 2024-09-10T21:20:27.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.644+0000 7f6e7fe0c640 1 Processor -- start 2024-09-10T21:20:27.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.644+0000 7f6e7fe0c640 1 -- start start 2024-09-10T21:20:27.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7fe0c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78103ca0 0x7f6e781992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7fe0c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e781045c0 0x7f6e78199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7dba8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78103ca0 0x7f6e781992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7dba8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78103ca0 0x7f6e781992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57364/0 (socket says 172.21.15.33:57364) 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7dba8640 1 -- 172.21.15.33:0/2312426157 learned_addr learned my addr 172.21.15.33:0/2312426157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e7810d100 0x7f6e7819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78112d70 con 0x7f6e78103ca0 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78112bf0 con 0x7f6e7810d100 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e78112ef0 con 0x7f6e781045c0 2024-09-10T21:20:27.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e7810d100 0x7f6e7819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.645+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e781045c0 0x7f6e78199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.646+0000 7f6e7dba8640 1 -- 172.21.15.33:0/2312426157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e781045c0 msgr2=0x7f6e78199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.646+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e781045c0 0x7f6e78199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.646+0000 7f6e7dba8640 1 -- 172.21.15.33:0/2312426157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e7810d100 msgr2=0x7f6e7819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.646+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e7810d100 0x7f6e7819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.646+0000 7f6e7dba8640 1 -- 172.21.15.33:0/2312426157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7819f390 con 0x7f6e78103ca0 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.646+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78103ca0 0x7f6e781992e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9f0 tx=0x7f6e7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.647+0000 7f6e6effd640 1 -- 172.21.15.33:0/2312426157 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7400cd80 con 0x7f6e78103ca0 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.647+0000 7f6e6effd640 1 -- 172.21.15.33:0/2312426157 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e740108e0 con 0x7f6e78103ca0 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.647+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7819f680 con 0x7f6e78103ca0 2024-09-10T21:20:27.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.647+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e781a5d40 con 0x7f6e78103ca0 2024-09-10T21:20:27.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.648+0000 7f6e6effd640 1 -- 172.21.15.33:0/2312426157 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e74018860 con 0x7f6e78103ca0 2024-09-10T21:20:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.651+0000 7f6e821e1740 1 Processor -- start 2024-09-10T21:20:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.651+0000 7f6e6effd640 1 -- 172.21.15.33:0/2312426157 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6e74010430 con 0x7f6e78103ca0 2024-09-10T21:20:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e821e1740 1 -- start start 2024-09-10T21:20:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb5510830 0x557bb5506df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb551b430 0x557bb551d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e821e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557bb547cea0 con 0x557bb5510830 2024-09-10T21:20:27.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e821e1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557bb546c2d0 con 0x557bb55064d0 2024-09-10T21:20:27.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e821e1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bb54700e0 con 0x557bb551b430 2024-09-10T21:20:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e7e3a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e7e3a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51048/0 (socket says 172.21.15.33:51048) 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e7e3a9640 1 -- 172.21.15.33:0/1389574314 learned_addr learned my addr 172.21.15.33:0/1389574314 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.652+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb5510830 0x557bb5506df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb551b430 0x557bb551d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7e3a9640 1 -- 172.21.15.33:0/1389574314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb551b430 msgr2=0x557bb551d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e6effd640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6e640b5360 0x7f6e640b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb551b430 0x557bb551d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7e3a9640 1 -- 172.21.15.33:0/1389574314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb5510830 msgr2=0x557bb5506df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb5510830 0x557bb5506df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7e3a9640 1 -- 172.21.15.33:0/1389574314 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb51ca650 con 0x557bb55064d0 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb551b430 0x557bb551d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb5510830 0x557bb5506df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:27.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e6effd640 1 -- 172.21.15.33:0/2312426157 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6e740e5c00 con 0x7f6e78103ca0 2024-09-10T21:20:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55068b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009fd0 tx=0x7f6e700118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca0ab5dd622c4a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6e640b5360 0x7f6e640b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.653+0000 7f6e63fff640 1 -- 172.21.15.33:0/1389574314 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e7001e070 con 0x557bb55064d0 2024-09-10T21:20:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.654+0000 7f6e63fff640 1 -- 172.21.15.33:0/1389574314 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e70004600 con 0x557bb55064d0 2024-09-10T21:20:27.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.654+0000 7f6e63fff640 1 -- 172.21.15.33:0/1389574314 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70019430 con 0x557bb55064d0 2024-09-10T21:20:27.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6e640b5360 0x7f6e640b7820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6e7001b000 tx=0x7f6e70019b60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:27.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 -- 172.21.15.33:0/1389574314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 msgr2=0x557bb55068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55068b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009fd0 tx=0x7f6e700118d0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 -- 172.21.15.33:0/1389574314 shutdown_connections 2024-09-10T21:20:27.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb551b430 0x557bb551d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb5510830 0x557bb5506df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1389574314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55068b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 -- 172.21.15.33:0/1389574314 >> 172.21.15.33:0/1389574314 conn(0x557bb5504300 msgr2=0x557bb55054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 -- 172.21.15.33:0/1389574314 shutdown_connections 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 -- 172.21.15.33:0/1389574314 wait complete. 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 -- start start 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.655+0000 7f6e821e1740 1 Processor -- start 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55100d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb550d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb550dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e821e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557bb52cd700 con 0x557bb551b430 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e821e1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557bb5407160 con 0x557bb55064d0 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e821e1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557bb551e220 con 0x557bb5510830 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e821e1740 10 client.0 _ll_register_callbacks cb 0x557bb54209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:27.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51060/0 (socket says 172.21.15.33:51060) 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 -- 172.21.15.33:0/3147965498 learned_addr learned my addr 172.21.15.33:0/3147965498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb550d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb550dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 -- 172.21.15.33:0/3147965498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 msgr2=0x557bb550d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb550d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 -- 172.21.15.33:0/3147965498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 msgr2=0x557bb550dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb550dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.656+0000 7f6e7dba8640 1 -- 172.21.15.33:0/3147965498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb5434c90 con 0x557bb55064d0 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb550dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55100d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9c0 tx=0x7f6e74009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd1bb5951815163 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:27.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e617fa640 1 -- 172.21.15.33:0/3147965498 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e740a8310 con 0x557bb55064d0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e617fa640 1 -- 172.21.15.33:0/3147965498 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6e740a6b70 con 0x557bb55064d0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e70004600 con 0x557bb55064d0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e617fa640 1 -- 172.21.15.33:0/3147965498 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6e740abbe0 con 0x557bb55064d0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e821e1740 10 client.27815 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.657+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x557bb5529ba0 con 0x557bb55064d0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e617fa640 1 -- 172.21.15.33:0/3147965498 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e70004600 con 0x557bb55064d0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e617fa640 1 -- 172.21.15.33:0/3147965498 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f6e740cf0b0 con 0x557bb55064d0 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e821e1740 10 client.27815 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e821e1740 10 client.27815 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e821e1740 20 client.27815 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:27.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e821e1740 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb552b100 0x557bb552d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e821e1740 4 client.27815 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:27.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557bb547edf0 con 0x557bb552b100 2024-09-10T21:20:27.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.658+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb552b100 0x557bb552d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.659+0000 7f6e617fa640 10 client.27815 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:20:27.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.659+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb552b100 0x557bb552d510 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef27289a2b1c51f6 server_cookie=30c94c369c30a098 in_seq=0 out_seq=0 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.660+0000 7f6e617fa640 1 -- 172.21.15.33:0/3147965498 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557bb547edf0 con 0x557bb552b100 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.660+0000 7f6e617fa640 10 client.27815 handle_command_reply: tid=0 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.660+0000 7f6e821e1740 1 client.27815 shutdown 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 msgr2=0x557bb55100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55100d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9c0 tx=0x7f6e74009310 comp rx=0 tx=0).stop 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 shutdown_connections 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb552b100 0x557bb552d510 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb550dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb550d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 --2- 172.21.15.33:0/3147965498 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb55100d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 >> 172.21.15.33:0/3147965498 conn(0x557bb5504300 msgr2=0x557bb5502c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e617fa640 0 client.27815 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:20:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 shutdown_connections 2024-09-10T21:20:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 1 -- 172.21.15.33:0/3147965498 wait complete. 2024-09-10T21:20:27.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.661+0000 7f6e821e1740 20 client.27815 trim_cache size 0 max 16384 2024-09-10T21:20:27.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 Processor -- start 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 -- start start 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb5506fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb55274e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb5502c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c00e430 con 0x557bb551b430 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c00ff70 con 0x557bb55064d0 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e821e1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c00eb00 con 0x557bb5510830 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e7e3a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb5502c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e7dba8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb5506fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e7e3a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb5502c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57384/0 (socket says 172.21.15.33:57384) 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e7e3a9640 1 -- 172.21.15.33:0/1818435365 learned_addr learned my addr 172.21.15.33:0/1818435365 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.678+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb55274e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e7dba8640 1 -- 172.21.15.33:0/1818435365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 msgr2=0x557bb55274e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb55274e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e7dba8640 1 -- 172.21.15.33:0/1818435365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 msgr2=0x557bb5502c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb5502c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e7dba8640 1 -- 172.21.15.33:0/1818435365 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557bb551b810 con 0x557bb55064d0 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb5502c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb5506fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e740ac400 tx=0x7f6e740e3780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54a30bdd663523f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e637fe640 1 -- 172.21.15.33:0/1818435365 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e740abdc0 con 0x557bb55064d0 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e637fe640 1 -- 172.21.15.33:0/1818435365 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e740b5290 con 0x557bb55064d0 2024-09-10T21:20:27.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.679+0000 7f6e637fe640 1 -- 172.21.15.33:0/1818435365 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e740ac240 con 0x557bb55064d0 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 -- 172.21.15.33:0/1818435365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 msgr2=0x557bb5506fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb5506fc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6e740ac400 tx=0x7f6e740e3780 comp rx=0 tx=0).stop 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 -- 172.21.15.33:0/1818435365 shutdown_connections 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb5502c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb5510830 0x557bb55274e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 --2- 172.21.15.33:0/1818435365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb55064d0 0x557bb5506fc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 -- 172.21.15.33:0/1818435365 >> 172.21.15.33:0/1818435365 conn(0x557bb5504300 msgr2=0x557bb55068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:27.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 -- 172.21.15.33:0/1818435365 shutdown_connections 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 -- 172.21.15.33:0/1818435365 wait complete. 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.680+0000 7f6e821e1740 1 -- start start 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 1 Processor -- start 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb55064d0 0x557bb5523550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb5510830 0x557bb5523a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb551e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c00f8a0 con 0x557bb551b430 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c00f1d0 con 0x557bb5510830 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c00c3f0 con 0x557bb55064d0 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e821e1740 10 client.0 _ll_register_callbacks cb 0x7f6e7400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e7e3a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb551e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e7e3a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb551e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57386/0 (socket says 172.21.15.33:57386) 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e7e3a9640 1 -- 172.21.15.33:0/2851935424 learned_addr learned my addr 172.21.15.33:0/2851935424 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.681+0000 7f6e7d3a7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb5510830 0x557bb5523a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb55064d0 0x557bb5523550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7d3a7640 1 -- 172.21.15.33:0/2851935424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb55064d0 msgr2=0x557bb5523550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb55064d0 0x557bb5523550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7d3a7640 1 -- 172.21.15.33:0/2851935424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 msgr2=0x557bb551e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb551e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7d3a7640 1 -- 172.21.15.33:0/2851935424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557bb55242c0 con 0x557bb5510830 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7e3a9640 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb551e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e7d3a7640 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb5510830 0x557bb5523a90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6e68006010 tx=0x7f6e68009520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=604b08143ad43ccc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e617fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:27.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.682+0000 7f6e617fa640 1 -- 172.21.15.33:0/2851935424 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68035040 con 0x557bb5510830 2024-09-10T21:20:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.683+0000 7f6e617fa640 1 -- 172.21.15.33:0/2851935424 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6e68005040 con 0x557bb5510830 2024-09-10T21:20:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.683+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557bb547a8a0 con 0x557bb5510830 2024-09-10T21:20:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.683+0000 7f6e617fa640 1 -- 172.21.15.33:0/2851935424 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f6e68005240 con 0x557bb5510830 2024-09-10T21:20:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.683+0000 7f6e821e1740 10 client.27821 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:27.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.683+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x557bb5598640 con 0x557bb5510830 2024-09-10T21:20:27.690 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:20:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e617fa640 1 -- 172.21.15.33:0/2851935424 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557bb547a8a0 con 0x557bb5510830 2024-09-10T21:20:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e617fa640 1 -- 172.21.15.33:0/2851935424 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f6e680035f0 con 0x557bb5510830 2024-09-10T21:20:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e821e1740 10 client.27821 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e821e1740 10 client.27821 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:20:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e821e1740 20 client.27821 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:27.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e821e1740 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb5598df0 0x557bb559b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e821e1740 4 client.27821 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.684+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557bb5477cd0 con 0x557bb5598df0 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.685+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb5598df0 0x557bb559b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.685+0000 7f6e617fa640 10 client.27821 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.685+0000 7f6e7dba8640 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb5598df0 0x557bb559b210 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=56e776be90ad2e4e server_cookie=de8025ae6456a4a9 in_seq=0 out_seq=0 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e617fa640 1 -- 172.21.15.33:0/2851935424 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557bb5477cd0 con 0x557bb5598df0 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e617fa640 10 client.27821 handle_command_reply: tid=0 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 client.27821 shutdown 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb5510830 msgr2=0x557bb5523a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb5510830 0x557bb5523a90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6e68006010 tx=0x7f6e68009520 comp rx=0 tx=0).stop 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 shutdown_connections 2024-09-10T21:20:27.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x557bb5598df0 0x557bb559b210 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557bb551b430 0x557bb551e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557bb5510830 0x557bb5523a90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 --2- 172.21.15.33:0/2851935424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557bb55064d0 0x557bb5523550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 >> 172.21.15.33:0/2851935424 conn(0x557bb5504300 msgr2=0x557bb5502c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e617fa640 0 client.27821 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 shutdown_connections 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 1 -- 172.21.15.33:0/2851935424 wait complete. 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.686+0000 7f6e821e1740 20 client.27821 trim_cache size 0 max 16384 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.687+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6e640b5360 msgr2=0x7f6e640b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.687+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6e640b5360 0x7f6e640b7820 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f6e7001b000 tx=0x7f6e70019b60 comp rx=0 tx=0).stop 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.687+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78103ca0 msgr2=0x7f6e781992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.687+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78103ca0 0x7f6e781992e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6e7400e9f0 tx=0x7f6e7400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.687+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 shutdown_connections 2024-09-10T21:20:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.687+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6e640b5360 0x7f6e640b7820 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.688+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e7810d100 0x7f6e7819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.688+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e781045c0 0x7f6e78199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.688+0000 7f6e7fe0c640 1 --2- 172.21.15.33:0/2312426157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e78103ca0 0x7f6e781992e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.688+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 >> 172.21.15.33:0/2312426157 conn(0x7f6e780fd700 msgr2=0x7f6e780fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.688+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 shutdown_connections 2024-09-10T21:20:27.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:27.688+0000 7f6e7fe0c640 1 -- 172.21.15.33:0/2312426157 wait complete. 2024-09-10T21:20:27.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:27 smithi104 ceph-mon[35325]: pgmap v840: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 801 op/s 2024-09-10T21:20:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:27 smithi174 ceph-mon[35635]: pgmap v840: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 801 op/s 2024-09-10T21:20:28.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 172733 42.66 MB/sec execute 88 sec latency 352.359 ms 2024-09-10T21:20:29.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 173248 42.56 MB/sec execute 89 sec latency 506.295 ms 2024-09-10T21:20:29.536 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:20:29.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:29 smithi104 ceph-mon[35325]: pgmap v841: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 964 op/s 2024-09-10T21:20:29.873 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:29 smithi174 ceph-mon[35635]: pgmap v841: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 964 op/s 2024-09-10T21:20:29.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:29 smithi033 ceph-mon[30860]: pgmap v841: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 13 MiB/s wr, 964 op/s 2024-09-10T21:20:30.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 173885 42.27 MB/sec execute 90 sec latency 82.473 ms 2024-09-10T21:20:30.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.438+0000 7f0d478b2640 1 -- 172.21.15.33:0/3074414432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 msgr2=0x7f0d40116c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.438+0000 7f0d478b2640 1 --2- 172.21.15.33:0/3074414432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 0x7f0d40116c60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009be0 tx=0x7f0d3802a750 comp rx=0 tx=0).stop 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.439+0000 7f0d478b2640 1 -- 172.21.15.33:0/3074414432 shutdown_connections 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.439+0000 7f0d478b2640 1 --2- 172.21.15.33:0/3074414432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 0x7f0d40116c60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.439+0000 7f0d478b2640 1 --2- 172.21.15.33:0/3074414432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d4010bb80 0x7f0d401142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.439+0000 7f0d478b2640 1 --2- 172.21.15.33:0/3074414432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 0x7f0d4010b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.439+0000 7f0d478b2640 1 -- 172.21.15.33:0/3074414432 >> 172.21.15.33:0/3074414432 conn(0x7f0d4006e2e0 msgr2=0x7f0d4006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.439+0000 7f0d478b2640 1 -- 172.21.15.33:0/3074414432 shutdown_connections 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.439+0000 7f0d478b2640 1 -- 172.21.15.33:0/3074414432 wait complete. 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.440+0000 7f0d478b2640 1 Processor -- start 2024-09-10T21:20:30.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.440+0000 7f0d478b2640 1 -- start start 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.440+0000 7f0d478b2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 0x7f0d40119570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.440+0000 7f0d478b2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d4010bb80 0x7f0d40117ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.440+0000 7f0d478b2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 0x7f0d401180f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.440+0000 7f0d478b2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4011b590 con 0x7f0d4010b260 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d478b2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4011b410 con 0x7f0d40114810 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d478b2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4011b710 con 0x7f0d4010bb80 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d4564e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 0x7f0d40119570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d4564e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 0x7f0d40119570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42912/0 (socket says 172.21.15.33:42912) 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d4564e640 1 -- 172.21.15.33:0/713067900 learned_addr learned my addr 172.21.15.33:0/713067900 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:30.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 0x7f0d401180f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d4010bb80 0x7f0d40117ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d45e4f640 1 -- 172.21.15.33:0/713067900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d4010bb80 msgr2=0x7f0d40117ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d4010bb80 0x7f0d40117ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d45e4f640 1 -- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 msgr2=0x7f0d40119570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 0x7f0d40119570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d45e4f640 1 -- 172.21.15.33:0/713067900 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d401b43a0 con 0x7f0d40114810 2024-09-10T21:20:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d4564e640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 0x7f0d40119570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.441+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 0x7f0d401180f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d3802ac70 tx=0x7f0d38003c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.442+0000 7f0d367fc640 1 -- 172.21.15.33:0/713067900 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38041020 con 0x7f0d40114810 2024-09-10T21:20:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.442+0000 7f0d367fc640 1 -- 172.21.15.33:0/713067900 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d38030970 con 0x7f0d40114810 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.442+0000 7f0d367fc640 1 -- 172.21.15.33:0/713067900 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38036040 con 0x7f0d40114810 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.442+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d401b4570 con 0x7f0d40114810 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.442+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d401b4970 con 0x7f0d40114810 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.445+0000 7f0d367fc640 1 -- 172.21.15.33:0/713067900 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0d38004460 con 0x7f0d40114810 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.446+0000 7f0d49c87740 1 Processor -- start 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.446+0000 7f0d49c87740 1 -- start start 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.446+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299d4d0 0x5624d299d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.446+0000 7f0d367fc640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b52e0 0x7f0d2c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.446+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d299ddf0 0x5624d29a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d29a9d80 0x5624d29ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d49c87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5624d2913ea0 con 0x5624d299ddf0 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d49c87740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5624d29032d0 con 0x5624d299d4d0 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d49c87740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5624d29070e0 con 0x5624d29a9d80 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299d4d0 0x5624d299d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299d4d0 0x5624d299d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41758/0 (socket says 172.21.15.33:41758) 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d367fc640 1 -- 172.21.15.33:0/713067900 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0d38106aa0 con 0x7f0d40114810 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 -- 172.21.15.33:0/1202975369 learned_addr learned my addr 172.21.15.33:0/1202975369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d299ddf0 0x5624d29a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d4564e640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b52e0 0x7f0d2c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 -- 172.21.15.33:0/1202975369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d29a9d80 msgr2=0x5624d29ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d29a9d80 0x5624d29ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 -- 172.21.15.33:0/1202975369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d299ddf0 msgr2=0x5624d29a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d299ddf0 0x5624d29a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.447+0000 7f0d44e4d640 1 -- 172.21.15.33:0/1202975369 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d2661650 con 0x5624d299d4d0 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.448+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299d4d0 0x5624d299d8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000bfd0 tx=0x7f0d3000b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a7fd49172f00d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.448+0000 7f0d4564e640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b52e0 0x7f0d2c0b77a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005fd0 tx=0x7f0d3c00b4a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.448+0000 7f0d2b7fe640 1 -- 172.21.15.33:0/1202975369 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30012c40 con 0x5624d299d4d0 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.448+0000 7f0d2b7fe640 1 -- 172.21.15.33:0/1202975369 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30012de0 con 0x5624d299d4d0 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.448+0000 7f0d2b7fe640 1 -- 172.21.15.33:0/1202975369 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d300047a0 con 0x5624d299d4d0 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 -- 172.21.15.33:0/1202975369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299d4d0 msgr2=0x5624d299d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299d4d0 0x5624d299d8b0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000bfd0 tx=0x7f0d3000b650 comp rx=0 tx=0).stop 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 -- 172.21.15.33:0/1202975369 shutdown_connections 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d29a9d80 0x5624d29ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d299ddf0 0x5624d29a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 --2- 172.21.15.33:0/1202975369 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299d4d0 0x5624d299d8b0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 -- 172.21.15.33:0/1202975369 >> 172.21.15.33:0/1202975369 conn(0x5624d299b300 msgr2=0x5624d299c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 -- 172.21.15.33:0/1202975369 shutdown_connections 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 -- 172.21.15.33:0/1202975369 wait complete. 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 -- start start 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.449+0000 7f0d49c87740 1 Processor -- start 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d29a9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d49c87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5624d2764700 con 0x5624d29a9d80 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d49c87740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5624d289e160 con 0x5624d299ddf0 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d49c87740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5624d29aca00 con 0x5624d299d4d0 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d45e4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d44e4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d45e4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42920/0 (socket says 172.21.15.33:42920) 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d44e4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41766/0 (socket says 172.21.15.33:41766) 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d45e4f640 1 -- 172.21.15.33:0/3811189367 learned_addr learned my addr 172.21.15.33:0/3811189367 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.450+0000 7f0d49c87740 10 client.0 _ll_register_callbacks cb 0x5624d28b79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d44e4d640 1 -- 172.21.15.33:0/3811189367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 msgr2=0x5624d29a9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d29a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d44e4d640 1 -- 172.21.15.33:0/3811189367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 msgr2=0x5624d29a7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d44e4d640 1 -- 172.21.15.33:0/3811189367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d28cbc90 con 0x5624d299ddf0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a6ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000b750 tx=0x7f0d300053d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3836aa883911e1eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d28ff9640 1 -- 172.21.15.33:0/3811189367 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30010950 con 0x5624d299ddf0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d30012de0 con 0x5624d299ddf0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.451+0000 7f0d28ff9640 1 -- 172.21.15.33:0/3811189367 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0d30010af0 con 0x5624d299ddf0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d28ff9640 1 -- 172.21.15.33:0/3811189367 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f0d30010cf0 con 0x5624d299ddf0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d49c87740 10 client.27833 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5624d29b8410 con 0x5624d299ddf0 2024-09-10T21:20:30.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d28ff9640 1 -- 172.21.15.33:0/3811189367 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30012de0 con 0x5624d299ddf0 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d28ff9640 1 -- 172.21.15.33:0/3811189367 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f0d3001f6a0 con 0x5624d299ddf0 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d49c87740 10 client.27833 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d49c87740 10 client.27833 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d49c87740 20 client.27833 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.452+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d29b97c0 0x5624d29bbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.453+0000 7f0d49c87740 4 client.27833 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.453+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624d2915df0 con 0x5624d29b97c0 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.453+0000 7f0d4564e640 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d29b97c0 0x5624d29bbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.454+0000 7f0d4564e640 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d29b97c0 0x5624d29bbbb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a7a5e00d579c23e9 server_cookie=aee29d1e03ea8540 in_seq=0 out_seq=0 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.454+0000 7f0d28ff9640 10 client.27833 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d28ff9640 1 -- 172.21.15.33:0/3811189367 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624d2915df0 con 0x5624d29b97c0 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d28ff9640 10 client.27833 handle_command_reply: tid=0 2024-09-10T21:20:30.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 client.27833 shutdown 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 msgr2=0x5624d29a6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a6ba0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000b750 tx=0x7f0d300053d0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 shutdown_connections 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d29b97c0 0x5624d29bbbb0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a6ba0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d28ff9640 0 client.27833 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3811189367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d29a9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 >> 172.21.15.33:0/3811189367 conn(0x5624d299b300 msgr2=0x5624d29a0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 shutdown_connections 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 1 -- 172.21.15.33:0/3811189367 wait complete. 2024-09-10T21:20:30.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.457+0000 7f0d49c87740 20 client.27833 trim_cache size 0 max 16384 2024-09-10T21:20:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.479+0000 7f0d49c87740 1 Processor -- start 2024-09-10T21:20:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.479+0000 7f0d49c87740 1 -- start start 2024-09-10T21:20:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.479+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d299ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.479+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.479+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d49c87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401bd150 con 0x5624d29a9d80 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d49c87740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401bec90 con 0x5624d299ddf0 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d49c87740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401bd820 con 0x5624d299d4d0 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42922/0 (socket says 172.21.15.33:42922) 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 -- 172.21.15.33:0/3287917452 learned_addr learned my addr 172.21.15.33:0/3287917452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d4564e640 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d299ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 -- 172.21.15.33:0/3287917452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 msgr2=0x5624d299ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d299ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 -- 172.21.15.33:0/3287917452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 msgr2=0x5624d29a88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 -- 172.21.15.33:0/3287917452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624d29a50a0 con 0x5624d29a9d80 2024-09-10T21:20:30.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d4564e640 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d299ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.480+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a3db0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0d380c97f0 tx=0x7f0d38004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b3d5210a0edce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.481+0000 7f0d2affd640 1 -- 172.21.15.33:0/3287917452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d380d60f0 con 0x5624d29a9d80 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.481+0000 7f0d2affd640 1 -- 172.21.15.33:0/3287917452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d380ccd30 con 0x5624d29a9d80 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.481+0000 7f0d2affd640 1 -- 172.21.15.33:0/3287917452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3807a150 con 0x5624d29a9d80 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 -- 172.21.15.33:0/3287917452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 msgr2=0x5624d29a3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a3db0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0d380c97f0 tx=0x7f0d38004ab0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 -- 172.21.15.33:0/3287917452 shutdown_connections 2024-09-10T21:20:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29a3db0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29a88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 --2- 172.21.15.33:0/3287917452 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d299ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 -- 172.21.15.33:0/3287917452 >> 172.21.15.33:0/3287917452 conn(0x5624d299b300 msgr2=0x5624d29a81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 -- 172.21.15.33:0/3287917452 shutdown_connections 2024-09-10T21:20:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 -- 172.21.15.33:0/3287917452 wait complete. 2024-09-10T21:20:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 -- start start 2024-09-10T21:20:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.482+0000 7f0d49c87740 1 Processor -- start 2024-09-10T21:20:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d29b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29b2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d49c87740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29acc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d44e4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d49c87740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401be5c0 con 0x5624d29a9d80 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d49c87740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401bdef0 con 0x5624d299ddf0 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d44e4d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41776/0 (socket says 172.21.15.33:41776) 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d49c87740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0d401bb110 con 0x5624d299d4d0 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d44e4d640 1 -- 172.21.15.33:0/242836333 learned_addr learned my addr 172.21.15.33:0/242836333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d49c87740 10 client.0 _ll_register_callbacks cb 0x7f0d3000bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.483+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d4564e640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d29b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d44e4d640 1 -- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 msgr2=0x5624d29b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d29b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d44e4d640 1 -- 172.21.15.33:0/242836333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 msgr2=0x5624d29acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d44e4d640 1 -- 172.21.15.33:0/242836333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624d29ad1c0 con 0x5624d299ddf0 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d45e4f640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d44e4d640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29b2210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d30000c00 tx=0x7f0d30005340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5664daaf95106472 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.484+0000 7f0d28ff9640 1 -- 172.21.15.33:0/242836333 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3001f670 con 0x5624d299ddf0 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.485+0000 7f0d28ff9640 1 -- 172.21.15.33:0/242836333 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0d3001f810 con 0x5624d299ddf0 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.485+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d380ccd30 con 0x5624d299ddf0 2024-09-10T21:20:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.485+0000 7f0d28ff9640 1 -- 172.21.15.33:0/242836333 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f0d3003c050 con 0x5624d299ddf0 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.485+0000 7f0d49c87740 10 client.27836 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.485+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x5624d2a26cd0 con 0x5624d299ddf0 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.485+0000 7f0d28ff9640 1 -- 172.21.15.33:0/242836333 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d30016040 con 0x5624d299ddf0 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d28ff9640 1 -- 172.21.15.33:0/242836333 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f0d30035460 con 0x5624d299ddf0 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d49c87740 10 client.27836 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d49c87740 10 client.27836 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d49c87740 20 client.27836 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d49c87740 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d2a27380 0x5624d2a297a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d49c87740 4 client.27836 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:20:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f0d380ccd30 con 0x5624d2a27380 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.486+0000 7f0d4564e640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d2a27380 0x5624d2a297a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.487+0000 7f0d28ff9640 10 client.27836 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.487+0000 7f0d4564e640 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d2a27380 0x5624d2a297a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=27958434f3e108e server_cookie=171cf3d17f23086b in_seq=0 out_seq=0 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d28ff9640 1 -- 172.21.15.33:0/242836333 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0d380ccd30 con 0x5624d2a27380 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d28ff9640 10 client.27836 handle_command_reply: tid=0 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 client.27836 shutdown 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 msgr2=0x5624d29b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29b2210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0d30000c00 tx=0x7f0d30005340 comp rx=0 tx=0).stop 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 shutdown_connections 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5624d2a27380 0x5624d2a297a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5624d29a9d80 0x5624d29acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d28ff9640 0 client.27836 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5624d299ddf0 0x5624d29b2210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 --2- 172.21.15.33:0/242836333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5624d299d4d0 0x5624d29b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.490+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 >> 172.21.15.33:0/242836333 conn(0x5624d299b300 msgr2=0x5624d29a89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.491+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 shutdown_connections 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.491+0000 7f0d49c87740 1 -- 172.21.15.33:0/242836333 wait complete. 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.491+0000 7f0d49c87740 20 client.27836 trim_cache size 0 max 16384 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b52e0 msgr2=0x7f0d2c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b52e0 0x7f0d2c0b77a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c005fd0 tx=0x7f0d3c00b4a0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 msgr2=0x7f0d401180f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 0x7f0d401180f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f0d3802ac70 tx=0x7f0d38003c90 comp rx=0 tx=0).stop 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 shutdown_connections 2024-09-10T21:20:30.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0d2c0b52e0 0x7f0d2c0b77a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0d40114810 0x7f0d401180f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0d4010bb80 0x7f0d40117ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 --2- 172.21.15.33:0/713067900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d4010b260 0x7f0d40119570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 >> 172.21.15.33:0/713067900 conn(0x7f0d4006e2e0 msgr2=0x7f0d40071790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 shutdown_connections 2024-09-10T21:20:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:30.492+0000 7f0d478b2640 1 -- 172.21.15.33:0/713067900 wait complete. 2024-09-10T21:20:31.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 174739 42.36 MB/sec execute 91 sec latency 90.986 ms 2024-09-10T21:20:31.385 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:20:31.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:31 smithi033 ceph-mon[30860]: pgmap v842: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 826 op/s 2024-09-10T21:20:31.729 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:31.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:31 smithi104 ceph-mon[35325]: pgmap v842: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 826 op/s 2024-09-10T21:20:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:31 smithi174 ceph-mon[35635]: pgmap v842: 97 pgs: 97 active+clean; 1.1 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 826 op/s 2024-09-10T21:20:32.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 175671 42.39 MB/sec execute 92 sec latency 147.363 ms 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 -- 172.21.15.33:0/692425144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd180103ca0 msgr2=0x7fd180104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 --2- 172.21.15.33:0/692425144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd180103ca0 0x7fd180104080 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd17400fb10 tx=0x7fd17402a160 comp rx=0 tx=0).stop 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 -- 172.21.15.33:0/692425144 shutdown_connections 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 --2- 172.21.15.33:0/692425144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd18010d100 0x7fd18010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 --2- 172.21.15.33:0/692425144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1801045c0 0x7fd18010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 --2- 172.21.15.33:0/692425144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd180103ca0 0x7fd180104080 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 -- 172.21.15.33:0/692425144 >> 172.21.15.33:0/692425144 conn(0x7fd1800fd700 msgr2=0x7fd1800ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:32.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 -- 172.21.15.33:0/692425144 shutdown_connections 2024-09-10T21:20:32.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.283+0000 7fd1860b1640 1 -- 172.21.15.33:0/692425144 wait complete. 2024-09-10T21:20:32.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.284+0000 7fd1860b1640 1 Processor -- start 2024-09-10T21:20:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.284+0000 7fd1860b1640 1 -- start start 2024-09-10T21:20:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.284+0000 7fd1860b1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd180103ca0 0x7fd1801970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.284+0000 7fd1860b1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1801045c0 0x7fd1801975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd180103ca0 0x7fd1801970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1801045c0 0x7fd1801975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd180103ca0 0x7fd1801970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42938/0 (socket says 172.21.15.33:42938) 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 -- 172.21.15.33:0/2186286508 learned_addr learned my addr 172.21.15.33:0/2186286508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd1860b1640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd18010d100 0x7fd18019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd180110bb0 con 0x7fd180103ca0 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd180110a30 con 0x7fd1801045c0 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd180110d30 con 0x7fd18010d100 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17ffff640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd18010d100 0x7fd18019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 -- 172.21.15.33:0/2186286508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd18010d100 msgr2=0x7fd18019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd18010d100 0x7fd18019c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 -- 172.21.15.33:0/2186286508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1801045c0 msgr2=0x7fd1801975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1801045c0 0x7fd1801975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17f7fe640 1 -- 172.21.15.33:0/2186286508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18019d150 con 0x7fd180103ca0 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.285+0000 7fd17effd640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1801045c0 0x7fd1801975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:32.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.286+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd180103ca0 0x7fd1801970a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd17400fc30 tx=0x7fd17400f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:32.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.286+0000 7fd17cff9640 1 -- 172.21.15.33:0/2186286508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17402ad70 con 0x7fd180103ca0 2024-09-10T21:20:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.286+0000 7fd17cff9640 1 -- 172.21.15.33:0/2186286508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd17400e660 con 0x7fd180103ca0 2024-09-10T21:20:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.286+0000 7fd17cff9640 1 -- 172.21.15.33:0/2186286508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740044e0 con 0x7fd180103ca0 2024-09-10T21:20:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.287+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18019d3e0 con 0x7fd180103ca0 2024-09-10T21:20:32.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.287+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18019d7b0 con 0x7fd180103ca0 2024-09-10T21:20:32.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.291+0000 7fd17cff9640 1 -- 172.21.15.33:0/2186286508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd17404c060 con 0x7fd180103ca0 2024-09-10T21:20:32.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.292+0000 7fd188486740 1 Processor -- start 2024-09-10T21:20:32.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.292+0000 7fd188486740 1 -- start start 2024-09-10T21:20:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.292+0000 7fd188486740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6593e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.292+0000 7fd188486740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d6594a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.292+0000 7fd17cff9640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1680b53e0 0x7fd1680b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd17ffff640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1680b53e0 0x7fd1680b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6593e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6593e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42952/0 (socket says 172.21.15.33:42952) 2024-09-10T21:20:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd188486740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd188486740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d658b4ea0 con 0x561d6593e0f0 2024-09-10T21:20:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd188486740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561d658a42d0 con 0x561d6594a9a0 2024-09-10T21:20:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd188486740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d658a80e0 con 0x561d6593ea10 2024-09-10T21:20:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d6594a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd17ffff640 1 -- 172.21.15.33:0/1507122885 learned_addr learned my addr 172.21.15.33:0/1507122885 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd17cff9640 1 -- 172.21.15.33:0/2186286508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd174105530 con 0x7fd180103ca0 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.293+0000 7fd17effd640 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17effd640 1 -- 172.21.15.33:0/1507122885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 msgr2=0x561d6594a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17effd640 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d6594a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17effd640 1 -- 172.21.15.33:0/1507122885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 msgr2=0x561d6593e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17effd640 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6593e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17effd640 1 -- 172.21.15.33:0/1507122885 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d65602650 con 0x561d6594a9a0 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d6594a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17ffff640 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6593e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17ffff640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1680b53e0 0x7fd1680b78a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd178003fd0 tx=0x7fd178003990 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd17effd640 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594cd90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd18006b950 tx=0x7fd170010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880020cb60d3f45f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd165ffb640 1 -- 172.21.15.33:0/1507122885 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170004220 con 0x561d6594a9a0 2024-09-10T21:20:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd165ffb640 1 -- 172.21.15.33:0/1507122885 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1700043c0 con 0x561d6594a9a0 2024-09-10T21:20:32.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.294+0000 7fd165ffb640 1 -- 172.21.15.33:0/1507122885 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170011870 con 0x561d6594a9a0 2024-09-10T21:20:32.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 -- 172.21.15.33:0/1507122885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 msgr2=0x561d6594cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594cd90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd18006b950 tx=0x7fd170010860 comp rx=0 tx=0).stop 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 -- 172.21.15.33:0/1507122885 shutdown_connections 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594cd90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d6594a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 --2- 172.21.15.33:0/1507122885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6593e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 -- 172.21.15.33:0/1507122885 >> 172.21.15.33:0/1507122885 conn(0x561d6593bf20 msgr2=0x561d6593d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 -- 172.21.15.33:0/1507122885 shutdown_connections 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.295+0000 7fd188486740 1 -- 172.21.15.33:0/1507122885 wait complete. 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 -- start start 2024-09-10T21:20:32.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 Processor -- start 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6594a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65947740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6594a9a0 0x561d65947da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561d65705700 con 0x561d6593e0f0 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561d6583f160 con 0x561d6593ea10 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.296+0000 7fd188486740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561d6594d620 con 0x561d6594a9a0 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd188486740 10 client.0 _ll_register_callbacks cb 0x561d658589e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:32.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6594a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6594a9a0 0x561d65947da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6594a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42968/0 (socket says 172.21.15.33:42968) 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6594a9a0 0x561d65947da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40292/0 (socket says 172.21.15.33:40292) 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 -- 172.21.15.33:0/1661837044 learned_addr learned my addr 172.21.15.33:0/1661837044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17effd640 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65947740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 -- 172.21.15.33:0/1661837044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6594a9a0 msgr2=0x561d65947da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6594a9a0 0x561d65947da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 -- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 msgr2=0x561d65947740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65947740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 -- 172.21.15.33:0/1661837044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d6586cc90 con 0x561d6593e0f0 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6594a2f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd1740c8f20 tx=0x7fd1740cb550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f2aea187868b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:32.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.297+0000 7fd15effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.298+0000 7fd15effd640 1 -- 172.21.15.33:0/1661837044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1740cbcb0 con 0x561d6593e0f0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.298+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1700043c0 con 0x561d6593e0f0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.298+0000 7fd15effd640 1 -- 172.21.15.33:0/1661837044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd1740c7f20 con 0x561d6593e0f0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.298+0000 7fd15effd640 1 -- 172.21.15.33:0/1661837044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fd1740d0790 con 0x561d6593e0f0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.298+0000 7fd188486740 10 client.18444 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.298+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x561d65958fb0 con 0x561d6593e0f0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.298+0000 7fd15effd640 1 -- 172.21.15.33:0/1661837044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1700043c0 con 0x561d6593e0f0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd15effd640 1 -- 172.21.15.33:0/1661837044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7fd1741050e0 con 0x561d6593e0f0 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd188486740 10 client.18444 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd188486740 10 client.18444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd188486740 20 client.18444 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd188486740 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d6595a460 0x561d6595c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd188486740 4 client.18444 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d658b6df0 con 0x561d6595a460 2024-09-10T21:20:32.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.299+0000 7fd17effd640 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d6595a460 0x561d6595c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.300+0000 7fd17effd640 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d6595a460 0x561d6595c8e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f398868bf6f45479 server_cookie=2d62d684b25881a7 in_seq=0 out_seq=0 2024-09-10T21:20:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.300+0000 7fd15effd640 10 client.18444 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd15effd640 1 -- 172.21.15.33:0/1661837044 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d658b6df0 con 0x561d6595a460 2024-09-10T21:20:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd15effd640 10 client.18444 handle_command_reply: tid=0 2024-09-10T21:20:32.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd188486740 1 client.18444 shutdown 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 msgr2=0x561d6594a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd188486740 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6594a2f0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fd1740c8f20 tx=0x7fd1740cb550 comp rx=0 tx=0).stop 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 shutdown_connections 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd188486740 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d6595a460 0x561d6595c8e0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.301+0000 7fd188486740 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6594a9a0 0x561d65947da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.302+0000 7fd188486740 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65947740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.302+0000 7fd188486740 1 --2- 172.21.15.33:0/1661837044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d6594a2f0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.302+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 >> 172.21.15.33:0/1661837044 conn(0x561d6593bf20 msgr2=0x561d65941ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.302+0000 7fd15effd640 0 client.18444 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.302+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 shutdown_connections 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.302+0000 7fd188486740 1 -- 172.21.15.33:0/1661837044 wait complete. 2024-09-10T21:20:32.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.302+0000 7fd188486740 20 client.18444 trim_cache size 0 max 16384 2024-09-10T21:20:32.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 Processor -- start 2024-09-10T21:20:32.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 -- start start 2024-09-10T21:20:32.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d65956850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d65949480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ac190 con 0x561d6593e0f0 2024-09-10T21:20:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801adcd0 con 0x561d6594a9a0 2024-09-10T21:20:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd188486740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ac860 con 0x561d6593ea10 2024-09-10T21:20:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd17effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d65949480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd17effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d65949480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40304/0 (socket says 172.21.15.33:40304) 2024-09-10T21:20:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41828/0 (socket says 172.21.15.33:41828) 2024-09-10T21:20:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.328+0000 7fd17effd640 1 -- 172.21.15.33:0/2816760443 learned_addr learned my addr 172.21.15.33:0/2816760443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d65956850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17effd640 1 -- 172.21.15.33:0/2816760443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 msgr2=0x561d6594bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17effd640 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17effd640 1 -- 172.21.15.33:0/2816760443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 msgr2=0x561d65956850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17effd640 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d65956850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17effd640 1 -- 172.21.15.33:0/2816760443 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d65945c40 con 0x561d6593ea10 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17ffff640 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd17effd640 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d65949480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd17000bef0 tx=0x7fd17000ba30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b88071998f452d50 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd1657fa640 1 -- 172.21.15.33:0/2816760443 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170029020 con 0x561d6593ea10 2024-09-10T21:20:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd1657fa640 1 -- 172.21.15.33:0/2816760443 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd170003b00 con 0x561d6593ea10 2024-09-10T21:20:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.329+0000 7fd1657fa640 1 -- 172.21.15.33:0/2816760443 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170011760 con 0x561d6593ea10 2024-09-10T21:20:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.330+0000 7fd188486740 1 -- 172.21.15.33:0/2816760443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 msgr2=0x561d65949480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.330+0000 7fd188486740 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d65949480 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd17000bef0 tx=0x7fd17000ba30 comp rx=0 tx=0).stop 2024-09-10T21:20:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.330+0000 7fd188486740 1 -- 172.21.15.33:0/2816760443 shutdown_connections 2024-09-10T21:20:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.330+0000 7fd188486740 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6594a9a0 0x561d6594bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.330+0000 7fd188486740 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593ea10 0x561d65949480 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.330+0000 7fd188486740 1 --2- 172.21.15.33:0/2816760443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6593e0f0 0x561d65956850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.330+0000 7fd188486740 1 -- 172.21.15.33:0/2816760443 >> 172.21.15.33:0/2816760443 conn(0x561d6593bf20 msgr2=0x561d65948d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.331+0000 7fd188486740 1 -- 172.21.15.33:0/2816760443 shutdown_connections 2024-09-10T21:20:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.331+0000 7fd188486740 1 -- 172.21.15.33:0/2816760443 wait complete. 2024-09-10T21:20:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.331+0000 7fd188486740 1 -- start start 2024-09-10T21:20:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.331+0000 7fd188486740 1 Processor -- start 2024-09-10T21:20:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd188486740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593e0f0 0x561d65952a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd188486740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65952ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd188486740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6594a9a0 0x561d65943960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd188486740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801ad600 con 0x561d6594a9a0 2024-09-10T21:20:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd188486740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801acf30 con 0x561d6593ea10 2024-09-10T21:20:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd188486740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1801aa150 con 0x561d6593e0f0 2024-09-10T21:20:32.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd188486740 10 client.0 _ll_register_callbacks cb 0x7fd1740cb6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593e0f0 0x561d65952a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6594a9a0 0x561d65943960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593e0f0 0x561d65952a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40310/0 (socket says 172.21.15.33:40310) 2024-09-10T21:20:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17f7fe640 1 -- 172.21.15.33:0/3747701905 learned_addr learned my addr 172.21.15.33:0/3747701905 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17effd640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65952ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17ffff640 1 -- 172.21.15.33:0/3747701905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593e0f0 msgr2=0x561d65952a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17ffff640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593e0f0 0x561d65952a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17ffff640 1 -- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 msgr2=0x561d65952ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17ffff640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65952ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17ffff640 1 -- 172.21.15.33:0/3747701905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d659440f0 con 0x561d6594a9a0 2024-09-10T21:20:32.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.332+0000 7fd17effd640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65952ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593e0f0 0x561d65952a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd17ffff640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6594a9a0 0x561d65943960 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd178015010 tx=0x7fd178015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2addc5ede064b011 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd15effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:20:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd15effd640 1 -- 172.21.15.33:0/3747701905 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd178012ea0 con 0x561d6594a9a0 2024-09-10T21:20:32.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd15effd640 1 -- 172.21.15.33:0/3747701905 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd17801c040 con 0x561d6594a9a0 2024-09-10T21:20:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd170003b00 con 0x561d6594a9a0 2024-09-10T21:20:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd15effd640 1 -- 172.21.15.33:0/3747701905 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7fd178004b20 con 0x561d6594a9a0 2024-09-10T21:20:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd188486740 10 client.18450 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.333+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x561d65945420 con 0x561d6594a9a0 2024-09-10T21:20:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd15effd640 1 -- 172.21.15.33:0/3747701905 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd178004cc0 con 0x561d6594a9a0 2024-09-10T21:20:32.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd15effd640 1 -- 172.21.15.33:0/3747701905 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7fd178039020 con 0x561d6594a9a0 2024-09-10T21:20:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd188486740 10 client.18450 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd188486740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd188486740 20 client.18450 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd188486740 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d659c86a0 0x561d659caac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:32.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd188486740 4 client.18450 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:20:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd170003b00 con 0x561d659c86a0 2024-09-10T21:20:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.334+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d659c86a0 0x561d659caac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.335+0000 7fd15effd640 10 client.18450 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.335+0000 7fd17f7fe640 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d659c86a0 0x561d659caac0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b92bc6da6d6c15ca server_cookie=6ca34aa2f666b3cc in_seq=0 out_seq=0 2024-09-10T21:20:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.335+0000 7fd15effd640 1 -- 172.21.15.33:0/3747701905 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd170003b00 con 0x561d659c86a0 2024-09-10T21:20:32.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.335+0000 7fd15effd640 10 client.18450 handle_command_reply: tid=0 2024-09-10T21:20:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.335+0000 7fd188486740 1 client.18450 shutdown 2024-09-10T21:20:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6594a9a0 msgr2=0x561d65943960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6594a9a0 0x561d65943960 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fd178015010 tx=0x7fd178015040 comp rx=0 tx=0).stop 2024-09-10T21:20:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 shutdown_connections 2024-09-10T21:20:32.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x561d659c86a0 0x561d659caac0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd15effd640 0 client.18450 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561d6594a9a0 0x561d65943960 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561d6593ea10 0x561d65952ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 --2- 172.21.15.33:0/3747701905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561d6593e0f0 0x561d65952a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 >> 172.21.15.33:0/3747701905 conn(0x561d6593bf20 msgr2=0x561d65949590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:32.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 shutdown_connections 2024-09-10T21:20:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 1 -- 172.21.15.33:0/3747701905 wait complete. 2024-09-10T21:20:32.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.336+0000 7fd188486740 20 client.18450 trim_cache size 0 max 16384 2024-09-10T21:20:32.355 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:20:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1680b53e0 msgr2=0x7fd1680b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1680b53e0 0x7fd1680b78a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fd178003fd0 tx=0x7fd178003990 comp rx=0 tx=0).stop 2024-09-10T21:20:32.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd180103ca0 msgr2=0x7fd1801970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd180103ca0 0x7fd1801970a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd17400fc30 tx=0x7fd17400f350 comp rx=0 tx=0).stop 2024-09-10T21:20:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 shutdown_connections 2024-09-10T21:20:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1680b53e0 0x7fd1680b78a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd18010d100 0x7fd18019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd1801045c0 0x7fd1801975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 --2- 172.21.15.33:0/2186286508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd180103ca0 0x7fd1801970a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.337+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 >> 172.21.15.33:0/2186286508 conn(0x7fd1800fd700 msgr2=0x7fd1800fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.338+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 shutdown_connections 2024-09-10T21:20:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:32.338+0000 7fd1860b1640 1 -- 172.21.15.33:0/2186286508 wait complete. 2024-09-10T21:20:33.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 176453 42.51 MB/sec execute 93 sec latency 166.364 ms 2024-09-10T21:20:33.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:33 smithi104 ceph-mon[35325]: pgmap v843: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T21:20:33.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:33 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:20:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:33 smithi174 ceph-mon[35635]: pgmap v843: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T21:20:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:33 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:20:33.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:33 smithi033 ceph-mon[30860]: pgmap v843: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 836 op/s 2024-09-10T21:20:33.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:33 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:20:34.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 176900 42.36 MB/sec execute 94 sec latency 447.990 ms 2024-09-10T21:20:34.174 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:20:34.174 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:20:34.555 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:34.739 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:20:34.739 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:20:34.739 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:34 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:20:34.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:20:34.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:20:34.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:34 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:20:34.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:20:34.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:20:34.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:34 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:20:35.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 177508 42.19 MB/sec execute 95 sec latency 91.477 ms 2024-09-10T21:20:35.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 -- 172.21.15.33:0/265492935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44107720 msgr2=0x7f5f44107b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/265492935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44107720 0x7f5f44107b60 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009900 tx=0x7f5f3802a410 comp rx=0 tx=0).stop 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 -- 172.21.15.33:0/265492935 shutdown_connections 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/265492935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f441080a0 0x7f5f4410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/265492935 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44107720 0x7f5f44107b60 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/265492935 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f44101d10 0x7f5f441020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 -- 172.21.15.33:0/265492935 >> 172.21.15.33:0/265492935 conn(0x7f5f440fd7c0 msgr2=0x7f5f440ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 -- 172.21.15.33:0/265492935 shutdown_connections 2024-09-10T21:20:35.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.140+0000 7f5f4a30b640 1 -- 172.21.15.33:0/265492935 wait complete. 2024-09-10T21:20:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.141+0000 7f5f4a30b640 1 Processor -- start 2024-09-10T21:20:35.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.141+0000 7f5f4a30b640 1 -- start start 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f4a30b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44101d10 0x7f5f4419b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f4a30b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f44107720 0x7f5f4419ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f4a30b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f441080a0 0x7f5f441a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f4a30b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44111d70 con 0x7f5f44101d10 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44101d10 0x7f5f4419b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f43fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44101d10 0x7f5f4419b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43002/0 (socket says 172.21.15.33:43002) 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f43fff640 1 -- 172.21.15.33:0/2437666233 learned_addr learned my addr 172.21.15.33:0/2437666233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:35.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f437fe640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f44107720 0x7f5f4419ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44111bf0 con 0x7f5f44107720 2024-09-10T21:20:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.142+0000 7f5f488a8640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f441080a0 0x7f5f441a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.143+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f44111ef0 con 0x7f5f441080a0 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.143+0000 7f5f43fff640 1 -- 172.21.15.33:0/2437666233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f441080a0 msgr2=0x7f5f441a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.143+0000 7f5f43fff640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f441080a0 0x7f5f441a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.143+0000 7f5f43fff640 1 -- 172.21.15.33:0/2437666233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f44107720 msgr2=0x7f5f4419ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.143+0000 7f5f43fff640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f44107720 0x7f5f4419ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.143+0000 7f5f43fff640 1 -- 172.21.15.33:0/2437666233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f441a15c0 con 0x7f5f44101d10 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.143+0000 7f5f43fff640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44101d10 0x7f5f4419b510 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400ea60 tx=0x7f5f3400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.144+0000 7f5f417fa640 1 -- 172.21.15.33:0/2437666233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3400cde0 con 0x7f5f44101d10 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.144+0000 7f5f417fa640 1 -- 172.21.15.33:0/2437666233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f340035f0 con 0x7f5f44101d10 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.144+0000 7f5f417fa640 1 -- 172.21.15.33:0/2437666233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f34010690 con 0x7f5f44101d10 2024-09-10T21:20:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.144+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f441a18b0 con 0x7f5f44101d10 2024-09-10T21:20:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.144+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f441a8140 con 0x7f5f44101d10 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 Processor -- start 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 -- start start 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7df2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55afbe755e60 con 0x55afbe7deee0 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55afbe745290 con 0x55afbe7df800 2024-09-10T21:20:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.147+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55afbe7490a0 con 0x55afbe7f3fa0 2024-09-10T21:20:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7df2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f417fa640 1 -- 172.21.15.33:0/2437666233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5f34010830 con 0x7f5f44101d10 2024-09-10T21:20:35.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7df2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43006/0 (socket says 172.21.15.33:43006) 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 -- 172.21.15.33:0/3108748151 learned_addr learned my addr 172.21.15.33:0/3108748151 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f43fff640 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f437fe640 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 -- 172.21.15.33:0/3108748151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 msgr2=0x55afbe7f6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 -- 172.21.15.33:0/3108748151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 msgr2=0x55afbe7f3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f488a8640 1 -- 172.21.15.33:0/3108748151 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afbe4a3930 con 0x55afbe7deee0 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.148+0000 7f5f43fff640 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.149+0000 7f5f488a8640 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7df2c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009fd0 tx=0x7f5f3c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37980613dd0eba1b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.150+0000 7f5f2a7fc640 1 -- 172.21.15.33:0/3108748151 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c004d20 con 0x55afbe7deee0 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.150+0000 7f5f2a7fc640 1 -- 172.21.15.33:0/3108748151 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3c004050 con 0x55afbe7deee0 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.150+0000 7f5f2a7fc640 1 -- 172.21.15.33:0/3108748151 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c005280 con 0x55afbe7deee0 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.150+0000 7f5f417fa640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f2c0b5360 0x7f5f2c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.150+0000 7f5f43fff640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f2c0b5360 0x7f5f2c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f417fa640 1 -- 172.21.15.33:0/2437666233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f340e5e60 con 0x7f5f44101d10 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3108748151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 msgr2=0x55afbe7df2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7df2c0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009fd0 tx=0x7f5f3c0118d0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3108748151 shutdown_connections 2024-09-10T21:20:35.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3108748151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7df2c0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3108748151 >> 172.21.15.33:0/3108748151 conn(0x55afbe7dcd10 msgr2=0x55afbe7ddeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:35.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.151+0000 7f5f43fff640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f2c0b5360 0x7f5f2c0b7820 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5f340ac570 tx=0x7f5f34014000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:35.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.152+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3108748151 shutdown_connections 2024-09-10T21:20:35.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.152+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3108748151 wait complete. 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.152+0000 7f5f4c6e0740 1 -- start start 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f4c6e0740 1 Processor -- start 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43018/0 (socket says 172.21.15.33:43018) 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f488a8640 1 -- 172.21.15.33:0/3610122513 learned_addr learned my addr 172.21.15.33:0/3610122513 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55afbe5a6890 con 0x55afbe7deee0 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55afbe6dfd40 con 0x55afbe7df800 2024-09-10T21:20:35.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55afbe7f6c20 con 0x55afbe7f3fa0 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.153+0000 7f5f437fe640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f4c6e0740 10 client.0 _ll_register_callbacks cb 0x55afbe743c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f488a8640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f437fe640 1 -- 172.21.15.33:0/3610122513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 msgr2=0x55afbe7f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f437fe640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f437fe640 1 -- 172.21.15.33:0/3610122513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 msgr2=0x55afbe7f0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f437fe640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7f0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f437fe640 1 -- 172.21.15.33:0/3610122513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afbe800030 con 0x55afbe7df800 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f488a8640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f437fe640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f14a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f3802aa00 tx=0x7f5f380042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2f52c07b0837d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f237fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.154+0000 7f5f237fe640 1 -- 172.21.15.33:0/3610122513 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38035040 con 0x55afbe7df800 2024-09-10T21:20:35.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c004050 con 0x55afbe7df800 2024-09-10T21:20:35.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f237fe640 1 -- 172.21.15.33:0/3610122513 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f38005450 con 0x55afbe7df800 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f237fe640 1 -- 172.21.15.33:0/3610122513 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f5f38005650 con 0x55afbe7df800 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f4c6e0740 10 client.27866 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55afbe8029f0 con 0x55afbe7df800 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f237fe640 1 -- 172.21.15.33:0/3610122513 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3c004050 con 0x55afbe7df800 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f237fe640 1 -- 172.21.15.33:0/3610122513 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f5f380038f0 con 0x55afbe7df800 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f4c6e0740 10 client.27866 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.155+0000 7f5f4c6e0740 10 client.27866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.156+0000 7f5f4c6e0740 20 client.27866 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.156+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe803da0 0x55afbe806230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.156+0000 7f5f4c6e0740 4 client.27866 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.156+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afbe757db0 con 0x55afbe803da0 2024-09-10T21:20:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.156+0000 7f5f488a8640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe803da0 0x55afbe806230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.156+0000 7f5f237fe640 10 client.27866 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.156+0000 7f5f488a8640 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe803da0 0x55afbe806230 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a593643948e6da81 server_cookie=4af6546c3868feb5 in_seq=0 out_seq=0 2024-09-10T21:20:35.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f237fe640 1 -- 172.21.15.33:0/3610122513 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55afbe757db0 con 0x55afbe803da0 2024-09-10T21:20:35.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f237fe640 10 client.27866 handle_command_reply: tid=0 2024-09-10T21:20:35.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 client.27866 shutdown 2024-09-10T21:20:35.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 msgr2=0x55afbe7f14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f14a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5f3802aa00 tx=0x7f5f380042e0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 shutdown_connections 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe803da0 0x55afbe806230 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f14a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/3610122513 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7f0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 >> 172.21.15.33:0/3610122513 conn(0x55afbe7dcd10 msgr2=0x55afbe7dbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.159+0000 7f5f237fe640 0 client.27866 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.160+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 shutdown_connections 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.160+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/3610122513 wait complete. 2024-09-10T21:20:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.160+0000 7f5f4c6e0740 20 client.27866 trim_cache size 0 max 16384 2024-09-10T21:20:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 Processor -- start 2024-09-10T21:20:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 -- start start 2024-09-10T21:20:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7eea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441ac380 con 0x55afbe7deee0 2024-09-10T21:20:35.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441adec0 con 0x55afbe7df800 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441aca50 con 0x55afbe7f3fa0 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7eea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43024/0 (socket says 172.21.15.33:43024) 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f437fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.184+0000 7f5f488a8640 1 -- 172.21.15.33:0/1802638619 learned_addr learned my addr 172.21.15.33:0/1802638619 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f488a8640 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f488a8640 1 -- 172.21.15.33:0/1802638619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 msgr2=0x55afbe7f2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f488a8640 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f488a8640 1 -- 172.21.15.33:0/1802638619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 msgr2=0x55afbe7f07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f488a8640 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f488a8640 1 -- 172.21.15.33:0/1802638619 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afbe7de8c0 con 0x55afbe7deee0 2024-09-10T21:20:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f488a8640 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7eea30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000f80 tx=0x7f5f3c003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbbe779b3c67499 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f29ffb640 1 -- 172.21.15.33:0/1802638619 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0045a0 con 0x55afbe7deee0 2024-09-10T21:20:35.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f29ffb640 1 -- 172.21.15.33:0/1802638619 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3c004050 con 0x55afbe7deee0 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.185+0000 7f5f29ffb640 1 -- 172.21.15.33:0/1802638619 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0055f0 con 0x55afbe7deee0 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.186+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1802638619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 msgr2=0x55afbe7eea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.186+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7eea30 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c000f80 tx=0x7f5f3c003f50 comp rx=0 tx=0).stop 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1802638619 shutdown_connections 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7df800 0x55afbe7f07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1802638619 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7deee0 0x55afbe7eea30 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1802638619 >> 172.21.15.33:0/1802638619 conn(0x55afbe7dcd10 msgr2=0x55afbe7f2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1802638619 shutdown_connections 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1802638619 wait complete. 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.187+0000 7f5f4c6e0740 1 -- start start 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 1 Processor -- start 2024-09-10T21:20:35.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7deee0 0x55afbe7fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7df800 0x55afbe7fc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441ad7f0 con 0x55afbe7df800 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441ad120 con 0x55afbe7f3fa0 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5f441aa380 con 0x55afbe7deee0 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f4c6e0740 10 client.0 _ll_register_callbacks cb 0x7f5f3c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7deee0 0x55afbe7fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f488a8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41880/0 (socket says 172.21.15.33:41880) 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f488a8640 1 -- 172.21.15.33:0/1370903173 learned_addr learned my addr 172.21.15.33:0/1370903173 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.188+0000 7f5f437fe640 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7df800 0x55afbe7fc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f488a8640 1 -- 172.21.15.33:0/1370903173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7deee0 msgr2=0x55afbe7fbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f488a8640 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7deee0 0x55afbe7fbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f488a8640 1 -- 172.21.15.33:0/1370903173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7df800 msgr2=0x55afbe7fc420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f488a8640 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7df800 0x55afbe7fc420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f488a8640 1 -- 172.21.15.33:0/1370903173 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afbe7f93f0 con 0x55afbe7f3fa0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f488a8640 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f8eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009370 tx=0x7f5f3c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f7dc03cd3f76ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f237fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f237fe640 1 -- 172.21.15.33:0/1370903173 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0045a0 con 0x55afbe7f3fa0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c004050 con 0x55afbe7f3fa0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.189+0000 7f5f237fe640 1 -- 172.21.15.33:0/1370903173 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5f3c004a00 con 0x55afbe7f3fa0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f237fe640 1 -- 172.21.15.33:0/1370903173 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f5f3c004c00 con 0x55afbe7f3fa0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f4c6e0740 10 client.27872 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55afbe871590 con 0x55afbe7f3fa0 2024-09-10T21:20:35.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f237fe640 1 -- 172.21.15.33:0/1370903173 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f3c004050 con 0x55afbe7f3fa0 2024-09-10T21:20:35.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f237fe640 1 -- 172.21.15.33:0/1370903173 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f5f3c038020 con 0x55afbe7f3fa0 2024-09-10T21:20:35.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f4c6e0740 10 client.27872 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f4c6e0740 10 client.27872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f4c6e0740 20 client.27872 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.190+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe871aa0 0x55afbe873ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.191+0000 7f5f4c6e0740 4 client.27872 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.191+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55afbe753860 con 0x55afbe871aa0 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.191+0000 7f5f437fe640 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe871aa0 0x55afbe873ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.191+0000 7f5f437fe640 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe871aa0 0x55afbe873ec0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdc57d32ea2a1997 server_cookie=9ecdc9f2b5d627ae in_seq=0 out_seq=0 2024-09-10T21:20:35.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.191+0000 7f5f237fe640 10 client.27872 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f237fe640 1 -- 172.21.15.33:0/1370903173 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55afbe753860 con 0x55afbe871aa0 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f237fe640 10 client.27872 handle_command_reply: tid=0 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 client.27872 shutdown 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7f3fa0 msgr2=0x55afbe7f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f8eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009370 tx=0x7f5f3c009fd0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 shutdown_connections 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55afbe871aa0 0x55afbe873ec0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f237fe640 0 client.27872 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55afbe7f3fa0 0x55afbe7f8eb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55afbe7df800 0x55afbe7fc420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 --2- 172.21.15.33:0/1370903173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55afbe7deee0 0x55afbe7fbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 >> 172.21.15.33:0/1370903173 conn(0x55afbe7dcd10 msgr2=0x55afbe7ff690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 shutdown_connections 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.195+0000 7f5f4c6e0740 1 -- 172.21.15.33:0/1370903173 wait complete. 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4c6e0740 20 client.27872 trim_cache size 0 max 16384 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "941658e5-2c5c-43b5-aa24-7d3c6632ea39", 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:20:35.201 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f2c0b5360 msgr2=0x7f5f2c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f2c0b5360 0x7f5f2c0b7820 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5f340ac570 tx=0x7f5f34014000 comp rx=0 tx=0).stop 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44101d10 msgr2=0x7f5f4419b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44101d10 0x7f5f4419b510 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f5f3400ea60 tx=0x7f5f3400ef20 comp rx=0 tx=0).stop 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 shutdown_connections 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5f2c0b5360 0x7f5f2c0b7820 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5f441080a0 0x7f5f441a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5f44107720 0x7f5f4419ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 --2- 172.21.15.33:0/2437666233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f44101d10 0x7f5f4419b510 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:35.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.196+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 >> 172.21.15.33:0/2437666233 conn(0x7f5f440fd7c0 msgr2=0x7f5f4406baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.197+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 shutdown_connections 2024-09-10T21:20:35.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:35.197+0000 7f5f4a30b640 1 -- 172.21.15.33:0/2437666233 wait complete. 2024-09-10T21:20:36.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 178091 41.99 MB/sec execute 96 sec latency 246.770 ms 2024-09-10T21:20:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:35 smithi174 ceph-mon[35635]: pgmap v844: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-10T21:20:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:20:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:20:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:35 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:20:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:35 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:36.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:35 smithi033 ceph-mon[30860]: pgmap v844: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-10T21:20:36.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:20:36.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:20:36.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:35 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:20:36.168 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:35 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:36.209 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:20:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:35 smithi104 ceph-mon[35325]: pgmap v844: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 831 op/s 2024-09-10T21:20:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:20:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:20:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:35 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:20:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:35 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:36.551 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:36.988 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 179181 42.26 MB/sec execute 97 sec latency 65.798 ms 2024-09-10T21:20:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 -- 172.21.15.33:0/3801343387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c10d130 msgr2=0x7f831c10f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 --2- 172.21.15.33:0/3801343387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c10d130 0x7f831c10f590 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f8314009ca0 tx=0x7f831402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 -- 172.21.15.33:0/3801343387 shutdown_connections 2024-09-10T21:20:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 --2- 172.21.15.33:0/3801343387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c10d130 0x7f831c10f590 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 --2- 172.21.15.33:0/3801343387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c1045f0 0x7f831c10cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 --2- 172.21.15.33:0/3801343387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f831c103cd0 0x7f831c1040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 -- 172.21.15.33:0/3801343387 >> 172.21.15.33:0/3801343387 conn(0x7f831c0fd770 msgr2=0x7f831c0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 -- 172.21.15.33:0/3801343387 shutdown_connections 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.096+0000 7f83236c0640 1 -- 172.21.15.33:0/3801343387 wait complete. 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 Processor -- start 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 -- start start 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c103cd0 0x7f831c19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f831c1045f0 0x7f831c19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c10d130 0x7f831c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f831c114ff0 con 0x7f831c103cd0 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f831c114e70 con 0x7f831c10d130 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f83236c0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f831c115170 con 0x7f831c1045f0 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f832145c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c103cd0 0x7f831c19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f832145c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c103cd0 0x7f831c19b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43042/0 (socket says 172.21.15.33:43042) 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f832145c640 1 -- 172.21.15.33:0/422803547 learned_addr learned my addr 172.21.15.33:0/422803547 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.097+0000 7f8320c5b640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f831c1045f0 0x7f831c19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f832145c640 1 -- 172.21.15.33:0/422803547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f831c1045f0 msgr2=0x7f831c19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f832145c640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f831c1045f0 0x7f831c19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f832145c640 1 -- 172.21.15.33:0/422803547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c10d130 msgr2=0x7f831c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f832145c640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c10d130 0x7f831c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f832145c640 1 -- 172.21.15.33:0/422803547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1a1530 con 0x7f831c103cd0 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f832145c640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c103cd0 0x7f831c19b480 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f831800e9f0 tx=0x7f831800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f83127fc640 1 -- 172.21.15.33:0/422803547 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831800cd80 con 0x7f831c103cd0 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f83127fc640 1 -- 172.21.15.33:0/422803547 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83180054d0 con 0x7f831c103cd0 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f83127fc640 1 -- 172.21.15.33:0/422803547 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318010690 con 0x7f831c103cd0 2024-09-10T21:20:37.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1a1820 con 0x7f831c103cd0 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.098+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1a7ee0 con 0x7f831c103cd0 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.103+0000 7f83127fc640 1 -- 172.21.15.33:0/422803547 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8318004e70 con 0x7f831c103cd0 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 Processor -- start 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 -- start start 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aaa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6abf160 0x55d8b6ac1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8b6a20e50 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d8b6a10280 con 0x55d8b6abf160 2024-09-10T21:20:37.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.105+0000 7f8325a95740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8b6a14090 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f8321c5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aaa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f832145c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f832145c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40408/0 (socket says 172.21.15.33:40408) 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f832145c640 1 -- 172.21.15.33:0/623098196 learned_addr learned my addr 172.21.15.33:0/623098196 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f83127fc640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83080b5360 0x7f83080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f8320c5b640 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6abf160 0x55d8b6ac1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f8321c5d640 1 -- 172.21.15.33:0/623098196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 msgr2=0x55d8b6abec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f8321c5d640 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f8321c5d640 1 -- 172.21.15.33:0/623098196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6abf160 msgr2=0x55d8b6ac1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f8321c5d640 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6abf160 0x55d8b6ac1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f8321c5d640 1 -- 172.21.15.33:0/623098196 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b676e650 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.106+0000 7f83127fc640 1 -- 172.21.15.33:0/422803547 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f83180e6020 con 0x7f831c103cd0 2024-09-10T21:20:37.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.107+0000 7f8321c5d640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83080b5360 0x7f83080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.107+0000 7f8321c5d640 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aaa480 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f8314004af0 tx=0x7f8314009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6379a9ba03ed76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.108+0000 7f8321c5d640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83080b5360 0x7f83080b7820 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8314009430 tx=0x7f831404b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.108+0000 7f83077fe640 1 -- 172.21.15.33:0/623098196 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831404d070 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.108+0000 7f83077fe640 1 -- 172.21.15.33:0/623098196 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8314041070 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.108+0000 7f83077fe640 1 -- 172.21.15.33:0/623098196 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831402f870 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.109+0000 7f8325a95740 1 -- 172.21.15.33:0/623098196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa0a0 msgr2=0x55d8b6aaa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.109+0000 7f8325a95740 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aaa480 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f8314004af0 tx=0x7f8314009800 comp rx=0 tx=0).stop 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.110+0000 7f8325a95740 1 -- 172.21.15.33:0/623098196 shutdown_connections 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.110+0000 7f8325a95740 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6abf160 0x55d8b6ac1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.110+0000 7f8325a95740 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.110+0000 7f8325a95740 1 --2- 172.21.15.33:0/623098196 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aaa480 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.110+0000 7f8325a95740 1 -- 172.21.15.33:0/623098196 >> 172.21.15.33:0/623098196 conn(0x55d8b6aa7ed0 msgr2=0x55d8b6aa9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:37.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.110+0000 7f8325a95740 1 -- 172.21.15.33:0/623098196 shutdown_connections 2024-09-10T21:20:37.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.110+0000 7f8325a95740 1 -- 172.21.15.33:0/623098196 wait complete. 2024-09-10T21:20:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.111+0000 7f8325a95740 1 -- start start 2024-09-10T21:20:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.111+0000 7f8325a95740 1 Processor -- start 2024-09-10T21:20:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8325a95740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6abea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8325a95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8325a95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6abf160 0x55d8b6abc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8325a95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8b6871700 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8325a95740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d8b69ab110 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8325a95740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8b6ac1de0 con 0x55d8b6abf160 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8325a95740 10 client.0 _ll_register_callbacks cb 0x55d8b69c4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8321c5d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6abf160 0x55d8b6abc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8321c5d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6abf160 0x55d8b6abc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:40424/0 (socket says 172.21.15.33:40424) 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.112+0000 7f8321c5d640 1 -- 172.21.15.33:0/1366901954 learned_addr learned my addr 172.21.15.33:0/1366901954 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f832145c640 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6abea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f8321c5d640 1 -- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 msgr2=0x55d8b6abea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f8321c5d640 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6abea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f8321c5d640 1 -- 172.21.15.33:0/1366901954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa9c0 msgr2=0x55d8b6abbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f8321c5d640 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f8321c5d640 1 -- 172.21.15.33:0/1366901954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b69d8c40 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f832145c640 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6abea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f8321c5d640 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6abf160 0x55d8b6abc4c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8314004870 tx=0x7f8314004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9735d332c0a48c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.113+0000 7f8304ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.114+0000 7f8304ff9640 1 -- 172.21.15.33:0/1366901954 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831404d070 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.114+0000 7f8304ff9640 1 -- 172.21.15.33:0/1366901954 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f831402f610 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.114+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8314041070 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.114+0000 7f8304ff9640 1 -- 172.21.15.33:0/1366901954 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f83140318e0 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.114+0000 7f8325a95740 10 client.27775 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.114+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55d8b6acd760 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8304ff9640 1 -- 172.21.15.33:0/1366901954 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8314056070 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8304ff9640 1 -- 172.21.15.33:0/1366901954 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f8314030ae0 con 0x55d8b6abf160 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8325a95740 10 client.27775 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:37.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8325a95740 10 client.27775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8325a95740 20 client.27775 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8325a95740 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6aceab0 0x55d8b6ad0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8325a95740 4 client.27775 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8314041070 con 0x55d8b6aceab0 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.115+0000 7f832145c640 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6aceab0 0x55d8b6ad0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.116+0000 7f8304ff9640 10 client.27775 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.116+0000 7f832145c640 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6aceab0 0x55d8b6ad0ea0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc7ec39a9ac04e78 server_cookie=e755880da35bfbd4 in_seq=0 out_seq=0 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.119+0000 7f8304ff9640 1 -- 172.21.15.33:0/1366901954 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f8314041070 con 0x55d8b6aceab0 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.119+0000 7f8304ff9640 10 client.27775 handle_command_reply: tid=0 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.119+0000 7f8325a95740 1 client.27775 shutdown 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6abf160 msgr2=0x55d8b6abc4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6abf160 0x55d8b6abc4c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8314004870 tx=0x7f8314004600 comp rx=0 tx=0).stop 2024-09-10T21:20:37.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 shutdown_connections 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6aceab0 0x55d8b6ad0ea0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6abf160 0x55d8b6abc4c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6abbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 --2- 172.21.15.33:0/1366901954 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6abea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 >> 172.21.15.33:0/1366901954 conn(0x55d8b6aa7ed0 msgr2=0x55d8b6aada40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8304ff9640 0 client.27775 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 shutdown_connections 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 1 -- 172.21.15.33:0/1366901954 wait complete. 2024-09-10T21:20:37.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.120+0000 7f8325a95740 20 client.27775 trim_cache size 0 max 16384 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 Processor -- start 2024-09-10T21:20:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 -- start start 2024-09-10T21:20:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aa7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6acb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1ac1a0 con 0x55d8b6abf160 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1adce0 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8325a95740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1ac870 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8320c5b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8320c5b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41912/0 (socket says 172.21.15.33:41912) 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8320c5b640 1 -- 172.21.15.33:0/3351669190 learned_addr learned my addr 172.21.15.33:0/3351669190 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f8321c5d640 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6acb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.144+0000 7f832145c640 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aa7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8320c5b640 1 -- 172.21.15.33:0/3351669190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa0a0 msgr2=0x55d8b6aa7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8320c5b640 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aa7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8320c5b640 1 -- 172.21.15.33:0/3351669190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 msgr2=0x55d8b6acb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8320c5b640 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6acb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8320c5b640 1 -- 172.21.15.33:0/3351669190 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8b6abedb0 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8320c5b640 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac15b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f830c009ab0 tx=0x7f830c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35f96b59bc08d0ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8306ffd640 1 -- 172.21.15.33:0/3351669190 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c0120d0 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8306ffd640 1 -- 172.21.15.33:0/3351669190 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830c012270 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.145+0000 7f8306ffd640 1 -- 172.21.15.33:0/3351669190 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f830c01b3c0 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 -- 172.21.15.33:0/3351669190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa9c0 msgr2=0x55d8b6ac15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac15b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f830c009ab0 tx=0x7f830c009f70 comp rx=0 tx=0).stop 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 -- 172.21.15.33:0/3351669190 shutdown_connections 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6acb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac15b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 --2- 172.21.15.33:0/3351669190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6aa7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 -- 172.21.15.33:0/3351669190 >> 172.21.15.33:0/3351669190 conn(0x55d8b6aa7ed0 msgr2=0x55d8b6abd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 -- 172.21.15.33:0/3351669190 shutdown_connections 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.146+0000 7f8325a95740 1 -- 172.21.15.33:0/3351669190 wait complete. 2024-09-10T21:20:37.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.147+0000 7f8325a95740 1 -- start start 2024-09-10T21:20:37.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.147+0000 7f8325a95740 1 Processor -- start 2024-09-10T21:20:37.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.147+0000 7f8325a95740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6ac72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.147+0000 7f8325a95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.147+0000 7f8325a95740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6ac40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6ac72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6ac72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:41928/0 (socket says 172.21.15.33:41928) 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 -- 172.21.15.33:0/3580894851 learned_addr learned my addr 172.21.15.33:0/3580894851 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1ad610 con 0x55d8b6abf160 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1acf40 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f831c1aa1c0 con 0x55d8b6aaa9c0 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f8325a95740 10 client.0 _ll_register_callbacks cb 0x7f831404b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f8320c5b640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f8321c5d640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6ac40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 -- 172.21.15.33:0/3580894851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 msgr2=0x55d8b6ac77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 -- 172.21.15.33:0/3580894851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 msgr2=0x55d8b6ac40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6ac40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f832145c640 1 -- 172.21.15.33:0/3580894851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8b6ac4790 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.148+0000 7f8321c5d640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6ac40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f832145c640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6ac72b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f831800e9c0 tx=0x7f83180e96b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876d898ab30df7fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f8304ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f8304ff9640 1 -- 172.21.15.33:0/3580894851 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318005680 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f8304ff9640 1 -- 172.21.15.33:0/3580894851 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8318005190 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f830c012270 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f8304ff9640 1 -- 172.21.15.33:0/3580894851 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) ==== 24+0+0 (secure 0 0 0) 0x7f83180ac2f0 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f8325a95740 10 client.27887 fetch_fsmap learned FSMap version 78 2024-09-10T21:20:37.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.149+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) -- 0x55d8b6b3c080 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8304ff9640 1 -- 172.21.15.33:0/3580894851 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83180acee0 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8304ff9640 1 -- 172.21.15.33:0/3580894851 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 78) ==== 8378+0+0 (secure 0 0 0) 0x7f83180b58e0 con 0x55d8b6aaa0a0 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8325a95740 10 client.27887 fetch_fsmap finished waiting for FSMap version 78 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8325a95740 10 client.27887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8325a95740 20 client.27887 populate_metadata read hostname 'smithi033' 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8325a95740 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6b3c740 0x55d8b6b3eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8325a95740 4 client.27887 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f830c012270 con 0x55d8b6b3c740 2024-09-10T21:20:37.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.150+0000 7f8320c5b640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6b3c740 0x55d8b6b3eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:20:37.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.152+0000 7f8320c5b640 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6b3c740 0x55d8b6b3eb60 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b62a820e68dda48 server_cookie=b096f950cd3b46be in_seq=0 out_seq=0 2024-09-10T21:20:37.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.152+0000 7f8304ff9640 10 client.27887 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:37.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8304ff9640 1 -- 172.21.15.33:0/3580894851 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6534 (crc 0 0 0) 0x7f830c012270 con 0x55d8b6b3c740 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8304ff9640 10 client.27887 handle_command_reply: tid=0 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 client.27887 shutdown 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 msgr2=0x55d8b6ac72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6ac72b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f831800e9c0 tx=0x7f83180e96b0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 shutdown_connections 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8b6b3c740 0x55d8b6b3eb60 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8b6abf160 0x55d8b6ac40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8b6aaa9c0 0x55d8b6ac77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 --2- 172.21.15.33:0/3580894851 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8b6aaa0a0 0x55d8b6ac72b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 >> 172.21.15.33:0/3580894851 conn(0x55d8b6aa7ed0 msgr2=0x55d8b6abdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.154+0000 7f8304ff9640 0 client.27887 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:20:37.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.155+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 shutdown_connections 2024-09-10T21:20:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.155+0000 7f8325a95740 1 -- 172.21.15.33:0/3580894851 wait complete. 2024-09-10T21:20:37.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.155+0000 7f8325a95740 20 client.27887 trim_cache size 0 max 16384 2024-09-10T21:20:37.160 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:20:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:20:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:20:37.160 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:20:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "941658e5-2c5c-43b5-aa24-7d3c6632ea39", 2024-09-10T21:20:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:20:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:20:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.161 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.163 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.164 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.166 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:20:37.168 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:20:37.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:20:37.173 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:20:37.175 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:20:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:20:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:20:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:20:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:20:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:20:37.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:20:37.176 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:20:37.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.156+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83080b5360 msgr2=0x7f83080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.156+0000 7f83236c0640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83080b5360 0x7f83080b7820 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8314009430 tx=0x7f831404b000 comp rx=0 tx=0).stop 2024-09-10T21:20:37.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c103cd0 msgr2=0x7f831c19b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c103cd0 0x7f831c19b480 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f831800e9f0 tx=0x7f831800edf0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 shutdown_connections 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f83080b5360 0x7f83080b7820 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f831c10d130 0x7f831c1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f831c1045f0 0x7f831c19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 --2- 172.21.15.33:0/422803547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f831c103cd0 0x7f831c19b480 secure :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f831800e9f0 tx=0x7f831800edf0 comp rx=0 tx=0).stop 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.157+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 >> 172.21.15.33:0/422803547 conn(0x7f831c0fd770 msgr2=0x7f831c0fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:20:37.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.158+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 shutdown_connections 2024-09-10T21:20:37.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:20:37.158+0000 7f83236c0640 1 -- 172.21.15.33:0/422803547 wait complete. 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:37.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:20:38.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 180017 42.25 MB/sec execute 98 sec latency 70.415 ms 2024-09-10T21:20:38.109 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:37 smithi033 ceph-mon[30860]: pgmap v845: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 742 op/s 2024-09-10T21:20:38.111 INFO:tasks.cephfs.filesystem:scrub status for tag:941658e5-2c5c-43b5-aa24-7d3c6632ea39 - {'path': '/', 'tag': '941658e5-2c5c-43b5-aa24-7d3c6632ea39', 'options': 'recursive,force'} 2024-09-10T21:20:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:37 smithi174 ceph-mon[35635]: pgmap v845: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 742 op/s 2024-09-10T21:20:38.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:37 smithi104 ceph-mon[35325]: pgmap v845: 97 pgs: 97 active+clean; 1.1 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 742 op/s 2024-09-10T21:20:39.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 180613 42.10 MB/sec execute 99 sec latency 314.685 ms 2024-09-10T21:20:40.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 181419 42.15 MB/sec execute 100 sec latency 307.833 ms 2024-09-10T21:20:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:39 smithi174 ceph-mon[35635]: pgmap v846: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 985 op/s 2024-09-10T21:20:40.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:39 smithi033 ceph-mon[30860]: pgmap v846: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 985 op/s 2024-09-10T21:20:40.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:39 smithi104 ceph-mon[35325]: pgmap v846: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 985 op/s 2024-09-10T21:20:41.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 182309 42.22 MB/sec execute 101 sec latency 128.819 ms 2024-09-10T21:20:42.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 183215 42.36 MB/sec execute 102 sec latency 54.616 ms 2024-09-10T21:20:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:41 smithi174 ceph-mon[35635]: pgmap v847: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 867 op/s 2024-09-10T21:20:42.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:41 smithi033 ceph-mon[30860]: pgmap v847: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 867 op/s 2024-09-10T21:20:42.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:41 smithi104 ceph-mon[35325]: pgmap v847: 97 pgs: 97 active+clean; 1.1 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 867 op/s 2024-09-10T21:20:43.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 184230 42.39 MB/sec execute 103 sec latency 69.309 ms 2024-09-10T21:20:44.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 184630 42.20 MB/sec execute 104 sec latency 475.113 ms 2024-09-10T21:20:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:43 smithi174 ceph-mon[35635]: pgmap v848: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 844 op/s 2024-09-10T21:20:44.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:43 smithi033 ceph-mon[30860]: pgmap v848: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 844 op/s 2024-09-10T21:20:44.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:43 smithi104 ceph-mon[35325]: pgmap v848: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 844 op/s 2024-09-10T21:20:45.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 185454 42.26 MB/sec execute 105 sec latency 70.420 ms 2024-09-10T21:20:46.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 186351 42.23 MB/sec execute 106 sec latency 71.642 ms 2024-09-10T21:20:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:45 smithi174 ceph-mon[35635]: pgmap v849: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:20:46.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:45 smithi033 ceph-mon[30860]: pgmap v849: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:20:46.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:45 smithi104 ceph-mon[35325]: pgmap v849: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 903 op/s 2024-09-10T21:20:47.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 187217 42.37 MB/sec execute 107 sec latency 134.053 ms 2024-09-10T21:20:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:20:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:20:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:20:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:20:47.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:20:47.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:20:47.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:20:47.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:20:47.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:20:47.177 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:20:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:20:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:20:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:20:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:20:47.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:20:48.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 188204 42.43 MB/sec execute 108 sec latency 50.563 ms 2024-09-10T21:20:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:47 smithi174 ceph-mon[35635]: pgmap v850: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-10T21:20:48.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:47 smithi033 ceph-mon[30860]: pgmap v850: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-10T21:20:48.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:47 smithi104 ceph-mon[35325]: pgmap v850: 97 pgs: 97 active+clean; 1.1 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 870 op/s 2024-09-10T21:20:49.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 188677 42.28 MB/sec execute 109 sec latency 368.857 ms 2024-09-10T21:20:50.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 189510 42.26 MB/sec execute 110 sec latency 75.978 ms 2024-09-10T21:20:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:49 smithi174 ceph-mon[35635]: pgmap v851: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:20:50.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:49 smithi033 ceph-mon[30860]: pgmap v851: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:20:50.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:49 smithi104 ceph-mon[35325]: pgmap v851: 97 pgs: 97 active+clean; 1.1 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 1.02k op/s 2024-09-10T21:20:51.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 190540 42.40 MB/sec execute 111 sec latency 51.630 ms 2024-09-10T21:20:52.057 INFO:tasks.workunit.client.0.smithi033.stdout: 10 191347 42.44 MB/sec execute 112 sec latency 285.782 ms 2024-09-10T21:20:52.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:51 smithi033 ceph-mon[30860]: pgmap v852: 97 pgs: 97 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-09-10T21:20:52.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:51 smithi104 ceph-mon[35325]: pgmap v852: 97 pgs: 97 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-09-10T21:20:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:51 smithi174 ceph-mon[35635]: pgmap v852: 97 pgs: 97 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-09-10T21:20:53.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 192162 42.49 MB/sec execute 113 sec latency 74.231 ms 2024-09-10T21:20:54.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 192643 42.28 MB/sec execute 114 sec latency 318.439 ms 2024-09-10T21:20:54.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:53 smithi033 ceph-mon[30860]: pgmap v853: 97 pgs: 97 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-09-10T21:20:54.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:53 smithi104 ceph-mon[35325]: pgmap v853: 97 pgs: 97 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-09-10T21:20:54.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:53 smithi174 ceph-mon[35635]: pgmap v853: 97 pgs: 97 active+clean; 1.1 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 842 op/s 2024-09-10T21:20:55.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 193469 42.27 MB/sec execute 115 sec latency 110.826 ms 2024-09-10T21:20:56.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 194365 42.36 MB/sec execute 116 sec latency 85.072 ms 2024-09-10T21:20:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:55 smithi174 ceph-mon[35635]: pgmap v854: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 979 op/s 2024-09-10T21:20:56.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:55 smithi033 ceph-mon[30860]: pgmap v854: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 979 op/s 2024-09-10T21:20:56.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:55 smithi104 ceph-mon[35325]: pgmap v854: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 979 op/s 2024-09-10T21:20:57.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 195149 42.37 MB/sec execute 117 sec latency 81.198 ms 2024-09-10T21:20:58.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 196180 42.44 MB/sec execute 118 sec latency 50.881 ms 2024-09-10T21:20:58.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:57 smithi033 ceph-mon[30860]: pgmap v855: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 897 op/s 2024-09-10T21:20:58.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:57 smithi104 ceph-mon[35325]: pgmap v855: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 897 op/s 2024-09-10T21:20:58.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:57 smithi174 ceph-mon[35635]: pgmap v855: 97 pgs: 97 active+clean; 1.1 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 897 op/s 2024-09-10T21:20:59.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 196858 42.42 MB/sec execute 119 sec latency 219.416 ms 2024-09-10T21:21:00.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 197632 42.44 MB/sec execute 120 sec latency 79.636 ms 2024-09-10T21:21:00.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:20:59 smithi033 ceph-mon[30860]: pgmap v856: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 958 op/s 2024-09-10T21:21:00.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:20:59 smithi104 ceph-mon[35325]: pgmap v856: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 958 op/s 2024-09-10T21:21:00.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:20:59 smithi174 ceph-mon[35635]: pgmap v856: 97 pgs: 97 active+clean; 1.2 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 958 op/s 2024-09-10T21:21:01.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 198298 42.34 MB/sec execute 121 sec latency 182.974 ms 2024-09-10T21:21:02.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 199209 42.41 MB/sec execute 122 sec latency 278.379 ms 2024-09-10T21:21:02.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:01 smithi033 ceph-mon[30860]: pgmap v857: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 919 op/s 2024-09-10T21:21:02.194 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:01 smithi174 ceph-mon[35635]: pgmap v857: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 919 op/s 2024-09-10T21:21:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:01 smithi104 ceph-mon[35325]: pgmap v857: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 919 op/s 2024-09-10T21:21:03.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 200062 42.48 MB/sec execute 123 sec latency 136.470 ms 2024-09-10T21:21:04.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 200640 42.40 MB/sec execute 124 sec latency 335.960 ms 2024-09-10T21:21:04.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:03 smithi033 ceph-mon[30860]: pgmap v858: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 820 op/s 2024-09-10T21:21:04.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:03 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:04.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:03 smithi104 ceph-mon[35325]: pgmap v858: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 820 op/s 2024-09-10T21:21:04.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:03 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:04.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:03 smithi174 ceph-mon[35635]: pgmap v858: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 820 op/s 2024-09-10T21:21:04.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:03 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:05.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 201211 42.31 MB/sec execute 125 sec latency 108.143 ms 2024-09-10T21:21:06.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 201967 42.20 MB/sec execute 126 sec latency 91.696 ms 2024-09-10T21:21:06.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:05 smithi033 ceph-mon[30860]: pgmap v859: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-09-10T21:21:06.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:05 smithi104 ceph-mon[35325]: pgmap v859: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-09-10T21:21:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:05 smithi174 ceph-mon[35635]: pgmap v859: 97 pgs: 97 active+clean; 1.2 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 934 op/s 2024-09-10T21:21:07.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 202850 42.25 MB/sec execute 127 sec latency 179.702 ms 2024-09-10T21:21:08.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 203745 42.34 MB/sec execute 128 sec latency 166.019 ms 2024-09-10T21:21:08.113 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:21:08.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:07 smithi033 ceph-mon[30860]: pgmap v860: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-10T21:21:08.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:07 smithi104 ceph-mon[35325]: pgmap v860: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-10T21:21:08.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:07 smithi174 ceph-mon[35635]: pgmap v860: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 824 op/s 2024-09-10T21:21:08.509 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:09.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 204414 42.33 MB/sec execute 129 sec latency 238.536 ms 2024-09-10T21:21:09.071 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:08 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 -- 172.21.15.33:0/2014716163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 msgr2=0x7ff128103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 --2- 172.21.15.33:0/2014716163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff128103180 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7ff118009900 tx=0x7ff11802a410 comp rx=0 tx=0).stop 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 -- 172.21.15.33:0/2014716163 shutdown_connections 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 --2- 172.21.15.33:0/2014716163 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff128109130 0x7ff12810f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 --2- 172.21.15.33:0/2014716163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1281087b0 0x7ff128108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 --2- 172.21.15.33:0/2014716163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff128103180 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 -- 172.21.15.33:0/2014716163 >> 172.21.15.33:0/2014716163 conn(0x7ff1280fe850 msgr2=0x7ff128100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 -- 172.21.15.33:0/2014716163 shutdown_connections 2024-09-10T21:21:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.101+0000 7ff12e727640 1 -- 172.21.15.33:0/2014716163 wait complete. 2024-09-10T21:21:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.102+0000 7ff12e727640 1 Processor -- start 2024-09-10T21:21:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.102+0000 7ff12e727640 1 -- start start 2024-09-10T21:21:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.103+0000 7ff12e727640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff1281992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.103+0000 7ff12e727640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1281087b0 0x7ff128199800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.103+0000 7ff127fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff1281992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff12e727640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff128109130 0x7ff12819eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff127fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff1281992c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56344/0 (socket says 172.21.15.33:56344) 2024-09-10T21:21:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff127fff640 1 -- 172.21.15.33:0/2594890639 learned_addr learned my addr 172.21.15.33:0/2594890639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff1277fe640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1281087b0 0x7ff128199800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff128109130 0x7ff12819eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff128112df0 con 0x7ff128102da0 2024-09-10T21:21:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff128112c70 con 0x7ff128109130 2024-09-10T21:21:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.104+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff128112f70 con 0x7ff1281087b0 2024-09-10T21:21:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.105+0000 7ff127fff640 1 -- 172.21.15.33:0/2594890639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1281087b0 msgr2=0x7ff128199800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.105+0000 7ff127fff640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1281087b0 0x7ff128199800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.105+0000 7ff127fff640 1 -- 172.21.15.33:0/2594890639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff128109130 msgr2=0x7ff12819eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.105+0000 7ff127fff640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff128109130 0x7ff12819eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.105+0000 7ff127fff640 1 -- 172.21.15.33:0/2594890639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12819f370 con 0x7ff128102da0 2024-09-10T21:21:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.105+0000 7ff127fff640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff1281992c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff118005940 tx=0x7ff1180040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.106+0000 7ff1257fa640 1 -- 172.21.15.33:0/2594890639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11803f030 con 0x7ff128102da0 2024-09-10T21:21:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.106+0000 7ff1257fa640 1 -- 172.21.15.33:0/2594890639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1180044c0 con 0x7ff128102da0 2024-09-10T21:21:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.106+0000 7ff1257fa640 1 -- 172.21.15.33:0/2594890639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff118035070 con 0x7ff128102da0 2024-09-10T21:21:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.106+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12819f600 con 0x7ff128102da0 2024-09-10T21:21:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.106+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1281a5d20 con 0x7ff128102da0 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 Processor -- start 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 -- start start 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab0a0 0x5560a0cab480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5560a0c21e50 con 0x5560a0cab9c0 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560a0c11280 con 0x5560a0cab0a0 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.110+0000 7ff130afc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560a0c15090 con 0x5560a0cb7950 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff12ccc4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab0a0 0x5560a0cab480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35824/0 (socket says 172.21.15.33:35824) 2024-09-10T21:21:09.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff1277fe640 1 -- 172.21.15.33:0/777199408 learned_addr learned my addr 172.21.15.33:0/777199408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:09.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff1257fa640 1 -- 172.21.15.33:0/2594890639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff11803b9d0 con 0x7ff128102da0 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff12ccc4640 1 -- 172.21.15.33:0/777199408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 msgr2=0x5560a0cb9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.111+0000 7ff12ccc4640 1 -- 172.21.15.33:0/777199408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 msgr2=0x5560a0cb7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.112+0000 7ff127fff640 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.112+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.112+0000 7ff12ccc4640 1 -- 172.21.15.33:0/777199408 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a096f650 con 0x5560a0cab0a0 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.112+0000 7ff127fff640 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:21:09.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.112+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab0a0 0x5560a0cab480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff120009f00 tx=0x7ff120011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=591def8d943e0969 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.113+0000 7ff10e7fc640 1 -- 172.21.15.33:0/777199408 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120004d20 con 0x5560a0cab0a0 2024-09-10T21:21:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.113+0000 7ff10e7fc640 1 -- 172.21.15.33:0/777199408 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff120004030 con 0x5560a0cab0a0 2024-09-10T21:21:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.113+0000 7ff10e7fc640 1 -- 172.21.15.33:0/777199408 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120005260 con 0x5560a0cab0a0 2024-09-10T21:21:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.113+0000 7ff1257fa640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1100b5360 0x7ff1100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.113+0000 7ff127fff640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1100b5360 0x7ff1100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.113+0000 7ff1257fa640 1 -- 172.21.15.33:0/2594890639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff118106230 con 0x7ff128102da0 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 -- 172.21.15.33:0/777199408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab0a0 msgr2=0x5560a0cab480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab0a0 0x5560a0cab480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff120009f00 tx=0x7ff120011810 comp rx=0 tx=0).stop 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff127fff640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1100b5360 0x7ff1100b7820 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff118031440 tx=0x7ff118005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 -- 172.21.15.33:0/777199408 shutdown_connections 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 --2- 172.21.15.33:0/777199408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab0a0 0x5560a0cab480 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 -- 172.21.15.33:0/777199408 >> 172.21.15.33:0/777199408 conn(0x5560a0ca8ed0 msgr2=0x5560a0caa070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.114+0000 7ff130afc740 1 -- 172.21.15.33:0/777199408 shutdown_connections 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.115+0000 7ff130afc740 1 -- 172.21.15.33:0/777199408 wait complete. 2024-09-10T21:21:09.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.115+0000 7ff130afc740 1 -- start start 2024-09-10T21:21:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.115+0000 7ff130afc740 1 Processor -- start 2024-09-10T21:21:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff130afc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0cb7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff130afc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab9c0 0x5560a0cb4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff130afc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff130afc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5560a0a72700 con 0x5560a0cab0a0 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff130afc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5560a0bac110 con 0x5560a0cab9c0 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff130afc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560a0cba5d0 con 0x5560a0cb7950 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff130afc740 10 client.0 _ll_register_callbacks cb 0x5560a0bc5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff12ccc4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0cb7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff12ccc4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0cb7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56350/0 (socket says 172.21.15.33:56350) 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff12ccc4640 1 -- 172.21.15.33:0/1198289760 learned_addr learned my addr 172.21.15.33:0/1198289760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.116+0000 7ff1277fe640 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab9c0 0x5560a0cb4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff12ccc4640 1 -- 172.21.15.33:0/1198289760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 msgr2=0x5560a0cb4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff12ccc4640 1 -- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab9c0 msgr2=0x5560a0cb4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab9c0 0x5560a0cb4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff12ccc4640 1 -- 172.21.15.33:0/1198289760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a0bd9c40 con 0x5560a0cab0a0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0cb7240 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff120003fc0 tx=0x7ff120004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2005a24aa3adad2e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff1077fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.117+0000 7ff1077fe640 1 -- 172.21.15.33:0/1198289760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff12002d060 con 0x5560a0cab0a0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.118+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff120004030 con 0x5560a0cab0a0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.118+0000 7ff1077fe640 1 -- 172.21.15.33:0/1198289760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff120004c40 con 0x5560a0cab0a0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.118+0000 7ff1077fe640 1 -- 172.21.15.33:0/1198289760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7ff120004e40 con 0x5560a0cab0a0 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.118+0000 7ff130afc740 10 client.18492 fetch_fsmap learned FSMap version 80 2024-09-10T21:21:09.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.118+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5560a0cca2b0 con 0x5560a0cab0a0 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.118+0000 7ff1077fe640 1 -- 172.21.15.33:0/1198289760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff120004030 con 0x5560a0cab0a0 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff1077fe640 1 -- 172.21.15.33:0/1198289760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 80) ==== 8366+0+0 (secure 0 0 0) 0x7ff120032070 con 0x5560a0cab0a0 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff130afc740 10 client.18492 fetch_fsmap finished waiting for FSMap version 80 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff130afc740 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff130afc740 20 client.18492 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff130afc740 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0ccba40 0x5560a0ccde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff130afc740 4 client.18492 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560a0c23da0 con 0x5560a0ccba40 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.119+0000 7ff1277fe640 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0ccba40 0x5560a0ccde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.120+0000 7ff1277fe640 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0ccba40 0x5560a0ccde60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af5098c0eb79abcf server_cookie=b0165c4e57ccf3 in_seq=0 out_seq=0 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.120+0000 7ff1077fe640 10 client.18492 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff1077fe640 1 -- 172.21.15.33:0/1198289760 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5560a0c23da0 con 0x5560a0ccba40 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff1077fe640 10 client.18492 handle_command_reply: tid=0 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 client.18492 shutdown 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 msgr2=0x5560a0cb7240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0cb7240 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7ff120003fc0 tx=0x7ff120004910 comp rx=0 tx=0).stop 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 shutdown_connections 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0ccba40 0x5560a0ccde60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cb7950 0x5560a0cb4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cab9c0 0x5560a0cb4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 --2- 172.21.15.33:0/1198289760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0cb7240 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.122+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 >> 172.21.15.33:0/1198289760 conn(0x5560a0ca8ed0 msgr2=0x5560a0caea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.123+0000 7ff1077fe640 0 client.18492 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.123+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 shutdown_connections 2024-09-10T21:21:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.123+0000 7ff130afc740 1 -- 172.21.15.33:0/1198289760 wait complete. 2024-09-10T21:21:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.123+0000 7ff130afc740 20 client.18492 trim_cache size 0 max 16384 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 Processor -- start 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 -- start start 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0ca3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab9c0 0x5560a0cb8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cb9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281ac1d0 con 0x5560a0cab0a0 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281add10 con 0x5560a0cb7950 2024-09-10T21:21:09.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.148+0000 7ff130afc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281ac8a0 con 0x5560a0cab9c0 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab9c0 0x5560a0cb8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab9c0 0x5560a0cb8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35844/0 (socket says 172.21.15.33:35844) 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 -- 172.21.15.33:0/1672587960 learned_addr learned my addr 172.21.15.33:0/1672587960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0ca3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cb9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 -- 172.21.15.33:0/1672587960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 msgr2=0x5560a0cb9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cb9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 -- 172.21.15.33:0/1672587960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 msgr2=0x5560a0ca3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0ca3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 -- 172.21.15.33:0/1672587960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a0cb5030 con 0x5560a0cab9c0 2024-09-10T21:21:09.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.149+0000 7ff1277fe640 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab9c0 0x5560a0cb8d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009ad0 tx=0x7ff11c004410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601020be856c9d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff10dffb640 1 -- 172.21.15.33:0/1672587960 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c003d90 con 0x5560a0cab9c0 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff10dffb640 1 -- 172.21.15.33:0/1672587960 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11c01e8e0 con 0x5560a0cab9c0 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff10dffb640 1 -- 172.21.15.33:0/1672587960 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c012870 con 0x5560a0cab9c0 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff130afc740 1 -- 172.21.15.33:0/1672587960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab9c0 msgr2=0x5560a0cb8d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff130afc740 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab9c0 0x5560a0cb8d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff11c009ad0 tx=0x7ff11c004410 comp rx=0 tx=0).stop 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff130afc740 1 -- 172.21.15.33:0/1672587960 shutdown_connections 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff130afc740 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cb9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff130afc740 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab9c0 0x5560a0cb8d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff130afc740 1 --2- 172.21.15.33:0/1672587960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab0a0 0x5560a0ca3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.150+0000 7ff130afc740 1 -- 172.21.15.33:0/1672587960 >> 172.21.15.33:0/1672587960 conn(0x5560a0ca8ed0 msgr2=0x5560a0cb2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.151+0000 7ff130afc740 1 -- 172.21.15.33:0/1672587960 shutdown_connections 2024-09-10T21:21:09.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.151+0000 7ff130afc740 1 -- 172.21.15.33:0/1672587960 wait complete. 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.151+0000 7ff130afc740 1 -- start start 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.151+0000 7ff130afc740 1 Processor -- start 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff130afc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab0a0 0x5560a0cb1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff130afc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff130afc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cbf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff12ccc4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab0a0 0x5560a0cb1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff130afc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281ad640 con 0x5560a0cab9c0 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff12ccc4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab0a0 0x5560a0cb1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:35854/0 (socket says 172.21.15.33:35854) 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff12ccc4640 1 -- 172.21.15.33:0/2056217807 learned_addr learned my addr 172.21.15.33:0/2056217807 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:09.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff1277fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56356/0 (socket says 172.21.15.33:56356) 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281acf70 con 0x5560a0cb7950 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1281aa190 con 0x5560a0cab0a0 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff130afc740 10 client.0 _ll_register_callbacks cb 0x7ff120009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.152+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cbf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff12ccc4640 1 -- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 msgr2=0x5560a0cbf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cbf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff12ccc4640 1 -- 172.21.15.33:0/2056217807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 msgr2=0x5560a0cb2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff12ccc4640 1 -- 172.21.15.33:0/2056217807 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a0cbffc0 con 0x5560a0cab0a0 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff12ccc4640 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab0a0 0x5560a0cb1be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff120009270 tx=0x7ff120000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15a053934b4602c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff1077fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:21:09.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff1077fe640 1 -- 172.21.15.33:0/2056217807 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120043060 con 0x5560a0cab0a0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff11c01e8e0 con 0x5560a0cab0a0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff1077fe640 1 -- 172.21.15.33:0/2056217807 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff1200051e0 con 0x5560a0cab0a0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.153+0000 7ff1077fe640 1 -- 172.21.15.33:0/2056217807 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7ff120038040 con 0x5560a0cab0a0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.154+0000 7ff130afc740 10 client.27799 fetch_fsmap learned FSMap version 80 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.154+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5560a0cb1330 con 0x5560a0cab0a0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.154+0000 7ff1077fe640 1 -- 172.21.15.33:0/2056217807 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff11c01e8e0 con 0x5560a0cab0a0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff1077fe640 1 -- 172.21.15.33:0/2056217807 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 80) ==== 8366+0+0 (secure 0 0 0) 0x7ff120048070 con 0x5560a0cab0a0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff130afc740 10 client.27799 fetch_fsmap finished waiting for FSMap version 80 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff130afc740 10 client.27799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff130afc740 20 client.27799 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff130afc740 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0d3d2f0 0x5560a0d3f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff130afc740 4 client.27799 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560a0c1f850 con 0x5560a0d3d2f0 2024-09-10T21:21:09.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.155+0000 7ff1277fe640 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0d3d2f0 0x5560a0d3f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.156+0000 7ff1077fe640 10 client.27799 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.156+0000 7ff1277fe640 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0d3d2f0 0x5560a0d3f710 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e0c9ce3e3eec8da server_cookie=10eda7db0197e952 in_seq=0 out_seq=0 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.156+0000 7ff1077fe640 1 -- 172.21.15.33:0/2056217807 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6338 (crc 0 0 0) 0x5560a0c1f850 con 0x5560a0d3d2f0 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.156+0000 7ff1077fe640 10 client.27799 handle_command_reply: tid=0 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.156+0000 7ff130afc740 1 client.27799 shutdown 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab0a0 msgr2=0x5560a0cb1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab0a0 0x5560a0cb1be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff120009270 tx=0x7ff120000f40 comp rx=0 tx=0).stop 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 shutdown_connections 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5560a0d3d2f0 0x5560a0d3f710 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff1077fe640 0 client.27799 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5560a0cb7950 0x5560a0cbf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5560a0cab9c0 0x5560a0cb2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 --2- 172.21.15.33:0/2056217807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a0cab0a0 0x5560a0cb1be0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 >> 172.21.15.33:0/2056217807 conn(0x5560a0ca8ed0 msgr2=0x5560a0ca9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.157+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 shutdown_connections 2024-09-10T21:21:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.158+0000 7ff130afc740 1 -- 172.21.15.33:0/2056217807 wait complete. 2024-09-10T21:21:09.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.158+0000 7ff130afc740 20 client.27799 trim_cache size 0 max 16384 2024-09-10T21:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:21:09.165 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:21:09.167 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.169 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:21:09.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:21:09.174 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:21:09.178 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:09.179 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:21:09.180 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:21:09.180 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:21:09.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1100b5360 msgr2=0x7ff1100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1100b5360 0x7ff1100b7820 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff118031440 tx=0x7ff118005fd0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 msgr2=0x7ff1281992c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff1281992c0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7ff118005940 tx=0x7ff1180040c0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 shutdown_connections 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff1100b5360 0x7ff1100b7820 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff128109130 0x7ff12819eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1281087b0 0x7ff128199800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 --2- 172.21.15.33:0/2594890639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff128102da0 0x7ff1281992c0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 >> 172.21.15.33:0/2594890639 conn(0x7ff1280fe850 msgr2=0x7ff128100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 shutdown_connections 2024-09-10T21:21:09.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:09.159+0000 7ff12e727640 1 -- 172.21.15.33:0/2594890639 wait complete. 2024-09-10T21:21:09.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:08 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:08 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:10.058 INFO:tasks.workunit.client.0.smithi033.stdout: 10 205020 42.25 MB/sec execute 130 sec latency 76.235 ms 2024-09-10T21:21:10.101 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:21:10.101 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:21:10.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:09 smithi104 ceph-mon[35325]: pgmap v861: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-10T21:21:10.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:09 smithi174 ceph-mon[35635]: pgmap v861: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-10T21:21:10.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:09 smithi033 ceph-mon[30860]: pgmap v861: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-10T21:21:10.506 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:11.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 205757 42.22 MB/sec execute 131 sec latency 70.780 ms 2024-09-10T21:21:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.072+0000 7f321b19f640 1 -- 172.21.15.33:0/1000433131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214108030 msgr2=0x7f321410e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.072+0000 7f321b19f640 1 --2- 172.21.15.33:0/1000433131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214108030 0x7f321410e830 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f320c009be0 tx=0x7f320c02a6c0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.072+0000 7f321b19f640 1 -- 172.21.15.33:0/1000433131 shutdown_connections 2024-09-10T21:21:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.072+0000 7f321b19f640 1 --2- 172.21.15.33:0/1000433131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214108030 0x7f321410e830 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.072+0000 7f321b19f640 1 --2- 172.21.15.33:0/1000433131 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32141076b0 0x7f3214107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.072+0000 7f321b19f640 1 --2- 172.21.15.33:0/1000433131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3214101ca0 0x7f3214102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.072+0000 7f321b19f640 1 -- 172.21.15.33:0/1000433131 >> 172.21.15.33:0/1000433131 conn(0x7f32140fd6f0 msgr2=0x7f32140ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.073+0000 7f321b19f640 1 -- 172.21.15.33:0/1000433131 shutdown_connections 2024-09-10T21:21:11.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.073+0000 7f321b19f640 1 -- 172.21.15.33:0/1000433131 wait complete. 2024-09-10T21:21:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.074+0000 7f321b19f640 1 Processor -- start 2024-09-10T21:21:11.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.074+0000 7f321b19f640 1 -- start start 2024-09-10T21:21:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.074+0000 7f321b19f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214101ca0 0x7f3214199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:11.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f321b19f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32141076b0 0x7f3214199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f321b19f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3214108030 0x7f321419eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f321b19f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3214112e10 con 0x7f3214108030 2024-09-10T21:21:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f321b19f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3214112c90 con 0x7f3214101ca0 2024-09-10T21:21:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f321b19f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3214112f90 con 0x7f32141076b0 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214101ca0 0x7f3214199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214101ca0 0x7f3214199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38040/0 (socket says 172.21.15.33:38040) 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 -- 172.21.15.33:0/3034595730 learned_addr learned my addr 172.21.15.33:0/3034595730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f321973c640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3214108030 0x7f321419eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 -- 172.21.15.33:0/3034595730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32141076b0 msgr2=0x7f3214199760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32141076b0 0x7f3214199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 -- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3214108030 msgr2=0x7f321419eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3214108030 0x7f321419eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.075+0000 7f3218f3b640 1 -- 172.21.15.33:0/3034595730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321419f2d0 con 0x7f3214101ca0 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.076+0000 7f3218f3b640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214101ca0 0x7f3214199220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f320800b2f0 tx=0x7f320800b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.076+0000 7f3211ffb640 1 -- 172.21.15.33:0/3034595730 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208022070 con 0x7f3214101ca0 2024-09-10T21:21:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.076+0000 7f3211ffb640 1 -- 172.21.15.33:0/3034595730 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f320801b9d0 con 0x7f3214101ca0 2024-09-10T21:21:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.076+0000 7f3211ffb640 1 -- 172.21.15.33:0/3034595730 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320801da90 con 0x7f3214101ca0 2024-09-10T21:21:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.076+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f321419f5c0 con 0x7f3214101ca0 2024-09-10T21:21:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.077+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f321410f220 con 0x7f3214101ca0 2024-09-10T21:21:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.078+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3214102080 con 0x7f3214101ca0 2024-09-10T21:21:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.080+0000 7f3211ffb640 1 -- 172.21.15.33:0/3034595730 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f320801dc70 con 0x7f3214101ca0 2024-09-10T21:21:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.081+0000 7f3211ffb640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f32040b54b0 0x7f32040b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.082+0000 7f3211ffb640 1 -- 172.21.15.33:0/3034595730 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f32080edb30 con 0x7f3214101ca0 2024-09-10T21:21:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.082+0000 7f3213fff640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f32040b54b0 0x7f32040b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.083+0000 7f3213fff640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f32040b54b0 0x7f32040b7970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3200005ab0 tx=0x7f3200005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.087+0000 7f3211ffb640 1 -- 172.21.15.33:0/3034595730 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f32080b49b0 con 0x7f3214101ca0 2024-09-10T21:21:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.442+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f321419a350 con 0x7f3214101ca0 2024-09-10T21:21:11.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.443+0000 7f3211ffb640 1 -- 172.21.15.33:0/3034595730 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+11974 (secure 0 0 0) 0x7f32080bd160 con 0x7f3214101ca0 2024-09-10T21:21:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:21:11.447 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":80,"btime":"2024-09-10T21:21:07:916733+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":80,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:21:06.997200+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:21:11.447 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 80 2024-09-10T21:21:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f32040b54b0 msgr2=0x7f32040b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f32040b54b0 0x7f32040b7970 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3200005ab0 tx=0x7f3200005a20 comp rx=0 tx=0).stop 2024-09-10T21:21:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214101ca0 msgr2=0x7f3214199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214101ca0 0x7f3214199220 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f320800b2f0 tx=0x7f320800b7b0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 shutdown_connections 2024-09-10T21:21:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f32040b54b0 0x7f32040b7970 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3214108030 0x7f321419eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32141076b0 0x7f3214199760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 --2- 172.21.15.33:0/3034595730 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3214101ca0 0x7f3214199220 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:11.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 >> 172.21.15.33:0/3034595730 conn(0x7f32140fd6f0 msgr2=0x7f32140ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:11.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.446+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 shutdown_connections 2024-09-10T21:21:11.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:11.447+0000 7f321b19f640 1 -- 172.21.15.33:0/3034595730 wait complete. 2024-09-10T21:21:12.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 206541 42.19 MB/sec execute 132 sec latency 109.359 ms 2024-09-10T21:21:12.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:11 smithi033 ceph-mon[30860]: pgmap v862: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 898 op/s 2024-09-10T21:21:12.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:11 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3034595730' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:21:12.231 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:21:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:11 smithi104 ceph-mon[35325]: pgmap v862: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 898 op/s 2024-09-10T21:21:12.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:11 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3034595730' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:21:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:11 smithi174 ceph-mon[35635]: pgmap v862: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 898 op/s 2024-09-10T21:21:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:11 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3034595730' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:21:12.594 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:13.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 207298 42.17 MB/sec execute 133 sec latency 135.333 ms 2024-09-10T21:21:14.001 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.455+0000 7fa2f0eec640 1 -- 172.21.15.33:0/1288195153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4040770 con 0x7fa2ec106ed0 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.455+0000 7fa2f4152640 1 -- 172.21.15.33:0/1288195153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec106ed0 msgr2=0x7fa2ec107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.455+0000 7fa2f4152640 1 --2- 172.21.15.33:0/1288195153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec107320 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa2e401a910 tx=0x7fa2e403b430 comp rx=0 tx=0).stop 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.456+0000 7fa2f4152640 1 -- 172.21.15.33:0/1288195153 shutdown_connections 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.456+0000 7fa2f4152640 1 --2- 172.21.15.33:0/1288195153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec107320 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.456+0000 7fa2f4152640 1 --2- 172.21.15.33:0/1288195153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2ec1023b0 0x7fa2ec106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.456+0000 7fa2f4152640 1 --2- 172.21.15.33:0/1288195153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec101a90 0x7fa2ec101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.456+0000 7fa2f4152640 1 -- 172.21.15.33:0/1288195153 >> 172.21.15.33:0/1288195153 conn(0x7fa2ec0fd700 msgr2=0x7fa2ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:14.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.456+0000 7fa2f4152640 1 -- 172.21.15.33:0/1288195153 shutdown_connections 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.457+0000 7fa2f4152640 1 -- 172.21.15.33:0/1288195153 wait complete. 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.457+0000 7fa2f4152640 1 Processor -- start 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f4152640 1 -- start start 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f4152640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec101a90 0x7fa2ec111ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f4152640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2ec1023b0 0x7fa2ec1123e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f4152640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f4152640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec10a6c0 con 0x7fa2ec101a90 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f4152640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec10a540 con 0x7fa2ec106ed0 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f4152640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2ec10a840 con 0x7fa2ec1023b0 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f26ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f26ef640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38060/0 (socket says 172.21.15.33:38060) 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f26ef640 1 -- 172.21.15.33:0/2788647737 learned_addr learned my addr 172.21.15.33:0/2788647737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:14.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec101a90 0x7fa2ec111ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.458+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2ec1023b0 0x7fa2ec1123e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.459+0000 7fa2f26ef640 1 -- 172.21.15.33:0/2788647737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2ec1023b0 msgr2=0x7fa2ec1123e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.459+0000 7fa2f26ef640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2ec1023b0 0x7fa2ec1123e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.459+0000 7fa2f26ef640 1 -- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec101a90 msgr2=0x7fa2ec111ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.459+0000 7fa2f26ef640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec101a90 0x7fa2ec111ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.459+0000 7fa2f26ef640 1 -- 172.21.15.33:0/2788647737 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2ec19d0d0 con 0x7fa2ec106ed0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.459+0000 7fa2f26ef640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec19c950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40048f0 tx=0x7fa2e4004920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.459+0000 7fa2e2ffd640 1 -- 172.21.15.33:0/2788647737 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e404f070 con 0x7fa2ec106ed0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.460+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2ec19d360 con 0x7fa2ec106ed0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.460+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2ec1a3ad0 con 0x7fa2ec106ed0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.460+0000 7fa2e2ffd640 1 -- 172.21.15.33:0/2788647737 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e40409c0 con 0x7fa2ec106ed0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.460+0000 7fa2e2ffd640 1 -- 172.21.15.33:0/2788647737 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e4046030 con 0x7fa2ec106ed0 2024-09-10T21:21:14.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.463+0000 7fa2f6527740 1 Processor -- start 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.464+0000 7fa2f6527740 1 -- start start 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e815a10 0x56458e821460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e8219a0 0x56458e823d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f6527740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56458e78bea0 con 0x56458e8219a0 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f6527740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56458e77b2d0 con 0x56458e8150f0 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f6527740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56458e77f0e0 con 0x56458e815a10 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f1eee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e815a10 0x56458e821460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f1eee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e815a10 0x56458e821460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46930/0 (socket says 172.21.15.33:46930) 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.465+0000 7fa2f1eee640 1 -- 172.21.15.33:0/1264848363 learned_addr learned my addr 172.21.15.33:0/1264848363 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.466+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.466+0000 7fa2f1eee640 1 -- 172.21.15.33:0/1264848363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 msgr2=0x56458e8154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.466+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.466+0000 7fa2f1eee640 1 -- 172.21.15.33:0/1264848363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e8219a0 msgr2=0x56458e823d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.466+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e8219a0 0x56458e823d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.466+0000 7fa2f1eee640 1 -- 172.21.15.33:0/1264848363 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458e4d9650 con 0x56458e815a10 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.467+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e815a10 0x56458e821460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ac20 tx=0x7fa2dc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90771f34b7694267 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.468+0000 7fa2d7fff640 1 -- 172.21.15.33:0/1264848363 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc01d070 con 0x56458e815a10 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.468+0000 7fa2d7fff640 1 -- 172.21.15.33:0/1264848363 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2dc018af0 con 0x56458e815a10 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.468+0000 7fa2d7fff640 1 -- 172.21.15.33:0/1264848363 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc010ba0 con 0x56458e815a10 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2e2ffd640 1 -- 172.21.15.33:0/2788647737 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa2e4005340 con 0x7fa2ec106ed0 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2f6527740 1 -- 172.21.15.33:0/1264848363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e815a10 msgr2=0x56458e821460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2f6527740 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e815a10 0x56458e821460 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc00ac20 tx=0x7fa2dc00f790 comp rx=0 tx=0).stop 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2f6527740 1 -- 172.21.15.33:0/1264848363 shutdown_connections 2024-09-10T21:21:14.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2f6527740 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e8219a0 0x56458e823d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2f6527740 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e815a10 0x56458e821460 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2f6527740 1 --2- 172.21.15.33:0/1264848363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.469+0000 7fa2f6527740 1 -- 172.21.15.33:0/1264848363 >> 172.21.15.33:0/1264848363 conn(0x56458e812f20 msgr2=0x56458e8140c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.470+0000 7fa2f6527740 1 -- 172.21.15.33:0/1264848363 shutdown_connections 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.470+0000 7fa2f6527740 1 -- 172.21.15.33:0/1264848363 wait complete. 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.470+0000 7fa2f6527740 1 -- start start 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.470+0000 7fa2f6527740 1 Processor -- start 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2e2ffd640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2d80b5330 0x7fa2d80b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2e2ffd640 1 -- 172.21.15.33:0/2788647737 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa2e4116d40 con 0x7fa2ec106ed0 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8212a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2d80b5330 0x7fa2d80b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f6527740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56458e5dc700 con 0x56458e815a10 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f6527740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56458e716160 con 0x56458e8150f0 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f6527740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56458e824620 con 0x56458e8219a0 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f6527740 10 client.0 _ll_register_callbacks cb 0x56458e72f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f16ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f16ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38280/0 (socket says 172.21.15.33:38280) 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f16ed640 1 -- 172.21.15.33:0/208026798 learned_addr learned my addr 172.21.15.33:0/208026798 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.471+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f1eee640 1 -- 172.21.15.33:0/208026798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 msgr2=0x56458e81ed00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81ed00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f1eee640 1 -- 172.21.15.33:0/208026798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 msgr2=0x56458e81e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f1eee640 1 -- 172.21.15.33:0/208026798 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458e743c90 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2d80b5330 0x7fa2d80b77f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800afd0 tx=0x7fa2e8013b20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8212a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc010880 tx=0x7fa2dc018670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c897df10330f08df server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2d57fa640 1 -- 172.21.15.33:0/208026798 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2dc027030 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2dc018af0 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.472+0000 7fa2d57fa640 1 -- 172.21.15.33:0/208026798 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa2dc0187e0 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2d57fa640 1 -- 172.21.15.33:0/208026798 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa2dc01d040 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 10 client.27920 fetch_fsmap learned FSMap version 80 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x56458e82ffe0 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2d57fa640 1 -- 172.21.15.33:0/208026798 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2dc018af0 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2d57fa640 1 -- 172.21.15.33:0/208026798 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 80) ==== 8366+0+0 (secure 0 0 0) 0x7fa2dc010bf0 con 0x56458e8150f0 2024-09-10T21:21:14.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 10 client.27920 fetch_fsmap finished waiting for FSMap version 80 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 10 client.27920 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 20 client.27920 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e8318c0 0x56458e833d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 4 client.27920 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56458e78ddf0 con 0x56458e8318c0 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.473+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e8318c0 0x56458e833d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.474+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e8318c0 0x56458e833d80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=19c02260c321ef1b server_cookie=b44b6012aff6449f in_seq=0 out_seq=0 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.474+0000 7fa2d57fa640 10 client.27920 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2d57fa640 1 -- 172.21.15.33:0/208026798 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56458e78ddf0 con 0x56458e8318c0 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2d57fa640 10 client.27920 handle_command_reply: tid=0 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 client.27920 shutdown 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 msgr2=0x56458e8212a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8212a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc010880 tx=0x7fa2dc018670 comp rx=0 tx=0).stop 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 shutdown_connections 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e8318c0 0x56458e833d80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 --2- 172.21.15.33:0/208026798 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e8212a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 >> 172.21.15.33:0/208026798 conn(0x56458e812f20 msgr2=0x56458e818a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2d57fa640 0 client.27920 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 shutdown_connections 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.477+0000 7fa2f6527740 1 -- 172.21.15.33:0/208026798 wait complete. 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.478+0000 7fa2f6527740 20 client.27920 trim_cache size 0 max 16384 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 Processor -- start 2024-09-10T21:21:14.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 -- start start 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e812760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000e4b0 con 0x56458e815a10 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000fff0 con 0x56458e8150f0 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f6527740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000eb80 con 0x56458e8219a0 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f16ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f1eee640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e812760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f16ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38288/0 (socket says 172.21.15.33:38288) 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f16ed640 1 -- 172.21.15.33:0/2516717431 learned_addr learned my addr 172.21.15.33:0/2516717431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f1eee640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e812760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38092/0 (socket says 172.21.15.33:38092) 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.498+0000 7fa2f26ef640 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f16ed640 1 -- 172.21.15.33:0/2516717431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 msgr2=0x56458e81d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f16ed640 1 -- 172.21.15.33:0/2516717431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 msgr2=0x56458e812760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e812760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f16ed640 1 -- 172.21.15.33:0/2516717431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56458e80e5c0 con 0x56458e815a10 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f26ef640 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e812760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.499+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81cf70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800a990 tx=0x7fa2e801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218a3515eb1dfe6a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.500+0000 7fa2d77fe640 1 -- 172.21.15.33:0/2516717431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8003970 con 0x56458e815a10 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.500+0000 7fa2d77fe640 1 -- 172.21.15.33:0/2516717431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e8004020 con 0x56458e815a10 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.500+0000 7fa2d77fe640 1 -- 172.21.15.33:0/2516717431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8010670 con 0x56458e815a10 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 -- 172.21.15.33:0/2516717431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 msgr2=0x56458e81cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81cf70 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800a990 tx=0x7fa2e801d000 comp rx=0 tx=0).stop 2024-09-10T21:21:14.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 -- 172.21.15.33:0/2516717431 shutdown_connections 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8219a0 0x56458e81d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e81cf70 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 --2- 172.21.15.33:0/2516717431 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8150f0 0x56458e812760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 -- 172.21.15.33:0/2516717431 >> 172.21.15.33:0/2516717431 conn(0x56458e812f20 msgr2=0x56458e819950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 -- 172.21.15.33:0/2516717431 shutdown_connections 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.501+0000 7fa2f6527740 1 -- 172.21.15.33:0/2516717431 wait complete. 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.502+0000 7fa2f6527740 1 -- start start 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.502+0000 7fa2f6527740 1 Processor -- start 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.503+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8150f0 0x56458e8387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.503+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e838cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.503+0000 7fa2f6527740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8219a0 0x56458e839230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.503+0000 7fa2f6527740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000f920 con 0x56458e815a10 2024-09-10T21:21:14.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.503+0000 7fa2f6527740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000f250 con 0x56458e8219a0 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.503+0000 7fa2f6527740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d000c470 con 0x56458e8150f0 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.503+0000 7fa2f6527740 10 client.0 _ll_register_callbacks cb 0x7fa2dc01b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e838cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e838cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38304/0 (socket says 172.21.15.33:38304) 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 -- 172.21.15.33:0/504500065 learned_addr learned my addr 172.21.15.33:0/504500065 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8150f0 0x56458e8387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f26ef640 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8219a0 0x56458e839230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 -- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8150f0 msgr2=0x56458e8387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8150f0 0x56458e8387b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 -- 172.21.15.33:0/504500065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8219a0 msgr2=0x56458e839230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8219a0 0x56458e839230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.504+0000 7fa2f16ed640 1 -- 172.21.15.33:0/504500065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56458e828be0 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.505+0000 7fa2f16ed640 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e838cf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa2e802b040 tx=0x7fa2e8012300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210da9fe0bf0c2a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.505+0000 7fa2d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.505+0000 7fa2d57fa640 1 -- 172.21.15.33:0/504500065 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8004c50 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2d57fa640 1 -- 172.21.15.33:0/504500065 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa2e8004df0 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2e8004020 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2d57fa640 1 -- 172.21.15.33:0/504500065 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fa2e8003970 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2f6527740 10 client.18519 fetch_fsmap learned FSMap version 80 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x56458e829ee0 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2d57fa640 1 -- 172.21.15.33:0/504500065 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2e80041d0 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2d57fa640 1 -- 172.21.15.33:0/504500065 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 80) ==== 8366+0+0 (secure 0 0 0) 0x7fa2e803b020 con 0x56458e815a10 2024-09-10T21:21:14.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2f6527740 10 client.18519 fetch_fsmap finished waiting for FSMap version 80 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.506+0000 7fa2f6527740 10 client.18519 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.507+0000 7fa2f6527740 20 client.18519 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.507+0000 7fa2f6527740 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e89e850 0x56458e8a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.507+0000 7fa2f6527740 4 client.18519 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.507+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa2e8004020 con 0x56458e89e850 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.507+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e89e850 0x56458e8a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.509+0000 7fa2d57fa640 10 client.18519 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.509+0000 7fa2f1eee640 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e89e850 0x56458e8a0c40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd082129cd54e33b server_cookie=d12026c4e7256af2 in_seq=0 out_seq=0 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.510+0000 7fa2d57fa640 1 -- 172.21.15.33:0/504500065 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa2e8004020 con 0x56458e89e850 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.510+0000 7fa2d57fa640 10 client.18519 handle_command_reply: tid=0 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.510+0000 7fa2f6527740 1 client.18519 shutdown 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.510+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 msgr2=0x56458e838cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.510+0000 7fa2f6527740 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e838cf0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa2e802b040 tx=0x7fa2e8012300 comp rx=0 tx=0).stop 2024-09-10T21:21:14.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 shutdown_connections 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56458e89e850 0x56458e8a0c40 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56458e8219a0 0x56458e839230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56458e815a10 0x56458e838cf0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 --2- 172.21.15.33:0/504500065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56458e8150f0 0x56458e8387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 >> 172.21.15.33:0/504500065 conn(0x56458e812f20 msgr2=0x56458e81cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2d57fa640 0 client.18519 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 shutdown_connections 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 1 -- 172.21.15.33:0/504500065 wait complete. 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.511+0000 7fa2f6527740 20 client.18519 trim_cache size 0 max 16384 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.512+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2d80b5330 msgr2=0x7fa2d80b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.512+0000 7fa2f4152640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2d80b5330 0x7fa2d80b77f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa2e800afd0 tx=0x7fa2e8013b20 comp rx=0 tx=0).stop 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.512+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec106ed0 msgr2=0x7fa2ec19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.512+0000 7fa2f4152640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec19c950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa2e40048f0 tx=0x7fa2e4004920 comp rx=0 tx=0).stop 2024-09-10T21:21:14.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 shutdown_connections 2024-09-10T21:21:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa2d80b5330 0x7fa2d80b77f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2ec106ed0 0x7fa2ec19c950 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa2ec1023b0 0x7fa2ec1123e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 --2- 172.21.15.33:0/2788647737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2ec101a90 0x7fa2ec111ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 >> 172.21.15.33:0/2788647737 conn(0x7fa2ec0fd700 msgr2=0x7fa2ec104800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 shutdown_connections 2024-09-10T21:21:14.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:13.513+0000 7fa2f4152640 1 -- 172.21.15.33:0/2788647737 wait complete. 2024-09-10T21:21:14.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 208112 42.20 MB/sec execute 134 sec latency 185.588 ms 2024-09-10T21:21:14.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:13 smithi104 ceph-mon[35325]: pgmap v863: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 802 op/s 2024-09-10T21:21:14.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:13 smithi174 ceph-mon[35635]: pgmap v863: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 802 op/s 2024-09-10T21:21:14.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:13 smithi033 ceph-mon[30860]: pgmap v863: 97 pgs: 97 active+clean; 1.1 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 802 op/s 2024-09-10T21:21:15.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 208705 42.13 MB/sec execute 135 sec latency 267.342 ms 2024-09-10T21:21:15.377 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:21:15.761 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:16.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 209408 42.17 MB/sec execute 136 sec latency 198.607 ms 2024-09-10T21:21:16.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:15 smithi033 ceph-mon[30860]: pgmap v864: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-09-10T21:21:16.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:15 smithi104 ceph-mon[35325]: pgmap v864: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 -- 172.21.15.33:0/1454166277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 msgr2=0x7fbd74107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 --2- 172.21.15.33:0/1454166277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74107b00 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009900 tx=0x7fbd5c02a410 comp rx=0 tx=0).stop 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 -- 172.21.15.33:0/1454166277 shutdown_connections 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 --2- 172.21.15.33:0/1454166277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd74108040 0x7fbd7410e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 --2- 172.21.15.33:0/1454166277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74107b00 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 --2- 172.21.15.33:0/1454166277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd74101cb0 0x7fbd74102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 -- 172.21.15.33:0/1454166277 >> 172.21.15.33:0/1454166277 conn(0x7fbd740fd700 msgr2=0x7fbd740ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 -- 172.21.15.33:0/1454166277 shutdown_connections 2024-09-10T21:21:16.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.313+0000 7fbd7a607640 1 -- 172.21.15.33:0/1454166277 wait complete. 2024-09-10T21:21:16.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.314+0000 7fbd7a607640 1 Processor -- start 2024-09-10T21:21:16.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.314+0000 7fbd7a607640 1 -- start start 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.314+0000 7fbd7a607640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd74101cb0 0x7fbd74199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd7a607640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd7a607640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd74108040 0x7fbd7419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd7a607640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74111d00 con 0x7fbd741076c0 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd7a607640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74111b80 con 0x7fbd74101cb0 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd7a607640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd74111e80 con 0x7fbd74108040 2024-09-10T21:21:16.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38328/0 (socket says 172.21.15.33:38328) 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd6bfff640 1 -- 172.21.15.33:0/3225803092 learned_addr learned my addr 172.21.15.33:0/3225803092 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd73fff640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd74101cb0 0x7fbd74199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd74108040 0x7fbd7419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd73fff640 1 -- 172.21.15.33:0/3225803092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd74108040 msgr2=0x7fbd7419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd73fff640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd74108040 0x7fbd7419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd73fff640 1 -- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 msgr2=0x7fbd74199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd73fff640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd73fff640 1 -- 172.21.15.33:0/3225803092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7419f360 con 0x7fbd74101cb0 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.315+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:16.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.316+0000 7fbd73fff640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd74101cb0 0x7fbd74199550 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400d720 tx=0x7fbd6400dbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:16.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.316+0000 7fbd71ffb640 1 -- 172.21.15.33:0/3225803092 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd64017070 con 0x7fbd74101cb0 2024-09-10T21:21:16.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.316+0000 7fbd71ffb640 1 -- 172.21.15.33:0/3225803092 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6400de30 con 0x7fbd74101cb0 2024-09-10T21:21:16.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.316+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd7419f650 con 0x7fbd74101cb0 2024-09-10T21:21:16.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.316+0000 7fbd71ffb640 1 -- 172.21.15.33:0/3225803092 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640047a0 con 0x7fbd74101cb0 2024-09-10T21:21:16.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.316+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd74112460 con 0x7fbd74101cb0 2024-09-10T21:21:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.318+0000 7fbd71ffb640 1 -- 172.21.15.33:0/3225803092 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbd640099d0 con 0x7fbd74101cb0 2024-09-10T21:21:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 Processor -- start 2024-09-10T21:21:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 -- start start 2024-09-10T21:21:16.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a5217354d0 0x55a5217358b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521741840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521741d80 0x55a521744170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5216abea0 con 0x55a521741d80 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a52169b2d0 con 0x55a5217354d0 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a52169f0e0 con 0x55a521735df0 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521741840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.319+0000 7fbd78ba4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a5217354d0 0x55a5217358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a5217354d0 0x55a5217358b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38144/0 (socket says 172.21.15.33:38144) 2024-09-10T21:21:16.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521741d80 0x55a521744170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 -- 172.21.15.33:0/4193777879 learned_addr learned my addr 172.21.15.33:0/4193777879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd73fff640 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521741840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46990/0 (socket says 172.21.15.33:46990) 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd71ffb640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbd600b52e0 0x7fbd600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 -- 172.21.15.33:0/4193777879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 msgr2=0x55a521741840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521741840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 -- 172.21.15.33:0/4193777879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521741d80 msgr2=0x55a521744170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521741d80 0x55a521744170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 -- 172.21.15.33:0/4193777879 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5213f9650 con 0x55a5217354d0 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbd600b52e0 0x7fbd600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd71ffb640 1 -- 172.21.15.33:0/3225803092 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbd640e57a0 con 0x7fbd74101cb0 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a5217354d0 0x55a5217358b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c011590 tx=0x7fbd6c011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab5ada4a05f9b459 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:16.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd6a7fc640 1 -- 172.21.15.33:0/4193777879 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c017050 con 0x55a5217354d0 2024-09-10T21:21:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd6a7fc640 1 -- 172.21.15.33:0/4193777879 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6c00fc90 con 0x55a5217354d0 2024-09-10T21:21:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.320+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbd600b52e0 0x7fbd600b77a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c022000 tx=0x7fbd6c023000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:16.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd6a7fc640 1 -- 172.21.15.33:0/4193777879 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c01ba00 con 0x55a5217354d0 2024-09-10T21:21:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4193777879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a5217354d0 msgr2=0x55a5217358b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a5217354d0 0x55a5217358b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c011590 tx=0x7fbd6c011a50 comp rx=0 tx=0).stop 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4193777879 shutdown_connections 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521741d80 0x55a521744170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521741840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4193777879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a5217354d0 0x55a5217358b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4193777879 >> 172.21.15.33:0/4193777879 conn(0x55a521733300 msgr2=0x55a5217344a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4193777879 shutdown_connections 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.321+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4193777879 wait complete. 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 -- start start 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 Processor -- start 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a5217415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521735df0 0x55a52173ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521741d80 0x55a52173ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a5214fc700 con 0x55a5217354d0 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a521636160 con 0x55a521735df0 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a521744aa0 con 0x55a521741d80 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd7c9dc740 10 client.0 _ll_register_callbacks cb 0x55a52164f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a5217415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a5217415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38350/0 (socket says 172.21.15.33:38350) 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd73fff640 1 -- 172.21.15.33:0/449644660 learned_addr learned my addr 172.21.15.33:0/449644660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.322+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521735df0 0x55a52173ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521741d80 0x55a52173ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd73fff640 1 -- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521741d80 msgr2=0x55a52173ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd73fff640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521741d80 0x55a52173ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd73fff640 1 -- 172.21.15.33:0/449644660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521735df0 msgr2=0x55a52173ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd73fff640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521735df0 0x55a52173ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd73fff640 1 -- 172.21.15.33:0/449644660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a521663c90 con 0x55a5217354d0 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521741d80 0x55a52173ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521735df0 0x55a52173ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd73fff640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a5217415c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009610 tx=0x7fbd640e5a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=826b482018ed8a92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd537fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd537fe640 1 -- 172.21.15.33:0/449644660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd640ab6d0 con 0x55a5217354d0 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd537fe640 1 -- 172.21.15.33:0/449644660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbd640a81d0 con 0x55a5217354d0 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6c00fc90 con 0x55a5217354d0 2024-09-10T21:21:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd537fe640 1 -- 172.21.15.33:0/449644660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbd64005020 con 0x55a5217354d0 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd7c9dc740 10 client.18531 fetch_fsmap learned FSMap version 80 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.323+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55a521750390 con 0x55a5217354d0 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd537fe640 1 -- 172.21.15.33:0/449644660 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6c00fc90 con 0x55a5217354d0 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd537fe640 1 -- 172.21.15.33:0/449644660 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 80) ==== 8366+0+0 (secure 0 0 0) 0x7fbd64097160 con 0x55a5217354d0 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd7c9dc740 10 client.18531 fetch_fsmap finished waiting for FSMap version 80 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd7c9dc740 10 client.18531 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd7c9dc740 20 client.18531 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a521751850 0x55a521753cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd7c9dc740 4 client.18531 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.324+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5216addf0 con 0x55a521751850 2024-09-10T21:21:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:15 smithi174 ceph-mon[35635]: pgmap v864: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 883 op/s 2024-09-10T21:21:16.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.418+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a521751850 0x55a521753cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.421+0000 7fbd537fe640 10 client.18531 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:21:16.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.421+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a521751850 0x55a521753cf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec335f8590beb182 server_cookie=5f607be7714cafd9 in_seq=0 out_seq=0 2024-09-10T21:21:16.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.426+0000 7fbd537fe640 1 -- 172.21.15.33:0/449644660 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a5216addf0 con 0x55a521751850 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.426+0000 7fbd537fe640 10 client.18531 handle_command_reply: tid=0 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.426+0000 7fbd7c9dc740 1 client.18531 shutdown 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 msgr2=0x55a5217415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a5217415c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fbd64009610 tx=0x7fbd640e5a40 comp rx=0 tx=0).stop 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 shutdown_connections 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a521751850 0x55a521753cf0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd537fe640 0 client.18531 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521741d80 0x55a52173ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521735df0 0x55a52173ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/449644660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a5217415c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 >> 172.21.15.33:0/449644660 conn(0x55a521733300 msgr2=0x55a521738e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 shutdown_connections 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/449644660 wait complete. 2024-09-10T21:21:16.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.427+0000 7fbd7c9dc740 20 client.18531 trim_cache size 0 max 16384 2024-09-10T21:21:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 Processor -- start 2024-09-10T21:21:16.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 -- start start 2024-09-10T21:21:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5217354d0 0x55a521732c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521735df0 0x55a52173df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a52173d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ac1d0 con 0x55a521735df0 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741add10 con 0x55a521741d80 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.451+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ac8a0 con 0x55a5217354d0 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521735df0 0x55a52173df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd78ba4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a52173d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd6bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521735df0 0x55a52173df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38362/0 (socket says 172.21.15.33:38362) 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd78ba4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a52173d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38168/0 (socket says 172.21.15.33:38168) 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd6bfff640 1 -- 172.21.15.33:0/2941423352 learned_addr learned my addr 172.21.15.33:0/2941423352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd73fff640 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5217354d0 0x55a521732c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd78ba4640 1 -- 172.21.15.33:0/2941423352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5217354d0 msgr2=0x55a521732c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5217354d0 0x55a521732c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd78ba4640 1 -- 172.21.15.33:0/2941423352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521735df0 msgr2=0x55a52173df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521735df0 0x55a52173df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd78ba4640 1 -- 172.21.15.33:0/2941423352 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a521741930 con 0x55a521741d80 2024-09-10T21:21:16.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.452+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521735df0 0x55a52173df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:16.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.456+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a52173d290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c01b580 tx=0x7fbd6c009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=483a05d391c23a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.457+0000 7fbd69ffb640 1 -- 172.21.15.33:0/2941423352 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c003bb0 con 0x55a521741d80 2024-09-10T21:21:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.457+0000 7fbd69ffb640 1 -- 172.21.15.33:0/2941423352 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6c009610 con 0x55a521741d80 2024-09-10T21:21:16.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.457+0000 7fbd69ffb640 1 -- 172.21.15.33:0/2941423352 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c01b810 con 0x55a521741d80 2024-09-10T21:21:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.458+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/2941423352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 msgr2=0x55a52173d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.458+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a52173d290 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c01b580 tx=0x7fbd6c009b30 comp rx=0 tx=0).stop 2024-09-10T21:21:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.461+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/2941423352 shutdown_connections 2024-09-10T21:21:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.461+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a52173d290 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.461+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a521735df0 0x55a52173df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.461+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/2941423352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5217354d0 0x55a521732c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.461+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/2941423352 >> 172.21.15.33:0/2941423352 conn(0x55a521733300 msgr2=0x55a521734410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.462+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/2941423352 shutdown_connections 2024-09-10T21:21:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.462+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/2941423352 wait complete. 2024-09-10T21:21:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.463+0000 7fbd7c9dc740 1 -- start start 2024-09-10T21:21:16.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.463+0000 7fbd7c9dc740 1 Processor -- start 2024-09-10T21:21:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.464+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a52174a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.464+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521745a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.464+0000 7fbd7c9dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a521745f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.464+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741ad640 con 0x55a5217354d0 2024-09-10T21:21:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.464+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741acf70 con 0x55a521741d80 2024-09-10T21:21:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.464+0000 7fbd7c9dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd741aa190 con 0x55a521735df0 2024-09-10T21:21:16.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.464+0000 7fbd7c9dc740 10 client.0 _ll_register_callbacks cb 0x7fbd640046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a52174a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd73fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a52174a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38372/0 (socket says 172.21.15.33:38372) 2024-09-10T21:21:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd73fff640 1 -- 172.21.15.33:0/4245597126 learned_addr learned my addr 172.21.15.33:0/4245597126 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd78ba4640 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a521745f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521745a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd6bfff640 1 -- 172.21.15.33:0/4245597126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 msgr2=0x55a521745f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a521745f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd6bfff640 1 -- 172.21.15.33:0/4245597126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 msgr2=0x55a52174a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a52174a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.465+0000 7fbd6bfff640 1 -- 172.21.15.33:0/4245597126 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a521746770 con 0x55a521735df0 2024-09-10T21:21:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.466+0000 7fbd6bfff640 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521745a00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55a52173f6f0 tx=0x7fbd5c004420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac78e59ab8732c43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:16.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.466+0000 7fbd537fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:21:16.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.466+0000 7fbd537fe640 1 -- 172.21.15.33:0/4245597126 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c035040 con 0x55a521735df0 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.467+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd6c009610 con 0x55a521735df0 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.467+0000 7fbd537fe640 1 -- 172.21.15.33:0/4245597126 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbd5c005450 con 0x55a521735df0 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.467+0000 7fbd537fe640 1 -- 172.21.15.33:0/4245597126 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbd5c005650 con 0x55a521735df0 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.467+0000 7fbd7c9dc740 10 client.27829 fetch_fsmap learned FSMap version 80 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.467+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55a5217be870 con 0x55a521735df0 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd537fe640 1 -- 172.21.15.33:0/4245597126 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd6c009610 con 0x55a521735df0 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd537fe640 1 -- 172.21.15.33:0/4245597126 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 80) ==== 8366+0+0 (secure 0 0 0) 0x7fbd5c003b10 con 0x55a521735df0 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd7c9dc740 10 client.27829 fetch_fsmap finished waiting for FSMap version 80 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd7c9dc740 10 client.27829 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd7c9dc740 20 client.27829 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a5217befc0 0x55a5217c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:16.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd7c9dc740 4 client.27829 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:21:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5216a98a0 con 0x55a5217befc0 2024-09-10T21:21:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.468+0000 7fbd73fff640 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a5217befc0 0x55a5217c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.471+0000 7fbd73fff640 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a5217befc0 0x55a5217c13e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=53190099d08a565a server_cookie=9805a494d4dc398d in_seq=0 out_seq=0 2024-09-10T21:21:16.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.471+0000 7fbd537fe640 10 client.27829 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:21:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.472+0000 7fbd537fe640 1 -- 172.21.15.33:0/4245597126 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5216a98a0 con 0x55a5217befc0 2024-09-10T21:21:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.472+0000 7fbd537fe640 10 client.27829 handle_command_reply: tid=0 2024-09-10T21:21:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.472+0000 7fbd7c9dc740 1 client.27829 shutdown 2024-09-10T21:21:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.472+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 msgr2=0x55a521745a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.472+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521745a00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x55a52173f6f0 tx=0x7fbd5c004420 comp rx=0 tx=0).stop 2024-09-10T21:21:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 shutdown_connections 2024-09-10T21:21:16.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a521741d80 0x55a521745f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55a5217befc0 0x55a5217c13e0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd537fe640 0 client.27829 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a521735df0 0x55a521745a00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 --2- 172.21.15.33:0/4245597126 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a5217354d0 0x55a52174a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 >> 172.21.15.33:0/4245597126 conn(0x55a521733300 msgr2=0x55a521740040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 shutdown_connections 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 1 -- 172.21.15.33:0/4245597126 wait complete. 2024-09-10T21:21:16.477 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:21:16.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.473+0000 7fbd7c9dc740 20 client.27829 trim_cache size 0 max 16384 2024-09-10T21:21:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.474+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbd600b52e0 msgr2=0x7fbd600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.474+0000 7fbd7a607640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbd600b52e0 0x7fbd600b77a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c022000 tx=0x7fbd6c023000 comp rx=0 tx=0).stop 2024-09-10T21:21:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.474+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd74101cb0 msgr2=0x7fbd74199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.474+0000 7fbd7a607640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd74101cb0 0x7fbd74199550 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbd6400d720 tx=0x7fbd6400dbe0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 shutdown_connections 2024-09-10T21:21:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbd600b52e0 0x7fbd600b77a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd74108040 0x7fbd7419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd741076c0 0x7fbd74199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 --2- 172.21.15.33:0/3225803092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbd74101cb0 0x7fbd74199550 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 >> 172.21.15.33:0/3225803092 conn(0x7fbd740fd700 msgr2=0x7fbd7406ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 shutdown_connections 2024-09-10T21:21:16.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:16.476+0000 7fbd7a607640 1 -- 172.21.15.33:0/3225803092 wait complete. 2024-09-10T21:21:17.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 209931 41.99 MB/sec execute 137 sec latency 201.725 ms 2024-09-10T21:21:17.306 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:21:17.676 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:18.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 210864 41.99 MB/sec execute 138 sec latency 70.844 ms 2024-09-10T21:21:18.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:17 smithi033 ceph-mon[30860]: pgmap v865: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.4 MiB/s wr, 819 op/s 2024-09-10T21:21:18.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:17 smithi104 ceph-mon[35325]: pgmap v865: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.4 MiB/s wr, 819 op/s 2024-09-10T21:21:18.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:17 smithi174 ceph-mon[35635]: pgmap v865: 97 pgs: 97 active+clean; 1.1 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.4 MiB/s wr, 819 op/s 2024-09-10T21:21:19.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 211792 42.17 MB/sec execute 139 sec latency 173.780 ms 2024-09-10T21:21:19.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.117+0000 7f549c244640 1 -- 172.21.15.33:0/1506306988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494075e60 msgr2=0x7f5494113600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.117+0000 7f549c244640 1 --2- 172.21.15.33:0/1506306988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494075e60 0x7f5494113600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5484009860 tx=0x7f548402a360 comp rx=0 tx=0).stop 2024-09-10T21:21:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.119+0000 7f549c244640 1 -- 172.21.15.33:0/1506306988 shutdown_connections 2024-09-10T21:21:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.119+0000 7f549c244640 1 --2- 172.21.15.33:0/1506306988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5494113b40 0x7f5494115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.119+0000 7f549c244640 1 --2- 172.21.15.33:0/1506306988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494075e60 0x7f5494113600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.119+0000 7f549c244640 1 --2- 172.21.15.33:0/1506306988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 0x7f5494075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.119+0000 7f549c244640 1 -- 172.21.15.33:0/1506306988 >> 172.21.15.33:0/1506306988 conn(0x7f54940fe870 msgr2=0x7f5494100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.120+0000 7f549c244640 1 -- 172.21.15.33:0/1506306988 shutdown_connections 2024-09-10T21:21:19.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.120+0000 7f549c244640 1 -- 172.21.15.33:0/1506306988 wait complete. 2024-09-10T21:21:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.120+0000 7f549c244640 1 Processor -- start 2024-09-10T21:21:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.120+0000 7f549c244640 1 -- start start 2024-09-10T21:21:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.121+0000 7f549c244640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 0x7f549406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.121+0000 7f549c244640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5494075e60 0x7f549406d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.121+0000 7f549c244640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494113b40 0x7f5494072740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.121+0000 7f549c244640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f54941193c0 con 0x7f5494075540 2024-09-10T21:21:19.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.121+0000 7f549c244640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5494119240 con 0x7f5494075e60 2024-09-10T21:21:19.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.121+0000 7f549c244640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5494119540 con 0x7f5494113b40 2024-09-10T21:21:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.123+0000 7f5499fe0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 0x7f549406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.123+0000 7f5499fe0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 0x7f549406ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38392/0 (socket says 172.21.15.33:38392) 2024-09-10T21:21:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.123+0000 7f5499fe0640 1 -- 172.21.15.33:0/3180765610 learned_addr learned my addr 172.21.15.33:0/3180765610 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:19.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.123+0000 7f549a7e1640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494113b40 0x7f5494072740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.123+0000 7f54997df640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5494075e60 0x7f549406d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.129+0000 7f5499fe0640 1 -- 172.21.15.33:0/3180765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494113b40 msgr2=0x7f5494072740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.129+0000 7f5499fe0640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494113b40 0x7f5494072740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.129+0000 7f5499fe0640 1 -- 172.21.15.33:0/3180765610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5494075e60 msgr2=0x7f549406d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.129+0000 7f5499fe0640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5494075e60 0x7f549406d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.129+0000 7f5499fe0640 1 -- 172.21.15.33:0/3180765610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5494072ec0 con 0x7f5494075540 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.129+0000 7f5499fe0640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 0x7f549406ce60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f548c00efb0 tx=0x7f548c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.130+0000 7f548affd640 1 -- 172.21.15.33:0/3180765610 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c01b070 con 0x7f5494075540 2024-09-10T21:21:19.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.130+0000 7f548affd640 1 -- 172.21.15.33:0/3180765610 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c00ba20 con 0x7f5494075540 2024-09-10T21:21:19.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.131+0000 7f548affd640 1 -- 172.21.15.33:0/3180765610 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c016b30 con 0x7f5494075540 2024-09-10T21:21:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.133+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54940731b0 con 0x7f5494075540 2024-09-10T21:21:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.133+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54940738b0 con 0x7f5494075540 2024-09-10T21:21:19.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.139+0000 7f549e619740 1 Processor -- start 2024-09-10T21:21:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f549e619740 1 -- start start 2024-09-10T21:21:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f549e619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f109d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f549e619740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f549e619740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f549e619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5573f1013ea0 con 0x5573f109d0f0 2024-09-10T21:21:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f549e619740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5573f10032d0 con 0x5573f10a99a0 2024-09-10T21:21:19.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f549e619740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573f10070e0 con 0x5573f109da10 2024-09-10T21:21:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.142+0000 7f548affd640 1 -- 172.21.15.33:0/3180765610 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f548c00b040 con 0x7f5494075540 2024-09-10T21:21:19.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.144+0000 7f548affd640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54800b5300 0x7f54800b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.144+0000 7f548affd640 1 -- 172.21.15.33:0/3180765610 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f548c0e59a0 con 0x7f5494075540 2024-09-10T21:21:19.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.145+0000 7f54997df640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.145+0000 7f5499fe0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.145+0000 7f54997df640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38200/0 (socket says 172.21.15.33:38200) 2024-09-10T21:21:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.145+0000 7f54997df640 1 -- 172.21.15.33:0/2416832332 learned_addr learned my addr 172.21.15.33:0/2416832332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.145+0000 7f549a7e1640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54800b5300 0x7f54800b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.145+0000 7f549a7e1640 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f109d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.146+0000 7f5499fe0640 1 -- 172.21.15.33:0/2416832332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 msgr2=0x5573f10abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.146+0000 7f5499fe0640 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10abd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.146+0000 7f5499fe0640 1 -- 172.21.15.33:0/2416832332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 msgr2=0x5573f109d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.146+0000 7f5499fe0640 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f109d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.146+0000 7f5499fe0640 1 -- 172.21.15.33:0/2416832332 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f0d61650 con 0x5573f109da10 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.147+0000 7f5499fe0640 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a9460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f548c00ef80 tx=0x7f548c0aba00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dbe61bd850c9970 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.147+0000 7f549a7e1640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54800b5300 0x7f54800b77c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f549000b590 tx=0x7f549000b520 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.147+0000 7f547ffff640 1 -- 172.21.15.33:0/2416832332 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c0ac1d0 con 0x5573f109da10 2024-09-10T21:21:19.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.149+0000 7f547ffff640 1 -- 172.21.15.33:0/2416832332 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c0b0c00 con 0x5573f109da10 2024-09-10T21:21:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.150+0000 7f549e619740 1 -- 172.21.15.33:0/2416832332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 msgr2=0x5573f10a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.150+0000 7f549e619740 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a9460 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f548c00ef80 tx=0x7f548c0aba00 comp rx=0 tx=0).stop 2024-09-10T21:21:19.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.152+0000 7f549e619740 1 -- 172.21.15.33:0/2416832332 shutdown_connections 2024-09-10T21:21:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.152+0000 7f549e619740 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10abd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.152+0000 7f549e619740 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a9460 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.152+0000 7f549e619740 1 --2- 172.21.15.33:0/2416832332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f109d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.152+0000 7f549e619740 1 -- 172.21.15.33:0/2416832332 >> 172.21.15.33:0/2416832332 conn(0x5573f109af20 msgr2=0x5573f109c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.159+0000 7f549e619740 1 -- 172.21.15.33:0/2416832332 shutdown_connections 2024-09-10T21:21:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.159+0000 7f549e619740 1 -- 172.21.15.33:0/2416832332 wait complete. 2024-09-10T21:21:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.159+0000 7f549e619740 1 -- start start 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.159+0000 7f549e619740 1 Processor -- start 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549e619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549e619740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549e619740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549e619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5573f0e64700 con 0x5573f109d0f0 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549e619740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5573f0f9e160 con 0x5573f10a99a0 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549e619740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573f10bd630 con 0x5573f109da10 2024-09-10T21:21:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549e619740 10 client.0 _ll_register_callbacks cb 0x5573f0fb79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f549a7e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f54997df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f54997df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47044/0 (socket says 172.21.15.33:47044) 2024-09-10T21:21:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.160+0000 7f54997df640 1 -- 172.21.15.33:0/1386023629 learned_addr learned my addr 172.21.15.33:0/1386023629 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.163+0000 7f5499fe0640 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.164+0000 7f549a7e1640 1 -- 172.21.15.33:0/1386023629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 msgr2=0x5573f10a6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.164+0000 7f549a7e1640 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.164+0000 7f549a7e1640 1 -- 172.21.15.33:0/1386023629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 msgr2=0x5573f10a9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.164+0000 7f549a7e1640 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10a9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.164+0000 7f549a7e1640 1 -- 172.21.15.33:0/1386023629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f0fcbc90 con 0x5573f10a99a0 2024-09-10T21:21:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.164+0000 7f549a7e1640 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10a6be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f549001e730 tx=0x7f54900046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c80eade3574a21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.165+0000 7f547d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.165+0000 7f547d7fa640 1 -- 172.21.15.33:0/1386023629 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490025070 con 0x5573f10a99a0 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.165+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f548c0b0c00 con 0x5573f10a99a0 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.165+0000 7f547d7fa640 1 -- 172.21.15.33:0/1386023629 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5490010aa0 con 0x5573f10a99a0 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.165+0000 7f547d7fa640 1 -- 172.21.15.33:0/1386023629 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5490010ca0 con 0x5573f10a99a0 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.165+0000 7f549e619740 10 client.27953 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.165+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5573f10c8fc0 con 0x5573f10a99a0 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.166+0000 7f547d7fa640 1 -- 172.21.15.33:0/1386023629 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f548c0b0c00 con 0x5573f10a99a0 2024-09-10T21:21:19.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.167+0000 7f547d7fa640 1 -- 172.21.15.33:0/1386023629 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f549001c040 con 0x5573f10a99a0 2024-09-10T21:21:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.167+0000 7f549e619740 10 client.27953 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.167+0000 7f549e619740 10 client.27953 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:21:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.167+0000 7f549e619740 20 client.27953 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.167+0000 7f549e619740 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10ca920 0x5573f10ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.167+0000 7f549e619740 4 client.27953 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.167+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573f1015df0 con 0x5573f10ca920 2024-09-10T21:21:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.168+0000 7f5499fe0640 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10ca920 0x5573f10ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.169+0000 7f5499fe0640 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10ca920 0x5573f10ccda0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2fc325edaad23371 server_cookie=5f647317f9eff562 in_seq=0 out_seq=0 2024-09-10T21:21:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.170+0000 7f547d7fa640 10 client.27953 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:21:19.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.172+0000 7f547d7fa640 1 -- 172.21.15.33:0/1386023629 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5573f1015df0 con 0x5573f10ca920 2024-09-10T21:21:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.172+0000 7f547d7fa640 10 client.27953 handle_command_reply: tid=0 2024-09-10T21:21:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.172+0000 7f549e619740 1 client.27953 shutdown 2024-09-10T21:21:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.176+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 msgr2=0x5573f10a6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.176+0000 7f549e619740 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10a6be0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f549001e730 tx=0x7f54900046c0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.178+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 shutdown_connections 2024-09-10T21:21:19.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.178+0000 7f549e619740 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10a6be0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.178+0000 7f549e619740 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10a6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.178+0000 7f549e619740 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10ca920 0x5573f10ccda0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.178+0000 7f549e619740 1 --2- 172.21.15.33:0/1386023629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10a9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.178+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 >> 172.21.15.33:0/1386023629 conn(0x5573f109af20 msgr2=0x5573f10a0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:19.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.178+0000 7f547d7fa640 0 client.27953 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:21:19.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.179+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 shutdown_connections 2024-09-10T21:21:19.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.179+0000 7f549e619740 1 -- 172.21.15.33:0/1386023629 wait complete. 2024-09-10T21:21:19.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.180+0000 7f549e619740 20 client.27953 trim_cache size 0 max 16384 2024-09-10T21:21:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.209+0000 7f549e619740 1 Processor -- start 2024-09-10T21:21:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f549e619740 1 -- start start 2024-09-10T21:21:19.215 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:18 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:19.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f549e619740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f109d0f0 0x5573f109bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f549e619740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f109a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f549e619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f10a99a0 0x5573f10aad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f549e619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f549410d560 con 0x5573f10a99a0 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f549e619740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f549410f0a0 con 0x5573f109d0f0 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f549e619740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f549410dc30 con 0x5573f109da10 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f54997df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f109a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f54997df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f109a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47046/0 (socket says 172.21.15.33:47046) 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.210+0000 7f54997df640 1 -- 172.21.15.33:0/1280813088 learned_addr learned my addr 172.21.15.33:0/1280813088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f5499fe0640 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f109d0f0 0x5573f109bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f549a7e1640 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f10a99a0 0x5573f10aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f54997df640 1 -- 172.21.15.33:0/1280813088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f109d0f0 msgr2=0x5573f109bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f54997df640 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f109d0f0 0x5573f109bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f54997df640 1 -- 172.21.15.33:0/1280813088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f10a99a0 msgr2=0x5573f10aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f54997df640 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f10a99a0 0x5573f10aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f54997df640 1 -- 172.21.15.33:0/1280813088 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f109b9a0 con 0x5573f109da10 2024-09-10T21:21:19.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.211+0000 7f549a7e1640 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f10a99a0 0x5573f10aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:19.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.212+0000 7f54997df640 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f109a760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f548402a880 tx=0x7f5484030b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d7ec64ea6208237 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:19.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.212+0000 7f547f7fe640 1 -- 172.21.15.33:0/1280813088 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484037040 con 0x5573f109da10 2024-09-10T21:21:19.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.212+0000 7f547f7fe640 1 -- 172.21.15.33:0/1280813088 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5484064070 con 0x5573f109da10 2024-09-10T21:21:19.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.212+0000 7f547f7fe640 1 -- 172.21.15.33:0/1280813088 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840038e0 con 0x5573f109da10 2024-09-10T21:21:19.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 -- 172.21.15.33:0/1280813088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 msgr2=0x5573f109a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f109a760 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f548402a880 tx=0x7f5484030b70 comp rx=0 tx=0).stop 2024-09-10T21:21:19.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 -- 172.21.15.33:0/1280813088 shutdown_connections 2024-09-10T21:21:19.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f10a99a0 0x5573f10aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f109a760 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 --2- 172.21.15.33:0/1280813088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f109d0f0 0x5573f109bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 -- 172.21.15.33:0/1280813088 >> 172.21.15.33:0/1280813088 conn(0x5573f109af20 msgr2=0x5573f10a4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:19.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 -- 172.21.15.33:0/1280813088 shutdown_connections 2024-09-10T21:21:19.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.214+0000 7f549e619740 1 -- 172.21.15.33:0/1280813088 wait complete. 2024-09-10T21:21:19.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.215+0000 7f549e619740 1 -- start start 2024-09-10T21:21:19.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.215+0000 7f549e619740 1 Processor -- start 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.216+0000 7f549e619740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10c2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.216+0000 7f549e619740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10c2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.216+0000 7f549e619740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.216+0000 7f549e619740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f549410e9d0 con 0x5573f109d0f0 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.216+0000 7f549e619740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f549410e300 con 0x5573f10a99a0 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.216+0000 7f549e619740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f549410b5f0 con 0x5573f109da10 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.216+0000 7f549e619740 10 client.0 _ll_register_callbacks cb 0x7f549001e3e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.217+0000 7f549a7e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.217+0000 7f549a7e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10bd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38224/0 (socket says 172.21.15.33:38224) 2024-09-10T21:21:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.217+0000 7f549a7e1640 1 -- 172.21.15.33:0/2011779609 learned_addr learned my addr 172.21.15.33:0/2011779609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.217+0000 7f5499fe0640 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.217+0000 7f54997df640 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10c2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.218+0000 7f549a7e1640 1 -- 172.21.15.33:0/2011779609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 msgr2=0x5573f10c2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.218+0000 7f549a7e1640 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10c2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.218+0000 7f549a7e1640 1 -- 172.21.15.33:0/2011779609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 msgr2=0x5573f10c2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.218+0000 7f549a7e1640 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10c2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.218+0000 7f549a7e1640 1 -- 172.21.15.33:0/2011779609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f10c34e0 con 0x5573f10a99a0 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.218+0000 7f549a7e1640 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10bd8b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f549000bac0 tx=0x7f5490010b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1437f8b27cab861 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.218+0000 7f547d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.219+0000 7f547d7fa640 1 -- 172.21.15.33:0/2011779609 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490038030 con 0x5573f10a99a0 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.219+0000 7f547d7fa640 1 -- 172.21.15.33:0/2011779609 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f54900122c0 con 0x5573f10a99a0 2024-09-10T21:21:19.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.219+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5484064070 con 0x5573f10a99a0 2024-09-10T21:21:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.219+0000 7f547d7fa640 1 -- 172.21.15.33:0/2011779609 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5490043050 con 0x5573f10a99a0 2024-09-10T21:21:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.219+0000 7f549e619740 10 client.27959 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.219+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5573f10b26c0 con 0x5573f10a99a0 2024-09-10T21:21:19.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f547d7fa640 1 -- 172.21.15.33:0/2011779609 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5484064070 con 0x5573f10a99a0 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f547d7fa640 1 -- 172.21.15.33:0/2011779609 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f549001c070 con 0x5573f10a99a0 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f549e619740 10 client.27959 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f549e619740 10 client.27959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f549e619740 20 client.27959 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f549e619740 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10b2e70 0x5573f10b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f549e619740 4 client.27959 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573f10118a0 con 0x5573f10b2e70 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.220+0000 7f5499fe0640 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10b2e70 0x5573f10b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.221+0000 7f547d7fa640 10 client.27959 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:21:19.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.221+0000 7f5499fe0640 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10b2e70 0x5573f10b5290 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=35b4449ac537e031 server_cookie=6216738dd7a07c42 in_seq=0 out_seq=0 2024-09-10T21:21:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.224+0000 7f547d7fa640 1 -- 172.21.15.33:0/2011779609 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573f10118a0 con 0x5573f10b2e70 2024-09-10T21:21:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f547d7fa640 10 client.27959 handle_command_reply: tid=0 2024-09-10T21:21:19.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 client.27959 shutdown 2024-09-10T21:21:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 msgr2=0x5573f10bd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10bd8b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f549000bac0 tx=0x7f5490010b50 comp rx=0 tx=0).stop 2024-09-10T21:21:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 shutdown_connections 2024-09-10T21:21:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5573f10a99a0 0x5573f10bd8b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f109da10 0x5573f10c2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5573f10b2e70 0x5573f10b5290 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 --2- 172.21.15.33:0/2011779609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5573f109d0f0 0x5573f10c2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 >> 172.21.15.33:0/2011779609 conn(0x5573f109af20 msgr2=0x5573f10a7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f547d7fa640 0 client.27959 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.225+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 shutdown_connections 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.226+0000 7f549e619740 1 -- 172.21.15.33:0/2011779609 wait complete. 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.226+0000 7f549e619740 20 client.27959 trim_cache size 0 max 16384 2024-09-10T21:21:19.230 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54800b5300 msgr2=0x7f54800b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54800b5300 0x7f54800b77c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f549000b590 tx=0x7f549000b520 comp rx=0 tx=0).stop 2024-09-10T21:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 msgr2=0x7f549406ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 0x7f549406ce60 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f548c00efb0 tx=0x7f548c00c800 comp rx=0 tx=0).stop 2024-09-10T21:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 shutdown_connections 2024-09-10T21:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f54800b5300 0x7f54800b77c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5494113b40 0x7f5494072740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5494075e60 0x7f549406d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 --2- 172.21.15.33:0/3180765610 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5494075540 0x7f549406ce60 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 >> 172.21.15.33:0/3180765610 conn(0x7f54940fe870 msgr2=0x7f54940ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 shutdown_connections 2024-09-10T21:21:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:19.227+0000 7f549c244640 1 -- 172.21.15.33:0/3180765610 wait complete. 2024-09-10T21:21:19.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:18 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:19.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:18 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:21:20.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 212118 42.00 MB/sec execute 140 sec latency 281.230 ms 2024-09-10T21:21:20.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:19 smithi104 ceph-mon[35325]: pgmap v866: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:21:20.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:19 smithi174 ceph-mon[35635]: pgmap v866: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:21:20.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:19 smithi033 ceph-mon[30860]: pgmap v866: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:21:20.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:21:21.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 212842 41.92 MB/sec execute 141 sec latency 68.792 ms 2024-09-10T21:21:21.077 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.599+0000 7f3b782c6640 1 -- 172.21.15.33:0/1592002375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 msgr2=0x7f3b7010f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.599+0000 7f3b782c6640 1 --2- 172.21.15.33:0/1592002375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 0x7f3b7010f650 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009ca0 tx=0x7f3b6002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.600+0000 7f3b782c6640 1 -- 172.21.15.33:0/1592002375 shutdown_connections 2024-09-10T21:21:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.600+0000 7f3b782c6640 1 --2- 172.21.15.33:0/1592002375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 0x7f3b70116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.600+0000 7f3b782c6640 1 --2- 172.21.15.33:0/1592002375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 0x7f3b7010f650 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.600+0000 7f3b782c6640 1 --2- 172.21.15.33:0/1592002375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b7010a3b0 0x7f3b7010a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.600+0000 7f3b782c6640 1 -- 172.21.15.33:0/1592002375 >> 172.21.15.33:0/1592002375 conn(0x7f3b7006e2e0 msgr2=0x7f3b7006e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:21.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.600+0000 7f3b782c6640 1 -- 172.21.15.33:0/1592002375 shutdown_connections 2024-09-10T21:21:21.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.600+0000 7f3b782c6640 1 -- 172.21.15.33:0/1592002375 wait complete. 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.601+0000 7f3b782c6640 1 Processor -- start 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.601+0000 7f3b782c6640 1 -- start start 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b782c6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b7010a3b0 0x7f3b701a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b782c6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 0x7f3b701a8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b782c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 0x7f3b701ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b782c6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b70119330 con 0x7f3b7010f210 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b782c6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701191b0 con 0x7f3b7010a3b0 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b782c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701194b0 con 0x7f3b7010fb90 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b75861640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 0x7f3b701a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b76863640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 0x7f3b701ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b76062640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b7010a3b0 0x7f3b701a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b76863640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 0x7f3b701ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36362/0 (socket says 172.21.15.33:36362) 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.602+0000 7f3b76863640 1 -- 172.21.15.33:0/4120287976 learned_addr learned my addr 172.21.15.33:0/4120287976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.603+0000 7f3b76863640 1 -- 172.21.15.33:0/4120287976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b7010a3b0 msgr2=0x7f3b701a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.603+0000 7f3b76863640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b7010a3b0 0x7f3b701a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.603+0000 7f3b76863640 1 -- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 msgr2=0x7f3b701a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.603+0000 7f3b76863640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 0x7f3b701a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.603+0000 7f3b76863640 1 -- 172.21.15.33:0/4120287976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b701ae180 con 0x7f3b7010fb90 2024-09-10T21:21:21.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.603+0000 7f3b76863640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 0x7f3b701ad9b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b68008a50 tx=0x7f3b68008f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.604+0000 7f3b6f7fe640 1 -- 172.21.15.33:0/4120287976 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68015030 con 0x7f3b7010fb90 2024-09-10T21:21:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.604+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b701ae470 con 0x7f3b7010fb90 2024-09-10T21:21:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.604+0000 7f3b6f7fe640 1 -- 172.21.15.33:0/4120287976 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b6800ecd0 con 0x7f3b7010fb90 2024-09-10T21:21:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.605+0000 7f3b6f7fe640 1 -- 172.21.15.33:0/4120287976 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b68003e60 con 0x7f3b7010fb90 2024-09-10T21:21:21.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.605+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b701b4b30 con 0x7f3b7010fb90 2024-09-10T21:21:21.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.609+0000 7f3b7a69b740 1 Processor -- start 2024-09-10T21:21:21.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b6f7fe640 1 -- 172.21.15.33:0/4120287976 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3b68004020 con 0x7f3b7010fb90 2024-09-10T21:21:21.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b7a69b740 1 -- start start 2024-09-10T21:21:21.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169e4d0 0x55785169e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169edf0 0x5578516aa840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 0x5578516ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557851614ea0 con 0x55785169edf0 2024-09-10T21:21:21.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5578516042d0 con 0x55785169e4d0 2024-09-10T21:21:21.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.610+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578516080e0 con 0x5578516aad80 2024-09-10T21:21:21.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.611+0000 7f3b75861640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169e4d0 0x55785169e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.611+0000 7f3b75861640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169e4d0 0x55785169e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51572/0 (socket says 172.21.15.33:51572) 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.611+0000 7f3b75861640 1 -- 172.21.15.33:0/3244437614 learned_addr learned my addr 172.21.15.33:0/3244437614 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.611+0000 7f3b76062640 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169edf0 0x5578516aa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.611+0000 7f3b75861640 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 0x5578516ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.612+0000 7f3b76062640 1 -- 172.21.15.33:0/3244437614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 msgr2=0x5578516ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.612+0000 7f3b76062640 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 0x5578516ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.612+0000 7f3b76062640 1 -- 172.21.15.33:0/3244437614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169e4d0 msgr2=0x55785169e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.612+0000 7f3b76062640 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169e4d0 0x55785169e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.612+0000 7f3b76062640 1 -- 172.21.15.33:0/3244437614 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557851362650 con 0x55785169edf0 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.612+0000 7f3b75861640 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169e4d0 0x55785169e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.613+0000 7f3b6f7fe640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3b580b5360 0x7f3b580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.613+0000 7f3b76062640 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169edf0 0x5578516aa840 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400bbe0 tx=0x7f3b6400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=863a138a84cbdac3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:21.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.613+0000 7f3b6f7fe640 1 -- 172.21.15.33:0/4120287976 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3b680e5a10 con 0x7f3b7010fb90 2024-09-10T21:21:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.614+0000 7f3b6cde6640 1 -- 172.21.15.33:0/3244437614 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64015070 con 0x55785169edf0 2024-09-10T21:21:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.614+0000 7f3b6cde6640 1 -- 172.21.15.33:0/3244437614 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b640035f0 con 0x55785169edf0 2024-09-10T21:21:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.614+0000 7f3b6cde6640 1 -- 172.21.15.33:0/3244437614 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b640116a0 con 0x55785169edf0 2024-09-10T21:21:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.614+0000 7f3b75861640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3b580b5360 0x7f3b580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.615+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3244437614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169edf0 msgr2=0x5578516aa840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.615+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169edf0 0x5578516aa840 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400bbe0 tx=0x7f3b6400f730 comp rx=0 tx=0).stop 2024-09-10T21:21:21.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.616+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3244437614 shutdown_connections 2024-09-10T21:21:21.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.616+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 0x5578516ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.616+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169edf0 0x5578516aa840 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.616+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3244437614 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169e4d0 0x55785169e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.616+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3244437614 >> 172.21.15.33:0/3244437614 conn(0x55785169c300 msgr2=0x55785169d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.617+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3244437614 shutdown_connections 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.617+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3244437614 wait complete. 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.617+0000 7f3b7a69b740 1 -- start start 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.618+0000 7f3b75861640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3b580b5360 0x7f3b580b7820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3b60031cb0 tx=0x7f3b60005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.618+0000 7f3b7a69b740 1 Processor -- start 2024-09-10T21:21:21.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.618+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169e4d0 0x5578516aa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.619+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.619+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578516aad80 0x5578516a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.619+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557851465700 con 0x5578516aad80 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.619+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55785159f160 con 0x55785169edf0 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.619+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578516ada50 con 0x55785169e4d0 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.619+0000 7f3b7a69b740 10 client.0 _ll_register_callbacks cb 0x5578515b89e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.620+0000 7f3b76863640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578516aad80 0x5578516a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.620+0000 7f3b76863640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578516aad80 0x5578516a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60016/0 (socket says 172.21.15.33:60016) 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.620+0000 7f3b76863640 1 -- 172.21.15.33:0/1857535358 learned_addr learned my addr 172.21.15.33:0/1857535358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.620+0000 7f3b76863640 1 -- 172.21.15.33:0/1857535358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169e4d0 msgr2=0x5578516aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.620+0000 7f3b76863640 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169e4d0 0x5578516aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.621+0000 7f3b76863640 1 -- 172.21.15.33:0/1857535358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 msgr2=0x5578516a7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.621+0000 7f3b75861640 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.621+0000 7f3b76863640 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516a7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.621+0000 7f3b76863640 1 -- 172.21.15.33:0/1857535358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578515ccc90 con 0x5578516aad80 2024-09-10T21:21:21.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.621+0000 7f3b75861640 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:21:21.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.621+0000 7f3b76863640 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578516aad80 0x5578516a8080 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3b680abed0 tx=0x7f3b680ab940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf10aeaa44368df6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:21.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.621+0000 7f3b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:21.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.622+0000 7f3b55ffb640 1 -- 172.21.15.33:0/1857535358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680abce0 con 0x5578516aad80 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.622+0000 7f3b55ffb640 1 -- 172.21.15.33:0/1857535358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3b68004340 con 0x5578516aad80 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.622+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b640035f0 con 0x5578516aad80 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.622+0000 7f3b55ffb640 1 -- 172.21.15.33:0/1857535358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3b680ac8d0 con 0x5578516aad80 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.623+0000 7f3b7a69b740 10 client.18567 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.623+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5578516b93e0 con 0x5578516aad80 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.623+0000 7f3b55ffb640 1 -- 172.21.15.33:0/1857535358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b640035f0 con 0x5578516aad80 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.623+0000 7f3b55ffb640 1 -- 172.21.15.33:0/1857535358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f3b680a8440 con 0x5578516aad80 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.624+0000 7f3b7a69b740 10 client.18567 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.624+0000 7f3b7a69b740 10 client.18567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.624+0000 7f3b7a69b740 20 client.18567 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.624+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5578516ba8a0 0x5578516bcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.624+0000 7f3b7a69b740 4 client.18567 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:21.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.624+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557851616df0 con 0x5578516ba8a0 2024-09-10T21:21:21.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.624+0000 7f3b76062640 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5578516ba8a0 0x5578516bcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.625+0000 7f3b76062640 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5578516ba8a0 0x5578516bcd20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c4c264fc59f78cf server_cookie=27f0a67930c0b63e in_seq=0 out_seq=0 2024-09-10T21:21:21.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.625+0000 7f3b55ffb640 10 client.18567 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:21:21.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.630+0000 7f3b55ffb640 1 -- 172.21.15.33:0/1857535358 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557851616df0 con 0x5578516ba8a0 2024-09-10T21:21:21.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.630+0000 7f3b55ffb640 10 client.18567 handle_command_reply: tid=0 2024-09-10T21:21:21.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.630+0000 7f3b7a69b740 1 client.18567 shutdown 2024-09-10T21:21:21.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.630+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578516aad80 msgr2=0x5578516a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.630+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578516aad80 0x5578516a8080 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f3b680abed0 tx=0x7f3b680ab940 comp rx=0 tx=0).stop 2024-09-10T21:21:21.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.632+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 shutdown_connections 2024-09-10T21:21:21.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.632+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5578516ba8a0 0x5578516bcd20 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.632+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5578516aad80 0x5578516a8080 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.632+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516a7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.632+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/1857535358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169e4d0 0x5578516aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.632+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 >> 172.21.15.33:0/1857535358 conn(0x55785169c300 msgr2=0x5578516a1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:21.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.632+0000 7f3b55ffb640 0 client.18567 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:21:21.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.633+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 shutdown_connections 2024-09-10T21:21:21.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.633+0000 7f3b7a69b740 1 -- 172.21.15.33:0/1857535358 wait complete. 2024-09-10T21:21:21.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.633+0000 7f3b7a69b740 20 client.18567 trim_cache size 0 max 16384 2024-09-10T21:21:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.656+0000 7f3b7a69b740 1 Processor -- start 2024-09-10T21:21:21.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.656+0000 7f3b7a69b740 1 -- start start 2024-09-10T21:21:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516bcd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169edf0 0x55785169bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578516aad80 0x5578516ac170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701bd360 con 0x55785169e4d0 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701beea0 con 0x5578516aad80 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701bda30 con 0x55785169edf0 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578516aad80 0x5578516ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578516aad80 0x5578516ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51598/0 (socket says 172.21.15.33:51598) 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 -- 172.21.15.33:0/3967233823 learned_addr learned my addr 172.21.15.33:0/3967233823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76062640 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516bcd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b75861640 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169edf0 0x55785169bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 -- 172.21.15.33:0/3967233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169edf0 msgr2=0x55785169bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169edf0 0x55785169bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 -- 172.21.15.33:0/3967233823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 msgr2=0x5578516bcd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516bcd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.657+0000 7f3b76863640 1 -- 172.21.15.33:0/3967233823 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578516a7850 con 0x5578516aad80 2024-09-10T21:21:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.658+0000 7f3b76863640 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578516aad80 0x5578516ac170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3b680e5cb0 tx=0x7f3b680aba70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91b5f2e0b9e3b90c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.658+0000 7f3b57fff640 1 -- 172.21.15.33:0/3967233823 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b680b0cb0 con 0x5578516aad80 2024-09-10T21:21:21.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.658+0000 7f3b57fff640 1 -- 172.21.15.33:0/3967233823 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b680b5760 con 0x5578516aad80 2024-09-10T21:21:21.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.659+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3967233823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578516aad80 msgr2=0x5578516ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.659+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578516aad80 0x5578516ac170 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3b680e5cb0 tx=0x7f3b680aba70 comp rx=0 tx=0).stop 2024-09-10T21:21:21.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.660+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3967233823 shutdown_connections 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.660+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5578516aad80 0x5578516ac170 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.660+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55785169edf0 0x55785169bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.660+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3967233823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516bcd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.660+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3967233823 >> 172.21.15.33:0/3967233823 conn(0x55785169c300 msgr2=0x5578516a6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.660+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3967233823 shutdown_connections 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.660+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3967233823 wait complete. 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 -- start start 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 Processor -- start 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516c2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 0x5578516c2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701be7d0 con 0x55785169e4d0 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701be100 con 0x55785169edf0 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b701bb320 con 0x5578516aad80 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b7a69b740 10 client.0 _ll_register_callbacks cb 0x7f3b680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b75861640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b76062640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b76062640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516c2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60046/0 (socket says 172.21.15.33:60046) 2024-09-10T21:21:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.661+0000 7f3b76062640 1 -- 172.21.15.33:0/3686947226 learned_addr learned my addr 172.21.15.33:0/3686947226 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b76062640 1 -- 172.21.15.33:0/3686947226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 msgr2=0x5578516c2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b76062640 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 0x5578516c2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b76062640 1 -- 172.21.15.33:0/3686947226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 msgr2=0x5578516c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b76062640 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b76062640 1 -- 172.21.15.33:0/3686947226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578516b2400 con 0x55785169e4d0 2024-09-10T21:21:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b75861640 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b76062640 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516c2550 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8bb58a4a49cd6c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b55ffb640 1 -- 172.21.15.33:0/3686947226 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b64015040 con 0x55785169e4d0 2024-09-10T21:21:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b55ffb640 1 -- 172.21.15.33:0/3686947226 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3b64003f60 con 0x55785169e4d0 2024-09-10T21:21:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.662+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3b680b5760 con 0x55785169e4d0 2024-09-10T21:21:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b55ffb640 1 -- 172.21.15.33:0/3686947226 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3b64033050 con 0x55785169e4d0 2024-09-10T21:21:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 10 client.18573 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x557851727b50 con 0x55785169e4d0 2024-09-10T21:21:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b55ffb640 1 -- 172.21.15.33:0/3686947226 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b680b5760 con 0x55785169e4d0 2024-09-10T21:21:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b55ffb640 1 -- 172.21.15.33:0/3686947226 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f3b64024420 con 0x55785169e4d0 2024-09-10T21:21:21.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 10 client.18573 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 10 client.18573 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:21:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 20 client.18573 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557851728040 0x55785172a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 4 client.18573 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:21:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.663+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5578516128a0 con 0x557851728040 2024-09-10T21:21:21.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.664+0000 7f3b75861640 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557851728040 0x55785172a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.664+0000 7f3b55ffb640 10 client.18573 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:21:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.664+0000 7f3b75861640 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557851728040 0x55785172a460 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=55299b2ae86f3962 server_cookie=ea3ae23bd6cd9a18 in_seq=0 out_seq=0 2024-09-10T21:21:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b55ffb640 1 -- 172.21.15.33:0/3686947226 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5578516128a0 con 0x557851728040 2024-09-10T21:21:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b55ffb640 10 client.18573 handle_command_reply: tid=0 2024-09-10T21:21:21.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 client.18573 shutdown 2024-09-10T21:21:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 msgr2=0x5578516c2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516c2550 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f3b640058c0 tx=0x7f3b64005f50 comp rx=0 tx=0).stop 2024-09-10T21:21:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 shutdown_connections 2024-09-10T21:21:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578516aad80 0x5578516c2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55785169edf0 0x5578516c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557851728040 0x55785172a460 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 --2- 172.21.15.33:0/3686947226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55785169e4d0 0x5578516c2550 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b55ffb640 0 client.18573 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:21:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 >> 172.21.15.33:0/3686947226 conn(0x55785169c300 msgr2=0x5578516a6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 shutdown_connections 2024-09-10T21:21:21.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.665+0000 7f3b7a69b740 1 -- 172.21.15.33:0/3686947226 wait complete. 2024-09-10T21:21:21.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.666+0000 7f3b7a69b740 20 client.18573 trim_cache size 0 max 16384 2024-09-10T21:21:21.679 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:21:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3b580b5360 msgr2=0x7f3b580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3b580b5360 0x7f3b580b7820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3b60031cb0 tx=0x7f3b60005d80 comp rx=0 tx=0).stop 2024-09-10T21:21:21.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 msgr2=0x7f3b701ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 0x7f3b701ad9b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b68008a50 tx=0x7f3b68008f10 comp rx=0 tx=0).stop 2024-09-10T21:21:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 shutdown_connections 2024-09-10T21:21:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3b580b5360 0x7f3b580b7820 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b7010fb90 0x7f3b701ad9b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b7010f210 0x7f3b701a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 --2- 172.21.15.33:0/4120287976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3b7010a3b0 0x7f3b701a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 >> 172.21.15.33:0/4120287976 conn(0x7f3b7006e2e0 msgr2=0x7f3b70073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 shutdown_connections 2024-09-10T21:21:21.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:21.667+0000 7f3b782c6640 1 -- 172.21.15.33:0/4120287976 wait complete. 2024-09-10T21:21:22.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 213699 41.90 MB/sec execute 142 sec latency 167.910 ms 2024-09-10T21:21:22.246 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:21 smithi033 ceph-mon[30860]: pgmap v867: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 903 op/s 2024-09-10T21:21:22.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:21 smithi104 ceph-mon[35325]: pgmap v867: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 903 op/s 2024-09-10T21:21:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:21 smithi174 ceph-mon[35635]: pgmap v867: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 903 op/s 2024-09-10T21:21:22.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:21:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:23.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 214623 42.01 MB/sec execute 143 sec latency 68.652 ms 2024-09-10T21:21:23.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.478+0000 7f4d084d5640 1 -- 172.21.15.33:0/1845185781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d000754e0 msgr2=0x7f4d000758c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.478+0000 7f4d084d5640 1 --2- 172.21.15.33:0/1845185781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d000754e0 0x7f4d000758c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009900 tx=0x7f4cfc02a410 comp rx=0 tx=0).stop 2024-09-10T21:21:23.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.478+0000 7f4d084d5640 1 -- 172.21.15.33:0/1845185781 shutdown_connections 2024-09-10T21:21:23.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.478+0000 7f4d084d5640 1 --2- 172.21.15.33:0/1845185781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d0010f3b0 0x7f4d001117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.478+0000 7f4d084d5640 1 --2- 172.21.15.33:0/1845185781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d00075e00 0x7f4d0010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.478+0000 7f4d084d5640 1 --2- 172.21.15.33:0/1845185781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d000754e0 0x7f4d000758c0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.478+0000 7f4d084d5640 1 -- 172.21.15.33:0/1845185781 >> 172.21.15.33:0/1845185781 conn(0x7f4d000fe870 msgr2=0x7f4d00100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:23.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.479+0000 7f4d084d5640 1 -- 172.21.15.33:0/1845185781 shutdown_connections 2024-09-10T21:21:23.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.479+0000 7f4d084d5640 1 -- 172.21.15.33:0/1845185781 wait complete. 2024-09-10T21:21:23.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.479+0000 7f4d084d5640 1 Processor -- start 2024-09-10T21:21:23.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.480+0000 7f4d084d5640 1 -- start start 2024-09-10T21:21:23.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.480+0000 7f4d084d5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d000754e0 0x7f4d0019b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.480+0000 7f4d084d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d00075e00 0x7f4d0019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.480+0000 7f4d084d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d0010f3b0 0x7f4d001a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.480+0000 7f4d084d5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00114f60 con 0x7f4d00075e00 2024-09-10T21:21:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.480+0000 7f4d084d5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4d00114de0 con 0x7f4d000754e0 2024-09-10T21:21:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.480+0000 7f4d084d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d001150e0 con 0x7f4d0010f3b0 2024-09-10T21:21:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.481+0000 7f4d06271640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d000754e0 0x7f4d0019b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.481+0000 7f4d06271640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d000754e0 0x7f4d0019b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51616/0 (socket says 172.21.15.33:51616) 2024-09-10T21:21:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.481+0000 7f4d06271640 1 -- 172.21.15.33:0/2112532306 learned_addr learned my addr 172.21.15.33:0/2112532306 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.481+0000 7f4d06a72640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d0010f3b0 0x7f4d001a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.481+0000 7f4d05a70640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d00075e00 0x7f4d0019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.481+0000 7f4d06271640 1 -- 172.21.15.33:0/2112532306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d0010f3b0 msgr2=0x7f4d001a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.481+0000 7f4d06271640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d0010f3b0 0x7f4d001a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4d06271640 1 -- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d00075e00 msgr2=0x7f4d0019ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4d06271640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d00075e00 0x7f4d0019ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4d06271640 1 -- 172.21.15.33:0/2112532306 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d001a1580 con 0x7f4d000754e0 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4d05a70640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d00075e00 0x7f4d0019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4d06271640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d000754e0 0x7f4d0019b4d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005940 tx=0x7f4cfc0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4cf77fe640 1 -- 172.21.15.33:0/2112532306 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc03f030 con 0x7f4d000754e0 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4cf77fe640 1 -- 172.21.15.33:0/2112532306 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cfc0044c0 con 0x7f4d000754e0 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.482+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d001a1810 con 0x7f4d000754e0 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.483+0000 7f4cf77fe640 1 -- 172.21.15.33:0/2112532306 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc035070 con 0x7f4d000754e0 2024-09-10T21:21:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.483+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d001a7f30 con 0x7f4d000754e0 2024-09-10T21:21:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 Processor -- start 2024-09-10T21:21:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 -- start start 2024-09-10T21:21:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7eb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8bd752ea0 con 0x55d8bd7e8d80 2024-09-10T21:21:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d8bd7422d0 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.486+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8bd7460e0 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7eb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7eb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60070/0 (socket says 172.21.15.33:60070) 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 -- 172.21.15.33:0/25167626 learned_addr learned my addr 172.21.15.33:0/25167626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d06a72640 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4cf77fe640 1 -- 172.21.15.33:0/2112532306 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4cfc0035f0 con 0x7f4d000754e0 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 -- 172.21.15.33:0/25167626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 msgr2=0x55d8bd7e8840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e8840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 -- 172.21.15.33:0/25167626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 msgr2=0x55d8bd7dc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7dc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 -- 172.21.15.33:0/25167626 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bd4a0650 con 0x55d8bd7e8d80 2024-09-10T21:21:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d06a72640 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7dc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.487+0000 7f4d05a70640 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7eb170 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009cf0 tx=0x7f4cf00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e293c1631abf721 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.488+0000 7f4cf4ff9640 1 -- 172.21.15.33:0/25167626 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0012450 con 0x55d8bd7e8d80 2024-09-10T21:21:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.488+0000 7f4cf4ff9640 1 -- 172.21.15.33:0/25167626 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cf0012aa0 con 0x55d8bd7e8d80 2024-09-10T21:21:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.488+0000 7f4cf4ff9640 1 -- 172.21.15.33:0/25167626 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf000fb80 con 0x55d8bd7e8d80 2024-09-10T21:21:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.488+0000 7f4cf77fe640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cec0b5360 0x7f4cec0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.488+0000 7f4cf77fe640 1 -- 172.21.15.33:0/2112532306 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cfc105fe0 con 0x7f4d000754e0 2024-09-10T21:21:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.488+0000 7f4d06a72640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cec0b5360 0x7f4cec0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/25167626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 msgr2=0x55d8bd7eb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7eb170 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009cf0 tx=0x7f4cf00117b0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/25167626 shutdown_connections 2024-09-10T21:21:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7eb170 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/25167626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7dc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/25167626 >> 172.21.15.33:0/25167626 conn(0x55d8bd7da300 msgr2=0x55d8bd7db4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/25167626 shutdown_connections 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/25167626 wait complete. 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d06a72640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cec0b5360 0x7f4cec0b7820 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009fd0 tx=0x7f4cf8009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 -- start start 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.489+0000 7f4d0a8aa740 1 Processor -- start 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d8bd5a3700 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d8bd6dd160 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8bd7eba00 con 0x55d8bd7e8d80 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d0a8aa740 10 client.0 _ll_register_callbacks cb 0x55d8bd6f69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d06271640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d05a70640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d06271640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60084/0 (socket says 172.21.15.33:60084) 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d06271640 1 -- 172.21.15.33:0/804420503 learned_addr learned my addr 172.21.15.33:0/804420503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d06a72640 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d05a70640 1 -- 172.21.15.33:0/804420503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 msgr2=0x55d8bd7e6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d05a70640 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d05a70640 1 -- 172.21.15.33:0/804420503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dc4d0 msgr2=0x55d8bd7e8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d05a70640 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d05a70640 1 -- 172.21.15.33:0/804420503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bd70ac90 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.490+0000 7f4d06271640 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4d05a70640 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e5a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0005830 tx=0x7f4cf0005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468e080d932e8b5f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4ce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/804420503 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0029020 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/804420503 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cf0004040 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cf0012aa0 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/804420503 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4cf0004240 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4d0a8aa740 10 client.27986 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d8bd7ffb70 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.491+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/804420503 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cf0012aa0 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/804420503 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f4cf000fce0 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4d0a8aa740 10 client.27986 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4d0a8aa740 10 client.27986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4d0a8aa740 20 client.27986 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd800f20 0x55d8bd803310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4d0a8aa740 4 client.27986 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8bd754df0 con 0x55d8bd800f20 2024-09-10T21:21:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.492+0000 7f4d06271640 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd800f20 0x55d8bd803310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.493+0000 7f4d06271640 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd800f20 0x55d8bd803310 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5473b1219d16b568 server_cookie=60edcb9333eabd29 in_seq=0 out_seq=0 2024-09-10T21:21:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.493+0000 7f4ce9ffb640 10 client.27986 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.494+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/804420503 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8bd754df0 con 0x55d8bd800f20 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.494+0000 7f4ce9ffb640 10 client.27986 handle_command_reply: tid=0 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.494+0000 7f4d0a8aa740 1 client.27986 shutdown 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.494+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dcdf0 msgr2=0x55d8bd7e5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.494+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e5a60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0005830 tx=0x7f4cf0005740 comp rx=0 tx=0).stop 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 shutdown_connections 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd800f20 0x55d8bd803310 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7e5a60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/804420503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 >> 172.21.15.33:0/804420503 conn(0x55d8bd7da300 msgr2=0x55d8bd7dfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4ce9ffb640 0 client.27986 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 shutdown_connections 2024-09-10T21:21:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/804420503 wait complete. 2024-09-10T21:21:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.495+0000 7f4d0a8aa740 20 client.27986 trim_cache size 0 max 16384 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.518+0000 7f4d0a8aa740 1 Processor -- start 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.518+0000 7f4d0a8aa740 1 -- start start 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.518+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.518+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7ea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7ea6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400e4c0 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4010000 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400eb90 con 0x55d8bd7e8d80 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51664/0 (socket says 172.21.15.33:51664) 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 -- 172.21.15.33:0/1752290311 learned_addr learned my addr 172.21.15.33:0/1752290311 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d05a70640 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 -- 172.21.15.33:0/1752290311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 msgr2=0x55d8bd7ea6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7ea6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 -- 172.21.15.33:0/1752290311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dcdf0 msgr2=0x55d8bd7ea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7ea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.519+0000 7f4d06271640 1 -- 172.21.15.33:0/1752290311 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bd7e5810 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.520+0000 7f4d05a70640 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:23.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.520+0000 7f4d06271640 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e77a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc103bd0 tx=0x7f4cfc106330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a8d733e8eddaf0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:23.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.520+0000 7f4cebfff640 1 -- 172.21.15.33:0/1752290311 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0d1120 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.520+0000 7f4cebfff640 1 -- 172.21.15.33:0/1752290311 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cfc0ccf40 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.520+0000 7f4cebfff640 1 -- 172.21.15.33:0/1752290311 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0ccaa0 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/1752290311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 msgr2=0x55d8bd7e77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e77a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc103bd0 tx=0x7f4cfc106330 comp rx=0 tx=0).stop 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/1752290311 shutdown_connections 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7ea6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/1752290311 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7e77a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/1752290311 >> 172.21.15.33:0/1752290311 conn(0x55d8bd7da300 msgr2=0x55d8bd7e7090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/1752290311 shutdown_connections 2024-09-10T21:21:23.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.521+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/1752290311 wait complete. 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 -- start start 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 Processor -- start 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7f9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400f930 con 0x55d8bd7e8d80 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400f260 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce400c480 con 0x55d8bd7dcdf0 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d0a8aa740 10 client.0 _ll_register_callbacks cb 0x7f4cf8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.522+0000 7f4d06271640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51676/0 (socket says 172.21.15.33:51676) 2024-09-10T21:21:23.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 -- 172.21.15.33:0/2016438649 learned_addr learned my addr 172.21.15.33:0/2016438649 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d05a70640 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06a72640 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 -- 172.21.15.33:0/2016438649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 msgr2=0x55d8bd7f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 -- 172.21.15.33:0/2016438649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 msgr2=0x55d8bd7e1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 -- 172.21.15.33:0/2016438649 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bd7e22e0 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d06271640 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7f9790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006010 tx=0x7f4cfc02a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b4135893f41410b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4ce9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/2016438649 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc0f3020 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cfc0ccf40 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.523+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/2016438649 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4cfc0ccca0 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.524+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/2016438649 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4cfc0d1320 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.524+0000 7f4d0a8aa740 10 client.27992 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.524+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d8bd7e35a0 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.524+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/2016438649 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cfc0ccf40 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/2016438649 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f4cfc0d59d0 con 0x55d8bd7dc4d0 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4d0a8aa740 10 client.27992 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4d0a8aa740 10 client.27992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4d0a8aa740 20 client.27992 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd86eb80 0x55d8bd870f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4d0a8aa740 4 client.27992 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d8bd7508a0 con 0x55d8bd86eb80 2024-09-10T21:21:23.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.525+0000 7f4d05a70640 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd86eb80 0x55d8bd870f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:23.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.526+0000 7f4d05a70640 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd86eb80 0x55d8bd870f70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3a35819a34fdd09 server_cookie=168aba908192380b in_seq=0 out_seq=0 2024-09-10T21:21:23.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.526+0000 7f4ce9ffb640 10 client.27992 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:23.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4ce9ffb640 1 -- 172.21.15.33:0/2016438649 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d8bd7508a0 con 0x55d8bd86eb80 2024-09-10T21:21:23.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4ce9ffb640 10 client.27992 handle_command_reply: tid=0 2024-09-10T21:21:23.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 client.27992 shutdown 2024-09-10T21:21:23.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 msgr2=0x55d8bd7f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7f9790 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc006010 tx=0x7f4cfc02a410 comp rx=0 tx=0).stop 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 shutdown_connections 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d8bd86eb80 0x55d8bd870f70 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4ce9ffb640 0 client.27992 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d8bd7e8d80 0x55d8bd7e1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8bd7dcdf0 0x55d8bd7f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 --2- 172.21.15.33:0/2016438649 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d8bd7dc4d0 0x55d8bd7f9790 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 >> 172.21.15.33:0/2016438649 conn(0x55d8bd7da300 msgr2=0x55d8bd7e78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.527+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 shutdown_connections 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d0a8aa740 1 -- 172.21.15.33:0/2016438649 wait complete. 2024-09-10T21:21:23.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d0a8aa740 20 client.27992 trim_cache size 0 max 16384 2024-09-10T21:21:23.536 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:21:23.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cec0b5360 msgr2=0x7f4cec0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cec0b5360 0x7f4cec0b7820 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009fd0 tx=0x7f4cf8009500 comp rx=0 tx=0).stop 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d000754e0 msgr2=0x7f4d0019b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d000754e0 0x7f4d0019b4d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005940 tx=0x7f4cfc0040c0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 shutdown_connections 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4cec0b5360 0x7f4cec0b7820 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d0010f3b0 0x7f4d001a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d00075e00 0x7f4d0019ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.528+0000 7f4d084d5640 1 --2- 172.21.15.33:0/2112532306 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4d000754e0 0x7f4d0019b4d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.529+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 >> 172.21.15.33:0/2112532306 conn(0x7f4d000fe870 msgr2=0x7f4d000ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.529+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 shutdown_connections 2024-09-10T21:21:23.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:23.529+0000 7f4d084d5640 1 -- 172.21.15.33:0/2112532306 wait complete. 2024-09-10T21:21:24.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 215405 42.05 MB/sec execute 144 sec latency 70.502 ms 2024-09-10T21:21:24.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:23 smithi104 ceph-mon[35325]: pgmap v868: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 763 op/s 2024-09-10T21:21:24.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:23 smithi174 ceph-mon[35635]: pgmap v868: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 763 op/s 2024-09-10T21:21:24.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:23 smithi033 ceph-mon[30860]: pgmap v868: 97 pgs: 97 active+clean; 1.2 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 763 op/s 2024-09-10T21:21:25.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 215860 41.91 MB/sec execute 145 sec latency 290.867 ms 2024-09-10T21:21:26.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 216721 41.93 MB/sec execute 146 sec latency 78.885 ms 2024-09-10T21:21:26.161 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:21:26.161 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:21:26.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:25 smithi104 ceph-mon[35325]: pgmap v869: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T21:21:26.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:25 smithi174 ceph-mon[35635]: pgmap v869: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T21:21:26.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:25 smithi033 ceph-mon[30860]: pgmap v869: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T21:21:26.496 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 -- 172.21.15.33:0/1449207781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1044a0 msgr2=0x7fe4bc104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/1449207781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1044a0 0x7fe4bc104900 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009900 tx=0x7fe4b002a410 comp rx=0 tx=0).stop 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 -- 172.21.15.33:0/1449207781 shutdown_connections 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/1449207781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4bc104e40 0x7fe4bc10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/1449207781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1044a0 0x7fe4bc104900 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/1449207781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4bc1023c0 0x7fe4bc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 -- 172.21.15.33:0/1449207781 >> 172.21.15.33:0/1449207781 conn(0x7fe4bc078140 msgr2=0x7fe4bc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 -- 172.21.15.33:0/1449207781 shutdown_connections 2024-09-10T21:21:27.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.044+0000 7fe4c22ec640 1 -- 172.21.15.33:0/1449207781 wait complete. 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.045+0000 7fe4c22ec640 1 Processor -- start 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.045+0000 7fe4c22ec640 1 -- start start 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c22ec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1023c0 0x7fe4bc199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c22ec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4bc1044a0 0x7fe4bc1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c22ec640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4bc104e40 0x7fe4bc19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c22ec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc112e70 con 0x7fe4bc1023c0 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c22ec640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc101f20 con 0x7fe4bc104e40 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c22ec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4bc112ff0 con 0x7fe4bc1044a0 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1023c0 0x7fe4bc199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4bc1044a0 0x7fe4bc1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4bc1044a0 0x7fe4bc1997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36464/0 (socket says 172.21.15.33:36464) 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/2101088580 learned_addr learned my addr 172.21.15.33:0/2101088580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1023c0 0x7fe4bc199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60128/0 (socket says 172.21.15.33:60128) 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c0889640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4bc104e40 0x7fe4bc19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c0889640 1 -- 172.21.15.33:0/2101088580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4bc1044a0 msgr2=0x7fe4bc1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c0889640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4bc1044a0 0x7fe4bc1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c0889640 1 -- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1023c0 msgr2=0x7fe4bc199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c0889640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1023c0 0x7fe4bc199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.046+0000 7fe4c0889640 1 -- 172.21.15.33:0/2101088580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4bc19f340 con 0x7fe4bc104e40 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.047+0000 7fe4c0889640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4bc104e40 0x7fe4bc19eb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4005fd0 tx=0x7fe4b400ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.047+0000 7fe4b97fa640 1 -- 172.21.15.33:0/2101088580 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400cd80 con 0x7fe4bc104e40 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.047+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4bc19f630 con 0x7fe4bc104e40 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.047+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4bc110c60 con 0x7fe4bc104e40 2024-09-10T21:21:27.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.047+0000 7fe4b97fa640 1 -- 172.21.15.33:0/2101088580 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4b4005020 con 0x7fe4bc104e40 2024-09-10T21:21:27.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.047+0000 7fe4b97fa640 1 -- 172.21.15.33:0/2101088580 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4010690 con 0x7fe4bc104e40 2024-09-10T21:21:27.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.050+0000 7fe4c46c1740 1 Processor -- start 2024-09-10T21:21:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4b97fa640 1 -- 172.21.15.33:0/2101088580 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe4b4004a40 con 0x7fe4bc104e40 2024-09-10T21:21:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4c46c1740 1 -- start start 2024-09-10T21:21:27.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60ea982c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60eaa4790 0x55c60eaa6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c60ea0ee60 con 0x55c60eaa4790 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c60e9fe290 con 0x55c60ea98800 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c60ea020a0 con 0x55c60ea97ee0 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60eaa4790 0x55c60eaa6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60eaa4790 0x55c60eaa6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60132/0 (socket says 172.21.15.33:60132) 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4bbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51700/0 (socket says 172.21.15.33:51700) 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/4098580231 learned_addr learned my addr 172.21.15.33:0/4098580231 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:27.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.051+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60ea982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4bbfff640 1 -- 172.21.15.33:0/4098580231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 msgr2=0x55c60ea982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60ea982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4b97fa640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4a40b5360 0x7fe4a40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4bbfff640 1 -- 172.21.15.33:0/4098580231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60eaa4790 msgr2=0x55c60eaa6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60eaa4790 0x55c60eaa6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4bbfff640 1 -- 172.21.15.33:0/4098580231 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60e75c930 con 0x55c60ea98800 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4b97fa640 1 -- 172.21.15.33:0/2101088580 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe4b40e5f60 con 0x7fe4bc104e40 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4c0889640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4a40b5360 0x7fe4a40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa4250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac010d30 tx=0x7fe4ac005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d743fd6084530cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.052+0000 7fe4a27fc640 1 -- 172.21.15.33:0/4098580231 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac010460 con 0x55c60ea98800 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.053+0000 7fe4a27fc640 1 -- 172.21.15.33:0/4098580231 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ac0035f0 con 0x55c60ea98800 2024-09-10T21:21:27.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.053+0000 7fe4a27fc640 1 -- 172.21.15.33:0/4098580231 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac019670 con 0x55c60ea98800 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.053+0000 7fe4c0889640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4a40b5360 0x7fe4a40b7820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe4b401a040 tx=0x7fe4b40059b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.053+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4098580231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 msgr2=0x55c60eaa4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.053+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa4250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac010d30 tx=0x7fe4ac005f90 comp rx=0 tx=0).stop 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4098580231 shutdown_connections 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60eaa4790 0x55c60eaa6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa4250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4098580231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60ea982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4098580231 >> 172.21.15.33:0/4098580231 conn(0x55c60ea95d10 msgr2=0x55c60ea96eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4098580231 shutdown_connections 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4098580231 wait complete. 2024-09-10T21:21:27.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.054+0000 7fe4c46c1740 1 -- start start 2024-09-10T21:21:27.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.055+0000 7fe4c46c1740 1 Processor -- start 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.055+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea97ee0 0x55c60eaa16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.055+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60eaa1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea97ee0 0x55c60eaa16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea97ee0 0x55c60eaa16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51712/0 (socket says 172.21.15.33:51712) 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa4790 0x55c60eaa2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/4074342468 learned_addr learned my addr 172.21.15.33:0/4074342468 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c60e85f890 con 0x55c60ea98800 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60eaa1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c60e998d40 con 0x55c60ea97ee0 2024-09-10T21:21:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c60eaa74e0 con 0x55c60eaa4790 2024-09-10T21:21:27.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.056+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa4790 0x55c60eaa2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 217528 41.88 MB/sec execute 147 sec latency 92.760 ms 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/4074342468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa4790 msgr2=0x55c60eaa2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa4790 0x55c60eaa2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/4074342468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 msgr2=0x55c60eaa1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60eaa1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/4074342468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c60eab08d0 con 0x55c60ea97ee0 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4c46c1740 10 client.0 _ll_register_callbacks cb 0x55c60e9fcc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60eaa1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4ac0035f0 con 0x55c60ea97ee0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea97ee0 0x55c60eaa16d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0005940 tx=0x7fe4b002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d295f2c19aaae86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe49b7fe640 10 client.28004 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe49b7fe640 1 -- 172.21.15.33:0/4074342468 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0035040 con 0x55c60ea97ee0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe49b7fe640 1 -- 172.21.15.33:0/4074342468 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe4b0005450 con 0x55c60ea97ee0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe49b7fe640 1 -- 172.21.15.33:0/4074342468 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe4b0005650 con 0x55c60ea97ee0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4c46c1740 10 client.28004 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.057+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=81}) -- 0x55c60eab3160 con 0x55c60ea97ee0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe49b7fe640 1 -- 172.21.15.33:0/4074342468 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fe4b0004770 con 0x55c60ea97ee0 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe4c46c1740 10 client.28004 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe4c46c1740 10 client.28004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe4c46c1740 20 client.28004 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eab4500 0x55c60eab6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe4c46c1740 4 client.28004 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c60ea10db0 con 0x55c60eab4500 2024-09-10T21:21:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.058+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eab4500 0x55c60eab6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.059+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eab4500 0x55c60eab6950 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c69ccd83a69136a server_cookie=dfa0f596023676e2 in_seq=0 out_seq=0 2024-09-10T21:21:27.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.059+0000 7fe49b7fe640 10 client.28004 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.061+0000 7fe49b7fe640 1 -- 172.21.15.33:0/4074342468 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c60ea10db0 con 0x55c60eab4500 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.061+0000 7fe49b7fe640 10 client.28004 handle_command_reply: tid=0 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.061+0000 7fe4c46c1740 1 client.28004 shutdown 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea97ee0 msgr2=0x55c60eaa16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea97ee0 0x55c60eaa16d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0005940 tx=0x7fe4b002a410 comp rx=0 tx=0).stop 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 shutdown_connections 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eab4500 0x55c60eab6950 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa4790 0x55c60eaa2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60eaa1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe49b7fe640 0 client.28004 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:27.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/4074342468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea97ee0 0x55c60eaa16d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 >> 172.21.15.33:0/4074342468 conn(0x55c60ea95d10 msgr2=0x55c60ea94f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 shutdown_connections 2024-09-10T21:21:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 1 -- 172.21.15.33:0/4074342468 wait complete. 2024-09-10T21:21:27.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.062+0000 7fe4c46c1740 20 client.28004 trim_cache size 0 max 16384 2024-09-10T21:21:27.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 Processor -- start 2024-09-10T21:21:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 -- start start 2024-09-10T21:21:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60eaa3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60ea9e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60eaa4790 0x55c60ea9ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a800e440 con 0x55c60ea98800 2024-09-10T21:21:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a800ff80 con 0x55c60eaa4790 2024-09-10T21:21:27.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.084+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a800eb10 con 0x55c60ea97ee0 2024-09-10T21:21:27.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60eaa3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60eaa4790 0x55c60ea9ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60eaa3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36484/0 (socket says 172.21.15.33:36484) 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/2615782868 learned_addr learned my addr 172.21.15.33:0/2615782868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60ea9e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/2615782868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 msgr2=0x55c60eaa3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60eaa3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/2615782868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 msgr2=0x55c60ea9e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60ea9e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/2615782868 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c60eaa05c0 con 0x55c60eaa4790 2024-09-10T21:21:27.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.085+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60ea9e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:27.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.086+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60eaa4790 0x55c60ea9ee20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe4b002a450 tx=0x7fe4b0049000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1b9bd8a6bb16d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:27.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.086+0000 7fe4a1ffb640 1 -- 172.21.15.33:0/2615782868 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0000e60 con 0x55c60eaa4790 2024-09-10T21:21:27.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.086+0000 7fe4a1ffb640 1 -- 172.21.15.33:0/2615782868 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4ac0035f0 con 0x55c60eaa4790 2024-09-10T21:21:27.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.086+0000 7fe4a1ffb640 1 -- 172.21.15.33:0/2615782868 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0035070 con 0x55c60eaa4790 2024-09-10T21:21:27.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 -- 172.21.15.33:0/2615782868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60eaa4790 msgr2=0x55c60ea9ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60eaa4790 0x55c60ea9ee20 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe4b002a450 tx=0x7fe4b0049000 comp rx=0 tx=0).stop 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 -- 172.21.15.33:0/2615782868 shutdown_connections 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60eaa4790 0x55c60ea9ee20 secure :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe4b002a450 tx=0x7fe4b0049000 comp rx=0 tx=0).stop 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea98800 0x55c60ea9e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/2615782868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60ea97ee0 0x55c60eaa3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 -- 172.21.15.33:0/2615782868 >> 172.21.15.33:0/2615782868 conn(0x55c60ea95d10 msgr2=0x55c60eaa2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 -- 172.21.15.33:0/2615782868 shutdown_connections 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.087+0000 7fe4c46c1740 1 -- 172.21.15.33:0/2615782868 wait complete. 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 -- start start 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 Processor -- start 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea97ee0 0x55c60eaad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa9ad0 0x55c60eaa9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a800f8b0 con 0x55c60ea97ee0 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a800f1e0 con 0x55c60ea98800 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a800c400 con 0x55c60eaa9ad0 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4c46c1740 10 client.0 _ll_register_callbacks cb 0x7fe4b0031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea97ee0 0x55c60eaad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4bb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea97ee0 0x55c60eaad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60168/0 (socket says 172.21.15.33:60168) 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.088+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/453838145 learned_addr learned my addr 172.21.15.33:0/453838145 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa9ad0 0x55c60eaa9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/453838145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa9ad0 msgr2=0x55c60eaa9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa9ad0 0x55c60eaa9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/453838145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 msgr2=0x55c60eaa94f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa94f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4bb7fe640 1 -- 172.21.15.33:0/453838145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c60eb216e0 con 0x55c60ea97ee0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4bb7fe640 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea97ee0 0x55c60eaad0b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009a20 tx=0x7fe4b0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c89644d2e17656 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe49b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe49b7fe640 1 -- 172.21.15.33:0/453838145 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0073060 con 0x55c60ea97ee0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe49b7fe640 1 -- 172.21.15.33:0/453838145 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe4b0004030 con 0x55c60ea97ee0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.089+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4ac0035f0 con 0x55c60ea97ee0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe49b7fe640 1 -- 172.21.15.33:0/453838145 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fe4b0035070 con 0x55c60ea97ee0 2024-09-10T21:21:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 10 client.18609 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55c60eb22a00 con 0x55c60ea97ee0 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe49b7fe640 1 -- 172.21.15.33:0/453838145 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4b0004230 con 0x55c60ea97ee0 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe49b7fe640 1 -- 172.21.15.33:0/453838145 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fe4b0078070 con 0x55c60ea97ee0 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 10 client.18609 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 10 client.18609 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 20 client.18609 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eb231b0 0x55c60eb255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 4 client.18609 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.090+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fe4ac0035f0 con 0x55c60eb231b0 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.091+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eb231b0 0x55c60eb255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.091+0000 7fe49b7fe640 10 client.18609 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.091+0000 7fe4bbfff640 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eb231b0 0x55c60eb255d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e41cde24b73f6f server_cookie=bf96c91a0a8182e0 in_seq=0 out_seq=0 2024-09-10T21:21:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe49b7fe640 1 -- 172.21.15.33:0/453838145 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fe4ac0035f0 con 0x55c60eb231b0 2024-09-10T21:21:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe49b7fe640 10 client.18609 handle_command_reply: tid=0 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 client.18609 shutdown 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea97ee0 msgr2=0x55c60eaad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea97ee0 0x55c60eaad0b0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009a20 tx=0x7fe4b0005c90 comp rx=0 tx=0).stop 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 shutdown_connections 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c60eb231b0 0x55c60eb255d0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c60eaa9ad0 0x55c60eaa9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c60ea98800 0x55c60eaa94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe49b7fe640 0 client.18609 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 --2- 172.21.15.33:0/453838145 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c60ea97ee0 0x55c60eaad0b0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 >> 172.21.15.33:0/453838145 conn(0x55c60ea95d10 msgr2=0x55c60eaa3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 shutdown_connections 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.093+0000 7fe4c46c1740 1 -- 172.21.15.33:0/453838145 wait complete. 2024-09-10T21:21:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c46c1740 20 client.18609 trim_cache size 0 max 16384 2024-09-10T21:21:27.101 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:21:27.101 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:21:27.101 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "1737cca1-c03b-4eef-8451-596ad722fe8c", 2024-09-10T21:21:27.101 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:21:27.101 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4a40b5360 msgr2=0x7fe4a40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4a40b5360 0x7fe4a40b7820 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe4b401a040 tx=0x7fe4b40059b0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4bc104e40 msgr2=0x7fe4bc19eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4bc104e40 0x7fe4bc19eb70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4005fd0 tx=0x7fe4b400ee60 comp rx=0 tx=0).stop 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 shutdown_connections 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4a40b5360 0x7fe4a40b7820 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe4bc104e40 0x7fe4bc19eb70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4bc1044a0 0x7fe4bc1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 --2- 172.21.15.33:0/2101088580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4bc1023c0 0x7fe4bc199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 >> 172.21.15.33:0/2101088580 conn(0x7fe4bc078140 msgr2=0x7fe4bc0ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 shutdown_connections 2024-09-10T21:21:27.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:27.094+0000 7fe4c22ec640 1 -- 172.21.15.33:0/2101088580 wait complete. 2024-09-10T21:21:28.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:21:28.059 INFO:tasks.workunit.client.0.smithi033.stdout: 10 218093 41.83 MB/sec execute 148 sec latency 395.414 ms 2024-09-10T21:21:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: pgmap v870: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 774 op/s 2024-09-10T21:21:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:21:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:21:28.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:21:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:21:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:21:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:21:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:21:28.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:21:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.317 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:21:28.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: pgmap v870: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 774 op/s 2024-09-10T21:21:28.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:21:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:21:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:21:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:21:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:21:28.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:21:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:21:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:21:28.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: pgmap v870: 97 pgs: 97 active+clean; 1.2 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 774 op/s 2024-09-10T21:21:28.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:21:28.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:21:28.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:28.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 218958 41.92 MB/sec execute 149 sec latency 144.384 ms 2024-09-10T21:21:29.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.807+0000 7f34e52dc640 1 -- 172.21.15.33:0/1563564738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108f90 msgr2=0x7f34e010f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.807+0000 7f34e52dc640 1 --2- 172.21.15.33:0/1563564738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108f90 0x7f34e010f790 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009ca0 tx=0x7f34d802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 -- 172.21.15.33:0/1563564738 shutdown_connections 2024-09-10T21:21:29.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 --2- 172.21.15.33:0/1563564738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108f90 0x7f34e010f790 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 --2- 172.21.15.33:0/1563564738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108610 0x7f34e0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 --2- 172.21.15.33:0/1563564738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e0102c00 0x7f34e0102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 -- 172.21.15.33:0/1563564738 >> 172.21.15.33:0/1563564738 conn(0x7f34e00fe670 msgr2=0x7f34e0100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 -- 172.21.15.33:0/1563564738 shutdown_connections 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 -- 172.21.15.33:0/1563564738 wait complete. 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.808+0000 7f34e52dc640 1 Processor -- start 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e52dc640 1 -- start start 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e52dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e0102c00 0x7f34e0198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e52dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108610 0x7f34e01994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e52dc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108f90 0x7f34e019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e52dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0112c50 con 0x7f34e0108610 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e4adb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108f90 0x7f34e019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34dffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e0102c00 0x7f34e0198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e4adb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108f90 0x7f34e019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51748/0 (socket says 172.21.15.33:51748) 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e4adb640 1 -- 172.21.15.33:0/157259140 learned_addr learned my addr 172.21.15.33:0/157259140 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34dffff640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e0102c00 0x7f34e0198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36524/0 (socket says 172.21.15.33:36524) 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0112ad0 con 0x7f34e0108f90 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34e0112dd0 con 0x7f34e0102c00 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.809+0000 7f34df7fe640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108610 0x7f34e01994c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34e4adb640 1 -- 172.21.15.33:0/157259140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e0102c00 msgr2=0x7f34e0198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34e4adb640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e0102c00 0x7f34e0198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34e4adb640 1 -- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108610 msgr2=0x7f34e01994c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34e4adb640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108610 0x7f34e01994c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34e4adb640 1 -- 172.21.15.33:0/157259140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34e019f030 con 0x7f34e0108f90 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34df7fe640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108610 0x7f34e01994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34e4adb640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108f90 0x7f34e019e860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005cc0 tx=0x7f34d8031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34dd7fa640 1 -- 172.21.15.33:0/157259140 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d8003b70 con 0x7f34e0108f90 2024-09-10T21:21:29.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34dd7fa640 1 -- 172.21.15.33:0/157259140 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d80318f0 con 0x7f34e0108f90 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.810+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34e019f2c0 con 0x7f34e0108f90 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.811+0000 7f34dd7fa640 1 -- 172.21.15.33:0/157259140 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d802f950 con 0x7f34e0108f90 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.811+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34e01a59e0 con 0x7f34e0108f90 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.813+0000 7f34e8913740 1 Processor -- start 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.814+0000 7f34dd7fa640 1 -- 172.21.15.33:0/157259140 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f34d802fb50 con 0x7f34e0108f90 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.814+0000 7f34e8913740 1 -- start start 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34e8913740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0c650 0x5616c4a0ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34e8913740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34e8913740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a18f00 0x5616c4a1b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34e8913740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616c4982e50 con 0x5616c4a0c650 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34e8913740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616c4972280 con 0x5616c4a18f00 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34e8913740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5616c4976090 con 0x5616c4a0cf70 2024-09-10T21:21:29.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0c650 0x5616c4a0ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0c650 0x5616c4a0ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60194/0 (socket says 172.21.15.33:60194) 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 -- 172.21.15.33:0/3673822608 learned_addr learned my addr 172.21.15.33:0/3673822608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34dffff640 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a18f00 0x5616c4a1b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 -- 172.21.15.33:0/3673822608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 msgr2=0x5616c4a189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 -- 172.21.15.33:0/3673822608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a18f00 msgr2=0x5616c4a1b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a18f00 0x5616c4a1b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34df7fe640 1 -- 172.21.15.33:0/3673822608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616c46d0650 con 0x5616c4a0c650 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.815+0000 7f34dd7fa640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f34c80b5390 0x7f34c80b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.816+0000 7f34dffff640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f34c80b5390 0x7f34c80b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.816+0000 7f34df7fe640 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0c650 0x5616c4a0ca30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f34d401ca50 tx=0x7f34d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f1ec8f25fde032 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.816+0000 7f34dd7fa640 1 -- 172.21.15.33:0/157259140 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f34d8106cf0 con 0x7f34e0108f90 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.816+0000 7f34c67fc640 1 -- 172.21.15.33:0/3673822608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4022070 con 0x5616c4a0c650 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.816+0000 7f34c67fc640 1 -- 172.21.15.33:0/3673822608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d4013a50 con 0x5616c4a0c650 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34c67fc640 1 -- 172.21.15.33:0/3673822608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d40035f0 con 0x5616c4a0c650 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 -- 172.21.15.33:0/3673822608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0c650 msgr2=0x5616c4a0ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0c650 0x5616c4a0ca30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f34d401ca50 tx=0x7f34d4009c00 comp rx=0 tx=0).stop 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34dffff640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f34c80b5390 0x7f34c80b7850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5616c4a0b360 tx=0x7f34d0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 -- 172.21.15.33:0/3673822608 shutdown_connections 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a18f00 0x5616c4a1b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 --2- 172.21.15.33:0/3673822608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0c650 0x5616c4a0ca30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 -- 172.21.15.33:0/3673822608 >> 172.21.15.33:0/3673822608 conn(0x5616c4a0a480 msgr2=0x5616c4a0b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 -- 172.21.15.33:0/3673822608 shutdown_connections 2024-09-10T21:21:29.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.817+0000 7f34e8913740 1 -- 172.21.15.33:0/3673822608 wait complete. 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.818+0000 7f34e8913740 1 -- start start 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.818+0000 7f34e8913740 1 Processor -- start 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e8913740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a187d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e8913740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a15c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e8913740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a16200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34dffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a187d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e4adb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a16200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e4adb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a16200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60196/0 (socket says 172.21.15.33:60196) 2024-09-10T21:21:29.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34dffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a187d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51770/0 (socket says 172.21.15.33:51770) 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a15c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e4adb640 1 -- 172.21.15.33:0/4031170485 learned_addr learned my addr 172.21.15.33:0/4031170485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5616c47d3700 con 0x5616c4a18f00 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5616c490d110 con 0x5616c4a0c650 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5616c4a1bbd0 con 0x5616c4a0cf70 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.819+0000 7f34e8913740 10 client.0 _ll_register_callbacks cb 0x5616c4926990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e4adb640 1 -- 172.21.15.33:0/4031170485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 msgr2=0x5616c4a15c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e4adb640 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a15c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e4adb640 1 -- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 msgr2=0x5616c4a187d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e4adb640 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a187d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e4adb640 1 -- 172.21.15.33:0/4031170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616c493ac40 con 0x5616c4a18f00 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34dffff640 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a187d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:29.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e4adb640 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a16200 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f34d80c9a80 tx=0x7f34d80ccce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fc9f34488152c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34bf7fe640 1 -- 172.21.15.33:0/4031170485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80d1f90 con 0x5616c4a18f00 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d4013a50 con 0x5616c4a18f00 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34bf7fe640 1 -- 172.21.15.33:0/4031170485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f34d80c9720 con 0x5616c4a18f00 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34bf7fe640 1 -- 172.21.15.33:0/4031170485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f34d80d6a40 con 0x5616c4a18f00 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e8913740 10 client.18621 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.820+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5616c4a27570 con 0x5616c4a18f00 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34bf7fe640 1 -- 172.21.15.33:0/4031170485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d4013a50 con 0x5616c4a18f00 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34bf7fe640 1 -- 172.21.15.33:0/4031170485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f34d80d6640 con 0x5616c4a18f00 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34e8913740 10 client.18621 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34e8913740 10 client.18621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34e8913740 20 client.18621 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:29.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34e8913740 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a28a00 0x5616c4a2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34e8913740 4 client.18621 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616c4984da0 con 0x5616c4a28a00 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.821+0000 7f34dffff640 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a28a00 0x5616c4a2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.822+0000 7f34bf7fe640 10 client.18621 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.822+0000 7f34dffff640 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a28a00 0x5616c4a2ae60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c295305f96e43230 server_cookie=fb827bf5f92a8b3b in_seq=0 out_seq=0 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34bf7fe640 1 -- 172.21.15.33:0/4031170485 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616c4984da0 con 0x5616c4a28a00 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34bf7fe640 10 client.18621 handle_command_reply: tid=0 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 client.18621 shutdown 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 msgr2=0x5616c4a16200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a16200 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f34d80c9a80 tx=0x7f34d80ccce0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 shutdown_connections 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a28a00 0x5616c4a2ae60 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a16200 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a15c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 --2- 172.21.15.33:0/4031170485 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a187d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 >> 172.21.15.33:0/4031170485 conn(0x5616c4a0a480 msgr2=0x5616c4a0fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34bf7fe640 0 client.18621 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 shutdown_connections 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 1 -- 172.21.15.33:0/4031170485 wait complete. 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.823+0000 7f34e8913740 20 client.18621 trim_cache size 0 max 16384 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 Processor -- start 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 -- start start 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a09e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a0b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a1a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01abdd0 con 0x5616c4a18f00 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01ad910 con 0x5616c4a0c650 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e8913740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01ac4a0 con 0x5616c4a0cf70 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a1a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a1a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60206/0 (socket says 172.21.15.33:60206) 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 -- 172.21.15.33:0/1830291947 learned_addr learned my addr 172.21.15.33:0/1830291947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34dffff640 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a09e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34df7fe640 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a0b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 -- 172.21.15.33:0/1830291947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 msgr2=0x5616c4a0b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a0b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 -- 172.21.15.33:0/1830291947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 msgr2=0x5616c4a09e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a09e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.846+0000 7f34e4adb640 1 -- 172.21.15.33:0/1830291947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616c4a0cda0 con 0x5616c4a18f00 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.847+0000 7f34e4adb640 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a1a2f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009a60 tx=0x7f34d80ccd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7daa4a7e775a0d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.847+0000 7f34c5ffb640 1 -- 172.21.15.33:0/1830291947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80c9b00 con 0x5616c4a18f00 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.847+0000 7f34c5ffb640 1 -- 172.21.15.33:0/1830291947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d8036070 con 0x5616c4a18f00 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.847+0000 7f34c5ffb640 1 -- 172.21.15.33:0/1830291947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d80cd710 con 0x5616c4a18f00 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.847+0000 7f34e8913740 1 -- 172.21.15.33:0/1830291947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 msgr2=0x5616c4a1a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.847+0000 7f34e8913740 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a1a2f0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f34d8009a60 tx=0x7f34d80ccd50 comp rx=0 tx=0).stop 2024-09-10T21:21:29.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- 172.21.15.33:0/1830291947 shutdown_connections 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a18f00 0x5616c4a1a2f0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a0cf70 0x5616c4a0b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 --2- 172.21.15.33:0/1830291947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a09e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- 172.21.15.33:0/1830291947 >> 172.21.15.33:0/1830291947 conn(0x5616c4a0a480 msgr2=0x5616c4a144a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- 172.21.15.33:0/1830291947 shutdown_connections 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- 172.21.15.33:0/1830291947 wait complete. 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- start start 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 Processor -- start 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a213b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0cf70 0x5616c4a21930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a18f00 0x5616c4a1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01ad240 con 0x5616c4a0cf70 2024-09-10T21:21:29.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01acb70 con 0x5616c4a0c650 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.848+0000 7f34e8913740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34e01a9db0 con 0x5616c4a18f00 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0cf70 0x5616c4a21930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0cf70 0x5616c4a21930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60212/0 (socket says 172.21.15.33:60212) 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 -- 172.21.15.33:0/3892741537 learned_addr learned my addr 172.21.15.33:0/3892741537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34e8913740 10 client.0 _ll_register_callbacks cb 0x7f34d8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34dffff640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a213b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34e4adb640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a18f00 0x5616c4a1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 -- 172.21.15.33:0/3892741537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a18f00 msgr2=0x5616c4a1c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a18f00 0x5616c4a1c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 -- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 msgr2=0x5616c4a213b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a213b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 -- 172.21.15.33:0/3892741537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616c4a1c930 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34e4adb640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a18f00 0x5616c4a1c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34df7fe640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0cf70 0x5616c4a21930 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f34d401ca20 tx=0x7f34d4013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab491944eb1259d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34bf7fe640 1 -- 172.21.15.33:0/3892741537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34d4022040 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34bf7fe640 1 -- 172.21.15.33:0/3892741537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f34d401c310 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.849+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34d8036070 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34bf7fe640 1 -- 172.21.15.33:0/3892741537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f34d401c510 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 10 client.18627 fetch_fsmap learned FSMap version 81 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5616c4a96670 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34bf7fe640 1 -- 172.21.15.33:0/3892741537 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34d8036070 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34bf7fe640 1 -- 172.21.15.33:0/3892741537 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f34d4030050 con 0x5616c4a0cf70 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 10 client.18627 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 10 client.18627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:21:29.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 20 client.18627 populate_metadata read hostname 'smithi033' 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a96bb0 0x5616c4a98fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 4 client.18627 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616c4980850 con 0x5616c4a96bb0 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.850+0000 7f34dffff640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a96bb0 0x5616c4a98fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.852+0000 7f34dffff640 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a96bb0 0x5616c4a98fd0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8781895ff4c02c4a server_cookie=4bbb575a41dba24b in_seq=0 out_seq=0 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.852+0000 7f34bf7fe640 10 client.18627 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.852+0000 7f34bf7fe640 1 -- 172.21.15.33:0/3892741537 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6818 (crc 0 0 0) 0x5616c4980850 con 0x5616c4a96bb0 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.852+0000 7f34bf7fe640 10 client.18627 handle_command_reply: tid=0 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.852+0000 7f34e8913740 1 client.18627 shutdown 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.852+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0cf70 msgr2=0x5616c4a21930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.852+0000 7f34e8913740 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0cf70 0x5616c4a21930 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f34d401ca20 tx=0x7f34d4013f90 comp rx=0 tx=0).stop 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 shutdown_connections 2024-09-10T21:21:29.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5616c4a96bb0 0x5616c4a98fd0 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5616c4a18f00 0x5616c4a1c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34bf7fe640 0 client.18627 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5616c4a0cf70 0x5616c4a21930 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 --2- 172.21.15.33:0/3892741537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5616c4a0c650 0x5616c4a213b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 >> 172.21.15.33:0/3892741537 conn(0x5616c4a0a480 msgr2=0x5616c4a17250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:21:29.157 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "1737cca1-c03b-4eef-8451-596ad722fe8c", 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:21:29.158 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.161 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.162 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:21:29.163 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.166 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.167 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.168 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:21:29.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:21:29.172 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:21:29.173 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:21:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 shutdown_connections 2024-09-10T21:21:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 1 -- 172.21.15.33:0/3892741537 wait complete. 2024-09-10T21:21:29.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.853+0000 7f34e8913740 20 client.18627 trim_cache size 0 max 16384 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f34c80b5390 msgr2=0x7f34c80b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f34c80b5390 0x7f34c80b7850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x5616c4a0b360 tx=0x7f34d0005f90 comp rx=0 tx=0).stop 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108f90 msgr2=0x7f34e019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108f90 0x7f34e019e860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f34d8005cc0 tx=0x7f34d8031700 comp rx=0 tx=0).stop 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 shutdown_connections 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f34c80b5390 0x7f34c80b7850 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f34e0108f90 0x7f34e019e860 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34e0108610 0x7f34e01994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 --2- 172.21.15.33:0/157259140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34e0102c00 0x7f34e0198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:21:29.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 >> 172.21.15.33:0/157259140 conn(0x7f34e00fe670 msgr2=0x7f34e00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:21:29.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 shutdown_connections 2024-09-10T21:21:29.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:21:28.854+0000 7f34e52dc640 1 -- 172.21.15.33:0/157259140 wait complete. 2024-09-10T21:21:29.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:29 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:29 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:29 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:29 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:29 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:29 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:29 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:29 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:29 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:21:29.935 INFO:tasks.cephfs.filesystem:scrub status for tag:1737cca1-c03b-4eef-8451-596ad722fe8c - {'path': '/', 'tag': '1737cca1-c03b-4eef-8451-596ad722fe8c', 'options': 'recursive,force'} 2024-09-10T21:21:30.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 219408 41.80 MB/sec execute 150 sec latency 322.909 ms 2024-09-10T21:21:30.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:30 smithi033 ceph-mon[30860]: pgmap v871: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 925 op/s 2024-09-10T21:21:30.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:30 smithi104 ceph-mon[35325]: pgmap v871: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 925 op/s 2024-09-10T21:21:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:30 smithi174 ceph-mon[35635]: pgmap v871: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 925 op/s 2024-09-10T21:21:31.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 220066 41.73 MB/sec execute 151 sec latency 74.813 ms 2024-09-10T21:21:32.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 220946 41.74 MB/sec execute 152 sec latency 82.019 ms 2024-09-10T21:21:32.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:32 smithi104 ceph-mon[35325]: pgmap v872: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 879 op/s 2024-09-10T21:21:32.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:32 smithi174 ceph-mon[35635]: pgmap v872: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 879 op/s 2024-09-10T21:21:32.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:32 smithi033 ceph-mon[30860]: pgmap v872: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 879 op/s 2024-09-10T21:21:33.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 221855 41.80 MB/sec execute 153 sec latency 87.034 ms 2024-09-10T21:21:34.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 222758 41.86 MB/sec execute 154 sec latency 56.339 ms 2024-09-10T21:21:34.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:34 smithi104 ceph-mon[35325]: pgmap v873: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-09-10T21:21:34.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:34 smithi174 ceph-mon[35635]: pgmap v873: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-09-10T21:21:34.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:34 smithi033 ceph-mon[30860]: pgmap v873: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-09-10T21:21:35.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 223236 41.75 MB/sec execute 155 sec latency 362.420 ms 2024-09-10T21:21:35.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:21:35.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.313 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.314 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:21:35.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:21:35.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:21:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:21:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:21:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:21:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:21:35.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:21:35.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:21:35.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:21:36.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 224145 41.82 MB/sec execute 156 sec latency 92.268 ms 2024-09-10T21:21:36.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:36 smithi104 ceph-mon[35325]: pgmap v874: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 940 op/s 2024-09-10T21:21:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:36 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-10T21:21:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:36 smithi104 ceph-mon[35325]: Unable to set osd_memory_target on smithi033 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-10T21:21:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:36 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T21:21:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:36 smithi104 ceph-mon[35325]: Unable to set osd_memory_target on smithi174 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-10T21:21:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:36 smithi104 ceph-mon[35325]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-10T21:21:36.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:36 smithi104 ceph-mon[35325]: Unable to set osd_memory_target on smithi104 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-10T21:21:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:36 smithi174 ceph-mon[35635]: pgmap v874: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 940 op/s 2024-09-10T21:21:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:36 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-10T21:21:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:36 smithi174 ceph-mon[35635]: Unable to set osd_memory_target on smithi033 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-10T21:21:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:36 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T21:21:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:36 smithi174 ceph-mon[35635]: Unable to set osd_memory_target on smithi174 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-10T21:21:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:36 smithi174 ceph-mon[35635]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-10T21:21:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:36 smithi174 ceph-mon[35635]: Unable to set osd_memory_target on smithi104 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-10T21:21:36.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:36 smithi033 ceph-mon[30860]: pgmap v874: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 940 op/s 2024-09-10T21:21:36.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:36 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi033 to 177.0M 2024-09-10T21:21:36.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:36 smithi033 ceph-mon[30860]: Unable to set osd_memory_target on smithi033 to 185687040: error parsing value: Value '185687040' is below minimum 939524096 2024-09-10T21:21:36.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:36 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi174 to 176.0M 2024-09-10T21:21:36.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:36 smithi033 ceph-mon[30860]: Unable to set osd_memory_target on smithi174 to 184574566: error parsing value: Value '184574566' is below minimum 939524096 2024-09-10T21:21:36.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:36 smithi033 ceph-mon[30860]: Adjusting osd_memory_target on smithi104 to 176.0M 2024-09-10T21:21:36.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:36 smithi033 ceph-mon[30860]: Unable to set osd_memory_target on smithi104 to 184586752: error parsing value: Value '184586752' is below minimum 939524096 2024-09-10T21:21:37.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 224738 41.72 MB/sec execute 157 sec latency 191.871 ms 2024-09-10T21:21:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:37 smithi104 ceph-mon[35325]: pgmap v875: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 841 op/s 2024-09-10T21:21:37.327 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:37 smithi033 ceph-mon[30860]: pgmap v875: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 841 op/s 2024-09-10T21:21:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:37 smithi174 ceph-mon[35635]: pgmap v875: 97 pgs: 97 active+clean; 1.2 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 841 op/s 2024-09-10T21:21:38.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 225744 41.80 MB/sec execute 158 sec latency 94.465 ms 2024-09-10T21:21:39.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 226722 41.88 MB/sec execute 159 sec latency 71.415 ms 2024-09-10T21:21:39.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:39 smithi104 ceph-mon[35325]: pgmap v876: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 965 op/s 2024-09-10T21:21:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:39 smithi174 ceph-mon[35635]: pgmap v876: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 965 op/s 2024-09-10T21:21:39.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:39 smithi033 ceph-mon[30860]: pgmap v876: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 13 MiB/s wr, 965 op/s 2024-09-10T21:21:40.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 227346 41.83 MB/sec execute 160 sec latency 366.037 ms 2024-09-10T21:21:41.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 228069 41.81 MB/sec execute 161 sec latency 152.343 ms 2024-09-10T21:21:41.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:41 smithi104 ceph-mon[35325]: pgmap v877: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 881 op/s 2024-09-10T21:21:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:21:41.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:41 smithi174 ceph-mon[35635]: pgmap v877: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 881 op/s 2024-09-10T21:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:21:41.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:41 smithi033 ceph-mon[30860]: pgmap v877: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 881 op/s 2024-09-10T21:21:41.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:21:41.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:21:42.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 228767 41.82 MB/sec execute 162 sec latency 257.937 ms 2024-09-10T21:21:42.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:21:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:21:42.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:21:42.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:21:42.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:21:42.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:21:42.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:21:42.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:21:42.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:21:43.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 229588 41.78 MB/sec execute 163 sec latency 94.244 ms 2024-09-10T21:21:43.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:43 smithi104 ceph-mon[35325]: pgmap v878: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 825 op/s 2024-09-10T21:21:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:43 smithi174 ceph-mon[35635]: pgmap v878: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 825 op/s 2024-09-10T21:21:43.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:43 smithi033 ceph-mon[30860]: pgmap v878: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 825 op/s 2024-09-10T21:21:44.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 230666 41.90 MB/sec execute 164 sec latency 137.739 ms 2024-09-10T21:21:45.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 231363 41.91 MB/sec execute 165 sec latency 267.533 ms 2024-09-10T21:21:45.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:45 smithi104 ceph-mon[35325]: pgmap v879: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 912 op/s 2024-09-10T21:21:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:45 smithi174 ceph-mon[35635]: pgmap v879: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 912 op/s 2024-09-10T21:21:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:45 smithi033 ceph-mon[30860]: pgmap v879: 97 pgs: 97 active+clean; 1.2 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 912 op/s 2024-09-10T21:21:46.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 231978 41.85 MB/sec execute 166 sec latency 76.712 ms 2024-09-10T21:21:47.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 232809 41.81 MB/sec execute 167 sec latency 144.437 ms 2024-09-10T21:21:47.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:47 smithi104 ceph-mon[35325]: pgmap v880: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 816 op/s 2024-09-10T21:21:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:47 smithi174 ceph-mon[35635]: pgmap v880: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 816 op/s 2024-09-10T21:21:47.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:47 smithi033 ceph-mon[30860]: pgmap v880: 97 pgs: 97 active+clean; 1.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 816 op/s 2024-09-10T21:21:48.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 233608 41.85 MB/sec execute 168 sec latency 347.267 ms 2024-09-10T21:21:49.060 INFO:tasks.workunit.client.0.smithi033.stdout: 10 234508 41.90 MB/sec execute 169 sec latency 51.747 ms 2024-09-10T21:21:49.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:49 smithi033 ceph-mon[30860]: pgmap v881: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 918 op/s 2024-09-10T21:21:49.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:49 smithi104 ceph-mon[35325]: pgmap v881: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 918 op/s 2024-09-10T21:21:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:49 smithi174 ceph-mon[35635]: pgmap v881: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 918 op/s 2024-09-10T21:21:50.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 235041 41.79 MB/sec execute 170 sec latency 265.751 ms 2024-09-10T21:21:51.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 235837 41.82 MB/sec execute 171 sec latency 123.861 ms 2024-09-10T21:21:51.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:51 smithi104 ceph-mon[35325]: pgmap v882: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 873 op/s 2024-09-10T21:21:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:51 smithi174 ceph-mon[35635]: pgmap v882: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 873 op/s 2024-09-10T21:21:51.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:51 smithi033 ceph-mon[30860]: pgmap v882: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 873 op/s 2024-09-10T21:21:52.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 236698 41.85 MB/sec execute 172 sec latency 89.856 ms 2024-09-10T21:21:53.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 237426 41.83 MB/sec execute 173 sec latency 84.143 ms 2024-09-10T21:21:53.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:53 smithi104 ceph-mon[35325]: pgmap v883: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 858 op/s 2024-09-10T21:21:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:53 smithi174 ceph-mon[35635]: pgmap v883: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 858 op/s 2024-09-10T21:21:53.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:53 smithi033 ceph-mon[30860]: pgmap v883: 97 pgs: 97 active+clean; 1.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 858 op/s 2024-09-10T21:21:54.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 238231 41.83 MB/sec execute 174 sec latency 119.178 ms 2024-09-10T21:21:55.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 238879 41.78 MB/sec execute 175 sec latency 225.819 ms 2024-09-10T21:21:55.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:55 smithi104 ceph-mon[35325]: pgmap v884: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:21:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:55 smithi174 ceph-mon[35635]: pgmap v884: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:21:55.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:55 smithi033 ceph-mon[30860]: pgmap v884: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:21:56.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 239621 41.79 MB/sec execute 176 sec latency 292.552 ms 2024-09-10T21:21:57.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 240478 41.82 MB/sec execute 177 sec latency 96.856 ms 2024-09-10T21:21:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:57 smithi104 ceph-mon[35325]: pgmap v885: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 745 op/s 2024-09-10T21:21:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:57 smithi174 ceph-mon[35635]: pgmap v885: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 745 op/s 2024-09-10T21:21:57.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:57 smithi033 ceph-mon[30860]: pgmap v885: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 745 op/s 2024-09-10T21:21:58.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 241287 41.84 MB/sec execute 178 sec latency 86.802 ms 2024-09-10T21:21:59.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 242085 41.85 MB/sec execute 179 sec latency 76.739 ms 2024-09-10T21:21:59.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:21:59 smithi104 ceph-mon[35325]: pgmap v886: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 939 op/s 2024-09-10T21:21:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:21:59 smithi174 ceph-mon[35635]: pgmap v886: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 939 op/s 2024-09-10T21:21:59.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:21:59 smithi033 ceph-mon[30860]: pgmap v886: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 939 op/s 2024-09-10T21:21:59.937 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:22:00.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 242886 41.84 MB/sec execute 180 sec latency 130.620 ms 2024-09-10T21:22:00.330 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:01.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 243417 41.75 MB/sec execute 181 sec latency 309.118 ms 2024-09-10T21:22:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 -- 172.21.15.33:0/2251965012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38104510 msgr2=0x7fea3810caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 --2- 172.21.15.33:0/2251965012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38104510 0x7fea3810caf0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fea24009900 tx=0x7fea2402a410 comp rx=0 tx=0).stop 2024-09-10T21:22:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 -- 172.21.15.33:0/2251965012 shutdown_connections 2024-09-10T21:22:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 --2- 172.21.15.33:0/2251965012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea3810d0c0 0x7fea3810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 --2- 172.21.15.33:0/2251965012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38104510 0x7fea3810caf0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 --2- 172.21.15.33:0/2251965012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea38103b10 0x7fea38103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 -- 172.21.15.33:0/2251965012 >> 172.21.15.33:0/2251965012 conn(0x7fea380fd700 msgr2=0x7fea380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 -- 172.21.15.33:0/2251965012 shutdown_connections 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.074+0000 7fea3eada640 1 -- 172.21.15.33:0/2251965012 wait complete. 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 Processor -- start 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 -- start start 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38103b10 0x7fea3819b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea38104510 0x7fea3819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3810d0c0 0x7fea381a0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea38112da0 con 0x7fea38103b10 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea38112c20 con 0x7fea38104510 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3eada640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea38112f20 con 0x7fea3810d0c0 2024-09-10T21:22:01.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3c876640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38103b10 0x7fea3819b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3d077640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3810d0c0 0x7fea381a0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3c876640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38103b10 0x7fea3819b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34434/0 (socket says 172.21.15.33:34434) 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3d077640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3810d0c0 0x7fea381a0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39092/0 (socket says 172.21.15.33:39092) 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.075+0000 7fea3c876640 1 -- 172.21.15.33:0/279434948 learned_addr learned my addr 172.21.15.33:0/279434948 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.076+0000 7fea3c876640 1 -- 172.21.15.33:0/279434948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3810d0c0 msgr2=0x7fea381a0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.076+0000 7fea3c876640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3810d0c0 0x7fea381a0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.076+0000 7fea3c876640 1 -- 172.21.15.33:0/279434948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea38104510 msgr2=0x7fea3819ba90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.076+0000 7fea3c876640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea38104510 0x7fea3819ba90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.076+0000 7fea3c876640 1 -- 172.21.15.33:0/279434948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea381a1600 con 0x7fea38103b10 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.076+0000 7fea3d077640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3810d0c0 0x7fea381a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:01.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.076+0000 7fea3c876640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38103b10 0x7fea3819b550 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e9f0 tx=0x7fea2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.077+0000 7fea35ffb640 1 -- 172.21.15.33:0/279434948 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00cd80 con 0x7fea38103b10 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.077+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea381a18f0 con 0x7fea38103b10 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.077+0000 7fea35ffb640 1 -- 172.21.15.33:0/279434948 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c0108e0 con 0x7fea38103b10 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.077+0000 7fea35ffb640 1 -- 172.21.15.33:0/279434948 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c018980 con 0x7fea38103b10 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.078+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea381a7fb0 con 0x7fea38103b10 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.083+0000 7fea40eaf740 1 Processor -- start 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.083+0000 7fea40eaf740 1 -- start start 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.084+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf77070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.084+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf83000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.084+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf85930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.084+0000 7fea40eaf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5559adee4e50 con 0x5559adf83540 2024-09-10T21:22:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.084+0000 7fea40eaf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5559aded4280 con 0x5559adf775b0 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.084+0000 7fea40eaf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5559aded8090 con 0x5559adf76c90 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.084+0000 7fea35ffb640 1 -- 172.21.15.33:0/279434948 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fea2c010430 con 0x7fea38103b10 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.085+0000 7fea3c876640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf83000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.085+0000 7fea3c876640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf83000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49448/0 (socket says 172.21.15.33:49448) 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.085+0000 7fea3c876640 1 -- 172.21.15.33:0/2991484212 learned_addr learned my addr 172.21.15.33:0/2991484212 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.085+0000 7fea3d077640 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf77070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.085+0000 7fea37fff640 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf85930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.086+0000 7fea37fff640 1 -- 172.21.15.33:0/2991484212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 msgr2=0x5559adf77070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.086+0000 7fea37fff640 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf77070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.086+0000 7fea37fff640 1 -- 172.21.15.33:0/2991484212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 msgr2=0x5559adf83000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.086+0000 7fea37fff640 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf83000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.086+0000 7fea37fff640 1 -- 172.21.15.33:0/2991484212 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559adc32650 con 0x5559adf83540 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.086+0000 7fea35ffb640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea280b5430 0x7fea280b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.087+0000 7fea37fff640 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf85930 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fea24009230 tx=0x7fea240095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff6d625ccf0a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.087+0000 7fea35ffb640 1 -- 172.21.15.33:0/279434948 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fea2c0e6040 con 0x7fea38103b10 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.087+0000 7fea22ffd640 1 -- 172.21.15.33:0/2991484212 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2403e070 con 0x5559adf83540 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.087+0000 7fea3d077640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea280b5430 0x7fea280b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.087+0000 7fea22ffd640 1 -- 172.21.15.33:0/2991484212 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24004400 con 0x5559adf83540 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.087+0000 7fea22ffd640 1 -- 172.21.15.33:0/2991484212 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24035040 con 0x5559adf83540 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 -- 172.21.15.33:0/2991484212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 msgr2=0x5559adf85930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf85930 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fea24009230 tx=0x7fea240095b0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea3d077640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea280b5430 0x7fea280b78f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea30005fd0 tx=0x7fea30009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 -- 172.21.15.33:0/2991484212 shutdown_connections 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf85930 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf83000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2991484212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf77070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 -- 172.21.15.33:0/2991484212 >> 172.21.15.33:0/2991484212 conn(0x5559adf74ac0 msgr2=0x5559adf75c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 -- 172.21.15.33:0/2991484212 shutdown_connections 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.088+0000 7fea40eaf740 1 -- 172.21.15.33:0/2991484212 wait complete. 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.089+0000 7fea40eaf740 1 -- start start 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.089+0000 7fea40eaf740 1 Processor -- start 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf82dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf775b0 0x5559adf80220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf83540 0x5559adf807c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf775b0 0x5559adf80220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea40eaf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5559add35700 con 0x5559adf775b0 2024-09-10T21:22:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea3c876640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf82dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf775b0 0x5559adf80220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34450/0 (socket says 172.21.15.33:34450) 2024-09-10T21:22:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea37fff640 1 -- 172.21.15.33:0/2729753605 learned_addr learned my addr 172.21.15.33:0/2729753605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.090+0000 7fea3d077640 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf83540 0x5559adf807c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5559ade6f110 con 0x5559adf83540 2024-09-10T21:22:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5559adf86160 con 0x5559adf76c90 2024-09-10T21:22:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea40eaf740 10 client.0 _ll_register_callbacks cb 0x5559ade88990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea3c876640 1 -- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf83540 msgr2=0x5559adf807c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea3c876640 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf83540 0x5559adf807c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea3c876640 1 -- 172.21.15.33:0/2729753605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf775b0 msgr2=0x5559adf80220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea3c876640 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf775b0 0x5559adf80220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea3c876640 1 -- 172.21.15.33:0/2729753605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ade9cc40 con 0x5559adf76c90 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea3c876640 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf82dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0ac750 tx=0x7fea2c0a8660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cace84964d8e75e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.091+0000 7fea17fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.092+0000 7fea17fff640 1 -- 172.21.15.33:0/2729753605 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0abf30 con 0x5559adf76c90 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.092+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea24004400 con 0x5559adf76c90 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.092+0000 7fea17fff640 1 -- 172.21.15.33:0/2729753605 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fea2c0aba20 con 0x5559adf76c90 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.092+0000 7fea17fff640 1 -- 172.21.15.33:0/2729753605 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fea2c0ac2d0 con 0x5559adf76c90 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.092+0000 7fea40eaf740 10 client.27916 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:01.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.092+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5559adf91af0 con 0x5559adf76c90 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea17fff640 1 -- 172.21.15.33:0/2729753605 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24004400 con 0x5559adf76c90 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea17fff640 1 -- 172.21.15.33:0/2729753605 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fea2c0adaf0 con 0x5559adf76c90 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea40eaf740 10 client.27916 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea40eaf740 10 client.27916 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea40eaf740 20 client.27916 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559adf953a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea40eaf740 4 client.27916 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.093+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559adee6da0 con 0x5559adf92f50 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.094+0000 7fea37fff640 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559adf953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.094+0000 7fea37fff640 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559adf953a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=251a4301a9e1cc4a server_cookie=7c7c0a98f5492329 in_seq=0 out_seq=0 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.094+0000 7fea17fff640 10 client.27916 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:01.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.095+0000 7fea17fff640 1 -- 172.21.15.33:0/2729753605 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5559adee6da0 con 0x5559adf92f50 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.095+0000 7fea17fff640 10 client.27916 handle_command_reply: tid=0 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.095+0000 7fea40eaf740 1 client.27916 shutdown 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.095+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 msgr2=0x5559adf82dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.095+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf82dd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0ac750 tx=0x7fea2c0a8660 comp rx=0 tx=0).stop 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.096+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 shutdown_connections 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.096+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559adf953a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.096+0000 7fea17fff640 0 client.27916 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.096+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf83540 0x5559adf807c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.096+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf775b0 0x5559adf80220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.096+0000 7fea40eaf740 1 --2- 172.21.15.33:0/2729753605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf82dd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.096+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 >> 172.21.15.33:0/2729753605 conn(0x5559adf74ac0 msgr2=0x5559adf6b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.097+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 shutdown_connections 2024-09-10T21:22:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.097+0000 7fea40eaf740 1 -- 172.21.15.33:0/2729753605 wait complete. 2024-09-10T21:22:01.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.097+0000 7fea40eaf740 20 client.27916 trim_cache size 0 max 16384 2024-09-10T21:22:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 Processor -- start 2024-09-10T21:22:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 -- start start 2024-09-10T21:22:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf76c90 0x5559adf7d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf775b0 0x5559adf7d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf9ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea381ac230 con 0x5559adf83540 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea381add70 con 0x5559adf76c90 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.127+0000 7fea40eaf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea381ac900 con 0x5559adf775b0 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3d077640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf9ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf76c90 0x5559adf7d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf76c90 0x5559adf7d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49454/0 (socket says 172.21.15.33:49454) 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 -- 172.21.15.33:0/1948669444 learned_addr learned my addr 172.21.15.33:0/1948669444 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea37fff640 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf775b0 0x5559adf7d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 -- 172.21.15.33:0/1948669444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf775b0 msgr2=0x5559adf7d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf775b0 0x5559adf7d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 -- 172.21.15.33:0/1948669444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 msgr2=0x5559adf9ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf9ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea3c876640 1 -- 172.21.15.33:0/1948669444 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559adf67b60 con 0x5559adf76c90 2024-09-10T21:22:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.128+0000 7fea37fff640 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf775b0 0x5559adf7d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:22:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.129+0000 7fea3c876640 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf76c90 0x5559adf7d430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e9c0 tx=0x7fea2c0a8790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4da06ab3883119 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.129+0000 7fea227fc640 1 -- 172.21.15.33:0/1948669444 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0ac590 con 0x5559adf76c90 2024-09-10T21:22:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.129+0000 7fea227fc640 1 -- 172.21.15.33:0/1948669444 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c0acec0 con 0x5559adf76c90 2024-09-10T21:22:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.129+0000 7fea227fc640 1 -- 172.21.15.33:0/1948669444 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c018e50 con 0x5559adf76c90 2024-09-10T21:22:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 -- 172.21.15.33:0/1948669444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf76c90 msgr2=0x5559adf7d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf76c90 0x5559adf7d430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e9c0 tx=0x7fea2c0a8790 comp rx=0 tx=0).stop 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 -- 172.21.15.33:0/1948669444 shutdown_connections 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf9ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf775b0 0x5559adf7d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1948669444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf76c90 0x5559adf7d430 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 -- 172.21.15.33:0/1948669444 >> 172.21.15.33:0/1948669444 conn(0x5559adf74ac0 msgr2=0x5559adf75b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 -- 172.21.15.33:0/1948669444 shutdown_connections 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.130+0000 7fea40eaf740 1 -- 172.21.15.33:0/1948669444 wait complete. 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.131+0000 7fea40eaf740 1 -- start start 2024-09-10T21:22:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.131+0000 7fea40eaf740 1 Processor -- start 2024-09-10T21:22:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.131+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.131+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf87a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.131+0000 7fea40eaf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf87f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea40eaf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fea381ad6a0 con 0x5559adf83540 2024-09-10T21:22:01.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea40eaf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fea381acfd0 con 0x5559adf775b0 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea40eaf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea381aa250 con 0x5559adf76c90 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea40eaf740 10 client.0 _ll_register_callbacks cb 0x7fea2c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf87f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf87f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34472/0 (socket says 172.21.15.33:34472) 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 -- 172.21.15.33:0/1757544207 learned_addr learned my addr 172.21.15.33:0/1757544207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea37fff640 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf87a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 -- 172.21.15.33:0/1757544207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 msgr2=0x5559adf874c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 -- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 msgr2=0x5559adf87a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf87a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 -- 172.21.15.33:0/1757544207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559adffeff0 con 0x5559adf83540 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea3d077640 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf87f40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fea30003e70 tx=0x7fea30004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df926c2a0db0799b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.132+0000 7fea17fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.133+0000 7fea17fff640 1 -- 172.21.15.33:0/1757544207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3002f060 con 0x5559adf83540 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.133+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2c0acec0 con 0x5559adf83540 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.133+0000 7fea17fff640 1 -- 172.21.15.33:0/1757544207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fea300045c0 con 0x5559adf83540 2024-09-10T21:22:01.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.133+0000 7fea17fff640 1 -- 172.21.15.33:0/1757544207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fea300047c0 con 0x5559adf83540 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.133+0000 7fea40eaf740 10 client.18645 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.133+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5559ae0002d0 con 0x5559adf83540 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea17fff640 1 -- 172.21.15.33:0/1757544207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c0acec0 con 0x5559adf83540 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea17fff640 1 -- 172.21.15.33:0/1757544207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fea30034070 con 0x5559adf83540 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea40eaf740 10 client.18645 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea40eaf740 10 client.18645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea40eaf740 20 client.18645 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559ae002af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea40eaf740 4 client.18645 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:22:01.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559adee2850 con 0x5559adf92f50 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.134+0000 7fea3c876640 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559ae002af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea3c876640 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559ae002af0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7118e8016fed9130 server_cookie=df7da12ee3cf1a52 in_seq=0 out_seq=0 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea17fff640 10 client.18645 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea17fff640 1 -- 172.21.15.33:0/1757544207 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6622 (crc 0 0 0) 0x5559adee2850 con 0x5559adf92f50 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea17fff640 10 client.18645 handle_command_reply: tid=0 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 client.18645 shutdown 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 msgr2=0x5559adf87f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf87f40 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fea30003e70 tx=0x7fea30004a40 comp rx=0 tx=0).stop 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 shutdown_connections 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5559adf92f50 0x5559ae002af0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5559adf83540 0x5559adf87f40 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5559adf775b0 0x5559adf87a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 --2- 172.21.15.33:0/1757544207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5559adf76c90 0x5559adf874c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 >> 172.21.15.33:0/1757544207 conn(0x5559adf74ac0 msgr2=0x5559adf6abb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.135+0000 7fea17fff640 0 client.18645 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.136+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 shutdown_connections 2024-09-10T21:22:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.136+0000 7fea40eaf740 1 -- 172.21.15.33:0/1757544207 wait complete. 2024-09-10T21:22:01.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.136+0000 7fea40eaf740 20 client.18645 trim_cache size 0 max 16384 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.147 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.148 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.149 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.150 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.151 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.152 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:22:01.153 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.155 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:22:01.156 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.157 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:22:01.158 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.160 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:22:01.161 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:22:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea280b5430 msgr2=0x7fea280b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea280b5430 0x7fea280b78f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea30005fd0 tx=0x7fea30009500 comp rx=0 tx=0).stop 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38103b10 msgr2=0x7fea3819b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38103b10 0x7fea3819b550 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00e9f0 tx=0x7fea2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 shutdown_connections 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fea280b5430 0x7fea280b78f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea3810d0c0 0x7fea381a0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fea38104510 0x7fea3819ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 --2- 172.21.15.33:0/279434948 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fea38103b10 0x7fea3819b550 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 >> 172.21.15.33:0/279434948 conn(0x7fea380fd700 msgr2=0x7fea3810f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 shutdown_connections 2024-09-10T21:22:01.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:01.138+0000 7fea3eada640 1 -- 172.21.15.33:0/279434948 wait complete. 2024-09-10T21:22:01.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:01 smithi104 ceph-mon[35325]: pgmap v887: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 846 op/s 2024-09-10T21:22:01.830 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:01 smithi033 ceph-mon[30860]: pgmap v887: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 846 op/s 2024-09-10T21:22:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:01 smithi174 ceph-mon[35635]: pgmap v887: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 846 op/s 2024-09-10T21:22:02.061 INFO:tasks.workunit.client.0.smithi033.stdout: 10 244230 41.73 MB/sec execute 182 sec latency 95.196 ms 2024-09-10T21:22:02.072 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:22:02.072 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:22:02.421 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:03.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 245248 41.87 MB/sec execute 183 sec latency 54.861 ms 2024-09-10T21:22:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.060+0000 7fd6ec985640 1 -- 172.21.15.33:0/1783160487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8103990 msgr2=0x7fd6e8103d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.061+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1783160487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8103990 0x7fd6e8103d70 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009900 tx=0x7fd6d802a480 comp rx=0 tx=0).stop 2024-09-10T21:22:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.062+0000 7fd6ec985640 1 -- 172.21.15.33:0/1783160487 shutdown_connections 2024-09-10T21:22:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.062+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1783160487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8101bf0 0x7fd6e81117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.062+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1783160487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8101030 0x7fd6e8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.062+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1783160487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8103990 0x7fd6e8103d70 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.063+0000 7fd6ec985640 1 -- 172.21.15.33:0/1783160487 >> 172.21.15.33:0/1783160487 conn(0x7fd6e8078140 msgr2=0x7fd6e80ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.063+0000 7fd6ec985640 1 -- 172.21.15.33:0/1783160487 shutdown_connections 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.063+0000 7fd6ec985640 1 -- 172.21.15.33:0/1783160487 wait complete. 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.065+0000 7fd6ec985640 1 Processor -- start 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.065+0000 7fd6ec985640 1 -- start start 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.065+0000 7fd6ec985640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8101030 0x7fd6e819b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6ec985640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8101bf0 0x7fd6e819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6ec985640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8103990 0x7fd6e81a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6ec985640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8115070 con 0x7fd6e8101030 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6ec985640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e8114ef0 con 0x7fd6e8103990 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6ec985640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e81151f0 con 0x7fd6e8101bf0 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6e6d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8103990 0x7fd6e81a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6e6d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8103990 0x7fd6e81a0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49476/0 (socket says 172.21.15.33:49476) 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6e6d9d640 1 -- 172.21.15.33:0/1982613979 learned_addr learned my addr 172.21.15.33:0/1982613979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.066+0000 7fd6e659c640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8101030 0x7fd6e819b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.067+0000 7fd6e6d9d640 1 -- 172.21.15.33:0/1982613979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8101bf0 msgr2=0x7fd6e819bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.067+0000 7fd6e6d9d640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8101bf0 0x7fd6e819bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.067+0000 7fd6e6d9d640 1 -- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8101030 msgr2=0x7fd6e819b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.067+0000 7fd6e6d9d640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8101030 0x7fd6e819b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.067+0000 7fd6e6d9d640 1 -- 172.21.15.33:0/1982613979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e81a1630 con 0x7fd6e8103990 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.068+0000 7fd6e6d9d640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8103990 0x7fd6e81a0e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000e8f0 tx=0x7fd6e000edb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.068+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1982613979 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0004d20 con 0x7fd6e8103990 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.068+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1982613979 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6e0005020 con 0x7fd6e8103990 2024-09-10T21:22:03.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.068+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e81a1920 con 0x7fd6e8103990 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.068+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e81a8060 con 0x7fd6e8103990 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.068+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1982613979 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00116b0 con 0x7fd6e8103990 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.070+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e8101470 con 0x7fd6e8103990 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.074+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1982613979 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd6e000f040 con 0x7fd6e8103990 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.075+0000 7fd6d77fe640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6d00b5420 0x7fd6d00b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.075+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1982613979 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd6e00e5e30 con 0x7fd6e8103990 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.075+0000 7fd6e659c640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6d00b5420 0x7fd6d00b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.075+0000 7fd6e659c640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6d00b5420 0x7fd6d00b78e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000c00 tx=0x7fd6d8031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:03.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.078+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1982613979 <== mon.2 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fd6e0016030 con 0x7fd6e8103990 2024-09-10T21:22:03.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.399+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6e81a8420 con 0x7fd6e8103990 2024-09-10T21:22:03.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.400+0000 7fd6d77fe640 1 -- 172.21.15.33:0/1982613979 <== mon.2 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+11973 (secure 0 0 0) 0x7fd6e00c5ee0 con 0x7fd6e8103990 2024-09-10T21:22:03.405 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:22:03.405 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2024-09-10T21:22:03.405 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"btime":"2024-09-10T21:21:17:963637+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":81,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:21:17.000563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:22:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.404+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6d00b5420 msgr2=0x7fd6d00b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:03.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.404+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6d00b5420 0x7fd6d00b78e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8000c00 tx=0x7fd6d8031400 comp rx=0 tx=0).stop 2024-09-10T21:22:03.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.404+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8103990 msgr2=0x7fd6e81a0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:03.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.404+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8103990 0x7fd6e81a0e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000e8f0 tx=0x7fd6e000edb0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.406+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 shutdown_connections 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.406+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd6d00b5420 0x7fd6d00b78e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.406+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6e8103990 0x7fd6e81a0e60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.406+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6e8101bf0 0x7fd6e819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.406+0000 7fd6ec985640 1 --2- 172.21.15.33:0/1982613979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6e8101030 0x7fd6e819b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.406+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 >> 172.21.15.33:0/1982613979 conn(0x7fd6e8078140 msgr2=0x7fd6e80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.407+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 shutdown_connections 2024-09-10T21:22:03.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:03.407+0000 7fd6ec985640 1 -- 172.21.15.33:0/1982613979 wait complete. 2024-09-10T21:22:03.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:03 smithi033 ceph-mon[30860]: pgmap v888: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 772 op/s 2024-09-10T21:22:03.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:03 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1982613979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:22:03.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:03 smithi104 ceph-mon[35325]: pgmap v888: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 772 op/s 2024-09-10T21:22:03.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:03 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1982613979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:22:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:03 smithi174 ceph-mon[35635]: pgmap v888: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.9 MiB/s wr, 772 op/s 2024-09-10T21:22:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:03 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1982613979' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:22:04.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 245941 41.82 MB/sec execute 184 sec latency 89.241 ms 2024-09-10T21:22:04.760 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:22:05.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 246610 41.79 MB/sec execute 185 sec latency 328.875 ms 2024-09-10T21:22:05.191 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:05.684 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:05 smithi033 ceph-mon[30860]: pgmap v889: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 840 op/s 2024-09-10T21:22:05.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:05 smithi104 ceph-mon[35325]: pgmap v889: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 840 op/s 2024-09-10T21:22:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:05 smithi174 ceph-mon[35635]: pgmap v889: 97 pgs: 97 active+clean; 1.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 840 op/s 2024-09-10T21:22:05.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.928+0000 7f57d317e640 1 -- 172.21.15.33:0/2567769401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10b260 msgr2=0x7f57cc10b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.928+0000 7f57d317e640 1 --2- 172.21.15.33:0/2567769401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10b260 0x7f57cc10b640 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009ca0 tx=0x7f57bc02a950 comp rx=0 tx=0).stop 2024-09-10T21:22:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.928+0000 7f57d317e640 1 -- 172.21.15.33:0/2567769401 shutdown_connections 2024-09-10T21:22:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.928+0000 7f57d317e640 1 --2- 172.21.15.33:0/2567769401 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57cc114810 0x7f57cc116c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.928+0000 7f57d317e640 1 --2- 172.21.15.33:0/2567769401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10bb80 0x7f57cc1142d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.928+0000 7f57d317e640 1 --2- 172.21.15.33:0/2567769401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10b260 0x7f57cc10b640 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.928+0000 7f57d317e640 1 -- 172.21.15.33:0/2567769401 >> 172.21.15.33:0/2567769401 conn(0x7f57cc06dfc0 msgr2=0x7f57cc06e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.929+0000 7f57d317e640 1 -- 172.21.15.33:0/2567769401 shutdown_connections 2024-09-10T21:22:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.929+0000 7f57d317e640 1 -- 172.21.15.33:0/2567769401 wait complete. 2024-09-10T21:22:05.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.930+0000 7f57d317e640 1 Processor -- start 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.931+0000 7f57d317e640 1 -- start start 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.931+0000 7f57d317e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10b260 0x7f57cc1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.931+0000 7f57d317e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10bb80 0x7f57cc1a0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57d317e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57cc114810 0x7f57cc1a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57d317e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc11a4b0 con 0x7f57cc10bb80 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10bb80 0x7f57cc1a0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10bb80 0x7f57cc1a0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34500/0 (socket says 172.21.15.33:34500) 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 -- 172.21.15.33:0/2263322848 learned_addr learned my addr 172.21.15.33:0/2263322848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10b260 0x7f57cc1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57d317e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc11a330 con 0x7f57cc114810 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57d171b640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57cc114810 0x7f57cc1a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc11a630 con 0x7f57cc10b260 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 -- 172.21.15.33:0/2263322848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10b260 msgr2=0x7f57cc1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10b260 0x7f57cc1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 -- 172.21.15.33:0/2263322848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57cc114810 msgr2=0x7f57cc1a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57cc114810 0x7f57cc1a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.932+0000 7f57cbfff640 1 -- 172.21.15.33:0/2263322848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc1a6a30 con 0x7f57cc10bb80 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.933+0000 7f57cbfff640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10bb80 0x7f57cc1a0ec0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f57c000ea60 tx=0x7f57c000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.933+0000 7f57c9ffb640 1 -- 172.21.15.33:0/2263322848 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c000cde0 con 0x7f57cc10bb80 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.933+0000 7f57c9ffb640 1 -- 172.21.15.33:0/2263322848 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c00035f0 con 0x7f57cc10bb80 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.933+0000 7f57c9ffb640 1 -- 172.21.15.33:0/2263322848 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c0010690 con 0x7f57cc10bb80 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.933+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc1a6d20 con 0x7f57cc10bb80 2024-09-10T21:22:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.934+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc1ad3e0 con 0x7f57cc10bb80 2024-09-10T21:22:05.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.937+0000 7f57d5553740 1 Processor -- start 2024-09-10T21:22:05.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.937+0000 7f57d5553740 1 -- start start 2024-09-10T21:22:05.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d5553740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363984d0 0x5592363988b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d5553740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559236398df0 0x5592363a4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d5553740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d5553740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55923630eea0 con 0x5592363a4d80 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d5553740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5592362fe2d0 con 0x5592363984d0 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d5553740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592363020e0 con 0x559236398df0 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d0f1a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559236398df0 0x5592363a4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57c9ffb640 1 -- 172.21.15.33:0/2263322848 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f57c0004000 con 0x7f57cc10bb80 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d0f1a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559236398df0 0x5592363a4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39170/0 (socket says 172.21.15.33:39170) 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1837927542 learned_addr learned my addr 172.21.15.33:0/1837927542 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57d171b640 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363984d0 0x5592363988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.938+0000 7f57cbfff640 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1837927542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363984d0 msgr2=0x5592363988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363984d0 0x5592363988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1837927542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 msgr2=0x5592363a7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1837927542 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55923605c650 con 0x559236398df0 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559236398df0 0x5592363a4840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009ca0 tx=0x7f57bc009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f93177c658ae81f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57b2ffd640 1 -- 172.21.15.33:0/1837927542 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc049070 con 0x559236398df0 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57b2ffd640 1 -- 172.21.15.33:0/1837927542 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57bc003e20 con 0x559236398df0 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.939+0000 7f57b2ffd640 1 -- 172.21.15.33:0/1837927542 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc034070 con 0x559236398df0 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57c9ffb640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57b40b5430 0x7f57b40b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d171b640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57b40b5430 0x7f57b40b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57c9ffb640 1 -- 172.21.15.33:0/2263322848 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f57c00e5dd0 con 0x7f57cc10bb80 2024-09-10T21:22:05.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 -- 172.21.15.33:0/1837927542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559236398df0 msgr2=0x5592363a4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559236398df0 0x5592363a4840 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009ca0 tx=0x7f57bc009800 comp rx=0 tx=0).stop 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 -- 172.21.15.33:0/1837927542 shutdown_connections 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559236398df0 0x5592363a4840 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 --2- 172.21.15.33:0/1837927542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363984d0 0x5592363988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 -- 172.21.15.33:0/1837927542 >> 172.21.15.33:0/1837927542 conn(0x559236396300 msgr2=0x5592363974a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.940+0000 7f57d5553740 1 -- 172.21.15.33:0/1837927542 shutdown_connections 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.941+0000 7f57d171b640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57b40b5430 0x7f57b40b78f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f57cc10c240 tx=0x7f57c4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.941+0000 7f57d5553740 1 -- 172.21.15.33:0/1837927542 wait complete. 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.941+0000 7f57d5553740 1 -- start start 2024-09-10T21:22:05.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.941+0000 7f57d5553740 1 Processor -- start 2024-09-10T21:22:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d5553740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363a4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d5553740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559236398df0 0x5592363a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d5553740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363a4d80 0x5592363a2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d5553740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55923615f700 con 0x559236398df0 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d5553740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559236299160 con 0x5592363a4d80 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d5553740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592363a79a0 con 0x5592363984d0 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559236398df0 0x5592363a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559236398df0 0x5592363a1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34532/0 (socket says 172.21.15.33:34532) 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 -- 172.21.15.33:0/2169825070 learned_addr learned my addr 172.21.15.33:0/2169825070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57d5553740 10 client.0 _ll_register_callbacks cb 0x5592362b29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 -- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 msgr2=0x5592363a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363a4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 -- 172.21.15.33:0/2169825070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363a4d80 msgr2=0x5592363a2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363a4d80 0x5592363a2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.942+0000 7f57cbfff640 1 -- 172.21.15.33:0/2169825070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592362c6c90 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.943+0000 7f57cbfff640 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559236398df0 0x5592363a1ae0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009230 tx=0x7f57c0003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7bd0b6cb9aba11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.943+0000 7f57abfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.943+0000 7f57abfff640 1 -- 172.21.15.33:0/2169825070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00abea0 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.943+0000 7f57abfff640 1 -- 172.21.15.33:0/2169825070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f57c00ab7b0 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.943+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc003e20 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.943+0000 7f57abfff640 1 -- 172.21.15.33:0/2169825070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f57c00abcc0 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.944+0000 7f57d5553740 10 client.18663 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.944+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5592363b1bb0 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.944+0000 7f57abfff640 1 -- 172.21.15.33:0/2169825070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57c00b5420 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.944+0000 7f57abfff640 1 -- 172.21.15.33:0/2169825070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f57c0093ce0 con 0x559236398df0 2024-09-10T21:22:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.944+0000 7f57d5553740 10 client.18663 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:05.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.944+0000 7f57d5553740 10 client.18663 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:22:05.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.945+0000 7f57d5553740 20 client.18663 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:05.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.945+0000 7f57d5553740 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5592363b3010 0x5592363b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.945+0000 7f57d5553740 4 client.18663 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:05.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.945+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f57bc003e20 con 0x5592363b3010 2024-09-10T21:22:05.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.945+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5592363b3010 0x5592363b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.946+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5592363b3010 0x5592363b3490 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c3cf1a641462263f server_cookie=76baccac68bd3165 in_seq=0 out_seq=0 2024-09-10T21:22:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.946+0000 7f57abfff640 10 client.18663 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.947+0000 7f57abfff640 1 -- 172.21.15.33:0/2169825070 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f57bc003e20 con 0x5592363b3010 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.947+0000 7f57abfff640 10 client.18663 handle_command_reply: tid=0 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.947+0000 7f57d5553740 1 client.18663 shutdown 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.947+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559236398df0 msgr2=0x5592363a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.947+0000 7f57d5553740 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559236398df0 0x5592363a1ae0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f57c0009230 tx=0x7f57c0003cf0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 shutdown_connections 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5592363b3010 0x5592363b3490 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5592363a4d80 0x5592363a2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57abfff640 0 client.18663 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559236398df0 0x5592363a1ae0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 --2- 172.21.15.33:0/2169825070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 >> 172.21.15.33:0/2169825070 conn(0x559236396300 msgr2=0x55923639be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 shutdown_connections 2024-09-10T21:22:05.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 1 -- 172.21.15.33:0/2169825070 wait complete. 2024-09-10T21:22:05.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.948+0000 7f57d5553740 20 client.18663 trim_cache size 0 max 16384 2024-09-10T21:22:05.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.964+0000 7f57d5553740 1 Processor -- start 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.964+0000 7f57d5553740 1 -- start start 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d5553740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d5553740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x559236395310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d5553740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d5553740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b4aa0 con 0x5592363a4d80 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d5553740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b65e0 con 0x559236398df0 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d5553740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b5170 con 0x5592363984d0 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d0f1a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d0f1a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39186/0 (socket says 172.21.15.33:39186) 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1744942095 learned_addr learned my addr 172.21.15.33:0/1744942095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57d171b640 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.965+0000 7f57cbfff640 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x559236395310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1744942095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 msgr2=0x559236395310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x559236395310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1744942095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 msgr2=0x5592363a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57d0f1a640 1 -- 172.21.15.33:0/1744942095 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592363a2480 con 0x5592363984d0 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363b0bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57bc004ab0 tx=0x7f57bc0095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba446ab65a650882 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57b27fc640 1 -- 172.21.15.33:0/1744942095 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc034040 con 0x5592363984d0 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57b27fc640 1 -- 172.21.15.33:0/1744942095 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57bc02fc10 con 0x5592363984d0 2024-09-10T21:22:05.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.966+0000 7f57b27fc640 1 -- 172.21.15.33:0/1744942095 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc049040 con 0x5592363984d0 2024-09-10T21:22:05.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.967+0000 7f57d5553740 1 -- 172.21.15.33:0/1744942095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 msgr2=0x5592363b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.967+0000 7f57d5553740 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363b0bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f57bc004ab0 tx=0x7f57bc0095c0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.967+0000 7f57d5553740 1 -- 172.21.15.33:0/1744942095 shutdown_connections 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.967+0000 7f57d5553740 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363a4d80 0x5592363a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.967+0000 7f57d5553740 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x559236395310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.967+0000 7f57d5553740 1 --2- 172.21.15.33:0/1744942095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363984d0 0x5592363b0bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.967+0000 7f57d5553740 1 -- 172.21.15.33:0/1744942095 >> 172.21.15.33:0/1744942095 conn(0x559236396300 msgr2=0x559236394c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.968+0000 7f57d5553740 1 -- 172.21.15.33:0/1744942095 shutdown_connections 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.968+0000 7f57d5553740 1 -- 172.21.15.33:0/1744942095 wait complete. 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.968+0000 7f57d5553740 1 -- start start 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.968+0000 7f57d5553740 1 Processor -- start 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d5553740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363984d0 0x5592363ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d5553740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x5592363aac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d5553740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363a4d80 0x5592363ab180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d0f1a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363984d0 0x5592363ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d5553740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b5f10 con 0x5592363984d0 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d5553740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b5840 con 0x559236398df0 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d5553740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc1b2a60 con 0x5592363a4d80 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d5553740 10 client.0 _ll_register_callbacks cb 0x7f57c00e9500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d0f1a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363984d0 0x5592363ad1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34552/0 (socket says 172.21.15.33:34552) 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57cbfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x5592363aac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d0f1a640 1 -- 172.21.15.33:0/712574738 learned_addr learned my addr 172.21.15.33:0/712574738 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57d171b640 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363a4d80 0x5592363ab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57cbfff640 1 -- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363a4d80 msgr2=0x5592363ab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57cbfff640 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363a4d80 0x5592363ab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57cbfff640 1 -- 172.21.15.33:0/712574738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363984d0 msgr2=0x5592363ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57cbfff640 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363984d0 0x5592363ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.969+0000 7f57cbfff640 1 -- 172.21.15.33:0/712574738 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592363ab740 con 0x559236398df0 2024-09-10T21:22:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57cbfff640 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x5592363aac40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x559236396ea0 tx=0x7f57c00a89f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf10bdc4cbf57b1f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57abfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57abfff640 1 -- 172.21.15.33:0/712574738 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c00abea0 con 0x559236398df0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc02fc10 con 0x559236398df0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57abfff640 1 -- 172.21.15.33:0/712574738 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f57c00d33b0 con 0x559236398df0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57abfff640 1 -- 172.21.15.33:0/712574738 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f57c00a83f0 con 0x559236398df0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57d5553740 10 client.28055 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x559236426860 con 0x559236398df0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.970+0000 7f57abfff640 1 -- 172.21.15.33:0/712574738 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57bc02fc10 con 0x559236398df0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57abfff640 1 -- 172.21.15.33:0/712574738 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f57c00ab220 con 0x559236398df0 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57d5553740 10 client.28055 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57d5553740 10 client.28055 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57d5553740 20 client.28055 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57d5553740 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x559236426e40 0x559236429230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:05.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57d5553740 4 client.28055 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:22:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559236310df0 con 0x559236426e40 2024-09-10T21:22:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.971+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x559236426e40 0x559236429230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.974+0000 7f57d0f1a640 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x559236426e40 0x559236429230 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=977e239cd6b20952 server_cookie=f0ac8d385e2884d7 in_seq=0 out_seq=0 2024-09-10T21:22:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.974+0000 7f57abfff640 10 client.28055 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:05.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.975+0000 7f57abfff640 1 -- 172.21.15.33:0/712574738 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559236310df0 con 0x559236426e40 2024-09-10T21:22:05.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.975+0000 7f57abfff640 10 client.28055 handle_command_reply: tid=0 2024-09-10T21:22:05.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.976+0000 7f57d5553740 1 client.28055 shutdown 2024-09-10T21:22:05.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.976+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 msgr2=0x5592363aac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.976+0000 7f57d5553740 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x5592363aac40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x559236396ea0 tx=0x7f57c00a89f0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 shutdown_connections 2024-09-10T21:22:05.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x559236426e40 0x559236429230 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57abfff640 0 client.28055 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592363a4d80 0x5592363ab180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559236398df0 0x5592363aac40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 --2- 172.21.15.33:0/712574738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5592363984d0 0x5592363ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 >> 172.21.15.33:0/712574738 conn(0x559236396300 msgr2=0x5592363a3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 shutdown_connections 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 1 -- 172.21.15.33:0/712574738 wait complete. 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.977+0000 7f57d5553740 20 client.28055 trim_cache size 0 max 16384 2024-09-10T21:22:05.982 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.978+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57b40b5430 msgr2=0x7f57b40b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.978+0000 7f57d317e640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57b40b5430 0x7f57b40b78f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f57cc10c240 tx=0x7f57c4005f90 comp rx=0 tx=0).stop 2024-09-10T21:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.978+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10bb80 msgr2=0x7f57cc1a0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.978+0000 7f57d317e640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10bb80 0x7f57cc1a0ec0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f57c000ea60 tx=0x7f57c000ef20 comp rx=0 tx=0).stop 2024-09-10T21:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 shutdown_connections 2024-09-10T21:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f57b40b5430 0x7f57b40b78f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f57cc114810 0x7f57cc1a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57cc10bb80 0x7f57cc1a0ec0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 --2- 172.21.15.33:0/2263322848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57cc10b260 0x7f57cc1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:05.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 >> 172.21.15.33:0/2263322848 conn(0x7f57cc06dfc0 msgr2=0x7f57cc072900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:05.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 shutdown_connections 2024-09-10T21:22:05.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:05.979+0000 7f57d317e640 1 -- 172.21.15.33:0/2263322848 wait complete. 2024-09-10T21:22:06.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 247136 41.74 MB/sec execute 186 sec latency 300.971 ms 2024-09-10T21:22:07.005 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:22:07.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 247812 41.68 MB/sec execute 187 sec latency 221.494 ms 2024-09-10T21:22:07.311 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:07.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:07 smithi104 ceph-mon[35325]: pgmap v890: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 831 op/s 2024-09-10T21:22:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:07 smithi174 ceph-mon[35635]: pgmap v890: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 831 op/s 2024-09-10T21:22:07.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:07 smithi033 ceph-mon[30860]: pgmap v890: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.8 MiB/s wr, 831 op/s 2024-09-10T21:22:08.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 248591 41.65 MB/sec execute 188 sec latency 113.299 ms 2024-09-10T21:22:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 -- 172.21.15.33:0/3493580133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140104600 msgr2=0x7f414010caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 --2- 172.21.15.33:0/3493580133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140104600 0x7f414010caf0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4134009900 tx=0x7f413402a410 comp rx=0 tx=0).stop 2024-09-10T21:22:08.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 -- 172.21.15.33:0/3493580133 shutdown_connections 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 --2- 172.21.15.33:0/3493580133 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f414010d140 0x7f414010f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 --2- 172.21.15.33:0/3493580133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4140104600 0x7f414010caf0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 --2- 172.21.15.33:0/3493580133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4140103ce0 0x7f41401040c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 -- 172.21.15.33:0/3493580133 >> 172.21.15.33:0/3493580133 conn(0x7f41400fd740 msgr2=0x7f41400ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 -- 172.21.15.33:0/3493580133 shutdown_connections 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.186+0000 7f414480a640 1 -- 172.21.15.33:0/3493580133 wait complete. 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.187+0000 7f414480a640 1 Processor -- start 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.187+0000 7f414480a640 1 -- start start 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f414480a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4140103ce0 0x7f4140199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f414480a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4140104600 0x7f41401998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f414480a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414010d140 0x7f414019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f414480a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4140112db0 con 0x7f414010d140 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f414480a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4140112c30 con 0x7f4140104600 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f414480a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4140112f30 con 0x7f4140103ce0 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4140104600 0x7f41401998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4140103ce0 0x7f4140199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4140104600 0x7f41401998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49546/0 (socket says 172.21.15.33:49546) 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 -- 172.21.15.33:0/2023720243 learned_addr learned my addr 172.21.15.33:0/2023720243 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413ed9d640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414010d140 0x7f414019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 -- 172.21.15.33:0/2023720243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4140103ce0 msgr2=0x7f4140199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4140103ce0 0x7f4140199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 -- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414010d140 msgr2=0x7f414019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414010d140 0x7f414019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.188+0000 7f413dd9b640 1 -- 172.21.15.33:0/2023720243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414019f410 con 0x7f4140104600 2024-09-10T21:22:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.189+0000 7f413dd9b640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4140104600 0x7f41401998a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4134032fd0 tx=0x7f4134031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.189+0000 7f412f7fe640 1 -- 172.21.15.33:0/2023720243 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413403f030 con 0x7f4140104600 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.189+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414019f6a0 con 0x7f4140104600 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.189+0000 7f412f7fe640 1 -- 172.21.15.33:0/2023720243 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4134004490 con 0x7f4140104600 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.189+0000 7f412f7fe640 1 -- 172.21.15.33:0/2023720243 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134035070 con 0x7f4140104600 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.189+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41401a5dc0 con 0x7f4140104600 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 Processor -- start 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 -- start start 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e74d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e760580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b7e6c3ea0 con 0x555b7e74ddf0 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555b7e6b32d0 con 0x555b7e75e190 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f4146bdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b7e6b70e0 con 0x555b7e74d4d0 2024-09-10T21:22:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f413e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f413e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34580/0 (socket says 172.21.15.33:34580) 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f413e59c640 1 -- 172.21.15.33:0/2569672258 learned_addr learned my addr 172.21.15.33:0/2569672258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.192+0000 7f413ed9d640 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e74d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f413e59c640 1 -- 172.21.15.33:0/2569672258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 msgr2=0x555b7e74d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f413e59c640 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e74d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f413e59c640 1 -- 172.21.15.33:0/2569672258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 msgr2=0x555b7e760580 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f413e59c640 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e760580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f413e59c640 1 -- 172.21.15.33:0/2569672258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7e411650 con 0x555b7e74ddf0 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f412f7fe640 1 -- 172.21.15.33:0/2023720243 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f413403ba30 con 0x7f4140104600 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f413e59c640 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f413000b790 tx=0x7f413000bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6f41c6a4d45805 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f412cff9640 1 -- 172.21.15.33:0/2569672258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130003dd0 con 0x555b7e74ddf0 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f412cff9640 1 -- 172.21.15.33:0/2569672258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4130015070 con 0x555b7e74ddf0 2024-09-10T21:22:08.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.193+0000 7f412cff9640 1 -- 172.21.15.33:0/2569672258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130011440 con 0x555b7e74ddf0 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f4146bdf740 1 -- 172.21.15.33:0/2569672258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 msgr2=0x555b7e75dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f4146bdf740 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f413000b790 tx=0x7f413000bc50 comp rx=0 tx=0).stop 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f412f7fe640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41280b52e0 0x7f41280b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f4146bdf740 1 -- 172.21.15.33:0/2569672258 shutdown_connections 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f4146bdf740 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e760580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f4146bdf740 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f4146bdf740 1 --2- 172.21.15.33:0/2569672258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e74d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f4146bdf740 1 -- 172.21.15.33:0/2569672258 >> 172.21.15.33:0/2569672258 conn(0x555b7e74b300 msgr2=0x555b7e74c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f412f7fe640 1 -- 172.21.15.33:0/2023720243 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4134106140 con 0x7f4140104600 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.194+0000 7f413ed9d640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41280b52e0 0x7f41280b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.195+0000 7f4146bdf740 1 -- 172.21.15.33:0/2569672258 shutdown_connections 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.195+0000 7f4146bdf740 1 -- 172.21.15.33:0/2569672258 wait complete. 2024-09-10T21:22:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.195+0000 7f413ed9d640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41280b52e0 0x7f41280b77a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4138005fd0 tx=0x7f4138009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.195+0000 7f4146bdf740 1 -- start start 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 1 Processor -- start 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74d4d0 0x555b7e75d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e756d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x555b7e514700 con 0x555b7e74d4d0 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555b7e64e160 con 0x555b7e75e190 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b7e760e40 con 0x555b7e74ddf0 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f413e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74d4d0 0x555b7e75d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f4146bdf740 10 client.0 _ll_register_callbacks cb 0x555b7e6679e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f413e59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74d4d0 0x555b7e75d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34586/0 (socket says 172.21.15.33:34586) 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f413e59c640 1 -- 172.21.15.33:0/4145755635 learned_addr learned my addr 172.21.15.33:0/4145755635 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f413dd9b640 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.196+0000 7f413ed9d640 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e756d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f413e59c640 1 -- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74ddf0 msgr2=0x555b7e75dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f413e59c640 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f413e59c640 1 -- 172.21.15.33:0/4145755635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 msgr2=0x555b7e756d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f413e59c640 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e756d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f413e59c640 1 -- 172.21.15.33:0/4145755635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7e67bc90 con 0x555b7e74d4d0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f413e59c640 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74d4d0 0x555b7e75d6a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f413000b760 tx=0x7f4130004420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41a92a608bc0b81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f4125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f4125ffb640 1 -- 172.21.15.33:0/4145755635 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130028060 con 0x555b7e74d4d0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f4125ffb640 1 -- 172.21.15.33:0/4145755635 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4130023660 con 0x555b7e74d4d0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.197+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4130015070 con 0x555b7e74d4d0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.198+0000 7f4125ffb640 1 -- 172.21.15.33:0/4145755635 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4130003fa0 con 0x555b7e74d4d0 2024-09-10T21:22:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.198+0000 7f4146bdf740 10 client.18681 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.198+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x555b7e7683b0 con 0x555b7e74d4d0 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.198+0000 7f4125ffb640 1 -- 172.21.15.33:0/4145755635 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4130023d10 con 0x555b7e74d4d0 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4125ffb640 1 -- 172.21.15.33:0/4145755635 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f413002d070 con 0x555b7e74d4d0 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4146bdf740 10 client.18681 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4146bdf740 10 client.18681 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4146bdf740 20 client.18681 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4146bdf740 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e769810 0x555b7e76bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4146bdf740 4 client.18681 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4130015070 con 0x555b7e769810 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f413dd9b640 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e769810 0x555b7e76bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f4125ffb640 10 client.18681 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.199+0000 7f413dd9b640 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e769810 0x555b7e76bc00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45e1721b10901b15 server_cookie=7340dc0f015d07cd in_seq=0 out_seq=0 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4125ffb640 1 -- 172.21.15.33:0/4145755635 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f4130015070 con 0x555b7e769810 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4125ffb640 10 client.18681 handle_command_reply: tid=0 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 client.18681 shutdown 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74d4d0 msgr2=0x555b7e75d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74d4d0 0x555b7e75d6a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f413000b760 tx=0x7f4130004420 comp rx=0 tx=0).stop 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 shutdown_connections 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e756d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e769810 0x555b7e76bc00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74ddf0 0x555b7e75dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 --2- 172.21.15.33:0/4145755635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74d4d0 0x555b7e75d6a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4125ffb640 0 client.18681 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 >> 172.21.15.33:0/4145755635 conn(0x555b7e74b300 msgr2=0x555b7e750ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 shutdown_connections 2024-09-10T21:22:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 1 -- 172.21.15.33:0/4145755635 wait complete. 2024-09-10T21:22:08.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.205+0000 7f4146bdf740 20 client.18681 trim_cache size 0 max 16384 2024-09-10T21:22:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 Processor -- start 2024-09-10T21:22:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 -- start start 2024-09-10T21:22:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e755fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e74ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e751d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41401ac260 con 0x555b7e74ddf0 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f41401adda0 con 0x555b7e75e190 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f4146bdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41401ac930 con 0x555b7e74d4d0 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f413dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e74ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f413dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e74ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34588/0 (socket says 172.21.15.33:34588) 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f413dd9b640 1 -- 172.21.15.33:0/652036781 learned_addr learned my addr 172.21.15.33:0/652036781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f413e59c640 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e755fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.223+0000 7f413ed9d640 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e751d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.224+0000 7f413dd9b640 1 -- 172.21.15.33:0/652036781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 msgr2=0x555b7e755fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.224+0000 7f413dd9b640 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e755fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.224+0000 7f413dd9b640 1 -- 172.21.15.33:0/652036781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 msgr2=0x555b7e751d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.224+0000 7f413dd9b640 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e751d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.224+0000 7f413dd9b640 1 -- 172.21.15.33:0/652036781 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7e74dc20 con 0x555b7e74ddf0 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.224+0000 7f413dd9b640 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e74ac10 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f41340cc8f0 tx=0x7f413402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b48e79d719a52bb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.224+0000 7f4127fff640 1 -- 172.21.15.33:0/652036781 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4134049040 con 0x555b7e74ddf0 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4127fff640 1 -- 172.21.15.33:0/652036781 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41340d13e0 con 0x555b7e74ddf0 2024-09-10T21:22:08.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4127fff640 1 -- 172.21.15.33:0/652036781 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41340c8c20 con 0x555b7e74ddf0 2024-09-10T21:22:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4146bdf740 1 -- 172.21.15.33:0/652036781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 msgr2=0x555b7e74ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4146bdf740 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e74ac10 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f41340cc8f0 tx=0x7f413402a410 comp rx=0 tx=0).stop 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4146bdf740 1 -- 172.21.15.33:0/652036781 shutdown_connections 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4146bdf740 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e75e190 0x555b7e751d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4146bdf740 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e74ac10 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4146bdf740 1 --2- 172.21.15.33:0/652036781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e74d4d0 0x555b7e755fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.225+0000 7f4146bdf740 1 -- 172.21.15.33:0/652036781 >> 172.21.15.33:0/652036781 conn(0x555b7e74b300 msgr2=0x555b7e760450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.226+0000 7f4146bdf740 1 -- 172.21.15.33:0/652036781 shutdown_connections 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.226+0000 7f4146bdf740 1 -- 172.21.15.33:0/652036781 wait complete. 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.226+0000 7f4146bdf740 1 -- start start 2024-09-10T21:22:08.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.226+0000 7f4146bdf740 1 Processor -- start 2024-09-10T21:22:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e74d4d0 0x555b7e766b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e762030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f4146bdf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e75e190 0x555b7e762700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f4146bdf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f41401ad6d0 con 0x555b7e74ddf0 2024-09-10T21:22:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f4146bdf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f41401ad000 con 0x555b7e74d4d0 2024-09-10T21:22:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f4146bdf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41401aa220 con 0x555b7e75e190 2024-09-10T21:22:08.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e75e190 0x555b7e762700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f413e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e74d4d0 0x555b7e766b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e75e190 0x555b7e762700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39256/0 (socket says 172.21.15.33:39256) 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f413ed9d640 1 -- 172.21.15.33:0/3605321770 learned_addr learned my addr 172.21.15.33:0/3605321770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f413dd9b640 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e762030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.227+0000 7f4146bdf740 10 client.0 _ll_register_callbacks cb 0x7f4138009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f413ed9d640 1 -- 172.21.15.33:0/3605321770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e74d4d0 msgr2=0x555b7e766b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f413ed9d640 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e74d4d0 0x555b7e766b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f413ed9d640 1 -- 172.21.15.33:0/3605321770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 msgr2=0x555b7e762030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f413ed9d640 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e762030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f413ed9d640 1 -- 172.21.15.33:0/3605321770 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7e7d53d0 con 0x555b7e75e190 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f413dd9b640 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e762030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f413ed9d640 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e75e190 0x555b7e762700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4138003f50 tx=0x7f4138004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ad20310b8490044 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f4125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f4125ffb640 1 -- 172.21.15.33:0/3605321770 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138035020 con 0x555b7e75e190 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.228+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41340d13e0 con 0x555b7e75e190 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4125ffb640 1 -- 172.21.15.33:0/3605321770 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f41380250d0 con 0x555b7e75e190 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4125ffb640 1 -- 172.21.15.33:0/3605321770 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4138025330 con 0x555b7e75e190 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 10 client.27955 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x555b7e7d65c0 con 0x555b7e75e190 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4125ffb640 1 -- 172.21.15.33:0/3605321770 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41340d13e0 con 0x555b7e75e190 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4125ffb640 1 -- 172.21.15.33:0/3605321770 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f413800f870 con 0x555b7e75e190 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 10 client.27955 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 10 client.27955 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 20 client.27955 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e7d6d70 0x555b7e7d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 4 client.27955 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:22:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.229+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b7e6c5df0 con 0x555b7e7d6d70 2024-09-10T21:22:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.230+0000 7f413e59c640 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e7d6d70 0x555b7e7d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.230+0000 7f4125ffb640 10 client.27955 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.230+0000 7f413e59c640 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e7d6d70 0x555b7e7d9190 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a6729446bf26539 server_cookie=7d6ed43cf8a6598a in_seq=0 out_seq=0 2024-09-10T21:22:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4125ffb640 1 -- 172.21.15.33:0/3605321770 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b7e6c5df0 con 0x555b7e7d6d70 2024-09-10T21:22:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4125ffb640 10 client.27955 handle_command_reply: tid=0 2024-09-10T21:22:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 client.27955 shutdown 2024-09-10T21:22:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e75e190 msgr2=0x555b7e762700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e75e190 0x555b7e762700 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4138003f50 tx=0x7f4138004a40 comp rx=0 tx=0).stop 2024-09-10T21:22:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 shutdown_connections 2024-09-10T21:22:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x555b7e7d6d70 0x555b7e7d9190 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4125ffb640 0 client.27955 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b7e75e190 0x555b7e762700 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555b7e74ddf0 0x555b7e762030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 --2- 172.21.15.33:0/3605321770 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555b7e74d4d0 0x555b7e766b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 >> 172.21.15.33:0/3605321770 conn(0x555b7e74b300 msgr2=0x555b7e755310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 shutdown_connections 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 1 -- 172.21.15.33:0/3605321770 wait complete. 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.231+0000 7f4146bdf740 20 client.27955 trim_cache size 0 max 16384 2024-09-10T21:22:08.239 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:22:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41280b52e0 msgr2=0x7f41280b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41280b52e0 0x7f41280b77a0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4138005fd0 tx=0x7f4138009500 comp rx=0 tx=0).stop 2024-09-10T21:22:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4140104600 msgr2=0x7f41401998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4140104600 0x7f41401998a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4134032fd0 tx=0x7f4134031a20 comp rx=0 tx=0).stop 2024-09-10T21:22:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 shutdown_connections 2024-09-10T21:22:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f41280b52e0 0x7f41280b77a0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f414010d140 0x7f414019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4140104600 0x7f41401998a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 --2- 172.21.15.33:0/2023720243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4140103ce0 0x7f4140199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 >> 172.21.15.33:0/2023720243 conn(0x7f41400fd740 msgr2=0x7f41400fe4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 shutdown_connections 2024-09-10T21:22:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:08.232+0000 7f414480a640 1 -- 172.21.15.33:0/2023720243 wait complete. 2024-09-10T21:22:09.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 249445 41.73 MB/sec execute 189 sec latency 94.059 ms 2024-09-10T21:22:09.501 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:22:09.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:09 smithi104 ceph-mon[35325]: pgmap v891: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 933 op/s 2024-09-10T21:22:09.826 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:09 smithi174 ceph-mon[35635]: pgmap v891: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 933 op/s 2024-09-10T21:22:09.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:09 smithi033 ceph-mon[30860]: pgmap v891: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 12 MiB/s wr, 933 op/s 2024-09-10T21:22:10.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 250265 41.74 MB/sec execute 190 sec latency 58.545 ms 2024-09-10T21:22:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.402+0000 7fba47493640 1 -- 172.21.15.33:0/3847868779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba4010a3b0 msgr2=0x7fba4010a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.402+0000 7fba47493640 1 --2- 172.21.15.33:0/3847868779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba4010a3b0 0x7fba4010a790 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009d50 tx=0x7fba3c02a840 comp rx=0 tx=0).stop 2024-09-10T21:22:10.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.403+0000 7fba47493640 1 -- 172.21.15.33:0/3847868779 shutdown_connections 2024-09-10T21:22:10.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.403+0000 7fba47493640 1 --2- 172.21.15.33:0/3847868779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba400759f0 0x7fba4007c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.403+0000 7fba47493640 1 --2- 172.21.15.33:0/3847868779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba40075070 0x7fba400754b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.403+0000 7fba47493640 1 --2- 172.21.15.33:0/3847868779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba4010a3b0 0x7fba4010a790 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.403+0000 7fba47493640 1 -- 172.21.15.33:0/3847868779 >> 172.21.15.33:0/3847868779 conn(0x7fba4006dfc0 msgr2=0x7fba4006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.403+0000 7fba47493640 1 -- 172.21.15.33:0/3847868779 shutdown_connections 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.404+0000 7fba47493640 1 -- 172.21.15.33:0/3847868779 wait complete. 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.405+0000 7fba47493640 1 Processor -- start 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.405+0000 7fba47493640 1 -- start start 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.406+0000 7fba47493640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba40075070 0x7fba4007bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.406+0000 7fba47493640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba400759f0 0x7fba40076e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.406+0000 7fba47493640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba4010a3b0 0x7fba400773b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.406+0000 7fba47493640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba4007f590 con 0x7fba40075070 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.406+0000 7fba47493640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba4007f410 con 0x7fba4010a3b0 2024-09-10T21:22:10.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.406+0000 7fba47493640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba4007f710 con 0x7fba400759f0 2024-09-10T21:22:10.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.408+0000 7fba44a2e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba400759f0 0x7fba40076e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.408+0000 7fba44a2e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba400759f0 0x7fba40076e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55652/0 (socket says 172.21.15.33:55652) 2024-09-10T21:22:10.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.408+0000 7fba44a2e640 1 -- 172.21.15.33:0/3217108351 learned_addr learned my addr 172.21.15.33:0/3217108351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:10.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.408+0000 7fba4522f640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba40075070 0x7fba4007bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.409+0000 7fba45a30640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba4010a3b0 0x7fba400773b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.410+0000 7fba44a2e640 1 -- 172.21.15.33:0/3217108351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba4010a3b0 msgr2=0x7fba400773b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.410+0000 7fba44a2e640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba4010a3b0 0x7fba400773b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.410+0000 7fba44a2e640 1 -- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba40075070 msgr2=0x7fba4007bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.410+0000 7fba44a2e640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba40075070 0x7fba4007bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.410+0000 7fba44a2e640 1 -- 172.21.15.33:0/3217108351 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba400779f0 con 0x7fba400759f0 2024-09-10T21:22:10.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.410+0000 7fba44a2e640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba400759f0 0x7fba40076e70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba3000e9f0 tx=0x7fba3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:10.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.411+0000 7fba367fc640 1 -- 172.21.15.33:0/3217108351 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3000cd80 con 0x7fba400759f0 2024-09-10T21:22:10.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.411+0000 7fba367fc640 1 -- 172.21.15.33:0/3217108351 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba300108e0 con 0x7fba400759f0 2024-09-10T21:22:10.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.411+0000 7fba367fc640 1 -- 172.21.15.33:0/3217108351 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba30018880 con 0x7fba400759f0 2024-09-10T21:22:10.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.411+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba401bcf00 con 0x7fba400759f0 2024-09-10T21:22:10.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.411+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba401bd330 con 0x7fba400759f0 2024-09-10T21:22:10.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 Processor -- start 2024-09-10T21:22:10.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 -- start start 2024-09-10T21:22:10.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820f2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b8205cea0 con 0x561b820f29a0 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561b8204c2d0 con 0x561b820e6a10 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.418+0000 7fba49868740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b820500e0 con 0x561b820e60f0 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba4522f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba4522f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820f2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39404/0 (socket says 172.21.15.33:39404) 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba4522f640 1 -- 172.21.15.33:0/3615807499 learned_addr learned my addr 172.21.15.33:0/3615807499 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba367fc640 1 -- 172.21.15.33:0/3217108351 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fba30018a20 con 0x7fba400759f0 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba45a30640 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba4522f640 1 -- 172.21.15.33:0/3615807499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 msgr2=0x561b820e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:10.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba4522f640 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba44a2e640 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba4522f640 1 -- 172.21.15.33:0/3615807499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 msgr2=0x561b820f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:10.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.419+0000 7fba4522f640 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.420+0000 7fba4522f640 1 -- 172.21.15.33:0/3615807499 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b81daa650 con 0x561b820e6a10 2024-09-10T21:22:10.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.420+0000 7fba45a30640 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:22:10.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.420+0000 7fba44a2e640 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:22:10.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.421+0000 7fba367fc640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fba2c0b54b0 0x7fba2c0b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.421+0000 7fba45a30640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fba2c0b54b0 0x7fba2c0b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.422+0000 7fba367fc640 1 -- 172.21.15.33:0/3217108351 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fba300e6f30 con 0x7fba400759f0 2024-09-10T21:22:10.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.422+0000 7fba4522f640 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820f2460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009d50 tx=0x7fba3c009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dff5505f4515837 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:10.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.422+0000 7fba45a30640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fba2c0b54b0 0x7fba2c0b7970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fba38005fd0 tx=0x7fba38009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.426+0000 7fba2b7fe640 1 -- 172.21.15.33:0/3615807499 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c040030 con 0x561b820e6a10 2024-09-10T21:22:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.426+0000 7fba2b7fe640 1 -- 172.21.15.33:0/3615807499 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba3c009af0 con 0x561b820e6a10 2024-09-10T21:22:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 -- 172.21.15.33:0/3615807499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 msgr2=0x561b820f2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820f2460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009d50 tx=0x7fba3c009900 comp rx=0 tx=0).stop 2024-09-10T21:22:10.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 -- 172.21.15.33:0/3615807499 shutdown_connections 2024-09-10T21:22:10.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820f2460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 --2- 172.21.15.33:0/3615807499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 -- 172.21.15.33:0/3615807499 >> 172.21.15.33:0/3615807499 conn(0x561b820e3f20 msgr2=0x561b820e50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 -- 172.21.15.33:0/3615807499 shutdown_connections 2024-09-10T21:22:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.427+0000 7fba49868740 1 -- 172.21.15.33:0/3615807499 wait complete. 2024-09-10T21:22:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.428+0000 7fba49868740 1 -- start start 2024-09-10T21:22:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.428+0000 7fba49868740 1 Processor -- start 2024-09-10T21:22:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba49868740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820f2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba49868740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820ef490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba49868740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba49868740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x561b81ead700 con 0x561b820e6a10 2024-09-10T21:22:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba49868740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x561b81fe7160 con 0x561b820e60f0 2024-09-10T21:22:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba49868740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b820f5610 con 0x561b820f29a0 2024-09-10T21:22:10.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba49868740 10 client.0 _ll_register_callbacks cb 0x561b820009e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba45a30640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba45a30640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ef9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55674/0 (socket says 172.21.15.33:55674) 2024-09-10T21:22:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.429+0000 7fba45a30640 1 -- 172.21.15.33:0/4268929127 learned_addr learned my addr 172.21.15.33:0/4268929127 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.430+0000 7fba4522f640 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.430+0000 7fba44a2e640 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820ef490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.430+0000 7fba45a30640 1 -- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 msgr2=0x561b820f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.430+0000 7fba45a30640 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.430+0000 7fba45a30640 1 -- 172.21.15.33:0/4268929127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 msgr2=0x561b820ef490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:10.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.430+0000 7fba45a30640 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820ef490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.430+0000 7fba45a30640 1 -- 172.21.15.33:0/4268929127 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b82014c90 con 0x561b820f29a0 2024-09-10T21:22:10.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.431+0000 7fba45a30640 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ef9d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba38009950 tx=0x7fba38004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163ed8e317273316 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:10.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.431+0000 7fba28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:22:10.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.431+0000 7fba28ff9640 1 -- 172.21.15.33:0/4268929127 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3801f620 con 0x561b820f29a0 2024-09-10T21:22:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.431+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba3c009af0 con 0x561b820f29a0 2024-09-10T21:22:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.431+0000 7fba28ff9640 1 -- 172.21.15.33:0/4268929127 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fba3801f7c0 con 0x561b820f29a0 2024-09-10T21:22:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.432+0000 7fba28ff9640 1 -- 172.21.15.33:0/4268929127 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fba38012e10 con 0x561b820f29a0 2024-09-10T21:22:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.432+0000 7fba49868740 10 client.27964 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.432+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x561b82100df0 con 0x561b820f29a0 2024-09-10T21:22:10.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.433+0000 7fba28ff9640 1 -- 172.21.15.33:0/4268929127 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba3c009af0 con 0x561b820f29a0 2024-09-10T21:22:10.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.433+0000 7fba28ff9640 1 -- 172.21.15.33:0/4268929127 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fba38032b60 con 0x561b820f29a0 2024-09-10T21:22:10.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.434+0000 7fba49868740 10 client.27964 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:10.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.434+0000 7fba49868740 10 client.27964 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:22:10.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.434+0000 7fba49868740 20 client.27964 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:10.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.434+0000 7fba49868740 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b821026a0 0x561b82104b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.434+0000 7fba49868740 4 client.27964 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:10.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.434+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b8205edf0 con 0x561b821026a0 2024-09-10T21:22:10.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.441+0000 7fba4522f640 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b821026a0 0x561b82104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.445+0000 7fba4522f640 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b821026a0 0x561b82104b00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d55247d6448ef2cf server_cookie=60ba4c474ae40cfc in_seq=0 out_seq=0 2024-09-10T21:22:10.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.445+0000 7fba28ff9640 10 client.27964 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:22:10.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.451+0000 7fba28ff9640 1 -- 172.21.15.33:0/4268929127 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561b8205edf0 con 0x561b821026a0 2024-09-10T21:22:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.451+0000 7fba28ff9640 10 client.27964 handle_command_reply: tid=0 2024-09-10T21:22:10.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.452+0000 7fba49868740 1 client.27964 shutdown 2024-09-10T21:22:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.452+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 msgr2=0x561b820ef9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.452+0000 7fba49868740 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ef9d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fba38009950 tx=0x7fba38004a40 comp rx=0 tx=0).stop 2024-09-10T21:22:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 shutdown_connections 2024-09-10T21:22:10.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ef9d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b821026a0 0x561b82104b00 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820ef490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 --2- 172.21.15.33:0/4268929127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 >> 172.21.15.33:0/4268929127 conn(0x561b820e3f20 msgr2=0x561b820e9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba28ff9640 0 client.27964 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 shutdown_connections 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 1 -- 172.21.15.33:0/4268929127 wait complete. 2024-09-10T21:22:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.453+0000 7fba49868740 20 client.27964 trim_cache size 0 max 16384 2024-09-10T21:22:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.482+0000 7fba49868740 1 Processor -- start 2024-09-10T21:22:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.482+0000 7fba49868740 1 -- start start 2024-09-10T21:22:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.482+0000 7fba49868740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820edd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.482+0000 7fba49868740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820ee250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba49868740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba49868740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400e500 con 0x561b820f29a0 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba49868740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba24010040 con 0x561b820e6a10 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba49868740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400ebd0 con 0x561b820e60f0 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820edd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820edd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55688/0 (socket says 172.21.15.33:55688) 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 -- 172.21.15.33:0/788841724 learned_addr learned my addr 172.21.15.33:0/788841724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba45a30640 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 -- 172.21.15.33:0/788841724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 msgr2=0x561b820ee250 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820ee250 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 -- 172.21.15.33:0/788841724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 msgr2=0x561b820ec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820ec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba4522f640 1 -- 172.21.15.33:0/788841724 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b820e6840 con 0x561b820e60f0 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.483+0000 7fba45a30640 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.484+0000 7fba4522f640 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820edd10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x561b820e4e00 tx=0x7fba3c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99537aa889e81706 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.484+0000 7fba2affd640 1 -- 172.21.15.33:0/788841724 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c02e030 con 0x561b820e60f0 2024-09-10T21:22:10.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.484+0000 7fba2affd640 1 -- 172.21.15.33:0/788841724 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba3c003aa0 con 0x561b820e60f0 2024-09-10T21:22:10.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.484+0000 7fba2affd640 1 -- 172.21.15.33:0/788841724 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c040040 con 0x561b820e60f0 2024-09-10T21:22:10.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.486+0000 7fba49868740 1 -- 172.21.15.33:0/788841724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 msgr2=0x561b820edd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.486+0000 7fba49868740 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820edd10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x561b820e4e00 tx=0x7fba3c000bc0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.486+0000 7fba49868740 1 -- 172.21.15.33:0/788841724 shutdown_connections 2024-09-10T21:22:10.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.486+0000 7fba49868740 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820f29a0 0x561b820ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.486+0000 7fba49868740 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e6a10 0x561b820ee250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.486+0000 7fba49868740 1 --2- 172.21.15.33:0/788841724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820e60f0 0x561b820edd10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.486+0000 7fba49868740 1 -- 172.21.15.33:0/788841724 >> 172.21.15.33:0/788841724 conn(0x561b820e3f20 msgr2=0x561b820ef400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.487+0000 7fba49868740 1 -- 172.21.15.33:0/788841724 shutdown_connections 2024-09-10T21:22:10.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.487+0000 7fba49868740 1 -- 172.21.15.33:0/788841724 wait complete. 2024-09-10T21:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.487+0000 7fba49868740 1 -- start start 2024-09-10T21:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 1 Processor -- start 2024-09-10T21:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820fb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400f970 con 0x561b820e6a10 2024-09-10T21:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400f2a0 con 0x561b820e60f0 2024-09-10T21:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba2400c4c0 con 0x561b820f29a0 2024-09-10T21:22:10.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.488+0000 7fba49868740 10 client.0 _ll_register_callbacks cb 0x7fba380099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.489+0000 7fba44a2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820fb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.489+0000 7fba4522f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.489+0000 7fba44a2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820fb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37056/0 (socket says 172.21.15.33:37056) 2024-09-10T21:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.489+0000 7fba44a2e640 1 -- 172.21.15.33:0/1101003779 learned_addr learned my addr 172.21.15.33:0/1101003779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.489+0000 7fba45a30640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba4522f640 1 -- 172.21.15.33:0/1101003779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 msgr2=0x561b820ec1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:10.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba4522f640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ec1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba4522f640 1 -- 172.21.15.33:0/1101003779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 msgr2=0x561b820fb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba4522f640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820fb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba4522f640 1 -- 172.21.15.33:0/1101003779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b820eca20 con 0x561b820e60f0 2024-09-10T21:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba44a2e640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba45a30640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.490+0000 7fba4522f640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820fadb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009940 tx=0x7fba3c032630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3148c72ffcf7d9ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:10.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.491+0000 7fba28ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.493+0000 7fba28ff9640 1 -- 172.21.15.33:0/1101003779 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c05a060 con 0x561b820e60f0 2024-09-10T21:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.493+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba3c003aa0 con 0x561b820e60f0 2024-09-10T21:22:10.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.493+0000 7fba28ff9640 1 -- 172.21.15.33:0/1101003779 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fba3c0035f0 con 0x561b820e60f0 2024-09-10T21:22:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.500+0000 7fba28ff9640 1 -- 172.21.15.33:0/1101003779 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fba3c02e070 con 0x561b820e60f0 2024-09-10T21:22:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.500+0000 7fba49868740 10 client.28085 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:10.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.500+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x561b820f95b0 con 0x561b820e60f0 2024-09-10T21:22:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.503+0000 7fba28ff9640 1 -- 172.21.15.33:0/1101003779 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba3c003aa0 con 0x561b820e60f0 2024-09-10T21:22:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.503+0000 7fba28ff9640 1 -- 172.21.15.33:0/1101003779 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fba3c05f070 con 0x561b820e60f0 2024-09-10T21:22:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.503+0000 7fba49868740 10 client.28085 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.503+0000 7fba49868740 10 client.28085 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:22:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.503+0000 7fba49868740 20 client.28085 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.504+0000 7fba49868740 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b8216f890 0x561b82171cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.504+0000 7fba49868740 4 client.28085 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:22:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.504+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b8205a8a0 con 0x561b8216f890 2024-09-10T21:22:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.506+0000 7fba44a2e640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b8216f890 0x561b82171cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:10.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.512+0000 7fba44a2e640 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b8216f890 0x561b82171cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=36c69f42f8ea0dda server_cookie=cacd12760fbd189f in_seq=0 out_seq=0 2024-09-10T21:22:10.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.512+0000 7fba28ff9640 10 client.28085 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:22:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba28ff9640 1 -- 172.21.15.33:0/1101003779 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b8205a8a0 con 0x561b8216f890 2024-09-10T21:22:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba28ff9640 10 client.28085 handle_command_reply: tid=0 2024-09-10T21:22:10.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 client.28085 shutdown 2024-09-10T21:22:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 msgr2=0x561b820fadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820fadb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fba3c009940 tx=0x7fba3c032630 comp rx=0 tx=0).stop 2024-09-10T21:22:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 shutdown_connections 2024-09-10T21:22:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b820f29a0 0x561b820ec1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x561b8216f890 0x561b82171cb0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x561b820e6a10 0x561b820fb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 --2- 172.21.15.33:0/1101003779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x561b820e60f0 0x561b820fadb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 >> 172.21.15.33:0/1101003779 conn(0x561b820e3f20 msgr2=0x561b820edc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.515+0000 7fba28ff9640 0 client.28085 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:22:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.516+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 shutdown_connections 2024-09-10T21:22:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.516+0000 7fba49868740 1 -- 172.21.15.33:0/1101003779 wait complete. 2024-09-10T21:22:10.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.516+0000 7fba49868740 20 client.28085 trim_cache size 0 max 16384 2024-09-10T21:22:10.523 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:22:10.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fba2c0b54b0 msgr2=0x7fba2c0b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fba2c0b54b0 0x7fba2c0b7970 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fba38005fd0 tx=0x7fba38009500 comp rx=0 tx=0).stop 2024-09-10T21:22:10.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba400759f0 msgr2=0x7fba40076e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:10.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba400759f0 0x7fba40076e70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba3000e9f0 tx=0x7fba3000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 shutdown_connections 2024-09-10T21:22:10.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fba2c0b54b0 0x7fba2c0b7970 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba4010a3b0 0x7fba400773b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba400759f0 0x7fba40076e70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 --2- 172.21.15.33:0/3217108351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba40075070 0x7fba4007bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.517+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 >> 172.21.15.33:0/3217108351 conn(0x7fba4006dfc0 msgr2=0x7fba400730d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.518+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 shutdown_connections 2024-09-10T21:22:10.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:10.518+0000 7fba47493640 1 -- 172.21.15.33:0/3217108351 wait complete. 2024-09-10T21:22:11.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 250679 41.63 MB/sec execute 191 sec latency 336.585 ms 2024-09-10T21:22:11.255 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:22:11.622 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:11.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:11 smithi033 ceph-mon[30860]: pgmap v892: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 785 op/s 2024-09-10T21:22:11.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:11 smithi104 ceph-mon[35325]: pgmap v892: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 785 op/s 2024-09-10T21:22:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:11 smithi174 ceph-mon[35635]: pgmap v892: 97 pgs: 97 active+clean; 1.2 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 785 op/s 2024-09-10T21:22:12.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 251376 41.59 MB/sec execute 192 sec latency 106.678 ms 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 -- 172.21.15.33:0/3602855110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c109150 msgr2=0x7f1f6c10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 --2- 172.21.15.33:0/3602855110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c109150 0x7f1f6c10f950 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009be0 tx=0x7f1f6402a6c0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 -- 172.21.15.33:0/3602855110 shutdown_connections 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 --2- 172.21.15.33:0/3602855110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c109150 0x7f1f6c10f950 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 --2- 172.21.15.33:0/3602855110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f6c1087d0 0x7f1f6c108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 --2- 172.21.15.33:0/3602855110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c102dc0 0x7f1f6c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 -- 172.21.15.33:0/3602855110 >> 172.21.15.33:0/3602855110 conn(0x7f1f6c0fe870 msgr2=0x7f1f6c100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:12.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 -- 172.21.15.33:0/3602855110 shutdown_connections 2024-09-10T21:22:12.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.152+0000 7f1f74736640 1 -- 172.21.15.33:0/3602855110 wait complete. 2024-09-10T21:22:12.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.153+0000 7f1f74736640 1 Processor -- start 2024-09-10T21:22:12.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.153+0000 7f1f74736640 1 -- start start 2024-09-10T21:22:12.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f74736640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f6c102dc0 0x7f1f6c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f74736640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c1087d0 0x7f1f6c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f74736640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c109150 0x7f1f6c19eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f74736640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c112dd0 con 0x7f1f6c1087d0 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f74736640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c112c50 con 0x7f1f6c109150 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f74736640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c112f50 con 0x7f1f6c102dc0 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c109150 0x7f1f6c19eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c109150 0x7f1f6c19eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39434/0 (socket says 172.21.15.33:39434) 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 -- 172.21.15.33:0/4084327797 learned_addr learned my addr 172.21.15.33:0/4084327797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c1087d0 0x7f1f6c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 -- 172.21.15.33:0/4084327797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f6c102dc0 msgr2=0x7f1f6c1992a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f6c102dc0 0x7f1f6c1992a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 -- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c1087d0 msgr2=0x7f1f6c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c1087d0 0x7f1f6c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.154+0000 7f1f72cd3640 1 -- 172.21.15.33:0/4084327797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6c19f350 con 0x7f1f6c109150 2024-09-10T21:22:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.155+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c109150 0x7f1f6c19eb80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009b30 tx=0x7f1f64003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.155+0000 7f1f637fe640 1 -- 172.21.15.33:0/4084327797 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64040020 con 0x7f1f6c109150 2024-09-10T21:22:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.155+0000 7f1f637fe640 1 -- 172.21.15.33:0/4084327797 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f64030970 con 0x7f1f6c109150 2024-09-10T21:22:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.155+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6c19f5e0 con 0x7f1f6c109150 2024-09-10T21:22:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.155+0000 7f1f637fe640 1 -- 172.21.15.33:0/4084327797 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64036040 con 0x7f1f6c109150 2024-09-10T21:22:12.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.155+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6c1a5d00 con 0x7f1f6c109150 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.158+0000 7f1f76b0b740 1 Processor -- start 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f637fe640 1 -- 172.21.15.33:0/4084327797 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1f640045c0 con 0x7f1f6c109150 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f76b0b740 1 -- start start 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e2848354d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e284835a10 0x55e284841460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848419a0 0x55e284843d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2847abea0 con 0x55e2848350f0 2024-09-10T21:22:12.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e28479b2d0 con 0x55e284835a10 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e28479f0e0 con 0x55e2848419a0 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f71cd1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e2848354d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f71cd1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e2848354d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37082/0 (socket says 172.21.15.33:37082) 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f71cd1640 1 -- 172.21.15.33:0/488968697 learned_addr learned my addr 172.21.15.33:0/488968697 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f724d2640 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e284835a10 0x55e284841460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.159+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848419a0 0x55e284843d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f71cd1640 1 -- 172.21.15.33:0/488968697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848419a0 msgr2=0x55e284843d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848419a0 0x55e284843d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f71cd1640 1 -- 172.21.15.33:0/488968697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e284835a10 msgr2=0x55e284841460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e284835a10 0x55e284841460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f71cd1640 1 -- 172.21.15.33:0/488968697 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2844f9650 con 0x55e2848350f0 2024-09-10T21:22:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f724d2640 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e284835a10 0x55e284841460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f637fe640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f580b5360 0x7f1f580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e2848354d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005bc0 tx=0x7f1f5c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed22a33145b0c41 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f60ff9640 1 -- 172.21.15.33:0/488968697 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c004050 con 0x55e2848350f0 2024-09-10T21:22:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f724d2640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f580b5360 0x7f1f580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f637fe640 1 -- 172.21.15.33:0/4084327797 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1f64106ac0 con 0x7f1f6c109150 2024-09-10T21:22:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f60ff9640 1 -- 172.21.15.33:0/488968697 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5c0041f0 con 0x55e2848350f0 2024-09-10T21:22:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.160+0000 7f1f60ff9640 1 -- 172.21.15.33:0/488968697 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00f760 con 0x55e2848350f0 2024-09-10T21:22:12.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 -- 172.21.15.33:0/488968697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 msgr2=0x55e2848354d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e2848354d0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c005bc0 tx=0x7f1f5c005850 comp rx=0 tx=0).stop 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f724d2640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f580b5360 0x7f1f580b7820 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55e284833e00 tx=0x7f1f68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 -- 172.21.15.33:0/488968697 shutdown_connections 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848419a0 0x55e284843d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e284835a10 0x55e284841460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/488968697 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e2848354d0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 -- 172.21.15.33:0/488968697 >> 172.21.15.33:0/488968697 conn(0x55e284832f20 msgr2=0x55e2848340c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 -- 172.21.15.33:0/488968697 shutdown_connections 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.161+0000 7f1f76b0b740 1 -- 172.21.15.33:0/488968697 wait complete. 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 -- start start 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 Processor -- start 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848350f0 0x55e284841380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e284835a10 0x55e28483e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e28483ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e2845fc700 con 0x55e284835a10 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e284736160 con 0x55e2848419a0 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e284848910 con 0x55e2848350f0 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f76b0b740 10 client.0 _ll_register_callbacks cb 0x55e28474f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f71cd1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e284835a10 0x55e28483e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f71cd1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e284835a10 0x55e28483e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37090/0 (socket says 172.21.15.33:37090) 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.162+0000 7f1f71cd1640 1 -- 172.21.15.33:0/2853490229 learned_addr learned my addr 172.21.15.33:0/2853490229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f724d2640 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848350f0 0x55e284841380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e28483ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f71cd1640 1 -- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848350f0 msgr2=0x55e284841380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848350f0 0x55e284841380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f71cd1640 1 -- 172.21.15.33:0/2853490229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 msgr2=0x55e28483ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e28483ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f71cd1640 1 -- 172.21.15.33:0/2853490229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e284763c90 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e284835a10 0x55e28483e870 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009710 tx=0x7f1f5c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b526da254087f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f55ffb640 1 -- 172.21.15.33:0/2853490229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c0037f0 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f55ffb640 1 -- 172.21.15.33:0/2853490229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1f5c00f500 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5c0041f0 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f55ffb640 1 -- 172.21.15.33:0/2853490229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1f5c00f700 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f76b0b740 10 client.18711 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.163+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55e2848543d0 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f55ffb640 1 -- 172.21.15.33:0/2853490229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f5c0041f0 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f55ffb640 1 -- 172.21.15.33:0/2853490229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f1f5c01f630 con 0x55e284835a10 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f76b0b740 10 client.18711 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:12.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f76b0b740 10 client.18711 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:22:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f76b0b740 20 client.18711 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284857b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f76b0b740 4 client.18711 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.164+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2847addf0 con 0x55e284855720 2024-09-10T21:22:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.165+0000 7f1f724d2640 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284857b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.165+0000 7f1f724d2640 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284857b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a06d83966a3d03 server_cookie=e960bb3f75cb6061 in_seq=0 out_seq=0 2024-09-10T21:22:12.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.165+0000 7f1f55ffb640 10 client.18711 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:22:12.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.168+0000 7f1f55ffb640 1 -- 172.21.15.33:0/2853490229 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e2847addf0 con 0x55e284855720 2024-09-10T21:22:12.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.168+0000 7f1f55ffb640 10 client.18711 handle_command_reply: tid=0 2024-09-10T21:22:12.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.168+0000 7f1f76b0b740 1 client.18711 shutdown 2024-09-10T21:22:12.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e284835a10 msgr2=0x55e28483e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e284835a10 0x55e28483e870 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009710 tx=0x7f1f5c005c70 comp rx=0 tx=0).stop 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 shutdown_connections 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e28483ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284857b10 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e284835a10 0x55e28483e870 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f55ffb640 0 client.18711 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/2853490229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2848350f0 0x55e284841380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 >> 172.21.15.33:0/2853490229 conn(0x55e284832f20 msgr2=0x55e284838ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 shutdown_connections 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 1 -- 172.21.15.33:0/2853490229 wait complete. 2024-09-10T21:22:12.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.169+0000 7f1f76b0b740 20 client.18711 trim_cache size 0 max 16384 2024-09-10T21:22:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 Processor -- start 2024-09-10T21:22:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 -- start start 2024-09-10T21:22:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e284832760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e2848402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284840830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1ac1c0 con 0x55e2848350f0 2024-09-10T21:22:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1add00 con 0x55e2848419a0 2024-09-10T21:22:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.186+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1ac890 con 0x55e284835a10 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f724d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e284832760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284840830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284840830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39452/0 (socket says 172.21.15.33:39452) 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f724d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e284832760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37102/0 (socket says 172.21.15.33:37102) 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f71cd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e2848402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 -- 172.21.15.33:0/1282280052 learned_addr learned my addr 172.21.15.33:0/1282280052 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 -- 172.21.15.33:0/1282280052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 msgr2=0x55e2848402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/1282280052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e2848402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 -- 172.21.15.33:0/1282280052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 msgr2=0x55e284832760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/1282280052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e284832760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 -- 172.21.15.33:0/1282280052 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2848339a0 con 0x55e2848419a0 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/1282280052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284840830 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f641045c0 tx=0x7f1f64004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a54620faa9399d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.187+0000 7f1f57fff640 1 -- 172.21.15.33:0/1282280052 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640d6160 con 0x55e2848419a0 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f57fff640 1 -- 172.21.15.33:0/1282280052 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f640cd980 con 0x55e2848419a0 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f57fff640 1 -- 172.21.15.33:0/1282280052 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f640c94f0 con 0x55e2848419a0 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 -- 172.21.15.33:0/1282280052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 msgr2=0x55e284840830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/1282280052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284840830 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f1f641045c0 tx=0x7f1f64004990 comp rx=0 tx=0).stop 2024-09-10T21:22:12.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 -- 172.21.15.33:0/1282280052 shutdown_connections 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/1282280052 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284840830 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/1282280052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e2848402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/1282280052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e284832760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 -- 172.21.15.33:0/1282280052 >> 172.21.15.33:0/1282280052 conn(0x55e284832f20 msgr2=0x55e28483fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.188+0000 7f1f76b0b740 1 -- 172.21.15.33:0/1282280052 shutdown_connections 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 -- 172.21.15.33:0/1282280052 wait complete. 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 -- start start 2024-09-10T21:22:12.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 Processor -- start 2024-09-10T21:22:12.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e28484dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e28484e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284848c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1ad630 con 0x55e2848350f0 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1acf60 con 0x55e2848419a0 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6c1aa180 con 0x55e284835a10 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.189+0000 7f1f76b0b740 10 client.0 _ll_register_callbacks cb 0x7f1f68009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f71cd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e28484e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e28484dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f72cd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284848c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e28484dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37118/0 (socket says 172.21.15.33:37118) 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f71cd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e28484e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55758/0 (socket says 172.21.15.33:55758) 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 -- 172.21.15.33:0/773105845 learned_addr learned my addr 172.21.15.33:0/773105845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:12.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 -- 172.21.15.33:0/773105845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 msgr2=0x55e28484e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e28484e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 -- 172.21.15.33:0/773105845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 msgr2=0x55e284848c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284848c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 -- 172.21.15.33:0/773105845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e28484e880 con 0x55e2848350f0 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f72cd3640 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284848c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f724d2640 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e28484dc50 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1f68012fd0 tx=0x7f1f68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84012dfac2e1d38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.190+0000 7f1f55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f55ffb640 1 -- 172.21.15.33:0/773105845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68034060 con 0x55e2848350f0 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f55ffb640 1 -- 172.21.15.33:0/773105845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1f6802f620 con 0x55e2848350f0 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f640cd980 con 0x55e2848350f0 2024-09-10T21:22:12.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f55ffb640 1 -- 172.21.15.33:0/773105845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1f6802f820 con 0x55e2848350f0 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f76b0b740 10 client.18717 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55e284844440 con 0x55e2848350f0 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f55ffb640 1 -- 172.21.15.33:0/773105845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f68029410 con 0x55e2848350f0 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f55ffb640 1 -- 172.21.15.33:0/773105845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f1f68039070 con 0x55e2848350f0 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f76b0b740 10 client.18717 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.191+0000 7f1f76b0b740 10 client.18717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.192+0000 7f1f76b0b740 20 client.18717 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.192+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284846c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:12.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.192+0000 7f1f76b0b740 4 client.18717 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.192+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1f640cd980 con 0x55e284855720 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.192+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284846c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.193+0000 7f1f71cd1640 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284846c30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f72b1f0a836d91c server_cookie=7f7b895f9103a0c3 in_seq=0 out_seq=0 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.193+0000 7f1f55ffb640 10 client.18717 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.193+0000 7f1f55ffb640 1 -- 172.21.15.33:0/773105845 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1f640cd980 con 0x55e284855720 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.193+0000 7f1f55ffb640 10 client.18717 handle_command_reply: tid=0 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.193+0000 7f1f76b0b740 1 client.18717 shutdown 2024-09-10T21:22:12.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 msgr2=0x55e28484dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e28484dc50 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1f68012fd0 tx=0x7f1f68004730 comp rx=0 tx=0).stop 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 shutdown_connections 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e2848419a0 0x55e284848c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e284835a10 0x55e28484e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55e284855720 0x55e284846c30 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 --2- 172.21.15.33:0/773105845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e2848350f0 0x55e28484dc50 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 >> 172.21.15.33:0/773105845 conn(0x55e284832f20 msgr2=0x55e284840400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f55ffb640 0 client.18717 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:22:12.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 shutdown_connections 2024-09-10T21:22:12.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 1 -- 172.21.15.33:0/773105845 wait complete. 2024-09-10T21:22:12.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.194+0000 7f1f76b0b740 20 client.18717 trim_cache size 0 max 16384 2024-09-10T21:22:12.203 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:22:12.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f580b5360 msgr2=0x7f1f580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f580b5360 0x7f1f580b7820 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x55e284833e00 tx=0x7f1f68009260 comp rx=0 tx=0).stop 2024-09-10T21:22:12.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c109150 msgr2=0x7f1f6c19eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:12.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c109150 0x7f1f6c19eb80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009b30 tx=0x7f1f64003c10 comp rx=0 tx=0).stop 2024-09-10T21:22:12.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 shutdown_connections 2024-09-10T21:22:12.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f580b5360 0x7f1f580b7820 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f6c109150 0x7f1f6c19eb80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f6c1087d0 0x7f1f6c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 --2- 172.21.15.33:0/4084327797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f6c102dc0 0x7f1f6c1992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:12.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 >> 172.21.15.33:0/4084327797 conn(0x7f1f6c0fe870 msgr2=0x7f1f6c0ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:12.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 shutdown_connections 2024-09-10T21:22:12.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:12.195+0000 7f1f74736640 1 -- 172.21.15.33:0/4084327797 wait complete. 2024-09-10T21:22:13.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 252280 41.65 MB/sec execute 193 sec latency 78.609 ms 2024-09-10T21:22:13.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:22:13.436 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:13.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:13 smithi104 ceph-mon[35325]: pgmap v893: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 801 op/s 2024-09-10T21:22:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:13 smithi174 ceph-mon[35635]: pgmap v893: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 801 op/s 2024-09-10T21:22:13.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:13 smithi033 ceph-mon[30860]: pgmap v893: 97 pgs: 97 active+clean; 1.2 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 801 op/s 2024-09-10T21:22:14.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 252980 41.66 MB/sec execute 194 sec latency 63.173 ms 2024-09-10T21:22:15.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 253710 41.57 MB/sec execute 195 sec latency 81.096 ms 2024-09-10T21:22:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.771+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/873809968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 msgr2=0x7f5ef4101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.771+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/873809968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4101630 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009900 tx=0x7f5ee402a410 comp rx=0 tx=0).stop 2024-09-10T21:22:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.772+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/873809968 shutdown_connections 2024-09-10T21:22:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.772+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/873809968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef4105f40 0x7f5ef410f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.772+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/873809968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef41055a0 0x7f5ef4105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.772+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/873809968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4101630 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.772+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/873809968 >> 172.21.15.33:0/873809968 conn(0x7f5ef4078140 msgr2=0x7f5ef40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.772+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/873809968 shutdown_connections 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.772+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/873809968 wait complete. 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.773+0000 7f5ef8bc1640 1 Processor -- start 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.773+0000 7f5ef8bc1640 1 -- start start 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.773+0000 7f5ef8bc1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4074800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef8bc1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef41055a0 0x7f5ef4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef8bc1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef4105f40 0x7f5ef406dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef8bc1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4112db0 con 0x7f5ef4101250 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4074800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37134/0 (socket says 172.21.15.33:37134) 2024-09-10T21:22:15.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef259c640 1 -- 172.21.15.33:0/4024899206 learned_addr learned my addr 172.21.15.33:0/4024899206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4112c30 con 0x7f5ef41055a0 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4112f30 con 0x7f5ef4105f40 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef4105f40 0x7f5ef406dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef2d9d640 1 -- 172.21.15.33:0/4024899206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef41055a0 msgr2=0x7f5ef4074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef41055a0 0x7f5ef4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef2d9d640 1 -- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 msgr2=0x7f5ef4074800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4074800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef2d9d640 1 -- 172.21.15.33:0/4024899206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef406e4e0 con 0x7f5ef4105f40 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.774+0000 7f5ef259c640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.775+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef4105f40 0x7f5ef406dc20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eec005fd0 tx=0x7f5eec00ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.775+0000 7f5eeb7fe640 1 -- 172.21.15.33:0/4024899206 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec00cd80 con 0x7f5ef4105f40 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.775+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef406e7d0 con 0x7f5ef4105f40 2024-09-10T21:22:15.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.775+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef4073750 con 0x7f5ef4105f40 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.776+0000 7f5eeb7fe640 1 -- 172.21.15.33:0/4024899206 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5eec005020 con 0x7f5ef4105f40 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.776+0000 7f5eeb7fe640 1 -- 172.21.15.33:0/4024899206 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eec010690 con 0x7f5ef4105f40 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.779+0000 7f5efaf96740 1 Processor -- start 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5eeb7fe640 1 -- 172.21.15.33:0/4024899206 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5eec004a40 con 0x7f5ef4105f40 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5efaf96740 1 -- start start 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0de460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0de9a0 0x56111d0e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5efaf96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56111d048ea0 con 0x56111d0d20f0 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5efaf96740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56111d0382d0 con 0x56111d0de9a0 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5efaf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56111d03c0e0 con 0x56111d0d2a10 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.780+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0d24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37148/0 (socket says 172.21.15.33:37148) 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 -- 172.21.15.33:0/105647917 learned_addr learned my addr 172.21.15.33:0/105647917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5ef259c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0de460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0de9a0 0x56111d0e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 -- 172.21.15.33:0/105647917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 msgr2=0x56111d0de460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0de460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 -- 172.21.15.33:0/105647917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0de9a0 msgr2=0x56111d0e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0de9a0 0x56111d0e0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 -- 172.21.15.33:0/105647917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111cd96650 con 0x56111d0d20f0 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eeb7fe640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ee00b5360 0x7f5ee00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5ef259c640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ee00b5360 0x7f5ee00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eebfff640 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0d24d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0147f0 tx=0x7f5edc009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75173bcb3c522bad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5eeb7fe640 1 -- 172.21.15.33:0/4024899206 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5eec0e60a0 con 0x7f5ef4105f40 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5ee8ff9640 1 -- 172.21.15.33:0/105647917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc005220 con 0x56111d0d20f0 2024-09-10T21:22:15.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5ee8ff9640 1 -- 172.21.15.33:0/105647917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5edc004c50 con 0x56111d0d20f0 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.781+0000 7f5ee8ff9640 1 -- 172.21.15.33:0/105647917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc00d970 con 0x56111d0d20f0 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 -- 172.21.15.33:0/105647917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 msgr2=0x56111d0d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5ef259c640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ee00b5360 0x7f5ee00b7820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4031440 tx=0x7f5ee403c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0d24d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0147f0 tx=0x7f5edc009380 comp rx=0 tx=0).stop 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 -- 172.21.15.33:0/105647917 shutdown_connections 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0de9a0 0x56111d0e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0de460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 --2- 172.21.15.33:0/105647917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0d24d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 -- 172.21.15.33:0/105647917 >> 172.21.15.33:0/105647917 conn(0x56111d0cff20 msgr2=0x56111d0d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 -- 172.21.15.33:0/105647917 shutdown_connections 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.782+0000 7f5efaf96740 1 -- 172.21.15.33:0/105647917 wait complete. 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 -- start start 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 Processor -- start 2024-09-10T21:22:15.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d20f0 0x56111d0de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56111ce99700 con 0x56111d0de9a0 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56111cfd3160 con 0x56111d0d2a10 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56111d0e1620 con 0x56111d0d20f0 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5efaf96740 10 client.0 _ll_register_callbacks cb 0x56111cfec9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5eebfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0db6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39498/0 (socket says 172.21.15.33:39498) 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5eebfff640 1 -- 172.21.15.33:0/1306558713 learned_addr learned my addr 172.21.15.33:0/1306558713 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.783+0000 7f5ef259c640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d20f0 0x56111d0de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5eebfff640 1 -- 172.21.15.33:0/1306558713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d20f0 msgr2=0x56111d0de260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5eebfff640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d20f0 0x56111d0de260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5eebfff640 1 -- 172.21.15.33:0/1306558713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 msgr2=0x56111d0dbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5eebfff640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0dbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5eebfff640 1 -- 172.21.15.33:0/1306558713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111d000c90 con 0x56111d0d2a10 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5eebfff640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0db6b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5edc004c20 tx=0x7f5edc005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d455d83d4e7b6492 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/1306558713 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc005e50 con 0x56111d0d2a10 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/1306558713 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5edc00d6f0 con 0x56111d0d2a10 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5edc004c50 con 0x56111d0d2a10 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/1306558713 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5edc0144b0 con 0x56111d0d2a10 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5efaf96740 10 client.28112 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.784+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x56111d0ecf80 con 0x56111d0d2a10 2024-09-10T21:22:15.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/1306558713 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5edc004c50 con 0x56111d0d2a10 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/1306558713 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f5edc00c070 con 0x56111d0d2a10 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5efaf96740 10 client.28112 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5efaf96740 10 client.28112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5efaf96740 20 client.28112 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d0f0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5efaf96740 4 client.28112 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.785+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56111d04adf0 con 0x56111d0ee320 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.786+0000 7f5ef259c640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d0f0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.786+0000 7f5ef259c640 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d0f0710 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbf16904fc894c21 server_cookie=e2a6bfea586bf243 in_seq=0 out_seq=0 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.786+0000 7f5ed9ffb640 10 client.28112 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/1306558713 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56111d04adf0 con 0x56111d0ee320 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5ed9ffb640 10 client.28112 handle_command_reply: tid=0 2024-09-10T21:22:15.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 client.28112 shutdown 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 msgr2=0x56111d0db6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0db6b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5edc004c20 tx=0x7f5edc005250 comp rx=0 tx=0).stop 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 shutdown_connections 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d0f0710 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5ed9ffb640 0 client.28112 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0db6b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1306558713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d20f0 0x56111d0de260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 >> 172.21.15.33:0/1306558713 conn(0x56111d0cff20 msgr2=0x56111d0d5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 shutdown_connections 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 1 -- 172.21.15.33:0/1306558713 wait complete. 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.789+0000 7f5efaf96740 20 client.28112 trim_cache size 0 max 16384 2024-09-10T21:22:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 Processor -- start 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 -- start start 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0ea8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0de9a0 0x56111d0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed400e670 con 0x56111d0d20f0 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed40101b0 con 0x56111d0d2a10 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5efaf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed400ed40 con 0x56111d0de9a0 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0ea8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0ea8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37170/0 (socket says 172.21.15.33:37170) 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 -- 172.21.15.33:0/1556106260 learned_addr learned my addr 172.21.15.33:0/1556106260 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0de9a0 0x56111d0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5eebfff640 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 -- 172.21.15.33:0/1556106260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0de9a0 msgr2=0x56111d0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0de9a0 0x56111d0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 -- 172.21.15.33:0/1556106260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 msgr2=0x56111d0d16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0d16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef259c640 1 -- 172.21.15.33:0/1556106260 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56111d0cb5c0 con 0x56111d0d20f0 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5ef2d9d640 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0de9a0 0x56111d0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.804+0000 7f5eebfff640 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5ef259c640 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0ea8b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4000c00 tx=0x7f5ee4042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729cb24dd08b325b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5edbfff640 1 -- 172.21.15.33:0/1556106260 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4035040 con 0x56111d0d20f0 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5edbfff640 1 -- 172.21.15.33:0/1556106260 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee403ad20 con 0x56111d0d20f0 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5edbfff640 1 -- 172.21.15.33:0/1556106260 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4003e40 con 0x56111d0d20f0 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5efaf96740 1 -- 172.21.15.33:0/1556106260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 msgr2=0x56111d0ea8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0ea8b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4000c00 tx=0x7f5ee4042000 comp rx=0 tx=0).stop 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5efaf96740 1 -- 172.21.15.33:0/1556106260 shutdown_connections 2024-09-10T21:22:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0de9a0 0x56111d0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d2a10 0x56111d0d16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5efaf96740 1 --2- 172.21.15.33:0/1556106260 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0d20f0 0x56111d0ea8b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.805+0000 7f5efaf96740 1 -- 172.21.15.33:0/1556106260 >> 172.21.15.33:0/1556106260 conn(0x56111d0cff20 msgr2=0x56111d0d0fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 -- 172.21.15.33:0/1556106260 shutdown_connections 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 -- 172.21.15.33:0/1556106260 wait complete. 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 -- start start 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 Processor -- start 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d20f0 0x56111d0e2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0e27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0e6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed400fae0 con 0x56111d0de9a0 2024-09-10T21:22:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed400f410 con 0x56111d0d20f0 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed400c630 con 0x56111d0d2a10 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5efaf96740 10 client.0 _ll_register_callbacks cb 0x7f5edc009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5ef2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5ef259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d20f0 0x56111d0e2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.806+0000 7f5ef259c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d20f0 0x56111d0e2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39522/0 (socket says 172.21.15.33:39522) 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef259c640 1 -- 172.21.15.33:0/3485212016 learned_addr learned my addr 172.21.15.33:0/3485212016 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5eebfff640 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0e27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef2d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0e6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37178/0 (socket says 172.21.15.33:37178) 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef259c640 1 -- 172.21.15.33:0/3485212016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 msgr2=0x56111d0e27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef259c640 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0e27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef259c640 1 -- 172.21.15.33:0/3485212016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 msgr2=0x56111d0e6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef259c640 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0e6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef259c640 1 -- 172.21.15.33:0/3485212016 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111d0e6e60 con 0x56111d0d20f0 2024-09-10T21:22:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ef259c640 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d20f0 0x56111d0e2230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40096c0 tx=0x7f5ee4005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148e9d766bcd7e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/3485212016 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4067060 con 0x56111d0d20f0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ee403ad20 con 0x56111d0d20f0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.807+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/3485212016 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5ee40035f0 con 0x56111d0d20f0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.808+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/3485212016 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5ee4035030 con 0x56111d0d20f0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.810+0000 7f5efaf96740 10 client.28118 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.810+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x56111d15aad0 con 0x56111d0d20f0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.810+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/3485212016 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee403ad20 con 0x56111d0d20f0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.811+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/3485212016 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f5ee4058070 con 0x56111d0d20f0 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.811+0000 7f5efaf96740 10 client.28118 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.811+0000 7f5efaf96740 10 client.28118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.811+0000 7f5efaf96740 20 client.28118 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.811+0000 7f5efaf96740 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.811+0000 7f5efaf96740 4 client.28118 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.811+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56111d0468a0 con 0x56111d0ee320 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.813+0000 7f5eebfff640 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.815+0000 7f5eebfff640 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d15d2a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c346bb30a43f4650 server_cookie=d7c4686860371585 in_seq=0 out_seq=0 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.815+0000 7f5ed9ffb640 10 client.28118 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5ed9ffb640 1 -- 172.21.15.33:0/3485212016 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56111d0468a0 con 0x56111d0ee320 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5ed9ffb640 10 client.28118 handle_command_reply: tid=0 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 client.28118 shutdown 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d20f0 msgr2=0x56111d0e2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d20f0 0x56111d0e2230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f5ee40096c0 tx=0x7f5ee4005e50 comp rx=0 tx=0).stop 2024-09-10T21:22:15.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 shutdown_connections 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56111d0ee320 0x56111d15d2a0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56111d0de9a0 0x56111d0e6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56111d0d2a10 0x56111d0e27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 --2- 172.21.15.33:0/3485212016 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56111d0d20f0 0x56111d0e2230 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 >> 172.21.15.33:0/3485212016 conn(0x56111d0cff20 msgr2=0x56111d0dcce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5ed9ffb640 0 client.28118 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 shutdown_connections 2024-09-10T21:22:15.214 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 1 -- 172.21.15.33:0/3485212016 wait complete. 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.817+0000 7f5efaf96740 20 client.28118 trim_cache size 0 max 16384 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ee00b5360 msgr2=0x7f5ee00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ee00b5360 0x7f5ee00b7820 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4031440 tx=0x7f5ee403c040 comp rx=0 tx=0).stop 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef4105f40 msgr2=0x7f5ef406dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef4105f40 0x7f5ef406dc20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5eec005fd0 tx=0x7f5eec00ee60 comp rx=0 tx=0).stop 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 shutdown_connections 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5ee00b5360 0x7f5ee00b7820 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ef4105f40 0x7f5ef406dc20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ef41055a0 0x7f5ef4074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 --2- 172.21.15.33:0/4024899206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ef4101250 0x7f5ef4074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 >> 172.21.15.33:0/4024899206 conn(0x7f5ef4078140 msgr2=0x7f5ef40ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:15.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 shutdown_connections 2024-09-10T21:22:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:14.818+0000 7f5ef8bc1640 1 -- 172.21.15.33:0/4024899206 wait complete. 2024-09-10T21:22:15.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:15 smithi104 ceph-mon[35325]: pgmap v894: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 931 op/s 2024-09-10T21:22:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:15 smithi174 ceph-mon[35635]: pgmap v894: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 931 op/s 2024-09-10T21:22:15.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:15 smithi033 ceph-mon[30860]: pgmap v894: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 931 op/s 2024-09-10T21:22:16.062 INFO:tasks.workunit.client.0.smithi033.stdout: 10 254389 41.58 MB/sec execute 196 sec latency 303.302 ms 2024-09-10T21:22:17.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 255145 41.58 MB/sec execute 197 sec latency 96.037 ms 2024-09-10T21:22:17.108 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:22:17.108 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:22:17.421 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:17.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:17 smithi104 ceph-mon[35325]: pgmap v895: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-10T21:22:17.851 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:17 smithi033 ceph-mon[30860]: pgmap v895: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-10T21:22:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:17 smithi174 ceph-mon[35635]: pgmap v895: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-09-10T21:22:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 -- 172.21.15.33:0/1951733648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 msgr2=0x7fc5d0104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/1951733648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d0104080 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0009900 tx=0x7fc5c002a410 comp rx=0 tx=0).stop 2024-09-10T21:22:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 -- 172.21.15.33:0/1951733648 shutdown_connections 2024-09-10T21:22:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/1951733648 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5d010d100 0x7fc5d010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/1951733648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5d01045c0 0x7fc5d010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/1951733648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d0104080 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 -- 172.21.15.33:0/1951733648 >> 172.21.15.33:0/1951733648 conn(0x7fc5d00fd700 msgr2=0x7fc5d00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 -- 172.21.15.33:0/1951733648 shutdown_connections 2024-09-10T21:22:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.956+0000 7fc5d5f65640 1 -- 172.21.15.33:0/1951733648 wait complete. 2024-09-10T21:22:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.957+0000 7fc5d5f65640 1 Processor -- start 2024-09-10T21:22:17.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.957+0000 7fc5d5f65640 1 -- start start 2024-09-10T21:22:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.957+0000 7fc5d5f65640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.957+0000 7fc5d5f65640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5d01045c0 0x7fc5d019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5d5f65640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5d010d100 0x7fc5d01a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5d5f65640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0112dc0 con 0x7fc5d0103ca0 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5d5f65640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0112c40 con 0x7fc5d01045c0 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5d5f65640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d0112f40 con 0x7fc5d010d100 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37192/0 (socket says 172.21.15.33:37192) 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 -- 172.21.15.33:0/2963257605 learned_addr learned my addr 172.21.15.33:0/2963257605 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cffff640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5d010d100 0x7fc5d01a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5d01045c0 0x7fc5d019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 -- 172.21.15.33:0/2963257605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5d010d100 msgr2=0x7fc5d01a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5d010d100 0x7fc5d01a0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 -- 172.21.15.33:0/2963257605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5d01045c0 msgr2=0x7fc5d019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5d01045c0 0x7fc5d019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 -- 172.21.15.33:0/2963257605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d01a1520 con 0x7fc5d0103ca0 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.958+0000 7fc5cf7fe640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d019b470 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0005ab0 tx=0x7fc5c0003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.959+0000 7fc5ccff9640 1 -- 172.21.15.33:0/2963257605 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c003f070 con 0x7fc5d0103ca0 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.959+0000 7fc5ccff9640 1 -- 172.21.15.33:0/2963257605 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c0031d60 con 0x7fc5d0103ca0 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.959+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d01a17b0 con 0x7fc5d0103ca0 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.959+0000 7fc5ccff9640 1 -- 172.21.15.33:0/2963257605 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c0035030 con 0x7fc5d0103ca0 2024-09-10T21:22:17.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.959+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d01a7ed0 con 0x7fc5d0103ca0 2024-09-10T21:22:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 Processor -- start 2024-09-10T21:22:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 -- start start 2024-09-10T21:22:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d710ee0 0x55925d7112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d71d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55925d687e60 con 0x55925d711800 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55925d677290 con 0x55925d710ee0 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5d833a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55925d67b0a0 con 0x55925d71d790 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d710ee0 0x55925d7112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d710ee0 0x55925d7112c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:39552/0 (socket says 172.21.15.33:39552) 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5cffff640 1 -- 172.21.15.33:0/495916816 learned_addr learned my addr 172.21.15.33:0/495916816 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5ccff9640 1 -- 172.21.15.33:0/2963257605 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc5c004c060 con 0x7fc5d0103ca0 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.962+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5cf7fe640 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d71d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5cffff640 1 -- 172.21.15.33:0/495916816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 msgr2=0x55925d71fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:17.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5cffff640 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5cffff640 1 -- 172.21.15.33:0/495916816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 msgr2=0x55925d71d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:17.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5cffff640 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d71d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5cffff640 1 -- 172.21.15.33:0/495916816 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925d3d5930 con 0x55925d710ee0 2024-09-10T21:22:17.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5cffff640 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d710ee0 0x55925d7112c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009fd0 tx=0x7fc5c80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d03a7a5e1d99cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5b5ffb640 1 -- 172.21.15.33:0/495916816 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004d20 con 0x55925d710ee0 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5b5ffb640 1 -- 172.21.15.33:0/495916816 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8004050 con 0x55925d710ee0 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.963+0000 7fc5b5ffb640 1 -- 172.21.15.33:0/495916816 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8005280 con 0x55925d710ee0 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.964+0000 7fc5ccff9640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5b80b5360 0x7fc5b80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.964+0000 7fc5ccff9640 1 -- 172.21.15.33:0/2963257605 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc5c0105fa0 con 0x7fc5d0103ca0 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.964+0000 7fc5cf7fe640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5b80b5360 0x7fc5b80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5cf7fe640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5b80b5360 0x7fc5b80b7820 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00c8dd0 tx=0x7fc5c003b460 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 -- 172.21.15.33:0/495916816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d710ee0 msgr2=0x55925d7112c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:17.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d710ee0 0x55925d7112c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009fd0 tx=0x7fc5c80118d0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 -- 172.21.15.33:0/495916816 shutdown_connections 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d71d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 --2- 172.21.15.33:0/495916816 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d710ee0 0x55925d7112c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 -- 172.21.15.33:0/495916816 >> 172.21.15.33:0/495916816 conn(0x55925d70ed10 msgr2=0x55925d70feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 -- 172.21.15.33:0/495916816 shutdown_connections 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.965+0000 7fc5d833a740 1 -- 172.21.15.33:0/495916816 wait complete. 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 -- start start 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 Processor -- start 2024-09-10T21:22:17.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d711800 0x55925d71ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55925d4d8890 con 0x55925d710ee0 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55925d611d40 con 0x55925d711800 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55925d720440 con 0x55925d71d790 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.966+0000 7fc5d833a740 10 client.0 _ll_register_callbacks cb 0x55925d675c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37206/0 (socket says 172.21.15.33:37206) 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 -- 172.21.15.33:0/3972749438 learned_addr learned my addr 172.21.15.33:0/3972749438 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 -- 172.21.15.33:0/3972749438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 msgr2=0x55925d71b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 -- 172.21.15.33:0/3972749438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d711800 msgr2=0x55925d71ad30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:22:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d711800 0x55925d71ad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 -- 172.21.15.33:0/3972749438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925d732110 con 0x55925d710ee0 2024-09-10T21:22:17.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5cffff640 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71a7f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8004920 tx=0x7fc5c8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6b4101f74887f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:17.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.967+0000 7fc5aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:17.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5aeffd640 1 -- 172.21.15.33:0/3972749438 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004c50 con 0x55925d710ee0 2024-09-10T21:22:17.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c8004050 con 0x55925d710ee0 2024-09-10T21:22:17.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5aeffd640 1 -- 172.21.15.33:0/3972749438 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc5c8004df0 con 0x55925d710ee0 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5aeffd640 1 -- 172.21.15.33:0/3972749438 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc5c8005040 con 0x55925d710ee0 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5d833a740 10 client.18747 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55925d734aa0 con 0x55925d710ee0 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5aeffd640 1 -- 172.21.15.33:0/3972749438 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c8004050 con 0x55925d710ee0 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5aeffd640 1 -- 172.21.15.33:0/3972749438 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fc5c800f6e0 con 0x55925d710ee0 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5d833a740 10 client.18747 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5d833a740 10 client.18747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.968+0000 7fc5d833a740 20 client.18747 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.969+0000 7fc5d833a740 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d735eb0 0x55925d7382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.969+0000 7fc5d833a740 4 client.18747 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.969+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55925d689db0 con 0x55925d735eb0 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.971+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d735eb0 0x55925d7382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:17.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.971+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d735eb0 0x55925d7382a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cd87e227f29246 server_cookie=b4f5834a7129d9d8 in_seq=0 out_seq=0 2024-09-10T21:22:17.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.971+0000 7fc5aeffd640 10 client.18747 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.974+0000 7fc5aeffd640 1 -- 172.21.15.33:0/3972749438 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55925d689db0 con 0x55925d735eb0 2024-09-10T21:22:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.974+0000 7fc5aeffd640 10 client.18747 handle_command_reply: tid=0 2024-09-10T21:22:17.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.974+0000 7fc5d833a740 1 client.18747 shutdown 2024-09-10T21:22:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 msgr2=0x55925d71a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71a7f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8004920 tx=0x7fc5c8000f40 comp rx=0 tx=0).stop 2024-09-10T21:22:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 shutdown_connections 2024-09-10T21:22:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d735eb0 0x55925d7382a0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d71d790 0x55925d71b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d711800 0x55925d71ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 --2- 172.21.15.33:0/3972749438 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71a7f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 >> 172.21.15.33:0/3972749438 conn(0x55925d70ed10 msgr2=0x55925d70df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5aeffd640 0 client.18747 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 shutdown_connections 2024-09-10T21:22:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 1 -- 172.21.15.33:0/3972749438 wait complete. 2024-09-10T21:22:17.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.975+0000 7fc5d833a740 20 client.18747 trim_cache size 0 max 16384 2024-09-10T21:22:18.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 Processor -- start 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 -- start start 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d711800 0x55925d717a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d717f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01ac1c0 con 0x55925d710ee0 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01add00 con 0x55925d71d790 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.996+0000 7fc5d833a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01ac890 con 0x55925d711800 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37218/0 (socket says 172.21.15.33:37218) 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 -- 172.21.15.33:0/1741372858 learned_addr learned my addr 172.21.15.33:0/1741372858 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d711800 0x55925d717a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:18.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d717f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 -- 172.21.15.33:0/1741372858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d711800 msgr2=0x55925d717a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d711800 0x55925d717a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 -- 172.21.15.33:0/1741372858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 msgr2=0x55925d717f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d717f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 -- 172.21.15.33:0/1741372858 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55925d70eb40 con 0x55925d710ee0 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d711800 0x55925d717a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71aed0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009bb0 tx=0x7fc5c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb21e8fb92096c0d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5b57fa640 1 -- 172.21.15.33:0/1741372858 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004e60 con 0x55925d710ee0 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.997+0000 7fc5b57fa640 1 -- 172.21.15.33:0/1741372858 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c800f4e0 con 0x55925d710ee0 2024-09-10T21:22:18.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5b57fa640 1 -- 172.21.15.33:0/1741372858 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80057b0 con 0x55925d710ee0 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5d833a740 1 -- 172.21.15.33:0/1741372858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 msgr2=0x55925d71aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71aed0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009bb0 tx=0x7fc5c80118d0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5d833a740 1 -- 172.21.15.33:0/1741372858 shutdown_connections 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d717f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d711800 0x55925d717a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1741372858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d710ee0 0x55925d71aed0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.998+0000 7fc5d833a740 1 -- 172.21.15.33:0/1741372858 >> 172.21.15.33:0/1741372858 conn(0x55925d70ed10 msgr2=0x55925d71a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 -- 172.21.15.33:0/1741372858 shutdown_connections 2024-09-10T21:22:18.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 -- 172.21.15.33:0/1741372858 wait complete. 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 -- start start 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 Processor -- start 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d710ee0 0x55925d72a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d72a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d7a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01ad630 con 0x55925d711800 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01acf60 con 0x55925d71d790 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d01aa1c0 con 0x55925d710ee0 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:17.999+0000 7fc5d833a740 10 client.0 _ll_register_callbacks cb 0x7fc5d006c4d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d72a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d72a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37230/0 (socket says 172.21.15.33:37230) 2024-09-10T21:22:18.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 -- 172.21.15.33:0/1926601881 learned_addr learned my addr 172.21.15.33:0/1926601881 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d710ee0 0x55925d72a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d7a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 -- 172.21.15.33:0/1926601881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d710ee0 msgr2=0x55925d72a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d710ee0 0x55925d72a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 -- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 msgr2=0x55925d7a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d7a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 -- 172.21.15.33:0/1926601881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55925d7a21e0 con 0x55925d711800 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5ceffd640 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d72a850 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4000c00 tx=0x7fc5c4013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcdef3a32f28919 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5aeffd640 1 -- 172.21.15.33:0/1926601881 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4024440 con 0x55925d711800 2024-09-10T21:22:18.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5aeffd640 1 -- 172.21.15.33:0/1926601881 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc5c4011440 con 0x55925d711800 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.000+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5c800f4e0 con 0x55925d711800 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5aeffd640 1 -- 172.21.15.33:0/1926601881 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc5c4011640 con 0x55925d711800 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 10 client.18753 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55925d7a3480 con 0x55925d711800 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5aeffd640 1 -- 172.21.15.33:0/1926601881 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c800f4e0 con 0x55925d711800 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5aeffd640 1 -- 172.21.15.33:0/1926601881 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fc5c4022070 con 0x55925d711800 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 10 client.18753 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 10 client.18753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 20 client.18753 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d7a3c30 0x55925d7a6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 4 client.18753 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:22:18.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.001+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55925d685860 con 0x55925d7a3c30 2024-09-10T21:22:18.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.005+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d7a3c30 0x55925d7a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:18.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.005+0000 7fc5cffff640 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d7a3c30 0x55925d7a6050 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=841d488bac98ade2 server_cookie=81f90285fe479d99 in_seq=0 out_seq=0 2024-09-10T21:22:18.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.005+0000 7fc5aeffd640 10 client.18753 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5aeffd640 1 -- 172.21.15.33:0/1926601881 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55925d685860 con 0x55925d7a3c30 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5aeffd640 10 client.18753 handle_command_reply: tid=0 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 client.18753 shutdown 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 msgr2=0x55925d72a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d72a850 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4000c00 tx=0x7fc5c4013830 comp rx=0 tx=0).stop 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 shutdown_connections 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55925d7a3c30 0x55925d7a6050 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55925d71d790 0x55925d7a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55925d711800 0x55925d72a850 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5aeffd640 0 client.18753 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 --2- 172.21.15.33:0/1926601881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55925d710ee0 0x55925d72a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 >> 172.21.15.33:0/1926601881 conn(0x55925d70ed10 msgr2=0x55925d71c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 shutdown_connections 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 1 -- 172.21.15.33:0/1926601881 wait complete. 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.008+0000 7fc5d833a740 20 client.18753 trim_cache size 0 max 16384 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:22:18.013 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "13d23824-38f6-4822-b207-44e9d8d4829e", 2024-09-10T21:22:18.014 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:22:18.014 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:22:18.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5b80b5360 msgr2=0x7fc5b80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5b80b5360 0x7fc5b80b7820 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00c8dd0 tx=0x7fc5c003b460 comp rx=0 tx=0).stop 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 msgr2=0x7fc5d019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d019b470 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fc5c0005ab0 tx=0x7fc5c0003c80 comp rx=0 tx=0).stop 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 shutdown_connections 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5b80b5360 0x7fc5b80b7820 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5d010d100 0x7fc5d01a0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5d01045c0 0x7fc5d019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 --2- 172.21.15.33:0/2963257605 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5d0103ca0 0x7fc5d019b470 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 >> 172.21.15.33:0/2963257605 conn(0x7fc5d00fd700 msgr2=0x7fc5d010f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 shutdown_connections 2024-09-10T21:22:18.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:18.009+0000 7fc5d5f65640 1 -- 172.21.15.33:0/2963257605 wait complete. 2024-09-10T21:22:18.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 255939 41.60 MB/sec execute 198 sec latency 78.970 ms 2024-09-10T21:22:18.584 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:18 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:22:18.585 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:18 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:18.585 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:18 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:22:18.585 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:18 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:22:18.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:18 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:22:18.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:18 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:18.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:18 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:22:18.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:18 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:22:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:18 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:22:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:18 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:18 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:22:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:18 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:22:18.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:22:19.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 256277 41.50 MB/sec execute 199 sec latency 564.394 ms 2024-09-10T21:22:19.235 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:19.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:22:19.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:22:19.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:22:19.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.663 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:22:19.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: pgmap v896: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 856 op/s 2024-09-10T21:22:19.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.664 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.768+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/2076883815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0104560 msgr2=0x7f9bd010dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.768+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/2076883815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0104560 0x7f9bd010dfa0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009be0 tx=0x7f9bc802a6c0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.768+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/2076883815 shutdown_connections 2024-09-10T21:22:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.768+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/2076883815 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0104560 0x7f9bd010dfa0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.768+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/2076883815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd0103be0 0x7f9bd0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.768+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/2076883815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bd0101720 0x7f9bd0101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.768+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/2076883815 >> 172.21.15.33:0/2076883815 conn(0x7f9bd00fca20 msgr2=0x7f9bd00fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:19.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.771+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/2076883815 shutdown_connections 2024-09-10T21:22:19.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.771+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/2076883815 wait complete. 2024-09-10T21:22:19.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.772+0000 7f9bd7fe5640 1 Processor -- start 2024-09-10T21:22:19.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.772+0000 7f9bd7fe5640 1 -- start start 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.773+0000 7f9bd7fe5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0101720 0x7f9bd00748c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.773+0000 7f9bd7fe5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd0103be0 0x7f9bd0074e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.773+0000 7f9bd7fe5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bd0104560 0x7f9bd006d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.773+0000 7f9bd7fe5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0111c60 con 0x7f9bd0101720 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.773+0000 7f9bd7fe5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0111ae0 con 0x7f9bd0103be0 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.773+0000 7f9bd7fe5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd0111de0 con 0x7f9bd0104560 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.773+0000 7f9bd5d81640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0101720 0x7f9bd00748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd6582640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bd0104560 0x7f9bd006d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5d81640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0101720 0x7f9bd00748c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54818/0 (socket says 172.21.15.33:54818) 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5d81640 1 -- 172.21.15.33:0/1371536747 learned_addr learned my addr 172.21.15.33:0/1371536747 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:19.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5580640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd0103be0 0x7f9bd0074e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5d81640 1 -- 172.21.15.33:0/1371536747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bd0104560 msgr2=0x7f9bd006d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bd0104560 0x7f9bd006d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5d81640 1 -- 172.21.15.33:0/1371536747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd0103be0 msgr2=0x7f9bd0074e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd0103be0 0x7f9bd0074e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.774+0000 7f9bd5d81640 1 -- 172.21.15.33:0/1371536747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd006e120 con 0x7f9bd0101720 2024-09-10T21:22:19.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.775+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0101720 0x7f9bd00748c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00e9c0 tx=0x7f9bcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:19.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.775+0000 7f9bc6ffd640 1 -- 172.21.15.33:0/1371536747 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc00cde0 con 0x7f9bd0101720 2024-09-10T21:22:19.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.776+0000 7f9bc6ffd640 1 -- 172.21.15.33:0/1371536747 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc0035f0 con 0x7f9bd0101720 2024-09-10T21:22:19.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.776+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd006e410 con 0x7f9bd0101720 2024-09-10T21:22:19.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.776+0000 7f9bc6ffd640 1 -- 172.21.15.33:0/1371536747 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc010690 con 0x7f9bd0101720 2024-09-10T21:22:19.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.776+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd01b4b20 con 0x7f9bd0101720 2024-09-10T21:22:19.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.779+0000 7f9bda3ba740 1 Processor -- start 2024-09-10T21:22:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.780+0000 7f9bda3ba740 1 -- start start 2024-09-10T21:22:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.780+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54480 0x556505c54860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.780+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c607f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.780+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c60d30 0x556505c63120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.780+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556505bcae50 con 0x556505c60d30 2024-09-10T21:22:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.780+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556505bba280 con 0x556505c54da0 2024-09-10T21:22:19.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.780+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556505bbe090 con 0x556505c54480 2024-09-10T21:22:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.781+0000 7f9bd5580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c60d30 0x556505c63120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.781+0000 7f9bd5580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c60d30 0x556505c63120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54822/0 (socket says 172.21.15.33:54822) 2024-09-10T21:22:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.781+0000 7f9bd5580640 1 -- 172.21.15.33:0/2017177238 learned_addr learned my addr 172.21.15.33:0/2017177238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:19.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.781+0000 7f9bd6582640 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54480 0x556505c54860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.782+0000 7f9bd5580640 1 -- 172.21.15.33:0/2017177238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54480 msgr2=0x556505c54860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:19.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.782+0000 7f9bd5580640 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54480 0x556505c54860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.782+0000 7f9bd5580640 1 -- 172.21.15.33:0/2017177238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 msgr2=0x556505c607f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:22:19.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.782+0000 7f9bd5580640 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c607f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.782+0000 7f9bd5580640 1 -- 172.21.15.33:0/2017177238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556505918650 con 0x556505c60d30 2024-09-10T21:22:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.783+0000 7f9bd5580640 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c60d30 0x556505c63120 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009c20 tx=0x7f9bc00117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d64678e00055180 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.783+0000 7f9bbbfff640 1 -- 172.21.15.33:0/2017177238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0012450 con 0x556505c60d30 2024-09-10T21:22:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.783+0000 7f9bc6ffd640 1 -- 172.21.15.33:0/1371536747 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9bcc004000 con 0x7f9bd0101720 2024-09-10T21:22:19.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.784+0000 7f9bbbfff640 1 -- 172.21.15.33:0/2017177238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc0012aa0 con 0x556505c60d30 2024-09-10T21:22:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.784+0000 7f9bc6ffd640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9bbc0b53e0 0x7f9bbc0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bda3ba740 1 -- 172.21.15.33:0/2017177238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c60d30 msgr2=0x556505c63120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c60d30 0x556505c63120 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009c20 tx=0x7f9bc00117b0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bd6582640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9bbc0b53e0 0x7f9bbc0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bda3ba740 1 -- 172.21.15.33:0/2017177238 shutdown_connections 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c60d30 0x556505c63120 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c607f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/2017177238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54480 0x556505c54860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bda3ba740 1 -- 172.21.15.33:0/2017177238 >> 172.21.15.33:0/2017177238 conn(0x556505c522b0 msgr2=0x556505c53450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.785+0000 7f9bc6ffd640 1 -- 172.21.15.33:0/1371536747 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9bcc0e6020 con 0x7f9bd0101720 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.786+0000 7f9bda3ba740 1 -- 172.21.15.33:0/2017177238 shutdown_connections 2024-09-10T21:22:19.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.786+0000 7f9bd6582640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9bbc0b53e0 0x7f9bbc0b78a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8006010 tx=0x7f9bc8004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.786+0000 7f9bda3ba740 1 -- 172.21.15.33:0/2017177238 wait complete. 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 -- start start 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 Processor -- start 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c60720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54da0 0x556505c5db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c60d30 0x556505c5e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556505a1b700 con 0x556505c54480 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556505b55110 con 0x556505c60d30 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556505c74a10 con 0x556505c54da0 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.787+0000 7f9bda3ba740 10 client.0 _ll_register_callbacks cb 0x556505b6e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:19.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.788+0000 7f9bd6582640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c60d30 0x556505c5e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.788+0000 7f9bd6582640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c60d30 0x556505c5e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:42452/0 (socket says 172.21.15.33:42452) 2024-09-10T21:22:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.788+0000 7f9bd6582640 1 -- 172.21.15.33:0/3337704921 learned_addr learned my addr 172.21.15.33:0/3337704921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.789+0000 7f9bd6582640 1 -- 172.21.15.33:0/3337704921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54da0 msgr2=0x556505c5db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.789+0000 7f9bd6582640 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54da0 0x556505c5db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.789+0000 7f9bd6582640 1 -- 172.21.15.33:0/3337704921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 msgr2=0x556505c60720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.789+0000 7f9bd6582640 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c60720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.789+0000 7f9bd6582640 1 -- 172.21.15.33:0/3337704921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556505b82c40 con 0x556505c60d30 2024-09-10T21:22:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.789+0000 7f9bd6582640 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c60d30 0x556505c5e300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009be0 tx=0x7f9bc802a6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86be03992ec6aafb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.790+0000 7f9bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.790+0000 7f9bb97fa640 1 -- 172.21.15.33:0/3337704921 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc804f020 con 0x556505c60d30 2024-09-10T21:22:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.790+0000 7f9bb97fa640 1 -- 172.21.15.33:0/3337704921 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9bc8030b70 con 0x556505c60d30 2024-09-10T21:22:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.791+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556505bccda0 con 0x556505c60d30 2024-09-10T21:22:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.792+0000 7f9bb97fa640 1 -- 172.21.15.33:0/3337704921 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f9bc8036040 con 0x556505c60d30 2024-09-10T21:22:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.792+0000 7f9bda3ba740 10 client.28142 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.792+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x556505c805a0 con 0x556505c60d30 2024-09-10T21:22:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.794+0000 7f9bb97fa640 1 -- 172.21.15.33:0/3337704921 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556505bccda0 con 0x556505c60d30 2024-09-10T21:22:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.794+0000 7f9bb97fa640 1 -- 172.21.15.33:0/3337704921 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f9bc804b480 con 0x556505c60d30 2024-09-10T21:22:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.795+0000 7f9bda3ba740 10 client.28142 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.795+0000 7f9bda3ba740 10 client.28142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.795+0000 7f9bda3ba740 20 client.28142 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.795+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505c81a50 0x556505c83e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.795+0000 7f9bda3ba740 4 client.28142 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:19.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.795+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556505bc8850 con 0x556505c81a50 2024-09-10T21:22:19.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.796+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505c81a50 0x556505c83e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.799+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505c81a50 0x556505c83e60 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c7103e3ada93b25 server_cookie=7c474bf6687ce07a in_seq=0 out_seq=0 2024-09-10T21:22:19.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.799+0000 7f9bb97fa640 10 client.28142 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:19.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.800+0000 7f9bb97fa640 1 -- 172.21.15.33:0/3337704921 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556505bc8850 con 0x556505c81a50 2024-09-10T21:22:19.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.800+0000 7f9bb97fa640 10 client.28142 handle_command_reply: tid=0 2024-09-10T21:22:19.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.800+0000 7f9bda3ba740 1 client.28142 shutdown 2024-09-10T21:22:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c60d30 msgr2=0x556505c5e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c60d30 0x556505c5e300 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009be0 tx=0x7f9bc802a6c0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 shutdown_connections 2024-09-10T21:22:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505c81a50 0x556505c83e60 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c60d30 0x556505c5e300 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c54da0 0x556505c5db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/3337704921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c60720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bb97fa640 0 client.28142 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:19.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.801+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 >> 172.21.15.33:0/3337704921 conn(0x556505c522b0 msgr2=0x556505c57e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.802+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 shutdown_connections 2024-09-10T21:22:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.802+0000 7f9bda3ba740 1 -- 172.21.15.33:0/3337704921 wait complete. 2024-09-10T21:22:19.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.802+0000 7f9bda3ba740 20 client.28142 trim_cache size 0 max 16384 2024-09-10T21:22:19.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:22:19.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:22:19.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:22:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:22:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: pgmap v896: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 856 op/s 2024-09-10T21:22:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.812 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.828+0000 7f9bda3ba740 1 Processor -- start 2024-09-10T21:22:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.828+0000 7f9bda3ba740 1 -- start start 2024-09-10T21:22:19.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54480 0x556505c4d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54da0 0x556505c5c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c5c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01b06d0 con 0x556505c54da0 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01b65e0 con 0x556505c54480 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01b5170 con 0x556505c60d30 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bd5580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54da0 0x556505c5c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bd5580640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54da0 0x556505c5c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54848/0 (socket says 172.21.15.33:54848) 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bd5580640 1 -- 172.21.15.33:0/385066048 learned_addr learned my addr 172.21.15.33:0/385066048 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bd6582640 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c5c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.829+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54480 0x556505c4d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bd5580640 1 -- 172.21.15.33:0/385066048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 msgr2=0x556505c5c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bd5580640 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c5c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bd5580640 1 -- 172.21.15.33:0/385066048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54480 msgr2=0x556505c4d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:19.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bd5580640 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54480 0x556505c4d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bd5580640 1 -- 172.21.15.33:0/385066048 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556505c60a90 con 0x556505c54da0 2024-09-10T21:22:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bd5580640 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54da0 0x556505c5c3f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b70 tx=0x7f9bc0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8c83a305fdaa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bbb7fe640 1 -- 172.21.15.33:0/385066048 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0028020 con 0x556505c54da0 2024-09-10T21:22:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bbb7fe640 1 -- 172.21.15.33:0/385066048 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bc0009650 con 0x556505c54da0 2024-09-10T21:22:19.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.830+0000 7f9bbb7fe640 1 -- 172.21.15.33:0/385066048 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc000fe90 con 0x556505c54da0 2024-09-10T21:22:19.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 -- 172.21.15.33:0/385066048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54da0 msgr2=0x556505c5c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54da0 0x556505c5c3f0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b70 tx=0x7f9bc0005c90 comp rx=0 tx=0).stop 2024-09-10T21:22:19.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 -- 172.21.15.33:0/385066048 shutdown_connections 2024-09-10T21:22:19.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c5c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54da0 0x556505c5c3f0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/385066048 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54480 0x556505c4d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 -- 172.21.15.33:0/385066048 >> 172.21.15.33:0/385066048 conn(0x556505c522b0 msgr2=0x556505c58e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 -- 172.21.15.33:0/385066048 shutdown_connections 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 -- 172.21.15.33:0/385066048 wait complete. 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.831+0000 7f9bda3ba740 1 -- start start 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 1 Processor -- start 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c79d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c7a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c74ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01b5f10 con 0x556505c54480 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01b5840 con 0x556505c54da0 2024-09-10T21:22:19.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd01ae690 con 0x556505c60d30 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bda3ba740 10 client.0 _ll_register_callbacks cb 0x7f9bc8042070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bd6582640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c74ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bd5d81640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c79d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bd6582640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c74ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47006/0 (socket says 172.21.15.33:47006) 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bd5580640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c7a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.832+0000 7f9bd6582640 1 -- 172.21.15.33:0/4195645301 learned_addr learned my addr 172.21.15.33:0/4195645301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bd5d81640 1 -- 172.21.15.33:0/4195645301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 msgr2=0x556505c74ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c74ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bd5d81640 1 -- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 msgr2=0x556505c7a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:19.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c7a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bd5d81640 1 -- 172.21.15.33:0/4195645301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556505c75220 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bd5580640 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c7a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bd5d81640 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c79d30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x556505c53190 tx=0x7f9bcc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f460d5d9b1702bd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bb97fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bb97fa640 1 -- 172.21.15.33:0/4195645301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bcc0d3020 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bb97fa640 1 -- 172.21.15.33:0/4195645301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9bcc0a8a10 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc0009650 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bb97fa640 1 -- 172.21.15.33:0/4195645301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f9bcc0acea0 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bda3ba740 10 client.18768 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x556505cef100 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.833+0000 7f9bb97fa640 1 -- 172.21.15.33:0/4195645301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bcc0cf030 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bb97fa640 1 -- 172.21.15.33:0/4195645301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f9bcc0cfaf0 con 0x556505c54480 2024-09-10T21:22:19.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bda3ba740 10 client.18768 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bda3ba740 10 client.18768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bda3ba740 20 client.18768 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505cef880 0x556505cf1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bda3ba740 4 client.18768 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9bc0009650 con 0x556505cef880 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.834+0000 7f9bd5580640 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505cef880 0x556505cf1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.835+0000 7f9bb97fa640 10 client.18768 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:19.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.835+0000 7f9bd5580640 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505cef880 0x556505cf1ca0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33c36ce677382686 server_cookie=c6180ccf918709c in_seq=0 out_seq=0 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.837+0000 7f9bb97fa640 1 -- 172.21.15.33:0/4195645301 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7102 (crc 0 0 0) 0x7f9bc0009650 con 0x556505cef880 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.837+0000 7f9bb97fa640 10 client.18768 handle_command_reply: tid=0 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.837+0000 7f9bda3ba740 1 client.18768 shutdown 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 msgr2=0x556505c79d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c79d30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x556505c53190 tx=0x7f9bcc00c860 comp rx=0 tx=0).stop 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 shutdown_connections 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x556505cef880 0x556505cf1ca0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556505c60d30 0x556505c74ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556505c54da0 0x556505c7a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 --2- 172.21.15.33:0/4195645301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556505c54480 0x556505c79d30 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 >> 172.21.15.33:0/4195645301 conn(0x556505c522b0 msgr2=0x556505c5f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:19.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bb97fa640 0 client.18768 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 shutdown_connections 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.838+0000 7f9bda3ba740 1 -- 172.21.15.33:0/4195645301 wait complete. 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bda3ba740 20 client.18768 trim_cache size 0 max 16384 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "13d23824-38f6-4822-b207-44e9d8d4829e", 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.846 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:22:19.847 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.848 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:22:19.849 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.850 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.851 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.853 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:22:19.854 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.855 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.856 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.857 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.858 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.859 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.860 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:22:19.861 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9bbc0b53e0 msgr2=0x7f9bbc0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9bbc0b53e0 0x7f9bbc0b78a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8006010 tx=0x7f9bc8004a40 comp rx=0 tx=0).stop 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0101720 msgr2=0x7f9bd00748c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0101720 0x7f9bd00748c0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f9bcc00e9c0 tx=0x7f9bcc00ee80 comp rx=0 tx=0).stop 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 shutdown_connections 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9bbc0b53e0 0x7f9bbc0b78a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bd0104560 0x7f9bd006d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bd0103be0 0x7f9bd0074e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.839+0000 7f9bd7fe5640 1 --2- 172.21.15.33:0/1371536747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bd0101720 0x7f9bd00748c0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.840+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 >> 172.21.15.33:0/1371536747 conn(0x7f9bd00fca20 msgr2=0x7f9bd00fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.840+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 shutdown_connections 2024-09-10T21:22:19.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:19.840+0000 7f9bd7fe5640 1 -- 172.21.15.33:0/1371536747 wait complete. 2024-09-10T21:22:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:22:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:22:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:22:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:22:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: pgmap v896: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 856 op/s 2024-09-10T21:22:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:20.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 256837 41.42 MB/sec execute 200 sec latency 119.497 ms 2024-09-10T21:22:20.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:20.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:22:21.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 257372 41.33 MB/sec execute 201 sec latency 354.855 ms 2024-09-10T21:22:21.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:21 smithi104 ceph-mon[35325]: pgmap v897: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-10T21:22:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:21 smithi174 ceph-mon[35635]: pgmap v897: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-10T21:22:21.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:21 smithi033 ceph-mon[30860]: pgmap v897: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 878 op/s 2024-09-10T21:22:21.969 INFO:tasks.cephfs.filesystem:scrub status for tag:13d23824-38f6-4822-b207-44e9d8d4829e - {'path': '/', 'tag': '13d23824-38f6-4822-b207-44e9d8d4829e', 'options': 'recursive,force'} 2024-09-10T21:22:22.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 258226 41.35 MB/sec execute 202 sec latency 72.557 ms 2024-09-10T21:22:23.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 259185 41.44 MB/sec execute 203 sec latency 65.839 ms 2024-09-10T21:22:23.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:23 smithi104 ceph-mon[35325]: pgmap v898: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 815 op/s 2024-09-10T21:22:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:23 smithi174 ceph-mon[35635]: pgmap v898: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 815 op/s 2024-09-10T21:22:23.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:23 smithi033 ceph-mon[30860]: pgmap v898: 97 pgs: 97 active+clean; 1.2 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 815 op/s 2024-09-10T21:22:24.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 259758 41.39 MB/sec execute 204 sec latency 82.172 ms 2024-09-10T21:22:25.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 260508 41.35 MB/sec execute 205 sec latency 77.264 ms 2024-09-10T21:22:25.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:25 smithi104 ceph-mon[35325]: pgmap v899: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 921 op/s 2024-09-10T21:22:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:25 smithi174 ceph-mon[35635]: pgmap v899: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 921 op/s 2024-09-10T21:22:25.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:25 smithi033 ceph-mon[30860]: pgmap v899: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 921 op/s 2024-09-10T21:22:26.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 261098 41.31 MB/sec execute 206 sec latency 374.919 ms 2024-09-10T21:22:27.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 261733 41.30 MB/sec execute 207 sec latency 88.711 ms 2024-09-10T21:22:27.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:27 smithi104 ceph-mon[35325]: pgmap v900: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 804 op/s 2024-09-10T21:22:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:27 smithi174 ceph-mon[35635]: pgmap v900: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 804 op/s 2024-09-10T21:22:27.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:27 smithi033 ceph-mon[30860]: pgmap v900: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.7 MiB/s wr, 804 op/s 2024-09-10T21:22:28.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 262578 41.31 MB/sec execute 208 sec latency 93.214 ms 2024-09-10T21:22:29.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 263481 41.35 MB/sec execute 209 sec latency 60.564 ms 2024-09-10T21:22:29.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:29 smithi104 ceph-mon[35325]: pgmap v901: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:22:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:29 smithi174 ceph-mon[35635]: pgmap v901: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:22:29.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:29 smithi033 ceph-mon[30860]: pgmap v901: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:22:30.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 264264 41.36 MB/sec execute 210 sec latency 60.739 ms 2024-09-10T21:22:31.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 264698 41.25 MB/sec execute 211 sec latency 338.520 ms 2024-09-10T21:22:31.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:31 smithi104 ceph-mon[35325]: pgmap v902: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-10T21:22:31.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:22:31.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:22:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:31 smithi174 ceph-mon[35635]: pgmap v902: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-10T21:22:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:22:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:22:31.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:31 smithi033 ceph-mon[30860]: pgmap v902: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 910 op/s 2024-09-10T21:22:31.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:22:31.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:22:32.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 265374 41.24 MB/sec execute 212 sec latency 91.404 ms 2024-09-10T21:22:32.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:32 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:22:32.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:32 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:22:32.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:32 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:22:32.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:32 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:22:32.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:32 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:22:32.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:32 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:22:32.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:32 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:22:32.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:32 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:22:32.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:32 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:22:33.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 266211 41.23 MB/sec execute 213 sec latency 263.944 ms 2024-09-10T21:22:33.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:33 smithi104 ceph-mon[35325]: pgmap v903: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 811 op/s 2024-09-10T21:22:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:33 smithi174 ceph-mon[35635]: pgmap v903: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 811 op/s 2024-09-10T21:22:33.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:33 smithi033 ceph-mon[30860]: pgmap v903: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 811 op/s 2024-09-10T21:22:34.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 267120 41.29 MB/sec execute 214 sec latency 72.234 ms 2024-09-10T21:22:35.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 267911 41.33 MB/sec execute 215 sec latency 127.259 ms 2024-09-10T21:22:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:35 smithi174 ceph-mon[35635]: pgmap v904: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 900 op/s 2024-09-10T21:22:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:35 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:22:35.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:35 smithi033 ceph-mon[30860]: pgmap v904: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 900 op/s 2024-09-10T21:22:35.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:35 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:22:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:35 smithi104 ceph-mon[35325]: pgmap v904: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 900 op/s 2024-09-10T21:22:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:35 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:22:36.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 268444 41.26 MB/sec execute 216 sec latency 311.220 ms 2024-09-10T21:22:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:22:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:22:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:22:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:22:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:22:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:22:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:22:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:22:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:22:37.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 268990 41.22 MB/sec execute 217 sec latency 374.041 ms 2024-09-10T21:22:38.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:37 smithi104 ceph-mon[35325]: pgmap v905: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 756 op/s 2024-09-10T21:22:38.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 269682 41.12 MB/sec execute 218 sec latency 88.976 ms 2024-09-10T21:22:38.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:37 smithi174 ceph-mon[35635]: pgmap v905: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 756 op/s 2024-09-10T21:22:38.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:37 smithi033 ceph-mon[30860]: pgmap v905: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 756 op/s 2024-09-10T21:22:39.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 270623 41.23 MB/sec execute 219 sec latency 88.898 ms 2024-09-10T21:22:40.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:39 smithi104 ceph-mon[35325]: pgmap v906: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 881 op/s 2024-09-10T21:22:40.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 271548 41.26 MB/sec execute 220 sec latency 50.109 ms 2024-09-10T21:22:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:39 smithi174 ceph-mon[35635]: pgmap v906: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 881 op/s 2024-09-10T21:22:40.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:39 smithi033 ceph-mon[30860]: pgmap v906: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 881 op/s 2024-09-10T21:22:41.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 272233 41.24 MB/sec execute 221 sec latency 151.472 ms 2024-09-10T21:22:42.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:41 smithi104 ceph-mon[35325]: pgmap v907: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 921 op/s 2024-09-10T21:22:42.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 272829 41.22 MB/sec execute 222 sec latency 317.074 ms 2024-09-10T21:22:42.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:41 smithi174 ceph-mon[35635]: pgmap v907: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 921 op/s 2024-09-10T21:22:42.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:41 smithi033 ceph-mon[30860]: pgmap v907: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 921 op/s 2024-09-10T21:22:43.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 273511 41.17 MB/sec execute 223 sec latency 95.545 ms 2024-09-10T21:22:44.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:43 smithi104 ceph-mon[35325]: pgmap v908: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:22:44.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 274421 41.19 MB/sec execute 224 sec latency 69.737 ms 2024-09-10T21:22:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:43 smithi174 ceph-mon[35635]: pgmap v908: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:22:44.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:43 smithi033 ceph-mon[30860]: pgmap v908: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:22:45.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 275344 41.23 MB/sec execute 225 sec latency 118.564 ms 2024-09-10T21:22:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:45 smithi104 ceph-mon[35325]: pgmap v909: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 841 op/s 2024-09-10T21:22:46.063 INFO:tasks.workunit.client.0.smithi033.stdout: 10 276178 41.28 MB/sec execute 226 sec latency 51.255 ms 2024-09-10T21:22:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:45 smithi174 ceph-mon[35635]: pgmap v909: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 841 op/s 2024-09-10T21:22:46.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:45 smithi033 ceph-mon[30860]: pgmap v909: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 841 op/s 2024-09-10T21:22:47.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 276641 41.18 MB/sec execute 227 sec latency 335.982 ms 2024-09-10T21:22:48.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 277421 41.21 MB/sec execute 228 sec latency 168.347 ms 2024-09-10T21:22:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:47 smithi174 ceph-mon[35635]: pgmap v910: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:22:48.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:47 smithi033 ceph-mon[30860]: pgmap v910: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:22:48.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:47 smithi104 ceph-mon[35325]: pgmap v910: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:22:49.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 278192 41.22 MB/sec execute 229 sec latency 75.483 ms 2024-09-10T21:22:50.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 278921 41.17 MB/sec execute 230 sec latency 73.022 ms 2024-09-10T21:22:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:49 smithi174 ceph-mon[35635]: pgmap v911: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 895 op/s 2024-09-10T21:22:50.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:49 smithi033 ceph-mon[30860]: pgmap v911: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 895 op/s 2024-09-10T21:22:50.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:49 smithi104 ceph-mon[35325]: pgmap v911: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 895 op/s 2024-09-10T21:22:51.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 279818 41.21 MB/sec execute 231 sec latency 60.247 ms 2024-09-10T21:22:51.970 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:22:52.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 280334 41.15 MB/sec execute 232 sec latency 268.083 ms 2024-09-10T21:22:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:51 smithi174 ceph-mon[35635]: pgmap v912: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 846 op/s 2024-09-10T21:22:52.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:51 smithi033 ceph-mon[30860]: pgmap v912: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 846 op/s 2024-09-10T21:22:52.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:51 smithi104 ceph-mon[35325]: pgmap v912: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 846 op/s 2024-09-10T21:22:52.350 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 -- 172.21.15.33:0/351862471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0250101c90 msgr2=0x7f0250102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 --2- 172.21.15.33:0/351862471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0250101c90 0x7f0250102070 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f0240009900 tx=0x7f024002a410 comp rx=0 tx=0).stop 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 -- 172.21.15.33:0/351862471 shutdown_connections 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 --2- 172.21.15.33:0/351862471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 0x7f025010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 --2- 172.21.15.33:0/351862471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02501076a0 0x7f0250107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 --2- 172.21.15.33:0/351862471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0250101c90 0x7f0250102070 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 -- 172.21.15.33:0/351862471 >> 172.21.15.33:0/351862471 conn(0x7f02500fd700 msgr2=0x7f02500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 -- 172.21.15.33:0/351862471 shutdown_connections 2024-09-10T21:22:52.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.841+0000 7f0254adb640 1 -- 172.21.15.33:0/351862471 wait complete. 2024-09-10T21:22:52.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.842+0000 7f0254adb640 1 Processor -- start 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.842+0000 7f0254adb640 1 -- start start 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.842+0000 7f0254adb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0250101c90 0x7f0250103550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.842+0000 7f0254adb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02501076a0 0x7f0250103a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.842+0000 7f0254adb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 0x7f0250103fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f0254adb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111ce0 con 0x7f02501076a0 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f0254adb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111b60 con 0x7f0250108020 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f0254adb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0250111e60 con 0x7f0250101c90 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0250101c90 0x7f0250103550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 0x7f0250103fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 0x7f0250103fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51610/0 (socket says 172.21.15.33:51610) 2024-09-10T21:22:52.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0250101c90 0x7f0250103550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:58106/0 (socket says 172.21.15.33:58106) 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 -- 172.21.15.33:0/2555121208 learned_addr learned my addr 172.21.15.33:0/2555121208 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024dd9b640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02501076a0 0x7f0250103a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 -- 172.21.15.33:0/2555121208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0250101c90 msgr2=0x7f0250103550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0250101c90 0x7f0250103550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 -- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02501076a0 msgr2=0x7f0250103a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02501076a0 0x7f0250103a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.843+0000 7f024ed9d640 1 -- 172.21.15.33:0/2555121208 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02501a76d0 con 0x7f0250108020 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.844+0000 7f024ed9d640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 0x7f0250103fd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f024800eb10 tx=0x7f024800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.844+0000 7f023f7fe640 1 -- 172.21.15.33:0/2555121208 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248004b20 con 0x7f0250108020 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.844+0000 7f023f7fe640 1 -- 172.21.15.33:0/2555121208 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0248004cc0 con 0x7f0250108020 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.844+0000 7f023f7fe640 1 -- 172.21.15.33:0/2555121208 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0248010670 con 0x7f0250108020 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.844+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02501a79c0 con 0x7f0250108020 2024-09-10T21:22:52.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.844+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02501a7eb0 con 0x7f0250108020 2024-09-10T21:22:52.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.847+0000 7f023f7fe640 1 -- 172.21.15.33:0/2555121208 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0248005020 con 0x7f0250108020 2024-09-10T21:22:52.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.848+0000 7f0256eb0740 1 Processor -- start 2024-09-10T21:22:52.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.848+0000 7f0256eb0740 1 -- start start 2024-09-10T21:22:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b3c480 0x55e388b3c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b53930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f0256eb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e388ab2e50 con 0x55e388b51540 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f0256eb0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e388aa2280 con 0x55e388b3cda0 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f0256eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e388aa6090 con 0x55e388b3c480 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f023f7fe640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02380b5360 0x7f02380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b3c480 0x55e388b3c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f024e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b3c480 0x55e388b3c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:58122/0 (socket says 172.21.15.33:58122) 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f024dd9b640 1 -- 172.21.15.33:0/3417843946 learned_addr learned my addr 172.21.15.33:0/3417843946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:52.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.849+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b53930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.850+0000 7f024dd9b640 1 -- 172.21.15.33:0/3417843946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 msgr2=0x55e388b51000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.850+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b51000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.850+0000 7f024dd9b640 1 -- 172.21.15.33:0/3417843946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 msgr2=0x55e388b53930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.850+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b53930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.850+0000 7f024dd9b640 1 -- 172.21.15.33:0/3417843946 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e388800650 con 0x55e388b3c480 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.850+0000 7f024ed9d640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02380b5360 0x7f02380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.850+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b3c480 0x55e388b3c860 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02440117e0 tx=0x7f02440097a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3cfdf0bf94cdd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.851+0000 7f023f7fe640 1 -- 172.21.15.33:0/2555121208 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f02480e6200 con 0x7f0250108020 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.851+0000 7f023cff9640 1 -- 172.21.15.33:0/3417843946 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244004030 con 0x55e388b3c480 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.851+0000 7f023cff9640 1 -- 172.21.15.33:0/3417843946 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f024400f7c0 con 0x55e388b3c480 2024-09-10T21:22:52.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.851+0000 7f023cff9640 1 -- 172.21.15.33:0/3417843946 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244003930 con 0x55e388b3c480 2024-09-10T21:22:52.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 -- 172.21.15.33:0/3417843946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b3c480 msgr2=0x55e388b3c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b3c480 0x55e388b3c860 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02440117e0 tx=0x7f02440097a0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f024ed9d640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02380b5360 0x7f02380b7820 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0248015040 tx=0x7f024800cf70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 -- 172.21.15.33:0/3417843946 shutdown_connections 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b53930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b51000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3417843946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b3c480 0x55e388b3c860 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f02440117e0 tx=0x7f02440097a0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 -- 172.21.15.33:0/3417843946 >> 172.21.15.33:0/3417843946 conn(0x55e388b3a2b0 msgr2=0x55e388b3b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 -- 172.21.15.33:0/3417843946 shutdown_connections 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.852+0000 7f0256eb0740 1 -- 172.21.15.33:0/3417843946 wait complete. 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 -- start start 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 Processor -- start 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b4e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b51540 0x55e388b4ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b4f010 0x55e388b5d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e388903700 con 0x55e388b3cda0 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e388a3d110 con 0x55e388b4f010 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e388b54160 con 0x55e388b51540 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f0256eb0740 10 client.0 _ll_register_callbacks cb 0x55e388a56990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f024e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b51540 0x55e388b4ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f024e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b51540 0x55e388b4ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:58126/0 (socket says 172.21.15.33:58126) 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f024e59c640 1 -- 172.21.15.33:0/3163563718 learned_addr learned my addr 172.21.15.33:0/3163563718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.853+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b4e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b4f010 0x55e388b5d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f024dd9b640 1 -- 172.21.15.33:0/3163563718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b51540 msgr2=0x55e388b4ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b51540 0x55e388b4ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f024dd9b640 1 -- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b4f010 msgr2=0x55e388b5d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b4f010 0x55e388b5d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f024dd9b640 1 -- 172.21.15.33:0/3163563718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e388a6ac40 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f024dd9b640 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b4e590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x55e388b3af90 tx=0x7f024400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5592928ebd5ee8b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.854+0000 7f0235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.855+0000 7f0235ffb640 1 -- 172.21.15.33:0/3163563718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244023070 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.855+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f024400f7c0 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.855+0000 7f0235ffb640 1 -- 172.21.15.33:0/3163563718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f02440051a0 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.855+0000 7f0235ffb640 1 -- 172.21.15.33:0/3163563718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f0244029040 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.855+0000 7f0256eb0740 10 client.18780 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.855+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55e388b60190 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.855+0000 7f0235ffb640 1 -- 172.21.15.33:0/3163563718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02440053a0 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0235ffb640 1 -- 172.21.15.33:0/3163563718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f02440314c0 con 0x55e388b3cda0 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0256eb0740 10 client.18780 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0256eb0740 10 client.18780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:52.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0256eb0740 20 client.18780 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388b614e0 0x55e388b638d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0256eb0740 4 client.18780 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f024400f7c0 con 0x55e388b614e0 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f024e59c640 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388b614e0 0x55e388b638d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f024e59c640 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388b614e0 0x55e388b638d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5527a12264ea996b server_cookie=2d71a4e84a3d569f in_seq=0 out_seq=0 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.856+0000 7f0235ffb640 10 client.18780 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.857+0000 7f0235ffb640 1 -- 172.21.15.33:0/3163563718 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f024400f7c0 con 0x55e388b614e0 2024-09-10T21:22:52.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.857+0000 7f0235ffb640 10 client.18780 handle_command_reply: tid=0 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.857+0000 7f0256eb0740 1 client.18780 shutdown 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.857+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 msgr2=0x55e388b4e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.857+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b4e590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x55e388b3af90 tx=0x7f024400ff60 comp rx=0 tx=0).stop 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.857+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 shutdown_connections 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.857+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388b614e0 0x55e388b638d0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0235ffb640 0 client.18780 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b4f010 0x55e388b5d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b51540 0x55e388b4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0256eb0740 1 --2- 172.21.15.33:0/3163563718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b4e590 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 >> 172.21.15.33:0/3163563718 conn(0x55e388b3a2b0 msgr2=0x55e388b3fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 shutdown_connections 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0256eb0740 1 -- 172.21.15.33:0/3163563718 wait complete. 2024-09-10T21:22:52.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.858+0000 7f0256eb0740 20 client.18780 trim_cache size 0 max 16384 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 Processor -- start 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 -- start start 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b5d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b4bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b51540 0x55e388b4e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ac1a0 con 0x55e388b3cda0 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f02501adce0 con 0x55e388b51540 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f0256eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ac870 con 0x55e388b4f010 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b5d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b5d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51082/0 (socket says 172.21.15.33:51082) 2024-09-10T21:22:52.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f024dd9b640 1 -- 172.21.15.33:0/2340386539 learned_addr learned my addr 172.21.15.33:0/2340386539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f024e59c640 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b4bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.879+0000 7f024dd9b640 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b51540 0x55e388b4e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f024dd9b640 1 -- 172.21.15.33:0/2340386539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 msgr2=0x55e388b4bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f024dd9b640 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b4bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f024dd9b640 1 -- 172.21.15.33:0/2340386539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b51540 msgr2=0x55e388b4e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f024dd9b640 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b51540 0x55e388b4e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f024dd9b640 1 -- 172.21.15.33:0/2340386539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e388b52090 con 0x55e388b3cda0 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f024dd9b640 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b5d390 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f02440117e0 tx=0x7f0244005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90545563e0153a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f0237fff640 1 -- 172.21.15.33:0/2340386539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244023020 con 0x55e388b3cda0 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f0237fff640 1 -- 172.21.15.33:0/2340386539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0244005b90 con 0x55e388b3cda0 2024-09-10T21:22:52.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.880+0000 7f0237fff640 1 -- 172.21.15.33:0/2340386539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02440053e0 con 0x55e388b3cda0 2024-09-10T21:22:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 -- 172.21.15.33:0/2340386539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 msgr2=0x55e388b5d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b5d390 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f02440117e0 tx=0x7f0244005fd0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 -- 172.21.15.33:0/2340386539 shutdown_connections 2024-09-10T21:22:52.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b51540 0x55e388b4e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b4bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 --2- 172.21.15.33:0/2340386539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b3cda0 0x55e388b5d390 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 -- 172.21.15.33:0/2340386539 >> 172.21.15.33:0/2340386539 conn(0x55e388b3a2b0 msgr2=0x55e388b52620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 -- 172.21.15.33:0/2340386539 shutdown_connections 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 -- 172.21.15.33:0/2340386539 wait complete. 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 -- start start 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.881+0000 7f0256eb0740 1 Processor -- start 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b59b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b68bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f0256eb0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b69110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f0256eb0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f02501ad610 con 0x55e388b51540 2024-09-10T21:22:52.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f0256eb0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f02501acf40 con 0x55e388b3cda0 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f0256eb0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02501aa160 con 0x55e388b4f010 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f0256eb0740 10 client.0 _ll_register_callbacks cb 0x7f0244009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b59b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b59b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51642/0 (socket says 172.21.15.33:51642) 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 -- 172.21.15.33:0/657045120 learned_addr learned my addr 172.21.15.33:0/657045120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024e59c640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b68bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b69110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 -- 172.21.15.33:0/657045120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 msgr2=0x55e388b68bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b68bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 -- 172.21.15.33:0/657045120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 msgr2=0x55e388b69110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b69110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 -- 172.21.15.33:0/657045120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e388b69880 con 0x55e388b3cda0 2024-09-10T21:22:52.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024e59c640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b68bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.882+0000 7f024dd9b640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b59b20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0244032040 tx=0x7f0244009f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6286ecb10620ab79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0235ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0235ffb640 1 -- 172.21.15.33:0/657045120 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0244023070 con 0x55e388b3cda0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0244005b90 con 0x55e388b3cda0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0235ffb640 1 -- 172.21.15.33:0/657045120 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0244005310 con 0x55e388b3cda0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0235ffb640 1 -- 172.21.15.33:0/657045120 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f0244031d70 con 0x55e388b3cda0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0256eb0740 10 client.28166 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55e388bce240 con 0x55e388b3cda0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.883+0000 7f0235ffb640 1 -- 172.21.15.33:0/657045120 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0244005b90 con 0x55e388b3cda0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0235ffb640 1 -- 172.21.15.33:0/657045120 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f0244042570 con 0x55e388b3cda0 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0256eb0740 10 client.28166 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0256eb0740 10 client.28166 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:22:52.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0256eb0740 20 client.28166 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:52.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0256eb0740 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388bce910 0x55e388bd0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:52.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0256eb0740 4 client.28166 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:22:52.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e388ab4da0 con 0x55e388bce910 2024-09-10T21:22:52.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f024e59c640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388bce910 0x55e388bd0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:52.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f024e59c640 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388bce910 0x55e388bd0d30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272c8ff7208dadf server_cookie=35629d72e514164d in_seq=0 out_seq=0 2024-09-10T21:22:52.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.884+0000 7f0235ffb640 10 client.28166 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:52.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.886+0000 7f0235ffb640 1 -- 172.21.15.33:0/657045120 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+6906 (crc 0 0 0) 0x55e388ab4da0 con 0x55e388bce910 2024-09-10T21:22:52.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.886+0000 7f0235ffb640 10 client.28166 handle_command_reply: tid=0 2024-09-10T21:22:52.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 client.28166 shutdown 2024-09-10T21:22:52.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 msgr2=0x55e388b59b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b59b20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0244032040 tx=0x7f0244009f30 comp rx=0 tx=0).stop 2024-09-10T21:22:52.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 shutdown_connections 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e388bce910 0x55e388bd0d30 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e388b51540 0x55e388b69110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e388b4f010 0x55e388b68bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 --2- 172.21.15.33:0/657045120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e388b3cda0 0x55e388b59b20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 >> 172.21.15.33:0/657045120 conn(0x55e388b3a2b0 msgr2=0x55e388b4e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0235ffb640 0 client.28166 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 shutdown_connections 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 1 -- 172.21.15.33:0/657045120 wait complete. 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.887+0000 7f0256eb0740 20 client.28166 trim_cache size 0 max 16384 2024-09-10T21:22:52.893 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.894 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.895 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:22:52.896 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:22:52.898 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.901 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:22:52.906 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:22:52.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02380b5360 msgr2=0x7f02380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02380b5360 0x7f02380b7820 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0248015040 tx=0x7f024800cf70 comp rx=0 tx=0).stop 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 msgr2=0x7f0250103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:52.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 0x7f0250103fd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f024800eb10 tx=0x7f024800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 shutdown_connections 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02380b5360 0x7f02380b7820 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0250108020 0x7f0250103fd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02501076a0 0x7f0250103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 --2- 172.21.15.33:0/2555121208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0250101c90 0x7f0250103550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 >> 172.21.15.33:0/2555121208 conn(0x7f02500fd700 msgr2=0x7f025006ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 shutdown_connections 2024-09-10T21:22:52.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:52.888+0000 7f0254adb640 1 -- 172.21.15.33:0/2555121208 wait complete. 2024-09-10T21:22:53.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 280952 41.12 MB/sec execute 233 sec latency 273.233 ms 2024-09-10T21:22:53.962 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:22:53.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:22:54.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 281751 41.15 MB/sec execute 234 sec latency 71.407 ms 2024-09-10T21:22:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:53 smithi174 ceph-mon[35635]: pgmap v913: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 805 op/s 2024-09-10T21:22:54.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:53 smithi033 ceph-mon[30860]: pgmap v913: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 805 op/s 2024-09-10T21:22:54.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:53 smithi104 ceph-mon[35325]: pgmap v913: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 805 op/s 2024-09-10T21:22:54.317 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:54.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 -- 172.21.15.33:0/361557780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c103ca0 msgr2=0x7f048c104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 --2- 172.21.15.33:0/361557780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c103ca0 0x7f048c104080 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f0488009900 tx=0x7f048802a410 comp rx=0 tx=0).stop 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 -- 172.21.15.33:0/361557780 shutdown_connections 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 --2- 172.21.15.33:0/361557780 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f048c10d100 0x7f048c10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 --2- 172.21.15.33:0/361557780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f048c1045c0 0x7f048c10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 --2- 172.21.15.33:0/361557780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c103ca0 0x7f048c104080 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 -- 172.21.15.33:0/361557780 >> 172.21.15.33:0/361557780 conn(0x7f048c0fd700 msgr2=0x7f048c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 -- 172.21.15.33:0/361557780 shutdown_connections 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.989+0000 7f0493f92640 1 -- 172.21.15.33:0/361557780 wait complete. 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 Processor -- start 2024-09-10T21:22:54.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 -- start start 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f048c103ca0 0x7f048c101950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c1045c0 0x7f048c101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f048c10d100 0x7f048c1023d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112d70 con 0x7f048c1045c0 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112bf0 con 0x7f048c103ca0 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0493f92640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112ef0 con 0x7f048c10d100 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f049152d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c1045c0 0x7f048c101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f049152d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c1045c0 0x7f048c101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51104/0 (socket says 172.21.15.33:51104) 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f049152d640 1 -- 172.21.15.33:0/2765539060 learned_addr learned my addr 172.21.15.33:0/2765539060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f0491d2e640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f048c103ca0 0x7f048c101950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:54.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.990+0000 7f049252f640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f048c10d100 0x7f048c1023d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f049152d640 1 -- 172.21.15.33:0/2765539060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f048c10d100 msgr2=0x7f048c1023d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f049152d640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f048c10d100 0x7f048c1023d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f049152d640 1 -- 172.21.15.33:0/2765539060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f048c103ca0 msgr2=0x7f048c101950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f049152d640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f048c103ca0 0x7f048c101950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:54.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f049152d640 1 -- 172.21.15.33:0/2765539060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c1a7760 con 0x7f048c1045c0 2024-09-10T21:22:54.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f049152d640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c1045c0 0x7f048c101e90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9c0 tx=0x7f047c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:54.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f0482ffd640 1 -- 172.21.15.33:0/2765539060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c004230 con 0x7f048c1045c0 2024-09-10T21:22:54.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c1a7a50 con 0x7f048c1045c0 2024-09-10T21:22:54.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f0482ffd640 1 -- 172.21.15.33:0/2765539060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c0043d0 con 0x7f048c1045c0 2024-09-10T21:22:54.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f0482ffd640 1 -- 172.21.15.33:0/2765539060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c003e60 con 0x7f048c1045c0 2024-09-10T21:22:54.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.991+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1a7f90 con 0x7f048c1045c0 2024-09-10T21:22:54.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.992+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c104080 con 0x7f048c1045c0 2024-09-10T21:22:54.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.994+0000 7f0482ffd640 1 -- 172.21.15.33:0/2765539060 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f047c00c8b0 con 0x7f048c1045c0 2024-09-10T21:22:54.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.995+0000 7f0482ffd640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f04780b52e0 0x7f04780b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:54.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.995+0000 7f0491d2e640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f04780b52e0 0x7f04780b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:54.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.995+0000 7f0482ffd640 1 -- 172.21.15.33:0/2765539060 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f047c0e61e0 con 0x7f048c1045c0 2024-09-10T21:22:54.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.996+0000 7f0491d2e640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f04780b52e0 0x7f04780b77a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0488031440 tx=0x7f0488005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:55.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:54.998+0000 7f0482ffd640 1 -- 172.21.15.33:0/2765539060 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f047c0ad0a0 con 0x7f048c1045c0 2024-09-10T21:22:55.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 282271 41.09 MB/sec execute 235 sec latency 136.048 ms 2024-09-10T21:22:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.294+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f048c02e5c0 con 0x7f048c1045c0 2024-09-10T21:22:55.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.295+0000 7f0482ffd640 1 -- 172.21.15.33:0/2765539060 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+11973 (secure 0 0 0) 0x7f047c0ac9a0 con 0x7f048c1045c0 2024-09-10T21:22:55.298 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:22:55.298 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"btime":"2024-09-10T21:21:17:963637+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":81,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:21:17.000563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:22:55.299 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f04780b52e0 msgr2=0x7f04780b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f04780b52e0 0x7f04780b77a0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0488031440 tx=0x7f0488005b50 comp rx=0 tx=0).stop 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c1045c0 msgr2=0x7f048c101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c1045c0 0x7f048c101e90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9c0 tx=0x7f047c00edc0 comp rx=0 tx=0).stop 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 shutdown_connections 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f04780b52e0 0x7f04780b77a0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f048c10d100 0x7f048c1023d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f048c1045c0 0x7f048c101e90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 --2- 172.21.15.33:0/2765539060 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f048c103ca0 0x7f048c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:55.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.298+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 >> 172.21.15.33:0/2765539060 conn(0x7f048c0fd700 msgr2=0x7f048c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:55.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.299+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 shutdown_connections 2024-09-10T21:22:55.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:55.300+0000 7f0493f92640 1 -- 172.21.15.33:0/2765539060 wait complete. 2024-09-10T21:22:55.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:55 smithi033 ceph-mon[30860]: pgmap v914: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 866 op/s 2024-09-10T21:22:55.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:55 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2765539060' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:22:56.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 282989 41.04 MB/sec execute 236 sec latency 93.151 ms 2024-09-10T21:22:56.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:22:56.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:55 smithi174 ceph-mon[35635]: pgmap v914: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 866 op/s 2024-09-10T21:22:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:55 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2765539060' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:22:56.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:55 smithi104 ceph-mon[35325]: pgmap v914: 97 pgs: 97 active+clean; 1.2 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 866 op/s 2024-09-10T21:22:56.315 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:55 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2765539060' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:22:56.568 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:57.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 283452 41.00 MB/sec execute 237 sec latency 277.808 ms 2024-09-10T21:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.374+0000 7f507edd3640 1 -- 172.21.15.33:0/2697966855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507810a530 msgr2=0x7f507810a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.374+0000 7f507edd3640 1 --2- 172.21.15.33:0/2697966855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507810a530 0x7f507810a930 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f5070009900 tx=0x7f507002a480 comp rx=0 tx=0).stop 2024-09-10T21:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 -- 172.21.15.33:0/2697966855 shutdown_connections 2024-09-10T21:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 --2- 172.21.15.33:0/2697966855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f507811c0a0 0x7f507811e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 --2- 172.21.15.33:0/2697966855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507810a530 0x7f507810a930 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 --2- 172.21.15.33:0/2697966855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507810a150 0x7f50781111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 -- 172.21.15.33:0/2697966855 >> 172.21.15.33:0/2697966855 conn(0x7f507806dfc0 msgr2=0x7f507806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 -- 172.21.15.33:0/2697966855 shutdown_connections 2024-09-10T21:22:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 -- 172.21.15.33:0/2697966855 wait complete. 2024-09-10T21:22:57.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.375+0000 7f507edd3640 1 Processor -- start 2024-09-10T21:22:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507edd3640 1 -- start start 2024-09-10T21:22:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507edd3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507810a150 0x7f50781a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507edd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f507810a530 0x7f50781a8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507edd3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507811c0a0 0x7f50781ad9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507edd3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5078121b20 con 0x7f507811c0a0 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507edd3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50781219a0 con 0x7f507810a150 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507edd3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5078121ca0 con 0x7f507810a530 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507811c0a0 0x7f50781ad9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507811c0a0 0x7f50781ad9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51122/0 (socket says 172.21.15.33:51122) 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 -- 172.21.15.33:0/928007044 learned_addr learned my addr 172.21.15.33:0/928007044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.376+0000 7f507cb6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507810a150 0x7f50781a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 -- 172.21.15.33:0/928007044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f507810a530 msgr2=0x7f50781a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f507810a530 0x7f50781a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 -- 172.21.15.33:0/928007044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507810a150 msgr2=0x7f50781a80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507810a150 0x7f50781a80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.377+0000 7f507d370640 1 -- 172.21.15.33:0/928007044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50781ae180 con 0x7f507811c0a0 2024-09-10T21:22:57.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.378+0000 7f507d370640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507811c0a0 0x7f50781ad9b0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f506800eaa0 tx=0x7f506800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.378+0000 7f5075ffb640 1 -- 172.21.15.33:0/928007044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506800cdb0 con 0x7f507811c0a0 2024-09-10T21:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.378+0000 7f5075ffb640 1 -- 172.21.15.33:0/928007044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5068005020 con 0x7f507811c0a0 2024-09-10T21:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.378+0000 7f5075ffb640 1 -- 172.21.15.33:0/928007044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50680116d0 con 0x7f507811c0a0 2024-09-10T21:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.378+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50781ae470 con 0x7f507811c0a0 2024-09-10T21:22:57.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.379+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50781b4b30 con 0x7f507811c0a0 2024-09-10T21:22:57.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.382+0000 7f50811a8740 1 Processor -- start 2024-09-10T21:22:57.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.383+0000 7f5075ffb640 1 -- 172.21.15.33:0/928007044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f506800f040 con 0x7f507811c0a0 2024-09-10T21:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f50811a8740 1 -- start start 2024-09-10T21:22:57.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f50811a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fecf4d0 0x56175fecf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f50811a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fedb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f50811a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fede170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f50811a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56175fe45ea0 con 0x56175fecf4d0 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f50811a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56175fe352d0 con 0x56175fecfdf0 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f50811a8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56175fe390e0 con 0x56175fedbd80 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f5077fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fecf4d0 0x56175fecf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f5077fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fecf4d0 0x56175fecf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51130/0 (socket says 172.21.15.33:51130) 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.384+0000 7f5077fff640 1 -- 172.21.15.33:0/2339135999 learned_addr learned my addr 172.21.15.33:0/2339135999 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:57.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.385+0000 7f5075ffb640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50600b5430 0x7f50600b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.385+0000 7f507cb6f640 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fedb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.385+0000 7f5075ffb640 1 -- 172.21.15.33:0/928007044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50680e5980 con 0x7f507811c0a0 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.385+0000 7f507d370640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50600b5430 0x7f50600b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.385+0000 7f5077fff640 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fede170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.386+0000 7f507d370640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50600b5430 0x7f50600b78f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f506800cfd0 tx=0x7f506801d000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.386+0000 7f5077fff640 1 -- 172.21.15.33:0/2339135999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 msgr2=0x56175fede170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.386+0000 7f5077fff640 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fede170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.386+0000 7f5077fff640 1 -- 172.21.15.33:0/2339135999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 msgr2=0x56175fedb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.386+0000 7f5077fff640 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fedb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.386+0000 7f5077fff640 1 -- 172.21.15.33:0/2339135999 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56175fb93650 con 0x56175fecf4d0 2024-09-10T21:22:57.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.386+0000 7f507cb6f640 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fedb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:57.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.387+0000 7f5077fff640 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fecf4d0 0x56175fecf8b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5070005ab0 tx=0x7f5070000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa126d0fb7655e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:57.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.387+0000 7f505effd640 1 -- 172.21.15.33:0/2339135999 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070035070 con 0x56175fecf4d0 2024-09-10T21:22:57.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.388+0000 7f505effd640 1 -- 172.21.15.33:0/2339135999 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50700316d0 con 0x56175fecf4d0 2024-09-10T21:22:57.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.388+0000 7f505effd640 1 -- 172.21.15.33:0/2339135999 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5070035070 con 0x56175fecf4d0 2024-09-10T21:22:57.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.389+0000 7f50811a8740 1 -- 172.21.15.33:0/2339135999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fecf4d0 msgr2=0x56175fecf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.389+0000 7f50811a8740 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fecf4d0 0x56175fecf8b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5070005ab0 tx=0x7f5070000bc0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.389+0000 7f50811a8740 1 -- 172.21.15.33:0/2339135999 shutdown_connections 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.389+0000 7f50811a8740 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fede170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.389+0000 7f50811a8740 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fedb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.389+0000 7f50811a8740 1 --2- 172.21.15.33:0/2339135999 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fecf4d0 0x56175fecf8b0 secure :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f5070005ab0 tx=0x7f5070000bc0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.389+0000 7f50811a8740 1 -- 172.21.15.33:0/2339135999 >> 172.21.15.33:0/2339135999 conn(0x56175fecd300 msgr2=0x56175fece4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.390+0000 7f50811a8740 1 -- 172.21.15.33:0/2339135999 shutdown_connections 2024-09-10T21:22:57.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.390+0000 7f50811a8740 1 -- 172.21.15.33:0/2339135999 wait complete. 2024-09-10T21:22:57.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.391+0000 7f50811a8740 1 -- start start 2024-09-10T21:22:57.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.391+0000 7f50811a8740 1 Processor -- start 2024-09-10T21:22:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.392+0000 7f50811a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 0x56175fed8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.392+0000 7f50811a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fedbd80 0x56175fed91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.393+0000 7f50811a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fef8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.393+0000 7f50811a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56175fc96700 con 0x56175fed9720 2024-09-10T21:22:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.393+0000 7f50811a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56175fdd0160 con 0x56175fedbd80 2024-09-10T21:22:57.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.393+0000 7f50811a8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56175feefa00 con 0x56175fecfdf0 2024-09-10T21:22:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.393+0000 7f50811a8740 10 client.0 _ll_register_callbacks cb 0x56175fde99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.393+0000 7f507cb6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fedbd80 0x56175fed91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.394+0000 7f507cb6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fedbd80 0x56175fed91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51678/0 (socket says 172.21.15.33:51678) 2024-09-10T21:22:57.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.394+0000 7f507cb6f640 1 -- 172.21.15.33:0/4065527286 learned_addr learned my addr 172.21.15.33:0/4065527286 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.395+0000 7f507cb6f640 1 -- 172.21.15.33:0/4065527286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 msgr2=0x56175fed8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.395+0000 7f507cb6f640 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 0x56175fed8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.395+0000 7f507cb6f640 1 -- 172.21.15.33:0/4065527286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 msgr2=0x56175fef8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.395+0000 7f507cb6f640 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fef8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.395+0000 7f507cb6f640 1 -- 172.21.15.33:0/4065527286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56175fdfdc90 con 0x56175fedbd80 2024-09-10T21:22:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.396+0000 7f507cb6f640 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fedbd80 0x56175fed91e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f506c0126d0 tx=0x7f506c012b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97ccd434b1cedd93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.396+0000 7f5057fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.396+0000 7f5057fff640 1 -- 172.21.15.33:0/4065527286 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c00f8c0 con 0x56175fedbd80 2024-09-10T21:22:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.396+0000 7f5057fff640 1 -- 172.21.15.33:0/4065527286 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f506c00fa60 con 0x56175fedbd80 2024-09-10T21:22:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.398+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50700316d0 con 0x56175fedbd80 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.399+0000 7f5057fff640 1 -- 172.21.15.33:0/4065527286 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f506c00fc60 con 0x56175fedbd80 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.399+0000 7f50811a8740 10 client.28184 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.399+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x56175fefb950 con 0x56175fedbd80 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.399+0000 7f5057fff640 1 -- 172.21.15.33:0/4065527286 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f506c00fe00 con 0x56175fedbd80 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.399+0000 7f5057fff640 1 -- 172.21.15.33:0/4065527286 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f506c0105b0 con 0x56175fedbd80 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.400+0000 7f50811a8740 10 client.28184 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.400+0000 7f50811a8740 10 client.28184 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.400+0000 7f50811a8740 20 client.28184 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.400+0000 7f50811a8740 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175feff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.400+0000 7f50811a8740 4 client.28184 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.400+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f50700316d0 con 0x56175fefd1a0 2024-09-10T21:22:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.401+0000 7f5077fff640 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175feff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.401+0000 7f5077fff640 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175feff660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a36a20440b32d478 server_cookie=d1ae31983fbec727 in_seq=0 out_seq=0 2024-09-10T21:22:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.401+0000 7f5057fff640 10 client.28184 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:57.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.406+0000 7f5057fff640 1 -- 172.21.15.33:0/4065527286 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f50700316d0 con 0x56175fefd1a0 2024-09-10T21:22:57.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.406+0000 7f5057fff640 10 client.28184 handle_command_reply: tid=0 2024-09-10T21:22:57.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.407+0000 7f50811a8740 1 client.28184 shutdown 2024-09-10T21:22:57.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fedbd80 msgr2=0x56175fed91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f50811a8740 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fedbd80 0x56175fed91e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f506c0126d0 tx=0x7f506c012b90 comp rx=0 tx=0).stop 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 shutdown_connections 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f50811a8740 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175feff660 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f50811a8740 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fef8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f50811a8740 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fedbd80 0x56175fed91e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f5057fff640 0 client.28184 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.408+0000 7f50811a8740 1 --2- 172.21.15.33:0/4065527286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 0x56175fed8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.409+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 >> 172.21.15.33:0/4065527286 conn(0x56175fecd300 msgr2=0x56175fed2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.409+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 shutdown_connections 2024-09-10T21:22:57.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.409+0000 7f50811a8740 1 -- 172.21.15.33:0/4065527286 wait complete. 2024-09-10T21:22:57.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.410+0000 7f50811a8740 20 client.28184 trim_cache size 0 max 16384 2024-09-10T21:22:57.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.436+0000 7f50811a8740 1 Processor -- start 2024-09-10T21:22:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.437+0000 7f50811a8740 1 -- start start 2024-09-10T21:22:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.437+0000 7f50811a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fef8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.437+0000 7f50811a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fed7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.437+0000 7f50811a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fed7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.437+0000 7f50811a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5078119ca0 con 0x56175fed9720 2024-09-10T21:22:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.437+0000 7f50811a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50781b64b0 con 0x56175fecfdf0 2024-09-10T21:22:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.437+0000 7f50811a8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50781b5040 con 0x56175fedbd80 2024-09-10T21:22:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.438+0000 7f5077fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fef8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.438+0000 7f5077fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fed7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.438+0000 7f5077fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fef8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51684/0 (socket says 172.21.15.33:51684) 2024-09-10T21:22:57.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.438+0000 7f5077fff640 1 -- 172.21.15.33:0/1585306775 learned_addr learned my addr 172.21.15.33:0/1585306775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.438+0000 7f507cb6f640 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fed7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.439+0000 7f507cb6f640 1 -- 172.21.15.33:0/1585306775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 msgr2=0x56175fed7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.439+0000 7f507cb6f640 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fed7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.439+0000 7f507cb6f640 1 -- 172.21.15.33:0/1585306775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 msgr2=0x56175fef8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.439+0000 7f507cb6f640 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fef8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.439+0000 7f507cb6f640 1 -- 172.21.15.33:0/1585306775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56175fecdd80 con 0x56175fed9720 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.439+0000 7f507cb6f640 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fed7520 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x56175fecccf0 tx=0x7f506c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9484b6cad81b1d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.440+0000 7f505e7fc640 1 -- 172.21.15.33:0/1585306775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c02b030 con 0x56175fed9720 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.440+0000 7f505e7fc640 1 -- 172.21.15.33:0/1585306775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f506c00f850 con 0x56175fed9720 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.440+0000 7f505e7fc640 1 -- 172.21.15.33:0/1585306775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c005710 con 0x56175fed9720 2024-09-10T21:22:57.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.441+0000 7f50811a8740 1 -- 172.21.15.33:0/1585306775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 msgr2=0x56175fed7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.441+0000 7f50811a8740 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fed7520 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x56175fecccf0 tx=0x7f506c00ff90 comp rx=0 tx=0).stop 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.442+0000 7f50811a8740 1 -- 172.21.15.33:0/1585306775 shutdown_connections 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.442+0000 7f50811a8740 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fedbd80 0x56175fed7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.442+0000 7f50811a8740 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fed9720 0x56175fed7520 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.442+0000 7f50811a8740 1 --2- 172.21.15.33:0/1585306775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fecfdf0 0x56175fef8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.442+0000 7f50811a8740 1 -- 172.21.15.33:0/1585306775 >> 172.21.15.33:0/1585306775 conn(0x56175fecd300 msgr2=0x56175fece300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.442+0000 7f50811a8740 1 -- 172.21.15.33:0/1585306775 shutdown_connections 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.443+0000 7f50811a8740 1 -- 172.21.15.33:0/1585306775 wait complete. 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.443+0000 7f50811a8740 1 -- start start 2024-09-10T21:22:57.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.443+0000 7f50811a8740 1 Processor -- start 2024-09-10T21:22:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f50811a8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 0x56175fef5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f50811a8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fed9720 0x56175feefc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f50811a8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fedbd80 0x56175fef0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f50811a8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50781b5de0 con 0x56175fedbd80 2024-09-10T21:22:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f50811a8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50781b5710 con 0x56175fed9720 2024-09-10T21:22:57.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f50811a8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5078117c60 con 0x56175fecfdf0 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f50811a8740 10 client.0 _ll_register_callbacks cb 0x7f506c009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f507cb6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fed9720 0x56175feefc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f507cb6f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fed9720 0x56175feefc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51688/0 (socket says 172.21.15.33:51688) 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f507cb6f640 1 -- 172.21.15.33:0/2799694074 learned_addr learned my addr 172.21.15.33:0/2799694074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f5077fff640 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fedbd80 0x56175fef0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.444+0000 7f5077fff640 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 0x56175fef5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f507cb6f640 1 -- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 msgr2=0x56175fef5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f507cb6f640 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 0x56175fef5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f507cb6f640 1 -- 172.21.15.33:0/2799694074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fedbd80 msgr2=0x56175fef0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f507cb6f640 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fedbd80 0x56175fef0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f507cb6f640 1 -- 172.21.15.33:0/2799694074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56175fef09c0 con 0x56175fed9720 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f507cb6f640 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fed9720 0x56175feefc50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f506c000c00 tx=0x7f506c005340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb752f93073bbd26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:57.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f5057fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f5057fff640 1 -- 172.21.15.33:0/2799694074 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f506c02b070 con 0x56175fed9720 2024-09-10T21:22:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f5057fff640 1 -- 172.21.15.33:0/2799694074 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f506c0035f0 con 0x56175fed9720 2024-09-10T21:22:57.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.445+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56175fef9610 con 0x56175fed9720 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.446+0000 7f5057fff640 1 -- 172.21.15.33:0/2799694074 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f506c031080 con 0x56175fed9720 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.446+0000 7f50811a8740 10 client.28190 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.446+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x56175ff6b190 con 0x56175fed9720 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.446+0000 7f5057fff640 1 -- 172.21.15.33:0/2799694074 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56175fef9610 con 0x56175fed9720 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.447+0000 7f5057fff640 1 -- 172.21.15.33:0/2799694074 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f506c0266a0 con 0x56175fed9720 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.447+0000 7f50811a8740 10 client.28190 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.447+0000 7f50811a8740 10 client.28190 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.447+0000 7f50811a8740 20 client.28190 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.447+0000 7f50811a8740 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175ff6d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:57.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.447+0000 7f50811a8740 4 client.28190 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:22:57.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.447+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56175fe47df0 con 0x56175fefd1a0 2024-09-10T21:22:57.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.450+0000 7f5077fff640 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175ff6d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:57.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.451+0000 7f5077fff640 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175ff6d880 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e4ae52274e733c9 server_cookie=2c03e7e838cd7182 in_seq=0 out_seq=0 2024-09-10T21:22:57.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.451+0000 7f5057fff640 10 client.28190 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.453+0000 7f5057fff640 1 -- 172.21.15.33:0/2799694074 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56175fe47df0 con 0x56175fefd1a0 2024-09-10T21:22:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.453+0000 7f5057fff640 10 client.28190 handle_command_reply: tid=0 2024-09-10T21:22:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.453+0000 7f50811a8740 1 client.28190 shutdown 2024-09-10T21:22:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.453+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fed9720 msgr2=0x56175feefc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.453+0000 7f50811a8740 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fed9720 0x56175feefc50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f506c000c00 tx=0x7f506c005340 comp rx=0 tx=0).stop 2024-09-10T21:22:57.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 shutdown_connections 2024-09-10T21:22:57.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56175fefd1a0 0x56175ff6d880 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56175fedbd80 0x56175fef0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56175fed9720 0x56175feefc50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 --2- 172.21.15.33:0/2799694074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56175fecfdf0 0x56175fef5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 >> 172.21.15.33:0/2799694074 conn(0x56175fecd300 msgr2=0x56175fed73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f5057fff640 0 client.28190 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 shutdown_connections 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 1 -- 172.21.15.33:0/2799694074 wait complete. 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.454+0000 7f50811a8740 20 client.28190 trim_cache size 0 max 16384 2024-09-10T21:22:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:22:57.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50600b5430 msgr2=0x7f50600b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50600b5430 0x7f50600b78f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f506800cfd0 tx=0x7f506801d000 comp rx=0 tx=0).stop 2024-09-10T21:22:57.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507811c0a0 msgr2=0x7f50781ad9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:57.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507811c0a0 0x7f50781ad9b0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f506800eaa0 tx=0x7f506800ef60 comp rx=0 tx=0).stop 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 shutdown_connections 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50600b5430 0x7f50600b78f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507811c0a0 0x7f50781ad9b0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f507810a530 0x7f50781a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 --2- 172.21.15.33:0/928007044 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f507810a150 0x7f50781a80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 >> 172.21.15.33:0/928007044 conn(0x7f507806dfc0 msgr2=0x7f507811c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 shutdown_connections 2024-09-10T21:22:57.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:57.455+0000 7f507edd3640 1 -- 172.21.15.33:0/928007044 wait complete. 2024-09-10T21:22:58.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 284215 40.99 MB/sec execute 238 sec latency 192.771 ms 2024-09-10T21:22:58.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:57 smithi174 ceph-mon[35635]: pgmap v915: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.5 MiB/s wr, 757 op/s 2024-09-10T21:22:58.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:57 smithi033 ceph-mon[30860]: pgmap v915: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.5 MiB/s wr, 757 op/s 2024-09-10T21:22:58.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:57 smithi104 ceph-mon[35325]: pgmap v915: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.5 MiB/s wr, 757 op/s 2024-09-10T21:22:58.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:22:59.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 285006 41.03 MB/sec execute 239 sec latency 84.219 ms 2024-09-10T21:22:59.101 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:22:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 -- 172.21.15.33:0/826151598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 msgr2=0x7f3e44108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/826151598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 0x7f3e44108c10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009900 tx=0x7f3e3802a410 comp rx=0 tx=0).stop 2024-09-10T21:22:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 -- 172.21.15.33:0/826151598 shutdown_connections 2024-09-10T21:22:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/826151598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 0x7f3e4410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/826151598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 0x7f3e44108c10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/826151598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e44102dc0 0x7f3e441031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 -- 172.21.15.33:0/826151598 >> 172.21.15.33:0/826151598 conn(0x7f3e440fe870 msgr2=0x7f3e44100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 -- 172.21.15.33:0/826151598 shutdown_connections 2024-09-10T21:22:59.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.788+0000 7f3e49d8b640 1 -- 172.21.15.33:0/826151598 wait complete. 2024-09-10T21:22:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.789+0000 7f3e49d8b640 1 Processor -- start 2024-09-10T21:22:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.789+0000 7f3e49d8b640 1 -- start start 2024-09-10T21:22:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.789+0000 7f3e49d8b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e44102dc0 0x7f3e440754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.789+0000 7f3e49d8b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 0x7f3e440759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e49d8b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 0x7f3e44075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e49d8b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112e10 con 0x7f3e441087d0 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e49d8b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112c90 con 0x7f3e44109150 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e49d8b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e44112f90 con 0x7f3e44102dc0 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 0x7f3e44075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 0x7f3e44075f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58996/0 (socket says 172.21.15.33:58996) 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 -- 172.21.15.33:0/1311478918 learned_addr learned my addr 172.21.15.33:0/1311478918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 0x7f3e440759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e437fe640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e44102dc0 0x7f3e440754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 -- 172.21.15.33:0/1311478918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e44102dc0 msgr2=0x7f3e440754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e44102dc0 0x7f3e440754a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 -- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 msgr2=0x7f3e440759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 0x7f3e440759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 -- 172.21.15.33:0/1311478918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e441a7680 con 0x7f3e44109150 2024-09-10T21:22:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.790+0000 7f3e43fff640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 0x7f3e44075f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00eb10 tx=0x7f3e3c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.791+0000 7f3e40ff9640 1 -- 172.21.15.33:0/1311478918 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c004b20 con 0x7f3e44109150 2024-09-10T21:22:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.791+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e441a7970 con 0x7f3e44109150 2024-09-10T21:22:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.791+0000 7f3e40ff9640 1 -- 172.21.15.33:0/1311478918 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3c004cc0 con 0x7f3e44109150 2024-09-10T21:22:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.791+0000 7f3e40ff9640 1 -- 172.21.15.33:0/1311478918 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3c010670 con 0x7f3e44109150 2024-09-10T21:22:59.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.791+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e441a7e20 con 0x7f3e44109150 2024-09-10T21:22:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.794+0000 7f3e40ff9640 1 -- 172.21.15.33:0/1311478918 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3e3c0108f0 con 0x7f3e44109150 2024-09-10T21:22:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.794+0000 7f3e4c160740 1 Processor -- start 2024-09-10T21:22:59.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.794+0000 7f3e4c160740 1 -- start start 2024-09-10T21:22:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.794+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c494d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c55460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c57d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e4c160740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3c0bbfea0 con 0x55d3c0c559a0 2024-09-10T21:22:59.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e4c160740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d3c0baf2d0 con 0x55d3c0c490f0 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e4c160740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d3c0bb30e0 con 0x55d3c0c49a10 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e42ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e42ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59006/0 (socket says 172.21.15.33:59006) 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e42ffd640 1 -- 172.21.15.33:0/2474133960 learned_addr learned my addr 172.21.15.33:0/2474133960 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e437fe640 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c55460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c57d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e437fe640 1 -- 172.21.15.33:0/2474133960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 msgr2=0x55d3c0c494d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e437fe640 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c494d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e437fe640 1 -- 172.21.15.33:0/2474133960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 msgr2=0x55d3c0c57d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e437fe640 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c57d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e437fe640 1 -- 172.21.15.33:0/2474133960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c090d650 con 0x55d3c0c49a10 2024-09-10T21:22:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.795+0000 7f3e40ff9640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3e2c0b5360 0x7f3e2c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.796+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3e2c0b5360 0x7f3e2c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.796+0000 7f3e40ff9640 1 -- 172.21.15.33:0/1311478918 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3e3c0e6290 con 0x7f3e44109150 2024-09-10T21:22:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.796+0000 7f3e437fe640 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c55460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009ac0 tx=0x7f3e34009f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b77c9dd1ceecb7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.796+0000 7f3e29ffb640 1 -- 172.21.15.33:0/2474133960 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400ecb0 con 0x55d3c0c49a10 2024-09-10T21:22:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.796+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3e2c0b5360 0x7f3e2c0b7820 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005c40 tx=0x7f3e38005780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.796+0000 7f3e29ffb640 1 -- 172.21.15.33:0/2474133960 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3400ee50 con 0x55d3c0c49a10 2024-09-10T21:22:59.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.796+0000 7f3e29ffb640 1 -- 172.21.15.33:0/2474133960 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34011470 con 0x55d3c0c49a10 2024-09-10T21:22:59.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 -- 172.21.15.33:0/2474133960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 msgr2=0x55d3c0c55460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c55460 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3e34009ac0 tx=0x7f3e34009f80 comp rx=0 tx=0).stop 2024-09-10T21:22:59.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 -- 172.21.15.33:0/2474133960 shutdown_connections 2024-09-10T21:22:59.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c57d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c55460 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 --2- 172.21.15.33:0/2474133960 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c494d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 -- 172.21.15.33:0/2474133960 >> 172.21.15.33:0/2474133960 conn(0x55d3c0c46f20 msgr2=0x55d3c0c480c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 -- 172.21.15.33:0/2474133960 shutdown_connections 2024-09-10T21:22:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 -- 172.21.15.33:0/2474133960 wait complete. 2024-09-10T21:22:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.798+0000 7f3e4c160740 1 -- start start 2024-09-10T21:22:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e4c160740 1 Processor -- start 2024-09-10T21:22:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c55210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e4c160740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d3c0a10700 con 0x55d3c0c49a10 2024-09-10T21:22:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e4c160740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d3c0b4a160 con 0x55d3c0c559a0 2024-09-10T21:22:59.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e4c160740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d3c0c58620 con 0x55d3c0c490f0 2024-09-10T21:22:59.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60294/0 (socket says 172.21.15.33:60294) 2024-09-10T21:22:59.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e437fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c55210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e42ffd640 1 -- 172.21.15.33:0/952237700 learned_addr learned my addr 172.21.15.33:0/952237700 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:59.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.799+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.800+0000 7f3e4c160740 10 client.0 _ll_register_callbacks cb 0x55d3c0b639e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:59.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.800+0000 7f3e42ffd640 1 -- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 msgr2=0x55d3c0c55210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.800+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c55210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.800+0000 7f3e42ffd640 1 -- 172.21.15.33:0/952237700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 msgr2=0x55d3c0c52be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.800+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.800+0000 7f3e42ffd640 1 -- 172.21.15.33:0/952237700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c0b77c90 con 0x55d3c0c49a10 2024-09-10T21:22:59.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.800+0000 7f3e437fe640 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c55210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:59.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52660 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009230 tx=0x7f3e3802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d27263ab61c77f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:59.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:22:59.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e22ffd640 1 -- 172.21.15.33:0/952237700 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3804f030 con 0x55d3c0c49a10 2024-09-10T21:22:59.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3400ee50 con 0x55d3c0c49a10 2024-09-10T21:22:59.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e22ffd640 1 -- 172.21.15.33:0/952237700 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3e38004250 con 0x55d3c0c49a10 2024-09-10T21:22:59.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e22ffd640 1 -- 172.21.15.33:0/952237700 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3e38035040 con 0x55d3c0c49a10 2024-09-10T21:22:59.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e4c160740 10 client.18819 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:59.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.801+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d3c0c63f00 con 0x55d3c0c49a10 2024-09-10T21:22:59.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e22ffd640 1 -- 172.21.15.33:0/952237700 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3400ee50 con 0x55d3c0c49a10 2024-09-10T21:22:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e22ffd640 1 -- 172.21.15.33:0/952237700 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f3e38048050 con 0x55d3c0c49a10 2024-09-10T21:22:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e4c160740 10 client.18819 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e4c160740 10 client.18819 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:22:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e4c160740 20 client.18819 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e4c160740 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0c65840 0x55d3c0c67d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e4c160740 4 client.18819 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:22:59.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3c0bc1df0 con 0x55d3c0c65840 2024-09-10T21:22:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.802+0000 7f3e437fe640 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0c65840 0x55d3c0c67d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.803+0000 7f3e22ffd640 10 client.18819 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:59.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.803+0000 7f3e437fe640 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0c65840 0x55d3c0c67d00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1f4b908bb6cfce39 server_cookie=f1b830d1a03d1b23 in_seq=0 out_seq=0 2024-09-10T21:22:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.806+0000 7f3e22ffd640 1 -- 172.21.15.33:0/952237700 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3c0bc1df0 con 0x55d3c0c65840 2024-09-10T21:22:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.806+0000 7f3e22ffd640 10 client.18819 handle_command_reply: tid=0 2024-09-10T21:22:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.806+0000 7f3e4c160740 1 client.18819 shutdown 2024-09-10T21:22:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 msgr2=0x55d3c0c52660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52660 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009230 tx=0x7f3e3802a410 comp rx=0 tx=0).stop 2024-09-10T21:22:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 shutdown_connections 2024-09-10T21:22:59.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52660 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0c65840 0x55d3c0c67d00 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 --2- 172.21.15.33:0/952237700 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c55210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 >> 172.21.15.33:0/952237700 conn(0x55d3c0c46f20 msgr2=0x55d3c0c4ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e22ffd640 0 client.18819 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 shutdown_connections 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 1 -- 172.21.15.33:0/952237700 wait complete. 2024-09-10T21:22:59.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.807+0000 7f3e4c160740 20 client.18819 trim_cache size 0 max 16384 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 Processor -- start 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 -- start start 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c61850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b05b0 con 0x55d3c0c49a10 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b20f0 con 0x55d3c0c559a0 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e4c160740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b0c80 con 0x55d3c0c490f0 2024-09-10T21:22:59.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.822+0000 7f3e42ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60310/0 (socket says 172.21.15.33:60310) 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e42ffd640 1 -- 172.21.15.33:0/3347291718 learned_addr learned my addr 172.21.15.33:0/3347291718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e437fe640 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c61850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e42ffd640 1 -- 172.21.15.33:0/3347291718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 msgr2=0x55d3c0c61850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c61850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e42ffd640 1 -- 172.21.15.33:0/3347291718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 msgr2=0x55d3c0c52b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e42ffd640 1 -- 172.21.15.33:0/3347291718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3c0c477a0 con 0x55d3c0c49a10 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e43fff640 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:22:59.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52630 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009850 tx=0x7f3e38005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11afe3ab9e58e5de server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e297fa640 1 -- 172.21.15.33:0/3347291718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3804f040 con 0x55d3c0c49a10 2024-09-10T21:22:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e297fa640 1 -- 172.21.15.33:0/3347291718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3804b480 con 0x55d3c0c49a10 2024-09-10T21:22:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.823+0000 7f3e297fa640 1 -- 172.21.15.33:0/3347291718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3805f460 con 0x55d3c0c49a10 2024-09-10T21:22:59.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 -- 172.21.15.33:0/3347291718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 msgr2=0x55d3c0c52630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52630 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009850 tx=0x7f3e38005900 comp rx=0 tx=0).stop 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 -- 172.21.15.33:0/3347291718 shutdown_connections 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c559a0 0x55d3c0c52b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c49a10 0x55d3c0c52630 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 --2- 172.21.15.33:0/3347291718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c490f0 0x55d3c0c61850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 -- 172.21.15.33:0/3347291718 >> 172.21.15.33:0/3347291718 conn(0x55d3c0c46f20 msgr2=0x55d3c0c4d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 -- 172.21.15.33:0/3347291718 shutdown_connections 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.824+0000 7f3e4c160740 1 -- 172.21.15.33:0/3347291718 wait complete. 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 -- start start 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 Processor -- start 2024-09-10T21:22:59.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c5de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c6cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c6d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b1a20 con 0x55d3c0c559a0 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441b1350 con 0x55d3c0c490f0 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e441ae570 con 0x55d3c0c49a10 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.825+0000 7f3e4c160740 10 client.0 _ll_register_callbacks cb 0x7f3e38041050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c5de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e43fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c6d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c5de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59038/0 (socket says 172.21.15.33:59038) 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 -- 172.21.15.33:0/1703685410 learned_addr learned my addr 172.21.15.33:0/1703685410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c6cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 -- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 msgr2=0x55d3c0c6cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c6cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 -- 172.21.15.33:0/1703685410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 msgr2=0x55d3c0c6d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c6d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 -- 172.21.15.33:0/1703685410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3c0c68070 con 0x55d3c0c490f0 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e437fe640 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c5de60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400e780 tx=0x7f3e34005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f44b309117bba1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e22ffd640 1 -- 172.21.15.33:0/1703685410 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3400ee50 con 0x55d3c0c490f0 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e22ffd640 1 -- 172.21.15.33:0/1703685410 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f3e340050d0 con 0x55d3c0c490f0 2024-09-10T21:22:59.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.826+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e3804b480 con 0x55d3c0c490f0 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e22ffd640 1 -- 172.21.15.33:0/1703685410 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f3e340052d0 con 0x55d3c0c490f0 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 10 client.28208 fetch_fsmap learned FSMap version 81 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d3c0c5a590 con 0x55d3c0c490f0 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e22ffd640 1 -- 172.21.15.33:0/1703685410 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e3804b480 con 0x55d3c0c490f0 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e22ffd640 1 -- 172.21.15.33:0/1703685410 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f3e3400f040 con 0x55d3c0c490f0 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 10 client.28208 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 10 client.28208 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 20 client.28208 populate_metadata read hostname 'smithi033' 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0cd2a80 0x55d3c0cd4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 4 client.28208 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.827+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3c0bbd8a0 con 0x55d3c0cd2a80 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.828+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0cd2a80 0x55d3c0cd4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:22:59.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.828+0000 7f3e22ffd640 10 client.28208 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:59.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.828+0000 7f3e42ffd640 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0cd2a80 0x55d3c0cd4ea0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=331c2a762fdfe33 server_cookie=e647da99d5a57b37 in_seq=0 out_seq=0 2024-09-10T21:22:59.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.835+0000 7f3e22ffd640 1 -- 172.21.15.33:0/1703685410 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3c0bbd8a0 con 0x55d3c0cd2a80 2024-09-10T21:22:59.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.835+0000 7f3e22ffd640 10 client.28208 handle_command_reply: tid=0 2024-09-10T21:22:59.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.835+0000 7f3e4c160740 1 client.28208 shutdown 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 msgr2=0x55d3c0c5de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c5de60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3e3400e780 tx=0x7f3e34005c50 comp rx=0 tx=0).stop 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 shutdown_connections 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d3c0c559a0 0x55d3c0c6d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55d3c0cd2a80 0x55d3c0cd4ea0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d3c0c49a10 0x55d3c0c6cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 --2- 172.21.15.33:0/1703685410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d3c0c490f0 0x55d3c0c5de60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e22ffd640 0 client.28208 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 >> 172.21.15.33:0/1703685410 conn(0x55d3c0c46f20 msgr2=0x55d3c0c525d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 shutdown_connections 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 1 -- 172.21.15.33:0/1703685410 wait complete. 2024-09-10T21:22:59.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.836+0000 7f3e4c160740 20 client.28208 trim_cache size 0 max 16384 2024-09-10T21:22:59.841 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:22:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3e2c0b5360 msgr2=0x7f3e2c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3e2c0b5360 0x7f3e2c0b7820 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3e38005c40 tx=0x7f3e38005780 comp rx=0 tx=0).stop 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 msgr2=0x7f3e44075f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 0x7f3e44075f20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3e3c00eb10 tx=0x7f3e3c00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 shutdown_connections 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3e2c0b5360 0x7f3e2c0b7820 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e44109150 0x7f3e44075f20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e441087d0 0x7f3e440759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 --2- 172.21.15.33:0/1311478918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e44102dc0 0x7f3e440754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 >> 172.21.15.33:0/1311478918 conn(0x7f3e440fe870 msgr2=0x7f3e44100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 shutdown_connections 2024-09-10T21:22:59.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:22:59.837+0000 7f3e49d8b640 1 -- 172.21.15.33:0/1311478918 wait complete. 2024-09-10T21:23:00.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 285664 40.97 MB/sec execute 240 sec latency 87.701 ms 2024-09-10T21:23:00.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:22:59 smithi174 ceph-mon[35635]: pgmap v916: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 840 op/s 2024-09-10T21:23:00.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:22:59 smithi033 ceph-mon[30860]: pgmap v916: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 840 op/s 2024-09-10T21:23:00.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:22:59 smithi104 ceph-mon[35325]: pgmap v916: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 840 op/s 2024-09-10T21:23:00.590 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:23:00.944 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:01.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 286462 41.00 MB/sec execute 241 sec latency 80.065 ms 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 -- 172.21.15.33:0/828187172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc108040 msgr2=0x7efffc10e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 --2- 172.21.15.33:0/828187172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc108040 0x7efffc10e840 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7efff4009ca0 tx=0x7efff402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 -- 172.21.15.33:0/828187172 shutdown_connections 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 --2- 172.21.15.33:0/828187172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc108040 0x7efffc10e840 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 --2- 172.21.15.33:0/828187172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc1076c0 0x7efffc107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 --2- 172.21.15.33:0/828187172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc101cb0 0x7efffc102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 -- 172.21.15.33:0/828187172 >> 172.21.15.33:0/828187172 conn(0x7efffc0fd740 msgr2=0x7efffc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 -- 172.21.15.33:0/828187172 shutdown_connections 2024-09-10T21:23:01.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.618+0000 7f0000a80640 1 -- 172.21.15.33:0/828187172 wait complete. 2024-09-10T21:23:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.619+0000 7f0000a80640 1 Processor -- start 2024-09-10T21:23:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.619+0000 7f0000a80640 1 -- start start 2024-09-10T21:23:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7f0000a80640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc101cb0 0x7efffc19a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7f0000a80640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc1076c0 0x7efffc19a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7f0000a80640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc108040 0x7efffc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7f0000a80640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efffc111d50 con 0x7efffc1076c0 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7f0000a80640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efffc111bd0 con 0x7efffc108040 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7f0000a80640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efffc111ed0 con 0x7efffc101cb0 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc108040 0x7efffc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc108040 0x7efffc19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59066/0 (socket says 172.21.15.33:59066) 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 -- 172.21.15.33:0/914060861 learned_addr learned my addr 172.21.15.33:0/914060861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffa59c640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc101cb0 0x7efffc19a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efff9d9b640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc1076c0 0x7efffc19a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 -- 172.21.15.33:0/914060861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc101cb0 msgr2=0x7efffc19a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc101cb0 0x7efffc19a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 -- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc1076c0 msgr2=0x7efffc19a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc1076c0 0x7efffc19a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efffad9d640 1 -- 172.21.15.33:0/914060861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc197f10 con 0x7efffc108040 2024-09-10T21:23:01.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.620+0000 7efff9d9b640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc1076c0 0x7efffc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.621+0000 7efffad9d640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc108040 0x7efffc19aac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efff402f540 tx=0x7efff40316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.621+0000 7effeb7fe640 1 -- 172.21.15.33:0/914060861 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff4003b70 con 0x7efffc108040 2024-09-10T21:23:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.621+0000 7effeb7fe640 1 -- 172.21.15.33:0/914060861 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff4003d10 con 0x7efffc108040 2024-09-10T21:23:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.621+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc1981a0 con 0x7efffc108040 2024-09-10T21:23:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.621+0000 7effeb7fe640 1 -- 172.21.15.33:0/914060861 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff402f990 con 0x7efffc108040 2024-09-10T21:23:01.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.621+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc198730 con 0x7efffc108040 2024-09-10T21:23:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.623+0000 7effeb7fe640 1 -- 172.21.15.33:0/914060861 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7efff402fb30 con 0x7efffc108040 2024-09-10T21:23:01.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 Processor -- start 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 -- start start 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea36394d0 0x559ea36398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3645840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3648170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ea35afea0 con 0x559ea36394d0 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ea359f2d0 con 0x559ea3645d80 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.624+0000 7f0002e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ea35a30e0 con 0x559ea3639df0 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea36394d0 0x559ea36398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea36394d0 0x559ea36398b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60334/0 (socket says 172.21.15.33:60334) 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 -- 172.21.15.33:0/3877971967 learned_addr learned my addr 172.21.15.33:0/3877971967 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3648170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7effeb7fe640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe40b53e0 0x7effe40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 -- 172.21.15.33:0/3877971967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 msgr2=0x559ea3645840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3645840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 -- 172.21.15.33:0/3877971967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 msgr2=0x559ea3648170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3648170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 -- 172.21.15.33:0/3877971967 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea32fd650 con 0x559ea36394d0 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efffad9d640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe40b53e0 0x7effe40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7efff9d9b640 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea36394d0 0x559ea36398b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efff0013800 tx=0x7efff00096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ee835e5c2ecd0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.625+0000 7effeb7fe640 1 -- 172.21.15.33:0/914060861 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efff4107000 con 0x7efffc108040 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.626+0000 7effe8ff9640 1 -- 172.21.15.33:0/3877971967 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0013d80 con 0x559ea36394d0 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.626+0000 7effe8ff9640 1 -- 172.21.15.33:0/3877971967 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff00130d0 con 0x559ea36394d0 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.626+0000 7efffad9d640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe40b53e0 0x7effe40b78a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efff40c7db0 tx=0x7efff40d1040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.626+0000 7effe8ff9640 1 -- 172.21.15.33:0/3877971967 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff000fcd0 con 0x559ea36394d0 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.626+0000 7f0002e55740 1 -- 172.21.15.33:0/3877971967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea36394d0 msgr2=0x559ea36398b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.626+0000 7f0002e55740 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea36394d0 0x559ea36398b0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7efff0013800 tx=0x7efff00096e0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 -- 172.21.15.33:0/3877971967 shutdown_connections 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3648170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3645840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 --2- 172.21.15.33:0/3877971967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea36394d0 0x559ea36398b0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 -- 172.21.15.33:0/3877971967 >> 172.21.15.33:0/3877971967 conn(0x559ea3637300 msgr2=0x559ea36384a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 -- 172.21.15.33:0/3877971967 shutdown_connections 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 -- 172.21.15.33:0/3877971967 wait complete. 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 -- start start 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.627+0000 7f0002e55740 1 Processor -- start 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7f0002e55740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea3645670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7f0002e55740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3642ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7f0002e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3645d80 0x559ea36430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7f0002e55740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559ea3400700 con 0x559ea3639df0 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7f0002e55740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559ea353a160 con 0x559ea36394d0 2024-09-10T21:23:01.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7f0002e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ea36489a0 con 0x559ea3645d80 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3642ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7f0002e55740 10 client.0 _ll_register_callbacks cb 0x559ea35539e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3642ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60336/0 (socket says 172.21.15.33:60336) 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 -- 172.21.15.33:0/21189762 learned_addr learned my addr 172.21.15.33:0/21189762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efff9d9b640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea3645670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efff9d9b640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3645d80 0x559ea36430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 -- 172.21.15.33:0/21189762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3645d80 msgr2=0x559ea36430f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3645d80 0x559ea36430f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 -- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 msgr2=0x559ea3645670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea3645670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.628+0000 7efffa59c640 1 -- 172.21.15.33:0/21189762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea3567c90 con 0x559ea3639df0 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7efff9d9b640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3645d80 0x559ea36430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:01.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7efffa59c640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3642ac0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7effec0116c0 tx=0x7effec011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9350c4b06a018f2c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7effe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7effe1ffb640 1 -- 172.21.15.33:0/21189762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec004460 con 0x559ea3639df0 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff00130d0 con 0x559ea3639df0 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7effe1ffb640 1 -- 172.21.15.33:0/21189762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7effec00f920 con 0x559ea3639df0 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7effe1ffb640 1 -- 172.21.15.33:0/21189762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7effec00fb20 con 0x559ea3639df0 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7f0002e55740 10 client.18837 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.629+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x559ea36543b0 con 0x559ea3639df0 2024-09-10T21:23:01.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7effe1ffb640 1 -- 172.21.15.33:0/21189762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff00130d0 con 0x559ea3639df0 2024-09-10T21:23:01.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7effe1ffb640 1 -- 172.21.15.33:0/21189762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7effec018070 con 0x559ea3639df0 2024-09-10T21:23:01.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7f0002e55740 10 client.18837 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7f0002e55740 10 client.18837 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7f0002e55740 20 client.18837 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7f0002e55740 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea3657bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7f0002e55740 4 client.18837 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ea35b1df0 con 0x559ea36557c0 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.630+0000 7efff9d9b640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea3657bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.631+0000 7effe1ffb640 10 client.18837 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.631+0000 7efff9d9b640 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea3657bb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=606a6d3254a79d3 server_cookie=9f691f275f56a6fa in_seq=0 out_seq=0 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7effe1ffb640 1 -- 172.21.15.33:0/21189762 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559ea35b1df0 con 0x559ea36557c0 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7effe1ffb640 10 client.18837 handle_command_reply: tid=0 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 client.18837 shutdown 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 msgr2=0x559ea3642ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3642ac0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7effec0116c0 tx=0x7effec011b80 comp rx=0 tx=0).stop 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 shutdown_connections 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3645d80 0x559ea36430f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea3657bb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7effe1ffb640 0 client.18837 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3642ac0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 --2- 172.21.15.33:0/21189762 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea3645670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 >> 172.21.15.33:0/21189762 conn(0x559ea3637300 msgr2=0x559ea363ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 shutdown_connections 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 1 -- 172.21.15.33:0/21189762 wait complete. 2024-09-10T21:23:01.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.632+0000 7f0002e55740 20 client.18837 trim_cache size 0 max 16384 2024-09-10T21:23:01.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 Processor -- start 2024-09-10T21:23:01.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 -- start start 2024-09-10T21:23:01.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea36394d0 0x559ea3638ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3651c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3647170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b0510 con 0x559ea3639df0 2024-09-10T21:23:01.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b2050 con 0x559ea3645d80 2024-09-10T21:23:01.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.646+0000 7f0002e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b0be0 con 0x559ea36394d0 2024-09-10T21:23:01.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea36394d0 0x559ea3638ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea36394d0 0x559ea3638ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55642/0 (socket says 172.21.15.33:55642) 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 -- 172.21.15.33:0/4072975232 learned_addr learned my addr 172.21.15.33:0/4072975232 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efffa59c640 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3651c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3647170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 -- 172.21.15.33:0/4072975232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 msgr2=0x559ea3647170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3647170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 -- 172.21.15.33:0/4072975232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 msgr2=0x559ea3651c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3651c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.647+0000 7efff9d9b640 1 -- 172.21.15.33:0/4072975232 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ea3637d80 con 0x559ea36394d0 2024-09-10T21:23:01.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.648+0000 7efff9d9b640 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea36394d0 0x559ea3638ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efff00059f0 tx=0x7efff0024040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3838e8c95f9323b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.648+0000 7effe3fff640 1 -- 172.21.15.33:0/4072975232 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff000fd90 con 0x559ea36394d0 2024-09-10T21:23:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.648+0000 7effe3fff640 1 -- 172.21.15.33:0/4072975232 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0005040 con 0x559ea36394d0 2024-09-10T21:23:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.648+0000 7effe3fff640 1 -- 172.21.15.33:0/4072975232 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff00048c0 con 0x559ea36394d0 2024-09-10T21:23:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 -- 172.21.15.33:0/4072975232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea36394d0 msgr2=0x559ea3638ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea36394d0 0x559ea3638ad0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7efff00059f0 tx=0x7efff0024040 comp rx=0 tx=0).stop 2024-09-10T21:23:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 -- 172.21.15.33:0/4072975232 shutdown_connections 2024-09-10T21:23:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea3645d80 0x559ea3647170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3639df0 0x559ea3651c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 --2- 172.21.15.33:0/4072975232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea36394d0 0x559ea3638ad0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 -- 172.21.15.33:0/4072975232 >> 172.21.15.33:0/4072975232 conn(0x559ea3637300 msgr2=0x559ea36383c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 -- 172.21.15.33:0/4072975232 shutdown_connections 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.649+0000 7f0002e55740 1 -- 172.21.15.33:0/4072975232 wait complete. 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.650+0000 7f0002e55740 1 -- start start 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.650+0000 7f0002e55740 1 Processor -- start 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.650+0000 7f0002e55740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea364e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7f0002e55740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3648c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7f0002e55740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3645d80 0x559ea3649310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7f0002e55740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b1980 con 0x559ea3645d80 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7f0002e55740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1b12b0 con 0x559ea36394d0 2024-09-10T21:23:01.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7f0002e55740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efffc1ae4f0 con 0x559ea3639df0 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7f0002e55740 10 client.0 _ll_register_callbacks cb 0x7effec009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea364e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea364e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59102/0 (socket says 172.21.15.33:59102) 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 -- 172.21.15.33:0/908187226 learned_addr learned my addr 172.21.15.33:0/908187226 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3645d80 0x559ea3649310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efffa59c640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3648c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 -- 172.21.15.33:0/908187226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 msgr2=0x559ea3648c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3648c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 -- 172.21.15.33:0/908187226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3645d80 msgr2=0x559ea3649310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3645d80 0x559ea3649310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efff9d9b640 1 -- 172.21.15.33:0/908187226 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ea36c2660 con 0x559ea36394d0 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.651+0000 7efffa59c640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3648c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7efff9d9b640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea364e5c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efff0013250 tx=0x7efff00133a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf0aa7ad4ce5a584 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7effe1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7effe1ffb640 1 -- 172.21.15.33:0/908187226 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff00054b0 con 0x559ea36394d0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff0005040 con 0x559ea36394d0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7effe1ffb640 1 -- 172.21.15.33:0/908187226 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7efff0004020 con 0x559ea36394d0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7effe1ffb640 1 -- 172.21.15.33:0/908187226 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7efff0005650 con 0x559ea36394d0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7f0002e55740 10 client.28226 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x559ea36c3870 con 0x559ea36394d0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7effe1ffb640 1 -- 172.21.15.33:0/908187226 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efff0005040 con 0x559ea36394d0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.652+0000 7effe1ffb640 1 -- 172.21.15.33:0/908187226 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7efff00035f0 con 0x559ea36394d0 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.653+0000 7f0002e55740 10 client.28226 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.653+0000 7f0002e55740 10 client.28226 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.653+0000 7f0002e55740 20 client.28226 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.653+0000 7f0002e55740 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea36c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.653+0000 7f0002e55740 4 client.28226 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:01.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.653+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559ea35ad8a0 con 0x559ea36557c0 2024-09-10T21:23:01.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.653+0000 7efffa59c640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea36c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7efffa59c640 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea36c5d30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b697ada465e309 server_cookie=90d25428fddfaef2 in_seq=0 out_seq=0 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7effe1ffb640 10 client.28226 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7effe1ffb640 1 -- 172.21.15.33:0/908187226 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559ea35ad8a0 con 0x559ea36557c0 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7effe1ffb640 10 client.28226 handle_command_reply: tid=0 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 client.28226 shutdown 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 msgr2=0x559ea364e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea364e5c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7efff0013250 tx=0x7efff00133a0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 shutdown_connections 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559ea36557c0 0x559ea36c5d30 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559ea3645d80 0x559ea3649310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ea3639df0 0x559ea3648c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 --2- 172.21.15.33:0/908187226 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559ea36394d0 0x559ea364e5c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7effe1ffb640 0 client.28226 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 >> 172.21.15.33:0/908187226 conn(0x559ea3637300 msgr2=0x559ea3635c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 shutdown_connections 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.654+0000 7f0002e55740 1 -- 172.21.15.33:0/908187226 wait complete. 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0002e55740 20 client.28226 trim_cache size 0 max 16384 2024-09-10T21:23:01.662 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:01.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe40b53e0 msgr2=0x7effe40b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe40b53e0 0x7effe40b78a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7efff40c7db0 tx=0x7efff40d1040 comp rx=0 tx=0).stop 2024-09-10T21:23:01.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc108040 msgr2=0x7efffc19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:01.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc108040 0x7efffc19aac0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7efff402f540 tx=0x7efff40316f0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 shutdown_connections 2024-09-10T21:23:01.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7effe40b53e0 0x7effe40b78a0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efffc108040 0x7efffc19aac0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efffc1076c0 0x7efffc19a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 --2- 172.21.15.33:0/914060861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efffc101cb0 0x7efffc19a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:01.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.655+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 >> 172.21.15.33:0/914060861 conn(0x7efffc0fd740 msgr2=0x7efffc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:01.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.656+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 shutdown_connections 2024-09-10T21:23:01.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:01.656+0000 7f0000a80640 1 -- 172.21.15.33:0/914060861 wait complete. 2024-09-10T21:23:02.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 286853 40.90 MB/sec execute 242 sec latency 297.020 ms 2024-09-10T21:23:02.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:01 smithi174 ceph-mon[35635]: pgmap v917: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-10T21:23:02.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:01 smithi033 ceph-mon[30860]: pgmap v917: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-10T21:23:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:01 smithi104 ceph-mon[35325]: pgmap v917: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-10T21:23:03.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 287505 40.89 MB/sec execute 243 sec latency 156.174 ms 2024-09-10T21:23:03.402 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:23:03.719 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:04.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 288359 40.92 MB/sec execute 244 sec latency 104.093 ms 2024-09-10T21:23:04.097 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:03 smithi033 ceph-mon[30860]: pgmap v918: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.6 MiB/s wr, 746 op/s 2024-09-10T21:23:04.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:03 smithi174 ceph-mon[35635]: pgmap v918: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.6 MiB/s wr, 746 op/s 2024-09-10T21:23:04.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:03 smithi104 ceph-mon[35325]: pgmap v918: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.6 MiB/s wr, 746 op/s 2024-09-10T21:23:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 -- 172.21.15.33:0/1192262911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 msgr2=0x7f00fc1117f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 --2- 172.21.15.33:0/1192262911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 0x7f00fc1117f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009ca0 tx=0x7f00f402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 -- 172.21.15.33:0/1192262911 shutdown_connections 2024-09-10T21:23:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 --2- 172.21.15.33:0/1192262911 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 0x7f00fc1117f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 --2- 172.21.15.33:0/1192262911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075f40 0x7f00fc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 --2- 172.21.15.33:0/1192262911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00fc075540 0x7f00fc075920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 -- 172.21.15.33:0/1192262911 >> 172.21.15.33:0/1192262911 conn(0x7f00fc0fe830 msgr2=0x7f00fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 -- 172.21.15.33:0/1192262911 shutdown_connections 2024-09-10T21:23:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.382+0000 7f0103f75640 1 -- 172.21.15.33:0/1192262911 wait complete. 2024-09-10T21:23:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 Processor -- start 2024-09-10T21:23:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 -- start start 2024-09-10T21:23:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075540 0x7f00fc19b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00fc075f40 0x7f00fc19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 0x7f00fc1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc114f70 con 0x7f00fc075540 2024-09-10T21:23:04.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc114df0 con 0x7f00fc10f400 2024-09-10T21:23:04.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0103f75640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1150f0 con 0x7f00fc075f40 2024-09-10T21:23:04.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0101d11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075540 0x7f00fc19b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0101d11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075540 0x7f00fc19b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60360/0 (socket says 172.21.15.33:60360) 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0101d11640 1 -- 172.21.15.33:0/920988561 learned_addr learned my addr 172.21.15.33:0/920988561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.383+0000 7f0102512640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 0x7f00fc1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.384+0000 7f0101510640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00fc075f40 0x7f00fc19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.384+0000 7f0101d11640 1 -- 172.21.15.33:0/920988561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00fc075f40 msgr2=0x7f00fc19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.384+0000 7f0101d11640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00fc075f40 0x7f00fc19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.384+0000 7f0101d11640 1 -- 172.21.15.33:0/920988561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 msgr2=0x7f00fc1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.384+0000 7f0101d11640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 0x7f00fc1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.384+0000 7f0101d11640 1 -- 172.21.15.33:0/920988561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc1a1530 con 0x7f00fc075540 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.384+0000 7f0101d11640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075540 0x7f00fc19b480 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9f0 tx=0x7f00f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.385+0000 7f00f2ffd640 1 -- 172.21.15.33:0/920988561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f800cd80 con 0x7f00fc075540 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.385+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc1a1820 con 0x7f00fc075540 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.385+0000 7f00f2ffd640 1 -- 172.21.15.33:0/920988561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f80108e0 con 0x7f00fc075540 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.385+0000 7f00f2ffd640 1 -- 172.21.15.33:0/920988561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8018980 con 0x7f00fc075540 2024-09-10T21:23:04.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.385+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc1a80b0 con 0x7f00fc075540 2024-09-10T21:23:04.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.388+0000 7f00f2ffd640 1 -- 172.21.15.33:0/920988561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f00f8010430 con 0x7f00fc075540 2024-09-10T21:23:04.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.389+0000 7f00f2ffd640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f00e80b52e0 0x7f00e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.389+0000 7f0101510640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f00e80b52e0 0x7f00e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.389+0000 7f010634a740 1 Processor -- start 2024-09-10T21:23:04.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.389+0000 7f00f2ffd640 1 -- 172.21.15.33:0/920988561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f00f80e61d0 con 0x7f00fc075540 2024-09-10T21:23:04.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f010634a740 1 -- start start 2024-09-10T21:23:04.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f010634a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42600c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f010634a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce4260600 0x557ce426c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f010634a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce426e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f010634a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ce41cdea0 con 0x557ce425fce0 2024-09-10T21:23:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f010634a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ce41bd2d0 con 0x557ce4260600 2024-09-10T21:23:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f010634a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ce41c10e0 con 0x557ce426c590 2024-09-10T21:23:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f0101510640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f00e80b52e0 0x7f00e80b77a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00fc1010e0 tx=0x7f00ec005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f0101d11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f0101510640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce4260600 0x557ce426c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f0101d11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60368/0 (socket says 172.21.15.33:60368) 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f0101d11640 1 -- 172.21.15.33:0/2414655127 learned_addr learned my addr 172.21.15.33:0/2414655127 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.390+0000 7f0102512640 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce426e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0101d11640 1 -- 172.21.15.33:0/2414655127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 msgr2=0x557ce426e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0101d11640 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce426e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0101d11640 1 -- 172.21.15.33:0/2414655127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce4260600 msgr2=0x557ce426c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0101d11640 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce4260600 0x557ce426c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0101d11640 1 -- 172.21.15.33:0/2414655127 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce3f1b650 con 0x557ce425fce0 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0101510640 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce4260600 0x557ce426c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0102512640 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce426e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f0101d11640 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42600c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f00f80ac3e0 tx=0x7f00f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc9117ccc1317d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f00e7fff640 1 -- 172.21.15.33:0/2414655127 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80b1020 con 0x557ce425fce0 2024-09-10T21:23:04.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f00e7fff640 1 -- 172.21.15.33:0/2414655127 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f80ac990 con 0x557ce425fce0 2024-09-10T21:23:04.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.391+0000 7f00e7fff640 1 -- 172.21.15.33:0/2414655127 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f8018b20 con 0x557ce425fce0 2024-09-10T21:23:04.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 -- 172.21.15.33:0/2414655127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 msgr2=0x557ce42600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42600c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f00f80ac3e0 tx=0x7f00f8000bc0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 -- 172.21.15.33:0/2414655127 shutdown_connections 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce426e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce4260600 0x557ce426c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 --2- 172.21.15.33:0/2414655127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42600c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 -- 172.21.15.33:0/2414655127 >> 172.21.15.33:0/2414655127 conn(0x557ce425db10 msgr2=0x557ce425ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 -- 172.21.15.33:0/2414655127 shutdown_connections 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.392+0000 7f010634a740 1 -- 172.21.15.33:0/2414655127 wait complete. 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 -- start start 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 Processor -- start 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce426bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce4260600 0x557ce4269350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce42699d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ce401e700 con 0x557ce4260600 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ce4158160 con 0x557ce425fce0 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ce426f240 con 0x557ce426c590 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f010634a740 10 client.0 _ll_register_callbacks cb 0x557ce41719e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f0101510640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce4260600 0x557ce4269350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f0101510640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce4260600 0x557ce4269350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60380/0 (socket says 172.21.15.33:60380) 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f0101d11640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce426bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.393+0000 7f0101510640 1 -- 172.21.15.33:0/1965087850 learned_addr learned my addr 172.21.15.33:0/1965087850 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0101510640 1 -- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 msgr2=0x557ce42699d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0102512640 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce42699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0101510640 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce42699d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0101510640 1 -- 172.21.15.33:0/1965087850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 msgr2=0x557ce426bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:04.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0101510640 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce426bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0101510640 1 -- 172.21.15.33:0/1965087850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce4185c90 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0101d11640 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce426bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f0101510640 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce4260600 0x557ce4269350 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f00ec003870 tx=0x7f00ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3405d49105765d93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.394+0000 7f00e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f00e57fa640 1 -- 172.21.15.33:0/1965087850 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ec00e6a0 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f00e57fa640 1 -- 172.21.15.33:0/1965087850 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f00ec00e840 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00f80ac990 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f00e57fa640 1 -- 172.21.15.33:0/1965087850 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f00ec00ea40 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 10 client.18855 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x557ce427acb0 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f00e57fa640 1 -- 172.21.15.33:0/1965087850 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f80ac990 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f00e57fa640 1 -- 172.21.15.33:0/1965087850 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f00ec027cd0 con 0x557ce4260600 2024-09-10T21:23:04.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 10 client.18855 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:04.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 10 client.18855 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:23:04.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 20 client.18855 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:04.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce427c160 0x557ce427e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 4 client.18855 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:04.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.395+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ce41cfdf0 con 0x557ce427c160 2024-09-10T21:23:04.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.396+0000 7f0101d11640 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce427c160 0x557ce427e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.396+0000 7f0101d11640 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce427c160 0x557ce427e5f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bb801672be8bc81 server_cookie=55c2b84aee99053f in_seq=0 out_seq=0 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.396+0000 7f00e57fa640 10 client.18855 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f00e57fa640 1 -- 172.21.15.33:0/1965087850 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ce41cfdf0 con 0x557ce427c160 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f00e57fa640 10 client.18855 handle_command_reply: tid=0 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 client.18855 shutdown 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce4260600 msgr2=0x557ce4269350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce4260600 0x557ce4269350 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f00ec003870 tx=0x7f00ec004730 comp rx=0 tx=0).stop 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 shutdown_connections 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce426c590 0x557ce42699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce427c160 0x557ce427e5f0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce4260600 0x557ce4269350 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 --2- 172.21.15.33:0/1965087850 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce426bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f00e57fa640 0 client.18855 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.399+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 >> 172.21.15.33:0/1965087850 conn(0x557ce425db10 msgr2=0x557ce42636b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.400+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 shutdown_connections 2024-09-10T21:23:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.400+0000 7f010634a740 1 -- 172.21.15.33:0/1965087850 wait complete. 2024-09-10T21:23:04.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.400+0000 7f010634a740 20 client.18855 trim_cache size 0 max 16384 2024-09-10T21:23:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 Processor -- start 2024-09-10T21:23:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 -- start start 2024-09-10T21:23:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4268840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce426c590 0x557ce4267bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1ac260 con 0x557ce425fce0 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1adda0 con 0x557ce426c590 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f010634a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1ac930 con 0x557ce4260600 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f0101d11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f0101d11640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42784d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60396/0 (socket says 172.21.15.33:60396) 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f0101d11640 1 -- 172.21.15.33:0/3140309612 learned_addr learned my addr 172.21.15.33:0/3140309612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f0101510640 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4268840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.417+0000 7f0102512640 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce426c590 0x557ce4267bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f0101d11640 1 -- 172.21.15.33:0/3140309612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 msgr2=0x557ce4268840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:04.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f0101d11640 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4268840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f0101d11640 1 -- 172.21.15.33:0/3140309612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce426c590 msgr2=0x557ce4267bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f0101d11640 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce426c590 0x557ce4267bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f0101d11640 1 -- 172.21.15.33:0/3140309612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ce425e590 con 0x557ce425fce0 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f0101d11640 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42784d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f00f80a9c00 tx=0x7f00f80a90e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd774776ce106387 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f00e77fe640 1 -- 172.21.15.33:0/3140309612 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80bcf90 con 0x557ce425fce0 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f00e77fe640 1 -- 172.21.15.33:0/3140309612 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f00f80b1470 con 0x557ce425fce0 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.418+0000 7f00e77fe640 1 -- 172.21.15.33:0/3140309612 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80b1040 con 0x557ce425fce0 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 -- 172.21.15.33:0/3140309612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 msgr2=0x557ce42784d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42784d0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f00f80a9c00 tx=0x7f00f80a90e0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 -- 172.21.15.33:0/3140309612 shutdown_connections 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce426c590 0x557ce4267bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4268840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 --2- 172.21.15.33:0/3140309612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce425fce0 0x557ce42784d0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 -- 172.21.15.33:0/3140309612 >> 172.21.15.33:0/3140309612 conn(0x557ce425db10 msgr2=0x557ce42645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 -- 172.21.15.33:0/3140309612 shutdown_connections 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 -- 172.21.15.33:0/3140309612 wait complete. 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 -- start start 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.419+0000 7f010634a740 1 Processor -- start 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f010634a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce4274f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f010634a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4283da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f010634a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce426c590 0x557ce4284530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f010634a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1ad6d0 con 0x557ce426c590 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f010634a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1ad000 con 0x557ce425fce0 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f010634a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc1aa220 con 0x557ce4260600 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f010634a740 10 client.0 _ll_register_callbacks cb 0x7f00ec005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:04.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f0101d11640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce4274f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f0102512640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce426c590 0x557ce4284530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f0101d11640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce4274f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59138/0 (socket says 172.21.15.33:59138) 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f0102512640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce426c590 0x557ce4284530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60402/0 (socket says 172.21.15.33:60402) 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f0101d11640 1 -- 172.21.15.33:0/1361049051 learned_addr learned my addr 172.21.15.33:0/1361049051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.420+0000 7f0101510640 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4283da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f0101d11640 1 -- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 msgr2=0x557ce4283da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f0101d11640 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4283da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f0101d11640 1 -- 172.21.15.33:0/1361049051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce426c590 msgr2=0x557ce4284530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f0101d11640 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce426c590 0x557ce4284530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f0101d11640 1 -- 172.21.15.33:0/1361049051 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ce42709c0 con 0x557ce425fce0 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f0101d11640 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce4274f50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c6a0 tx=0x7f00f8009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a7a88b95b62ceef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:04.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f00e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f00e57fa640 1 -- 172.21.15.33:0/1361049051 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f80cc060 con 0x557ce425fce0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557ce41cb8a0 con 0x557ce425fce0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f00e57fa640 1 -- 172.21.15.33:0/1361049051 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f00f80bf040 con 0x557ce425fce0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f00e57fa640 1 -- 172.21.15.33:0/1361049051 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f00f80b7030 con 0x557ce425fce0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f010634a740 10 client.28244 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.421+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x557ce42e9380 con 0x557ce425fce0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f00e57fa640 1 -- 172.21.15.33:0/1361049051 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557ce41cb8a0 con 0x557ce425fce0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f00e57fa640 1 -- 172.21.15.33:0/1361049051 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f00f80cfce0 con 0x557ce425fce0 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f010634a740 10 client.28244 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f010634a740 10 client.28244 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f010634a740 20 client.28244 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f010634a740 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce42e96b0 0x557ce42ebaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:04.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f010634a740 4 client.28244 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557ce41c8cd0 con 0x557ce42e96b0 2024-09-10T21:23:04.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.422+0000 7f0101510640 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce42e96b0 0x557ce42ebaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.423+0000 7f0101510640 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce42e96b0 0x557ce42ebaa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e57a31485c8dd455 server_cookie=25ba0fb09dd08484 in_seq=0 out_seq=0 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.423+0000 7f00e57fa640 10 client.28244 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.424+0000 7f00e57fa640 1 -- 172.21.15.33:0/1361049051 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557ce41c8cd0 con 0x557ce42e96b0 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.424+0000 7f00e57fa640 10 client.28244 handle_command_reply: tid=0 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.424+0000 7f010634a740 1 client.28244 shutdown 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.424+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 msgr2=0x557ce4274f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.424+0000 7f010634a740 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce4274f50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f00f800c6a0 tx=0x7f00f8009310 comp rx=0 tx=0).stop 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.424+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 shutdown_connections 2024-09-10T21:23:04.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ce42e96b0 0x557ce42ebaa0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ce426c590 0x557ce4284530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ce4260600 0x557ce4283da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 1 --2- 172.21.15.33:0/1361049051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ce425fce0 0x557ce4274f50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 >> 172.21.15.33:0/1361049051 conn(0x557ce425db10 msgr2=0x557ce4253c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f00e57fa640 0 client.28244 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 shutdown_connections 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 1 -- 172.21.15.33:0/1361049051 wait complete. 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f010634a740 20 client.28244 trim_cache size 0 max 16384 2024-09-10T21:23:04.432 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f00e80b52e0 msgr2=0x7f00e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f0103f75640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f00e80b52e0 0x7f00e80b77a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f00fc1010e0 tx=0x7f00ec005f90 comp rx=0 tx=0).stop 2024-09-10T21:23:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075540 msgr2=0x7f00fc19b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.425+0000 7f0103f75640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075540 0x7f00fc19b480 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9f0 tx=0x7f00f800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 shutdown_connections 2024-09-10T21:23:04.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f00e80b52e0 0x7f00e80b77a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f00fc10f400 0x7f00fc1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f00fc075f40 0x7f00fc19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 --2- 172.21.15.33:0/920988561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00fc075540 0x7f00fc19b480 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 >> 172.21.15.33:0/920988561 conn(0x7f00fc0fe830 msgr2=0x7f00fc0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 shutdown_connections 2024-09-10T21:23:04.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:04.426+0000 7f0103f75640 1 -- 172.21.15.33:0/920988561 wait complete. 2024-09-10T21:23:05.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 289119 40.92 MB/sec execute 245 sec latency 99.409 ms 2024-09-10T21:23:05.667 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:23:05.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:05 smithi033 ceph-mon[30860]: pgmap v919: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 794 op/s 2024-09-10T21:23:05.977 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:06.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 289803 40.91 MB/sec execute 246 sec latency 174.468 ms 2024-09-10T21:23:06.113 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:05 smithi104 ceph-mon[35325]: pgmap v919: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 794 op/s 2024-09-10T21:23:06.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:05 smithi174 ceph-mon[35635]: pgmap v919: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 794 op/s 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.502+0000 7fc57d633640 1 -- 172.21.15.33:0/246655779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 msgr2=0x7fc57810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.502+0000 7fc57d633640 1 --2- 172.21.15.33:0/246655779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 0x7fc57810f550 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc570009ca0 tx=0x7fc57002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 -- 172.21.15.33:0/246655779 shutdown_connections 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 --2- 172.21.15.33:0/246655779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 0x7fc57810f550 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 --2- 172.21.15.33:0/246655779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 0x7fc57810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 --2- 172.21.15.33:0/246655779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc578103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 -- 172.21.15.33:0/246655779 >> 172.21.15.33:0/246655779 conn(0x7fc5780fd700 msgr2=0x7fc5780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 -- 172.21.15.33:0/246655779 shutdown_connections 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 -- 172.21.15.33:0/246655779 wait complete. 2024-09-10T21:23:06.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.503+0000 7fc57d633640 1 Processor -- start 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc57d633640 1 -- start start 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc57d633640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc57d633640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc57d633640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc57d633640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc578112db0 con 0x7fc578104530 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc57d633640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc578112c30 con 0x7fc57810d0e0 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc57d633640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc578112f30 con 0x7fc578103b30 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60428/0 (socket says 172.21.15.33:60428) 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 -- 172.21.15.33:0/3927597233 learned_addr learned my addr 172.21.15.33:0/3927597233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc576ffd640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5777fe640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 -- 172.21.15.33:0/3927597233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 msgr2=0x7fc5781019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 -- 172.21.15.33:0/3927597233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc57810d0e0 msgr2=0x7fc578102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.504+0000 7fc5767fc640 1 -- 172.21.15.33:0/3927597233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5781a7620 con 0x7fc578104530 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.505+0000 7fc5767fc640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578104530 0x7fc578101ef0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00d950 tx=0x7fc56c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.505+0000 7fc55ffff640 1 -- 172.21.15.33:0/3927597233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c017070 con 0x7fc578104530 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.505+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5781a7910 con 0x7fc578104530 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.505+0000 7fc55ffff640 1 -- 172.21.15.33:0/3927597233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc56c009c30 con 0x7fc578104530 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.505+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5781a7ea0 con 0x7fc578104530 2024-09-10T21:23:06.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.505+0000 7fc55ffff640 1 -- 172.21.15.33:0/3927597233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56c00f620 con 0x7fc578104530 2024-09-10T21:23:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.508+0000 7fc57fa08740 1 Processor -- start 2024-09-10T21:23:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.508+0000 7fc57fa08740 1 -- start start 2024-09-10T21:23:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.508+0000 7fc55ffff640 1 -- 172.21.15.33:0/3927597233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc56c00f7c0 con 0x7fc578104530 2024-09-10T21:23:06.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b69828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b6991170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc57fa08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5632b68f8ea0 con 0x5632b69824d0 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc57fa08740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5632b68e82d0 con 0x5632b698ed80 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc57fa08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5632b68ec0e0 con 0x5632b6982df0 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b69828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b69828b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60444/0 (socket says 172.21.15.33:60444) 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 -- 172.21.15.33:0/1275284420 learned_addr learned my addr 172.21.15.33:0/1275284420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b6991170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 -- 172.21.15.33:0/1275284420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 msgr2=0x5632b698e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 -- 172.21.15.33:0/1275284420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 msgr2=0x5632b6991170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b6991170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5777fe640 1 -- 172.21.15.33:0/1275284420 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b6646650 con 0x5632b69824d0 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.509+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b6991170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.510+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b69828b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc57002aeb0 tx=0x7fc57002f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeb5deb571b7893 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.510+0000 7fc55ffff640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b5360 0x7fc5600b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.510+0000 7fc55d7fa640 1 -- 172.21.15.33:0/1275284420 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570031e30 con 0x5632b69824d0 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.510+0000 7fc55d7fa640 1 -- 172.21.15.33:0/1275284420 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc570036030 con 0x5632b69824d0 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.510+0000 7fc55d7fa640 1 -- 172.21.15.33:0/1275284420 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570004330 con 0x5632b69824d0 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.510+0000 7fc576ffd640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b5360 0x7fc5600b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.510+0000 7fc55ffff640 1 -- 172.21.15.33:0/3927597233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc56c0e54a0 con 0x7fc578104530 2024-09-10T21:23:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 -- 172.21.15.33:0/1275284420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 msgr2=0x5632b69828b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b69828b0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fc57002aeb0 tx=0x7fc57002f3c0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc576ffd640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b5360 0x7fc5600b7820 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x5632b69811e0 tx=0x7fc56800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 -- 172.21.15.33:0/1275284420 shutdown_connections 2024-09-10T21:23:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b6991170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1275284420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b69828b0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 -- 172.21.15.33:0/1275284420 >> 172.21.15.33:0/1275284420 conn(0x5632b6980300 msgr2=0x5632b69814a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 -- 172.21.15.33:0/1275284420 shutdown_connections 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.511+0000 7fc57fa08740 1 -- 172.21.15.33:0/1275284420 wait complete. 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 -- start start 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 Processor -- start 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b698e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b698c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5632b6749700 con 0x5632b69824d0 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5632b6883160 con 0x5632b698ed80 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5632b6991820 con 0x5632b6982df0 2024-09-10T21:23:06.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc57fa08740 10 client.0 _ll_register_callbacks cb 0x5632b689c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc576ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b698e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc576ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b698e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60454/0 (socket says 172.21.15.33:60454) 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc576ffd640 1 -- 172.21.15.33:0/1139828793 learned_addr learned my addr 172.21.15.33:0/1139828793 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b698c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.512+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc576ffd640 1 -- 172.21.15.33:0/1139828793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 msgr2=0x5632b698ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc576ffd640 1 -- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 msgr2=0x5632b698c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b698c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc576ffd640 1 -- 172.21.15.33:0/1139828793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b68b0c90 con 0x5632b69824d0 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b698c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:06.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b698e610 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc568003d40 tx=0x7fc5680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac1808900b80f5b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc55aff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc55aff5640 1 -- 172.21.15.33:0/1139828793 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc568005020 con 0x5632b69824d0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc55aff5640 1 -- 172.21.15.33:0/1139828793 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc5680051c0 con 0x5632b69824d0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.513+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc570036030 con 0x5632b69824d0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc55aff5640 1 -- 172.21.15.33:0/1139828793 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc5680053c0 con 0x5632b69824d0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 10 client.18873 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5632b699d150 con 0x5632b69824d0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc55aff5640 1 -- 172.21.15.33:0/1139828793 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc568005560 con 0x5632b69824d0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc55aff5640 1 -- 172.21.15.33:0/1139828793 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fc568024070 con 0x5632b69824d0 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 10 client.18873 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 10 client.18873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 20 client.18873 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:06.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b699e750 0x5632b69a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 4 client.18873 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc570036030 con 0x5632b699e750 2024-09-10T21:23:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.514+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b699e750 0x5632b69a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.515+0000 7fc55aff5640 10 client.18873 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.515+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b699e750 0x5632b69a0b90 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1f70068369e0963 server_cookie=6a4700d40df46e8a in_seq=0 out_seq=0 2024-09-10T21:23:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc55aff5640 1 -- 172.21.15.33:0/1139828793 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc570036030 con 0x5632b699e750 2024-09-10T21:23:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc55aff5640 10 client.18873 handle_command_reply: tid=0 2024-09-10T21:23:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 client.18873 shutdown 2024-09-10T21:23:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 msgr2=0x5632b698e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b698e610 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fc568003d40 tx=0x7fc5680046c0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 shutdown_connections 2024-09-10T21:23:06.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b699e750 0x5632b69a0b90 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b698ed80 0x5632b698c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1139828793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b69824d0 0x5632b698e610 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 >> 172.21.15.33:0/1139828793 conn(0x5632b6980300 msgr2=0x5632b6985e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc55aff5640 0 client.18873 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 shutdown_connections 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 1 -- 172.21.15.33:0/1139828793 wait complete. 2024-09-10T21:23:06.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.516+0000 7fc57fa08740 20 client.18873 trim_cache size 0 max 16384 2024-09-10T21:23:06.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.543+0000 7fc57fa08740 1 Processor -- start 2024-09-10T21:23:06.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.543+0000 7fc57fa08740 1 -- start start 2024-09-10T21:23:06.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.544+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b697fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.544+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.544+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6990170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.544+0000 7fc57fa08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc56400e4d0 con 0x5632b698ed80 2024-09-10T21:23:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.544+0000 7fc57fa08740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc564010010 con 0x5632b69824d0 2024-09-10T21:23:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.544+0000 7fc57fa08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc56400eba0 con 0x5632b6982df0 2024-09-10T21:23:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.545+0000 7fc5777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6990170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.545+0000 7fc5777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6990170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60462/0 (socket says 172.21.15.33:60462) 2024-09-10T21:23:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.545+0000 7fc5777fe640 1 -- 172.21.15.33:0/1345045275 learned_addr learned my addr 172.21.15.33:0/1345045275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:06.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.545+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.548+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b697fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.553+0000 7fc5777fe640 1 -- 172.21.15.33:0/1345045275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 msgr2=0x5632b698d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.553+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.553+0000 7fc5777fe640 1 -- 172.21.15.33:0/1345045275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 msgr2=0x5632b697fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.553+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b697fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.553+0000 7fc5777fe640 1 -- 172.21.15.33:0/1345045275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632b6980d80 con 0x5632b698ed80 2024-09-10T21:23:06.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.558+0000 7fc5777fe640 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6990170 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc570009ca0 tx=0x7fc570009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52c516a0c84ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:06.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.559+0000 7fc55cff9640 1 -- 172.21.15.33:0/1345045275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc570052030 con 0x5632b698ed80 2024-09-10T21:23:06.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.559+0000 7fc55cff9640 1 -- 172.21.15.33:0/1345045275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc570036040 con 0x5632b698ed80 2024-09-10T21:23:06.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.559+0000 7fc55cff9640 1 -- 172.21.15.33:0/1345045275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57004a040 con 0x5632b698ed80 2024-09-10T21:23:06.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.561+0000 7fc57fa08740 1 -- 172.21.15.33:0/1345045275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 msgr2=0x5632b6990170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.561+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6990170 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc570009ca0 tx=0x7fc570009a10 comp rx=0 tx=0).stop 2024-09-10T21:23:06.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.561+0000 7fc57fa08740 1 -- 172.21.15.33:0/1345045275 shutdown_connections 2024-09-10T21:23:06.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.561+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6990170 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.561+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b698d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.561+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1345045275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b697fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.561+0000 7fc57fa08740 1 -- 172.21.15.33:0/1345045275 >> 172.21.15.33:0/1345045275 conn(0x5632b6980300 msgr2=0x5632b698d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.566+0000 7fc57fa08740 1 -- 172.21.15.33:0/1345045275 shutdown_connections 2024-09-10T21:23:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.567+0000 7fc57fa08740 1 -- 172.21.15.33:0/1345045275 wait complete. 2024-09-10T21:23:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 -- start start 2024-09-10T21:23:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 Processor -- start 2024-09-10T21:23:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b6996de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b6997320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6991b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc56400f940 con 0x5632b698ed80 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc56400f270 con 0x5632b69824d0 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc56400c490 con 0x5632b6982df0 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.568+0000 7fc57fa08740 10 client.0 _ll_register_callbacks cb 0x7fc568003a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.570+0000 7fc576ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b6996de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.570+0000 7fc576ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b6996de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59188/0 (socket says 172.21.15.33:59188) 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.570+0000 7fc576ffd640 1 -- 172.21.15.33:0/1824609790 learned_addr learned my addr 172.21.15.33:0/1824609790 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.571+0000 7fc576ffd640 1 -- 172.21.15.33:0/1824609790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 msgr2=0x5632b6997320 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.571+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b6997320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.571+0000 7fc576ffd640 1 -- 172.21.15.33:0/1824609790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 msgr2=0x5632b6991b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.571+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6991b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.571+0000 7fc576ffd640 1 -- 172.21.15.33:0/1824609790 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632b6992300 con 0x5632b69824d0 2024-09-10T21:23:06.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.571+0000 7fc576ffd640 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b6996de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc56801bc60 tx=0x7fc56801bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d6d489e8de63bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:06.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.571+0000 7fc55aff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:23:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.576+0000 7fc55aff5640 1 -- 172.21.15.33:0/1824609790 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc56801b870 con 0x5632b69824d0 2024-09-10T21:23:06.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.576+0000 7fc55aff5640 1 -- 172.21.15.33:0/1824609790 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc568038420 con 0x5632b69824d0 2024-09-10T21:23:06.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.578+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc570036040 con 0x5632b69824d0 2024-09-10T21:23:06.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.580+0000 7fc55aff5640 1 -- 172.21.15.33:0/1824609790 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fc56803e050 con 0x5632b69824d0 2024-09-10T21:23:06.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.581+0000 7fc57fa08740 10 client.28259 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:06.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.581+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5632b6a0c160 con 0x5632b69824d0 2024-09-10T21:23:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.585+0000 7fc55aff5640 1 -- 172.21.15.33:0/1824609790 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc570036040 con 0x5632b69824d0 2024-09-10T21:23:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.585+0000 7fc55aff5640 1 -- 172.21.15.33:0/1824609790 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fc568024050 con 0x5632b69824d0 2024-09-10T21:23:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.587+0000 7fc57fa08740 10 client.28259 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.587+0000 7fc57fa08740 10 client.28259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.587+0000 7fc57fa08740 20 client.28259 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:06.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.587+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b6a0c8e0 0x5632b6a0cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:06.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.587+0000 7fc57fa08740 4 client.28259 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:06.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.587+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5632b68fadf0 con 0x5632b6a0c8e0 2024-09-10T21:23:06.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.592+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b6a0c8e0 0x5632b6a0cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:06.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.605+0000 7fc5767fc640 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b6a0c8e0 0x5632b6a0cd50 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d55300db2371bb9c server_cookie=24631579b8a20a3e in_seq=0 out_seq=0 2024-09-10T21:23:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.606+0000 7fc55aff5640 10 client.28259 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.606+0000 7fc55aff5640 1 -- 172.21.15.33:0/1824609790 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5632b68fadf0 con 0x5632b6a0c8e0 2024-09-10T21:23:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.606+0000 7fc55aff5640 10 client.28259 handle_command_reply: tid=0 2024-09-10T21:23:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.608+0000 7fc57fa08740 1 client.28259 shutdown 2024-09-10T21:23:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.608+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 msgr2=0x5632b6996de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.608+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b6996de0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc56801bc60 tx=0x7fc56801bf10 comp rx=0 tx=0).stop 2024-09-10T21:23:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.609+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 shutdown_connections 2024-09-10T21:23:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.609+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5632b6a0c8e0 0x5632b6a0cd50 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.609+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5632b698ed80 0x5632b6991b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.609+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5632b6982df0 0x5632b6997320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.609+0000 7fc57fa08740 1 --2- 172.21.15.33:0/1824609790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5632b69824d0 0x5632b6996de0 secure :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc56801bc60 tx=0x7fc56801bf10 comp rx=0 tx=0).stop 2024-09-10T21:23:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.609+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 >> 172.21.15.33:0/1824609790 conn(0x5632b6980300 msgr2=0x5632b698d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:06.614 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.609+0000 7fc55aff5640 0 client.28259 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.610+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 shutdown_connections 2024-09-10T21:23:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.610+0000 7fc57fa08740 1 -- 172.21.15.33:0/1824609790 wait complete. 2024-09-10T21:23:06.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.610+0000 7fc57fa08740 20 client.28259 trim_cache size 0 max 16384 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.615+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b5360 msgr2=0x7fc5600b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.615+0000 7fc57d633640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b5360 0x7fc5600b7820 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x5632b69811e0 tx=0x7fc56800b390 comp rx=0 tx=0).stop 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.615+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578104530 msgr2=0x7fc578101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.615+0000 7fc57d633640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578104530 0x7fc578101ef0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc56c00d950 tx=0x7fc56c00de10 comp rx=0 tx=0).stop 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.617+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 shutdown_connections 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.617+0000 7fc57d633640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b5360 0x7fc5600b7820 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.617+0000 7fc57d633640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.617+0000 7fc57d633640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.617+0000 7fc57d633640 1 --2- 172.21.15.33:0/3927597233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:06.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.617+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 >> 172.21.15.33:0/3927597233 conn(0x7fc5780fd700 msgr2=0x7fc5780fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:06.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.617+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 shutdown_connections 2024-09-10T21:23:06.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:06.618+0000 7fc57d633640 1 -- 172.21.15.33:0/3927597233 wait complete. 2024-09-10T21:23:07.064 INFO:tasks.workunit.client.0.smithi033.stdout: 10 290188 40.83 MB/sec execute 247 sec latency 326.199 ms 2024-09-10T21:23:08.065 INFO:tasks.workunit.client.0.smithi033.stdout: 10 290757 40.76 MB/sec execute 248 sec latency 110.062 ms 2024-09-10T21:23:08.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:07 smithi174 ceph-mon[35635]: pgmap v920: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 762 op/s 2024-09-10T21:23:08.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:07 smithi033 ceph-mon[30860]: pgmap v920: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 762 op/s 2024-09-10T21:23:08.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:07 smithi104 ceph-mon[35325]: pgmap v920: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 762 op/s 2024-09-10T21:23:08.453 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:23:08.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:23:08.768 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:09.065 INFO:tasks.workunit.client.0.smithi033.stdout: 10 291440 40.74 MB/sec execute 249 sec latency 69.228 ms 2024-09-10T21:23:09.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.309+0000 7f566ead6640 1 -- 172.21.15.33:0/3282913680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668109170 msgr2=0x7f566810f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.309+0000 7f566ead6640 1 --2- 172.21.15.33:0/3282913680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668109170 0x7f566810f970 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f5660009ca0 tx=0x7f566002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.310+0000 7f566ead6640 1 -- 172.21.15.33:0/3282913680 shutdown_connections 2024-09-10T21:23:09.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.310+0000 7f566ead6640 1 --2- 172.21.15.33:0/3282913680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5668109170 0x7f566810f970 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.310+0000 7f566ead6640 1 --2- 172.21.15.33:0/3282913680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f56681087f0 0x7f5668108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.310+0000 7f566ead6640 1 --2- 172.21.15.33:0/3282913680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5668102de0 0x7f56681031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.310+0000 7f566ead6640 1 -- 172.21.15.33:0/3282913680 >> 172.21.15.33:0/3282913680 conn(0x7f56680fe830 msgr2=0x7f5668100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:09.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.310+0000 7f566ead6640 1 -- 172.21.15.33:0/3282913680 shutdown_connections 2024-09-10T21:23:09.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.310+0000 7f566ead6640 1 -- 172.21.15.33:0/3282913680 wait complete. 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 Processor -- start 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 -- start start 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5668102de0 0x7f5668199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56681087f0 0x7f5668199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668109170 0x7f566819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5668112e30 con 0x7f56681087f0 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5668112cb0 con 0x7f5668109170 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566ead6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5668112fb0 con 0x7f5668102de0 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566c872640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5668102de0 0x7f5668199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f5667fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56681087f0 0x7f5668199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566c872640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5668102de0 0x7f5668199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:55774/0 (socket says 172.21.15.33:55774) 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566c872640 1 -- 172.21.15.33:0/1247263617 learned_addr learned my addr 172.21.15.33:0/1247263617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.311+0000 7f566d073640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668109170 0x7f566819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5667fff640 1 -- 172.21.15.33:0/1247263617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5668102de0 msgr2=0x7f5668199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5667fff640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5668102de0 0x7f5668199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5667fff640 1 -- 172.21.15.33:0/1247263617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668109170 msgr2=0x7f566819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5667fff640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668109170 0x7f566819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5667fff640 1 -- 172.21.15.33:0/1247263617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f566819f2f0 con 0x7f56681087f0 2024-09-10T21:23:09.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5667fff640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56681087f0 0x7f5668199780 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f565400e9f0 tx=0x7f565400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:09.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5665ffb640 1 -- 172.21.15.33:0/1247263617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565400ccc0 con 0x7f56681087f0 2024-09-10T21:23:09.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f566819f5e0 con 0x7f56681087f0 2024-09-10T21:23:09.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5665ffb640 1 -- 172.21.15.33:0/1247263617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56540108e0 con 0x7f56681087f0 2024-09-10T21:23:09.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.312+0000 7f5665ffb640 1 -- 172.21.15.33:0/1247263617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5654018880 con 0x7f56681087f0 2024-09-10T21:23:09.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.313+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5668075550 con 0x7f56681087f0 2024-09-10T21:23:09.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.315+0000 7f5665ffb640 1 -- 172.21.15.33:0/1247263617 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5654010430 con 0x7f56681087f0 2024-09-10T21:23:09.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 Processor -- start 2024-09-10T21:23:09.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 -- start start 2024-09-10T21:23:09.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce16ee0 0x55b44ce172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce17800 0x55b44ce23250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b44cd8de60 con 0x55b44ce23790 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b44cd7d290 con 0x55b44ce17800 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5670eab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b44cd810a0 con 0x55b44ce16ee0 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f566c872640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce17800 0x55b44ce23250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f566c872640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce17800 0x55b44ce23250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59200/0 (socket says 172.21.15.33:59200) 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f566c872640 1 -- 172.21.15.33:0/875895147 learned_addr learned my addr 172.21.15.33:0/875895147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5667fff640 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.316+0000 7f5665ffb640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56580b5360 0x7f56580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f566c872640 1 -- 172.21.15.33:0/875895147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce16ee0 msgr2=0x55b44ce172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f566c872640 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce16ee0 0x55b44ce172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f566c872640 1 -- 172.21.15.33:0/875895147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 msgr2=0x55b44ce25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f566c872640 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f566c872640 1 -- 172.21.15.33:0/875895147 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b44cadb930 con 0x55b44ce17800 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f5665ffb640 1 -- 172.21.15.33:0/1247263617 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f56540e5e80 con 0x7f56681087f0 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f566d073640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56580b5360 0x7f56580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f566c872640 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce17800 0x55b44ce23250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f565c009e10 tx=0x7f565c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=645a880355c6890c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.317+0000 7f5652ffd640 1 -- 172.21.15.33:0/875895147 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c01e070 con 0x55b44ce17800 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.318+0000 7f5652ffd640 1 -- 172.21.15.33:0/875895147 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c019ae0 con 0x55b44ce17800 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.318+0000 7f566d073640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56580b5360 0x7f56580b7820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5660006010 tx=0x7f566003f040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:09.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.318+0000 7f5652ffd640 1 -- 172.21.15.33:0/875895147 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c00fae0 con 0x55b44ce17800 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 -- 172.21.15.33:0/875895147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce17800 msgr2=0x55b44ce23250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce17800 0x55b44ce23250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f565c009e10 tx=0x7f565c011810 comp rx=0 tx=0).stop 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 -- 172.21.15.33:0/875895147 shutdown_connections 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce17800 0x55b44ce23250 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 --2- 172.21.15.33:0/875895147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce16ee0 0x55b44ce172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 -- 172.21.15.33:0/875895147 >> 172.21.15.33:0/875895147 conn(0x55b44ce14d10 msgr2=0x55b44ce15eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:09.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 -- 172.21.15.33:0/875895147 shutdown_connections 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 -- 172.21.15.33:0/875895147 wait complete. 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 -- start start 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.319+0000 7f5670eab740 1 Processor -- start 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5670eab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce16ee0 0x55b44ce206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5670eab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce20c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5670eab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce23790 0x55b44ce21170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5670eab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55b44cbde890 con 0x55b44ce16ee0 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5670eab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55b44cd17d40 con 0x55b44ce23790 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5670eab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b44ce264e0 con 0x55b44ce17800 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5670eab740 10 client.0 _ll_register_callbacks cb 0x55b44cd7bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce23790 0x55b44ce21170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce23790 0x55b44ce21170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59202/0 (socket says 172.21.15.33:59202) 2024-09-10T21:23:09.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 -- 172.21.15.33:0/385530188 learned_addr learned my addr 172.21.15.33:0/385530188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566c872640 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce16ee0 0x55b44ce206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f5667fff640 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce20c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 -- 172.21.15.33:0/385530188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 msgr2=0x55b44ce20c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce20c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 -- 172.21.15.33:0/385530188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce16ee0 msgr2=0x55b44ce206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce16ee0 0x55b44ce206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.320+0000 7f566d073640 1 -- 172.21.15.33:0/385530188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b44ce2f860 con 0x55b44ce23790 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f566d073640 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce23790 0x55b44ce21170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5660031730 tx=0x7f566002f3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12725d36dac26b79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f5647fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f5647fff640 1 -- 172.21.15.33:0/385530188 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5660004240 con 0x55b44ce23790 2024-09-10T21:23:09.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f5647fff640 1 -- 172.21.15.33:0/385530188 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f56600043e0 con 0x55b44ce23790 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565c019ae0 con 0x55b44ce23790 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f5647fff640 1 -- 172.21.15.33:0/385530188 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f56600045e0 con 0x55b44ce23790 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f5670eab740 10 client.28271 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.321+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55b44ce32210 con 0x55b44ce23790 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5647fff640 1 -- 172.21.15.33:0/385530188 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c019ae0 con 0x55b44ce23790 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5647fff640 1 -- 172.21.15.33:0/385530188 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f5660046040 con 0x55b44ce23790 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5670eab740 10 client.28271 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5670eab740 10 client.28271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5670eab740 20 client.28271 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5670eab740 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44ce33720 0x55b44ce35b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5670eab740 4 client.28271 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b44cd8fdb0 con 0x55b44ce33720 2024-09-10T21:23:09.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.322+0000 7f566c872640 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44ce33720 0x55b44ce35b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.323+0000 7f566c872640 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44ce33720 0x55b44ce35b50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ea920cde8af53d6 server_cookie=507b89bb7b321bda in_seq=0 out_seq=0 2024-09-10T21:23:09.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.323+0000 7f5647fff640 10 client.28271 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:09.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5647fff640 1 -- 172.21.15.33:0/385530188 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b44cd8fdb0 con 0x55b44ce33720 2024-09-10T21:23:09.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5647fff640 10 client.28271 handle_command_reply: tid=0 2024-09-10T21:23:09.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 client.28271 shutdown 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce23790 msgr2=0x55b44ce21170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce23790 0x55b44ce21170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5660031730 tx=0x7f566002f3b0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 shutdown_connections 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44ce33720 0x55b44ce35b50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5647fff640 0 client.28271 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce23790 0x55b44ce21170 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce20c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 --2- 172.21.15.33:0/385530188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce16ee0 0x55b44ce206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.324+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 >> 172.21.15.33:0/385530188 conn(0x55b44ce14d10 msgr2=0x55b44ce13f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.325+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 shutdown_connections 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.325+0000 7f5670eab740 1 -- 172.21.15.33:0/385530188 wait complete. 2024-09-10T21:23:09.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.325+0000 7f5670eab740 20 client.28271 trim_cache size 0 max 16384 2024-09-10T21:23:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.345+0000 7f5670eab740 1 Processor -- start 2024-09-10T21:23:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.345+0000 7f5670eab740 1 -- start start 2024-09-10T21:23:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f5670eab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce1ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f5670eab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce1d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f5670eab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce1de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f5670eab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56681b0620 con 0x55b44ce23790 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f5670eab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56681b2160 con 0x55b44ce16ee0 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f5670eab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56681b0cf0 con 0x55b44ce17800 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce1ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566d073640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce1de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce1ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59218/0 (socket says 172.21.15.33:59218) 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566d073640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce1de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60540/0 (socket says 172.21.15.33:60540) 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 -- 172.21.15.33:0/3501288443 learned_addr learned my addr 172.21.15.33:0/3501288443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f5667fff640 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce1d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 -- 172.21.15.33:0/3501288443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 msgr2=0x55b44ce1d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce1d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 -- 172.21.15.33:0/3501288443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 msgr2=0x55b44ce1de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce1de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.346+0000 7f566c872640 1 -- 172.21.15.33:0/3501288443 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b44ce15790 con 0x55b44ce16ee0 2024-09-10T21:23:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.347+0000 7f566c872640 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce1ff80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f565c013010 tx=0x7f565c00f760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=612f661758f6491 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.347+0000 7f56527fc640 1 -- 172.21.15.33:0/3501288443 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c01e040 con 0x55b44ce16ee0 2024-09-10T21:23:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.347+0000 7f56527fc640 1 -- 172.21.15.33:0/3501288443 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c02f030 con 0x55b44ce16ee0 2024-09-10T21:23:09.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.347+0000 7f56527fc640 1 -- 172.21.15.33:0/3501288443 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c019820 con 0x55b44ce16ee0 2024-09-10T21:23:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 -- 172.21.15.33:0/3501288443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 msgr2=0x55b44ce1ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce1ff80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f565c013010 tx=0x7f565c00f760 comp rx=0 tx=0).stop 2024-09-10T21:23:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 -- 172.21.15.33:0/3501288443 shutdown_connections 2024-09-10T21:23:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce23790 0x55b44ce1de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce17800 0x55b44ce1d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 --2- 172.21.15.33:0/3501288443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce1ff80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 -- 172.21.15.33:0/3501288443 >> 172.21.15.33:0/3501288443 conn(0x55b44ce14d10 msgr2=0x55b44ce22540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 -- 172.21.15.33:0/3501288443 shutdown_connections 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 -- 172.21.15.33:0/3501288443 wait complete. 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 -- start start 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.348+0000 7f5670eab740 1 Processor -- start 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5670eab740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce2b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5670eab740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce17800 0x55b44ce2bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5670eab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce23790 0x55b44ce26730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5670eab740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f56681b1a90 con 0x55b44ce17800 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5670eab740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f56681b13c0 con 0x55b44ce16ee0 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5670eab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56681ae600 con 0x55b44ce23790 2024-09-10T21:23:09.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f566c872640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce2b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f566c872640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce2b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:59230/0 (socket says 172.21.15.33:59230) 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f566c872640 1 -- 172.21.15.33:0/2509886029 learned_addr learned my addr 172.21.15.33:0/2509886029 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5667fff640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce17800 0x55b44ce2bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f5670eab740 10 client.0 _ll_register_callbacks cb 0x7f565c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.349+0000 7f566d073640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce23790 0x55b44ce26730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f566c872640 1 -- 172.21.15.33:0/2509886029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce23790 msgr2=0x55b44ce26730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f566c872640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce23790 0x55b44ce26730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f566c872640 1 -- 172.21.15.33:0/2509886029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce17800 msgr2=0x55b44ce2bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f566c872640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce17800 0x55b44ce2bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f566c872640 1 -- 172.21.15.33:0/2509886029 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b44ce26c70 con 0x55b44ce16ee0 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f566d073640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce23790 0x55b44ce26730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:09.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5667fff640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce17800 0x55b44ce2bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f566c872640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce2b780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f565c01c040 tx=0x7f565c005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644069b770b9249f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5647fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5647fff640 1 -- 172.21.15.33:0/2509886029 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c01e040 con 0x55b44ce16ee0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5647fff640 1 -- 172.21.15.33:0/2509886029 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f565c005030 con 0x55b44ce16ee0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565c02f030 con 0x55b44ce16ee0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5647fff640 1 -- 172.21.15.33:0/2509886029 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f565c009880 con 0x55b44ce16ee0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5670eab740 10 client.28277 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.350+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55b44cea0de0 con 0x55b44ce16ee0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5647fff640 1 -- 172.21.15.33:0/2509886029 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c02f030 con 0x55b44ce16ee0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5647fff640 1 -- 172.21.15.33:0/2509886029 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f565c00f7d0 con 0x55b44ce16ee0 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5670eab740 10 client.28277 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5670eab740 10 client.28277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:09.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5670eab740 20 client.28277 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5670eab740 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44cea1300 0x55b44cea3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5670eab740 4 client.28277 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:23:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b44cd8b860 con 0x55b44cea1300 2024-09-10T21:23:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.351+0000 7f5667fff640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44cea1300 0x55b44cea3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.352+0000 7f5647fff640 10 client.28277 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:09.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.352+0000 7f5667fff640 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44cea1300 0x55b44cea3720 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b54191b1f284fa91 server_cookie=3c89b83bf916b6bd in_seq=0 out_seq=0 2024-09-10T21:23:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.358+0000 7f5647fff640 1 -- 172.21.15.33:0/2509886029 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b44cd8b860 con 0x55b44cea1300 2024-09-10T21:23:09.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.358+0000 7f5647fff640 10 client.28277 handle_command_reply: tid=0 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.358+0000 7f5670eab740 1 client.28277 shutdown 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 msgr2=0x55b44ce2b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce2b780 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f565c01c040 tx=0x7f565c005f10 comp rx=0 tx=0).stop 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 shutdown_connections 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55b44cea1300 0x55b44cea3720 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b44ce23790 0x55b44ce26730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55b44ce17800 0x55b44ce2bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 --2- 172.21.15.33:0/2509886029 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55b44ce16ee0 0x55b44ce2b780 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 >> 172.21.15.33:0/2509886029 conn(0x55b44ce14d10 msgr2=0x55b44ce13750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5647fff640 0 client.28277 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 shutdown_connections 2024-09-10T21:23:09.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 1 -- 172.21.15.33:0/2509886029 wait complete. 2024-09-10T21:23:09.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.359+0000 7f5670eab740 20 client.28277 trim_cache size 0 max 16384 2024-09-10T21:23:09.367 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:23:09.367 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:23:09.367 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb", 2024-09-10T21:23:09.367 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:23:09.367 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:23:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56580b5360 msgr2=0x7f56580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56580b5360 0x7f56580b7820 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f5660006010 tx=0x7f566003f040 comp rx=0 tx=0).stop 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56681087f0 msgr2=0x7f5668199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56681087f0 0x7f5668199780 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f565400e9f0 tx=0x7f565400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 shutdown_connections 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f56580b5360 0x7f56580b7820 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5668109170 0x7f566819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f56681087f0 0x7f5668199780 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 --2- 172.21.15.33:0/1247263617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5668102de0 0x7f5668199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.360+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 >> 172.21.15.33:0/1247263617 conn(0x7f56680fe830 msgr2=0x7f5668100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.361+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 shutdown_connections 2024-09-10T21:23:09.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:09.361+0000 7f566ead6640 1 -- 172.21.15.33:0/1247263617 wait complete. 2024-09-10T21:23:10.065 INFO:tasks.workunit.client.0.smithi033.stdout: 10 291809 40.65 MB/sec execute 250 sec latency 498.159 ms 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: pgmap v921: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 860 op/s 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:23:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:09 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: pgmap v921: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 860 op/s 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:23:10.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:09 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:10.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: pgmap v921: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 860 op/s 2024-09-10T21:23:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:23:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:23:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:23:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:23:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:23:10.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:09 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:10.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:23:11.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 292607 40.70 MB/sec execute 251 sec latency 425.044 ms 2024-09-10T21:23:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:10 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:23:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:10 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:10 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:23:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:10 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:10 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:10 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:10 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:10 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:23:11.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:10 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:10 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:23:11.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:10 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:10 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:10 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:10 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:10 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:23:11.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:10 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:10 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:23:11.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:10 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:10 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:10 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:10 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:23:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 -- 172.21.15.33:0/2670140580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee841076a0 msgr2=0x7fee84107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 --2- 172.21.15.33:0/2670140580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee841076a0 0x7fee84107ae0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fee78009900 tx=0x7fee7802a410 comp rx=0 tx=0).stop 2024-09-10T21:23:11.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 -- 172.21.15.33:0/2670140580 shutdown_connections 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 --2- 172.21.15.33:0/2670140580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee84108020 0x7fee8410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 --2- 172.21.15.33:0/2670140580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee841076a0 0x7fee84107ae0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 --2- 172.21.15.33:0/2670140580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee84101c90 0x7fee84102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 -- 172.21.15.33:0/2670140580 >> 172.21.15.33:0/2670140580 conn(0x7fee840fd700 msgr2=0x7fee840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 -- 172.21.15.33:0/2670140580 shutdown_connections 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.572+0000 7fee8ac66640 1 -- 172.21.15.33:0/2670140580 wait complete. 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.573+0000 7fee8ac66640 1 Processor -- start 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.573+0000 7fee8ac66640 1 -- start start 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee8ac66640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee84101c90 0x7fee8419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee8ac66640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee841076a0 0x7fee8419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee8ac66640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee84108020 0x7fee841a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee841076a0 0x7fee8419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee88a02640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee84101c90 0x7fee8419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee841076a0 0x7fee8419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:60772/0 (socket says 172.21.15.33:60772) 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee89203640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee84108020 0x7fee841a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee89203640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee84108020 0x7fee841a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60464/0 (socket says 172.21.15.33:60464) 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee8ac66640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee84110c20 con 0x7fee84108020 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee83fff640 1 -- 172.21.15.33:0/503609848 learned_addr learned my addr 172.21.15.33:0/503609848 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:11.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee84110aa0 con 0x7fee841076a0 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.574+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee84110da0 con 0x7fee84101c90 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee83fff640 1 -- 172.21.15.33:0/503609848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee84101c90 msgr2=0x7fee8419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee83fff640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee84101c90 0x7fee8419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee83fff640 1 -- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee84108020 msgr2=0x7fee841a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee83fff640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee84108020 0x7fee841a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee83fff640 1 -- 172.21.15.33:0/503609848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee841a1470 con 0x7fee841076a0 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee83fff640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee841076a0 0x7fee8419b900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee78005ab0 tx=0x7fee78003d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee81ffb640 1 -- 172.21.15.33:0/503609848 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7803f070 con 0x7fee841076a0 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee81ffb640 1 -- 172.21.15.33:0/503609848 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78031d60 con 0x7fee841076a0 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee841a1700 con 0x7fee841076a0 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.575+0000 7fee81ffb640 1 -- 172.21.15.33:0/503609848 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78035030 con 0x7fee841076a0 2024-09-10T21:23:11.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.576+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee841a7e20 con 0x7fee841076a0 2024-09-10T21:23:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.579+0000 7fee8d03b740 1 Processor -- start 2024-09-10T21:23:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee8d03b740 1 -- start start 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c485f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c486b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c486dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee8d03b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5625c47d5e50 con 0x5625c485f0a0 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee8d03b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5625c47c5280 con 0x5625c486b950 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee8d03b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625c47c9090 con 0x5625c485f9c0 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee88a02640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c486b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee88a02640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c486b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54522/0 (socket says 172.21.15.33:54522) 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.580+0000 7fee88a02640 1 -- 172.21.15.33:0/1182673749 learned_addr learned my addr 172.21.15.33:0/1182673749 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.581+0000 7fee89203640 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c485f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.581+0000 7fee81ffb640 1 -- 172.21.15.33:0/503609848 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fee7803b4f0 con 0x7fee841076a0 2024-09-10T21:23:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.581+0000 7fee89203640 1 -- 172.21.15.33:0/1182673749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 msgr2=0x5625c486b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.581+0000 7fee89203640 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c486b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.581+0000 7fee89203640 1 -- 172.21.15.33:0/1182673749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 msgr2=0x5625c486dd40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:23:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.581+0000 7fee89203640 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c486dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.581+0000 7fee89203640 1 -- 172.21.15.33:0/1182673749 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c4523650 con 0x5625c485f0a0 2024-09-10T21:23:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.582+0000 7fee89203640 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c485f480 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fee7c007e40 tx=0x7fee7c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2b9aa1c8bfb16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.583+0000 7fee81ffb640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fee740b5350 0x7fee740b7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.583+0000 7fee6effd640 1 -- 172.21.15.33:0/1182673749 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c003b70 con 0x5625c485f0a0 2024-09-10T21:23:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.583+0000 7fee6effd640 1 -- 172.21.15.33:0/1182673749 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c003d10 con 0x5625c485f0a0 2024-09-10T21:23:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.583+0000 7fee6effd640 1 -- 172.21.15.33:0/1182673749 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c004490 con 0x5625c485f0a0 2024-09-10T21:23:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.583+0000 7fee81ffb640 1 -- 172.21.15.33:0/503609848 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fee781062b0 con 0x7fee841076a0 2024-09-10T21:23:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.583+0000 7fee88a02640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fee740b5350 0x7fee740b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 -- 172.21.15.33:0/1182673749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 msgr2=0x5625c485f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c485f480 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fee7c007e40 tx=0x7fee7c00f790 comp rx=0 tx=0).stop 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 -- 172.21.15.33:0/1182673749 shutdown_connections 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c486dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c486b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1182673749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c485f480 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 -- 172.21.15.33:0/1182673749 >> 172.21.15.33:0/1182673749 conn(0x5625c485ced0 msgr2=0x5625c485e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 -- 172.21.15.33:0/1182673749 shutdown_connections 2024-09-10T21:23:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.584+0000 7fee8d03b740 1 -- 172.21.15.33:0/1182673749 wait complete. 2024-09-10T21:23:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee88a02640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fee740b5350 0x7fee740b7810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5625c485ddb0 tx=0x7fee7000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 -- start start 2024-09-10T21:23:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 Processor -- start 2024-09-10T21:23:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c486b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c4868660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c4868c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5625c4626700 con 0x5625c485f0a0 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5625c4760110 con 0x5625c486b950 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.585+0000 7fee8d03b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5625c486e5d0 con 0x5625c485f9c0 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee8d03b740 10 client.0 _ll_register_callbacks cb 0x5625c4779990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c486b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c486b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60484/0 (socket says 172.21.15.33:60484) 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 -- 172.21.15.33:0/1928304797 learned_addr learned my addr 172.21.15.33:0/1928304797 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee89203640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c4868c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee83fff640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c4868660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 -- 172.21.15.33:0/1928304797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 msgr2=0x5625c4868660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c4868660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 -- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 msgr2=0x5625c4868c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:11.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c4868c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.586+0000 7fee88a02640 1 -- 172.21.15.33:0/1928304797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c478dc40 con 0x5625c485f0a0 2024-09-10T21:23:11.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.587+0000 7fee89203640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c4868c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:11.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.587+0000 7fee88a02640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c486b210 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fee700109c0 tx=0x7fee700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e96708237052 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:11.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.587+0000 7fee63fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:11.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.587+0000 7fee63fff640 1 -- 172.21.15.33:0/1928304797 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70032060 con 0x5625c485f0a0 2024-09-10T21:23:11.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.587+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7c003d10 con 0x5625c485f0a0 2024-09-10T21:23:11.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.588+0000 7fee63fff640 1 -- 172.21.15.33:0/1928304797 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fee7000f940 con 0x5625c485f0a0 2024-09-10T21:23:11.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.588+0000 7fee63fff640 1 -- 172.21.15.33:0/1928304797 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fee7000aa30 con 0x5625c485f0a0 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.588+0000 7fee8d03b740 10 client.18906 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.588+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5625c4879fe0 con 0x5625c485f0a0 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.588+0000 7fee63fff640 1 -- 172.21.15.33:0/1928304797 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c003d10 con 0x5625c485f0a0 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.588+0000 7fee63fff640 1 -- 172.21.15.33:0/1928304797 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fee70037070 con 0x5625c485f0a0 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.589+0000 7fee8d03b740 10 client.18906 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.589+0000 7fee8d03b740 10 client.18906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.589+0000 7fee8d03b740 20 client.18906 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.589+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c487b440 0x5625c487d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.589+0000 7fee8d03b740 4 client.18906 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.589+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5625c47d7da0 con 0x5625c487b440 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.589+0000 7fee83fff640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c487b440 0x5625c487d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.590+0000 7fee63fff640 10 client.18906 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:11.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.591+0000 7fee83fff640 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c487b440 0x5625c487d830 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dee177a0cd676ca6 server_cookie=19df508740c2f9dc in_seq=0 out_seq=0 2024-09-10T21:23:11.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee63fff640 1 -- 172.21.15.33:0/1928304797 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5625c47d7da0 con 0x5625c487b440 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee63fff640 10 client.18906 handle_command_reply: tid=0 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 client.18906 shutdown 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 msgr2=0x5625c486b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c486b210 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fee700109c0 tx=0x7fee700046c0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 shutdown_connections 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c487b440 0x5625c487d830 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee63fff640 0 client.18906 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c486b950 0x5625c4868c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c485f9c0 0x5625c4868660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 --2- 172.21.15.33:0/1928304797 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c486b210 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.594+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 >> 172.21.15.33:0/1928304797 conn(0x5625c485ced0 msgr2=0x5625c4862a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:11.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.595+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 shutdown_connections 2024-09-10T21:23:11.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.595+0000 7fee8d03b740 1 -- 172.21.15.33:0/1928304797 wait complete. 2024-09-10T21:23:11.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.595+0000 7fee8d03b740 20 client.18906 trim_cache size 0 max 16384 2024-09-10T21:23:11.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.619+0000 7fee8d03b740 1 Processor -- start 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.619+0000 7fee8d03b740 1 -- start start 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c4877860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f9c0 0x5625c48638f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4863e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee8d03b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ac150 con 0x5625c485f0a0 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee8d03b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee841adc90 con 0x5625c485f9c0 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee8d03b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ac820 con 0x5625c486b950 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c4877860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c4877860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60486/0 (socket says 172.21.15.33:60486) 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee83fff640 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f9c0 0x5625c48638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 -- 172.21.15.33:0/248581107 learned_addr learned my addr 172.21.15.33:0/248581107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:11.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee89203640 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4863e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 -- 172.21.15.33:0/248581107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 msgr2=0x5625c4863e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4863e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 -- 172.21.15.33:0/248581107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f9c0 msgr2=0x5625c48638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f9c0 0x5625c48638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 -- 172.21.15.33:0/248581107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625c486ae40 con 0x5625c485f0a0 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.620+0000 7fee88a02640 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c4877860 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fee7000a660 tx=0x7fee70010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8542c334af329ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.621+0000 7fee6e7fc640 1 -- 172.21.15.33:0/248581107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7003d030 con 0x5625c485f0a0 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.621+0000 7fee6e7fc640 1 -- 172.21.15.33:0/248581107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee700054e0 con 0x5625c485f0a0 2024-09-10T21:23:11.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.621+0000 7fee6e7fc640 1 -- 172.21.15.33:0/248581107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7002d410 con 0x5625c485f0a0 2024-09-10T21:23:11.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.621+0000 7fee8d03b740 1 -- 172.21.15.33:0/248581107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 msgr2=0x5625c4877860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.621+0000 7fee8d03b740 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c4877860 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fee7000a660 tx=0x7fee70010fd0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 -- 172.21.15.33:0/248581107 shutdown_connections 2024-09-10T21:23:11.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4863e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f9c0 0x5625c48638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 --2- 172.21.15.33:0/248581107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f0a0 0x5625c4877860 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 -- 172.21.15.33:0/248581107 >> 172.21.15.33:0/248581107 conn(0x5625c485ced0 msgr2=0x5625c4868410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 -- 172.21.15.33:0/248581107 shutdown_connections 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 -- 172.21.15.33:0/248581107 wait complete. 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 -- start start 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 Processor -- start 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f0a0 0x5625c4874260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f9c0 0x5625c4882ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.622+0000 7fee8d03b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4883670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee8d03b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fee841ad5c0 con 0x5625c485f9c0 2024-09-10T21:23:11.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee8d03b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fee841acef0 con 0x5625c485f0a0 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee8d03b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee841aa150 con 0x5625c486b950 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee8d03b740 10 client.0 _ll_register_callbacks cb 0x7fee7000a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f9c0 0x5625c4882ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f9c0 0x5625c4882ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60490/0 (socket says 172.21.15.33:60490) 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 -- 172.21.15.33:0/4155643689 learned_addr learned my addr 172.21.15.33:0/4155643689 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee89203640 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4883670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 -- 172.21.15.33:0/4155643689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 msgr2=0x5625c4883670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4883670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 -- 172.21.15.33:0/4155643689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f0a0 msgr2=0x5625c4874260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f0a0 0x5625c4874260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 -- 172.21.15.33:0/4155643689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625c4872ec0 con 0x5625c485f9c0 2024-09-10T21:23:11.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee83fff640 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f9c0 0x5625c4882ee0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fee780b40b0 tx=0x7fee7803b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a742cc378b6356 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee63fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee63fff640 1 -- 172.21.15.33:0/4155643689 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780c8040 con 0x5625c485f9c0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.623+0000 7fee63fff640 1 -- 172.21.15.33:0/4155643689 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fee780cccd0 con 0x5625c485f9c0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee700054e0 con 0x5625c485f9c0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee63fff640 1 -- 172.21.15.33:0/4155643689 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fee780cc580 con 0x5625c485f9c0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 10 client.18912 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5625c48e85b0 con 0x5625c485f9c0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee63fff640 1 -- 172.21.15.33:0/4155643689 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee700054e0 con 0x5625c485f9c0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee63fff640 1 -- 172.21.15.33:0/4155643689 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7fee780d5b70 con 0x5625c485f9c0 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 10 client.18912 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 10 client.18912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:11.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 20 client.18912 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c48e8c90 0x5625c48eb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.624+0000 7fee8d03b740 4 client.18912 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.625+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5625c47d3850 con 0x5625c48e8c90 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.625+0000 7fee88a02640 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c48e8c90 0x5625c48eb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.625+0000 7fee63fff640 10 client.18912 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.625+0000 7fee88a02640 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c48e8c90 0x5625c48eb0b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427a664759b9304f server_cookie=1d2b3fe21031be8c in_seq=0 out_seq=0 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.626+0000 7fee63fff640 1 -- 172.21.15.33:0/4155643689 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7386 (crc 0 0 0) 0x5625c47d3850 con 0x5625c48e8c90 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.626+0000 7fee63fff640 10 client.18912 handle_command_reply: tid=0 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.626+0000 7fee8d03b740 1 client.18912 shutdown 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f9c0 msgr2=0x5625c4882ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f9c0 0x5625c4882ee0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fee780b40b0 tx=0x7fee7803b890 comp rx=0 tx=0).stop 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 shutdown_connections 2024-09-10T21:23:11.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5625c48e8c90 0x5625c48eb0b0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5625c486b950 0x5625c4883670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5625c485f9c0 0x5625c4882ee0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 --2- 172.21.15.33:0/4155643689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5625c485f0a0 0x5625c4874260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee63fff640 0 client.18912 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 >> 172.21.15.33:0/4155643689 conn(0x5625c485ced0 msgr2=0x5625c48638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 shutdown_connections 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 1 -- 172.21.15.33:0/4155643689 wait complete. 2024-09-10T21:23:11.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.627+0000 7fee8d03b740 20 client.18912 trim_cache size 0 max 16384 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb", 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:23:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:23:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:23:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:23:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:23:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:23:11.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fee740b5350 msgr2=0x7fee740b7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fee740b5350 0x7fee740b7810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x5625c485ddb0 tx=0x7fee7000a2d0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee841076a0 msgr2=0x7fee8419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee841076a0 0x7fee8419b900 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fee78005ab0 tx=0x7fee78003d50 comp rx=0 tx=0).stop 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 shutdown_connections 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fee740b5350 0x7fee740b7810 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee84108020 0x7fee841a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fee841076a0 0x7fee8419b900 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 --2- 172.21.15.33:0/503609848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee84101c90 0x7fee8419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 >> 172.21.15.33:0/503609848 conn(0x7fee840fd700 msgr2=0x7fee84100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 shutdown_connections 2024-09-10T21:23:11.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:11.628+0000 7fee8ac66640 1 -- 172.21.15.33:0/503609848 wait complete. 2024-09-10T21:23:12.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 293151 40.64 MB/sec execute 252 sec latency 324.751 ms 2024-09-10T21:23:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:11 smithi174 ceph-mon[35635]: pgmap v922: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-10T21:23:12.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:11 smithi033 ceph-mon[30860]: pgmap v922: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-10T21:23:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:11 smithi104 ceph-mon[35325]: pgmap v922: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-10T21:23:12.454 INFO:tasks.cephfs.filesystem:scrub status for tag:3d46f877-fb1e-4f1a-80f8-e6908aafdfbb - {'path': '/', 'tag': '3d46f877-fb1e-4f1a-80f8-e6908aafdfbb', 'options': 'recursive,force'} 2024-09-10T21:23:13.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 293744 40.62 MB/sec execute 253 sec latency 127.539 ms 2024-09-10T21:23:14.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 294454 40.60 MB/sec execute 254 sec latency 103.328 ms 2024-09-10T21:23:14.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:13 smithi174 ceph-mon[35635]: pgmap v923: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 790 op/s 2024-09-10T21:23:14.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:13 smithi033 ceph-mon[30860]: pgmap v923: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 790 op/s 2024-09-10T21:23:14.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:13 smithi104 ceph-mon[35325]: pgmap v923: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 790 op/s 2024-09-10T21:23:15.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 295410 40.61 MB/sec execute 255 sec latency 105.035 ms 2024-09-10T21:23:16.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 296316 40.67 MB/sec execute 256 sec latency 62.980 ms 2024-09-10T21:23:16.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:15 smithi174 ceph-mon[35635]: pgmap v924: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 904 op/s 2024-09-10T21:23:16.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:15 smithi033 ceph-mon[30860]: pgmap v924: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 904 op/s 2024-09-10T21:23:16.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:15 smithi104 ceph-mon[35325]: pgmap v924: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 10 MiB/s wr, 904 op/s 2024-09-10T21:23:17.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 296821 40.63 MB/sec execute 257 sec latency 276.086 ms 2024-09-10T21:23:18.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 297449 40.57 MB/sec execute 258 sec latency 184.673 ms 2024-09-10T21:23:18.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:17 smithi174 ceph-mon[35635]: pgmap v925: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-09-10T21:23:18.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:17 smithi033 ceph-mon[30860]: pgmap v925: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-09-10T21:23:18.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:17 smithi104 ceph-mon[35325]: pgmap v925: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 9.2 MiB/s wr, 766 op/s 2024-09-10T21:23:19.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 298354 40.62 MB/sec execute 259 sec latency 64.366 ms 2024-09-10T21:23:20.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 299188 40.66 MB/sec execute 260 sec latency 88.395 ms 2024-09-10T21:23:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:19 smithi174 ceph-mon[35635]: pgmap v926: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 893 op/s 2024-09-10T21:23:20.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:19 smithi033 ceph-mon[30860]: pgmap v926: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 893 op/s 2024-09-10T21:23:20.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:19 smithi104 ceph-mon[35325]: pgmap v926: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 893 op/s 2024-09-10T21:23:21.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 299872 40.63 MB/sec execute 261 sec latency 67.567 ms 2024-09-10T21:23:22.066 INFO:tasks.workunit.client.0.smithi033.stdout: 10 300592 40.59 MB/sec execute 262 sec latency 204.707 ms 2024-09-10T21:23:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:21 smithi174 ceph-mon[35635]: pgmap v927: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-09-10T21:23:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:23:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:23:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:23:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:23:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:21 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:23:22.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:21 smithi033 ceph-mon[30860]: pgmap v927: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-09-10T21:23:22.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:23:22.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:23:22.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:23:22.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:23:22.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:21 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:23:22.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:21 smithi104 ceph-mon[35325]: pgmap v927: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 11 MiB/s wr, 872 op/s 2024-09-10T21:23:22.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:23:22.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:23:22.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:23:22.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:23:22.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:21 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:23:23.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 301194 40.59 MB/sec execute 263 sec latency 329.371 ms 2024-09-10T21:23:24.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 302013 40.59 MB/sec execute 264 sec latency 68.008 ms 2024-09-10T21:23:24.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:23 smithi174 ceph-mon[35635]: pgmap v928: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 750 op/s 2024-09-10T21:23:24.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:23 smithi033 ceph-mon[30860]: pgmap v928: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 750 op/s 2024-09-10T21:23:24.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:23 smithi104 ceph-mon[35325]: pgmap v928: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 750 op/s 2024-09-10T21:23:25.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 302907 40.61 MB/sec execute 265 sec latency 69.078 ms 2024-09-10T21:23:26.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 303880 40.69 MB/sec execute 266 sec latency 110.382 ms 2024-09-10T21:23:26.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:25 smithi174 ceph-mon[35635]: pgmap v929: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:23:26.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:25 smithi033 ceph-mon[30860]: pgmap v929: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:23:26.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:25 smithi104 ceph-mon[35325]: pgmap v929: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:23:27.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 304426 40.65 MB/sec execute 267 sec latency 198.899 ms 2024-09-10T21:23:28.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 304763 40.57 MB/sec execute 268 sec latency 350.726 ms 2024-09-10T21:23:28.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:27 smithi174 ceph-mon[35635]: pgmap v930: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-10T21:23:28.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:27 smithi033 ceph-mon[30860]: pgmap v930: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-10T21:23:28.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:27 smithi104 ceph-mon[35325]: pgmap v930: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 784 op/s 2024-09-10T21:23:29.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 305362 40.52 MB/sec execute 269 sec latency 108.542 ms 2024-09-10T21:23:30.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 306071 40.52 MB/sec execute 270 sec latency 84.177 ms 2024-09-10T21:23:30.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:29 smithi174 ceph-mon[35635]: pgmap v931: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 939 op/s 2024-09-10T21:23:30.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:29 smithi033 ceph-mon[30860]: pgmap v931: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 939 op/s 2024-09-10T21:23:30.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:29 smithi104 ceph-mon[35325]: pgmap v931: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 939 op/s 2024-09-10T21:23:31.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 306695 40.46 MB/sec execute 271 sec latency 107.805 ms 2024-09-10T21:23:32.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 307385 40.46 MB/sec execute 272 sec latency 87.037 ms 2024-09-10T21:23:32.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:31 smithi174 ceph-mon[35635]: pgmap v932: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:23:32.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:31 smithi033 ceph-mon[30860]: pgmap v932: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:23:32.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:31 smithi104 ceph-mon[35325]: pgmap v932: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 882 op/s 2024-09-10T21:23:33.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 307856 40.38 MB/sec execute 273 sec latency 352.039 ms 2024-09-10T21:23:34.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 308500 40.38 MB/sec execute 274 sec latency 117.090 ms 2024-09-10T21:23:34.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:33 smithi033 ceph-mon[30860]: pgmap v933: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.1 MiB/s wr, 771 op/s 2024-09-10T21:23:34.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:33 smithi104 ceph-mon[35325]: pgmap v933: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.1 MiB/s wr, 771 op/s 2024-09-10T21:23:34.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:33 smithi174 ceph-mon[35635]: pgmap v933: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.1 MiB/s wr, 771 op/s 2024-09-10T21:23:35.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 309127 40.38 MB/sec execute 275 sec latency 160.616 ms 2024-09-10T21:23:36.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 309778 40.37 MB/sec execute 276 sec latency 98.114 ms 2024-09-10T21:23:36.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:35 smithi033 ceph-mon[30860]: pgmap v934: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 875 op/s 2024-09-10T21:23:36.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:35 smithi104 ceph-mon[35325]: pgmap v934: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 875 op/s 2024-09-10T21:23:36.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:35 smithi174 ceph-mon[35635]: pgmap v934: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 875 op/s 2024-09-10T21:23:37.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 310469 40.35 MB/sec execute 277 sec latency 108.248 ms 2024-09-10T21:23:37.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:23:37.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:23:37.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:23:37.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:36 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:23:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:23:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:23:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:23:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:36 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:23:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:23:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:23:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:23:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:36 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:23:38.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 311064 40.31 MB/sec execute 278 sec latency 286.387 ms 2024-09-10T21:23:38.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:37 smithi033 ceph-mon[30860]: pgmap v935: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.3 MiB/s wr, 753 op/s 2024-09-10T21:23:38.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:37 smithi104 ceph-mon[35325]: pgmap v935: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.3 MiB/s wr, 753 op/s 2024-09-10T21:23:38.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:37 smithi174 ceph-mon[35635]: pgmap v935: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 9.3 MiB/s wr, 753 op/s 2024-09-10T21:23:39.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 311876 40.31 MB/sec execute 279 sec latency 71.498 ms 2024-09-10T21:23:40.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 312838 40.36 MB/sec execute 280 sec latency 56.543 ms 2024-09-10T21:23:40.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:39 smithi033 ceph-mon[30860]: pgmap v936: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 803 op/s 2024-09-10T21:23:40.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:39 smithi104 ceph-mon[35325]: pgmap v936: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 803 op/s 2024-09-10T21:23:40.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:39 smithi174 ceph-mon[35635]: pgmap v936: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 803 op/s 2024-09-10T21:23:41.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 313844 40.42 MB/sec execute 281 sec latency 90.464 ms 2024-09-10T21:23:42.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 314610 40.43 MB/sec execute 282 sec latency 71.599 ms 2024-09-10T21:23:42.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:41 smithi033 ceph-mon[30860]: pgmap v937: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 770 op/s 2024-09-10T21:23:42.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:41 smithi104 ceph-mon[35325]: pgmap v937: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 770 op/s 2024-09-10T21:23:42.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:41 smithi174 ceph-mon[35635]: pgmap v937: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 9.9 MiB/s wr, 770 op/s 2024-09-10T21:23:42.455 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:23:42.860 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:43.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 315070 40.37 MB/sec execute 283 sec latency 250.285 ms 2024-09-10T21:23:43.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.417+0000 7f4b2efc1640 1 -- 172.21.15.33:0/4264490322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b2810f3c0 msgr2=0x7f4b281117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.417+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/4264490322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b2810f3c0 0x7f4b281117b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009ca0 tx=0x7f4b2002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 -- 172.21.15.33:0/4264490322 shutdown_connections 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/4264490322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b2810f3c0 0x7f4b281117b0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/4264490322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b28075e00 0x7f4b2810ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/4264490322 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b280754e0 0x7f4b280758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 -- 172.21.15.33:0/4264490322 >> 172.21.15.33:0/4264490322 conn(0x7f4b280fe800 msgr2=0x7f4b28100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 -- 172.21.15.33:0/4264490322 shutdown_connections 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 -- 172.21.15.33:0/4264490322 wait complete. 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 Processor -- start 2024-09-10T21:23:43.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.418+0000 7f4b2efc1640 1 -- start start 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2efc1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b280754e0 0x7f4b2819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2efc1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b28075e00 0x7f4b2819b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2efc1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2810f3c0 0x7f4b281a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2efc1640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28114f90 con 0x7f4b28075e00 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2efc1640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28114e10 con 0x7f4b280754e0 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2efc1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b28115110 con 0x7f4b2810f3c0 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2d55e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2810f3c0 0x7f4b281a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b28075e00 0x7f4b2819b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2d55e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2810f3c0 0x7f4b281a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42206/0 (socket says 172.21.15.33:42206) 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.419+0000 7f4b2d55e640 1 -- 172.21.15.33:0/3570676556 learned_addr learned my addr 172.21.15.33:0/3570676556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b27fff640 1 -- 172.21.15.33:0/3570676556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2810f3c0 msgr2=0x7f4b281a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b27fff640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2810f3c0 0x7f4b281a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b27fff640 1 -- 172.21.15.33:0/3570676556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b280754e0 msgr2=0x7f4b2819b440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:23:43.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b27fff640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b280754e0 0x7f4b2819b440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b27fff640 1 -- 172.21.15.33:0/3570676556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b281a14f0 con 0x7f4b28075e00 2024-09-10T21:23:43.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b27fff640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b28075e00 0x7f4b2819b980 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400e9f0 tx=0x7f4b1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:43.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b25ffb640 1 -- 172.21.15.33:0/3570676556 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1400ccc0 con 0x7f4b28075e00 2024-09-10T21:23:43.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b281a17e0 con 0x7f4b28075e00 2024-09-10T21:23:43.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.420+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b281a7ea0 con 0x7f4b28075e00 2024-09-10T21:23:43.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.421+0000 7f4b25ffb640 1 -- 172.21.15.33:0/3570676556 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b140044d0 con 0x7f4b28075e00 2024-09-10T21:23:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.421+0000 7f4b25ffb640 1 -- 172.21.15.33:0/3570676556 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1401c070 con 0x7f4b28075e00 2024-09-10T21:23:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b25ffb640 1 -- 172.21.15.33:0/3570676556 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4b14005020 con 0x7f4b28075e00 2024-09-10T21:23:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 Processor -- start 2024-09-10T21:23:43.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 -- start start 2024-09-10T21:23:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d2620f4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4da0 0x55d2621007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d262103120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d26206ae50 con 0x55d2620f4da0 2024-09-10T21:23:43.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d26205a280 con 0x55d2620f4480 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.424+0000 7f4b31396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d26205e090 con 0x55d262100d30 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4da0 0x55d2621007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d262103120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4da0 0x55d2621007f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48390/0 (socket says 172.21.15.33:48390) 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/4051756926 learned_addr learned my addr 172.21.15.33:0/4051756926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d2620f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/4051756926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 msgr2=0x55d262103120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d262103120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/4051756926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 msgr2=0x55d2620f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d2620f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/4051756926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d261db8650 con 0x55d2620f4da0 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b25ffb640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4b180b52e0 0x7f4b180b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.425+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d2620f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.426+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4da0 0x55d2621007f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009bd0 tx=0x7f4b1c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d811a98efee29e75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.426+0000 7f4b25ffb640 1 -- 172.21.15.33:0/3570676556 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4b140e5ab0 con 0x7f4b28075e00 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.426+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4b180b52e0 0x7f4b180b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.426+0000 7f4b12ffd640 1 -- 172.21.15.33:0/4051756926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c005020 con 0x55d2620f4da0 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.426+0000 7f4b12ffd640 1 -- 172.21.15.33:0/4051756926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c0047a0 con 0x55d2620f4da0 2024-09-10T21:23:43.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.426+0000 7f4b12ffd640 1 -- 172.21.15.33:0/4051756926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c010690 con 0x55d2620f4da0 2024-09-10T21:23:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 -- 172.21.15.33:0/4051756926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4da0 msgr2=0x55d2621007f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4b180b52e0 0x7f4b180b77a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20005f30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4da0 0x55d2621007f0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009bd0 tx=0x7f4b1c00e720 comp rx=0 tx=0).stop 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 -- 172.21.15.33:0/4051756926 shutdown_connections 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d262103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4da0 0x55d2621007f0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 --2- 172.21.15.33:0/4051756926 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d2620f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 -- 172.21.15.33:0/4051756926 >> 172.21.15.33:0/4051756926 conn(0x55d2620f22b0 msgr2=0x55d2620f3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 -- 172.21.15.33:0/4051756926 shutdown_connections 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.427+0000 7f4b31396740 1 -- 172.21.15.33:0/4051756926 wait complete. 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 -- start start 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 Processor -- start 2024-09-10T21:23:43.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d262100550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d262100d30 0x55d2620fdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d261ebb700 con 0x55d262100d30 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d261ff5110 con 0x55d2620f4480 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2621039b0 con 0x55d2620f4da0 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.428+0000 7f4b31396740 10 client.0 _ll_register_callbacks cb 0x55d26200e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d262100d30 0x55d2620fdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2cd5d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d262100550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d262100d30 0x55d2620fdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48394/0 (socket says 172.21.15.33:48394) 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 -- 172.21.15.33:0/508846569 learned_addr learned my addr 172.21.15.33:0/508846569 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b27fff640 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 -- 172.21.15.33:0/508846569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 msgr2=0x55d2620fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 -- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 msgr2=0x55d262100550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d262100550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 -- 172.21.15.33:0/508846569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d262022c40 con 0x55d262100d30 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d262100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d262100d30 0x55d2620fdee0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009a30 tx=0x7f4b2002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e732c1afce588f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b07fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b07fff640 1 -- 172.21.15.33:0/508846569 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b20003c40 con 0x55d262100d30 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c0047a0 con 0x55d262100d30 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.429+0000 7f4b07fff640 1 -- 172.21.15.33:0/508846569 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4b20003de0 con 0x55d262100d30 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b07fff640 1 -- 172.21.15.33:0/508846569 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4b20004040 con 0x55d262100d30 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 10 client.18924 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d262117ac0 con 0x55d262100d30 2024-09-10T21:23:43.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b07fff640 1 -- 172.21.15.33:0/508846569 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c0047a0 con 0x55d262100d30 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b07fff640 1 -- 172.21.15.33:0/508846569 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f4b20051070 con 0x55d262100d30 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 10 client.18924 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 10 client.18924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 20 client.18924 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d26211b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 4 client.18924 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.430+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d26206cda0 con 0x55d262118fc0 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.431+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d26211b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.431+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d26211b440 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70fbf2d9161af9dd server_cookie=a9e4f689465a1a0f in_seq=0 out_seq=0 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.431+0000 7f4b07fff640 10 client.18924 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:43.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.432+0000 7f4b07fff640 1 -- 172.21.15.33:0/508846569 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d26206cda0 con 0x55d262118fc0 2024-09-10T21:23:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.432+0000 7f4b07fff640 10 client.18924 handle_command_reply: tid=0 2024-09-10T21:23:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 client.18924 shutdown 2024-09-10T21:23:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d262100d30 msgr2=0x55d2620fdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d262100d30 0x55d2620fdee0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f4b20009a30 tx=0x7f4b2002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 shutdown_connections 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d26211b440 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d262100d30 0x55d2620fdee0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b07fff640 0 client.18924 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 --2- 172.21.15.33:0/508846569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4480 0x55d262100550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 >> 172.21.15.33:0/508846569 conn(0x55d2620f22b0 msgr2=0x55d2620f7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 shutdown_connections 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.433+0000 7f4b31396740 1 -- 172.21.15.33:0/508846569 wait complete. 2024-09-10T21:23:43.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.434+0000 7f4b31396740 20 client.18924 trim_cache size 0 max 16384 2024-09-10T21:23:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 Processor -- start 2024-09-10T21:23:43.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 -- start start 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d2620fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d262100d30 0x55d262120d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c00e4c0 con 0x55d2620f4480 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c010000 con 0x55d262100d30 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.454+0000 7f4b31396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c00eb90 con 0x55d2620f4da0 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d2620fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d2620fabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48400/0 (socket says 172.21.15.33:48400) 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/3513685534 learned_addr learned my addr 172.21.15.33:0/3513685534 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:43.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d262100d30 0x55d262120d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/3513685534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 msgr2=0x55d2620fb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/3513685534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d262100d30 msgr2=0x55d262120d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d262100d30 0x55d262120d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/3513685534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2620fbea0 con 0x55d2620f4480 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.455+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d2620fabb0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009bd0 tx=0x7f4b1c004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24c7c642dc5e4fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.456+0000 7f4b127fc640 1 -- 172.21.15.33:0/3513685534 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c030070 con 0x55d2620f4480 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.456+0000 7f4b127fc640 1 -- 172.21.15.33:0/3513685534 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c009670 con 0x55d2620f4480 2024-09-10T21:23:43.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.456+0000 7f4b127fc640 1 -- 172.21.15.33:0/3513685534 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c028070 con 0x55d2620f4480 2024-09-10T21:23:43.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.456+0000 7f4b31396740 1 -- 172.21.15.33:0/3513685534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 msgr2=0x55d2620fabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.456+0000 7f4b31396740 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d2620fabb0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c009bd0 tx=0x7f4b1c004f40 comp rx=0 tx=0).stop 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.456+0000 7f4b31396740 1 -- 172.21.15.33:0/3513685534 shutdown_connections 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d262100d30 0x55d262120d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2620f4da0 0x55d2620fb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 --2- 172.21.15.33:0/3513685534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d2620fabb0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 -- 172.21.15.33:0/3513685534 >> 172.21.15.33:0/3513685534 conn(0x55d2620f22b0 msgr2=0x55d2620f8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 -- 172.21.15.33:0/3513685534 shutdown_connections 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 -- 172.21.15.33:0/3513685534 wait complete. 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 -- start start 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 Processor -- start 2024-09-10T21:23:43.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.457+0000 7f4b31396740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d262111a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b31396740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4da0 0x55d26210cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b31396740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d26210d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b31396740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c00f930 con 0x55d2620f4480 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b31396740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c00f260 con 0x55d2620f4da0 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b31396740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b0c00c480 con 0x55d262100d30 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b31396740 10 client.0 _ll_register_callbacks cb 0x7f4b14005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d262111a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b27fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4da0 0x55d26210cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d262111a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48414/0 (socket says 172.21.15.33:48414) 2024-09-10T21:23:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/1976570209 learned_addr learned my addr 172.21.15.33:0/1976570209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2d55e640 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d26210d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/1976570209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 msgr2=0x55d26210d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d26210d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/1976570209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4da0 msgr2=0x55d26210cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4da0 0x55d26210cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.458+0000 7f4b2cd5d640 1 -- 172.21.15.33:0/1976570209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26210daf0 con 0x55d2620f4480 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b2cd5d640 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d262111a80 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c005da0 tx=0x7f4b1c009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2c08cc8b3bd3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b07fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b07fff640 1 -- 172.21.15.33:0/1976570209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c038020 con 0x55d2620f4480 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b07fff640 1 -- 172.21.15.33:0/1976570209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4b1c004700 con 0x55d2620f4480 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c009670 con 0x55d2620f4480 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b07fff640 1 -- 172.21.15.33:0/1976570209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f4b1c030040 con 0x55d2620f4480 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b31396740 10 client.18930 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d262185d30 con 0x55d2620f4480 2024-09-10T21:23:43.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.459+0000 7f4b07fff640 1 -- 172.21.15.33:0/1976570209 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b1c009670 con 0x55d2620f4480 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b07fff640 1 -- 172.21.15.33:0/1976570209 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f4b1c033070 con 0x55d2620f4480 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b31396740 10 client.18930 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b31396740 10 client.18930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b31396740 20 client.18930 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b31396740 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d262188270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b31396740 4 client.18930 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d262068850 con 0x55d262118fc0 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b27fff640 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d262188270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b07fff640 10 client.18930 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:43.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.460+0000 7f4b27fff640 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d262188270 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3310cec1dca9c1c2 server_cookie=77e63d3079df44c0 in_seq=0 out_seq=0 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b07fff640 1 -- 172.21.15.33:0/1976570209 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7190 (crc 0 0 0) 0x55d262068850 con 0x55d262118fc0 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b07fff640 10 client.18930 handle_command_reply: tid=0 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 client.18930 shutdown 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 msgr2=0x55d262111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d262111a80 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c005da0 tx=0x7f4b1c009f00 comp rx=0 tx=0).stop 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 shutdown_connections 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d262118fc0 0x55d262188270 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d262100d30 0x55d26210d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d2620f4da0 0x55d26210cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 --2- 172.21.15.33:0/1976570209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d2620f4480 0x55d262111a80 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b07fff640 0 client.18930 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:43.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 >> 172.21.15.33:0/1976570209 conn(0x55d2620f22b0 msgr2=0x55d2620fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 shutdown_connections 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.464+0000 7f4b31396740 1 -- 172.21.15.33:0/1976570209 wait complete. 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.465+0000 7f4b31396740 20 client.18930 trim_cache size 0 max 16384 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.471 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:23:43.472 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.473 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:23:43.474 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.476 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:23:43.477 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.479 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.480 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.481 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.482 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.483 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.484 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:23:43.486 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:23:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.465+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4b180b52e0 msgr2=0x7f4b180b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.465+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4b180b52e0 0x7f4b180b77a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b20006010 tx=0x7f4b20005f30 comp rx=0 tx=0).stop 2024-09-10T21:23:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.465+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b28075e00 msgr2=0x7f4b2819b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.465+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b28075e00 0x7f4b2819b980 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400e9f0 tx=0x7f4b1400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 shutdown_connections 2024-09-10T21:23:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f4b180b52e0 0x7f4b180b77a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b2810f3c0 0x7f4b281a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4b28075e00 0x7f4b2819b980 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 --2- 172.21.15.33:0/3570676556 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b280754e0 0x7f4b2819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 >> 172.21.15.33:0/3570676556 conn(0x7f4b280fe800 msgr2=0x7f4b28111570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 shutdown_connections 2024-09-10T21:23:43.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:43.466+0000 7f4b2efc1640 1 -- 172.21.15.33:0/3570676556 wait complete. 2024-09-10T21:23:44.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 315773 40.34 MB/sec execute 284 sec latency 89.085 ms 2024-09-10T21:23:44.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:43 smithi033 ceph-mon[30860]: pgmap v938: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.3 MiB/s wr, 631 op/s 2024-09-10T21:23:44.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:43 smithi104 ceph-mon[35325]: pgmap v938: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.3 MiB/s wr, 631 op/s 2024-09-10T21:23:44.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:43 smithi174 ceph-mon[35635]: pgmap v938: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.3 MiB/s wr, 631 op/s 2024-09-10T21:23:44.848 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:23:44.848 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:23:45.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 316674 40.37 MB/sec execute 285 sec latency 68.898 ms 2024-09-10T21:23:45.159 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:45.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.644+0000 7f9118092640 1 -- 172.21.15.33:0/3565225547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911010f3d0 msgr2=0x7f91101117c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:45.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.644+0000 7f9118092640 1 --2- 172.21.15.33:0/3565225547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911010f3d0 0x7f91101117c0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f9108009900 tx=0x7f910802a410 comp rx=0 tx=0).stop 2024-09-10T21:23:45.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 -- 172.21.15.33:0/3565225547 shutdown_connections 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 --2- 172.21.15.33:0/3565225547 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f911010f3d0 0x7f91101117c0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 --2- 172.21.15.33:0/3565225547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110075de0 0x7f911010ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 --2- 172.21.15.33:0/3565225547 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91100754c0 0x7f91100758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 -- 172.21.15.33:0/3565225547 >> 172.21.15.33:0/3565225547 conn(0x7f91100fe870 msgr2=0x7f9110100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 -- 172.21.15.33:0/3565225547 shutdown_connections 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 -- 172.21.15.33:0/3565225547 wait complete. 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 Processor -- start 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.645+0000 7f9118092640 1 -- start start 2024-09-10T21:23:45.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9118092640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91100754c0 0x7f911019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9118092640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110075de0 0x7f911019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9118092640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f911010f3d0 0x7f91101a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9118092640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9110114f90 con 0x7f91100754c0 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9118092640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9110114e10 con 0x7f911010f3d0 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9118092640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9110115110 con 0x7f9110075de0 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91100754c0 0x7f911019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91100754c0 0x7f911019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48446/0 (socket says 172.21.15.33:48446) 2024-09-10T21:23:45.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 -- 172.21.15.33:0/1824633806 learned_addr learned my addr 172.21.15.33:0/1824633806 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f911662f640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f911010f3d0 0x7f91101a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f911562d640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110075de0 0x7f911019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 -- 172.21.15.33:0/1824633806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110075de0 msgr2=0x7f911019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110075de0 0x7f911019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 -- 172.21.15.33:0/1824633806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f911010f3d0 msgr2=0x7f91101a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f911010f3d0 0x7f91101a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 -- 172.21.15.33:0/1824633806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91101a15d0 con 0x7f91100754c0 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.646+0000 7f9115e2e640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91100754c0 0x7f911019b520 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f910000ebc0 tx=0x7f910000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.647+0000 7f910effd640 1 -- 172.21.15.33:0/1824633806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100010630 con 0x7f91100754c0 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.647+0000 7f910effd640 1 -- 172.21.15.33:0/1824633806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9100010c80 con 0x7f91100754c0 2024-09-10T21:23:45.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.647+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91101a18c0 con 0x7f91100754c0 2024-09-10T21:23:45.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.647+0000 7f910effd640 1 -- 172.21.15.33:0/1824633806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9100018b60 con 0x7f91100754c0 2024-09-10T21:23:45.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.647+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91101a8150 con 0x7f91100754c0 2024-09-10T21:23:45.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.648+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90fc0052c0 con 0x7f91100754c0 2024-09-10T21:23:45.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.649+0000 7f910effd640 1 -- 172.21.15.33:0/1824633806 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9100018420 con 0x7f91100754c0 2024-09-10T21:23:45.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.650+0000 7f910effd640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f90f80b5360 0x7f90f80b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:45.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.651+0000 7f910effd640 1 -- 172.21.15.33:0/1824633806 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f91000e6330 con 0x7f91100754c0 2024-09-10T21:23:45.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.651+0000 7f911562d640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f90f80b5360 0x7f90f80b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:45.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.651+0000 7f911562d640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f90f80b5360 0x7f90f80b7820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9110111990 tx=0x7f9104005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:45.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.656+0000 7f910effd640 1 -- 172.21.15.33:0/1824633806 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f91000acba0 con 0x7f91100754c0 2024-09-10T21:23:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.940+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90fc0055b0 con 0x7f91100754c0 2024-09-10T21:23:45.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.941+0000 7f910effd640 1 -- 172.21.15.33:0/1824633806 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+11973 (secure 0 0 0) 0x7f91000b15d0 con 0x7f91100754c0 2024-09-10T21:23:45.945 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:23:45.945 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":81,"btime":"2024-09-10T21:21:17:963637+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":81,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:21:17.000563+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:23:45.945 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 81 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f90f80b5360 msgr2=0x7f90f80b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f90f80b5360 0x7f90f80b7820 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9110111990 tx=0x7f9104005f90 comp rx=0 tx=0).stop 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91100754c0 msgr2=0x7f911019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91100754c0 0x7f911019b520 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f910000ebc0 tx=0x7f910000c6a0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 shutdown_connections 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f90f80b5360 0x7f90f80b7820 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f911010f3d0 0x7f91101a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9110075de0 0x7f911019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 --2- 172.21.15.33:0/1824633806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f91100754c0 0x7f911019b520 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:45.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 >> 172.21.15.33:0/1824633806 conn(0x7f91100fe870 msgr2=0x7f91100ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 shutdown_connections 2024-09-10T21:23:45.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:45.944+0000 7f9118092640 1 -- 172.21.15.33:0/1824633806 wait complete. 2024-09-10T21:23:46.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 317503 40.40 MB/sec execute 286 sec latency 72.910 ms 2024-09-10T21:23:46.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:45 smithi033 ceph-mon[30860]: pgmap v939: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 830 op/s 2024-09-10T21:23:46.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:45 smithi104 ceph-mon[35325]: pgmap v939: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 830 op/s 2024-09-10T21:23:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:45 smithi174 ceph-mon[35635]: pgmap v939: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 830 op/s 2024-09-10T21:23:46.857 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:23:47.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 318103 40.38 MB/sec execute 287 sec latency 98.843 ms 2024-09-10T21:23:47.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:46 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1824633806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:23:47.207 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:47.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:46 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1824633806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:23:47.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:46 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1824633806' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:23:47.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.833+0000 7f89b53fe640 1 -- 172.21.15.33:0/2762520738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0101250 msgr2=0x7f89b0101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.833+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2762520738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0101250 0x7f89b0101630 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009860 tx=0x7f89a802a340 comp rx=0 tx=0).stop 2024-09-10T21:23:47.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.833+0000 7f89b53fe640 1 -- 172.21.15.33:0/2762520738 shutdown_connections 2024-09-10T21:23:47.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.833+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2762520738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b0113860 0x7f89b0115c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.833+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2762520738 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b0110f30 0x7f89b0113320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.833+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2762520738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0101250 0x7f89b0101630 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.833+0000 7f89b53fe640 1 -- 172.21.15.33:0/2762520738 >> 172.21.15.33:0/2762520738 conn(0x7f89b0078140 msgr2=0x7f89b00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.837+0000 7f89b53fe640 1 -- 172.21.15.33:0/2762520738 shutdown_connections 2024-09-10T21:23:47.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.837+0000 7f89b53fe640 1 -- 172.21.15.33:0/2762520738 wait complete. 2024-09-10T21:23:47.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.838+0000 7f89b53fe640 1 Processor -- start 2024-09-10T21:23:47.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.838+0000 7f89b53fe640 1 -- start start 2024-09-10T21:23:47.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89b53fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b0101250 0x7f89b01104b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89b53fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b0110f30 0x7f89b01109f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89b53fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0113860 0x7f89b010a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89b53fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0109020 con 0x7f89b0101250 2024-09-10T21:23:47.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89b53fe640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89b0108ea0 con 0x7f89b0113860 2024-09-10T21:23:47.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89b53fe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01091a0 con 0x7f89b0110f30 2024-09-10T21:23:47.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0113860 0x7f89b010a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0113860 0x7f89b010a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38402/0 (socket says 172.21.15.33:38402) 2024-09-10T21:23:47.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.839+0000 7f89af7fe640 1 -- 172.21.15.33:0/2137374217 learned_addr learned my addr 172.21.15.33:0/2137374217 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:47.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.840+0000 7f89ae7fc640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b0110f30 0x7f89b01109f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.840+0000 7f89af7fe640 1 -- 172.21.15.33:0/2137374217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b0110f30 msgr2=0x7f89b01109f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.840+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b0110f30 0x7f89b01109f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.840+0000 7f89af7fe640 1 -- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b0101250 msgr2=0x7f89b01104b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.840+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b0101250 0x7f89b01104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.840+0000 7f89af7fe640 1 -- 172.21.15.33:0/2137374217 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b010b210 con 0x7f89b0113860 2024-09-10T21:23:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.841+0000 7f89ae7fc640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b0110f30 0x7f89b01109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.841+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0113860 0x7f89b010a950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f89a000eda0 tx=0x7f89a000c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.841+0000 7f89a7fff640 1 -- 172.21.15.33:0/2137374217 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0003b70 con 0x7f89b0113860 2024-09-10T21:23:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.841+0000 7f89a7fff640 1 -- 172.21.15.33:0/2137374217 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a0003d10 con 0x7f89b0113860 2024-09-10T21:23:47.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.841+0000 7f89a7fff640 1 -- 172.21.15.33:0/2137374217 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0017920 con 0x7f89b0113860 2024-09-10T21:23:47.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.841+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b010b500 con 0x7f89b0113860 2024-09-10T21:23:47.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.841+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b01b4b40 con 0x7f89b0113860 2024-09-10T21:23:47.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.845+0000 7f89a7fff640 1 -- 172.21.15.33:0/2137374217 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f89a000b030 con 0x7f89b0113860 2024-09-10T21:23:47.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.847+0000 7f89a7fff640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f899c0b53a0 0x7f899c0b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.847+0000 7f89aeffd640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f899c0b53a0 0x7f899c0b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.848+0000 7f89a7fff640 1 -- 172.21.15.33:0/2137374217 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f89a00e5a90 con 0x7f89b0113860 2024-09-10T21:23:47.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.848+0000 7f89aeffd640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f899c0b53a0 0x7f899c0b7860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009860 tx=0x7f89a8005b20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.849+0000 7f89b77d3740 1 Processor -- start 2024-09-10T21:23:47.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.849+0000 7f89b77d3740 1 -- start start 2024-09-10T21:23:47.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 0x55a3560648b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356064df0 0x55a356070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a356073170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89b77d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a355fdaea0 con 0x55a356070d80 2024-09-10T21:23:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89b77d3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a355fca2d0 con 0x55a3560644d0 2024-09-10T21:23:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89b77d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a355fce0e0 con 0x55a356064df0 2024-09-10T21:23:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356064df0 0x55a356070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356064df0 0x55a356070840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42308/0 (socket says 172.21.15.33:42308) 2024-09-10T21:23:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89ae7fc640 1 -- 172.21.15.33:0/2402729193 learned_addr learned my addr 172.21.15.33:0/2402729193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89aeffd640 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 0x55a3560648b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.850+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a356073170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.851+0000 7f89af7fe640 1 -- 172.21.15.33:0/2402729193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356064df0 msgr2=0x55a356070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:47.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.851+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356064df0 0x55a356070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.851+0000 7f89af7fe640 1 -- 172.21.15.33:0/2402729193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 msgr2=0x55a3560648b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:47.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.851+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 0x55a3560648b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.851+0000 7f89af7fe640 1 -- 172.21.15.33:0/2402729193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a355d28650 con 0x55a356070d80 2024-09-10T21:23:47.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.851+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a356073170 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f89a00a94c0 tx=0x7f89a00abc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f96d5f431170cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.852+0000 7f89a57fa640 1 -- 172.21.15.33:0/2402729193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00ac950 con 0x55a356070d80 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.852+0000 7f89a57fa640 1 -- 172.21.15.33:0/2402729193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a00ac250 con 0x55a356070d80 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.852+0000 7f89a57fa640 1 -- 172.21.15.33:0/2402729193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a00b5350 con 0x55a356070d80 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.852+0000 7f89b77d3740 1 -- 172.21.15.33:0/2402729193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 msgr2=0x55a356073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a356073170 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f89a00a94c0 tx=0x7f89a00abc60 comp rx=0 tx=0).stop 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 -- 172.21.15.33:0/2402729193 shutdown_connections 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a356073170 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356064df0 0x55a356070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2402729193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 0x55a3560648b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 -- 172.21.15.33:0/2402729193 >> 172.21.15.33:0/2402729193 conn(0x55a356062300 msgr2=0x55a3560634a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 -- 172.21.15.33:0/2402729193 shutdown_connections 2024-09-10T21:23:47.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.853+0000 7f89b77d3740 1 -- 172.21.15.33:0/2402729193 wait complete. 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 -- start start 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 Processor -- start 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 0x55a356070710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356064df0 0x55a35606db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 0x55a35606e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55a355e2b700 con 0x55a356064df0 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55a355f65160 con 0x55a3560644d0 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.854+0000 7f89b77d3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a356073a00 con 0x55a356070d80 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89b77d3740 10 client.0 _ll_register_callbacks cb 0x55a355f7e9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:47.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356064df0 0x55a35606db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356064df0 0x55a35606db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48470/0 (socket says 172.21.15.33:48470) 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 -- 172.21.15.33:0/3104622833 learned_addr learned my addr 172.21.15.33:0/3104622833 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 -- 172.21.15.33:0/3104622833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 msgr2=0x55a35606e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 0x55a35606e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 -- 172.21.15.33:0/3104622833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 msgr2=0x55a356070710 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 0x55a356070710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.855+0000 7f89ae7fc640 1 -- 172.21.15.33:0/3104622833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a355f92c90 con 0x55a356064df0 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.856+0000 7f89967fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.856+0000 7f89ae7fc640 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356064df0 0x55a35606db60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8998011460 tx=0x7f8998011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13506cf7bd0695f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.856+0000 7f89967fc640 1 -- 172.21.15.33:0/3104622833 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8998005220 con 0x55a356064df0 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.856+0000 7f89967fc640 1 -- 172.21.15.33:0/3104622833 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8998005020 con 0x55a356064df0 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.856+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a00ac250 con 0x55a356064df0 2024-09-10T21:23:47.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89967fc640 1 -- 172.21.15.33:0/3104622833 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f8998005220 con 0x55a356064df0 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89b77d3740 10 client.18945 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55a35607f480 con 0x55a356064df0 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89967fc640 1 -- 172.21.15.33:0/3104622833 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a00ac250 con 0x55a356064df0 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89967fc640 1 -- 172.21.15.33:0/3104622833 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f899800f040 con 0x55a356064df0 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89b77d3740 10 client.18945 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89b77d3740 10 client.18945 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.857+0000 7f89b77d3740 20 client.18945 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.858+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a356080980 0x55a356082e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.858+0000 7f89b77d3740 4 client.18945 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:47.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.858+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a355fdcdf0 con 0x55a356080980 2024-09-10T21:23:47.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.860+0000 7f89aeffd640 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a356080980 0x55a356082e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.862+0000 7f89aeffd640 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a356080980 0x55a356082e40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95c9522ad140269e server_cookie=37f137741b1b2036 in_seq=0 out_seq=0 2024-09-10T21:23:47.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.862+0000 7f89967fc640 10 client.18945 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:23:47.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.865+0000 7f89967fc640 1 -- 172.21.15.33:0/3104622833 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a355fdcdf0 con 0x55a356080980 2024-09-10T21:23:47.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.865+0000 7f89967fc640 10 client.18945 handle_command_reply: tid=0 2024-09-10T21:23:47.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.865+0000 7f89b77d3740 1 client.18945 shutdown 2024-09-10T21:23:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.866+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356064df0 msgr2=0x55a35606db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.866+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356064df0 0x55a35606db60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8998011460 tx=0x7f8998011920 comp rx=0 tx=0).stop 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.867+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 shutdown_connections 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.867+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a356080980 0x55a356082e40 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.867+0000 7f89967fc640 0 client.18945 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.867+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 0x55a35606e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.867+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356064df0 0x55a35606db60 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.867+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3104622833 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a3560644d0 0x55a356070710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.867+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 >> 172.21.15.33:0/3104622833 conn(0x55a356062300 msgr2=0x55a356067ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.868+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 shutdown_connections 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.868+0000 7f89b77d3740 1 -- 172.21.15.33:0/3104622833 wait complete. 2024-09-10T21:23:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.868+0000 7f89b77d3740 20 client.18945 trim_cache size 0 max 16384 2024-09-10T21:23:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.889+0000 7f89b77d3740 1 Processor -- start 2024-09-10T21:23:47.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.890+0000 7f89b77d3740 1 -- start start 2024-09-10T21:23:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.890+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3560644d0 0x55a35607cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.891+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356072170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.891+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 0x55a3560726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.891+0000 7f89b77d3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01bd580 con 0x55a3560644d0 2024-09-10T21:23:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.891+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.891+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3560644d0 0x55a35607cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.891+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3560644d0 0x55a35607cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48486/0 (socket says 172.21.15.33:48486) 2024-09-10T21:23:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.891+0000 7f89aeffd640 1 -- 172.21.15.33:0/2148773327 learned_addr learned my addr 172.21.15.33:0/2148773327 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.892+0000 7f89b77d3740 1 -- 172.21.15.33:0/2148773327 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01bf0c0 con 0x55a356064df0 2024-09-10T21:23:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.892+0000 7f89b77d3740 1 -- 172.21.15.33:0/2148773327 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01bdc50 con 0x55a356070d80 2024-09-10T21:23:47.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.892+0000 7f89af7fe640 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 0x55a3560726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.896+0000 7f89aeffd640 1 -- 172.21.15.33:0/2148773327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 msgr2=0x55a3560726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:47.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.896+0000 7f89aeffd640 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 0x55a3560726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.896+0000 7f89aeffd640 1 -- 172.21.15.33:0/2148773327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 msgr2=0x55a356072170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.896+0000 7f89aeffd640 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356072170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.896+0000 7f89aeffd640 1 -- 172.21.15.33:0/2148773327 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a356064c20 con 0x55a3560644d0 2024-09-10T21:23:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.896+0000 7f89aeffd640 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3560644d0 0x55a35607cc80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009830 tx=0x7f89a803b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9926dcbe2833b335 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.897+0000 7f89a4ff9640 1 -- 172.21.15.33:0/2148773327 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8037040 con 0x55a3560644d0 2024-09-10T21:23:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.897+0000 7f89a4ff9640 1 -- 172.21.15.33:0/2148773327 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89a8030900 con 0x55a3560644d0 2024-09-10T21:23:47.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.897+0000 7f89a4ff9640 1 -- 172.21.15.33:0/2148773327 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8003bf0 con 0x55a3560644d0 2024-09-10T21:23:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 -- 172.21.15.33:0/2148773327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3560644d0 msgr2=0x55a35607cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3560644d0 0x55a35607cc80 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009830 tx=0x7f89a803b000 comp rx=0 tx=0).stop 2024-09-10T21:23:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 -- 172.21.15.33:0/2148773327 shutdown_connections 2024-09-10T21:23:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a356070d80 0x55a3560726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356072170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 --2- 172.21.15.33:0/2148773327 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a3560644d0 0x55a35607cc80 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 -- 172.21.15.33:0/2148773327 >> 172.21.15.33:0/2148773327 conn(0x55a356062300 msgr2=0x55a35606f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 -- 172.21.15.33:0/2148773327 shutdown_connections 2024-09-10T21:23:47.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.898+0000 7f89b77d3740 1 -- 172.21.15.33:0/2148773327 wait complete. 2024-09-10T21:23:47.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.899+0000 7f89b77d3740 1 -- start start 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.899+0000 7f89b77d3740 1 Processor -- start 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3560644d0 0x55a3560791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89b77d3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a3560741e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3560644d0 0x55a3560791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89aeffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3560644d0 0x55a3560791e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:42344/0 (socket says 172.21.15.33:42344) 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a3560741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89af7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a3560741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48498/0 (socket says 172.21.15.33:48498) 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89ae7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356073ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38430/0 (socket says 172.21.15.33:38430) 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89aeffd640 1 -- 172.21.15.33:0/3848300166 learned_addr learned my addr 172.21.15.33:0/3848300166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01be9f0 con 0x55a356070d80 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01be320 con 0x55a356064df0 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.900+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89b01bb500 con 0x55a3560644d0 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89aeffd640 1 -- 172.21.15.33:0/3848300166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 msgr2=0x55a356073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89aeffd640 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89aeffd640 1 -- 172.21.15.33:0/3848300166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 msgr2=0x55a3560741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89aeffd640 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a3560741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89aeffd640 1 -- 172.21.15.33:0/3848300166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a356074750 con 0x55a3560644d0 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89af7fe640 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a3560741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:47.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89b77d3740 10 client.0 _ll_register_callbacks cb 0x7f8998009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89a8030900 con 0x55a3560644d0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89aeffd640 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3560644d0 0x55a3560791e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005cd0 tx=0x7f89a80035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfd40243b074e9bd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.901+0000 7f89967fc640 10 client.28195 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.902+0000 7f89967fc640 1 -- 172.21.15.33:0/3848300166 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8037070 con 0x55a3560644d0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.902+0000 7f89967fc640 1 -- 172.21.15.33:0/3848300166 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f89a8030650 con 0x55a3560644d0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.902+0000 7f89967fc640 1 -- 172.21.15.33:0/3848300166 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f89a8004040 con 0x55a3560644d0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.902+0000 7f89b77d3740 10 client.28195 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.902+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=81}) -- 0x55a3560ee470 con 0x55a3560644d0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.903+0000 7f89967fc640 1 -- 172.21.15.33:0/3848300166 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f89a803a030 con 0x55a3560644d0 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.903+0000 7f89b77d3740 10 client.28195 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:47.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.903+0000 7f89b77d3740 10 client.28195 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:23:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.903+0000 7f89b77d3740 20 client.28195 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.903+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a3560eea80 0x55a3560f0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.903+0000 7f89b77d3740 4 client.28195 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.903+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a355fd88a0 con 0x55a3560eea80 2024-09-10T21:23:47.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.905+0000 7f89ae7fc640 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a3560eea80 0x55a3560f0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.905+0000 7f89ae7fc640 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a3560eea80 0x55a3560f0ee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2562e41fe952b940 server_cookie=2ba2ce2bec90ccc7 in_seq=0 out_seq=0 2024-09-10T21:23:47.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.905+0000 7f89967fc640 10 client.28195 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:23:47.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.915+0000 7f89967fc640 1 -- 172.21.15.33:0/3848300166 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a355fd88a0 con 0x55a3560eea80 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.915+0000 7f89967fc640 10 client.28195 handle_command_reply: tid=0 2024-09-10T21:23:47.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.915+0000 7f89b77d3740 1 client.28195 shutdown 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3560644d0 msgr2=0x55a3560791e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3560644d0 0x55a3560791e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005cd0 tx=0x7f89a80035f0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 shutdown_connections 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x55a3560eea80 0x55a3560f0ee0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55a356070d80 0x55a3560741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89967fc640 0 client.28195 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55a356064df0 0x55a356073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 --2- 172.21.15.33:0/3848300166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3560644d0 0x55a3560791e0 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f89a8005cd0 tx=0x7f89a80035f0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.916+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 >> 172.21.15.33:0/3848300166 conn(0x55a356062300 msgr2=0x55a356060c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.917+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 shutdown_connections 2024-09-10T21:23:47.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.917+0000 7f89b77d3740 1 -- 172.21.15.33:0/3848300166 wait complete. 2024-09-10T21:23:47.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.917+0000 7f89b77d3740 20 client.28195 trim_cache size 0 max 16384 2024-09-10T21:23:47.921 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f899c0b53a0 msgr2=0x7f899c0b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f899c0b53a0 0x7f899c0b7860 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f89a8009860 tx=0x7f89a8005b20 comp rx=0 tx=0).stop 2024-09-10T21:23:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0113860 msgr2=0x7f89b010a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0113860 0x7f89b010a950 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f89a000eda0 tx=0x7f89a000c6e0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 shutdown_connections 2024-09-10T21:23:47.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f899c0b53a0 0x7f899c0b7860 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f89b0113860 0x7f89b010a950 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89b0110f30 0x7f89b01109f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 --2- 172.21.15.33:0/2137374217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f89b0101250 0x7f89b01104b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 >> 172.21.15.33:0/2137374217 conn(0x7f89b0078140 msgr2=0x7f89b01140e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 shutdown_connections 2024-09-10T21:23:47.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:47.918+0000 7f89b53fe640 1 -- 172.21.15.33:0/2137374217 wait complete. 2024-09-10T21:23:48.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 318462 40.29 MB/sec execute 288 sec latency 406.775 ms 2024-09-10T21:23:48.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:47 smithi033 ceph-mon[30860]: pgmap v940: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 779 op/s 2024-09-10T21:23:48.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:47 smithi104 ceph-mon[35325]: pgmap v940: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 779 op/s 2024-09-10T21:23:48.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:47 smithi174 ceph-mon[35635]: pgmap v940: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.2 MiB/s wr, 779 op/s 2024-09-10T21:23:48.869 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:23:49.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 319149 40.27 MB/sec execute 289 sec latency 113.650 ms 2024-09-10T21:23:49.205 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:49.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 -- 172.21.15.33:0/2198965906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c40102de0 msgr2=0x7f8c401031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 --2- 172.21.15.33:0/2198965906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c40102de0 0x7f8c401031c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009900 tx=0x7f8c3002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 -- 172.21.15.33:0/2198965906 shutdown_connections 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 --2- 172.21.15.33:0/2198965906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40109170 0x7f8c4010f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 --2- 172.21.15.33:0/2198965906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c401087f0 0x7f8c40108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 --2- 172.21.15.33:0/2198965906 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c40102de0 0x7f8c401031c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 -- 172.21.15.33:0/2198965906 >> 172.21.15.33:0/2198965906 conn(0x7f8c400fe830 msgr2=0x7f8c40100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.910+0000 7f8c454ea640 1 -- 172.21.15.33:0/2198965906 shutdown_connections 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.911+0000 7f8c454ea640 1 -- 172.21.15.33:0/2198965906 wait complete. 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.911+0000 7f8c454ea640 1 Processor -- start 2024-09-10T21:23:49.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.912+0000 7f8c454ea640 1 -- start start 2024-09-10T21:23:49.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.912+0000 7f8c454ea640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40102de0 0x7f8c401990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.912+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40102de0 0x7f8c401990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.912+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40102de0 0x7f8c401990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35748/0 (socket says 172.21.15.33:35748) 2024-09-10T21:23:49.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c454ea640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c401087f0 0x7f8c40199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3effd640 1 -- 172.21.15.33:0/1261355898 learned_addr learned my addr 172.21.15.33:0/1261355898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c401087f0 0x7f8c40199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c454ea640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40109170 0x7f8c4019e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40112e80 con 0x7f8c40102de0 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40112d00 con 0x7f8c401087f0 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c40113000 con 0x7f8c40109170 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/1261355898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40109170 msgr2=0x7f8c4019e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40109170 0x7f8c4019e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40102de0 msgr2=0x7f8c401990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40102de0 0x7f8c401990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/1261355898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4019f190 con 0x7f8c401087f0 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.913+0000 7f8c3effd640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40102de0 0x7f8c401990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:49.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.914+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c401087f0 0x7f8c40199620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400e9f0 tx=0x7f8c3400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.914+0000 7f8c27fff640 1 -- 172.21.15.33:0/1261355898 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3400ccc0 con 0x7f8c401087f0 2024-09-10T21:23:49.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.914+0000 7f8c27fff640 1 -- 172.21.15.33:0/1261355898 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c340108e0 con 0x7f8c401087f0 2024-09-10T21:23:49.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.914+0000 7f8c27fff640 1 -- 172.21.15.33:0/1261355898 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c340188e0 con 0x7f8c401087f0 2024-09-10T21:23:49.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.914+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4019f480 con 0x7f8c401087f0 2024-09-10T21:23:49.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.914+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c40075380 con 0x7f8c401087f0 2024-09-10T21:23:49.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.918+0000 7f8c27fff640 1 -- 172.21.15.33:0/1261355898 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8c34010430 con 0x7f8c401087f0 2024-09-10T21:23:49.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.918+0000 7f8c478bf740 1 Processor -- start 2024-09-10T21:23:49.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.918+0000 7f8c478bf740 1 -- start start 2024-09-10T21:23:49.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bddd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bdebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c27fff640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c280b5430 0x7f8c280b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c3f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bddd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c3f7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bddd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55430/0 (socket says 172.21.15.33:55430) 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46226/0 (socket says 172.21.15.33:46226) 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55951bd53ea0 con 0x55951bde99a0 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c3f7fe640 1 -- 172.21.15.33:0/1337701712 learned_addr learned my addr 172.21.15.33:0/1337701712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55951bd432d0 con 0x55951bddd0f0 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55951bd470e0 con 0x55951bddda10 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bdebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c280b5430 0x7f8c280b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.919+0000 7f8c27fff640 1 -- 172.21.15.33:0/1261355898 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8c340e5fa0 con 0x7f8c401087f0 2024-09-10T21:23:49.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c3f7fe640 1 -- 172.21.15.33:0/1337701712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 msgr2=0x55951bde9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c3f7fe640 1 -- 172.21.15.33:0/1337701712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 msgr2=0x55951bdebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bdebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c3f7fe640 1 -- 172.21.15.33:0/1337701712 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951baa1650 con 0x55951bddd0f0 2024-09-10T21:23:49.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bddd4d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8c38013e30 tx=0x7f8c38010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39a8024123b8e9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c280b5430 0x7f8c280b78f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8c38010c40 tx=0x7f8c38015000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c257fa640 1 -- 172.21.15.33:0/1337701712 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38009d30 con 0x55951bddd0f0 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c257fa640 1 -- 172.21.15.33:0/1337701712 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c38013780 con 0x55951bddd0f0 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.920+0000 7f8c257fa640 1 -- 172.21.15.33:0/1337701712 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3800bd10 con 0x55951bddd0f0 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 msgr2=0x55951bddd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bddd4d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8c38013e30 tx=0x7f8c38010740 comp rx=0 tx=0).stop 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 shutdown_connections 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bdebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 --2- 172.21.15.33:0/1337701712 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bddd4d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 >> 172.21.15.33:0/1337701712 conn(0x55951bddaf20 msgr2=0x55951bddc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 shutdown_connections 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.921+0000 7f8c478bf740 1 -- 172.21.15.33:0/1337701712 wait complete. 2024-09-10T21:23:49.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 -- start start 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 Processor -- start 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bde92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55951bba4700 con 0x55951bde99a0 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55951bcde160 con 0x55951bddd0f0 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55951bdec5c0 con 0x55951bddda10 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c478bf740 10 client.0 _ll_register_callbacks cb 0x55951bcf79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c3f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c3e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c3f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35760/0 (socket says 172.21.15.33:35760) 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bde92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.922+0000 7f8c3f7fe640 1 -- 172.21.15.33:0/3404613932 learned_addr learned my addr 172.21.15.33:0/3404613932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:49.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c3effd640 1 -- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 msgr2=0x55951bde6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c3effd640 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c3effd640 1 -- 172.21.15.33:0/3404613932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 msgr2=0x55951bde6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c3effd640 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c3effd640 1 -- 172.21.15.33:0/3404613932 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951bd0bc90 con 0x55951bddd0f0 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c3effd640 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bde92d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8c300057c0 tx=0x7f8c30005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3584abb8dc3a08ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3404613932 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30035070 con 0x55951bddd0f0 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3404613932 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8c30004250 con 0x55951bddd0f0 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c38013780 con 0x55951bddd0f0 2024-09-10T21:23:49.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3404613932 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f8c30004450 con 0x55951bddd0f0 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c478bf740 10 client.28334 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.923+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55951bdf7f30 con 0x55951bddd0f0 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.924+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3404613932 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c38013780 con 0x55951bddd0f0 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.924+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3404613932 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f8c30056700 con 0x55951bddd0f0 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.925+0000 7f8c478bf740 10 client.28334 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.925+0000 7f8c478bf740 10 client.28334 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.925+0000 7f8c478bf740 20 client.28334 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.925+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951bdf93f0 0x55951bdfb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.925+0000 7f8c478bf740 4 client.28334 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.925+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55951bd55df0 con 0x55951bdf93f0 2024-09-10T21:23:49.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.925+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951bdf93f0 0x55951bdfb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.926+0000 7f8c22ff5640 10 client.28334 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:23:49.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.926+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951bdf93f0 0x55951bdfb800 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6cae0068f4839113 server_cookie=8f747a6c9253478e in_seq=0 out_seq=0 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.927+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3404613932 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55951bd55df0 con 0x55951bdf93f0 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.927+0000 7f8c22ff5640 10 client.28334 handle_command_reply: tid=0 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.927+0000 7f8c478bf740 1 client.28334 shutdown 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.927+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 msgr2=0x55951bde92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.927+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bde92d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8c300057c0 tx=0x7f8c30005a40 comp rx=0 tx=0).stop 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 shutdown_connections 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951bdf93f0 0x55951bdfb800 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddda10 0x55951bde6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3404613932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddd0f0 0x55951bde92d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c22ff5640 0 client.28334 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 >> 172.21.15.33:0/3404613932 conn(0x55951bddaf20 msgr2=0x55951bde0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 shutdown_connections 2024-09-10T21:23:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 1 -- 172.21.15.33:0/3404613932 wait complete. 2024-09-10T21:23:49.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.928+0000 7f8c478bf740 20 client.28334 trim_cache size 0 max 16384 2024-09-10T21:23:49.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.942+0000 7f8c478bf740 1 Processor -- start 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c478bf740 1 -- start start 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddda10 0x55951bde5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c478bf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401ac0c0 con 0x55951bde99a0 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c478bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401adc00 con 0x55951bddda10 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c478bf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401ac790 con 0x55951bddd0f0 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddda10 0x55951bde5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46240/0 (socket says 172.21.15.33:46240) 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3effd640 1 -- 172.21.15.33:0/3464784709 learned_addr learned my addr 172.21.15.33:0/3464784709 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3e7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddda10 0x55951bde5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55452/0 (socket says 172.21.15.33:55452) 2024-09-10T21:23:49.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/3464784709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 msgr2=0x55951bdf57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.943+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.944+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/3464784709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 msgr2=0x55951bde19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.944+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.944+0000 7f8c3e7fc640 1 -- 172.21.15.33:0/3464784709 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951bde9640 con 0x55951bddda10 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.944+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddda10 0x55951bde5c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8c340a8d30 tx=0x7f8c3400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f09cc0f64411b3b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.944+0000 7f8c24ff9640 1 -- 172.21.15.33:0/3464784709 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c340a6e60 con 0x55951bddda10 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.944+0000 7f8c24ff9640 1 -- 172.21.15.33:0/3464784709 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c340ac760 con 0x55951bddda10 2024-09-10T21:23:49.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.944+0000 7f8c24ff9640 1 -- 172.21.15.33:0/3464784709 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c340ab0a0 con 0x55951bddda10 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 -- 172.21.15.33:0/3464784709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddda10 msgr2=0x55951bde5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddda10 0x55951bde5c10 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8c340a8d30 tx=0x7f8c3400cf60 comp rx=0 tx=0).stop 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 -- 172.21.15.33:0/3464784709 shutdown_connections 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bde99a0 0x55951bde19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bddda10 0x55951bde5c10 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3464784709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 -- 172.21.15.33:0/3464784709 >> 172.21.15.33:0/3464784709 conn(0x55951bddaf20 msgr2=0x55951bddbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 -- 172.21.15.33:0/3464784709 shutdown_connections 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 -- 172.21.15.33:0/3464784709 wait complete. 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 -- start start 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.945+0000 7f8c478bf740 1 Processor -- start 2024-09-10T21:23:49.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bddda10 0x55951bdeadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c478bf740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bde99a0 0x55951bdeb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c478bf740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401ad530 con 0x55951bddda10 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c478bf740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401ace60 con 0x55951bde99a0 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c478bf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c401aa080 con 0x55951bddd0f0 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46254/0 (socket says 172.21.15.33:46254) 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 -- 172.21.15.33:0/3103657662 learned_addr learned my addr 172.21.15.33:0/3103657662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bddda10 0x55951bdeadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c478bf740 10 client.0 _ll_register_callbacks cb 0x7f8c34005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bde99a0 0x55951bdeb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 -- 172.21.15.33:0/3103657662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bde99a0 msgr2=0x55951bdeb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bde99a0 0x55951bdeb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 -- 172.21.15.33:0/3103657662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bddda10 msgr2=0x55951bdeadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bddda10 0x55951bdeadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 -- 172.21.15.33:0/3103657662 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951bdf0d60 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3f7fe640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bde99a0 0x55951bdeb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.946+0000 7f8c3effd640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf2100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c30000c00 tx=0x7f8c3003b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f28e8ae30e55207 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c22ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3103657662 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c30035070 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3103657662 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8c3003bcd0 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c340ac760 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3103657662 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f8c30009520 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 10 client.28210 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55951be663f0 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3103657662 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c30005520 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3103657662 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f8c30004a80 con 0x55951bddd0f0 2024-09-10T21:23:49.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 10 client.28210 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 10 client.28210 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 20 client.28210 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951be66bf0 0x55951be69010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 4 client.28210 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.947+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f8c340ac760 con 0x55951be66bf0 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951be66bf0 0x55951be69010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c3e7fc640 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951be66bf0 0x55951be69010 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb1432e549e5a876 server_cookie=8e743eff29d91060 in_seq=0 out_seq=0 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c22ff5640 10 client.28210 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c22ff5640 1 -- 172.21.15.33:0/3103657662 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8c340ac760 con 0x55951be66bf0 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c22ff5640 10 client.28210 handle_command_reply: tid=0 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c478bf740 1 client.28210 shutdown 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 msgr2=0x55951bdf2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.948+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf2100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c30000c00 tx=0x7f8c3003b7d0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 shutdown_connections 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55951bde99a0 0x55951bdeb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55951bddda10 0x55951bdeadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55951be66bf0 0x55951be69010 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 --2- 172.21.15.33:0/3103657662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55951bddd0f0 0x55951bdf2100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 >> 172.21.15.33:0/3103657662 conn(0x55951bddaf20 msgr2=0x55951bde19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c22ff5640 0 client.28210 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 shutdown_connections 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 1 -- 172.21.15.33:0/3103657662 wait complete. 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.949+0000 7f8c478bf740 20 client.28210 trim_cache size 0 max 16384 2024-09-10T21:23:49.957 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c280b5430 msgr2=0x7f8c280b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c280b5430 0x7f8c280b78f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8c38010c40 tx=0x7f8c38015000 comp rx=0 tx=0).stop 2024-09-10T21:23:49.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c401087f0 msgr2=0x7f8c40199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c401087f0 0x7f8c40199620 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c3400e9f0 tx=0x7f8c3400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 shutdown_connections 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8c280b5430 0x7f8c280b78f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c40109170 0x7f8c4019e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c401087f0 0x7f8c40199620 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 --2- 172.21.15.33:0/1261355898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8c40102de0 0x7f8c401990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 >> 172.21.15.33:0/1261355898 conn(0x7f8c400fe830 msgr2=0x7f8c401055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 shutdown_connections 2024-09-10T21:23:49.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:49.950+0000 7f8c454ea640 1 -- 172.21.15.33:0/1261355898 wait complete. 2024-09-10T21:23:50.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 319890 40.30 MB/sec execute 290 sec latency 123.010 ms 2024-09-10T21:23:50.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:49 smithi033 ceph-mon[30860]: pgmap v941: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 812 op/s 2024-09-10T21:23:50.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:49 smithi104 ceph-mon[35325]: pgmap v941: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 812 op/s 2024-09-10T21:23:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:49 smithi174 ceph-mon[35635]: pgmap v941: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 812 op/s 2024-09-10T21:23:50.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:23:51.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 320567 40.29 MB/sec execute 291 sec latency 108.720 ms 2024-09-10T21:23:51.285 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:52.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 321134 40.27 MB/sec execute 292 sec latency 281.554 ms 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.073+0000 7f50f3d00640 1 -- 172.21.15.33:0/2544835680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 msgr2=0x7f50ec103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.073+0000 7f50f3d00640 1 --2- 172.21.15.33:0/2544835680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 0x7f50ec103f10 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009900 tx=0x7f50e002a410 comp rx=0 tx=0).stop 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.074+0000 7f50f3d00640 1 -- 172.21.15.33:0/2544835680 shutdown_connections 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.074+0000 7f50f3d00640 1 --2- 172.21.15.33:0/2544835680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50ec10d0e0 0x7f50ec10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.074+0000 7f50f3d00640 1 --2- 172.21.15.33:0/2544835680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50ec104530 0x7f50ec10cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.074+0000 7f50f3d00640 1 --2- 172.21.15.33:0/2544835680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 0x7f50ec103f10 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.074+0000 7f50f3d00640 1 -- 172.21.15.33:0/2544835680 >> 172.21.15.33:0/2544835680 conn(0x7f50ec0fd700 msgr2=0x7f50ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.074+0000 7f50f3d00640 1 -- 172.21.15.33:0/2544835680 shutdown_connections 2024-09-10T21:23:52.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.074+0000 7f50f3d00640 1 -- 172.21.15.33:0/2544835680 wait complete. 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 Processor -- start 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 -- start start 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 0x7f50ec199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50ec104530 0x7f50ec1997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50ec10d0e0 0x7f50ec19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec112d60 con 0x7f50ec103b30 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec112be0 con 0x7f50ec104530 2024-09-10T21:23:52.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f229d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50ec10d0e0 0x7f50ec19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f229d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50ec10d0e0 0x7f50ec19eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46272/0 (socket says 172.21.15.33:46272) 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f229d640 1 -- 172.21.15.33:0/877474342 learned_addr learned my addr 172.21.15.33:0/877474342 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec112ee0 con 0x7f50ec10d0e0 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 0x7f50ec199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.075+0000 7f50f129b640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50ec104530 0x7f50ec1997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.076+0000 7f50f1a9c640 1 -- 172.21.15.33:0/877474342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50ec10d0e0 msgr2=0x7f50ec19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.076+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50ec10d0e0 0x7f50ec19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.076+0000 7f50f1a9c640 1 -- 172.21.15.33:0/877474342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50ec104530 msgr2=0x7f50ec1997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.076+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50ec104530 0x7f50ec1997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.076+0000 7f50f1a9c640 1 -- 172.21.15.33:0/877474342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50ec19f340 con 0x7f50ec103b30 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.076+0000 7f50f129b640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50ec104530 0x7f50ec1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:52.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.076+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 0x7f50ec199290 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f50e002a930 tx=0x7f50e0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.077+0000 7f50deffd640 1 -- 172.21.15.33:0/877474342 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e003f030 con 0x7f50ec103b30 2024-09-10T21:23:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.077+0000 7f50deffd640 1 -- 172.21.15.33:0/877474342 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e0004490 con 0x7f50ec103b30 2024-09-10T21:23:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.077+0000 7f50deffd640 1 -- 172.21.15.33:0/877474342 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0035070 con 0x7f50ec103b30 2024-09-10T21:23:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.077+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50ec19f5d0 con 0x7f50ec103b30 2024-09-10T21:23:52.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.077+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50ec101890 con 0x7f50ec103b30 2024-09-10T21:23:52.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.081+0000 7f50deffd640 1 -- 172.21.15.33:0/877474342 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f50e003b9c0 con 0x7f50ec103b30 2024-09-10T21:23:52.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.082+0000 7f50f60d5740 1 Processor -- start 2024-09-10T21:23:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.083+0000 7f50f60d5740 1 -- start start 2024-09-10T21:23:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.083+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e008b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.083+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522e15050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e008b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50deffd640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50d80b5430 0x7f50d80b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e008b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46282/0 (socket says 172.21.15.33:46282) 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 -- 172.21.15.33:0/3836154066 learned_addr learned my addr 172.21.15.33:0/3836154066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e17980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559522d76ea0 con 0x559522e00df0 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559522d662d0 con 0x559522e15590 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559522d6a0e0 con 0x559522e004d0 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522e15050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50deffd640 1 -- 172.21.15.33:0/877474342 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50e0105fe0 con 0x7f50ec103b30 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e17980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f129b640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50d80b5430 0x7f50d80b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 -- 172.21.15.33:0/3836154066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 msgr2=0x559522e17980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e17980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 -- 172.21.15.33:0/3836154066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 msgr2=0x559522e15050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522e15050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.084+0000 7f50f229d640 1 -- 172.21.15.33:0/3836154066 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559522ac4650 con 0x559522e004d0 2024-09-10T21:23:52.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.085+0000 7f50f129b640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50d80b5430 0x7f50d80b78f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50ec10f7c0 tx=0x7f50e8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:52.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.085+0000 7f50f229d640 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e008b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f50e4011a20 tx=0x7f50e4009da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=effe988ea7accd00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:52.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.085+0000 7f50d7fff640 1 -- 172.21.15.33:0/3836154066 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e401a030 con 0x559522e004d0 2024-09-10T21:23:52.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.085+0000 7f50d7fff640 1 -- 172.21.15.33:0/3836154066 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e400fa00 con 0x559522e004d0 2024-09-10T21:23:52.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.085+0000 7f50d7fff640 1 -- 172.21.15.33:0/3836154066 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4005500 con 0x559522e004d0 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.086+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 msgr2=0x559522e008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.086+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e008b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f50e4011a20 tx=0x7f50e4009da0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 shutdown_connections 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e17980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522e15050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3836154066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e008b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 >> 172.21.15.33:0/3836154066 conn(0x559522dfe300 msgr2=0x559522dff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 shutdown_connections 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 -- 172.21.15.33:0/3836154066 wait complete. 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 -- start start 2024-09-10T21:23:52.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.087+0000 7f50f60d5740 1 Processor -- start 2024-09-10T21:23:52.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e004d0 0x559522e14f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e00df0 0x559522e12350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e129d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f60d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559522bc7700 con 0x559522e15590 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f60d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x559522d01160 con 0x559522e004d0 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f60d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559522e18240 con 0x559522e00df0 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f60d5740 10 client.0 _ll_register_callbacks cb 0x559522d1a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f129b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e00df0 0x559522e12350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f129b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e00df0 0x559522e12350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46288/0 (socket says 172.21.15.33:46288) 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f129b640 1 -- 172.21.15.33:0/15666913 learned_addr learned my addr 172.21.15.33:0/15666913 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.088+0000 7f50f229d640 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e129d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.089+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e004d0 0x559522e14f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.089+0000 7f50f229d640 1 -- 172.21.15.33:0/15666913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e00df0 msgr2=0x559522e12350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.089+0000 7f50f229d640 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e00df0 0x559522e12350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.089+0000 7f50f229d640 1 -- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e004d0 msgr2=0x559522e14f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.089+0000 7f50f229d640 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e004d0 0x559522e14f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.089+0000 7f50f229d640 1 -- 172.21.15.33:0/15666913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559522d2ec90 con 0x559522e15590 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.090+0000 7f50f229d640 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e129d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f50e4014040 tx=0x7f50e4004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d9c72057db4310 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.090+0000 7f50d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.090+0000 7f50d57fa640 1 -- 172.21.15.33:0/15666913 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e401a040 con 0x559522e15590 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.090+0000 7f50d57fa640 1 -- 172.21.15.33:0/15666913 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50e4005400 con 0x559522e15590 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.090+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e400fa00 con 0x559522e15590 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50d57fa640 1 -- 172.21.15.33:0/15666913 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f50e402e050 con 0x559522e15590 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 10 client.18981 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:52.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x559522e23bf0 con 0x559522e15590 2024-09-10T21:23:52.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50d57fa640 1 -- 172.21.15.33:0/15666913 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e401f460 con 0x559522e15590 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50d57fa640 1 -- 172.21.15.33:0/15666913 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f50e40283f0 con 0x559522e15590 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 10 client.18981 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 10 client.18981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 20 client.18981 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e27330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 4 client.18981 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.091+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f50e400fa00 con 0x559522e24f40 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.092+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e27330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.093+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e27330 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a3d9b963c6d69fc server_cookie=f67324d12923a7fe in_seq=0 out_seq=0 2024-09-10T21:23:52.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.093+0000 7f50d57fa640 10 client.18981 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.095+0000 7f50d57fa640 1 -- 172.21.15.33:0/15666913 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f50e400fa00 con 0x559522e24f40 2024-09-10T21:23:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.095+0000 7f50d57fa640 10 client.18981 handle_command_reply: tid=0 2024-09-10T21:23:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.095+0000 7f50f60d5740 1 client.18981 shutdown 2024-09-10T21:23:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 msgr2=0x559522e129d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e129d0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f50e4014040 tx=0x7f50e4004cd0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 shutdown_connections 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e27330 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e129d0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50d57fa640 0 client.18981 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e00df0 0x559522e12350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 --2- 172.21.15.33:0/15666913 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e004d0 0x559522e14f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 >> 172.21.15.33:0/15666913 conn(0x559522dfe300 msgr2=0x559522e03e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 shutdown_connections 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 1 -- 172.21.15.33:0/15666913 wait complete. 2024-09-10T21:23:52.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.096+0000 7f50f60d5740 20 client.18981 trim_cache size 0 max 16384 2024-09-10T21:23:52.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.119+0000 7f50f60d5740 1 Processor -- start 2024-09-10T21:23:52.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.119+0000 7f50f60d5740 1 -- start start 2024-09-10T21:23:52.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.119+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.119+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522dff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.119+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e2cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f1a9c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f129b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522dff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f129b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522dff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35828/0 (socket says 172.21.15.33:35828) 2024-09-10T21:23:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f129b640 1 -- 172.21.15.33:0/2243516229 learned_addr learned my addr 172.21.15.33:0/2243516229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.119+0000 7f50f60d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1ac260 con 0x559522e00df0 2024-09-10T21:23:52.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f60d5740 1 -- 172.21.15.33:0/2243516229 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1adda0 con 0x559522e15590 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f60d5740 1 -- 172.21.15.33:0/2243516229 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1ac930 con 0x559522e004d0 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f229d640 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e2cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f229d640 1 -- 172.21.15.33:0/2243516229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 msgr2=0x559522e21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.120+0000 7f50f229d640 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e21470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.121+0000 7f50f229d640 1 -- 172.21.15.33:0/2243516229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 msgr2=0x559522dff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.121+0000 7f50f229d640 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522dff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.121+0000 7f50f229d640 1 -- 172.21.15.33:0/2243516229 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559522e12100 con 0x559522e15590 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.121+0000 7f50f129b640 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522dff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:52.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.121+0000 7f50f229d640 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e2cb10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009560 tx=0x7f50e40119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2055e7d94ea3c29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.121+0000 7f50d77fe640 1 -- 172.21.15.33:0/2243516229 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e4037030 con 0x559522e15590 2024-09-10T21:23:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50d77fe640 1 -- 172.21.15.33:0/2243516229 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e4011580 con 0x559522e15590 2024-09-10T21:23:52.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50d77fe640 1 -- 172.21.15.33:0/2243516229 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e401a070 con 0x559522e15590 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50f60d5740 1 -- 172.21.15.33:0/2243516229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 msgr2=0x559522e2cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50f60d5740 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e2cb10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f50e4009560 tx=0x7f50e40119a0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50f60d5740 1 -- 172.21.15.33:0/2243516229 shutdown_connections 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50f60d5740 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e15590 0x559522e2cb10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50f60d5740 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e00df0 0x559522dff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.122+0000 7f50f60d5740 1 --2- 172.21.15.33:0/2243516229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.123+0000 7f50f60d5740 1 -- 172.21.15.33:0/2243516229 >> 172.21.15.33:0/2243516229 conn(0x559522dfe300 msgr2=0x559522dff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:52.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.123+0000 7f50f60d5740 1 -- 172.21.15.33:0/2243516229 shutdown_connections 2024-09-10T21:23:52.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.123+0000 7f50f60d5740 1 -- 172.21.15.33:0/2243516229 wait complete. 2024-09-10T21:23:52.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.124+0000 7f50f60d5740 1 -- start start 2024-09-10T21:23:52.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.124+0000 7f50f60d5740 1 Processor -- start 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.125+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e1d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.125+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e00df0 0x559522e1dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.125+0000 7f50f60d5740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e1a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.125+0000 7f50f60d5740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1ad6d0 con 0x559522e15590 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f60d5740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1ad000 con 0x559522e00df0 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f60d5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50ec1aa220 con 0x559522e004d0 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f129b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e00df0 0x559522e1dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f129b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e00df0 0x559522e1dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55490/0 (socket says 172.21.15.33:55490) 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f129b640 1 -- 172.21.15.33:0/3333152974 learned_addr learned my addr 172.21.15.33:0/3333152974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e1d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f60d5740 10 client.0 _ll_register_callbacks cb 0x7f50e0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:52.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f229d640 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e1a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f1a9c640 1 -- 172.21.15.33:0/3333152974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e00df0 msgr2=0x559522e1dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e00df0 0x559522e1dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f1a9c640 1 -- 172.21.15.33:0/3333152974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 msgr2=0x559522e1a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.126+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e1a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.127+0000 7f50f1a9c640 1 -- 172.21.15.33:0/3333152974 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559522e1ab10 con 0x559522e004d0 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.127+0000 7f50f1a9c640 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e1d6b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50e00c9ac0 tx=0x7f50e004ace0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff23142233755151 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.127+0000 7f50d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.127+0000 7f50d57fa640 1 -- 172.21.15.33:0/3333152974 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50e0049040 con 0x559522e004d0 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.127+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50e4011580 con 0x559522e004d0 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.127+0000 7f50d57fa640 1 -- 172.21.15.33:0/3333152974 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f50e003bce0 con 0x559522e004d0 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.128+0000 7f50d57fa640 1 -- 172.21.15.33:0/3333152974 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f50e00d1050 con 0x559522e004d0 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.128+0000 7f50f60d5740 10 client.28228 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:52.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.128+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x559522e926f0 con 0x559522e004d0 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.128+0000 7f50d57fa640 1 -- 172.21.15.33:0/3333152974 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50e4011580 con 0x559522e004d0 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.129+0000 7f50d57fa640 1 -- 172.21.15.33:0/3333152974 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f50e004ad50 con 0x559522e004d0 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.129+0000 7f50f60d5740 10 client.28228 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.129+0000 7f50f60d5740 10 client.28228 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.129+0000 7f50f60d5740 20 client.28228 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.129+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.129+0000 7f50f60d5740 4 client.28228 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:52.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.129+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559522d78df0 con 0x559522e24f40 2024-09-10T21:23:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.130+0000 7f50f129b640 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.131+0000 7f50f129b640 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e94e10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41c6dbbf45ef8c47 server_cookie=22424b27539f54b6 in_seq=0 out_seq=0 2024-09-10T21:23:52.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.131+0000 7f50d57fa640 10 client.28228 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:52.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.132+0000 7f50d57fa640 1 -- 172.21.15.33:0/3333152974 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559522d78df0 con 0x559522e24f40 2024-09-10T21:23:52.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.132+0000 7f50d57fa640 10 client.28228 handle_command_reply: tid=0 2024-09-10T21:23:52.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.132+0000 7f50f60d5740 1 client.28228 shutdown 2024-09-10T21:23:52.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 msgr2=0x559522e1d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e1d6b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f50e00c9ac0 tx=0x7f50e004ace0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 shutdown_connections 2024-09-10T21:23:52.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x559522e24f40 0x559522e94e10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559522e15590 0x559522e1a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559522e00df0 0x559522e1dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 --2- 172.21.15.33:0/3333152974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559522e004d0 0x559522e1d6b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 >> 172.21.15.33:0/3333152974 conn(0x559522dfe300 msgr2=0x559522e13980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:52.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50d57fa640 0 client.28228 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:23:52.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 shutdown_connections 2024-09-10T21:23:52.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 1 -- 172.21.15.33:0/3333152974 wait complete. 2024-09-10T21:23:52.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.133+0000 7f50f60d5740 20 client.28228 trim_cache size 0 max 16384 2024-09-10T21:23:52.137 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.134+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50d80b5430 msgr2=0x7f50d80b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.134+0000 7f50f3d00640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50d80b5430 0x7f50d80b78f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f50ec10f7c0 tx=0x7f50e8005f90 comp rx=0 tx=0).stop 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.134+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 msgr2=0x7f50ec199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.134+0000 7f50f3d00640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 0x7f50ec199290 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f50e002a930 tx=0x7f50e0031a20 comp rx=0 tx=0).stop 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.134+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 shutdown_connections 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.134+0000 7f50f3d00640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f50d80b5430 0x7f50d80b78f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.135+0000 7f50f3d00640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50ec10d0e0 0x7f50ec19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.135+0000 7f50f3d00640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50ec104530 0x7f50ec1997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.135+0000 7f50f3d00640 1 --2- 172.21.15.33:0/877474342 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50ec103b30 0x7f50ec199290 secure :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f50e002a930 tx=0x7f50e0031a20 comp rx=0 tx=0).stop 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.135+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 >> 172.21.15.33:0/877474342 conn(0x7f50ec0fd700 msgr2=0x7f50ec0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.135+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 shutdown_connections 2024-09-10T21:23:52.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:52.135+0000 7f50f3d00640 1 -- 172.21.15.33:0/877474342 wait complete. 2024-09-10T21:23:52.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:51 smithi104 ceph-mon[35325]: pgmap v942: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:23:52.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:51 smithi174 ceph-mon[35635]: pgmap v942: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:23:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:51 smithi033 ceph-mon[30860]: pgmap v942: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:23:52.973 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:23:53.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 321725 40.28 MB/sec execute 293 sec latency 302.594 ms 2024-09-10T21:23:53.294 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 -- 172.21.15.33:0/2149059944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 msgr2=0x7f5d3c111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 --2- 172.21.15.33:0/2149059944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 0x7f5d3c111b70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009ca0 tx=0x7f5d3402a8f0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 -- 172.21.15.33:0/2149059944 shutdown_connections 2024-09-10T21:23:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 --2- 172.21.15.33:0/2149059944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 0x7f5d3c111b70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 --2- 172.21.15.33:0/2149059944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c10a9f0 0x7f5d3c10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 --2- 172.21.15.33:0/2149059944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d3c104fe0 0x7f5d3c1053c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 -- 172.21.15.33:0/2149059944 >> 172.21.15.33:0/2149059944 conn(0x7f5d3c1009f0 msgr2=0x7f5d3c102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:53.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.863+0000 7f5d42512640 1 -- 172.21.15.33:0/2149059944 shutdown_connections 2024-09-10T21:23:53.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.864+0000 7f5d42512640 1 -- 172.21.15.33:0/2149059944 wait complete. 2024-09-10T21:23:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.864+0000 7f5d42512640 1 Processor -- start 2024-09-10T21:23:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d42512640 1 -- start start 2024-09-10T21:23:53.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d42512640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d3c104fe0 0x7f5d3c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d42512640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c10a9f0 0x7f5d3c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d42512640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 0x7f5d3c1a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d42512640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c115070 con 0x7f5d3c10a9f0 2024-09-10T21:23:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d42512640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c114ef0 con 0x7f5d3c10b370 2024-09-10T21:23:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d42512640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1151f0 con 0x7f5d3c104fe0 2024-09-10T21:23:53.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d3bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d3c104fe0 0x7f5d3c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d3bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d3c104fe0 0x7f5d3c19b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46320/0 (socket says 172.21.15.33:46320) 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d3bfff640 1 -- 172.21.15.33:0/1769710256 learned_addr learned my addr 172.21.15.33:0/1769710256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.865+0000 7f5d40aaf640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 0x7f5d3c1a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d33fff640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c10a9f0 0x7f5d3c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d40aaf640 1 -- 172.21.15.33:0/1769710256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d3c104fe0 msgr2=0x7f5d3c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d40aaf640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d3c104fe0 0x7f5d3c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d40aaf640 1 -- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c10a9f0 msgr2=0x7f5d3c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d40aaf640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c10a9f0 0x7f5d3c19b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d40aaf640 1 -- 172.21.15.33:0/1769710256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a14d0 con 0x7f5d3c10b370 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d40aaf640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 0x7f5d3c1a0d00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d34006010 tx=0x7f5d3402f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.866+0000 7f5d39ffb640 1 -- 172.21.15.33:0/1769710256 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3402fbc0 con 0x7f5d3c10b370 2024-09-10T21:23:53.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.867+0000 7f5d39ffb640 1 -- 172.21.15.33:0/1769710256 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d3402d070 con 0x7f5d3c10b370 2024-09-10T21:23:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.867+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a1760 con 0x7f5d3c10b370 2024-09-10T21:23:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.867+0000 7f5d39ffb640 1 -- 172.21.15.33:0/1769710256 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34046770 con 0x7f5d3c10b370 2024-09-10T21:23:53.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.868+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a7f00 con 0x7f5d3c10b370 2024-09-10T21:23:53.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.871+0000 7f5d448e7740 1 Processor -- start 2024-09-10T21:23:53.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d448e7740 1 -- start start 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d24d0 0x5568396d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396e7590 0x5568396e9980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d448e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556839648ea0 con 0x5568396d24d0 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d448e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5568396382d0 con 0x5568396d2df0 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d448e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55683963c0e0 con 0x5568396e7590 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d3bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d3bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55496/0 (socket says 172.21.15.33:55496) 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d3bfff640 1 -- 172.21.15.33:0/2048036009 learned_addr learned my addr 172.21.15.33:0/2048036009 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.874+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d24d0 0x5568396d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.875+0000 7f5d33fff640 1 -- 172.21.15.33:0/2048036009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396e7590 msgr2=0x5568396e9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:53.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.875+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396e7590 0x5568396e9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.875+0000 7f5d33fff640 1 -- 172.21.15.33:0/2048036009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 msgr2=0x5568396e7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.875+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.875+0000 7f5d33fff640 1 -- 172.21.15.33:0/2048036009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556839396650 con 0x5568396d24d0 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.875+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d24d0 0x5568396d28b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400bb70 tx=0x7f5d2400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda3adf91359c656 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.876+0000 7f5d327fc640 1 -- 172.21.15.33:0/2048036009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24005220 con 0x5568396d24d0 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.876+0000 7f5d327fc640 1 -- 172.21.15.33:0/2048036009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d24004c50 con 0x5568396d24d0 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d327fc640 1 -- 172.21.15.33:0/2048036009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400f680 con 0x5568396d24d0 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d448e7740 1 -- 172.21.15.33:0/2048036009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d24d0 msgr2=0x5568396d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d24d0 0x5568396d28b0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400bb70 tx=0x7f5d2400d710 comp rx=0 tx=0).stop 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d448e7740 1 -- 172.21.15.33:0/2048036009 shutdown_connections 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396e7590 0x5568396e9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2048036009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d24d0 0x5568396d28b0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d39ffb640 1 -- 172.21.15.33:0/1769710256 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5d340305f0 con 0x7f5d3c10b370 2024-09-10T21:23:53.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.877+0000 7f5d448e7740 1 -- 172.21.15.33:0/2048036009 >> 172.21.15.33:0/2048036009 conn(0x5568396d0300 msgr2=0x5568396d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:53.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.878+0000 7f5d448e7740 1 -- 172.21.15.33:0/2048036009 shutdown_connections 2024-09-10T21:23:53.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.878+0000 7f5d448e7740 1 -- 172.21.15.33:0/2048036009 wait complete. 2024-09-10T21:23:53.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.878+0000 7f5d448e7740 1 -- start start 2024-09-10T21:23:53.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 1 Processor -- start 2024-09-10T21:23:53.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396de890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396dedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 0x5568396e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556839499700 con 0x5568396d2df0 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5568395d3160 con 0x5568396e7590 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5568396d8b90 con 0x5568396d24d0 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396dedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d3bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396dedd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35870/0 (socket says 172.21.15.33:35870) 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d3bfff640 1 -- 172.21.15.33:0/387962258 learned_addr learned my addr 172.21.15.33:0/387962258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.879+0000 7f5d448e7740 10 client.0 _ll_register_callbacks cb 0x5568395ec9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d33fff640 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396de890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d39ffb640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d280b52f0 0x7f5d280b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d3bfff640 1 -- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 msgr2=0x5568396de890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d39ffb640 1 -- 172.21.15.33:0/1769710256 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5d341071e0 con 0x7f5d3c10b370 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396de890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d3bfff640 1 -- 172.21.15.33:0/387962258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 msgr2=0x5568396e3f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 0x5568396e3f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.880+0000 7f5d3bfff640 1 -- 172.21.15.33:0/387962258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556839600c90 con 0x5568396d2df0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.881+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396dedd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c010500 tx=0x7f5d2c0109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256537151a7254ba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.881+0000 7f5d1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.881+0000 7f5d40aaf640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d280b52f0 0x7f5d280b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.882+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/387962258 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c018070 con 0x5568396d2df0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.882+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/387962258 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5d2c014640 con 0x5568396d2df0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.882+0000 7f5d40aaf640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d280b52f0 0x7f5d280b77b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5d3402a930 tx=0x7f5d34030dd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.882+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d24004c50 con 0x5568396d2df0 2024-09-10T21:23:53.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.883+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/387962258 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5d2c022050 con 0x5568396d2df0 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.883+0000 7f5d448e7740 10 client.18993 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.883+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5568396ed3f0 con 0x5568396d2df0 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.883+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/387962258 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d24004c50 con 0x5568396d2df0 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/387962258 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f5d2c01da10 con 0x5568396d2df0 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d448e7740 10 client.18993 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d448e7740 10 client.18993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d448e7740 20 client.18993 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d448e7740 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5568396eed20 0x5568396f1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d448e7740 4 client.18993 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55683964adf0 con 0x5568396eed20 2024-09-10T21:23:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.884+0000 7f5d33fff640 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5568396eed20 0x5568396f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.885+0000 7f5d33fff640 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5568396eed20 0x5568396f1140 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=570cae30a6cecd0e server_cookie=e1a6be280d852929 in_seq=0 out_seq=0 2024-09-10T21:23:53.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.885+0000 7f5d1b7fe640 10 client.18993 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.888+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/387962258 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55683964adf0 con 0x5568396eed20 2024-09-10T21:23:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.889+0000 7f5d1b7fe640 10 client.18993 handle_command_reply: tid=0 2024-09-10T21:23:53.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.889+0000 7f5d448e7740 1 client.18993 shutdown 2024-09-10T21:23:53.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.889+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 msgr2=0x5568396dedd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.889+0000 7f5d448e7740 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396dedd0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c010500 tx=0x7f5d2c0109c0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.891+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 shutdown_connections 2024-09-10T21:23:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.891+0000 7f5d448e7740 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 0x5568396e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.891+0000 7f5d448e7740 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x5568396eed20 0x5568396f1140 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.891+0000 7f5d448e7740 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396dedd0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.891+0000 7f5d448e7740 1 --2- 172.21.15.33:0/387962258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396de890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.891+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 >> 172.21.15.33:0/387962258 conn(0x5568396d0300 msgr2=0x5568396d5ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:53.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.892+0000 7f5d1b7fe640 0 client.18993 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.893+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 shutdown_connections 2024-09-10T21:23:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.893+0000 7f5d448e7740 1 -- 172.21.15.33:0/387962258 wait complete. 2024-09-10T21:23:53.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.894+0000 7f5d448e7740 20 client.18993 trim_cache size 0 max 16384 2024-09-10T21:23:53.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.924+0000 7f5d448e7740 1 Processor -- start 2024-09-10T21:23:53.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.925+0000 7f5d448e7740 1 -- start start 2024-09-10T21:23:53.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.925+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396f0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.925+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.925+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396e7590 0x5568396e4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.925+0000 7f5d33fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396f0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.926+0000 7f5d33fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396f0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46338/0 (socket says 172.21.15.33:46338) 2024-09-10T21:23:53.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.926+0000 7f5d33fff640 1 -- 172.21.15.33:0/954324487 learned_addr learned my addr 172.21.15.33:0/954324487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:53.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.926+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1ac1f0 con 0x5568396e7590 2024-09-10T21:23:53.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.926+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1add30 con 0x5568396d2df0 2024-09-10T21:23:53.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.926+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1ac8c0 con 0x5568396d24d0 2024-09-10T21:23:53.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.926+0000 7f5d33fff640 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396e7590 0x5568396e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.926+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.927+0000 7f5d3bfff640 1 -- 172.21.15.33:0/954324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 msgr2=0x5568396f0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:53.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.927+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396f0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.927+0000 7f5d3bfff640 1 -- 172.21.15.33:0/954324487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396e7590 msgr2=0x5568396e4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:53.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.927+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396e7590 0x5568396e4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.927+0000 7f5d3bfff640 1 -- 172.21.15.33:0/954324487 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568396dcbb0 con 0x5568396d2df0 2024-09-10T21:23:53.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.927+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e3dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5568396e4fe0 tx=0x7f5d2c006f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62afe9e30b8a7217 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:53.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.928+0000 7f5d31ffb640 1 -- 172.21.15.33:0/954324487 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c02b020 con 0x5568396d2df0 2024-09-10T21:23:53.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.928+0000 7f5d31ffb640 1 -- 172.21.15.33:0/954324487 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d2c0044f0 con 0x5568396d2df0 2024-09-10T21:23:53.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.928+0000 7f5d31ffb640 1 -- 172.21.15.33:0/954324487 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c018070 con 0x5568396d2df0 2024-09-10T21:23:53.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.929+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 msgr2=0x5568396e3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.929+0000 7f5d448e7740 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e3dc0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x5568396e4fe0 tx=0x7f5d2c006f30 comp rx=0 tx=0).stop 2024-09-10T21:23:53.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.930+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 shutdown_connections 2024-09-10T21:23:53.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.930+0000 7f5d448e7740 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396e7590 0x5568396e4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.930+0000 7f5d448e7740 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396d2df0 0x5568396e3dc0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.930+0000 7f5d448e7740 1 --2- 172.21.15.33:0/954324487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568396f0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.930+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 >> 172.21.15.33:0/954324487 conn(0x5568396d0300 msgr2=0x5568396d9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:53.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.931+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 shutdown_connections 2024-09-10T21:23:53.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.932+0000 7f5d448e7740 1 -- 172.21.15.33:0/954324487 wait complete. 2024-09-10T21:23:53.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.933+0000 7f5d448e7740 1 -- start start 2024-09-10T21:23:53.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.933+0000 7f5d448e7740 1 Processor -- start 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.933+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568397030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.933+0000 7f5d448e7740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396db560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d33fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568397030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d33fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568397030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46340/0 (socket says 172.21.15.33:46340) 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d33fff640 1 -- 172.21.15.33:0/2308618532 learned_addr learned my addr 172.21.15.33:0/2308618532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 0x5568396dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1ad660 con 0x5568396d2df0 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1acf90 con 0x5568396e7590 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1aa1d0 con 0x5568396d24d0 2024-09-10T21:23:53.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.934+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396db560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.935+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 0x5568396dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.935+0000 7f5d33fff640 1 -- 172.21.15.33:0/2308618532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 msgr2=0x5568396dbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:53.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.935+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 0x5568396dbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.935+0000 7f5d33fff640 1 -- 172.21.15.33:0/2308618532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 msgr2=0x5568396db560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.935+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396db560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.935+0000 7f5d33fff640 1 -- 172.21.15.33:0/2308618532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568396dc2d0 con 0x5568396d24d0 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.936+0000 7f5d448e7740 10 client.0 _ll_register_callbacks cb 0x7f5d2c007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.936+0000 7f5d33fff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568397030b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b9f0 tx=0x7f5d2404af90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13d7db99b1c59e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.936+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d2c0044f0 con 0x5568396d24d0 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.936+0000 7f5d1b7fe640 10 client.28240 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:23:53.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.938+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/2308618532 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2403e040 con 0x5568396d24d0 2024-09-10T21:23:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.938+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/2308618532 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f5d24039640 con 0x5568396d24d0 2024-09-10T21:23:53.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.938+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.946+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/2308618532 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f5d24039860 con 0x5568396d24d0 2024-09-10T21:23:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.946+0000 7f5d448e7740 10 client.28240 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:53.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.946+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=81}) -- 0x55683976cc00 con 0x5568396d24d0 2024-09-10T21:23:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.948+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/2308618532 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f5d24056070 con 0x5568396d24d0 2024-09-10T21:23:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.948+0000 7f5d448e7740 10 client.28240 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.948+0000 7f5d448e7740 10 client.28240 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:23:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.948+0000 7f5d448e7740 20 client.28240 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.948+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55683976d340 0x55683976f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.948+0000 7f5d448e7740 4 client.28240 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:53.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.948+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5d2c0044f0 con 0x55683976d340 2024-09-10T21:23:53.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.950+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55683976d340 0x55683976f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.951+0000 7f5d3bfff640 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55683976d340 0x55683976f760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e5e381b35f25d4af server_cookie=bb252249994a948c in_seq=0 out_seq=0 2024-09-10T21:23:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.951+0000 7f5d1b7fe640 10 client.28240 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:53.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.951+0000 7f5d1b7fe640 1 -- 172.21.15.33:0/2308618532 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5d2c0044f0 con 0x55683976d340 2024-09-10T21:23:53.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.952+0000 7f5d1b7fe640 10 client.28240 handle_command_reply: tid=0 2024-09-10T21:23:53.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.952+0000 7f5d448e7740 1 client.28240 shutdown 2024-09-10T21:23:53.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.954+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 msgr2=0x5568397030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.954+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568397030b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400b9f0 tx=0x7f5d2404af90 comp rx=0 tx=0).stop 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 shutdown_connections 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5568396e7590 0x5568396dbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55683976d340 0x55683976f760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5568396d2df0 0x5568396db560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 --2- 172.21.15.33:0/2308618532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568396d24d0 0x5568397030b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 >> 172.21.15.33:0/2308618532 conn(0x5568396d0300 msgr2=0x5568396dce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d1b7fe640 0 client.28240 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 shutdown_connections 2024-09-10T21:23:53.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.955+0000 7f5d448e7740 1 -- 172.21.15.33:0/2308618532 wait complete. 2024-09-10T21:23:53.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.956+0000 7f5d448e7740 20 client.28240 trim_cache size 0 max 16384 2024-09-10T21:23:53.960 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d280b52f0 msgr2=0x7f5d280b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d280b52f0 0x7f5d280b77b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5d3402a930 tx=0x7f5d34030dd0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 msgr2=0x7f5d3c1a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 0x7f5d3c1a0d00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d34006010 tx=0x7f5d3402f640 comp rx=0 tx=0).stop 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 shutdown_connections 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5d280b52f0 0x7f5d280b77b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d3c10b370 0x7f5d3c1a0d00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d3c10a9f0 0x7f5d3c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 --2- 172.21.15.33:0/1769710256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d3c104fe0 0x7f5d3c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.959+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 >> 172.21.15.33:0/1769710256 conn(0x7f5d3c1009f0 msgr2=0x7f5d3c102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.960+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 shutdown_connections 2024-09-10T21:23:53.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:53.960+0000 7f5d42512640 1 -- 172.21.15.33:0/1769710256 wait complete. 2024-09-10T21:23:54.019 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:53 smithi033 ceph-mon[30860]: pgmap v943: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 681 op/s 2024-09-10T21:23:54.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 322356 40.24 MB/sec execute 294 sec latency 159.614 ms 2024-09-10T21:23:54.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:53 smithi104 ceph-mon[35325]: pgmap v943: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 681 op/s 2024-09-10T21:23:54.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:53 smithi174 ceph-mon[35635]: pgmap v943: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 8.7 MiB/s wr, 681 op/s 2024-09-10T21:23:55.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:23:55.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 323124 40.23 MB/sec execute 295 sec latency 311.823 ms 2024-09-10T21:23:55.338 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:23:55.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.899+0000 7f9a039dc640 1 -- 172.21.15.33:0/3084409259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc109150 msgr2=0x7f99fc10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.899+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3084409259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc109150 0x7f99fc10f950 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009ca0 tx=0x7f99f402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 -- 172.21.15.33:0/3084409259 shutdown_connections 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3084409259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc109150 0x7f99fc10f950 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3084409259 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99fc1087d0 0x7f99fc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3084409259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99fc102dc0 0x7f99fc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 -- 172.21.15.33:0/3084409259 >> 172.21.15.33:0/3084409259 conn(0x7f99fc0fe870 msgr2=0x7f99fc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 -- 172.21.15.33:0/3084409259 shutdown_connections 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 -- 172.21.15.33:0/3084409259 wait complete. 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.900+0000 7f9a039dc640 1 Processor -- start 2024-09-10T21:23:55.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a039dc640 1 -- start start 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a039dc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99fc102dc0 0x7f99fc1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a039dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc1087d0 0x7f99fc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a039dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99fc109150 0x7f99fc19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a039dc640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc112e10 con 0x7f99fc1087d0 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a039dc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc112c90 con 0x7f99fc102dc0 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a039dc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc112f90 con 0x7f99fc109150 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a00f77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc1087d0 0x7f99fc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a01778640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99fc102dc0 0x7f99fc1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a00f77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc1087d0 0x7f99fc199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35896/0 (socket says 172.21.15.33:35896) 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a00f77640 1 -- 172.21.15.33:0/3423000591 learned_addr learned my addr 172.21.15.33:0/3423000591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a01f79640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99fc109150 0x7f99fc19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a00f77640 1 -- 172.21.15.33:0/3423000591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99fc109150 msgr2=0x7f99fc19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.901+0000 7f9a00f77640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99fc109150 0x7f99fc19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f9a00f77640 1 -- 172.21.15.33:0/3423000591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99fc102dc0 msgr2=0x7f99fc1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f9a00f77640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99fc102dc0 0x7f99fc1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f9a00f77640 1 -- 172.21.15.33:0/3423000591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc19f290 con 0x7f99fc1087d0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f9a00f77640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc1087d0 0x7f99fc199720 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00ea60 tx=0x7f99ec00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f99f27fc640 1 -- 172.21.15.33:0/3423000591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec00cde0 con 0x7f99fc1087d0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc19f580 con 0x7f99fc1087d0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f99f27fc640 1 -- 172.21.15.33:0/3423000591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec0035f0 con 0x7f99fc1087d0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f99f27fc640 1 -- 172.21.15.33:0/3423000591 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec010690 con 0x7f99fc1087d0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.902+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc1a5c40 con 0x7f99fc1087d0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.905+0000 7f9a05db1740 1 Processor -- start 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.905+0000 7f99f27fc640 1 -- 172.21.15.33:0/3423000591 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f99ec004000 con 0x7f99fc1087d0 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.905+0000 7f9a05db1740 1 -- start start 2024-09-10T21:23:55.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x5558425068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555842510830 0x555842506df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55584251b430 0x55584251d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a05db1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55584247cea0 con 0x5558425064d0 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a05db1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55584246c2d0 con 0x55584251b430 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a05db1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5558424700e0 con 0x555842510830 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a01778640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555842510830 0x555842506df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a01778640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555842510830 0x555842506df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46362/0 (socket says 172.21.15.33:46362) 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a01778640 1 -- 172.21.15.33:0/3368759320 learned_addr learned my addr 172.21.15.33:0/3368759320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a00f77640 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55584251b430 0x55584251d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.906+0000 7f9a01f79640 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x5558425068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f99f27fc640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99e80b52e0 0x7f99e80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f9a01f79640 1 -- 172.21.15.33:0/3368759320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555842510830 msgr2=0x555842506df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f9a01f79640 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555842510830 0x555842506df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f9a01f79640 1 -- 172.21.15.33:0/3368759320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55584251b430 msgr2=0x55584251d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f9a01f79640 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55584251b430 0x55584251d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f9a01f79640 1 -- 172.21.15.33:0/3368759320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558421ca650 con 0x5558425064d0 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f99f27fc640 1 -- 172.21.15.33:0/3423000591 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f99ec0e5eb0 con 0x7f99fc1087d0 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f9a01f79640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99e80b52e0 0x7f99e80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f9a01f79640 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x5558425068b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f99f4005e30 tx=0x7f99f4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2478d8aae1ce897e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f99e77fe640 1 -- 172.21.15.33:0/3368759320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4031d30 con 0x5558425064d0 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f99e77fe640 1 -- 172.21.15.33:0/3368759320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f4036030 con 0x5558425064d0 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.907+0000 7f99e77fe640 1 -- 172.21.15.33:0/3368759320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4004400 con 0x5558425064d0 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 -- 172.21.15.33:0/3368759320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 msgr2=0x5558425068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x5558425068b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f99f4005e30 tx=0x7f99f4005ef0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a01f79640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99e80b52e0 0x7f99e80b77a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f99f4006010 tx=0x7f99f4004740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 -- 172.21.15.33:0/3368759320 shutdown_connections 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55584251b430 0x55584251d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555842510830 0x555842506df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 --2- 172.21.15.33:0/3368759320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x5558425068b0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 -- 172.21.15.33:0/3368759320 >> 172.21.15.33:0/3368759320 conn(0x555842504300 msgr2=0x5558425054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 -- 172.21.15.33:0/3368759320 shutdown_connections 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.908+0000 7f9a05db1740 1 -- 172.21.15.33:0/3368759320 wait complete. 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.909+0000 7f9a05db1740 1 -- start start 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.909+0000 7f9a05db1740 1 Processor -- start 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842510070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x55584250d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584250da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a05db1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5558422cd700 con 0x5558425064d0 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a05db1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x555842407160 con 0x555842510830 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a05db1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55584251e0a0 con 0x55584251b430 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a05db1740 10 client.0 _ll_register_callbacks cb 0x5558424209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a01f79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584250da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a01f79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584250da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46372/0 (socket says 172.21.15.33:46372) 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.910+0000 7f9a01f79640 1 -- 172.21.15.33:0/1878855781 learned_addr learned my addr 172.21.15.33:0/1878855781 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.911+0000 7f9a01778640 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842510070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.911+0000 7f9a01f79640 1 -- 172.21.15.33:0/1878855781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 msgr2=0x55584250d4c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.911+0000 7f9a01f79640 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x55584250d4c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.911+0000 7f9a01f79640 1 -- 172.21.15.33:0/1878855781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 msgr2=0x555842510070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:55.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.911+0000 7f9a01f79640 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842510070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.911+0000 7f9a01f79640 1 -- 172.21.15.33:0/1878855781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555842434c90 con 0x55584251b430 2024-09-10T21:23:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.911+0000 7f9a01778640 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842510070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:23:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.912+0000 7f9a01f79640 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584250da20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99f4004d70 tx=0x7f99f4041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ecb4620cba9108 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.912+0000 7f99e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:23:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.912+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1878855781 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4060060 con 0x55584251b430 2024-09-10T21:23:55.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.912+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99f4036030 con 0x55584251b430 2024-09-10T21:23:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.912+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1878855781 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f99f4058070 con 0x55584251b430 2024-09-10T21:23:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.912+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1878855781 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f99f402fb60 con 0x55584251b430 2024-09-10T21:23:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.912+0000 7f9a05db1740 10 client.28252 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.913+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x555842529930 con 0x55584251b430 2024-09-10T21:23:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.913+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1878855781 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99f402fd00 con 0x55584251b430 2024-09-10T21:23:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.913+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1878855781 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f99f4003bd0 con 0x55584251b430 2024-09-10T21:23:55.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.913+0000 7f9a05db1740 10 client.28252 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.913+0000 7f9a05db1740 10 client.28252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.913+0000 7f9a05db1740 20 client.28252 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.914+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55584252ac80 0x55584252d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.914+0000 7f9a05db1740 4 client.28252 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.914+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f99f4036030 con 0x55584252ac80 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.915+0000 7f9a01778640 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55584252ac80 0x55584252d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.915+0000 7f99e4ff9640 10 client.28252 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.915+0000 7f9a01778640 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55584252ac80 0x55584252d070 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3da16af36e3ecd server_cookie=cd55df2f4c0b3558 in_seq=0 out_seq=0 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.916+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1878855781 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f99f4036030 con 0x55584252ac80 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.916+0000 7f99e4ff9640 10 client.28252 handle_command_reply: tid=0 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.916+0000 7f9a05db1740 1 client.28252 shutdown 2024-09-10T21:23:55.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 msgr2=0x55584250da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584250da20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f99f4004d70 tx=0x7f99f4041000 comp rx=0 tx=0).stop 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 shutdown_connections 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55584252ac80 0x55584252d070 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584250da20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f99e4ff9640 0 client.28252 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x55584250d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1878855781 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842510070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 >> 172.21.15.33:0/1878855781 conn(0x555842504300 msgr2=0x555842502c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 shutdown_connections 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 1 -- 172.21.15.33:0/1878855781 wait complete. 2024-09-10T21:23:55.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.917+0000 7f9a05db1740 20 client.28252 trim_cache size 0 max 16384 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.935+0000 7f9a05db1740 1 Processor -- start 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.935+0000 7f9a05db1740 1 -- start start 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842505350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x5558425068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x555842506df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a05db1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1ac1b0 con 0x5558425064d0 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a05db1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1adcf0 con 0x555842510830 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a05db1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1ac880 con 0x55584251b430 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a00f77640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x5558425068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a00f77640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x5558425068b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55572/0 (socket says 172.21.15.33:55572) 2024-09-10T21:23:55.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a00f77640 1 -- 172.21.15.33:0/439058550 learned_addr learned my addr 172.21.15.33:0/439058550 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a01778640 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842505350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a01f79640 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x555842506df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a00f77640 1 -- 172.21.15.33:0/439058550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 msgr2=0x555842506df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a00f77640 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x555842506df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a00f77640 1 -- 172.21.15.33:0/439058550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 msgr2=0x555842505350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.936+0000 7f9a00f77640 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842505350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.937+0000 7f9a00f77640 1 -- 172.21.15.33:0/439058550 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555842507330 con 0x555842510830 2024-09-10T21:23:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.938+0000 7f9a00f77640 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x5558425068b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0ac600 tx=0x7f99ec0096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=486ba74d05bc1424 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.938+0000 7f99e6ffd640 1 -- 172.21.15.33:0/439058550 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0a8cc0 con 0x555842510830 2024-09-10T21:23:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.938+0000 7f99e6ffd640 1 -- 172.21.15.33:0/439058550 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec0ac180 con 0x555842510830 2024-09-10T21:23:55.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.938+0000 7f99e6ffd640 1 -- 172.21.15.33:0/439058550 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0abde0 con 0x555842510830 2024-09-10T21:23:55.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 -- 172.21.15.33:0/439058550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 msgr2=0x5558425068b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x5558425068b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f99ec0ac600 tx=0x7f99ec0096c0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 -- 172.21.15.33:0/439058550 shutdown_connections 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x555842506df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x555842510830 0x5558425068b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 --2- 172.21.15.33:0/439058550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5558425064d0 0x555842505350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 -- 172.21.15.33:0/439058550 >> 172.21.15.33:0/439058550 conn(0x555842504300 msgr2=0x55584250bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 -- 172.21.15.33:0/439058550 shutdown_connections 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.939+0000 7f9a05db1740 1 -- 172.21.15.33:0/439058550 wait complete. 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.940+0000 7f9a05db1740 1 -- start start 2024-09-10T21:23:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.940+0000 7f9a05db1740 1 Processor -- start 2024-09-10T21:23:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.940+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5558425064d0 0x555842523380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.940+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555842510830 0x5558425238c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.940+0000 7f9a05db1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584251f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a05db1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1ad620 con 0x555842510830 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a05db1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1acf50 con 0x5558425064d0 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a05db1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1aa170 con 0x55584251b430 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a05db1740 10 client.0 _ll_register_callbacks cb 0x7f99f4049020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555842510830 0x5558425238c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555842510830 0x5558425238c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35934/0 (socket says 172.21.15.33:35934) 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 -- 172.21.15.33:0/1816764417 learned_addr learned my addr 172.21.15.33:0/1816764417 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a01f79640 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584251f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 -- 172.21.15.33:0/1816764417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 msgr2=0x55584251f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584251f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 -- 172.21.15.33:0/1816764417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5558425064d0 msgr2=0x555842523380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5558425064d0 0x555842523380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a00f77640 1 -- 172.21.15.33:0/1816764417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555842523f00 con 0x555842510830 2024-09-10T21:23:55.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.941+0000 7f9a01f79640 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584251f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:23:55.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.942+0000 7f9a00f77640 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555842510830 0x5558425238c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c760 tx=0x7f99ec005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0496920e606b67f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:23:55.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.942+0000 7f99e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:23:55.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.942+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1816764417 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0abfc0 con 0x555842510830 2024-09-10T21:23:55.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.942+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1816764417 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f99ec0a88e0 con 0x555842510830 2024-09-10T21:23:55.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.942+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99ec0ac180 con 0x555842510830 2024-09-10T21:23:55.948 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1816764417 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f99ec0ac440 con 0x555842510830 2024-09-10T21:23:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f9a05db1740 10 client.19011 fetch_fsmap learned FSMap version 81 2024-09-10T21:23:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x555842598ee0 con 0x555842510830 2024-09-10T21:23:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1816764417 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99ec0ac180 con 0x555842510830 2024-09-10T21:23:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1816764417 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f99ec0a7fc0 con 0x555842510830 2024-09-10T21:23:55.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f9a05db1740 10 client.19011 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:23:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f9a05db1740 10 client.19011 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:23:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.946+0000 7f9a05db1740 20 client.19011 populate_metadata read hostname 'smithi033' 2024-09-10T21:23:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.947+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5558425995f0 0x55584259b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:23:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.947+0000 7f9a05db1740 4 client.19011 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:23:55.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.947+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55584247edf0 con 0x5558425995f0 2024-09-10T21:23:55.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.948+0000 7f9a01778640 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5558425995f0 0x55584259b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:23:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f9a01778640 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5558425995f0 0x55584259b9e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb26486d30482f7 server_cookie=9bbffe6044bbb2 in_seq=0 out_seq=0 2024-09-10T21:23:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f99e4ff9640 10 client.19011 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f99e4ff9640 1 -- 172.21.15.33:0/1816764417 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55584247edf0 con 0x5558425995f0 2024-09-10T21:23:55.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f99e4ff9640 10 client.19011 handle_command_reply: tid=0 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f9a05db1740 1 client.19011 shutdown 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555842510830 msgr2=0x5558425238c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555842510830 0x5558425238c0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c760 tx=0x7f99ec005b20 comp rx=0 tx=0).stop 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 shutdown_connections 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.950+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5558425995f0 0x55584259b9e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55584251b430 0x55584251f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x555842510830 0x5558425238c0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f99e4ff9640 0 client.19011 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f9a05db1740 1 --2- 172.21.15.33:0/1816764417 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5558425064d0 0x555842523380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 >> 172.21.15.33:0/1816764417 conn(0x555842504300 msgr2=0x555842502c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 shutdown_connections 2024-09-10T21:23:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f9a05db1740 1 -- 172.21.15.33:0/1816764417 wait complete. 2024-09-10T21:23:55.956 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:23:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.951+0000 7f9a05db1740 20 client.19011 trim_cache size 0 max 16384 2024-09-10T21:23:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99e80b52e0 msgr2=0x7f99e80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99e80b52e0 0x7f99e80b77a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f99f4006010 tx=0x7f99f4004740 comp rx=0 tx=0).stop 2024-09-10T21:23:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc1087d0 msgr2=0x7f99fc199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:23:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc1087d0 0x7f99fc199720 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00ea60 tx=0x7f99ec00ef20 comp rx=0 tx=0).stop 2024-09-10T21:23:55.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 shutdown_connections 2024-09-10T21:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f99e80b52e0 0x7f99e80b77a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99fc109150 0x7f99fc19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f99fc1087d0 0x7f99fc199720 secure :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00ea60 tx=0x7f99ec00ef20 comp rx=0 tx=0).stop 2024-09-10T21:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.952+0000 7f9a039dc640 1 --2- 172.21.15.33:0/3423000591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99fc102dc0 0x7f99fc1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.953+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 >> 172.21.15.33:0/3423000591 conn(0x7f99fc0fe870 msgr2=0x7f99fc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.953+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 shutdown_connections 2024-09-10T21:23:55.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:23:55.953+0000 7f9a039dc640 1 -- 172.21.15.33:0/3423000591 wait complete. 2024-09-10T21:23:56.067 INFO:tasks.workunit.client.0.smithi033.stdout: 10 323799 40.24 MB/sec execute 296 sec latency 64.905 ms 2024-09-10T21:23:56.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:55 smithi033 ceph-mon[30860]: pgmap v944: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 847 op/s 2024-09-10T21:23:56.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:55 smithi104 ceph-mon[35325]: pgmap v944: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 847 op/s 2024-09-10T21:23:56.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:55 smithi174 ceph-mon[35635]: pgmap v944: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 847 op/s 2024-09-10T21:23:57.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 324524 40.23 MB/sec execute 297 sec latency 171.841 ms 2024-09-10T21:23:58.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 325098 40.21 MB/sec execute 298 sec latency 289.035 ms 2024-09-10T21:23:58.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:57 smithi104 ceph-mon[35325]: pgmap v945: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 714 op/s 2024-09-10T21:23:58.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:57 smithi174 ceph-mon[35635]: pgmap v945: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 714 op/s 2024-09-10T21:23:58.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:57 smithi033 ceph-mon[30860]: pgmap v945: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.7 MiB/s wr, 714 op/s 2024-09-10T21:23:59.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 325751 40.19 MB/sec execute 299 sec latency 77.542 ms 2024-09-10T21:23:59.740 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:23:59.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:24:00.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 326440 40.18 MB/sec execute 300 sec latency 88.312 ms 2024-09-10T21:24:00.081 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:00.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:23:59 smithi033 ceph-mon[30860]: pgmap v946: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 790 op/s 2024-09-10T21:24:00.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:23:59 smithi104 ceph-mon[35325]: pgmap v946: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 790 op/s 2024-09-10T21:24:00.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:23:59 smithi174 ceph-mon[35635]: pgmap v946: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 790 op/s 2024-09-10T21:24:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 -- 172.21.15.33:0/2783493381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c104550 msgr2=0x7f641c10cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 --2- 172.21.15.33:0/2783493381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c104550 0x7f641c10cb30 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f640c009900 tx=0x7f640c02a410 comp rx=0 tx=0).stop 2024-09-10T21:24:00.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 -- 172.21.15.33:0/2783493381 shutdown_connections 2024-09-10T21:24:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 --2- 172.21.15.33:0/2783493381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 0x7f641c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 --2- 172.21.15.33:0/2783493381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c104550 0x7f641c10cb30 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 --2- 172.21.15.33:0/2783493381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f641c103b50 0x7f641c103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 -- 172.21.15.33:0/2783493381 >> 172.21.15.33:0/2783493381 conn(0x7f641c0fd700 msgr2=0x7f641c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 -- 172.21.15.33:0/2783493381 shutdown_connections 2024-09-10T21:24:00.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.617+0000 7f64247b5640 1 -- 172.21.15.33:0/2783493381 wait complete. 2024-09-10T21:24:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.618+0000 7f64247b5640 1 Processor -- start 2024-09-10T21:24:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.618+0000 7f64247b5640 1 -- start start 2024-09-10T21:24:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f64247b5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c103b50 0x7f641c19b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f64247b5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f641c104550 0x7f641c19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f64247b5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 0x7f641c1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f64247b5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f641c112da0 con 0x7f641c103b50 2024-09-10T21:24:00.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f64247b5640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f641c112c20 con 0x7f641c10d100 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f64247b5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f641c112f20 con 0x7f641c104550 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 0x7f641c1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 0x7f641c1a0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56802/0 (socket says 172.21.15.33:56802) 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 -- 172.21.15.33:0/2749148471 learned_addr learned my addr 172.21.15.33:0/2749148471 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422551640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c103b50 0x7f641c19b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 -- 172.21.15.33:0/2749148471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f641c104550 msgr2=0x7f641c19ba50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f641c104550 0x7f641c19ba50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 -- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c103b50 msgr2=0x7f641c19b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c103b50 0x7f641c19b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.619+0000 7f6422d52640 1 -- 172.21.15.33:0/2749148471 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f641c1a15c0 con 0x7f641c10d100 2024-09-10T21:24:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.620+0000 7f6422d52640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 0x7f641c1a0df0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f641400b970 tx=0x7f641400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.620+0000 7f64137fe640 1 -- 172.21.15.33:0/2749148471 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6414019070 con 0x7f641c10d100 2024-09-10T21:24:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.620+0000 7f64137fe640 1 -- 172.21.15.33:0/2749148471 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6414014440 con 0x7f641c10d100 2024-09-10T21:24:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.620+0000 7f64137fe640 1 -- 172.21.15.33:0/2749148471 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140075b0 con 0x7f641c10d100 2024-09-10T21:24:00.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.620+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f641c1a18b0 con 0x7f641c10d100 2024-09-10T21:24:00.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.621+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f641c1a7f70 con 0x7f641c10d100 2024-09-10T21:24:00.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.624+0000 7f64137fe640 1 -- 172.21.15.33:0/2749148471 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f641400cb40 con 0x7f641c10d100 2024-09-10T21:24:00.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.625+0000 7f64137fe640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64080b5360 0x7f64080b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.625+0000 7f6422551640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64080b5360 0x7f64080b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.625+0000 7f64137fe640 1 -- 172.21.15.33:0/2749148471 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f64140e54b0 con 0x7f641c10d100 2024-09-10T21:24:00.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.626+0000 7f6422551640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64080b5360 0x7f64080b7820 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6418009b60 tx=0x7f6418005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:00.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.627+0000 7f6426b8a740 1 Processor -- start 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.627+0000 7f6426b8a740 1 -- start start 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.627+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.627+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e7d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.627+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e7fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6426b8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d213de7e60 con 0x55d213e7d790 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6426b8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d213dd7290 con 0x55d213e71800 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e7fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6426b8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d213ddb0a0 con 0x55d213e70ee0 2024-09-10T21:24:00.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e7fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38416/0 (socket says 172.21.15.33:38416) 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 -- 172.21.15.33:0/3513588176 learned_addr learned my addr 172.21.15.33:0/3513588176 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6421d50640 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e7d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422551640 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 -- 172.21.15.33:0/3513588176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 msgr2=0x55d213e712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 -- 172.21.15.33:0/3513588176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 msgr2=0x55d213e7d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e7d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 -- 172.21.15.33:0/3513588176 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d213b35930 con 0x55d213e7d790 2024-09-10T21:24:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.628+0000 7f6422d52640 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e7fb80 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f64140e2ef0 tx=0x7f6414014b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76908cdcbddf90bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.629+0000 7f6410ff9640 1 -- 172.21.15.33:0/3513588176 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140ac370 con 0x55d213e7d790 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.629+0000 7f6410ff9640 1 -- 172.21.15.33:0/3513588176 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64140abc70 con 0x55d213e7d790 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.629+0000 7f6410ff9640 1 -- 172.21.15.33:0/3513588176 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140b4d70 con 0x55d213e7d790 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 -- 172.21.15.33:0/3513588176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 msgr2=0x55d213e7fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e7fb80 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f64140e2ef0 tx=0x7f6414014b60 comp rx=0 tx=0).stop 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 -- 172.21.15.33:0/3513588176 shutdown_connections 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e7fb80 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e7d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3513588176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 -- 172.21.15.33:0/3513588176 >> 172.21.15.33:0/3513588176 conn(0x55d213e6ed10 msgr2=0x55d213e6feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 -- 172.21.15.33:0/3513588176 shutdown_connections 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 -- 172.21.15.33:0/3513588176 wait complete. 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 -- start start 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.630+0000 7f6426b8a740 1 Processor -- start 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e70ee0 0x55d213e7a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e71800 0x55d213e7ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e7d790 0x55d213e7b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6426b8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d213c38890 con 0x55d213e70ee0 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6426b8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d213d71d40 con 0x55d213e7d790 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6426b8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d213e803e0 con 0x55d213e71800 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6426b8a740 10 client.0 _ll_register_callbacks cb 0x55d213dd5c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:00.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6421d50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e71800 0x55d213e7ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6421d50640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e71800 0x55d213e7ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:41616/0 (socket says 172.21.15.33:41616) 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6421d50640 1 -- 172.21.15.33:0/1069236111 learned_addr learned my addr 172.21.15.33:0/1069236111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.631+0000 7f6422551640 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e70ee0 0x55d213e7a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6422d52640 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e7d790 0x55d213e7b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6422551640 1 -- 172.21.15.33:0/1069236111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e71800 msgr2=0x55d213e7ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6422551640 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e71800 0x55d213e7ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6422551640 1 -- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e7d790 msgr2=0x55d213e7b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6422551640 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e7d790 0x55d213e7b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6422551640 1 -- 172.21.15.33:0/1069236111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d213e898f0 con 0x55d213e70ee0 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6422551640 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e70ee0 0x55d213e7a830 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6418011870 tx=0x7f6418004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faedbe0030f807d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:00.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.632+0000 7f6405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:24:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.633+0000 7f6405ffb640 1 -- 172.21.15.33:0/1069236111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6418020dc0 con 0x55d213e70ee0 2024-09-10T21:24:00.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.633+0000 7f6405ffb640 1 -- 172.21.15.33:0/1069236111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f64180283e0 con 0x55d213e70ee0 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.633+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64140abc70 con 0x55d213e70ee0 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.633+0000 7f6405ffb640 1 -- 172.21.15.33:0/1069236111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f64180285e0 con 0x55d213e70ee0 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.633+0000 7f6426b8a740 10 client.19023 fetch_fsmap learned FSMap version 81 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.633+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d213e8c300 con 0x55d213e70ee0 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6405ffb640 1 -- 172.21.15.33:0/1069236111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64140abc70 con 0x55d213e70ee0 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6405ffb640 1 -- 172.21.15.33:0/1069236111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f6418027070 con 0x55d213e70ee0 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6426b8a740 10 client.19023 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6426b8a740 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6426b8a740 20 client.19023 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6426b8a740 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213e8d7b0 0x55d213e8fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6426b8a740 4 client.19023 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d213de9db0 con 0x55d213e8d7b0 2024-09-10T21:24:00.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.634+0000 7f6421d50640 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213e8d7b0 0x55d213e8fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.635+0000 7f6405ffb640 10 client.19023 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:00.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.635+0000 7f6421d50640 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213e8d7b0 0x55d213e8fba0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce03a219693a6e2 server_cookie=1867ec373beb093b in_seq=0 out_seq=0 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.637+0000 7f6405ffb640 1 -- 172.21.15.33:0/1069236111 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d213de9db0 con 0x55d213e8d7b0 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.637+0000 7f6405ffb640 10 client.19023 handle_command_reply: tid=0 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.637+0000 7f6426b8a740 1 client.19023 shutdown 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.637+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e70ee0 msgr2=0x55d213e7a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.637+0000 7f6426b8a740 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e70ee0 0x55d213e7a830 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f6418011870 tx=0x7f6418004730 comp rx=0 tx=0).stop 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 shutdown_connections 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213e8d7b0 0x55d213e8fba0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6405ffb640 0 client.19023 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e7d790 0x55d213e7b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e71800 0x55d213e7ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 --2- 172.21.15.33:0/1069236111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e70ee0 0x55d213e7a830 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 >> 172.21.15.33:0/1069236111 conn(0x55d213e6ed10 msgr2=0x55d213e6df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 shutdown_connections 2024-09-10T21:24:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 1 -- 172.21.15.33:0/1069236111 wait complete. 2024-09-10T21:24:00.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.638+0000 7f6426b8a740 20 client.19023 trim_cache size 0 max 16384 2024-09-10T21:24:00.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 Processor -- start 2024-09-10T21:24:00.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 -- start start 2024-09-10T21:24:00.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e77a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e77f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1b0890 con 0x55d213e7d790 2024-09-10T21:24:00.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1b23d0 con 0x55d213e71800 2024-09-10T21:24:00.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6426b8a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1b0f60 con 0x55d213e70ee0 2024-09-10T21:24:00.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6422d52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e77f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6422d52640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e77f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38432/0 (socket says 172.21.15.33:38432) 2024-09-10T21:24:00.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6422d52640 1 -- 172.21.15.33:0/3307254097 learned_addr learned my addr 172.21.15.33:0/3307254097 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6422551640 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.664+0000 7f6421d50640 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e77a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6422d52640 1 -- 172.21.15.33:0/3307254097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 msgr2=0x55d213e74650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6422d52640 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e74650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6422d52640 1 -- 172.21.15.33:0/3307254097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 msgr2=0x55d213e77a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6422d52640 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e77a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6422d52640 1 -- 172.21.15.33:0/3307254097 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d213e7fb80 con 0x55d213e7d790 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6422551640 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e74650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6422d52640 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e77f80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f641400b940 tx=0x7f6414093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921759d6adbb1ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:00.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6407fff640 1 -- 172.21.15.33:0/3307254097 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140b0750 con 0x55d213e7d790 2024-09-10T21:24:00.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6407fff640 1 -- 172.21.15.33:0/3307254097 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64140ac370 con 0x55d213e7d790 2024-09-10T21:24:00.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.665+0000 7f6407fff640 1 -- 172.21.15.33:0/3307254097 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140a7ee0 con 0x55d213e7d790 2024-09-10T21:24:00.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.666+0000 7f6426b8a740 1 -- 172.21.15.33:0/3307254097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 msgr2=0x55d213e77f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.666+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e77f80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f641400b940 tx=0x7f6414093040 comp rx=0 tx=0).stop 2024-09-10T21:24:00.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.666+0000 7f6426b8a740 1 -- 172.21.15.33:0/3307254097 shutdown_connections 2024-09-10T21:24:00.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.666+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e7d790 0x55d213e77f80 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.666+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e71800 0x55d213e77a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.666+0000 7f6426b8a740 1 --2- 172.21.15.33:0/3307254097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e70ee0 0x55d213e74650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.666+0000 7f6426b8a740 1 -- 172.21.15.33:0/3307254097 >> 172.21.15.33:0/3307254097 conn(0x55d213e6ed10 msgr2=0x55d213e73f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.667+0000 7f6426b8a740 1 -- 172.21.15.33:0/3307254097 shutdown_connections 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.667+0000 7f6426b8a740 1 -- 172.21.15.33:0/3307254097 wait complete. 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.667+0000 7f6426b8a740 1 -- start start 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.667+0000 7f6426b8a740 1 Processor -- start 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e70ee0 0x55d213e821d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e71800 0x55d213e82770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6426b8a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e7d790 0x55d213ef3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6426b8a740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1b1d00 con 0x55d213e71800 2024-09-10T21:24:00.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6422551640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e70ee0 0x55d213e821d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e71800 0x55d213e82770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e71800 0x55d213e82770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38444/0 (socket says 172.21.15.33:38444) 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 -- 172.21.15.33:0/548234156 learned_addr learned my addr 172.21.15.33:0/548234156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1b1630 con 0x55d213e70ee0 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f641c1ae850 con 0x55d213e7d790 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6426b8a740 10 client.0 _ll_register_callbacks cb 0x7f6418011c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6422d52640 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e7d790 0x55d213ef3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 -- 172.21.15.33:0/548234156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e7d790 msgr2=0x55d213ef3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e7d790 0x55d213ef3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 -- 172.21.15.33:0/548234156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e70ee0 msgr2=0x55d213e821d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e70ee0 0x55d213e821d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.668+0000 7f6421d50640 1 -- 172.21.15.33:0/548234156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d213ef3f50 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.669+0000 7f6421d50640 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e71800 0x55d213e82770 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f640c02a930 tx=0x7f640c004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b883415482d77a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.669+0000 7f6405ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.669+0000 7f6405ffb640 1 -- 172.21.15.33:0/548234156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c035040 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.669+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64140ac370 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.669+0000 7f6405ffb640 1 -- 172.21.15.33:0/548234156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f640c005450 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.669+0000 7f6405ffb640 1 -- 172.21.15.33:0/548234156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f640c054050 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 10 client.19029 fetch_fsmap learned FSMap version 81 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x55d213efb4a0 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6405ffb640 1 -- 172.21.15.33:0/548234156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64140ac370 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6405ffb640 1 -- 172.21.15.33:0/548234156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f640c03a040 con 0x55d213e71800 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 10 client.19029 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 10 client.19029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:00.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 20 client.19029 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213efbbd0 0x55d213efdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 4 client.19029 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:24:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d213de5860 con 0x55d213efbbd0 2024-09-10T21:24:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.670+0000 7f6422551640 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213efbbd0 0x55d213efdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.671+0000 7f6422551640 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213efbbd0 0x55d213efdff0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81acbd2960bcf968 server_cookie=925aacaf3495d940 in_seq=0 out_seq=0 2024-09-10T21:24:00.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.671+0000 7f6405ffb640 10 client.19029 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.673+0000 7f6405ffb640 1 -- 172.21.15.33:0/548234156 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d213de5860 con 0x55d213efbbd0 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6405ffb640 10 client.19029 handle_command_reply: tid=0 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 client.19029 shutdown 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e71800 msgr2=0x55d213e82770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e71800 0x55d213e82770 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f640c02a930 tx=0x7f640c004360 comp rx=0 tx=0).stop 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 shutdown_connections 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d213efbbd0 0x55d213efdff0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d213e7d790 0x55d213ef3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6405ffb640 0 client.19029 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d213e71800 0x55d213e82770 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 --2- 172.21.15.33:0/548234156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d213e70ee0 0x55d213e821d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 >> 172.21.15.33:0/548234156 conn(0x55d213e6ed10 msgr2=0x55d213e7af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 shutdown_connections 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "43b2cb50-7571-4216-b724-5dd1d60c9297", 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:24:00.680 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:24:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.674+0000 7f6426b8a740 1 -- 172.21.15.33:0/548234156 wait complete. 2024-09-10T21:24:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.675+0000 7f6426b8a740 20 client.19029 trim_cache size 0 max 16384 2024-09-10T21:24:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.675+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64080b5360 msgr2=0x7f64080b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.675+0000 7f64247b5640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64080b5360 0x7f64080b7820 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6418009b60 tx=0x7f6418005f90 comp rx=0 tx=0).stop 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.675+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 msgr2=0x7f641c1a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.675+0000 7f64247b5640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 0x7f641c1a0df0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f641400b970 tx=0x7f641400be30 comp rx=0 tx=0).stop 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 shutdown_connections 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f64080b5360 0x7f64080b7820 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f641c10d100 0x7f641c1a0df0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f641c104550 0x7f641c19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 --2- 172.21.15.33:0/2749148471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f641c103b50 0x7f641c19b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 >> 172.21.15.33:0/2749148471 conn(0x7f641c0fd700 msgr2=0x7f641c10f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 shutdown_connections 2024-09-10T21:24:00.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:00.676+0000 7f64247b5640 1 -- 172.21.15.33:0/2749148471 wait complete. 2024-09-10T21:24:01.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 326788 40.10 MB/sec execute 301 sec latency 208.527 ms 2024-09-10T21:24:01.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:01.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:01.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:00 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:01.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:00 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:01.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:01.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:01.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:00 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:01.350 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:00 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:00 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:00 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:01.715 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:24:02.027 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:02.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 327151 40.02 MB/sec execute 302 sec latency 476.697 ms 2024-09-10T21:24:02.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: pgmap v947: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:24:02.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:24:02.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:02.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:02.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:02.303 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.304 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:01 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: pgmap v947: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.312 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:01 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: pgmap v947: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:01 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.602+0000 7f13b988e640 1 -- 172.21.15.33:0/326956480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b4109160 msgr2=0x7f13b410f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.602+0000 7f13b988e640 1 --2- 172.21.15.33:0/326956480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b4109160 0x7f13b410f960 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ca0 tx=0x7f13ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.602+0000 7f13b988e640 1 -- 172.21.15.33:0/326956480 shutdown_connections 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.602+0000 7f13b988e640 1 --2- 172.21.15.33:0/326956480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b4109160 0x7f13b410f960 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.602+0000 7f13b988e640 1 --2- 172.21.15.33:0/326956480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13b41087e0 0x7f13b4108c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.602+0000 7f13b988e640 1 --2- 172.21.15.33:0/326956480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13b4102dd0 0x7f13b41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.602+0000 7f13b988e640 1 -- 172.21.15.33:0/326956480 >> 172.21.15.33:0/326956480 conn(0x7f13b40fe800 msgr2=0x7f13b4100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:02.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.603+0000 7f13b988e640 1 -- 172.21.15.33:0/326956480 shutdown_connections 2024-09-10T21:24:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.603+0000 7f13b988e640 1 -- 172.21.15.33:0/326956480 wait complete. 2024-09-10T21:24:02.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.604+0000 7f13b988e640 1 Processor -- start 2024-09-10T21:24:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.604+0000 7f13b988e640 1 -- start start 2024-09-10T21:24:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.605+0000 7f13b988e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13b4102dd0 0x7f13b4199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.605+0000 7f13b988e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b41087e0 0x7f13b4199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.605+0000 7f13b988e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13b4109160 0x7f13b419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.605+0000 7f13b988e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4112e30 con 0x7f13b41087e0 2024-09-10T21:24:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.605+0000 7f13b988e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4112cb0 con 0x7f13b4102dd0 2024-09-10T21:24:02.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.605+0000 7f13b988e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13b4112fb0 con 0x7f13b4109160 2024-09-10T21:24:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.606+0000 7f13b27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b41087e0 0x7f13b4199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.606+0000 7f13b27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b41087e0 0x7f13b4199840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38458/0 (socket says 172.21.15.33:38458) 2024-09-10T21:24:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.606+0000 7f13b27fc640 1 -- 172.21.15.33:0/1496877211 learned_addr learned my addr 172.21.15.33:0/1496877211 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.606+0000 7f13b37fe640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13b4109160 0x7f13b419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.606+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13b4102dd0 0x7f13b4199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.607+0000 7f13b27fc640 1 -- 172.21.15.33:0/1496877211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13b4109160 msgr2=0x7f13b419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.607+0000 7f13b27fc640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13b4109160 0x7f13b419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.607+0000 7f13b27fc640 1 -- 172.21.15.33:0/1496877211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13b4102dd0 msgr2=0x7f13b4199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.607+0000 7f13b27fc640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13b4102dd0 0x7f13b4199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.607+0000 7f13b27fc640 1 -- 172.21.15.33:0/1496877211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b419f3b0 con 0x7f13b41087e0 2024-09-10T21:24:02.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.607+0000 7f13b27fc640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b41087e0 0x7f13b4199840 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f13a800e9f0 tx=0x7f13a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.608+0000 7f13b888c640 1 -- 172.21.15.33:0/1496877211 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a800cd80 con 0x7f13b41087e0 2024-09-10T21:24:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.608+0000 7f13b888c640 1 -- 172.21.15.33:0/1496877211 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a80108e0 con 0x7f13b41087e0 2024-09-10T21:24:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.608+0000 7f13b888c640 1 -- 172.21.15.33:0/1496877211 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a8018980 con 0x7f13b41087e0 2024-09-10T21:24:02.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.608+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b419f6a0 con 0x7f13b41087e0 2024-09-10T21:24:02.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.609+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b41a5d60 con 0x7f13b41087e0 2024-09-10T21:24:02.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.613+0000 7f13b888c640 1 -- 172.21.15.33:0/1496877211 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f13a8010430 con 0x7f13b41087e0 2024-09-10T21:24:02.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 Processor -- start 2024-09-10T21:24:02.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 -- start start 2024-09-10T21:24:02.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d048d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d049dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d04a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5566d0405e50 con 0x5566d049e1d0 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566d03f5280 con 0x5566d049d850 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.614+0000 7f13bbc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566d03f9090 con 0x5566d048cf00 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d049dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d049dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56838/0 (socket says 172.21.15.33:56838) 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b2ffd640 1 -- 172.21.15.33:0/2618759301 learned_addr learned my addr 172.21.15.33:0/2618759301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b37fe640 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d048d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b888c640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f139c0b5430 0x7f139c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b2ffd640 1 -- 172.21.15.33:0/2618759301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 msgr2=0x5566d048d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:02.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d048d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b2ffd640 1 -- 172.21.15.33:0/2618759301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 msgr2=0x5566d04a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b888c640 1 -- 172.21.15.33:0/1496877211 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f13a80e5e80 con 0x7f13b41087e0 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.615+0000 7f13b27fc640 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d04a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.616+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d04a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.616+0000 7f13b2ffd640 1 -- 172.21.15.33:0/2618759301 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d0153650 con 0x5566d049d850 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.616+0000 7f13b37fe640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f139c0b5430 0x7f139c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.616+0000 7f13b27fc640 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d04a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.616+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d049dc90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009b60 tx=0x7f13a400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=746b9beb37e6543d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:02.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.616+0000 7f1399ffb640 1 -- 172.21.15.33:0/2618759301 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4016070 con 0x5566d049d850 2024-09-10T21:24:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.616+0000 7f1399ffb640 1 -- 172.21.15.33:0/2618759301 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a4009e50 con 0x5566d049d850 2024-09-10T21:24:02.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.617+0000 7f13bbc63740 1 -- 172.21.15.33:0/2618759301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 msgr2=0x5566d049dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.617+0000 7f13b37fe640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f139c0b5430 0x7f139c0b78f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006010 tx=0x7f13ac0095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.617+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d049dc90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009b60 tx=0x7f13a400e6c0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 -- 172.21.15.33:0/2618759301 shutdown_connections 2024-09-10T21:24:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d04a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d049dc90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2618759301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d048d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 -- 172.21.15.33:0/2618759301 >> 172.21.15.33:0/2618759301 conn(0x5566d048f420 msgr2=0x5566d048f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 -- 172.21.15.33:0/2618759301 shutdown_connections 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 -- 172.21.15.33:0/2618759301 wait complete. 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.618+0000 7f13bbc63740 1 -- start start 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 1 Processor -- start 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d048cf00 0x5566d04a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049d850 0x5566d04a3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049e1d0 0x5566d04a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5566d0256700 con 0x5566d049d850 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5566d0390110 con 0x5566d048cf00 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566d04a7280 con 0x5566d049e1d0 2024-09-10T21:24:02.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.619+0000 7f13bbc63740 10 client.0 _ll_register_callbacks cb 0x5566d03a9990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.620+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049e1d0 0x5566d04a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.620+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049e1d0 0x5566d04a4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:41666/0 (socket says 172.21.15.33:41666) 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.620+0000 7f13b37fe640 1 -- 172.21.15.33:0/250244206 learned_addr learned my addr 172.21.15.33:0/250244206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.621+0000 7f13b37fe640 1 -- 172.21.15.33:0/250244206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d048cf00 msgr2=0x5566d04a6870 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.621+0000 7f13b37fe640 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d048cf00 0x5566d04a6870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.621+0000 7f13b37fe640 1 -- 172.21.15.33:0/250244206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049d850 msgr2=0x5566d04a3cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.621+0000 7f13b37fe640 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049d850 0x5566d04a3cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.621+0000 7f13b37fe640 1 -- 172.21.15.33:0/250244206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d03bdc40 con 0x5566d049e1d0 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.622+0000 7f13b37fe640 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049e1d0 0x5566d04a4200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ca0 tx=0x7f13ac02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f932e70d18a4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:02.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.622+0000 7f1392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:24:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.622+0000 7f1392ffd640 1 -- 172.21.15.33:0/250244206 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0318d0 con 0x5566d049e1d0 2024-09-10T21:24:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.622+0000 7f1392ffd640 1 -- 172.21.15.33:0/250244206 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f13ac003b80 con 0x5566d049e1d0 2024-09-10T21:24:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.623+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13a4009e50 con 0x5566d049e1d0 2024-09-10T21:24:02.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.623+0000 7f1392ffd640 1 -- 172.21.15.33:0/250244206 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f13ac003d80 con 0x5566d049e1d0 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.623+0000 7f13bbc63740 10 client.28282 fetch_fsmap learned FSMap version 81 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.623+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5566d04b2a60 con 0x5566d049e1d0 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.624+0000 7f1392ffd640 1 -- 172.21.15.33:0/250244206 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13a4009e50 con 0x5566d049e1d0 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.624+0000 7f1392ffd640 1 -- 172.21.15.33:0/250244206 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f13ac02f8b0 con 0x5566d049e1d0 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.624+0000 7f13bbc63740 10 client.28282 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.624+0000 7f13bbc63740 10 client.28282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.624+0000 7f13bbc63740 20 client.28282 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.624+0000 7f13bbc63740 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d04b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.625+0000 7f13bbc63740 4 client.28282 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.625+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566d0407da0 con 0x5566d04b3fc0 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.625+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d04b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.626+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d04b6450 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14d8bf5ef0f2e0 server_cookie=cd13d6136e4b3efc in_seq=0 out_seq=0 2024-09-10T21:24:02.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.626+0000 7f1392ffd640 10 client.28282 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f1392ffd640 1 -- 172.21.15.33:0/250244206 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5566d0407da0 con 0x5566d04b3fc0 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f1392ffd640 10 client.28282 handle_command_reply: tid=0 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 client.28282 shutdown 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049e1d0 msgr2=0x5566d04a4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049e1d0 0x5566d04a4200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009ca0 tx=0x7f13ac02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 shutdown_connections 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d04b6450 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f1392ffd640 0 client.28282 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049e1d0 0x5566d04a4200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049d850 0x5566d04a3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 --2- 172.21.15.33:0/250244206 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d048cf00 0x5566d04a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 >> 172.21.15.33:0/250244206 conn(0x5566d048f420 msgr2=0x5566d048ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.627+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 shutdown_connections 2024-09-10T21:24:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.628+0000 7f13bbc63740 1 -- 172.21.15.33:0/250244206 wait complete. 2024-09-10T21:24:02.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.628+0000 7f13bbc63740 20 client.28282 trim_cache size 0 max 16384 2024-09-10T21:24:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 Processor -- start 2024-09-10T21:24:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 -- start start 2024-09-10T21:24:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d049c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d048c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d048c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ac270 con 0x5566d049e1d0 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41addb0 con 0x5566d049d850 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.654+0000 7f13bbc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ac940 con 0x5566d048cf00 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d048c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d048c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38474/0 (socket says 172.21.15.33:38474) 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 -- 172.21.15.33:0/2317566679 learned_addr learned my addr 172.21.15.33:0/2317566679 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d049c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b27fc640 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d048c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 -- 172.21.15.33:0/2317566679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 msgr2=0x5566d049c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d049c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 -- 172.21.15.33:0/2317566679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 msgr2=0x5566d048c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d048c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 -- 172.21.15.33:0/2317566679 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566d048dc60 con 0x5566d049e1d0 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.655+0000 7f13b37fe640 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d048c570 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009430 tx=0x7f13ac005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935354bf3a76e085 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.656+0000 7f13997fa640 1 -- 172.21.15.33:0/2317566679 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0318d0 con 0x5566d049e1d0 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.656+0000 7f13997fa640 1 -- 172.21.15.33:0/2317566679 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac036040 con 0x5566d049e1d0 2024-09-10T21:24:02.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.656+0000 7f13997fa640 1 -- 172.21.15.33:0/2317566679 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ac030d00 con 0x5566d049e1d0 2024-09-10T21:24:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 -- 172.21.15.33:0/2317566679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 msgr2=0x5566d048c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d048c570 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f13ac009430 tx=0x7f13ac005bd0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 -- 172.21.15.33:0/2317566679 shutdown_connections 2024-09-10T21:24:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d049e1d0 0x5566d048c570 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049d850 0x5566d048c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 --2- 172.21.15.33:0/2317566679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d048cf00 0x5566d049c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 -- 172.21.15.33:0/2317566679 >> 172.21.15.33:0/2317566679 conn(0x5566d048f420 msgr2=0x5566d049edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 -- 172.21.15.33:0/2317566679 shutdown_connections 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 -- 172.21.15.33:0/2317566679 wait complete. 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.657+0000 7f13bbc63740 1 -- start start 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 1 Processor -- start 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d048cf00 0x5566d04ac550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049d850 0x5566d04aca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049e1d0 0x5566d04bad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ad6e0 con 0x5566d048cf00 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41ad010 con 0x5566d049e1d0 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13b41aa230 con 0x5566d049d850 2024-09-10T21:24:02.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13bbc63740 10 client.0 _ll_register_callbacks cb 0x7f13ac009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049e1d0 0x5566d04bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.658+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d048cf00 0x5566d04ac550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d048cf00 0x5566d04ac550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38482/0 (socket says 172.21.15.33:38482) 2024-09-10T21:24:02.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 -- 172.21.15.33:0/4042873351 learned_addr learned my addr 172.21.15.33:0/4042873351 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049e1d0 0x5566d04bad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56854/0 (socket says 172.21.15.33:56854) 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b27fc640 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049d850 0x5566d04aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 -- 172.21.15.33:0/4042873351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049d850 msgr2=0x5566d04aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049d850 0x5566d04aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 -- 172.21.15.33:0/4042873351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049e1d0 msgr2=0x5566d04bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049e1d0 0x5566d04bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 -- 172.21.15.33:0/4042873351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566d04ad2c0 con 0x5566d048cf00 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b27fc640 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049d850 0x5566d04aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f13b2ffd640 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d048cf00 0x5566d04ac550 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009ab0 tx=0x7f13a40094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc9fd534d05b95e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.659+0000 7f1392ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:24:02.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.660+0000 7f1392ffd640 1 -- 172.21.15.33:0/4042873351 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13a4029040 con 0x5566d048cf00 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.660+0000 7f1392ffd640 1 -- 172.21.15.33:0/4042873351 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f13a4004620 con 0x5566d048cf00 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.660+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13ac036040 con 0x5566d048cf00 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.660+0000 7f1392ffd640 1 -- 172.21.15.33:0/4042873351 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f13a4016040 con 0x5566d048cf00 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.660+0000 7f13bbc63740 10 client.19041 fetch_fsmap learned FSMap version 81 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.660+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) -- 0x5566d04bc390 con 0x5566d048cf00 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.660+0000 7f1392ffd640 1 -- 172.21.15.33:0/4042873351 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13ac036040 con 0x5566d048cf00 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f1392ffd640 1 -- 172.21.15.33:0/4042873351 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 81) ==== 8362+0+0 (secure 0 0 0) 0x7f13a400f040 con 0x5566d048cf00 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13bbc63740 10 client.19041 fetch_fsmap finished waiting for FSMap version 81 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13bbc63740 10 client.19041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13bbc63740 20 client.19041 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13bbc63740 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d0523f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13bbc63740 4 client.19041 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:24:02.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5566d0403850 con 0x5566d04b3fc0 2024-09-10T21:24:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13b27fc640 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d0523f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f13b27fc640 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d0523f80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5f5c05d3ab82cb server_cookie=c021989c3ad146a9 in_seq=0 out_seq=0 2024-09-10T21:24:02.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.661+0000 7f1392ffd640 10 client.19041 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f1392ffd640 1 -- 172.21.15.33:0/4042873351 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7670 (crc 0 0 0) 0x5566d0403850 con 0x5566d04b3fc0 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f1392ffd640 10 client.19041 handle_command_reply: tid=0 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 client.19041 shutdown 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d048cf00 msgr2=0x5566d04ac550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d048cf00 0x5566d04ac550 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009ab0 tx=0x7f13a40094f0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 shutdown_connections 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5566d04b3fc0 0x5566d0523f80 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5566d049e1d0 0x5566d04bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566d049d850 0x5566d04aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 --2- 172.21.15.33:0/4042873351 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5566d048cf00 0x5566d04ac550 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f1392ffd640 0 client.19041 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 >> 172.21.15.33:0/4042873351 conn(0x5566d048f420 msgr2=0x5566d04a52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 shutdown_connections 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 1 -- 172.21.15.33:0/4042873351 wait complete. 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.664+0000 7f13bbc63740 20 client.19041 trim_cache size 0 max 16384 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:24:02.669 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "43b2cb50-7571-4216-b724-5dd1d60c9297", 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:24:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.671 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.672 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.673 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.674 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:24:02.675 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.676 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.677 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:24:02.678 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.679 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.680 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.681 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:24:02.682 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.683 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.684 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.685 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:24:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f139c0b5430 msgr2=0x7f139c0b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f139c0b5430 0x7f139c0b78f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006010 tx=0x7f13ac0095c0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b41087e0 msgr2=0x7f13b4199840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b41087e0 0x7f13b4199840 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f13a800e9f0 tx=0x7f13a800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 shutdown_connections 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f139c0b5430 0x7f139c0b78f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13b4109160 0x7f13b419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f13b41087e0 0x7f13b4199840 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 --2- 172.21.15.33:0/1496877211 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f13b4102dd0 0x7f13b4199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 >> 172.21.15.33:0/1496877211 conn(0x7f13b40fe800 msgr2=0x7f13b40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 shutdown_connections 2024-09-10T21:24:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:02.665+0000 7f13b988e640 1 -- 172.21.15.33:0/1496877211 wait complete. 2024-09-10T21:24:03.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 327745 40.00 MB/sec execute 303 sec latency 274.588 ms 2024-09-10T21:24:03.293 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:02 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:03.293 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:02 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:03.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:02 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:03.307 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:02 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:03.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:02 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:03.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:02 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:03.474 INFO:tasks.cephfs.filesystem:scrub status for tag:43b2cb50-7571-4216-b724-5dd1d60c9297 - {'path': '/', 'tag': '43b2cb50-7571-4216-b724-5dd1d60c9297', 'options': 'recursive,force'} 2024-09-10T21:24:04.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 328252 39.95 MB/sec execute 304 sec latency 391.176 ms 2024-09-10T21:24:04.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:03 smithi104 ceph-mon[35325]: pgmap v948: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.0 MiB/s wr, 644 op/s 2024-09-10T21:24:04.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:03 smithi174 ceph-mon[35635]: pgmap v948: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.0 MiB/s wr, 644 op/s 2024-09-10T21:24:04.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:03 smithi033 ceph-mon[30860]: pgmap v948: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.0 MiB/s wr, 644 op/s 2024-09-10T21:24:05.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 328985 39.95 MB/sec execute 305 sec latency 105.287 ms 2024-09-10T21:24:06.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 329795 39.98 MB/sec execute 306 sec latency 74.457 ms 2024-09-10T21:24:06.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:05 smithi104 ceph-mon[35325]: pgmap v949: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T21:24:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:05 smithi174 ceph-mon[35635]: pgmap v949: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T21:24:06.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:05 smithi033 ceph-mon[30860]: pgmap v949: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 885 op/s 2024-09-10T21:24:07.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 330638 40.01 MB/sec execute 307 sec latency 76.210 ms 2024-09-10T21:24:08.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 331233 39.98 MB/sec execute 308 sec latency 121.155 ms 2024-09-10T21:24:08.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:07 smithi104 ceph-mon[35325]: pgmap v950: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.2 MiB/s wr, 784 op/s 2024-09-10T21:24:08.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:07 smithi174 ceph-mon[35635]: pgmap v950: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.2 MiB/s wr, 784 op/s 2024-09-10T21:24:08.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:07 smithi033 ceph-mon[30860]: pgmap v950: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 9.2 MiB/s wr, 784 op/s 2024-09-10T21:24:09.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 331704 39.93 MB/sec execute 309 sec latency 286.306 ms 2024-09-10T21:24:10.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 332553 39.92 MB/sec execute 310 sec latency 112.142 ms 2024-09-10T21:24:10.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:09 smithi104 ceph-mon[35325]: pgmap v951: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-09-10T21:24:10.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:09 smithi174 ceph-mon[35635]: pgmap v951: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-09-10T21:24:10.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:09 smithi033 ceph-mon[30860]: pgmap v951: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 852 op/s 2024-09-10T21:24:11.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 333529 39.97 MB/sec execute 311 sec latency 57.332 ms 2024-09-10T21:24:11.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:10 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:11.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:10 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:11.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:10 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:12.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 334436 39.99 MB/sec execute 312 sec latency 67.704 ms 2024-09-10T21:24:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:11 smithi104 ceph-mon[35325]: pgmap v952: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 851 op/s 2024-09-10T21:24:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:11 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:24:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:11 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:24:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:12 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:24:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:12 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:24:12.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:12 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:24:12.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:12 smithi174 ceph-mon[35635]: pgmap v952: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 851 op/s 2024-09-10T21:24:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:12 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:24:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:12 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:24:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:12 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:24:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:12 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:24:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:12 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:24:12.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:11 smithi033 ceph-mon[30860]: pgmap v952: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 851 op/s 2024-09-10T21:24:12.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:11 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:24:12.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:11 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:24:12.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:11 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:24:12.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:12 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:24:12.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:12 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:24:13.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 335333 40.04 MB/sec execute 313 sec latency 74.761 ms 2024-09-10T21:24:14.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 335693 39.98 MB/sec execute 314 sec latency 301.370 ms 2024-09-10T21:24:14.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:14 smithi104 ceph-mon[35325]: pgmap v953: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.4 MiB/s wr, 725 op/s 2024-09-10T21:24:14.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:14 smithi174 ceph-mon[35635]: pgmap v953: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.4 MiB/s wr, 725 op/s 2024-09-10T21:24:14.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:14 smithi033 ceph-mon[30860]: pgmap v953: 97 pgs: 97 active+clean; 1.2 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.4 MiB/s wr, 725 op/s 2024-09-10T21:24:15.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 336219 39.94 MB/sec execute 315 sec latency 181.002 ms 2024-09-10T21:24:16.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 336890 39.93 MB/sec execute 316 sec latency 89.914 ms 2024-09-10T21:24:16.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:16 smithi104 ceph-mon[35325]: pgmap v954: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 890 op/s 2024-09-10T21:24:16.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:16 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:16.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:16 smithi174 ceph-mon[35635]: pgmap v954: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 890 op/s 2024-09-10T21:24:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:16 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:16.428 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:16 smithi033 ceph-mon[30860]: pgmap v954: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 890 op/s 2024-09-10T21:24:16.429 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:16 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:17.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 337530 39.90 MB/sec execute 317 sec latency 177.966 ms 2024-09-10T21:24:18.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 338264 39.90 MB/sec execute 318 sec latency 95.209 ms 2024-09-10T21:24:18.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:18 smithi104 ceph-mon[35325]: pgmap v955: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 778 op/s 2024-09-10T21:24:18.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:18 smithi174 ceph-mon[35635]: pgmap v955: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 778 op/s 2024-09-10T21:24:18.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:18 smithi033 ceph-mon[30860]: pgmap v955: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.7 MiB/s wr, 778 op/s 2024-09-10T21:24:19.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 338680 39.85 MB/sec execute 319 sec latency 285.461 ms 2024-09-10T21:24:20.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 339215 39.79 MB/sec execute 320 sec latency 189.298 ms 2024-09-10T21:24:20.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:20 smithi104 ceph-mon[35325]: pgmap v956: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 814 op/s 2024-09-10T21:24:20.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:20 smithi174 ceph-mon[35635]: pgmap v956: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 814 op/s 2024-09-10T21:24:20.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:20 smithi033 ceph-mon[30860]: pgmap v956: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 814 op/s 2024-09-10T21:24:21.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 339893 39.78 MB/sec execute 321 sec latency 113.304 ms 2024-09-10T21:24:22.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 340692 39.82 MB/sec execute 322 sec latency 252.984 ms 2024-09-10T21:24:22.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:22 smithi104 ceph-mon[35325]: pgmap v957: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 806 op/s 2024-09-10T21:24:22.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:22 smithi174 ceph-mon[35635]: pgmap v957: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 806 op/s 2024-09-10T21:24:22.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:22 smithi033 ceph-mon[30860]: pgmap v957: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 806 op/s 2024-09-10T21:24:23.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 341354 39.83 MB/sec execute 323 sec latency 71.656 ms 2024-09-10T21:24:24.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 341676 39.75 MB/sec execute 324 sec latency 326.333 ms 2024-09-10T21:24:24.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:24 smithi104 ceph-mon[35325]: pgmap v958: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 692 op/s 2024-09-10T21:24:24.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:24 smithi174 ceph-mon[35635]: pgmap v958: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 692 op/s 2024-09-10T21:24:24.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:24 smithi033 ceph-mon[30860]: pgmap v958: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.9 MiB/s wr, 692 op/s 2024-09-10T21:24:25.068 INFO:tasks.workunit.client.0.smithi033.stdout: 10 342227 39.69 MB/sec execute 325 sec latency 110.596 ms 2024-09-10T21:24:26.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 343078 39.71 MB/sec execute 326 sec latency 65.606 ms 2024-09-10T21:24:26.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:26 smithi104 ceph-mon[35325]: pgmap v959: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 830 op/s 2024-09-10T21:24:26.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:26 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:26.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:26 smithi174 ceph-mon[35635]: pgmap v959: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 830 op/s 2024-09-10T21:24:26.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:26 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:26.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:26 smithi033 ceph-mon[30860]: pgmap v959: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 830 op/s 2024-09-10T21:24:26.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:26 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:24:27.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 343899 39.73 MB/sec execute 327 sec latency 89.407 ms 2024-09-10T21:24:28.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 344675 39.74 MB/sec execute 328 sec latency 52.757 ms 2024-09-10T21:24:28.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:28 smithi104 ceph-mon[35325]: pgmap v960: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 715 op/s 2024-09-10T21:24:28.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:28 smithi174 ceph-mon[35635]: pgmap v960: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 715 op/s 2024-09-10T21:24:28.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:28 smithi033 ceph-mon[30860]: pgmap v960: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 715 op/s 2024-09-10T21:24:29.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 345291 39.72 MB/sec execute 329 sec latency 344.919 ms 2024-09-10T21:24:30.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 345993 39.74 MB/sec execute 330 sec latency 97.126 ms 2024-09-10T21:24:30.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:30 smithi104 ceph-mon[35325]: pgmap v961: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 703 op/s 2024-09-10T21:24:30.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:30 smithi174 ceph-mon[35635]: pgmap v961: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 703 op/s 2024-09-10T21:24:30.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:30 smithi033 ceph-mon[30860]: pgmap v961: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 703 op/s 2024-09-10T21:24:31.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 346711 39.72 MB/sec execute 331 sec latency 65.645 ms 2024-09-10T21:24:32.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 347343 39.70 MB/sec execute 332 sec latency 96.504 ms 2024-09-10T21:24:32.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:32 smithi104 ceph-mon[35325]: pgmap v962: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 785 op/s 2024-09-10T21:24:32.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:32 smithi174 ceph-mon[35635]: pgmap v962: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 785 op/s 2024-09-10T21:24:32.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:32 smithi033 ceph-mon[30860]: pgmap v962: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 785 op/s 2024-09-10T21:24:33.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 348092 39.68 MB/sec execute 333 sec latency 71.132 ms 2024-09-10T21:24:33.476 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:24:33.881 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:34.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 348752 39.65 MB/sec execute 334 sec latency 204.797 ms 2024-09-10T21:24:34.308 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:34 smithi033 ceph-mon[30860]: pgmap v963: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.8 MiB/s wr, 665 op/s 2024-09-10T21:24:34.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:34 smithi104 ceph-mon[35325]: pgmap v963: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.8 MiB/s wr, 665 op/s 2024-09-10T21:24:34.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:34 smithi174 ceph-mon[35635]: pgmap v963: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.8 MiB/s wr, 665 op/s 2024-09-10T21:24:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 -- 172.21.15.33:0/92644962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0104580 msgr2=0x7fb0a010cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/92644962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0104580 0x7fb0a010cb60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb094009860 tx=0x7fb09402a340 comp rx=0 tx=0).stop 2024-09-10T21:24:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 -- 172.21.15.33:0/92644962 shutdown_connections 2024-09-10T21:24:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/92644962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a010d130 0x7fb0a010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/92644962 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0104580 0x7fb0a010cb60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/92644962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a0103b80 0x7fb0a0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 -- 172.21.15.33:0/92644962 >> 172.21.15.33:0/92644962 conn(0x7fb0a00fd6f0 msgr2=0x7fb0a00ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 -- 172.21.15.33:0/92644962 shutdown_connections 2024-09-10T21:24:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.424+0000 7fb0a4f72640 1 -- 172.21.15.33:0/92644962 wait complete. 2024-09-10T21:24:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.425+0000 7fb0a4f72640 1 Processor -- start 2024-09-10T21:24:34.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb0a4f72640 1 -- start start 2024-09-10T21:24:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb0a4f72640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0103b80 0x7fb0a0101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb0a4f72640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a0104580 0x7fb0a0101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb0a4f72640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a010d130 0x7fb0a0102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb09e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0103b80 0x7fb0a0101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb0a4f72640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0112d70 con 0x7fb0a010d130 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb0a4f72640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0112bf0 con 0x7fb0a0103b80 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb0a4f72640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a0112ef0 con 0x7fb0a0104580 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a0104580 0x7fb0a0101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a0104580 0x7fb0a0101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59664/0 (socket says 172.21.15.33:59664) 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09dd9b640 1 -- 172.21.15.33:0/3304735583 learned_addr learned my addr 172.21.15.33:0/3304735583 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb09e59c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0103b80 0x7fb0a0101a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53270/0 (socket says 172.21.15.33:53270) 2024-09-10T21:24:34.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.426+0000 7fb09ed9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a010d130 0x7fb0a0102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09e59c640 1 -- 172.21.15.33:0/3304735583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a0104580 msgr2=0x7fb0a0101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09e59c640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a0104580 0x7fb0a0101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09e59c640 1 -- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a010d130 msgr2=0x7fb0a0102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09e59c640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a010d130 0x7fb0a0102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09e59c640 1 -- 172.21.15.33:0/3304735583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a01a76a0 con 0x7fb0a0103b80 2024-09-10T21:24:34.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.427+0000 7fb09e59c640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0103b80 0x7fb0a0101a00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb09000d6d0 tx=0x7fb09000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.428+0000 7fb08f7fe640 1 -- 172.21.15.33:0/3304735583 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090017070 con 0x7fb0a0103b80 2024-09-10T21:24:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.428+0000 7fb08f7fe640 1 -- 172.21.15.33:0/3304735583 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb090009820 con 0x7fb0a0103b80 2024-09-10T21:24:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.428+0000 7fb08f7fe640 1 -- 172.21.15.33:0/3304735583 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090017070 con 0x7fb0a0103b80 2024-09-10T21:24:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.428+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a01a7990 con 0x7fb0a0103b80 2024-09-10T21:24:34.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.428+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a01a7e80 con 0x7fb0a0103b80 2024-09-10T21:24:34.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 Processor -- start 2024-09-10T21:24:34.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 -- start start 2024-09-10T21:24:34.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb08f7fe640 1 -- 172.21.15.33:0/3304735583 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb090004bd0 con 0x7fb0a0103b80 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad3e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad53000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad55930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56052acb4e50 con 0x56052ad53540 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56052aca4280 con 0x56052ad3eda0 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb0a7347740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56052aca8090 con 0x56052ad3e480 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb09dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad55930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.432+0000 7fb09ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad3e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad3e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59672/0 (socket says 172.21.15.33:59672) 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09ed9d640 1 -- 172.21.15.33:0/2664202567 learned_addr learned my addr 172.21.15.33:0/2664202567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09dd9b640 1 -- 172.21.15.33:0/2664202567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 msgr2=0x56052ad3e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/2664202567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad3e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09dd9b640 1 -- 172.21.15.33:0/2664202567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 msgr2=0x56052ad53000 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/2664202567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad53000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09dd9b640 1 -- 172.21.15.33:0/2664202567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052aa02650 con 0x56052ad53540 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/2664202567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad55930 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb094005cd0 tx=0x7fb0940095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a082f94d6b712917 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb08cff9640 1 -- 172.21.15.33:0/2664202567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094040020 con 0x56052ad53540 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb08cff9640 1 -- 172.21.15.33:0/2664202567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb094005040 con 0x56052ad53540 2024-09-10T21:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.433+0000 7fb08cff9640 1 -- 172.21.15.33:0/2664202567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb094037040 con 0x56052ad53540 2024-09-10T21:24:34.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.434+0000 7fb08f7fe640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0880b56d0 0x7fb0880b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.434+0000 7fb09ed9d640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0880b56d0 0x7fb0880b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.434+0000 7fb08f7fe640 1 -- 172.21.15.33:0/3304735583 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb090005020 con 0x7fb0a0103b80 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.434+0000 7fb0a7347740 1 -- 172.21.15.33:0/2664202567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 msgr2=0x56052ad55930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.434+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2664202567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad55930 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fb094005cd0 tx=0x7fb0940095b0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 -- 172.21.15.33:0/2664202567 shutdown_connections 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2664202567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad55930 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2664202567 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad53000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2664202567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad3e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 -- 172.21.15.33:0/2664202567 >> 172.21.15.33:0/2664202567 conn(0x56052ad3c2b0 msgr2=0x56052ad3d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 -- 172.21.15.33:0/2664202567 shutdown_connections 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb09ed9d640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0880b56d0 0x7fb0880b7b90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e620 tx=0x7fb098010040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 -- 172.21.15.33:0/2664202567 wait complete. 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.435+0000 7fb0a7347740 1 -- start start 2024-09-10T21:24:34.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 1 Processor -- start 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad52e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad502c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad50940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56052ab05700 con 0x56052ad53540 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56052ac3f110 con 0x56052ad3eda0 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56052ad56160 con 0x56052ad3e480 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb0a7347740 10 client.0 _ll_register_callbacks cb 0x56052ac58990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb09e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad52e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb09e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad52e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59676/0 (socket says 172.21.15.33:59676) 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb09e59c640 1 -- 172.21.15.33:0/3215293412 learned_addr learned my addr 172.21.15.33:0/3215293412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb09ed9d640 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad50940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.436+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb09e59c640 1 -- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 msgr2=0x56052ad502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb09e59c640 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb09e59c640 1 -- 172.21.15.33:0/3215293412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 msgr2=0x56052ad50940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb09e59c640 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad50940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb09e59c640 1 -- 172.21.15.33:0/3215293412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052ac6cc40 con 0x56052ad3e480 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb09e59c640 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad52e70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0900e2c80 tx=0x7fb090009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffce2f16f3b63a2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb085ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb085ffb640 1 -- 172.21.15.33:0/3215293412 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090017070 con 0x56052ad3e480 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.437+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb094005040 con 0x56052ad3e480 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb085ffb640 1 -- 172.21.15.33:0/3215293412 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb0900a7bb0 con 0x56052ad3e480 2024-09-10T21:24:34.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb085ffb640 1 -- 172.21.15.33:0/3215293412 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb090020070 con 0x56052ad3e480 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 10 client.28300 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56052ad61ad0 con 0x56052ad3e480 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb085ffb640 1 -- 172.21.15.33:0/3215293412 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb094005040 con 0x56052ad3e480 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb085ffb640 1 -- 172.21.15.33:0/3215293412 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fb090005220 con 0x56052ad3e480 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 10 client.28300 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 10 client.28300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 20 client.28300 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052ad63410 0x56052ad65830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 4 client.28300 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.438+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56052acb6da0 con 0x56052ad63410 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.439+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052ad63410 0x56052ad65830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.439+0000 7fb085ffb640 10 client.28300 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:34.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.440+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052ad63410 0x56052ad65830 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85236db69d646e5d server_cookie=7658122446bfadb0 in_seq=0 out_seq=0 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb085ffb640 1 -- 172.21.15.33:0/3215293412 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56052acb6da0 con 0x56052ad63410 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb085ffb640 10 client.28300 handle_command_reply: tid=0 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 client.28300 shutdown 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 msgr2=0x56052ad52e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad52e70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb0900e2c80 tx=0x7fb090009ec0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 shutdown_connections 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052ad63410 0x56052ad65830 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad50940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 --2- 172.21.15.33:0/3215293412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad52e70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb085ffb640 0 client.28300 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 >> 172.21.15.33:0/3215293412 conn(0x56052ad3c2b0 msgr2=0x56052ad41e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 shutdown_connections 2024-09-10T21:24:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 1 -- 172.21.15.33:0/3215293412 wait complete. 2024-09-10T21:24:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.444+0000 7fb0a7347740 20 client.28300 trim_cache size 0 max 16384 2024-09-10T21:24:34.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.468+0000 7fb0a7347740 1 Processor -- start 2024-09-10T21:24:34.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.468+0000 7fb0a7347740 1 -- start start 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad5f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad4eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad4f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb0a7347740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a01ac2a0 con 0x56052ad53540 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb0a7347740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a01adea0 con 0x56052ad3eda0 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb0a7347740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a01aca30 con 0x56052ad3e480 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad5f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad5f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:59682/0 (socket says 172.21.15.33:59682) 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 -- 172.21.15.33:0/2082574879 learned_addr learned my addr 172.21.15.33:0/2082574879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad4eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09ed9d640 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad4f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 -- 172.21.15.33:0/2082574879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 msgr2=0x56052ad4eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad4eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 -- 172.21.15.33:0/2082574879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 msgr2=0x56052ad4f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad4f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.469+0000 7fb09e59c640 1 -- 172.21.15.33:0/2082574879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56052ad37b60 con 0x56052ad3e480 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.470+0000 7fb09e59c640 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad5f390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb0900b0220 tx=0x7fb090000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde0e070b5de27bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.470+0000 7fb087fff640 1 -- 172.21.15.33:0/2082574879 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090017050 con 0x56052ad3e480 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.470+0000 7fb087fff640 1 -- 172.21.15.33:0/2082574879 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0900ac040 con 0x56052ad3e480 2024-09-10T21:24:34.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.470+0000 7fb087fff640 1 -- 172.21.15.33:0/2082574879 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090005220 con 0x56052ad3e480 2024-09-10T21:24:34.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 -- 172.21.15.33:0/2082574879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 msgr2=0x56052ad5f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad5f390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb0900b0220 tx=0x7fb090000bc0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 -- 172.21.15.33:0/2082574879 shutdown_connections 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad53540 0x56052ad4f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad4eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 --2- 172.21.15.33:0/2082574879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad3e480 0x56052ad5f390 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 -- 172.21.15.33:0/2082574879 >> 172.21.15.33:0/2082574879 conn(0x56052ad3c2b0 msgr2=0x56052ad55370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 -- 172.21.15.33:0/2082574879 shutdown_connections 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 -- 172.21.15.33:0/2082574879 wait complete. 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.471+0000 7fb0a7347740 1 -- start start 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 1 Processor -- start 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad3e480 0x56052ad5b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad5b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad53540 0x56052ad583c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a01ad7d0 con 0x56052ad3e480 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a01ad100 con 0x56052ad3eda0 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a01aa260 con 0x56052ad53540 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb0a7347740 10 client.0 _ll_register_callbacks cb 0x7fb090005a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb09dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad5b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb09ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad53540 0x56052ad583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb09dd9b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad5b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53300/0 (socket says 172.21.15.33:53300) 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb09dd9b640 1 -- 172.21.15.33:0/231922876 learned_addr learned my addr 172.21.15.33:0/231922876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.472+0000 7fb09e59c640 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad3e480 0x56052ad5b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb09e59c640 1 -- 172.21.15.33:0/231922876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad53540 msgr2=0x56052ad583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb09e59c640 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad53540 0x56052ad583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb09e59c640 1 -- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 msgr2=0x56052ad5b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb09e59c640 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad5b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb09e59c640 1 -- 172.21.15.33:0/231922876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56052ad58900 con 0x56052ad3e480 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad5b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb09e59c640 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad3e480 0x56052ad5b3f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fb090009610 tx=0x7fb090005450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9130f78386d236fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb085ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb085ffb640 1 -- 172.21.15.33:0/231922876 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090017070 con 0x56052ad3e480 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0900ac040 con 0x56052ad3e480 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb085ffb640 1 -- 172.21.15.33:0/231922876 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb0900aadb0 con 0x56052ad3e480 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb085ffb640 1 -- 172.21.15.33:0/231922876 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb0900ab940 con 0x56052ad3e480 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb0a7347740 10 client.19059 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.473+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56052add0a80 con 0x56052ad3e480 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb085ffb640 1 -- 172.21.15.33:0/231922876 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0900ac040 con 0x56052ad3e480 2024-09-10T21:24:34.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb085ffb640 1 -- 172.21.15.33:0/231922876 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fb0900aa6a0 con 0x56052ad3e480 2024-09-10T21:24:34.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb0a7347740 10 client.19059 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:34.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb0a7347740 10 client.19059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:34.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb0a7347740 20 client.19059 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:34.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb0a7347740 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052add0f50 0x56052add3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb0a7347740 4 client.19059 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:24:34.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.474+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56052acb2850 con 0x56052add0f50 2024-09-10T21:24:34.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.475+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052add0f50 0x56052add3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.477+0000 7fb09dd9b640 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052add0f50 0x56052add3370 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57699dcfd04a6e63 server_cookie=3026766499b45b3d in_seq=0 out_seq=0 2024-09-10T21:24:34.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.477+0000 7fb085ffb640 10 client.19059 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:34.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb085ffb640 1 -- 172.21.15.33:0/231922876 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7474 (crc 0 0 0) 0x56052acb2850 con 0x56052add0f50 2024-09-10T21:24:34.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb085ffb640 10 client.19059 handle_command_reply: tid=0 2024-09-10T21:24:34.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 client.19059 shutdown 2024-09-10T21:24:34.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad3e480 msgr2=0x56052ad5b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad3e480 0x56052ad5b3f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fb090009610 tx=0x7fb090005450 comp rx=0 tx=0).stop 2024-09-10T21:24:34.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 shutdown_connections 2024-09-10T21:24:34.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56052add0f50 0x56052add3370 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56052ad53540 0x56052ad583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb085ffb640 0 client.19059 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:34.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56052ad3eda0 0x56052ad5b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 --2- 172.21.15.33:0/231922876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56052ad3e480 0x56052ad5b3f0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.479+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 >> 172.21.15.33:0/231922876 conn(0x56052ad3c2b0 msgr2=0x56052ad4ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.480+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 shutdown_connections 2024-09-10T21:24:34.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.480+0000 7fb0a7347740 1 -- 172.21.15.33:0/231922876 wait complete. 2024-09-10T21:24:34.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.480+0000 7fb0a7347740 20 client.19059 trim_cache size 0 max 16384 2024-09-10T21:24:34.484 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:24:34.484 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:24:34.484 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:24:34.484 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.485 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:24:34.486 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.487 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:24:34.488 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.489 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.490 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.491 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.492 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:24:34.493 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.494 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.495 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.496 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.497 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.498 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:24:34.499 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:34.500 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0880b56d0 msgr2=0x7fb0880b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0880b56d0 0x7fb0880b7b90 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb09800e620 tx=0x7fb098010040 comp rx=0 tx=0).stop 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0103b80 msgr2=0x7fb0a0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0103b80 0x7fb0a0101a00 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb09000d6d0 tx=0x7fb09000db90 comp rx=0 tx=0).stop 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 shutdown_connections 2024-09-10T21:24:34.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb0880b56d0 0x7fb0880b7b90 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0a010d130 0x7fb0a0102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0a0104580 0x7fb0a0101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 --2- 172.21.15.33:0/3304735583 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0a0103b80 0x7fb0a0101a00 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 >> 172.21.15.33:0/3304735583 conn(0x7fb0a00fd6f0 msgr2=0x7fb0a00fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 shutdown_connections 2024-09-10T21:24:34.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:34.481+0000 7fb0a4f72640 1 -- 172.21.15.33:0/3304735583 wait complete. 2024-09-10T21:24:35.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 349582 39.70 MB/sec execute 335 sec latency 109.503 ms 2024-09-10T21:24:35.767 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:24:35.768 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:24:36.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 350333 39.72 MB/sec execute 336 sec latency 88.541 ms 2024-09-10T21:24:36.115 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:36.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:36 smithi104 ceph-mon[35325]: pgmap v964: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 836 op/s 2024-09-10T21:24:36.387 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:36 smithi033 ceph-mon[30860]: pgmap v964: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 836 op/s 2024-09-10T21:24:36.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:36 smithi174 ceph-mon[35635]: pgmap v964: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 836 op/s 2024-09-10T21:24:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.648+0000 7f18a6242640 1 -- 172.21.15.33:0/1030103449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 msgr2=0x7f18a0107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.648+0000 7f18a6242640 1 --2- 172.21.15.33:0/1030103449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0107b00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1894009900 tx=0x7f189402a410 comp rx=0 tx=0).stop 2024-09-10T21:24:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 -- 172.21.15.33:0/1030103449 shutdown_connections 2024-09-10T21:24:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 --2- 172.21.15.33:0/1030103449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18a0108040 0x7f18a010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:36.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 --2- 172.21.15.33:0/1030103449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0107b00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 --2- 172.21.15.33:0/1030103449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18a0101cb0 0x7f18a0102090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 -- 172.21.15.33:0/1030103449 >> 172.21.15.33:0/1030103449 conn(0x7f18a00fd700 msgr2=0x7f18a00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 -- 172.21.15.33:0/1030103449 shutdown_connections 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 -- 172.21.15.33:0/1030103449 wait complete. 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.649+0000 7f18a6242640 1 Processor -- start 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f18a6242640 1 -- start start 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f18a6242640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18a0101cb0 0x7f18a0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f18a6242640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f18a6242640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18a0108040 0x7f18a019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f18a6242640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0110c00 con 0x7f18a01076c0 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f18a6242640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0110a80 con 0x7f18a0101cb0 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f18a6242640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18a0110d80 con 0x7f18a0108040 2024-09-10T21:24:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f189effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f189effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34774/0 (socket says 172.21.15.33:34774) 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f189effd640 1 -- 172.21.15.33:0/3646069402 learned_addr learned my addr 172.21.15.33:0/3646069402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f189ffff640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18a0108040 0x7f18a019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.650+0000 7f189f7fe640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18a0101cb0 0x7f18a0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189effd640 1 -- 172.21.15.33:0/3646069402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18a0108040 msgr2=0x7f18a019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189effd640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18a0108040 0x7f18a019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189effd640 1 -- 172.21.15.33:0/3646069402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18a0101cb0 msgr2=0x7f18a0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189effd640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18a0101cb0 0x7f18a0197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189effd640 1 -- 172.21.15.33:0/3646069402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a019d1c0 con 0x7f18a01076c0 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189ffff640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18a0108040 0x7f18a019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189f7fe640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18a0101cb0 0x7f18a0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189effd640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0197650 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1894005940 tx=0x7f18940042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189cff9640 1 -- 172.21.15.33:0/3646069402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189403f030 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189cff9640 1 -- 172.21.15.33:0/3646069402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1894003cb0 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a019d450 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.651+0000 7f189cff9640 1 -- 172.21.15.33:0/3646069402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1894035070 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.652+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a01a3b70 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.653+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a0102090 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.656+0000 7f189cff9640 1 -- 172.21.15.33:0/3646069402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1894003800 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.657+0000 7f189cff9640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f18880b75f0 0x7f18880b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.657+0000 7f189f7fe640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f18880b75f0 0x7f18880b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.657+0000 7f189cff9640 1 -- 172.21.15.33:0/3646069402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f18941068a0 con 0x7f18a01076c0 2024-09-10T21:24:36.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.658+0000 7f189f7fe640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f18880b75f0 0x7f18880b9ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1890009930 tx=0x7f1890008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:36.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:36.663+0000 7f189cff9640 1 -- 172.21.15.33:0/3646069402 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f1894049060 con 0x7f18a01076c0 2024-09-10T21:24:37.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.039+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18a0198240 con 0x7f18a01076c0 2024-09-10T21:24:37.042 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:24:37.043 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"btime":"2024-09-10T21:24:25:023764+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":84,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:24:24.175284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:24:37.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.040+0000 7f189cff9640 1 -- 172.21.15.33:0/3646069402 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+11980 (secure 0 0 0) 0x7f18a0198240 con 0x7f18a01076c0 2024-09-10T21:24:37.043 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.043+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f18880b75f0 msgr2=0x7f18880b9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.043+0000 7f18a6242640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f18880b75f0 0x7f18880b9ab0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1890009930 tx=0x7f1890008040 comp rx=0 tx=0).stop 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.043+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 msgr2=0x7f18a0197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.043+0000 7f18a6242640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0197650 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1894005940 tx=0x7f18940042f0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.043+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 shutdown_connections 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.044+0000 7f18a6242640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f18880b75f0 0x7f18880b9ab0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.044+0000 7f18a6242640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18a0108040 0x7f18a019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.044+0000 7f18a6242640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18a01076c0 0x7f18a0197650 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.044+0000 7f18a6242640 1 --2- 172.21.15.33:0/3646069402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18a0101cb0 0x7f18a0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.044+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 >> 172.21.15.33:0/3646069402 conn(0x7f18a00fd700 msgr2=0x7f18a00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.044+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 shutdown_connections 2024-09-10T21:24:37.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:37.044+0000 7f18a6242640 1 -- 172.21.15.33:0/3646069402 wait complete. 2024-09-10T21:24:37.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 350886 39.69 MB/sec execute 337 sec latency 87.186 ms 2024-09-10T21:24:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:37 smithi104 ceph-mon[35325]: pgmap v965: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 747 op/s 2024-09-10T21:24:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:37 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:24:37.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:37 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3646069402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:24:37.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:37 smithi033 ceph-mon[30860]: pgmap v965: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 747 op/s 2024-09-10T21:24:37.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:37 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:24:37.392 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:37 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3646069402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:24:37.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:37 smithi174 ceph-mon[35635]: pgmap v965: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 747 op/s 2024-09-10T21:24:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:37 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:24:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:37 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3646069402' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:24:37.838 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:24:38.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 351403 39.65 MB/sec execute 338 sec latency 111.323 ms 2024-09-10T21:24:39.069 INFO:tasks.workunit.client.0.smithi033.stdout: 10 351850 39.60 MB/sec execute 339 sec latency 376.287 ms 2024-09-10T21:24:39.247 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:39.642 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:39 smithi033 ceph-mon[30860]: pgmap v966: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 766 op/s 2024-09-10T21:24:39.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:39 smithi104 ceph-mon[35325]: pgmap v966: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 766 op/s 2024-09-10T21:24:39.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:39 smithi174 ceph-mon[35635]: pgmap v966: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.7 MiB/s wr, 766 op/s 2024-09-10T21:24:40.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 352488 39.56 MB/sec execute 340 sec latency 390.019 ms 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.195+0000 7f7d89569640 1 -- 172.21.15.33:0/957781474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d84104fb0 msgr2=0x7f7d84105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.195+0000 7f7d89569640 1 --2- 172.21.15.33:0/957781474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d84104fb0 0x7f7d84105390 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009900 tx=0x7f7d7402a420 comp rx=0 tx=0).stop 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.195+0000 7f7d89569640 1 -- 172.21.15.33:0/957781474 shutdown_connections 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.195+0000 7f7d89569640 1 --2- 172.21.15.33:0/957781474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410b340 0x7f7d84111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.195+0000 7f7d89569640 1 --2- 172.21.15.33:0/957781474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d8410a9c0 0x7f7d8410ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.195+0000 7f7d89569640 1 --2- 172.21.15.33:0/957781474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d84104fb0 0x7f7d84105390 secure :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009900 tx=0x7f7d7402a420 comp rx=0 tx=0).stop 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.195+0000 7f7d89569640 1 -- 172.21.15.33:0/957781474 >> 172.21.15.33:0/957781474 conn(0x7f7d84100a20 msgr2=0x7f7d84102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.196+0000 7f7d89569640 1 -- 172.21.15.33:0/957781474 shutdown_connections 2024-09-10T21:24:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.197+0000 7f7d89569640 1 -- 172.21.15.33:0/957781474 wait complete. 2024-09-10T21:24:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 Processor -- start 2024-09-10T21:24:40.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 -- start start 2024-09-10T21:24:40.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410a9c0 0x7f7d8419b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d8410b340 0x7f7d8419bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d841a0f40 0x7f7d841a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84115030 con 0x7f7d841a0f40 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d84114eb0 con 0x7f7d8410a9c0 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.198+0000 7f7d89569640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841151b0 con 0x7f7d8410b340 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.199+0000 7f7d82ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410a9c0 0x7f7d8419b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.199+0000 7f7d82ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410a9c0 0x7f7d8419b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56396/0 (socket says 172.21.15.33:56396) 2024-09-10T21:24:40.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.199+0000 7f7d82ffd640 1 -- 172.21.15.33:0/1070732166 learned_addr learned my addr 172.21.15.33:0/1070732166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.199+0000 7f7d827fc640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d8410b340 0x7f7d8419bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.200+0000 7f7d82ffd640 1 -- 172.21.15.33:0/1070732166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d8410b340 msgr2=0x7f7d8419bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.200+0000 7f7d82ffd640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d8410b340 0x7f7d8419bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.200+0000 7f7d82ffd640 1 -- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d841a0f40 msgr2=0x7f7d841a13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.200+0000 7f7d82ffd640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d841a0f40 0x7f7d841a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.200+0000 7f7d82ffd640 1 -- 172.21.15.33:0/1070732166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d841a1ad0 con 0x7f7d8410a9c0 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.200+0000 7f7d82ffd640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410a9c0 0x7f7d8419b660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005ab0 tx=0x7f7d74003b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.201+0000 7f7d6bfff640 1 -- 172.21.15.33:0/1070732166 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7403f020 con 0x7f7d8410a9c0 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.201+0000 7f7d6bfff640 1 -- 172.21.15.33:0/1070732166 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d74031b50 con 0x7f7d8410a9c0 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.201+0000 7f7d6bfff640 1 -- 172.21.15.33:0/1070732166 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74035070 con 0x7f7d8410a9c0 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.201+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d841a1d60 con 0x7f7d8410a9c0 2024-09-10T21:24:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.201+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d841a8380 con 0x7f7d8410a9c0 2024-09-10T21:24:40.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.204+0000 7f7d8b93e740 1 Processor -- start 2024-09-10T21:24:40.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.204+0000 7f7d8b93e740 1 -- start start 2024-09-10T21:24:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bed900 0x562b67bedce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b67b5bea0 con 0x562b67be8dd0 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b67b4b2d0 con 0x562b67bed900 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b67b4f0e0 con 0x562b67bee220 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d827fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bed900 0x562b67bedce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bed900 0x562b67bedce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56412/0 (socket says 172.21.15.33:56412) 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d827fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47446/0 (socket says 172.21.15.33:47446) 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 -- 172.21.15.33:0/1010270430 learned_addr learned my addr 172.21.15.33:0/1010270430 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d6bfff640 1 -- 172.21.15.33:0/1070732166 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7d740316a0 con 0x7f7d8410a9c0 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d82ffd640 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 -- 172.21.15.33:0/1010270430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 msgr2=0x562b67be8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 -- 172.21.15.33:0/1010270430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 msgr2=0x562b67be9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d837fe640 1 -- 172.21.15.33:0/1010270430 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b678a9650 con 0x562b67bed900 2024-09-10T21:24:40.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d827fc640 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:40.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.205+0000 7f7d82ffd640 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:40.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.206+0000 7f7d837fe640 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bed900 0x562b67bedce0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009ea0 tx=0x7f7d7c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c19a0251c5b7b722 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.206+0000 7f7d697fa640 1 -- 172.21.15.33:0/1010270430 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c004d20 con 0x562b67bed900 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.206+0000 7f7d697fa640 1 -- 172.21.15.33:0/1010270430 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d7c00ee60 con 0x562b67bed900 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.206+0000 7f7d6bfff640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b5360 0x7f7d6c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.206+0000 7f7d6bfff640 1 -- 172.21.15.33:0/1070732166 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d74106640 con 0x7f7d8410a9c0 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d697fa640 1 -- 172.21.15.33:0/1010270430 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c010830 con 0x562b67bed900 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d82ffd640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b5360 0x7f7d6c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1010270430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bed900 msgr2=0x562b67bedce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bed900 0x562b67bedce0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c009ea0 tx=0x7f7d7c009f90 comp rx=0 tx=0).stop 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1010270430 shutdown_connections 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1010270430 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bed900 0x562b67bedce0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1010270430 >> 172.21.15.33:0/1010270430 conn(0x562b67beb730 msgr2=0x562b67bec8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1010270430 shutdown_connections 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1010270430 wait complete. 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.207+0000 7f7d82ffd640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b5360 0x7f7d6c0b7820 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7d740c9390 tx=0x7f7d7402a420 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 -- start start 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 Processor -- start 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67be8dd0 0x562b67be8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67be67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67be8dd0 0x562b67be8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67be8dd0 0x562b67be8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56414/0 (socket says 172.21.15.33:56414) 2024-09-10T21:24:40.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d827fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67be67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d837fe640 1 -- 172.21.15.33:0/398981537 learned_addr learned my addr 172.21.15.33:0/398981537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b679ac700 con 0x562b67bed900 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b67ae6160 con 0x562b67be8dd0 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b67be9af0 con 0x562b67bee220 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d8b93e740 10 client.0 _ll_register_callbacks cb 0x562b67aff9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.208+0000 7f7d837fe640 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d837fe640 1 -- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 msgr2=0x562b67be6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d837fe640 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d837fe640 1 -- 172.21.15.33:0/398981537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 msgr2=0x562b67be67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d837fe640 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67be67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d837fe640 1 -- 172.21.15.33:0/398981537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b67b13c90 con 0x562b67be8dd0 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d837fe640 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67be8dd0 0x562b67be8350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c010490 tx=0x7f7d7c009ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7102e183a43d36ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d66ff5640 1 -- 172.21.15.33:0/398981537 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c01d040 con 0x562b67be8dd0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d7c00ee60 con 0x562b67be8dd0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.209+0000 7f7d66ff5640 1 -- 172.21.15.33:0/398981537 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d7c003b70 con 0x562b67be8dd0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d66ff5640 1 -- 172.21.15.33:0/398981537 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c003d70 con 0x562b67be8dd0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 10 client.28442 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x562b67c10e50 con 0x562b67be8dd0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d66ff5640 1 -- 172.21.15.33:0/398981537 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d7c0044e0 con 0x562b67be8dd0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d66ff5640 1 -- 172.21.15.33:0/398981537 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f7d7c031050 con 0x562b67be8dd0 2024-09-10T21:24:40.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 10 client.28442 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 10 client.28442 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 20 client.28442 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c12260 0x562b67c14650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 4 client.28442 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7d7c00ee60 con 0x562b67c12260 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.210+0000 7f7d827fc640 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c12260 0x562b67c14650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.211+0000 7f7d66ff5640 10 client.28442 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:24:40.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.211+0000 7f7d827fc640 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c12260 0x562b67c14650 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8855c5ef4d16db79 server_cookie=82aab87221bdf887 in_seq=0 out_seq=0 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d66ff5640 1 -- 172.21.15.33:0/398981537 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7d7c00ee60 con 0x562b67c12260 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d66ff5640 10 client.28442 handle_command_reply: tid=0 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 client.28442 shutdown 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67be8dd0 msgr2=0x562b67be8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67be8dd0 0x562b67be8350 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c010490 tx=0x7f7d7c009ae0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 shutdown_connections 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c12260 0x562b67c14650 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d66ff5640 0 client.28442 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bee220 0x562b67be6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67be67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/398981537 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67be8dd0 0x562b67be8350 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 >> 172.21.15.33:0/398981537 conn(0x562b67beb730 msgr2=0x562b67bf1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 shutdown_connections 2024-09-10T21:24:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 1 -- 172.21.15.33:0/398981537 wait complete. 2024-09-10T21:24:40.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.212+0000 7f7d8b93e740 20 client.28442 trim_cache size 0 max 16384 2024-09-10T21:24:40.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.226+0000 7f7d8b93e740 1 Processor -- start 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.226+0000 7f7d8b93e740 1 -- start start 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.226+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.226+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bed900 0x562b67be3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.226+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 0x562b67c19ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841ac740 con 0x562b67be8dd0 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841ae340 con 0x562b67bee220 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841aced0 con 0x562b67bed900 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bed900 0x562b67be3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bed900 0x562b67be3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39440/0 (socket says 172.21.15.33:39440) 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 -- 172.21.15.33:0/555556921 learned_addr learned my addr 172.21.15.33:0/555556921 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d837fe640 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 -- 172.21.15.33:0/555556921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 msgr2=0x562b67c19ce0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 0x562b67c19ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 -- 172.21.15.33:0/555556921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 msgr2=0x562b67be39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 -- 172.21.15.33:0/555556921 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b67be6560 con 0x562b67bed900 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d837fe640 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d827fc640 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bed900 0x562b67be3f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562b67c14c80 tx=0x7f7d78005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee8a022c1304eb89 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d68ff9640 1 -- 172.21.15.33:0/555556921 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7802a070 con 0x562b67bed900 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d68ff9640 1 -- 172.21.15.33:0/555556921 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d78003cc0 con 0x562b67bed900 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.227+0000 7f7d68ff9640 1 -- 172.21.15.33:0/555556921 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d780126b0 con 0x562b67bed900 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 -- 172.21.15.33:0/555556921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bed900 msgr2=0x562b67be3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bed900 0x562b67be3f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x562b67c14c80 tx=0x7f7d78005f90 comp rx=0 tx=0).stop 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 -- 172.21.15.33:0/555556921 shutdown_connections 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 0x562b67c19ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67bed900 0x562b67be3f00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/555556921 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67be8dd0 0x562b67be39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 -- 172.21.15.33:0/555556921 >> 172.21.15.33:0/555556921 conn(0x562b67beb730 msgr2=0x562b67bec7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 -- 172.21.15.33:0/555556921 shutdown_connections 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.228+0000 7f7d8b93e740 1 -- 172.21.15.33:0/555556921 wait complete. 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 -- start start 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 Processor -- start 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67be8dd0 0x562b67c0a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67c08740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 0x562b67c08c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841adc70 con 0x562b67bed900 2024-09-10T21:24:40.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841ad5a0 con 0x562b67bee220 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d841aa740 con 0x562b67be8dd0 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d8b93e740 10 client.0 _ll_register_callbacks cb 0x7f7d78009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67be8dd0 0x562b67c0a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d837fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67be8dd0 0x562b67c0a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39452/0 (socket says 172.21.15.33:39452) 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d837fe640 1 -- 172.21.15.33:0/1099208856 learned_addr learned my addr 172.21.15.33:0/1099208856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d827fc640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67c08740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.229+0000 7f7d837fe640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 0x562b67c08c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d837fe640 1 -- 172.21.15.33:0/1099208856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 msgr2=0x562b67c08c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d837fe640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 0x562b67c08c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d837fe640 1 -- 172.21.15.33:0/1099208856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 msgr2=0x562b67c08740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d837fe640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67c08740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d837fe640 1 -- 172.21.15.33:0/1099208856 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b67c092c0 con 0x562b67be8dd0 2024-09-10T21:24:40.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d827fc640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67c08740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d837fe640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67be8dd0 0x562b67c0a240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c010b30 tx=0x7f7d7c0021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6c20a689f6d445c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d66ff5640 1 -- 172.21.15.33:0/1099208856 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c009500 con 0x562b67be8dd0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d66ff5640 1 -- 172.21.15.33:0/1099208856 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d7c003970 con 0x562b67be8dd0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.230+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d78003cc0 con 0x562b67be8dd0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d66ff5640 1 -- 172.21.15.33:0/1099208856 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f7d7c00ee60 con 0x562b67be8dd0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 10 client.28327 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x562b67c7fd70 con 0x562b67be8dd0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d66ff5640 1 -- 172.21.15.33:0/1099208856 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d78003cc0 con 0x562b67be8dd0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d66ff5640 1 -- 172.21.15.33:0/1099208856 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f7d7c031070 con 0x562b67be8dd0 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 10 client.28327 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 10 client.28327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:24:40.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 20 client.28327 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c804f0 0x562b67c82910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 4 client.28327 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b67b5ddf0 con 0x562b67c804f0 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.231+0000 7f7d827fc640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c804f0 0x562b67c82910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d827fc640 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c804f0 0x562b67c82910 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d13d4526d63f39a server_cookie=55f5878d7d5837e in_seq=0 out_seq=0 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d66ff5640 10 client.28327 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d66ff5640 1 -- 172.21.15.33:0/1099208856 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b67b5ddf0 con 0x562b67c804f0 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d66ff5640 10 client.28327 handle_command_reply: tid=0 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d8b93e740 1 client.28327 shutdown 2024-09-10T21:24:40.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67be8dd0 msgr2=0x562b67c0a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67be8dd0 0x562b67c0a240 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c010b30 tx=0x7f7d7c0021d0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 shutdown_connections 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b67c804f0 0x562b67c82910 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b67bee220 0x562b67c08c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.232+0000 7f7d66ff5640 0 client.28327 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.233+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b67bed900 0x562b67c08740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.233+0000 7f7d8b93e740 1 --2- 172.21.15.33:0/1099208856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b67be8dd0 0x562b67c0a240 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.233+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 >> 172.21.15.33:0/1099208856 conn(0x562b67beb730 msgr2=0x562b67bf0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.233+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 shutdown_connections 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.233+0000 7f7d8b93e740 1 -- 172.21.15.33:0/1099208856 wait complete. 2024-09-10T21:24:40.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.233+0000 7f7d8b93e740 20 client.28327 trim_cache size 0 max 16384 2024-09-10T21:24:40.243 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:24:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b5360 msgr2=0x7f7d6c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b5360 0x7f7d6c0b7820 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7d740c9390 tx=0x7f7d7402a420 comp rx=0 tx=0).stop 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410a9c0 msgr2=0x7f7d8419b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410a9c0 0x7f7d8419b660 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7d74005ab0 tx=0x7f7d74003b90 comp rx=0 tx=0).stop 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 shutdown_connections 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b5360 0x7f7d6c0b7820 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d841a0f40 0x7f7d841a13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d8410b340 0x7f7d8419bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 --2- 172.21.15.33:0/1070732166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d8410a9c0 0x7f7d8419b660 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 >> 172.21.15.33:0/1070732166 conn(0x7f7d84100a20 msgr2=0x7f7d841022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 shutdown_connections 2024-09-10T21:24:40.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:40.234+0000 7f7d89569640 1 -- 172.21.15.33:0/1070732166 wait complete. 2024-09-10T21:24:41.055 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:24:41.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 353161 39.58 MB/sec execute 341 sec latency 210.036 ms 2024-09-10T21:24:41.424 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:41.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:41 smithi104 ceph-mon[35325]: pgmap v967: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-10T21:24:41.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:41.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:41 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:41.858 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:41 smithi033 ceph-mon[30860]: pgmap v967: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-10T21:24:41.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:41.859 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:41 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:41 smithi174 ceph-mon[35635]: pgmap v967: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 806 op/s 2024-09-10T21:24:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:41 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 -- 172.21.15.33:0/3455050437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a4101a90 msgr2=0x7f47a4101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/3455050437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a4101a90 0x7f47a4101e70 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f4794009900 tx=0x7f479402a410 comp rx=0 tx=0).stop 2024-09-10T21:24:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 -- 172.21.15.33:0/3455050437 shutdown_connections 2024-09-10T21:24:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/3455050437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4106ed0 0x7f47a4107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/3455050437 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a41023b0 0x7f47a4106880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/3455050437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a4101a90 0x7f47a4101e70 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 -- 172.21.15.33:0/3455050437 >> 172.21.15.33:0/3455050437 conn(0x7f47a40fd700 msgr2=0x7f47a40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 -- 172.21.15.33:0/3455050437 shutdown_connections 2024-09-10T21:24:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.967+0000 7f47ab8eb640 1 -- 172.21.15.33:0/3455050437 wait complete. 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.968+0000 7f47ab8eb640 1 Processor -- start 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.968+0000 7f47ab8eb640 1 -- start start 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47ab8eb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4101a90 0x7f47a419fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47ab8eb640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a41023b0 0x7f47a419ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47ab8eb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a4106ed0 0x7f47a41a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47ab8eb640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47a4119450 con 0x7f47a41023b0 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47ab8eb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41192d0 con 0x7f47a4106ed0 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47ab8eb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47a41195d0 con 0x7f47a4101a90 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a9687640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4101a90 0x7f47a419fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a8e86640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a41023b0 0x7f47a419ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a9687640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4101a90 0x7f47a419fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39476/0 (socket says 172.21.15.33:39476) 2024-09-10T21:24:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a9687640 1 -- 172.21.15.33:0/1346588358 learned_addr learned my addr 172.21.15.33:0/1346588358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a8e86640 1 -- 172.21.15.33:0/1346588358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4101a90 msgr2=0x7f47a419fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a8e86640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4101a90 0x7f47a419fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a8e86640 1 -- 172.21.15.33:0/1346588358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a4106ed0 msgr2=0x7f47a41a52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a9e88640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a4106ed0 0x7f47a41a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a8e86640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a4106ed0 0x7f47a41a52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a8e86640 1 -- 172.21.15.33:0/1346588358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a41a5ab0 con 0x7f47a41023b0 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a9687640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4101a90 0x7f47a419fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.969+0000 7f47a9e88640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a4106ed0 0x7f47a41a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.970+0000 7f47a8e86640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a41023b0 0x7f47a419ff40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f479800bdf0 tx=0x7f479800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.970+0000 7f47a27fc640 1 -- 172.21.15.33:0/1346588358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798019070 con 0x7f47a41023b0 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.970+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a41a5da0 con 0x7f47a41023b0 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.970+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41ac630 con 0x7f47a41023b0 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.970+0000 7f47a27fc640 1 -- 172.21.15.33:0/1346588358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479800ce10 con 0x7f47a41023b0 2024-09-10T21:24:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.970+0000 7f47a27fc640 1 -- 172.21.15.33:0/1346588358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798012860 con 0x7f47a41023b0 2024-09-10T21:24:41.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 Processor -- start 2024-09-10T21:24:41.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 -- start start 2024-09-10T21:24:41.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b750f0 0x556ab6b754d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b75a10 0x556ab6b81460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 0x556ab6b83d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ab6aebea0 con 0x556ab6b819a0 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556ab6adb2d0 con 0x556ab6b750f0 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47adcc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ab6adf0e0 con 0x556ab6b75a10 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47a9e88640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b750f0 0x556ab6b754d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47a9e88640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b750f0 0x556ab6b754d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56452/0 (socket says 172.21.15.33:56452) 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.976+0000 7f47a9e88640 1 -- 172.21.15.33:0/579756585 learned_addr learned my addr 172.21.15.33:0/579756585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f47a9687640 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b75a10 0x556ab6b81460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f47a9e88640 1 -- 172.21.15.33:0/579756585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b75a10 msgr2=0x556ab6b81460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:41.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f47a9e88640 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b75a10 0x556ab6b81460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f47a9e88640 1 -- 172.21.15.33:0/579756585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 msgr2=0x556ab6b83d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f47a9e88640 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 0x556ab6b83d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f47a9e88640 1 -- 172.21.15.33:0/579756585 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab6839650 con 0x556ab6b750f0 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f47a9e88640 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b750f0 0x556ab6b754d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f479c00a990 tx=0x7f479c00ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6310816c15015b14 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.977+0000 7f478b7fe640 1 -- 172.21.15.33:0/579756585 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c015070 con 0x556ab6b750f0 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.978+0000 7f478b7fe640 1 -- 172.21.15.33:0/579756585 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479c00cbe0 con 0x556ab6b750f0 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.978+0000 7f478b7fe640 1 -- 172.21.15.33:0/579756585 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c0185a0 con 0x556ab6b750f0 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47adcc0740 1 -- 172.21.15.33:0/579756585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b750f0 msgr2=0x556ab6b754d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47adcc0740 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b750f0 0x556ab6b754d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f479c00a990 tx=0x7f479c00ae50 comp rx=0 tx=0).stop 2024-09-10T21:24:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47adcc0740 1 -- 172.21.15.33:0/579756585 shutdown_connections 2024-09-10T21:24:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47adcc0740 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 0x556ab6b83d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47adcc0740 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b75a10 0x556ab6b81460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47adcc0740 1 --2- 172.21.15.33:0/579756585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b750f0 0x556ab6b754d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47adcc0740 1 -- 172.21.15.33:0/579756585 >> 172.21.15.33:0/579756585 conn(0x556ab6b72f20 msgr2=0x556ab6b740c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.979+0000 7f47a27fc640 1 -- 172.21.15.33:0/1346588358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4798012a00 con 0x7f47a41023b0 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.980+0000 7f47adcc0740 1 -- 172.21.15.33:0/579756585 shutdown_connections 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.981+0000 7f47a27fc640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f478c0b52f0 0x7f478c0b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.981+0000 7f47a27fc640 1 -- 172.21.15.33:0/1346588358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f47980e5380 con 0x7f47a41023b0 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.981+0000 7f47adcc0740 1 -- 172.21.15.33:0/579756585 wait complete. 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.981+0000 7f47a9687640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f478c0b52f0 0x7f478c0b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.982+0000 7f47adcc0740 1 -- start start 2024-09-10T21:24:41.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.982+0000 7f47adcc0740 1 Processor -- start 2024-09-10T21:24:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.982+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b812a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.982+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b7f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47adcc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556ab693c700 con 0x556ab6b7ec80 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47adcc0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556ab6a76160 con 0x556ab6b819a0 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47adcc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ab6b84510 con 0x556ab6b750f0 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47adcc0740 10 client.0 _ll_register_callbacks cb 0x556ab6a8f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47a8e86640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47a8e86640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56454/0 (socket says 172.21.15.33:56454) 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.983+0000 7f47a8e86640 1 -- 172.21.15.33:0/3538116642 learned_addr learned my addr 172.21.15.33:0/3538116642 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f47a8e86640 1 -- 172.21.15.33:0/3538116642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 msgr2=0x556ab6b812a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f47a8e86640 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b812a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f47a8e86640 1 -- 172.21.15.33:0/3538116642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 msgr2=0x556ab6b7f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f47a8e86640 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b7f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f47a8e86640 1 -- 172.21.15.33:0/3538116642 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab6aa3c90 con 0x556ab6b819a0 2024-09-10T21:24:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f47a8e86640 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7e6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47980a8180 tx=0x7f47980ab3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61692b1079714ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f4788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f4788ff9640 1 -- 172.21.15.33:0/3538116642 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980b0380 con 0x556ab6b819a0 2024-09-10T21:24:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.984+0000 7f4788ff9640 1 -- 172.21.15.33:0/3538116642 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f47980b4c00 con 0x556ab6b819a0 2024-09-10T21:24:41.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.985+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479c00cbe0 con 0x556ab6b819a0 2024-09-10T21:24:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.985+0000 7f4788ff9640 1 -- 172.21.15.33:0/3538116642 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f47980b4e80 con 0x556ab6b819a0 2024-09-10T21:24:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.985+0000 7f47adcc0740 10 client.28451 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.985+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x556ab6b90360 con 0x556ab6b819a0 2024-09-10T21:24:41.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.985+0000 7f4788ff9640 1 -- 172.21.15.33:0/3538116642 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f479c00cbe0 con 0x556ab6b819a0 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f4788ff9640 1 -- 172.21.15.33:0/3538116642 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f47980a7500 con 0x556ab6b819a0 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f47adcc0740 10 client.28451 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f47adcc0740 10 client.28451 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f47adcc0740 20 client.28451 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f47adcc0740 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6b91890 0x556ab6b93cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f47adcc0740 4 client.28451 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ab6aeddf0 con 0x556ab6b91890 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.986+0000 7f47a9687640 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6b91890 0x556ab6b93cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.987+0000 7f47a9687640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f478c0b52f0 0x7f478c0b77b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4794005ec0 tx=0x7f47940058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.987+0000 7f47a9687640 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6b91890 0x556ab6b93cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=760b2487d2ebe8a4 server_cookie=414c30eedd117a04 in_seq=0 out_seq=0 2024-09-10T21:24:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.987+0000 7f4788ff9640 10 client.28451 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:24:41.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.989+0000 7f4788ff9640 1 -- 172.21.15.33:0/3538116642 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ab6aeddf0 con 0x556ab6b91890 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.989+0000 7f4788ff9640 10 client.28451 handle_command_reply: tid=0 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.989+0000 7f47adcc0740 1 client.28451 shutdown 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.989+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 msgr2=0x556ab6b7e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.989+0000 7f47adcc0740 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7e6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47980a8180 tx=0x7f47980ab3e0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.990+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 shutdown_connections 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.990+0000 7f47adcc0740 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b7f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.990+0000 7f47adcc0740 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7e6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.990+0000 7f47adcc0740 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6b91890 0x556ab6b93cf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.990+0000 7f47adcc0740 1 --2- 172.21.15.33:0/3538116642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b812a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.990+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 >> 172.21.15.33:0/3538116642 conn(0x556ab6b72f20 msgr2=0x556ab6b78a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:41.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.990+0000 7f4788ff9640 0 client.28451 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:24:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.991+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 shutdown_connections 2024-09-10T21:24:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.991+0000 7f47adcc0740 1 -- 172.21.15.33:0/3538116642 wait complete. 2024-09-10T21:24:41.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:41.991+0000 7f47adcc0740 20 client.28451 trim_cache size 0 max 16384 2024-09-10T21:24:42.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.016+0000 7f47adcc0740 1 Processor -- start 2024-09-10T21:24:42.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.016+0000 7f47adcc0740 1 -- start start 2024-09-10T21:24:42.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b798b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:42.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b79df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:42.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:42.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47adcc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f47900304e0 con 0x556ab6b7ec80 2024-09-10T21:24:42.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47adcc0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f47900320e0 con 0x556ab6b819a0 2024-09-10T21:24:42.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47adcc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4790030c70 con 0x556ab6b750f0 2024-09-10T21:24:42.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47a9687640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b798b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:42.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47a9687640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b798b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39496/0 (socket says 172.21.15.33:39496) 2024-09-10T21:24:42.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47a9687640 1 -- 172.21.15.33:0/4145417585 learned_addr learned my addr 172.21.15.33:0/4145417585 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:42.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47a8e86640 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b79df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:42.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.017+0000 7f47a9e88640 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:42.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f47a8e86640 1 -- 172.21.15.33:0/4145417585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 msgr2=0x556ab6b798b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f47a8e86640 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b798b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f47a8e86640 1 -- 172.21.15.33:0/4145417585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 msgr2=0x556ab6b7cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f47a8e86640 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f47a8e86640 1 -- 172.21.15.33:0/4145417585 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ab6b7f450 con 0x556ab6b7ec80 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f47a8e86640 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b79df0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f47980ab8b0 tx=0x7f47980123e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb4d699e01aace4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f478affd640 1 -- 172.21.15.33:0/4145417585 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980c6080 con 0x556ab6b7ec80 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f478affd640 1 -- 172.21.15.33:0/4145417585 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47980cee50 con 0x556ab6b7ec80 2024-09-10T21:24:42.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.018+0000 7f478affd640 1 -- 172.21.15.33:0/4145417585 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47980b5090 con 0x556ab6b7ec80 2024-09-10T21:24:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.019+0000 7f47adcc0740 1 -- 172.21.15.33:0/4145417585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 msgr2=0x556ab6b79df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.019+0000 7f47adcc0740 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b79df0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f47980ab8b0 tx=0x7f47980123e0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.019+0000 7f47adcc0740 1 -- 172.21.15.33:0/4145417585 shutdown_connections 2024-09-10T21:24:42.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.019+0000 7f47adcc0740 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b819a0 0x556ab6b7cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.019+0000 7f47adcc0740 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b7ec80 0x556ab6b79df0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.019+0000 7f47adcc0740 1 --2- 172.21.15.33:0/4145417585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b798b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.019+0000 7f47adcc0740 1 -- 172.21.15.33:0/4145417585 >> 172.21.15.33:0/4145417585 conn(0x556ab6b72f20 msgr2=0x556ab6b6df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.020+0000 7f47adcc0740 1 -- 172.21.15.33:0/4145417585 shutdown_connections 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.020+0000 7f47adcc0740 1 -- 172.21.15.33:0/4145417585 wait complete. 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.020+0000 7f47adcc0740 1 -- start start 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.020+0000 7f47adcc0740 1 Processor -- start 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b85210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b7ec80 0x556ab6b85790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47adcc0740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 0x556ab6b89700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47adcc0740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4790031a10 con 0x556ab6b819a0 2024-09-10T21:24:42.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47adcc0740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4790031340 con 0x556ab6b7ec80 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47adcc0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f479002e4a0 con 0x556ab6b750f0 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47adcc0740 10 client.0 _ll_register_callbacks cb 0x7f47980097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47a9687640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b85210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47a9687640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b85210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39506/0 (socket says 172.21.15.33:39506) 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47a9687640 1 -- 172.21.15.33:0/1412280356 learned_addr learned my addr 172.21.15.33:0/1412280356 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.021+0000 7f47a9e88640 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 0x556ab6b89700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f47a9687640 1 -- 172.21.15.33:0/1412280356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b7ec80 msgr2=0x556ab6b85790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f47a9687640 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b7ec80 0x556ab6b85790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f47a9687640 1 -- 172.21.15.33:0/1412280356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 msgr2=0x556ab6b89700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f47a9687640 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 0x556ab6b89700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f47a9687640 1 -- 172.21.15.33:0/1412280356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ab6b89e10 con 0x556ab6b750f0 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f47a9687640 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b85210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f47940043f0 tx=0x7f479403e070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37473f08efda8f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:42.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f4788ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:24:42.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f4788ff9640 1 -- 172.21.15.33:0/1412280356 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4794035040 con 0x556ab6b750f0 2024-09-10T21:24:42.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f4788ff9640 1 -- 172.21.15.33:0/1412280356 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4794005300 con 0x556ab6b750f0 2024-09-10T21:24:42.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.022+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47980cee50 con 0x556ab6b750f0 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.023+0000 7f4788ff9640 1 -- 172.21.15.33:0/1412280356 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f4794005500 con 0x556ab6b750f0 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.023+0000 7f47adcc0740 10 client.28339 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.023+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x556ab6bff230 con 0x556ab6b750f0 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f4788ff9640 1 -- 172.21.15.33:0/1412280356 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4794040070 con 0x556ab6b750f0 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f4788ff9640 1 -- 172.21.15.33:0/1412280356 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f4794003830 con 0x556ab6b750f0 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f47adcc0740 10 client.28339 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f47adcc0740 10 client.28339 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f47adcc0740 20 client.28339 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:42.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f47adcc0740 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6bff990 0x556ab6c01db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:42.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f47adcc0740 4 client.28339 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:24:42.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f47980cee50 con 0x556ab6bff990 2024-09-10T21:24:42.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.025+0000 7f47a8e86640 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6bff990 0x556ab6c01db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:42.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.028+0000 7f47a8e86640 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6bff990 0x556ab6c01db0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a35474e36e4d943 server_cookie=f6339a0892c0aea2 in_seq=0 out_seq=0 2024-09-10T21:24:42.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.028+0000 7f4788ff9640 10 client.28339 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:24:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.029+0000 7f4788ff9640 1 -- 172.21.15.33:0/1412280356 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f47980cee50 con 0x556ab6bff990 2024-09-10T21:24:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.029+0000 7f4788ff9640 10 client.28339 handle_command_reply: tid=0 2024-09-10T21:24:42.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.029+0000 7f47adcc0740 1 client.28339 shutdown 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 msgr2=0x556ab6b85210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b85210 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f47940043f0 tx=0x7f479403e070 comp rx=0 tx=0).stop 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 shutdown_connections 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556ab6b819a0 0x556ab6b89700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556ab6b7ec80 0x556ab6b85790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556ab6bff990 0x556ab6c01db0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 --2- 172.21.15.33:0/1412280356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ab6b750f0 0x556ab6b85210 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f47940043f0 tx=0x7f479403e070 comp rx=0 tx=0).stop 2024-09-10T21:24:42.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 >> 172.21.15.33:0/1412280356 conn(0x556ab6b72f20 msgr2=0x556ab6b798b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f4788ff9640 0 client.28339 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:24:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 shutdown_connections 2024-09-10T21:24:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 1 -- 172.21.15.33:0/1412280356 wait complete. 2024-09-10T21:24:42.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.030+0000 7f47adcc0740 20 client.28339 trim_cache size 0 max 16384 2024-09-10T21:24:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:24:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f478c0b52f0 msgr2=0x7f478c0b77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f478c0b52f0 0x7f478c0b77b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f4794005ec0 tx=0x7f47940058d0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a41023b0 msgr2=0x7f47a419ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a41023b0 0x7f47a419ff40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f479800bdf0 tx=0x7f479800bee0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 shutdown_connections 2024-09-10T21:24:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f478c0b52f0 0x7f478c0b77b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f47a4106ed0 0x7f47a41a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a41023b0 0x7f47a419ff40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 --2- 172.21.15.33:0/1346588358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a4101a90 0x7f47a419fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.031+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 >> 172.21.15.33:0/1346588358 conn(0x7f47a40fd700 msgr2=0x7f47a4104850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.032+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 shutdown_connections 2024-09-10T21:24:42.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:42.032+0000 7f47ab8eb640 1 -- 172.21.15.33:0/1346588358 wait complete. 2024-09-10T21:24:42.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 353886 39.54 MB/sec execute 342 sec latency 71.423 ms 2024-09-10T21:24:42.882 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:24:43.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 354646 39.58 MB/sec execute 343 sec latency 198.288 ms 2024-09-10T21:24:43.272 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:43.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:43 smithi033 ceph-mon[30860]: pgmap v968: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.8 MiB/s wr, 630 op/s 2024-09-10T21:24:43.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:43 smithi104 ceph-mon[35325]: pgmap v968: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.8 MiB/s wr, 630 op/s 2024-09-10T21:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.809+0000 7f40758e7640 1 -- 172.21.15.33:0/251604147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a150 msgr2=0x7f40701111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:43.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.809+0000 7f40758e7640 1 --2- 172.21.15.33:0/251604147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a150 0x7f40701111b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f406801a910 tx=0x7f406803b4c0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.811+0000 7f40758e7640 1 -- 172.21.15.33:0/251604147 shutdown_connections 2024-09-10T21:24:43.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.811+0000 7f40758e7640 1 --2- 172.21.15.33:0/251604147 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f407011c0a0 0x7f407011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.811+0000 7f40758e7640 1 --2- 172.21.15.33:0/251604147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f407010a530 0x7f407010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.811+0000 7f40758e7640 1 --2- 172.21.15.33:0/251604147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a150 0x7f40701111b0 secure :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f406801a910 tx=0x7f406803b4c0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.811+0000 7f40758e7640 1 -- 172.21.15.33:0/251604147 >> 172.21.15.33:0/251604147 conn(0x7f407006dfc0 msgr2=0x7f407006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.812+0000 7f40758e7640 1 -- 172.21.15.33:0/251604147 shutdown_connections 2024-09-10T21:24:43.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.812+0000 7f40758e7640 1 -- 172.21.15.33:0/251604147 wait complete. 2024-09-10T21:24:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 Processor -- start 2024-09-10T21:24:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 -- start start 2024-09-10T21:24:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a530 0x7f407011b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f407011c0a0 0x7f40701179b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4070117ef0 0x7f40701183a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4070114100 con 0x7f407010a530 2024-09-10T21:24:43.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4070113f80 con 0x7f407011c0a0 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.813+0000 7f40758e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4070114280 con 0x7f4070117ef0 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.814+0000 7f406effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a530 0x7f407011b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.814+0000 7f406effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a530 0x7f407011b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47508/0 (socket says 172.21.15.33:47508) 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.814+0000 7f406effd640 1 -- 172.21.15.33:0/4254308984 learned_addr learned my addr 172.21.15.33:0/4254308984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.817+0000 7f406f7fe640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4070117ef0 0x7f40701183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.817+0000 7f406e7fc640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f407011c0a0 0x7f40701179b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.818+0000 7f406e7fc640 1 -- 172.21.15.33:0/4254308984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4070117ef0 msgr2=0x7f40701183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.818+0000 7f406e7fc640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4070117ef0 0x7f40701183a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.818+0000 7f406e7fc640 1 -- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a530 msgr2=0x7f407011b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:43.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.818+0000 7f406e7fc640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a530 0x7f407011b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.818+0000 7f406e7fc640 1 -- 172.21.15.33:0/4254308984 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40701ac100 con 0x7f407011c0a0 2024-09-10T21:24:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.819+0000 7f406e7fc640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f407011c0a0 0x7f40701179b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f405800b3f0 tx=0x7f405800b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.820+0000 7f40748e5640 1 -- 172.21.15.33:0/4254308984 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058015020 con 0x7f407011c0a0 2024-09-10T21:24:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.820+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4070075380 con 0x7f407011c0a0 2024-09-10T21:24:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.820+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4070075990 con 0x7f407011c0a0 2024-09-10T21:24:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.820+0000 7f40748e5640 1 -- 172.21.15.33:0/4254308984 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4058011ce0 con 0x7f407011c0a0 2024-09-10T21:24:43.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.820+0000 7f40748e5640 1 -- 172.21.15.33:0/4254308984 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058010d80 con 0x7f407011c0a0 2024-09-10T21:24:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.827+0000 7f40748e5640 1 -- 172.21.15.33:0/4254308984 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f405801a400 con 0x7f407011c0a0 2024-09-10T21:24:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.828+0000 7f40748e5640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f405c0b5360 0x7f405c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 Processor -- start 2024-09-10T21:24:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f406effd640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f405c0b5360 0x7f405c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 -- start start 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77d0f0 0x56081d77d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77da10 0x56081d789460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56081d6f3ea0 con 0x56081d77d0f0 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56081d6e32d0 con 0x56081d77da10 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.829+0000 7f4077cbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56081d6e70e0 con 0x56081d7899a0 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.830+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.830+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39526/0 (socket says 172.21.15.33:39526) 2024-09-10T21:24:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.830+0000 7f406f7fe640 1 -- 172.21.15.33:0/103101665 learned_addr learned my addr 172.21.15.33:0/103101665 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:43.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.830+0000 7f406e7fc640 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77da10 0x56081d789460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.831+0000 7f406f7fe640 1 -- 172.21.15.33:0/103101665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77da10 msgr2=0x56081d789460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.831+0000 7f406f7fe640 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77da10 0x56081d789460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.831+0000 7f406f7fe640 1 -- 172.21.15.33:0/103101665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77d0f0 msgr2=0x56081d77d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.831+0000 7f406f7fe640 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77d0f0 0x56081d77d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.831+0000 7f406f7fe640 1 -- 172.21.15.33:0/103101665 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56081d441650 con 0x56081d7899a0 2024-09-10T21:24:43.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.834+0000 7f40748e5640 1 -- 172.21.15.33:0/4254308984 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f40580e6500 con 0x7f407011c0a0 2024-09-10T21:24:43.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.835+0000 7f406f7fe640 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78bd90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4060009e90 tx=0x7f406000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4b34af2fb30c9c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:43.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.835+0000 7f406effd640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f405c0b5360 0x7f405c0b7820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4068005940 tx=0x7f40680058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.840+0000 7f4065ffb640 1 -- 172.21.15.33:0/103101665 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060004d20 con 0x56081d7899a0 2024-09-10T21:24:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.840+0000 7f4065ffb640 1 -- 172.21.15.33:0/103101665 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40600054d0 con 0x56081d7899a0 2024-09-10T21:24:43.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.840+0000 7f4065ffb640 1 -- 172.21.15.33:0/103101665 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600118d0 con 0x56081d7899a0 2024-09-10T21:24:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.841+0000 7f4077cbc740 1 -- 172.21.15.33:0/103101665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 msgr2=0x56081d78bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:43.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.841+0000 7f4077cbc740 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78bd90 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4060009e90 tx=0x7f406000f730 comp rx=0 tx=0).stop 2024-09-10T21:24:43.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.843+0000 7f4077cbc740 1 -- 172.21.15.33:0/103101665 shutdown_connections 2024-09-10T21:24:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.843+0000 7f4077cbc740 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78bd90 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.843+0000 7f4077cbc740 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77da10 0x56081d789460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.843+0000 7f4077cbc740 1 --2- 172.21.15.33:0/103101665 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77d0f0 0x56081d77d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.843+0000 7f4077cbc740 1 -- 172.21.15.33:0/103101665 >> 172.21.15.33:0/103101665 conn(0x56081d77af20 msgr2=0x56081d77c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:43.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.846+0000 7f4077cbc740 1 -- 172.21.15.33:0/103101665 shutdown_connections 2024-09-10T21:24:43.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.846+0000 7f4077cbc740 1 -- 172.21.15.33:0/103101665 wait complete. 2024-09-10T21:24:43.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.847+0000 7f4077cbc740 1 -- start start 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.847+0000 7f4077cbc740 1 Processor -- start 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.848+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d77d0f0 0x56081d7892b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.848+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77da10 0x56081d786700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.848+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d7899a0 0x56081d786d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.848+0000 7f4077cbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56081d544700 con 0x56081d77da10 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.848+0000 7f4077cbc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56081d67e160 con 0x56081d7899a0 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.848+0000 7f4077cbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56081d78c620 con 0x56081d77d0f0 2024-09-10T21:24:43.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.848+0000 7f4077cbc740 10 client.0 _ll_register_callbacks cb 0x56081d6979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.849+0000 7f406e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77da10 0x56081d786700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.849+0000 7f406e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77da10 0x56081d786700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47522/0 (socket says 172.21.15.33:47522) 2024-09-10T21:24:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.849+0000 7f406e7fc640 1 -- 172.21.15.33:0/3091817687 learned_addr learned my addr 172.21.15.33:0/3091817687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:43.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.850+0000 7f406effd640 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d77d0f0 0x56081d7892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.850+0000 7f406e7fc640 1 -- 172.21.15.33:0/3091817687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d77d0f0 msgr2=0x56081d7892b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.850+0000 7f406e7fc640 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d77d0f0 0x56081d7892b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.850+0000 7f406e7fc640 1 -- 172.21.15.33:0/3091817687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d7899a0 msgr2=0x56081d786d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.850+0000 7f406e7fc640 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d7899a0 0x56081d786d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.850+0000 7f406e7fc640 1 -- 172.21.15.33:0/3091817687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56081d6abc90 con 0x56081d77da10 2024-09-10T21:24:43.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.850+0000 7f406e7fc640 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77da10 0x56081d786700 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f40580ac9c0 tx=0x7f4058010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646da883a4efafc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.851+0000 7f404effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:24:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.853+0000 7f404effd640 1 -- 172.21.15.33:0/3091817687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580c7ce0 con 0x56081d77da10 2024-09-10T21:24:43.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.853+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40600054d0 con 0x56081d77da10 2024-09-10T21:24:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.853+0000 7f404effd640 1 -- 172.21.15.33:0/3091817687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f40580e60e0 con 0x56081d77da10 2024-09-10T21:24:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.855+0000 7f404effd640 1 -- 172.21.15.33:0/3091817687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f40580a8430 con 0x56081d77da10 2024-09-10T21:24:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.855+0000 7f4077cbc740 10 client.19101 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:43.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.855+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56081d797fe0 con 0x56081d77da10 2024-09-10T21:24:43.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.860+0000 7f404effd640 1 -- 172.21.15.33:0/3091817687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40600054d0 con 0x56081d77da10 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.860+0000 7f404effd640 1 -- 172.21.15.33:0/3091817687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f4058010380 con 0x56081d77da10 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.861+0000 7f4077cbc740 10 client.19101 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.861+0000 7f4077cbc740 10 client.19101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.861+0000 7f4077cbc740 20 client.19101 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.861+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d799440 0x56081d79b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.861+0000 7f4077cbc740 4 client.19101 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.861+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56081d6f5df0 con 0x56081d799440 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.861+0000 7f406effd640 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d799440 0x56081d79b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.862+0000 7f406effd640 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d799440 0x56081d79b890 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76ba5e983049239d server_cookie=a82b764bda1b8e03 in_seq=0 out_seq=0 2024-09-10T21:24:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.864+0000 7f404effd640 10 client.19101 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:24:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.865+0000 7f404effd640 1 -- 172.21.15.33:0/3091817687 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56081d6f5df0 con 0x56081d799440 2024-09-10T21:24:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.865+0000 7f404effd640 10 client.19101 handle_command_reply: tid=0 2024-09-10T21:24:43.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.865+0000 7f4077cbc740 1 client.19101 shutdown 2024-09-10T21:24:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77da10 msgr2=0x56081d786700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77da10 0x56081d786700 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f40580ac9c0 tx=0x7f4058010000 comp rx=0 tx=0).stop 2024-09-10T21:24:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 shutdown_connections 2024-09-10T21:24:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d7899a0 0x56081d786d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d799440 0x56081d79b890 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d77da10 0x56081d786700 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3091817687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d77d0f0 0x56081d7892b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.867+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 >> 172.21.15.33:0/3091817687 conn(0x56081d77af20 msgr2=0x56081d780a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.868+0000 7f404effd640 0 client.19101 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:24:43.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.868+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 shutdown_connections 2024-09-10T21:24:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.870+0000 7f4077cbc740 1 -- 172.21.15.33:0/3091817687 wait complete. 2024-09-10T21:24:43.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.873+0000 7f4077cbc740 20 client.19101 trim_cache size 0 max 16384 2024-09-10T21:24:43.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:43 smithi174 ceph-mon[35635]: pgmap v968: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.8 MiB/s wr, 630 op/s 2024-09-10T21:24:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.902+0000 7f4077cbc740 1 Processor -- start 2024-09-10T21:24:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.902+0000 7f4077cbc740 1 -- start start 2024-09-10T21:24:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.902+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77d0f0 0x56081d7958b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.902+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.902+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d785690 0x56081d78b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77d0f0 0x56081d7958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77d0f0 0x56081d7958b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56506/0 (socket says 172.21.15.33:56506) 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39550/0 (socket says 172.21.15.33:39550) 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d785690 0x56081d78b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d785690 0x56081d78b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47538/0 (socket says 172.21.15.33:47538) 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 -- 172.21.15.33:0/3240756458 learned_addr learned my addr 172.21.15.33:0/3240756458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f405000e5e0 con 0x56081d785690 2024-09-10T21:24:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f40500101e0 con 0x56081d77d0f0 2024-09-10T21:24:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f405000ed70 con 0x56081d7899a0 2024-09-10T21:24:43.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 -- 172.21.15.33:0/3240756458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 msgr2=0x56081d78ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 --2- 172.21.15.33:0/3240756458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 -- 172.21.15.33:0/3240756458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d785690 msgr2=0x56081d78b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 --2- 172.21.15.33:0/3240756458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d785690 0x56081d78b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.903+0000 7f406effd640 1 -- 172.21.15.33:0/3240756458 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56081d784c00 con 0x56081d77d0f0 2024-09-10T21:24:43.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.911+0000 7f406effd640 1 --2- 172.21.15.33:0/3240756458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77d0f0 0x56081d7958b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f406801a6d0 tx=0x7f4068071000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bcb0773ad81442a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.918+0000 7f40657fa640 1 -- 172.21.15.33:0/3240756458 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068088070 con 0x56081d77d0f0 2024-09-10T21:24:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.918+0000 7f40657fa640 1 -- 172.21.15.33:0/3240756458 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4068004e20 con 0x56081d77d0f0 2024-09-10T21:24:43.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.918+0000 7f40657fa640 1 -- 172.21.15.33:0/3240756458 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068046050 con 0x56081d77d0f0 2024-09-10T21:24:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.921+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77d0f0 msgr2=0x56081d7958b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.921+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3240756458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77d0f0 0x56081d7958b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f406801a6d0 tx=0x7f4068071000 comp rx=0 tx=0).stop 2024-09-10T21:24:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.921+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 shutdown_connections 2024-09-10T21:24:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.921+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3240756458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d785690 0x56081d78b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.921+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3240756458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d78ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.921+0000 7f4077cbc740 1 --2- 172.21.15.33:0/3240756458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d77d0f0 0x56081d7958b0 secure :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f406801a6d0 tx=0x7f4068071000 comp rx=0 tx=0).stop 2024-09-10T21:24:43.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.921+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 >> 172.21.15.33:0/3240756458 conn(0x56081d77af20 msgr2=0x56081d784f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:43.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.930+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 shutdown_connections 2024-09-10T21:24:43.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.934+0000 7f4077cbc740 1 -- 172.21.15.33:0/3240756458 wait complete. 2024-09-10T21:24:43.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.934+0000 7f4077cbc740 1 -- start start 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.934+0000 7f4077cbc740 1 Processor -- start 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.935+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d785690 0x56081d791cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.935+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d792230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.935+0000 7f4077cbc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d78c970 0x56081d78cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.935+0000 7f4077cbc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f405000fb10 con 0x56081d78c970 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.935+0000 7f4077cbc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f405000f440 con 0x56081d785690 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.935+0000 7f4077cbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f405000c5a0 con 0x56081d7899a0 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.935+0000 7f4077cbc740 10 client.0 _ll_register_callbacks cb 0x7f4058010f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:43.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.937+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d78c970 0x56081d78cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.937+0000 7f406f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d78c970 0x56081d78cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47550/0 (socket says 172.21.15.33:47550) 2024-09-10T21:24:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.937+0000 7f406f7fe640 1 -- 172.21.15.33:0/2724394037 learned_addr learned my addr 172.21.15.33:0/2724394037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.937+0000 7f406effd640 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d785690 0x56081d791cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.937+0000 7f406e7fc640 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d792230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:43.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.943+0000 7f406effd640 1 -- 172.21.15.33:0/2724394037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 msgr2=0x56081d792230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.943+0000 7f406effd640 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d792230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.943+0000 7f406effd640 1 -- 172.21.15.33:0/2724394037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d78c970 msgr2=0x56081d78cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.943+0000 7f406effd640 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d78c970 0x56081d78cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:43.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.943+0000 7f406effd640 1 -- 172.21.15.33:0/2724394037 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56081d78d620 con 0x56081d785690 2024-09-10T21:24:43.949 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.947+0000 7f406effd640 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d785690 0x56081d791cb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4068078000 tx=0x7f40680405a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0cd12bc6be05079 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:43.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.947+0000 7f404effd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:43.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.947+0000 7f404effd640 1 -- 172.21.15.33:0/2724394037 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068088070 con 0x56081d785690 2024-09-10T21:24:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.947+0000 7f404effd640 1 -- 172.21.15.33:0/2724394037 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4068004900 con 0x56081d785690 2024-09-10T21:24:43.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.950+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4068004e20 con 0x56081d785690 2024-09-10T21:24:43.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.953+0000 7f404effd640 1 -- 172.21.15.33:0/2724394037 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f4068046070 con 0x56081d785690 2024-09-10T21:24:43.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.954+0000 7f4077cbc740 10 client.28469 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:43.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.954+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56081d8073e0 con 0x56081d785690 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f404effd640 1 -- 172.21.15.33:0/2724394037 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4068004e20 con 0x56081d785690 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f404effd640 1 -- 172.21.15.33:0/2724394037 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f406806c040 con 0x56081d785690 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f4077cbc740 10 client.28469 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f4077cbc740 10 client.28469 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f4077cbc740 20 client.28469 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f4077cbc740 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d807940 0x56081d809d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f4077cbc740 4 client.28469 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:24:43.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:43.956+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56081d79bc20 con 0x56081d807940 2024-09-10T21:24:44.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.038+0000 7f406e7fc640 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d807940 0x56081d809d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.040+0000 7f404effd640 10 client.28469 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:24:44.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.040+0000 7f406e7fc640 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d807940 0x56081d809d60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2a0e2f8dea70177c server_cookie=a23b1d55a51beed0 in_seq=0 out_seq=0 2024-09-10T21:24:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f404effd640 1 -- 172.21.15.33:0/2724394037 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56081d79bc20 con 0x56081d807940 2024-09-10T21:24:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f404effd640 10 client.28469 handle_command_reply: tid=0 2024-09-10T21:24:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 client.28469 shutdown 2024-09-10T21:24:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d785690 msgr2=0x56081d791cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d785690 0x56081d791cb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4068078000 tx=0x7f40680405a0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 shutdown_connections 2024-09-10T21:24:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x56081d807940 0x56081d809d60 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56081d78c970 0x56081d78cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56081d7899a0 0x56081d792230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f404effd640 0 client.28469 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 --2- 172.21.15.33:0/2724394037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56081d785690 0x56081d791cb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.046+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 >> 172.21.15.33:0/2724394037 conn(0x56081d77af20 msgr2=0x56081d787d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.047+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 shutdown_connections 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.047+0000 7f4077cbc740 1 -- 172.21.15.33:0/2724394037 wait complete. 2024-09-10T21:24:44.050 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.047+0000 7f4077cbc740 20 client.28469 trim_cache size 0 max 16384 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f405c0b5360 msgr2=0x7f405c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f405c0b5360 0x7f405c0b7820 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4068005940 tx=0x7f40680058d0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f407011c0a0 msgr2=0x7f40701179b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f407011c0a0 0x7f40701179b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f405800b3f0 tx=0x7f405800b8b0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 shutdown_connections 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f405c0b5360 0x7f405c0b7820 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4070117ef0 0x7f40701183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f407011c0a0 0x7f40701179b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 --2- 172.21.15.33:0/4254308984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f407010a530 0x7f407011b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 >> 172.21.15.33:0/4254308984 conn(0x7f407006dfc0 msgr2=0x7f4070072c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 shutdown_connections 2024-09-10T21:24:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:44.048+0000 7f40758e7640 1 -- 172.21.15.33:0/4254308984 wait complete. 2024-09-10T21:24:44.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 355131 39.54 MB/sec execute 344 sec latency 299.128 ms 2024-09-10T21:24:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:44 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:44 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:45.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 355484 39.47 MB/sec execute 345 sec latency 465.401 ms 2024-09-10T21:24:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:44 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:44 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:45.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:44 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:45.176 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:44 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:45.600 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:24:45.847 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:45 smithi033 ceph-mon[30860]: pgmap v969: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:24:45.895 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:46.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 355959 39.45 MB/sec execute 346 sec latency 210.395 ms 2024-09-10T21:24:46.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:45 smithi174 ceph-mon[35635]: pgmap v969: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:24:46.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:45 smithi104 ceph-mon[35325]: pgmap v969: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 826 op/s 2024-09-10T21:24:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 -- 172.21.15.33:0/58770114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 msgr2=0x7fe96810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 --2- 172.21.15.33:0/58770114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 0x7fe96810f550 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fe960009900 tx=0x7fe96002a410 comp rx=0 tx=0).stop 2024-09-10T21:24:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 -- 172.21.15.33:0/58770114 shutdown_connections 2024-09-10T21:24:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 --2- 172.21.15.33:0/58770114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 0x7fe96810f550 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 --2- 172.21.15.33:0/58770114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe968104530 0x7fe96810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 --2- 172.21.15.33:0/58770114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 0x7fe968103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 -- 172.21.15.33:0/58770114 >> 172.21.15.33:0/58770114 conn(0x7fe9680fd700 msgr2=0x7fe9680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 -- 172.21.15.33:0/58770114 shutdown_connections 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.623+0000 7fe97003f640 1 -- 172.21.15.33:0/58770114 wait complete. 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 Processor -- start 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 -- start start 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 0x7fe968104b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe968104530 0x7fe968105070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 0x7fe9681055b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe968112d60 con 0x7fe96810d0e0 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe968112be0 con 0x7fe968103b30 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe97003f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe968112ee0 con 0x7fe968104530 2024-09-10T21:24:46.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe96dddb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 0x7fe968104b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe96e5dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 0x7fe9681055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe96e5dc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 0x7fe9681055b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47578/0 (socket says 172.21.15.33:47578) 2024-09-10T21:24:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe96d5da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe968104530 0x7fe968105070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe96e5dc640 1 -- 172.21.15.33:0/791037658 learned_addr learned my addr 172.21.15.33:0/791037658 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.624+0000 7fe96dddb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 0x7fe968104b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56540/0 (socket says 172.21.15.33:56540) 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe96dddb640 1 -- 172.21.15.33:0/791037658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe968104530 msgr2=0x7fe968105070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe96dddb640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe968104530 0x7fe968105070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe96dddb640 1 -- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 msgr2=0x7fe9681055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe96dddb640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 0x7fe9681055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe96dddb640 1 -- 172.21.15.33:0/791037658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9681abad0 con 0x7fe968103b30 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe96dddb640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 0x7fe968104b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe95800eb00 tx=0x7fe95800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe966ffd640 1 -- 172.21.15.33:0/791037658 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958010430 con 0x7fe968103b30 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe966ffd640 1 -- 172.21.15.33:0/791037658 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe958010a80 con 0x7fe968103b30 2024-09-10T21:24:46.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9681abdc0 con 0x7fe968103b30 2024-09-10T21:24:46.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe966ffd640 1 -- 172.21.15.33:0/791037658 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe958017a90 con 0x7fe968103b30 2024-09-10T21:24:46.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.625+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9681ac270 con 0x7fe968103b30 2024-09-10T21:24:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.628+0000 7fe972414740 1 Processor -- start 2024-09-10T21:24:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe972414740 1 -- start start 2024-09-10T21:24:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe972414740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dba78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe972414740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe972414740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbb6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe972414740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56350db1dea0 con 0x56350dba7df0 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe972414740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56350db0d2d0 con 0x56350dbb3d80 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe972414740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56350db110e0 con 0x56350dba74d0 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe96e5dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dba78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe96e5dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dba78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39592/0 (socket says 172.21.15.33:39592) 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe966ffd640 1 -- 172.21.15.33:0/791037658 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe958017c70 con 0x7fe968103b30 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe96e5dc640 1 -- 172.21.15.33:0/2960859426 learned_addr learned my addr 172.21.15.33:0/2960859426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe96d5da640 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbb6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.629+0000 7fe96dddb640 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96e5dc640 1 -- 172.21.15.33:0/2960859426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 msgr2=0x56350dbb6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbb6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96e5dc640 1 -- 172.21.15.33:0/2960859426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 msgr2=0x56350dbb3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96e5dc640 1 -- 172.21.15.33:0/2960859426 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350d86b650 con 0x56350dba74d0 2024-09-10T21:24:46.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96d5da640 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbb6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:46.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96dddb640 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:46.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dba78b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe960031bc0 tx=0x7fe960031bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df7baae2e2350458 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe94ffff640 1 -- 172.21.15.33:0/2960859426 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960040070 con 0x56350dba74d0 2024-09-10T21:24:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe94ffff640 1 -- 172.21.15.33:0/2960859426 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe960004030 con 0x56350dba74d0 2024-09-10T21:24:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe94ffff640 1 -- 172.21.15.33:0/2960859426 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960035040 con 0x56350dba74d0 2024-09-10T21:24:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.630+0000 7fe966ffd640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9500b52e0 0x7fe9500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe966ffd640 1 -- 172.21.15.33:0/791037658 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe9580e6600 con 0x7fe968103b30 2024-09-10T21:24:46.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe96dddb640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9500b52e0 0x7fe9500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 -- 172.21.15.33:0/2960859426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 msgr2=0x56350dba78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe96dddb640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9500b52e0 0x7fe9500b77a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe9580acd50 tx=0x7fe958000f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dba78b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe960031bc0 tx=0x7fe960031bf0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 -- 172.21.15.33:0/2960859426 shutdown_connections 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbb6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 --2- 172.21.15.33:0/2960859426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dba78b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 -- 172.21.15.33:0/2960859426 >> 172.21.15.33:0/2960859426 conn(0x56350dba5300 msgr2=0x56350dba64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 -- 172.21.15.33:0/2960859426 shutdown_connections 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.631+0000 7fe972414740 1 -- 172.21.15.33:0/2960859426 wait complete. 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 -- start start 2024-09-10T21:24:46.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 Processor -- start 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbb3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbb1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56350d96e700 con 0x56350dba7df0 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56350daa8160 con 0x56350dba74d0 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56350dbb69a0 con 0x56350dbb3d80 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe972414740 10 client.0 _ll_register_callbacks cb 0x56350dac19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.632+0000 7fe96e5dc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbb3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbb3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56564/0 (socket says 172.21.15.33:56564) 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 -- 172.21.15.33:0/50034669 learned_addr learned my addr 172.21.15.33:0/50034669 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbb1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96d5da640 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 -- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 msgr2=0x56350dbb1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbb1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 -- 172.21.15.33:0/50034669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 msgr2=0x56350dbb0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 -- 172.21.15.33:0/50034669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350dad5c90 con 0x56350dba74d0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbb3630 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe960009a20 tx=0x7fe9600051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8c04dac6f5852e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe94d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe94d7fa640 1 -- 172.21.15.33:0/50034669 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe960040070 con 0x56350dba74d0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe94d7fa640 1 -- 172.21.15.33:0/50034669 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe9600043f0 con 0x56350dba74d0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.633+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe960004030 con 0x56350dba74d0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe94d7fa640 1 -- 172.21.15.33:0/50034669 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe960035040 con 0x56350dba74d0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 10 client.28481 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56350dbc22d0 con 0x56350dba74d0 2024-09-10T21:24:46.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe94d7fa640 1 -- 172.21.15.33:0/50034669 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe960004030 con 0x56350dba74d0 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe94d7fa640 1 -- 172.21.15.33:0/50034669 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fe960004bd0 con 0x56350dba74d0 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 10 client.28481 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 10 client.28481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 20 client.28481 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dbc3620 0x56350dbc5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 4 client.28481 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56350db1fdf0 con 0x56350dbc3620 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.634+0000 7fe96d5da640 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dbc3620 0x56350dbc5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe96d5da640 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dbc3620 0x56350dbc5a10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f6a0c68f905192 server_cookie=48b013b59cfce41f in_seq=0 out_seq=0 2024-09-10T21:24:46.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe94d7fa640 10 client.28481 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:24:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe94d7fa640 1 -- 172.21.15.33:0/50034669 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56350db1fdf0 con 0x56350dbc3620 2024-09-10T21:24:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe94d7fa640 10 client.28481 handle_command_reply: tid=0 2024-09-10T21:24:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 client.28481 shutdown 2024-09-10T21:24:46.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 msgr2=0x56350dbb3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbb3630 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fe960009a20 tx=0x7fe9600051f0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 shutdown_connections 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbb1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dbc3620 0x56350dbc5a10 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 --2- 172.21.15.33:0/50034669 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbb3630 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 >> 172.21.15.33:0/50034669 conn(0x56350dba5300 msgr2=0x56350dbaaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.635+0000 7fe94d7fa640 0 client.28481 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.636+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 shutdown_connections 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.636+0000 7fe972414740 1 -- 172.21.15.33:0/50034669 wait complete. 2024-09-10T21:24:46.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.636+0000 7fe972414740 20 client.28481 trim_cache size 0 max 16384 2024-09-10T21:24:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 Processor -- start 2024-09-10T21:24:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 -- start start 2024-09-10T21:24:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dbb5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbadcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b4a20 con 0x56350dba7df0 2024-09-10T21:24:46.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b6620 con 0x56350dbb3d80 2024-09-10T21:24:46.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe972414740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b51b0 con 0x56350dba74d0 2024-09-10T21:24:46.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47598/0 (socket says 172.21.15.33:47598) 2024-09-10T21:24:46.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 -- 172.21.15.33:0/934287503 learned_addr learned my addr 172.21.15.33:0/934287503 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:46.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbadcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dbb5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 -- 172.21.15.33:0/934287503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 msgr2=0x56350dbb5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dbb5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 -- 172.21.15.33:0/934287503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 msgr2=0x56350dbadcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbadcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96d5da640 1 -- 172.21.15.33:0/934287503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56350dbb13b0 con 0x56350dba7df0 2024-09-10T21:24:46.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.649+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dbb5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:46.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe96d5da640 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb56b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00bd30 tx=0x7fe95c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee72d3cbafcbfc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe94f7fe640 1 -- 172.21.15.33:0/934287503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c004220 con 0x56350dba7df0 2024-09-10T21:24:46.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe94f7fe640 1 -- 172.21.15.33:0/934287503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c0043c0 con 0x56350dba7df0 2024-09-10T21:24:46.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe94f7fe640 1 -- 172.21.15.33:0/934287503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c0035f0 con 0x56350dba7df0 2024-09-10T21:24:46.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe972414740 1 -- 172.21.15.33:0/934287503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 msgr2=0x56350dbb56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe972414740 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb56b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fe95c00bd30 tx=0x7fe95c00b410 comp rx=0 tx=0).stop 2024-09-10T21:24:46.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe972414740 1 -- 172.21.15.33:0/934287503 shutdown_connections 2024-09-10T21:24:46.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe972414740 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dbb3d80 0x56350dbadcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe972414740 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbb56b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe972414740 1 --2- 172.21.15.33:0/934287503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dba74d0 0x56350dbb5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.650+0000 7fe972414740 1 -- 172.21.15.33:0/934287503 >> 172.21.15.33:0/934287503 conn(0x56350dba5300 msgr2=0x56350dbb28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 -- 172.21.15.33:0/934287503 shutdown_connections 2024-09-10T21:24:46.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 -- 172.21.15.33:0/934287503 wait complete. 2024-09-10T21:24:46.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 -- start start 2024-09-10T21:24:46.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 Processor -- start 2024-09-10T21:24:46.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbc9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbc5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbc61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b5f50 con 0x56350dba7df0 2024-09-10T21:24:46.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b5880 con 0x56350dba74d0 2024-09-10T21:24:46.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9681b29e0 con 0x56350dbb3d80 2024-09-10T21:24:46.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe972414740 10 client.0 _ll_register_callbacks cb 0x7fe95c00b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:46.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.651+0000 7fe96d5da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbc5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbc5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47602/0 (socket says 172.21.15.33:47602) 2024-09-10T21:24:46.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 -- 172.21.15.33:0/415486852 learned_addr learned my addr 172.21.15.33:0/415486852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:46.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbc61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbc9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 -- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 msgr2=0x56350dbc61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbc61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 -- 172.21.15.33:0/415486852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 msgr2=0x56350dbc9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:46.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbc9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 -- 172.21.15.33:0/415486852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56350dc30190 con 0x56350dba7df0 2024-09-10T21:24:46.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe96d5da640 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbc5a10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe95c026040 tx=0x7fe95c0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b980114527de5a0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe94d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:24:46.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe94d7fa640 1 -- 172.21.15.33:0/415486852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95c028040 con 0x56350dba7df0 2024-09-10T21:24:46.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe95c0043c0 con 0x56350dba7df0 2024-09-10T21:24:46.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe94d7fa640 1 -- 172.21.15.33:0/415486852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe95c005020 con 0x56350dba7df0 2024-09-10T21:24:46.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe94d7fa640 1 -- 172.21.15.33:0/415486852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe95c004e50 con 0x56350dba7df0 2024-09-10T21:24:46.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.652+0000 7fe972414740 10 client.19125 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:46.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56350dc313b0 con 0x56350dba7df0 2024-09-10T21:24:46.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe94d7fa640 1 -- 172.21.15.33:0/415486852 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe95c0043c0 con 0x56350dba7df0 2024-09-10T21:24:46.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe94d7fa640 1 -- 172.21.15.33:0/415486852 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fe95c02dac0 con 0x56350dba7df0 2024-09-10T21:24:46.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe972414740 10 client.19125 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:46.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe972414740 10 client.19125 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:24:46.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe972414740 20 client.19125 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:46.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe972414740 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dc31a90 0x56350dc33eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe972414740 4 client.19125 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:24:46.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56350db1b8a0 con 0x56350dc31a90 2024-09-10T21:24:46.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.653+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dc31a90 0x56350dc33eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe94d7fa640 10 client.19125 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:24:46.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe96e5dc640 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dc31a90 0x56350dc33eb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73f0640406acb453 server_cookie=4998dd36ff6468fd in_seq=0 out_seq=0 2024-09-10T21:24:46.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe94d7fa640 1 -- 172.21.15.33:0/415486852 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56350db1b8a0 con 0x56350dc31a90 2024-09-10T21:24:46.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe94d7fa640 10 client.19125 handle_command_reply: tid=0 2024-09-10T21:24:46.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 client.19125 shutdown 2024-09-10T21:24:46.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 msgr2=0x56350dbc5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbc5a10 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fe95c026040 tx=0x7fe95c0049d0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 shutdown_connections 2024-09-10T21:24:46.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56350dbb3d80 0x56350dbc61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x56350dc31a90 0x56350dc33eb0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56350dba7df0 0x56350dbc5a10 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 --2- 172.21.15.33:0/415486852 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56350dba74d0 0x56350dbc9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe94d7fa640 0 client.19125 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:24:46.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 >> 172.21.15.33:0/415486852 conn(0x56350dba5300 msgr2=0x56350dba3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 shutdown_connections 2024-09-10T21:24:46.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 1 -- 172.21.15.33:0/415486852 wait complete. 2024-09-10T21:24:46.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.654+0000 7fe972414740 20 client.19125 trim_cache size 0 max 16384 2024-09-10T21:24:46.676 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:24:46.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9500b52e0 msgr2=0x7fe9500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9500b52e0 0x7fe9500b77a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe9580acd50 tx=0x7fe958000f40 comp rx=0 tx=0).stop 2024-09-10T21:24:46.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 msgr2=0x7fe968104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 0x7fe968104b30 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe95800eb00 tx=0x7fe95800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 shutdown_connections 2024-09-10T21:24:46.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe9500b52e0 0x7fe9500b77a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe96810d0e0 0x7fe9681055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe968104530 0x7fe968105070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 --2- 172.21.15.33:0/791037658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe968103b30 0x7fe968104b30 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 >> 172.21.15.33:0/791037658 conn(0x7fe9680fd700 msgr2=0x7fe96810f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 shutdown_connections 2024-09-10T21:24:46.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:46.655+0000 7fe97003f640 1 -- 172.21.15.33:0/791037658 wait complete. 2024-09-10T21:24:47.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 356492 39.41 MB/sec execute 347 sec latency 185.738 ms 2024-09-10T21:24:47.490 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:24:47.835 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:48.072 INFO:tasks.workunit.client.0.smithi033.stdout: 10 357051 39.37 MB/sec execute 348 sec latency 143.025 ms 2024-09-10T21:24:48.133 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:47 smithi033 ceph-mon[30860]: pgmap v970: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2024-09-10T21:24:48.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:47 smithi174 ceph-mon[35635]: pgmap v970: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2024-09-10T21:24:48.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:47 smithi104 ceph-mon[35325]: pgmap v970: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.4 MiB/s wr, 644 op/s 2024-09-10T21:24:48.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.395+0000 7f1d375e7640 1 -- 172.21.15.33:0/3431210230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d3010d1a0 msgr2=0x7f1d3010f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.395+0000 7f1d375e7640 1 --2- 172.21.15.33:0/3431210230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d3010d1a0 0x7f1d3010f600 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009ca0 tx=0x7f1d2802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.395+0000 7f1d375e7640 1 -- 172.21.15.33:0/3431210230 shutdown_connections 2024-09-10T21:24:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.395+0000 7f1d375e7640 1 --2- 172.21.15.33:0/3431210230 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d3010d1a0 0x7f1d3010f600 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.395+0000 7f1d375e7640 1 --2- 172.21.15.33:0/3431210230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30104660 0x7f1d3010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.395+0000 7f1d375e7640 1 --2- 172.21.15.33:0/3431210230 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d30103d40 0x7f1d30104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.395+0000 7f1d375e7640 1 -- 172.21.15.33:0/3431210230 >> 172.21.15.33:0/3431210230 conn(0x7f1d300fd7e0 msgr2=0x7f1d300ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.398+0000 7f1d375e7640 1 -- 172.21.15.33:0/3431210230 shutdown_connections 2024-09-10T21:24:48.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.398+0000 7f1d375e7640 1 -- 172.21.15.33:0/3431210230 wait complete. 2024-09-10T21:24:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 Processor -- start 2024-09-10T21:24:48.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 -- start start 2024-09-10T21:24:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103d40 0x7f1d3019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d30104660 0x7f1d3019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3010d1a0 0x7f1d301a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30112ea0 con 0x7f1d30104660 2024-09-10T21:24:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30112d20 con 0x7f1d3010d1a0 2024-09-10T21:24:48.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d375e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d30113020 con 0x7f1d30103d40 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d34b82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d30104660 0x7f1d3019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d35383640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103d40 0x7f1d3019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d34b82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d30104660 0x7f1d3019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47614/0 (socket says 172.21.15.33:47614) 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.399+0000 7f1d34b82640 1 -- 172.21.15.33:0/1329661849 learned_addr learned my addr 172.21.15.33:0/1329661849 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d34b82640 1 -- 172.21.15.33:0/1329661849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103d40 msgr2=0x7f1d3019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d34b82640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103d40 0x7f1d3019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d34b82640 1 -- 172.21.15.33:0/1329661849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3010d1a0 msgr2=0x7f1d301a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d34b82640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3010d1a0 0x7f1d301a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d34b82640 1 -- 172.21.15.33:0/1329661849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d301a15d0 con 0x7f1d30104660 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d35383640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103d40 0x7f1d3019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d34b82640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d30104660 0x7f1d3019ba60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000e9c0 tx=0x7f1d2000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d267fc640 1 -- 172.21.15.33:0/1329661849 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004220 con 0x7f1d30104660 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d301a18c0 con 0x7f1d30104660 2024-09-10T21:24:48.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d267fc640 1 -- 172.21.15.33:0/1329661849 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d200043c0 con 0x7f1d30104660 2024-09-10T21:24:48.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.400+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d301a7f80 con 0x7f1d30104660 2024-09-10T21:24:48.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.401+0000 7f1d267fc640 1 -- 172.21.15.33:0/1329661849 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20010430 con 0x7f1d30104660 2024-09-10T21:24:48.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.403+0000 7f1d267fc640 1 -- 172.21.15.33:0/1329661849 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1d200035f0 con 0x7f1d30104660 2024-09-10T21:24:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.404+0000 7f1d267fc640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d1c0b53e0 0x7f1d1c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 Processor -- start 2024-09-10T21:24:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d35383640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d1c0b53e0 0x7f1d1c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d267fc640 1 -- 172.21.15.33:0/1329661849 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1d200e6090 con 0x7f1d30104660 2024-09-10T21:24:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 -- start start 2024-09-10T21:24:48.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c178935ea0 con 0x55c1789cbd80 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c1789252d0 con 0x55c1789bfdf0 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d399bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1789290e0 con 0x55c1789bf4d0 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d35383640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d35383640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39654/0 (socket says 172.21.15.33:39654) 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d35383640 1 -- 172.21.15.33:0/2149261411 learned_addr learned my addr 172.21.15.33:0/2149261411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d34b82640 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.405+0000 7f1d35b84640 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d35383640 1 -- 172.21.15.33:0/2149261411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 msgr2=0x55c1789cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d35383640 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d35383640 1 -- 172.21.15.33:0/2149261411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 msgr2=0x55c1789ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d35383640 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d35383640 1 -- 172.21.15.33:0/2149261411 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c178683650 con 0x55c1789bf4d0 2024-09-10T21:24:48.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d35383640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d1c0b53e0 0x7f1d1c0b78a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009db0 tx=0x7f1d2c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d35383640 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bf8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009860 tx=0x7f1d2c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=634094b13cc33c37 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d1b7fe640 1 -- 172.21.15.33:0/2149261411 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c013650 con 0x55c1789bf4d0 2024-09-10T21:24:48.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d1b7fe640 1 -- 172.21.15.33:0/2149261411 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d2c024070 con 0x55c1789bf4d0 2024-09-10T21:24:48.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.406+0000 7f1d1b7fe640 1 -- 172.21.15.33:0/2149261411 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c01fa80 con 0x55c1789bf4d0 2024-09-10T21:24:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.407+0000 7f1d399bc740 1 -- 172.21.15.33:0/2149261411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 msgr2=0x55c1789bf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.407+0000 7f1d399bc740 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bf8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009860 tx=0x7f1d2c01b000 comp rx=0 tx=0).stop 2024-09-10T21:24:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 -- 172.21.15.33:0/2149261411 shutdown_connections 2024-09-10T21:24:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 --2- 172.21.15.33:0/2149261411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bf8b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 -- 172.21.15.33:0/2149261411 >> 172.21.15.33:0/2149261411 conn(0x55c1789bd300 msgr2=0x55c1789be4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 -- 172.21.15.33:0/2149261411 shutdown_connections 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 -- 172.21.15.33:0/2149261411 wait complete. 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 -- start start 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.408+0000 7f1d399bc740 1 Processor -- start 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789c9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d34b82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d35383640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d34b82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56614/0 (socket says 172.21.15.33:56614) 2024-09-10T21:24:48.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d34b82640 1 -- 172.21.15.33:0/750580620 learned_addr learned my addr 172.21.15.33:0/750580620 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c178786700 con 0x55c1789cbd80 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c1788c0160 con 0x55c1789bfdf0 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1789ce9a0 con 0x55c1789bf4d0 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d35b84640 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.409+0000 7f1d399bc740 10 client.0 _ll_register_callbacks cb 0x55c1788d99e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d34b82640 1 -- 172.21.15.33:0/750580620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 msgr2=0x55c1789cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d34b82640 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d34b82640 1 -- 172.21.15.33:0/750580620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 msgr2=0x55c1789c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d34b82640 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d34b82640 1 -- 172.21.15.33:0/750580620 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1788edc90 con 0x55c1789bfdf0 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d34b82640 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789c8a80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1d200a9a80 tx=0x7f1d20003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63a86402f9dc2056 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:48.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d18ff9640 1 -- 172.21.15.33:0/750580620 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200cfd60 con 0x55c1789bfdf0 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d2c024070 con 0x55c1789bfdf0 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d18ff9640 1 -- 172.21.15.33:0/750580620 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1d200adb40 con 0x55c1789bfdf0 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d18ff9640 1 -- 172.21.15.33:0/750580620 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f1d200ac810 con 0x55c1789bfdf0 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d399bc740 10 client.28493 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.410+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55c1789da370 con 0x55c1789bfdf0 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d18ff9640 1 -- 172.21.15.33:0/750580620 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d200b12f0 con 0x55c1789bfdf0 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d18ff9640 1 -- 172.21.15.33:0/750580620 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f1d200a8160 con 0x55c1789bfdf0 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d399bc740 10 client.28493 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d399bc740 10 client.28493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d399bc740 20 client.28493 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d399bc740 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c1789db840 0x55c1789ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d399bc740 4 client.28493 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1d2c024070 con 0x55c1789db840 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.411+0000 7f1d35383640 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c1789db840 0x55c1789ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.412+0000 7f1d35383640 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c1789db840 0x55c1789ddc50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd057867f035dbbc server_cookie=bf16fed3010e7331 in_seq=0 out_seq=0 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.412+0000 7f1d18ff9640 10 client.28493 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d18ff9640 1 -- 172.21.15.33:0/750580620 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1d2c024070 con 0x55c1789db840 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d18ff9640 10 client.28493 handle_command_reply: tid=0 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 client.28493 shutdown 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 msgr2=0x55c1789c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789c8a80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1d200a9a80 tx=0x7f1d20003e80 comp rx=0 tx=0).stop 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 shutdown_connections 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c1789db840 0x55c1789ddc50 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789cbd80 0x55c1789c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789c8a80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 --2- 172.21.15.33:0/750580620 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 >> 172.21.15.33:0/750580620 conn(0x55c1789bd300 msgr2=0x55c1789c2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d18ff9640 0 client.28493 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 shutdown_connections 2024-09-10T21:24:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 1 -- 172.21.15.33:0/750580620 wait complete. 2024-09-10T21:24:48.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.413+0000 7f1d399bc740 20 client.28493 trim_cache size 0 max 16384 2024-09-10T21:24:48.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.432+0000 7f1d399bc740 1 Processor -- start 2024-09-10T21:24:48.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d399bc740 1 -- start start 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bfdf0 0x55c1789bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789cbd80 0x55c1789c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d399bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1400e450 con 0x55c1789bfdf0 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d399bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14010050 con 0x55c1789cbd80 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d399bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1400ebe0 con 0x55c1789bf4d0 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d35b84640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789cbd80 0x55c1789c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d35b84640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789cbd80 0x55c1789c5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56616/0 (socket says 172.21.15.33:56616) 2024-09-10T21:24:48.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d35b84640 1 -- 172.21.15.33:0/1925840272 learned_addr learned my addr 172.21.15.33:0/1925840272 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d35383640 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.433+0000 7f1d34b82640 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bfdf0 0x55c1789bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d35b84640 1 -- 172.21.15.33:0/1925840272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 msgr2=0x55c1789bead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d35b84640 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d35b84640 1 -- 172.21.15.33:0/1925840272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bfdf0 msgr2=0x55c1789bcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d35b84640 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bfdf0 0x55c1789bcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d35b84640 1 -- 172.21.15.33:0/1925840272 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1789ce4e0 con 0x55c1789cbd80 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d35b84640 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789cbd80 0x55c1789c5c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d28035000 tx=0x7f1d28009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37042fe38bb7fdfe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d1affd640 1 -- 172.21.15.33:0/1925840272 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28031af0 con 0x55c1789cbd80 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d1affd640 1 -- 172.21.15.33:0/1925840272 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d28036040 con 0x55c1789cbd80 2024-09-10T21:24:48.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.434+0000 7f1d1affd640 1 -- 172.21.15.33:0/1925840272 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28004040 con 0x55c1789cbd80 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 -- 172.21.15.33:0/1925840272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789cbd80 msgr2=0x55c1789c5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789cbd80 0x55c1789c5c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1d28035000 tx=0x7f1d28009a20 comp rx=0 tx=0).stop 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 -- 172.21.15.33:0/1925840272 shutdown_connections 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789cbd80 0x55c1789c5c90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bfdf0 0x55c1789bcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 --2- 172.21.15.33:0/1925840272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789bf4d0 0x55c1789bead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 -- 172.21.15.33:0/1925840272 >> 172.21.15.33:0/1925840272 conn(0x55c1789bd300 msgr2=0x55c1789be3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 -- 172.21.15.33:0/1925840272 shutdown_connections 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.435+0000 7f1d399bc740 1 -- 172.21.15.33:0/1925840272 wait complete. 2024-09-10T21:24:48.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.436+0000 7f1d399bc740 1 -- start start 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.436+0000 7f1d399bc740 1 Processor -- start 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.436+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bf4d0 0x55c1789d4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.436+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789d0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.436+0000 7f1d399bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789cbd80 0x55c1789d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d399bc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1400f980 con 0x55c1789bf4d0 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d399bc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1400f2b0 con 0x55c1789bfdf0 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d399bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1400c410 con 0x55c1789cbd80 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d399bc740 10 client.0 _ll_register_callbacks cb 0x7f1d200046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789cbd80 0x55c1789d0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d34b82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789d0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789cbd80 0x55c1789d0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39674/0 (socket says 172.21.15.33:39674) 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 -- 172.21.15.33:0/3050033505 learned_addr learned my addr 172.21.15.33:0/3050033505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:48.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d34b82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789d0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:56624/0 (socket says 172.21.15.33:56624) 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35383640 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bf4d0 0x55c1789d4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 -- 172.21.15.33:0/3050033505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 msgr2=0x55c1789d0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789d0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 -- 172.21.15.33:0/3050033505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bf4d0 msgr2=0x55c1789d4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bf4d0 0x55c1789d4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 -- 172.21.15.33:0/3050033505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1789c93b0 con 0x55c1789cbd80 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.437+0000 7f1d35b84640 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789cbd80 0x55c1789d0c40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1d2802aeb0 tx=0x7f1d28003970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f574dc31c0790a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d18ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d18ff9640 1 -- 172.21.15.33:0/3050033505 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28054060 con 0x55c1789cbd80 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d18ff9640 1 -- 172.21.15.33:0/3050033505 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1d2802f610 con 0x55c1789cbd80 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d28036040 con 0x55c1789cbd80 2024-09-10T21:24:48.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d18ff9640 1 -- 172.21.15.33:0/3050033505 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f1d28031e60 con 0x55c1789cbd80 2024-09-10T21:24:48.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d399bc740 10 client.28384 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:48.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55c178a49340 con 0x55c1789cbd80 2024-09-10T21:24:48.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.438+0000 7f1d18ff9640 1 -- 172.21.15.33:0/3050033505 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d28036040 con 0x55c1789cbd80 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d18ff9640 1 -- 172.21.15.33:0/3050033505 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f1d28030890 con 0x55c1789cbd80 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d399bc740 10 client.28384 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d399bc740 10 client.28384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d399bc740 20 client.28384 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d399bc740 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c178a49a60 0x55c178a4be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d399bc740 4 client.28384 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c178937df0 con 0x55c178a49a60 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.439+0000 7f1d35383640 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c178a49a60 0x55c178a4be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d35383640 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c178a49a60 0x55c178a4be80 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe5b81dad9ea7159 server_cookie=c9caec462b937db6 in_seq=0 out_seq=0 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d18ff9640 10 client.28384 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d18ff9640 1 -- 172.21.15.33:0/3050033505 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c178937df0 con 0x55c178a49a60 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d18ff9640 10 client.28384 handle_command_reply: tid=0 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 client.28384 shutdown 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789cbd80 msgr2=0x55c1789d0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789cbd80 0x55c1789d0c40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1d2802aeb0 tx=0x7f1d28003970 comp rx=0 tx=0).stop 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 shutdown_connections 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c178a49a60 0x55c178a4be80 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1789cbd80 0x55c1789d0c40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d18ff9640 0 client.28384 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c1789bfdf0 0x55c1789d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 --2- 172.21.15.33:0/3050033505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c1789bf4d0 0x55c1789d4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 >> 172.21.15.33:0/3050033505 conn(0x55c1789bd300 msgr2=0x55c1789ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 shutdown_connections 2024-09-10T21:24:48.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 1 -- 172.21.15.33:0/3050033505 wait complete. 2024-09-10T21:24:48.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.440+0000 7f1d399bc740 20 client.28384 trim_cache size 0 max 16384 2024-09-10T21:24:48.448 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:24:48.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d1c0b53e0 msgr2=0x7f1d1c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d1c0b53e0 0x7f1d1c0b78a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009db0 tx=0x7f1d2c009110 comp rx=0 tx=0).stop 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d30104660 msgr2=0x7f1d3019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d30104660 0x7f1d3019ba60 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000e9c0 tx=0x7f1d2000edc0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 shutdown_connections 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1d1c0b53e0 0x7f1d1c0b78a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d3010d1a0 0x7f1d301a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1d30104660 0x7f1d3019ba60 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 --2- 172.21.15.33:0/1329661849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d30103d40 0x7f1d3019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 >> 172.21.15.33:0/1329661849 conn(0x7f1d300fd7e0 msgr2=0x7f1d300fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 shutdown_connections 2024-09-10T21:24:48.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:48.441+0000 7f1d375e7640 1 -- 172.21.15.33:0/1329661849 wait complete. 2024-09-10T21:24:49.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 357654 39.34 MB/sec execute 349 sec latency 189.393 ms 2024-09-10T21:24:49.146 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.146 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.146 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:24:49.147 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:24:49.147 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:24:49.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:24:49.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:24:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:24:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:24:50.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 358443 39.33 MB/sec execute 350 sec latency 281.027 ms 2024-09-10T21:24:50.187 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:24:50.187 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:24:50.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:50 smithi104 ceph-mon[35325]: pgmap v971: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-10T21:24:50.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:50 smithi174 ceph-mon[35635]: pgmap v971: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-10T21:24:50.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:50 smithi033 ceph-mon[30860]: pgmap v971: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.0 MiB/s wr, 709 op/s 2024-09-10T21:24:50.555 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:51.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 359408 39.40 MB/sec execute 351 sec latency 61.078 ms 2024-09-10T21:24:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.112+0000 7f2b224a2640 1 -- 172.21.15.33:0/656869568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c1023b0 msgr2=0x7f2b1c106880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.112+0000 7f2b224a2640 1 --2- 172.21.15.33:0/656869568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c1023b0 0x7f2b1c106880 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2b10009900 tx=0x7f2b1002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.112+0000 7f2b224a2640 1 -- 172.21.15.33:0/656869568 shutdown_connections 2024-09-10T21:24:51.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.112+0000 7f2b224a2640 1 --2- 172.21.15.33:0/656869568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c106ed0 0x7f2b1c107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.112+0000 7f2b224a2640 1 --2- 172.21.15.33:0/656869568 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c1023b0 0x7f2b1c106880 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.112+0000 7f2b224a2640 1 --2- 172.21.15.33:0/656869568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 0x7f2b1c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.112+0000 7f2b224a2640 1 -- 172.21.15.33:0/656869568 >> 172.21.15.33:0/656869568 conn(0x7f2b1c0fd700 msgr2=0x7f2b1c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.113+0000 7f2b224a2640 1 -- 172.21.15.33:0/656869568 shutdown_connections 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.113+0000 7f2b224a2640 1 -- 172.21.15.33:0/656869568 wait complete. 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.113+0000 7f2b224a2640 1 Processor -- start 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b224a2640 1 -- start start 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b224a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b224a2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c1023b0 0x7f2b1c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b224a2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c106ed0 0x7f2b1c1a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b224a2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c119390 con 0x7f2b1c1023b0 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b224a2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c119210 con 0x7f2b1c106ed0 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b224a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c119510 con 0x7f2b1c101a90 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47250/0 (socket says 172.21.15.33:47250) 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1746219822 learned_addr learned my addr 172.21.15.33:0/1746219822 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.114+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c1023b0 0x7f2b1c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c106ed0 0x7f2b1c1a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1746219822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c106ed0 msgr2=0x7f2b1c1a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c106ed0 0x7f2b1c1a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c1023b0 msgr2=0x7f2b1c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c1023b0 0x7f2b1c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1746219822 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c1a5950 con 0x7f2b1c101a90 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c106ed0 0x7f2b1c1a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.115+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f8a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c890 tx=0x7f2b0c00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.116+0000 7f2b197fa640 1 -- 172.21.15.33:0/1746219822 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c004220 con 0x7f2b1c101a90 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.116+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c1a5c40 con 0x7f2b1c101a90 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.116+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c1ac4d0 con 0x7f2b1c101a90 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.116+0000 7f2b197fa640 1 -- 172.21.15.33:0/1746219822 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0c0043c0 con 0x7f2b1c101a90 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.116+0000 7f2b197fa640 1 -- 172.21.15.33:0/1746219822 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c004a00 con 0x7f2b1c101a90 2024-09-10T21:24:51.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.119+0000 7f2b24877740 1 Processor -- start 2024-09-10T21:24:51.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.120+0000 7f2b24877740 1 -- start start 2024-09-10T21:24:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.120+0000 7f2b24877740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a34f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.120+0000 7f2b24877740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a35b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.120+0000 7f2b24877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a35db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.120+0000 7f2b24877740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e79a2c5e60 con 0x55e79a34f800 2024-09-10T21:24:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.120+0000 7f2b24877740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e79a2b5290 con 0x55e79a34eee0 2024-09-10T21:24:51.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.120+0000 7f2b24877740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e79a2b90a0 con 0x55e79a35b790 2024-09-10T21:24:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.121+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a35db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.121+0000 7f2b197fa640 1 -- 172.21.15.33:0/1746219822 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2b0c004ba0 con 0x7f2b1c101a90 2024-09-10T21:24:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.121+0000 7f2b20a3f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a34f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.121+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a35b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a35db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:47256/0 (socket says 172.21.15.33:47256) 2024-09-10T21:24:51.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1708920480 learned_addr learned my addr 172.21.15.33:0/1708920480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1708920480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 msgr2=0x55e79a35db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a35db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1708920480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 msgr2=0x55e79a34f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a34f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1bfff640 1 -- 172.21.15.33:0/1708920480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79a013930 con 0x55e79a34f800 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a35db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.122+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a34f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.123+0000 7f2b197fa640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b040b53e0 0x7f2b040b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.123+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b040b53e0 0x7f2b040b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.123+0000 7f2b197fa640 1 -- 172.21.15.33:0/1746219822 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2b0c0e5730 con 0x7f2b1c101a90 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.124+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a35b250 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0e3160 tx=0x7f2b0c0ed610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3edc0258c107746 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.124+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b040b53e0 0x7f2b040b78a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009fd0 tx=0x7f2b14009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.124+0000 7f2b027fc640 1 -- 172.21.15.33:0/1708920480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c01d030 con 0x55e79a34f800 2024-09-10T21:24:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.124+0000 7f2b027fc640 1 -- 172.21.15.33:0/1708920480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0c0ab9c0 con 0x55e79a34f800 2024-09-10T21:24:51.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.124+0000 7f2b027fc640 1 -- 172.21.15.33:0/1708920480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0abc80 con 0x55e79a34f800 2024-09-10T21:24:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 -- 172.21.15.33:0/1708920480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 msgr2=0x55e79a35b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a35b250 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0e3160 tx=0x7f2b0c0ed610 comp rx=0 tx=0).stop 2024-09-10T21:24:51.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 -- 172.21.15.33:0/1708920480 shutdown_connections 2024-09-10T21:24:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a35db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a35b250 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 --2- 172.21.15.33:0/1708920480 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a34f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 -- 172.21.15.33:0/1708920480 >> 172.21.15.33:0/1708920480 conn(0x55e79a34cd10 msgr2=0x55e79a34deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.125+0000 7f2b24877740 1 -- 172.21.15.33:0/1708920480 shutdown_connections 2024-09-10T21:24:51.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.126+0000 7f2b24877740 1 -- 172.21.15.33:0/1708920480 wait complete. 2024-09-10T21:24:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.126+0000 7f2b24877740 1 -- start start 2024-09-10T21:24:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 1 Processor -- start 2024-09-10T21:24:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 0x55e79a3587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34f800 0x55e79a358d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a35b790 0x55e79a359250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e79a116890 con 0x55e79a35b790 2024-09-10T21:24:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e79a24fd40 con 0x55e79a34f800 2024-09-10T21:24:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e79a35e440 con 0x55e79a34eee0 2024-09-10T21:24:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b24877740 10 client.0 _ll_register_callbacks cb 0x55e79a2b3c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34f800 0x55e79a358d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34f800 0x55e79a358d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57468/0 (socket says 172.21.15.33:57468) 2024-09-10T21:24:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/682075871 learned_addr learned my addr 172.21.15.33:0/682075871 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/682075871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 msgr2=0x55e79a3587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 0x55e79a3587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/682075871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a35b790 msgr2=0x55e79a359250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a35b790 0x55e79a359250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.127+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/682075871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79a367930 con 0x55e79a34f800 2024-09-10T21:24:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34f800 0x55e79a358d10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b1002a9e0 tx=0x7f2b100045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ba7d1a87ecc8449 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2afb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2afb7fe640 1 -- 172.21.15.33:0/682075871 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10035040 con 0x55e79a34f800 2024-09-10T21:24:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2afb7fe640 1 -- 172.21.15.33:0/682075871 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2b100055d0 con 0x55e79a34f800 2024-09-10T21:24:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b0c0ab9c0 con 0x55e79a34f800 2024-09-10T21:24:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2afb7fe640 1 -- 172.21.15.33:0/682075871 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f2b10004750 con 0x55e79a34f800 2024-09-10T21:24:51.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2b24877740 10 client.28508 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.128+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55e79a36a310 con 0x55e79a34f800 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2afb7fe640 1 -- 172.21.15.33:0/682075871 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b0c0ab9c0 con 0x55e79a34f800 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2afb7fe640 1 -- 172.21.15.33:0/682075871 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f2b10003b70 con 0x55e79a34f800 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2b24877740 10 client.28508 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2b24877740 10 client.28508 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2b24877740 20 client.28508 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2b24877740 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a36b870 0x55e79a36dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2b24877740 4 client.28508 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:51.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e79a2c7db0 con 0x55e79a36b870 2024-09-10T21:24:51.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.129+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a36b870 0x55e79a36dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.130+0000 7f2afb7fe640 10 client.28508 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:51.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.130+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a36b870 0x55e79a36dcc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c9b2a422b7caae8 server_cookie=5db71687c77895e in_seq=0 out_seq=0 2024-09-10T21:24:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.132+0000 7f2afb7fe640 1 -- 172.21.15.33:0/682075871 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e79a2c7db0 con 0x55e79a36b870 2024-09-10T21:24:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.132+0000 7f2afb7fe640 10 client.28508 handle_command_reply: tid=0 2024-09-10T21:24:51.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.132+0000 7f2b24877740 1 client.28508 shutdown 2024-09-10T21:24:51.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.132+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34f800 msgr2=0x55e79a358d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.132+0000 7f2b24877740 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34f800 0x55e79a358d10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b1002a9e0 tx=0x7f2b100045a0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 shutdown_connections 2024-09-10T21:24:51.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a36b870 0x55e79a36dcc0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a35b790 0x55e79a359250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34f800 0x55e79a358d10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 --2- 172.21.15.33:0/682075871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 0x55e79a3587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 >> 172.21.15.33:0/682075871 conn(0x55e79a34cd10 msgr2=0x55e79a34bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2afb7fe640 0 client.28508 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:51.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 shutdown_connections 2024-09-10T21:24:51.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.133+0000 7f2b24877740 1 -- 172.21.15.33:0/682075871 wait complete. 2024-09-10T21:24:51.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.134+0000 7f2b24877740 20 client.28508 trim_cache size 0 max 16384 2024-09-10T21:24:51.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 Processor -- start 2024-09-10T21:24:51.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 -- start start 2024-09-10T21:24:51.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 0x55e79a34e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a355a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a35b790 0x55e79a355f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0800e4f0 con 0x55e79a34f800 2024-09-10T21:24:51.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b080100f0 con 0x55e79a35b790 2024-09-10T21:24:51.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.159+0000 7f2b24877740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0800ec80 con 0x55e79a34eee0 2024-09-10T21:24:51.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a355a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a355a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59936/0 (socket says 172.21.15.33:59936) 2024-09-10T21:24:51.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1577930167 learned_addr learned my addr 172.21.15.33:0/1577930167 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:51.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a35b790 0x55e79a355f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 0x55e79a34e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1577930167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 msgr2=0x55e79a34e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 0x55e79a34e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1577930167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a35b790 msgr2=0x55e79a355f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a35b790 0x55e79a355f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/1577930167 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e79a356cd0 con 0x55e79a34f800 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.160+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a355a00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2b10031440 tx=0x7f2b10001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a4ab611b4d7a0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.161+0000 7f2b01ffb640 1 -- 172.21.15.33:0/1577930167 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10035070 con 0x55e79a34f800 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.161+0000 7f2b01ffb640 1 -- 172.21.15.33:0/1577930167 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b10041070 con 0x55e79a34f800 2024-09-10T21:24:51.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.161+0000 7f2b01ffb640 1 -- 172.21.15.33:0/1577930167 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10004d00 con 0x55e79a34f800 2024-09-10T21:24:51.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.161+0000 7f2b24877740 1 -- 172.21.15.33:0/1577930167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 msgr2=0x55e79a355a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.161+0000 7f2b24877740 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a355a00 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2b10031440 tx=0x7f2b10001e50 comp rx=0 tx=0).stop 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 -- 172.21.15.33:0/1577930167 shutdown_connections 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a35b790 0x55e79a355f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a355a00 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 --2- 172.21.15.33:0/1577930167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a34eee0 0x55e79a34e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 -- 172.21.15.33:0/1577930167 >> 172.21.15.33:0/1577930167 conn(0x55e79a34cd10 msgr2=0x55e79a3528b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 -- 172.21.15.33:0/1577930167 shutdown_connections 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 -- 172.21.15.33:0/1577930167 wait complete. 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 -- start start 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.162+0000 7f2b24877740 1 Processor -- start 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b24877740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a361ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b24877740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a362010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b24877740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a3d77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b24877740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0800fa20 con 0x55e79a34f800 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a361ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a362010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a362010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59938/0 (socket says 172.21.15.33:59938) 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b1bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a361ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57494/0 (socket says 172.21.15.33:57494) 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b1b7fe640 1 -- 172.21.15.33:0/889978210 learned_addr learned my addr 172.21.15.33:0/889978210 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a3d77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b24877740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0800f350 con 0x55e79a34eee0 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0800c4b0 con 0x55e79a35b790 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b24877740 10 client.0 _ll_register_callbacks cb 0x7f2b10009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.163+0000 7f2b1bfff640 1 -- 172.21.15.33:0/889978210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 msgr2=0x55e79a3d77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a3d77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b1bfff640 1 -- 172.21.15.33:0/889978210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 msgr2=0x55e79a362010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:51.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a362010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b1bfff640 1 -- 172.21.15.33:0/889978210 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e79a3d7d20 con 0x55e79a34eee0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b20a3f640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a3d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a362010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b1bfff640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a361ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55e79a352060 tx=0x7f2b0c004570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca9ddc2d598a843 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2afb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2afb7fe640 1 -- 172.21.15.33:0/889978210 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c01d040 con 0x55e79a34eee0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b10041070 con 0x55e79a34eee0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2afb7fe640 1 -- 172.21.15.33:0/889978210 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2b0c0eb6d0 con 0x55e79a34eee0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2afb7fe640 1 -- 172.21.15.33:0/889978210 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c0ab620 con 0x55e79a34eee0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.164+0000 7f2b24877740 10 client.28514 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55e79a3d8f20 con 0x55e79a34eee0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2afb7fe640 1 -- 172.21.15.33:0/889978210 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b10041070 con 0x55e79a34eee0 2024-09-10T21:24:51.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2afb7fe640 1 -- 172.21.15.33:0/889978210 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f2b0c005290 con 0x55e79a34eee0 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b24877740 10 client.28514 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b24877740 10 client.28514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b24877740 20 client.28514 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b24877740 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a3d9250 0x55e79a3db640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b24877740 4 client.28514 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e79a2c3860 con 0x55e79a3d9250 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.165+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a3d9250 0x55e79a3db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.166+0000 7f2b1b7fe640 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a3d9250 0x55e79a3db640 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f1e5df588ed9b3 server_cookie=9db1aaafffb5d42c in_seq=0 out_seq=0 2024-09-10T21:24:51.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.166+0000 7f2afb7fe640 10 client.28514 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2afb7fe640 1 -- 172.21.15.33:0/889978210 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e79a2c3860 con 0x55e79a3d9250 2024-09-10T21:24:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2afb7fe640 10 client.28514 handle_command_reply: tid=0 2024-09-10T21:24:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 client.28514 shutdown 2024-09-10T21:24:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 msgr2=0x55e79a361ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a361ad0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x55e79a352060 tx=0x7f2b0c004570 comp rx=0 tx=0).stop 2024-09-10T21:24:51.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 shutdown_connections 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e79a3d9250 0x55e79a3db640 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e79a35b790 0x55e79a3d77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e79a34f800 0x55e79a362010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 --2- 172.21.15.33:0/889978210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e79a34eee0 0x55e79a361ad0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2afb7fe640 0 client.28514 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 >> 172.21.15.33:0/889978210 conn(0x55e79a34cd10 msgr2=0x55e79a359e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 shutdown_connections 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 1 -- 172.21.15.33:0/889978210 wait complete. 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.168+0000 7f2b24877740 20 client.28514 trim_cache size 0 max 16384 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:24:51.174 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:24:51.175 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07", 2024-09-10T21:24:51.175 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:24:51.175 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:24:51.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b040b53e0 msgr2=0x7f2b040b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b040b53e0 0x7f2b040b78a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009fd0 tx=0x7f2b14009500 comp rx=0 tx=0).stop 2024-09-10T21:24:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 msgr2=0x7f2b1c19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f8a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c890 tx=0x7f2b0c00cd50 comp rx=0 tx=0).stop 2024-09-10T21:24:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 shutdown_connections 2024-09-10T21:24:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2b040b53e0 0x7f2b040b78a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b1c106ed0 0x7f2b1c1a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b1c1023b0 0x7f2b1c19fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 --2- 172.21.15.33:0/1746219822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b1c101a90 0x7f2b1c19f8a0 secure :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c890 tx=0x7f2b0c00cd50 comp rx=0 tx=0).stop 2024-09-10T21:24:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.169+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 >> 172.21.15.33:0/1746219822 conn(0x7f2b1c0fd700 msgr2=0x7f2b1c1047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.170+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 shutdown_connections 2024-09-10T21:24:51.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:51.170+0000 7f2b224a2640 1 -- 172.21.15.33:0/1746219822 wait complete. 2024-09-10T21:24:51.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:51 smithi174 ceph-mon[35635]: pgmap v972: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-10T21:24:51.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:51 smithi033 ceph-mon[30860]: pgmap v972: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-10T21:24:51.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:51 smithi104 ceph-mon[35325]: pgmap v972: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 776 op/s 2024-09-10T21:24:52.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 359704 39.35 MB/sec execute 352 sec latency 467.095 ms 2024-09-10T21:24:52.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:52 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:24:52.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.427 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:52 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:24:52.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:52.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:52 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:53.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 360307 39.33 MB/sec execute 353 sec latency 518.155 ms 2024-09-10T21:24:53.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:24:53.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:53 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:53.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:53 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:53 smithi174 ceph-mon[35635]: pgmap v973: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.9 MiB/s wr, 623 op/s 2024-09-10T21:24:53.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:53 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:53.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:53 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:53.426 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:53 smithi033 ceph-mon[30860]: pgmap v973: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.9 MiB/s wr, 623 op/s 2024-09-10T21:24:53.555 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:24:53.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:53 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:53.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:53 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:24:53.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:53 smithi104 ceph-mon[35325]: pgmap v973: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.9 MiB/s wr, 623 op/s 2024-09-10T21:24:54.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 360809 39.28 MB/sec execute 354 sec latency 149.476 ms 2024-09-10T21:24:54.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.160+0000 7fd141e48640 1 -- 172.21.15.33:0/2924749590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10f5d0 msgr2=0x7fd13c115dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.160+0000 7fd141e48640 1 --2- 172.21.15.33:0/2924749590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10f5d0 0x7fd13c115dd0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd134009900 tx=0x7fd13402a440 comp rx=0 tx=0).stop 2024-09-10T21:24:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.161+0000 7fd141e48640 1 -- 172.21.15.33:0/2924749590 shutdown_connections 2024-09-10T21:24:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.161+0000 7fd141e48640 1 --2- 172.21.15.33:0/2924749590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10f5d0 0x7fd13c115dd0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.161+0000 7fd141e48640 1 --2- 172.21.15.33:0/2924749590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd13c10ec50 0x7fd13c10f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.161+0000 7fd141e48640 1 --2- 172.21.15.33:0/2924749590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c109250 0x7fd13c109630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.161+0000 7fd141e48640 1 -- 172.21.15.33:0/2924749590 >> 172.21.15.33:0/2924749590 conn(0x7fd13c078900 msgr2=0x7fd13c07ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.163+0000 7fd141e48640 1 -- 172.21.15.33:0/2924749590 shutdown_connections 2024-09-10T21:24:54.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.164+0000 7fd141e48640 1 -- 172.21.15.33:0/2924749590 wait complete. 2024-09-10T21:24:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.165+0000 7fd141e48640 1 Processor -- start 2024-09-10T21:24:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.165+0000 7fd141e48640 1 -- start start 2024-09-10T21:24:54.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.166+0000 7fd141e48640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c109250 0x7fd13c074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.166+0000 7fd141e48640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10ec50 0x7fd13c074c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.166+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10ec50 0x7fd13c074c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.166+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10ec50 0x7fd13c074c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57520/0 (socket says 172.21.15.33:57520) 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.166+0000 7fd13bfff640 1 -- 172.21.15.33:0/2104711764 learned_addr learned my addr 172.21.15.33:0/2104711764 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.166+0000 7fd140e46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c109250 0x7fd13c074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd141e48640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd13c10f5d0 0x7fd13c06da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c119310 con 0x7fd13c10f5d0 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c119190 con 0x7fd13c10ec50 2024-09-10T21:24:54.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c119490 con 0x7fd13c109250 2024-09-10T21:24:54.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd13bfff640 1 -- 172.21.15.33:0/2104711764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c109250 msgr2=0x7fd13c074750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd13bfff640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c109250 0x7fd13c074750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd13bfff640 1 -- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd13c10f5d0 msgr2=0x7fd13c06da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd13bfff640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd13c10f5d0 0x7fd13c06da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.167+0000 7fd13bfff640 1 -- 172.21.15.33:0/2104711764 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13c06e340 con 0x7fd13c10ec50 2024-09-10T21:24:54.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.168+0000 7fd13bfff640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10ec50 0x7fd13c074c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b450 tx=0x7fd12400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.171+0000 7fd139ffb640 1 -- 172.21.15.33:0/2104711764 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124015020 con 0x7fd13c10ec50 2024-09-10T21:24:54.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.171+0000 7fd139ffb640 1 -- 172.21.15.33:0/2104711764 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd124011d40 con 0x7fd13c10ec50 2024-09-10T21:24:54.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.171+0000 7fd139ffb640 1 -- 172.21.15.33:0/2104711764 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd124010e40 con 0x7fd13c10ec50 2024-09-10T21:24:54.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.171+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd13c06e630 con 0x7fd13c10ec50 2024-09-10T21:24:54.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.172+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd13c073610 con 0x7fd13c10ec50 2024-09-10T21:24:54.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.175+0000 7fd14547f740 1 Processor -- start 2024-09-10T21:24:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd14547f740 1 -- start start 2024-09-10T21:24:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd14547f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955989280 0x558955989660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd139ffb640 1 -- 172.21.15.33:0/2104711764 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fd124010500 con 0x7fd13c10ec50 2024-09-10T21:24:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd14547f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989ba0 0x5589559955f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd14547f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955995b30 0x558955997f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd14547f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5589558ffe50 con 0x558955995b30 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd14547f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5589558ef280 con 0x558955989280 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd14547f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589558f3090 con 0x558955989ba0 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955995b30 0x558955997f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955995b30 0x558955997f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59966/0 (socket says 172.21.15.33:59966) 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.178+0000 7fd13bfff640 1 -- 172.21.15.33:0/1316182278 learned_addr learned my addr 172.21.15.33:0/1316182278 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.179+0000 7fd140e46640 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989ba0 0x5589559955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.179+0000 7fd13bfff640 1 -- 172.21.15.33:0/1316182278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989ba0 msgr2=0x5589559955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.179+0000 7fd13bfff640 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989ba0 0x5589559955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.179+0000 7fd13bfff640 1 -- 172.21.15.33:0/1316182278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955989280 msgr2=0x558955989660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.179+0000 7fd13bfff640 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955989280 0x558955989660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.179+0000 7fd13bfff640 1 -- 172.21.15.33:0/1316182278 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55895564d650 con 0x558955995b30 2024-09-10T21:24:54.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.179+0000 7fd13bfff640 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955995b30 0x558955997f20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd1240ebc20 tx=0x7fd1240ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5dfc0dc126c63e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.180+0000 7fd132ffd640 1 -- 172.21.15.33:0/1316182278 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240f4070 con 0x558955995b30 2024-09-10T21:24:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.180+0000 7fd132ffd640 1 -- 172.21.15.33:0/1316182278 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1240e5de0 con 0x558955995b30 2024-09-10T21:24:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.181+0000 7fd14547f740 1 -- 172.21.15.33:0/1316182278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955995b30 msgr2=0x558955997f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.181+0000 7fd14547f740 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955995b30 0x558955997f20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd1240ebc20 tx=0x7fd1240ed670 comp rx=0 tx=0).stop 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd139ffb640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1280b55b0 0x7fd1280b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd139ffb640 1 -- 172.21.15.33:0/2104711764 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd1240e5830 con 0x7fd13c10ec50 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd14547f740 1 -- 172.21.15.33:0/1316182278 shutdown_connections 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd14547f740 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955995b30 0x558955997f20 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd14547f740 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989ba0 0x5589559955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd14547f740 1 --2- 172.21.15.33:0/1316182278 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955989280 0x558955989660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd14547f740 1 -- 172.21.15.33:0/1316182278 >> 172.21.15.33:0/1316182278 conn(0x5589559870b0 msgr2=0x558955988250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.183+0000 7fd141647640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1280b55b0 0x7fd1280b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.184+0000 7fd14547f740 1 -- 172.21.15.33:0/1316182278 shutdown_connections 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.184+0000 7fd14547f740 1 -- 172.21.15.33:0/1316182278 wait complete. 2024-09-10T21:24:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.184+0000 7fd14547f740 1 -- start start 2024-09-10T21:24:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.184+0000 7fd14547f740 1 Processor -- start 2024-09-10T21:24:54.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd141647640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1280b55b0 0x7fd1280b7a70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd1340096c0 tx=0x7fd134031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd14547f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989280 0x5589559954e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd14547f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989ba0 0x558955992930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd14547f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955995b30 0x558955993000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd14547f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558955750700 con 0x558955989ba0 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd14547f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55895588a110 con 0x558955995b30 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd14547f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558955998800 con 0x558955989280 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.185+0000 7fd14547f740 10 client.0 _ll_register_callbacks cb 0x5589558a3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989ba0 0x558955992930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989ba0 0x558955992930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59974/0 (socket says 172.21.15.33:59974) 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 -- 172.21.15.33:0/982594425 learned_addr learned my addr 172.21.15.33:0/982594425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd141647640 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955995b30 0x558955993000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 -- 172.21.15.33:0/982594425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989280 msgr2=0x5589559954e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989280 0x5589559954e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 -- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955995b30 msgr2=0x558955993000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955995b30 0x558955993000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.186+0000 7fd13bfff640 1 -- 172.21.15.33:0/982594425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589558b7c40 con 0x558955989ba0 2024-09-10T21:24:54.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.190+0000 7fd13bfff640 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989ba0 0x558955992930 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd124009fd0 tx=0x7fd1240eb240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36b6eb576b839e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.190+0000 7fd11bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:24:54.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.191+0000 7fd11bfff640 1 -- 172.21.15.33:0/982594425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240f4070 con 0x558955989ba0 2024-09-10T21:24:54.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.191+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558955901da0 con 0x558955989ba0 2024-09-10T21:24:54.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.191+0000 7fd11bfff640 1 -- 172.21.15.33:0/982594425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd1240ac210 con 0x558955989ba0 2024-09-10T21:24:54.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.191+0000 7fd11bfff640 1 -- 172.21.15.33:0/982594425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fd1240a8600 con 0x558955989ba0 2024-09-10T21:24:54.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.194+0000 7fd14547f740 10 client.19164 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:54.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.194+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x5589559a4350 con 0x558955989ba0 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd11bfff640 1 -- 172.21.15.33:0/982594425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558955901da0 con 0x558955989ba0 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd11bfff640 1 -- 172.21.15.33:0/982594425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fd1240971a0 con 0x558955989ba0 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd14547f740 10 client.19164 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd14547f740 10 client.19164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd14547f740 20 client.19164 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd14547f740 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589559a5800 0x5589559a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd14547f740 4 client.19164 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:54.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.201+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589558fd850 con 0x5589559a5800 2024-09-10T21:24:54.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.208+0000 7fd140e46640 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589559a5800 0x5589559a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.220+0000 7fd140e46640 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589559a5800 0x5589559a7c80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56ddd3da9f896c5c server_cookie=439e226bf754832d in_seq=0 out_seq=0 2024-09-10T21:24:54.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.220+0000 7fd11bfff640 10 client.19164 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:54.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.222+0000 7fd11bfff640 1 -- 172.21.15.33:0/982594425 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589558fd850 con 0x5589559a5800 2024-09-10T21:24:54.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.222+0000 7fd11bfff640 10 client.19164 handle_command_reply: tid=0 2024-09-10T21:24:54.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.224+0000 7fd14547f740 1 client.19164 shutdown 2024-09-10T21:24:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.224+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989ba0 msgr2=0x558955992930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.224+0000 7fd14547f740 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989ba0 0x558955992930 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd124009fd0 tx=0x7fd1240eb240 comp rx=0 tx=0).stop 2024-09-10T21:24:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 shutdown_connections 2024-09-10T21:24:54.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd14547f740 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5589559a5800 0x5589559a7c80 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd14547f740 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558955995b30 0x558955993000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd14547f740 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989ba0 0x558955992930 secure :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fd124009fd0 tx=0x7fd1240eb240 comp rx=0 tx=0).stop 2024-09-10T21:24:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd14547f740 1 --2- 172.21.15.33:0/982594425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955989280 0x5589559954e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 >> 172.21.15.33:0/982594425 conn(0x5589559870b0 msgr2=0x55895598cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd11bfff640 0 client.19164 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:54.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.227+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 shutdown_connections 2024-09-10T21:24:54.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.228+0000 7fd14547f740 1 -- 172.21.15.33:0/982594425 wait complete. 2024-09-10T21:24:54.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.228+0000 7fd14547f740 20 client.19164 trim_cache size 0 max 16384 2024-09-10T21:24:54.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.265+0000 7fd14547f740 1 Processor -- start 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.265+0000 7fd14547f740 1 -- start start 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.265+0000 7fd14547f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd14547f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd14547f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955991900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd14547f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1bd190 con 0x558955989280 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd14547f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1bed90 con 0x55895598e250 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd14547f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1bd920 con 0x558955995b30 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd141647640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955991900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd141647640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955991900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57558/0 (socket says 172.21.15.33:57558) 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd141647640 1 -- 172.21.15.33:0/3340489153 learned_addr learned my addr 172.21.15.33:0/3340489153 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:54.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd13bfff640 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd140e46640 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd13bfff640 1 -- 172.21.15.33:0/3340489153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 msgr2=0x558955991900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:54.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd13bfff640 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955991900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd13bfff640 1 -- 172.21.15.33:0/3340489153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 msgr2=0x5589559a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.266+0000 7fd13bfff640 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559a7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.267+0000 7fd13bfff640 1 -- 172.21.15.33:0/3340489153 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558955992d90 con 0x558955995b30 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.267+0000 7fd13bfff640 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559a1ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558955992140 tx=0x7fd1240ed670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d16249f3d244a3cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.268+0000 7fd1327fc640 1 -- 172.21.15.33:0/3340489153 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240f4070 con 0x558955995b30 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.268+0000 7fd1327fc640 1 -- 172.21.15.33:0/3340489153 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1240eba70 con 0x558955995b30 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.268+0000 7fd1327fc640 1 -- 172.21.15.33:0/3340489153 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240edea0 con 0x558955995b30 2024-09-10T21:24:54.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.269+0000 7fd14547f740 1 -- 172.21.15.33:0/3340489153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 msgr2=0x5589559a1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.269+0000 7fd14547f740 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559a1ae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x558955992140 tx=0x7fd1240ed670 comp rx=0 tx=0).stop 2024-09-10T21:24:54.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.269+0000 7fd14547f740 1 -- 172.21.15.33:0/3340489153 shutdown_connections 2024-09-10T21:24:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.269+0000 7fd14547f740 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955991900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.269+0000 7fd14547f740 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559a1ae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.269+0000 7fd14547f740 1 --2- 172.21.15.33:0/3340489153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.269+0000 7fd14547f740 1 -- 172.21.15.33:0/3340489153 >> 172.21.15.33:0/3340489153 conn(0x5589559870b0 msgr2=0x5589559911f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.270+0000 7fd14547f740 1 -- 172.21.15.33:0/3340489153 shutdown_connections 2024-09-10T21:24:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.270+0000 7fd14547f740 1 -- 172.21.15.33:0/3340489153 wait complete. 2024-09-10T21:24:54.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.270+0000 7fd14547f740 1 -- start start 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 1 Processor -- start 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559902b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955996ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1be6c0 con 0x558955989280 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1bdff0 con 0x55895598e250 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd13c1bb150 con 0x558955995b30 2024-09-10T21:24:54.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd14547f740 10 client.0 _ll_register_callbacks cb 0x7fd12c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955996ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd13bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955996ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:57572/0 (socket says 172.21.15.33:57572) 2024-09-10T21:24:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.271+0000 7fd13bfff640 1 -- 172.21.15.33:0/1019545096 learned_addr learned my addr 172.21.15.33:0/1019545096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:24:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.272+0000 7fd141647640 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.272+0000 7fd140e46640 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559902b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.272+0000 7fd13bfff640 1 -- 172.21.15.33:0/1019545096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 msgr2=0x5589559971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:54.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.272+0000 7fd13bfff640 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.272+0000 7fd13bfff640 1 -- 172.21.15.33:0/1019545096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 msgr2=0x5589559902b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:54.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.272+0000 7fd13bfff640 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559902b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.272+0000 7fd13bfff640 1 -- 172.21.15.33:0/1019545096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558955997920 con 0x55895598e250 2024-09-10T21:24:54.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.273+0000 7fd13bfff640 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955996ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd12400be10 tx=0x7fd1240eb400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d768b314d804956 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.273+0000 7fd11bfff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:24:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.274+0000 7fd11bfff640 1 -- 172.21.15.33:0/1019545096 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240f4050 con 0x55895598e250 2024-09-10T21:24:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.274+0000 7fd11bfff640 1 -- 172.21.15.33:0/1019545096 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fd1240b50b0 con 0x55895598e250 2024-09-10T21:24:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.274+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1240eba70 con 0x55895598e250 2024-09-10T21:24:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.274+0000 7fd11bfff640 1 -- 172.21.15.33:0/1019545096 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fd1240ab720 con 0x55895598e250 2024-09-10T21:24:54.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.275+0000 7fd14547f740 10 client.28526 fetch_fsmap learned FSMap version 84 2024-09-10T21:24:54.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.275+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x558955997920 con 0x55895598e250 2024-09-10T21:24:54.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.275+0000 7fd11bfff640 1 -- 172.21.15.33:0/1019545096 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1240eba70 con 0x55895598e250 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd11bfff640 1 -- 172.21.15.33:0/1019545096 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fd1240c1b30 con 0x55895598e250 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd14547f740 10 client.28526 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd14547f740 10 client.28526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd14547f740 20 client.28526 populate_metadata read hostname 'smithi033' 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd14547f740 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558955a128b0 0x558955a14ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd14547f740 4 client.28526 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5589558fac80 con 0x558955a128b0 2024-09-10T21:24:54.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.276+0000 7fd140e46640 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558955a128b0 0x558955a14ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.277+0000 7fd140e46640 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558955a128b0 0x558955a14ca0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec1ef3e017f467e server_cookie=dfe6b86e49155199 in_seq=0 out_seq=0 2024-09-10T21:24:54.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.277+0000 7fd11bfff640 10 client.28526 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd11bfff640 1 -- 172.21.15.33:0/1019545096 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7954 (crc 0 0 0) 0x5589558fac80 con 0x558955a128b0 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd11bfff640 10 client.28526 handle_command_reply: tid=0 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 client.28526 shutdown 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 msgr2=0x558955996ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955996ca0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd12400be10 tx=0x7fd1240eb400 comp rx=0 tx=0).stop 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 shutdown_connections 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558955a128b0 0x558955a14ca0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558955995b30 0x5589559971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55895598e250 0x558955996ca0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd11bfff640 0 client.28526 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 --2- 172.21.15.33:0/1019545096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558955989280 0x5589559902b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 >> 172.21.15.33:0/1019545096 conn(0x5589559870b0 msgr2=0x558955991a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 shutdown_connections 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 1 -- 172.21.15.33:0/1019545096 wait complete. 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.278+0000 7fd14547f740 20 client.28526 trim_cache size 0 max 16384 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:24:54.283 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07", 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.284 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:24:54.285 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.287 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:24:54.288 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.289 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.291 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:24:54.292 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.293 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.294 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:24:54.295 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.296 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:24:54.298 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.299 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:24:54.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:24:54.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:24:54.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:54.301 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:24:54.301 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:24:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1280b55b0 msgr2=0x7fd1280b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1280b55b0 0x7fd1280b7a70 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd1340096c0 tx=0x7fd134031400 comp rx=0 tx=0).stop 2024-09-10T21:24:54.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10ec50 msgr2=0x7fd13c074c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10ec50 0x7fd13c074c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd12400b450 tx=0x7fd12400b910 comp rx=0 tx=0).stop 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 shutdown_connections 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fd1280b55b0 0x7fd1280b7a70 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd13c10f5d0 0x7fd13c06da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd13c10ec50 0x7fd13c074c90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 --2- 172.21.15.33:0/2104711764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd13c109250 0x7fd13c074750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 >> 172.21.15.33:0/2104711764 conn(0x7fd13c078900 msgr2=0x7fd13c07acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 shutdown_connections 2024-09-10T21:24:54.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:24:54.280+0000 7fd141e48640 1 -- 172.21.15.33:0/2104711764 wait complete. 2024-09-10T21:24:55.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 361306 39.25 MB/sec execute 355 sec latency 256.662 ms 2024-09-10T21:24:55.547 INFO:tasks.cephfs.filesystem:scrub status for tag:4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07 - {'path': '/', 'tag': '4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07', 'options': 'recursive,force'} 2024-09-10T21:24:55.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:55 smithi104 ceph-mon[35325]: pgmap v974: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:24:55.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:55 smithi174 ceph-mon[35635]: pgmap v974: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:24:55.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:55 smithi033 ceph-mon[30860]: pgmap v974: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 891 op/s 2024-09-10T21:24:56.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 362014 39.23 MB/sec execute 356 sec latency 144.727 ms 2024-09-10T21:24:57.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 362784 39.25 MB/sec execute 357 sec latency 91.361 ms 2024-09-10T21:24:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:57 smithi104 ceph-mon[35325]: pgmap v975: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.2 MiB/s wr, 709 op/s 2024-09-10T21:24:57.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:57 smithi174 ceph-mon[35635]: pgmap v975: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.2 MiB/s wr, 709 op/s 2024-09-10T21:24:57.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:57 smithi033 ceph-mon[30860]: pgmap v975: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.2 MiB/s wr, 709 op/s 2024-09-10T21:24:58.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 363494 39.23 MB/sec execute 358 sec latency 62.648 ms 2024-09-10T21:24:59.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 364318 39.25 MB/sec execute 359 sec latency 85.173 ms 2024-09-10T21:24:59.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:24:59 smithi104 ceph-mon[35325]: pgmap v976: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.3 MiB/s wr, 766 op/s 2024-09-10T21:24:59.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:24:59 smithi174 ceph-mon[35635]: pgmap v976: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.3 MiB/s wr, 766 op/s 2024-09-10T21:24:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:24:59 smithi033 ceph-mon[30860]: pgmap v976: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.3 MiB/s wr, 766 op/s 2024-09-10T21:25:00.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 364883 39.24 MB/sec execute 360 sec latency 288.247 ms 2024-09-10T21:25:01.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 365624 39.25 MB/sec execute 361 sec latency 71.459 ms 2024-09-10T21:25:01.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:01 smithi104 ceph-mon[35325]: pgmap v977: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 838 op/s 2024-09-10T21:25:01.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:01 smithi174 ceph-mon[35635]: pgmap v977: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 838 op/s 2024-09-10T21:25:01.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:01 smithi033 ceph-mon[30860]: pgmap v977: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 10 MiB/s wr, 838 op/s 2024-09-10T21:25:02.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 365999 39.20 MB/sec execute 362 sec latency 398.154 ms 2024-09-10T21:25:03.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 366026 39.09 MB/sec execute 363 sec latency 1346.117 ms 2024-09-10T21:25:03.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:03 smithi104 ceph-mon[35325]: pgmap v978: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 751 op/s 2024-09-10T21:25:03.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:03 smithi174 ceph-mon[35635]: pgmap v978: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 751 op/s 2024-09-10T21:25:03.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:03 smithi033 ceph-mon[30860]: pgmap v978: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 751 op/s 2024-09-10T21:25:04.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 366228 39.01 MB/sec execute 364 sec latency 1617.422 ms 2024-09-10T21:25:05.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 366464 38.93 MB/sec execute 365 sec latency 139.778 ms 2024-09-10T21:25:05.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:05 smithi104 ceph-mon[35325]: pgmap v979: 97 pgs: 97 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-09-10T21:25:05.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:05 smithi174 ceph-mon[35635]: pgmap v979: 97 pgs: 97 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-09-10T21:25:05.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:05 smithi033 ceph-mon[30860]: pgmap v979: 97 pgs: 97 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 18 MiB/s wr, 1.02k op/s 2024-09-10T21:25:06.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 367044 38.91 MB/sec execute 366 sec latency 100.729 ms 2024-09-10T21:25:06.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:06 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:06.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:06 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:06.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:06 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:07.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 367819 38.88 MB/sec execute 367 sec latency 223.808 ms 2024-09-10T21:25:07.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:07 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:07.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:07 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:07.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:07 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:07.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:07 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:07.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:07 smithi104 ceph-mon[35325]: pgmap v980: 97 pgs: 97 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 901 op/s 2024-09-10T21:25:07.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:07 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:07 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:07 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:07 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:07 smithi174 ceph-mon[35635]: pgmap v980: 97 pgs: 97 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 901 op/s 2024-09-10T21:25:07.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:07 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:07.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:07 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:07.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:07 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:07.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:07 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:07.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:07 smithi033 ceph-mon[30860]: pgmap v980: 97 pgs: 97 active+clean; 1.3 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 20 MiB/s wr, 901 op/s 2024-09-10T21:25:08.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 368703 38.94 MB/sec execute 368 sec latency 213.752 ms 2024-09-10T21:25:09.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 369154 38.94 MB/sec execute 369 sec latency 248.054 ms 2024-09-10T21:25:09.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:09 smithi104 ceph-mon[35325]: pgmap v981: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 995 op/s 2024-09-10T21:25:09.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:09 smithi174 ceph-mon[35635]: pgmap v981: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 995 op/s 2024-09-10T21:25:09.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:09 smithi033 ceph-mon[30860]: pgmap v981: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 21 MiB/s wr, 995 op/s 2024-09-10T21:25:10.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 369739 38.94 MB/sec execute 370 sec latency 77.158 ms 2024-09-10T21:25:11.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 370260 38.90 MB/sec execute 371 sec latency 144.860 ms 2024-09-10T21:25:11.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:11 smithi104 ceph-mon[35325]: pgmap v982: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-10T21:25:11.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:11 smithi174 ceph-mon[35635]: pgmap v982: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-10T21:25:11.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:11 smithi033 ceph-mon[30860]: pgmap v982: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 23 MiB/s wr, 1.20k op/s 2024-09-10T21:25:12.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 370827 38.86 MB/sec execute 372 sec latency 290.693 ms 2024-09-10T21:25:13.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 371598 38.87 MB/sec execute 373 sec latency 72.027 ms 2024-09-10T21:25:13.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:13 smithi104 ceph-mon[35325]: pgmap v983: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-09-10T21:25:13.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:13 smithi174 ceph-mon[35635]: pgmap v983: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-09-10T21:25:13.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:13 smithi033 ceph-mon[30860]: pgmap v983: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 21 MiB/s wr, 1.01k op/s 2024-09-10T21:25:14.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 372457 38.92 MB/sec execute 374 sec latency 56.028 ms 2024-09-10T21:25:15.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 373122 38.90 MB/sec execute 375 sec latency 181.542 ms 2024-09-10T21:25:15.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:15 smithi104 ceph-mon[35325]: pgmap v984: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 1.13k op/s 2024-09-10T21:25:15.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:15 smithi174 ceph-mon[35635]: pgmap v984: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 1.13k op/s 2024-09-10T21:25:15.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:15 smithi033 ceph-mon[30860]: pgmap v984: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 24 MiB/s wr, 1.13k op/s 2024-09-10T21:25:16.073 INFO:tasks.workunit.client.0.smithi033.stdout: 10 373668 38.86 MB/sec execute 376 sec latency 331.287 ms 2024-09-10T21:25:17.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 374419 38.86 MB/sec execute 377 sec latency 115.387 ms 2024-09-10T21:25:17.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:17 smithi104 ceph-mon[35325]: pgmap v985: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 932 op/s 2024-09-10T21:25:17.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:17 smithi174 ceph-mon[35635]: pgmap v985: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 932 op/s 2024-09-10T21:25:17.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:17 smithi033 ceph-mon[30860]: pgmap v985: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 16 MiB/s wr, 932 op/s 2024-09-10T21:25:18.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 375236 38.90 MB/sec execute 378 sec latency 126.818 ms 2024-09-10T21:25:19.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 375724 38.87 MB/sec execute 379 sec latency 389.514 ms 2024-09-10T21:25:19.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:19 smithi104 ceph-mon[35325]: pgmap v986: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 879 op/s 2024-09-10T21:25:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:19 smithi174 ceph-mon[35635]: pgmap v986: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 879 op/s 2024-09-10T21:25:19.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:19 smithi033 ceph-mon[30860]: pgmap v986: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 879 op/s 2024-09-10T21:25:20.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 376426 38.85 MB/sec execute 380 sec latency 73.341 ms 2024-09-10T21:25:21.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 377256 38.86 MB/sec execute 381 sec latency 75.243 ms 2024-09-10T21:25:21.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:21 smithi104 ceph-mon[35325]: pgmap v987: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 929 op/s 2024-09-10T21:25:21.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:21 smithi174 ceph-mon[35635]: pgmap v987: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 929 op/s 2024-09-10T21:25:21.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:21 smithi033 ceph-mon[30860]: pgmap v987: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 929 op/s 2024-09-10T21:25:22.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 378009 38.87 MB/sec execute 382 sec latency 189.489 ms 2024-09-10T21:25:23.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 378593 38.84 MB/sec execute 383 sec latency 292.152 ms 2024-09-10T21:25:23.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:23 smithi104 ceph-mon[35325]: pgmap v988: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 667 op/s 2024-09-10T21:25:23.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:23 smithi174 ceph-mon[35635]: pgmap v988: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 667 op/s 2024-09-10T21:25:23.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:23 smithi033 ceph-mon[30860]: pgmap v988: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 12 MiB/s wr, 667 op/s 2024-09-10T21:25:24.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 379300 38.85 MB/sec execute 384 sec latency 71.200 ms 2024-09-10T21:25:25.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 380023 38.87 MB/sec execute 385 sec latency 89.483 ms 2024-09-10T21:25:25.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:25:25.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:25 smithi104 ceph-mon[35325]: pgmap v989: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-09-10T21:25:25.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:25 smithi174 ceph-mon[35635]: pgmap v989: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-09-10T21:25:25.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:25 smithi033 ceph-mon[30860]: pgmap v989: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-09-10T21:25:26.020 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:26.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 380629 38.83 MB/sec execute 386 sec latency 286.428 ms 2024-09-10T21:25:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 -- 172.21.15.33:0/1869573310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c0102da0 msgr2=0x7f76c0103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 --2- 172.21.15.33:0/1869573310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c0102da0 0x7f76c0103180 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009900 tx=0x7f76bc02a410 comp rx=0 tx=0).stop 2024-09-10T21:25:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 -- 172.21.15.33:0/1869573310 shutdown_connections 2024-09-10T21:25:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 --2- 172.21.15.33:0/1869573310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c0109130 0x7f76c010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 --2- 172.21.15.33:0/1869573310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76c01087b0 0x7f76c0108bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 --2- 172.21.15.33:0/1869573310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c0102da0 0x7f76c0103180 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 -- 172.21.15.33:0/1869573310 >> 172.21.15.33:0/1869573310 conn(0x7f76c00fe850 msgr2=0x7f76c0100c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 -- 172.21.15.33:0/1869573310 shutdown_connections 2024-09-10T21:25:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.917+0000 7f76c8507640 1 -- 172.21.15.33:0/1869573310 wait complete. 2024-09-10T21:25:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 Processor -- start 2024-09-10T21:25:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 -- start start 2024-09-10T21:25:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76c0102da0 0x7f76c0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01087b0 0x7f76c01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c0109130 0x7f76c019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112e00 con 0x7f76c01087b0 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112c80 con 0x7f76c0109130 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c8507640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76c0112f80 con 0x7f76c0102da0 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c5aa2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01087b0 0x7f76c01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c5aa2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01087b0 0x7f76c01997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55530/0 (socket says 172.21.15.33:55530) 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c5aa2640 1 -- 172.21.15.33:0/2057560336 learned_addr learned my addr 172.21.15.33:0/2057560336 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.918+0000 7f76c62a3640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76c0102da0 0x7f76c0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c5aa2640 1 -- 172.21.15.33:0/2057560336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76c0102da0 msgr2=0x7f76c0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76c0102da0 0x7f76c0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c5aa2640 1 -- 172.21.15.33:0/2057560336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c0109130 msgr2=0x7f76c019eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c0109130 0x7f76c019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c5aa2640 1 -- 172.21.15.33:0/2057560336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c019f320 con 0x7f76c01087b0 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01087b0 0x7f76c01997b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f76b000e9f0 tx=0x7f76b000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76b77fe640 1 -- 172.21.15.33:0/2057560336 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b000cd80 con 0x7f76c01087b0 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76b77fe640 1 -- 172.21.15.33:0/2057560336 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b00108e0 con 0x7f76c01087b0 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c019f610 con 0x7f76c01087b0 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76b77fe640 1 -- 172.21.15.33:0/2057560336 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b0018980 con 0x7f76c01087b0 2024-09-10T21:25:26.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.919+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c01a5cd0 con 0x7f76c01087b0 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 Processor -- start 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 -- start start 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76b77fe640 1 -- 172.21.15.33:0/2057560336 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f76b0010430 con 0x7f76c01087b0 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b70d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70dda0 0x56269b7197f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b71c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56269b683e50 con 0x56269b70dda0 2024-09-10T21:25:26.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56269b673280 con 0x56269b70d480 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.922+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56269b677090 con 0x56269b719d30 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b71c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b71c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:60350/0 (socket says 172.21.15.33:60350) 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 -- 172.21.15.33:0/1858390984 learned_addr learned my addr 172.21.15.33:0/1858390984 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b70d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c62a3640 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70dda0 0x56269b7197f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 -- 172.21.15.33:0/1858390984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 msgr2=0x56269b70d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b70d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 -- 172.21.15.33:0/1858390984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70dda0 msgr2=0x56269b7197f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70dda0 0x56269b7197f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 -- 172.21.15.33:0/1858390984 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56269b3d1650 con 0x56269b719d30 2024-09-10T21:25:26.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.923+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b71c120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76b00ee190 tx=0x7f76b00ee650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb13334cc6eb685 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.924+0000 7f76b77fe640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f76ac0b52e0 0x7f76ac0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.924+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f76ac0b52e0 0x7f76ac0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.924+0000 7f76b4ff9640 1 -- 172.21.15.33:0/1858390984 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b00f9070 con 0x56269b719d30 2024-09-10T21:25:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.924+0000 7f76b77fe640 1 -- 172.21.15.33:0/2057560336 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f76b00e5f70 con 0x7f76c01087b0 2024-09-10T21:25:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.924+0000 7f76b4ff9640 1 -- 172.21.15.33:0/1858390984 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b00ac240 con 0x56269b719d30 2024-09-10T21:25:26.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.924+0000 7f76b4ff9640 1 -- 172.21.15.33:0/1858390984 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b00a85d0 con 0x56269b719d30 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1858390984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 msgr2=0x56269b71c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f76ac0b52e0 0x7f76ac0b77a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009270 tx=0x7f76b8009200 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b71c120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f76b00ee190 tx=0x7f76b00ee650 comp rx=0 tx=0).stop 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1858390984 shutdown_connections 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b71c120 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70dda0 0x56269b7197f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1858390984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b70d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1858390984 >> 172.21.15.33:0/1858390984 conn(0x56269b70b2b0 msgr2=0x56269b70c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1858390984 shutdown_connections 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1858390984 wait complete. 2024-09-10T21:25:26.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.925+0000 7f76ca8dc740 1 -- start start 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 1 Processor -- start 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70d480 0x56269b719640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b716a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b7170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56269b4d4700 con 0x56269b70d480 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56269b60e110 con 0x56269b70dda0 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56269b71c9b0 con 0x56269b719d30 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76ca8dc740 10 client.0 _ll_register_callbacks cb 0x56269b627990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76c62a3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70d480 0x56269b719640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76c6aa4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b7170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76c6aa4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b7170c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:60356/0 (socket says 172.21.15.33:60356) 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76c6aa4640 1 -- 172.21.15.33:0/3642202944 learned_addr learned my addr 172.21.15.33:0/3642202944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:26.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.926+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b716a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76c6aa4640 1 -- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 msgr2=0x56269b716a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b716a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76c6aa4640 1 -- 172.21.15.33:0/3642202944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70d480 msgr2=0x56269b719640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70d480 0x56269b719640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76c6aa4640 1 -- 172.21.15.33:0/3642202944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56269b63bc40 con 0x56269b719d30 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76c62a3640 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70d480 0x56269b719640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b7170c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f76b8003f50 tx=0x7f76b8004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf9c5b3465e10c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76a9ffb640 1 -- 172.21.15.33:0/3642202944 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b800f950 con 0x56269b719d30 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76a9ffb640 1 -- 172.21.15.33:0/3642202944 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f76b800faf0 con 0x56269b719d30 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.927+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76b00ac240 con 0x56269b719d30 2024-09-10T21:25:26.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76a9ffb640 1 -- 172.21.15.33:0/3642202944 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f76b800fcf0 con 0x56269b719d30 2024-09-10T21:25:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 10 client.28420 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56269b728370 con 0x56269b719d30 2024-09-10T21:25:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76a9ffb640 1 -- 172.21.15.33:0/3642202944 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b00ac240 con 0x56269b719d30 2024-09-10T21:25:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76a9ffb640 1 -- 172.21.15.33:0/3642202944 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f76b8026400 con 0x56269b719d30 2024-09-10T21:25:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 10 client.28420 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:26.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 10 client.28420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 20 client.28420 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b729830 0x56269b72bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 4 client.28420 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.928+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56269b685da0 con 0x56269b729830 2024-09-10T21:25:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.929+0000 7f76c62a3640 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b729830 0x56269b72bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.929+0000 7f76c62a3640 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b729830 0x56269b72bc70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=674648e8d1a1bee2 server_cookie=623ec87186b9b254 in_seq=0 out_seq=0 2024-09-10T21:25:26.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.929+0000 7f76a9ffb640 10 client.28420 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76a9ffb640 1 -- 172.21.15.33:0/3642202944 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56269b685da0 con 0x56269b729830 2024-09-10T21:25:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76a9ffb640 10 client.28420 handle_command_reply: tid=0 2024-09-10T21:25:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 client.28420 shutdown 2024-09-10T21:25:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 msgr2=0x56269b7170c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b7170c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f76b8003f50 tx=0x7f76b8004a40 comp rx=0 tx=0).stop 2024-09-10T21:25:26.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 shutdown_connections 2024-09-10T21:25:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b729830 0x56269b72bc70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b719d30 0x56269b7170c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b716a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/3642202944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b70d480 0x56269b719640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 >> 172.21.15.33:0/3642202944 conn(0x56269b70b2b0 msgr2=0x56269b710e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76a9ffb640 0 client.28420 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 shutdown_connections 2024-09-10T21:25:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 1 -- 172.21.15.33:0/3642202944 wait complete. 2024-09-10T21:25:26.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.933+0000 7f76ca8dc740 20 client.28420 trim_cache size 0 max 16384 2024-09-10T21:25:26.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 Processor -- start 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 -- start start 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70d480 0x56269b725be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b7062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b70a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01ae4e0 con 0x56269b719d30 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01b00e0 con 0x56269b70dda0 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01aec70 con 0x56269b70d480 2024-09-10T21:25:26.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76c6aa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b70a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76c6aa4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b70a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55552/0 (socket says 172.21.15.33:55552) 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76c6aa4640 1 -- 172.21.15.33:0/237529631 learned_addr learned my addr 172.21.15.33:0/237529631 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76c62a3640 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70d480 0x56269b725be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.953+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b7062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76c6aa4640 1 -- 172.21.15.33:0/237529631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70d480 msgr2=0x56269b725be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70d480 0x56269b725be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76c6aa4640 1 -- 172.21.15.33:0/237529631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 msgr2=0x56269b7062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b7062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76c6aa4640 1 -- 172.21.15.33:0/237529631 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56269b70cde0 con 0x56269b719d30 2024-09-10T21:25:26.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b70a2c0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000f80 tx=0x7f76b8009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31cd7e7c8d58a1a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76abfff640 1 -- 172.21.15.33:0/237529631 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b803e070 con 0x56269b719d30 2024-09-10T21:25:26.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76abfff640 1 -- 172.21.15.33:0/237529631 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b8003d10 con 0x56269b719d30 2024-09-10T21:25:26.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.954+0000 7f76abfff640 1 -- 172.21.15.33:0/237529631 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b8035040 con 0x56269b719d30 2024-09-10T21:25:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 -- 172.21.15.33:0/237529631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 msgr2=0x56269b70a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b70a2c0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f76b8000f80 tx=0x7f76b8009e00 comp rx=0 tx=0).stop 2024-09-10T21:25:26.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 -- 172.21.15.33:0/237529631 shutdown_connections 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b70a2c0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70dda0 0x56269b7062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/237529631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70d480 0x56269b725be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 -- 172.21.15.33:0/237529631 >> 172.21.15.33:0/237529631 conn(0x56269b70b2b0 msgr2=0x56269b709bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 -- 172.21.15.33:0/237529631 shutdown_connections 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.955+0000 7f76ca8dc740 1 -- 172.21.15.33:0/237529631 wait complete. 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 -- start start 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 Processor -- start 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b721ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70dda0 0x56269b722480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b71cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01afa10 con 0x56269b719d30 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01af340 con 0x56269b70d480 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76c01ac4a0 con 0x56269b70dda0 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.956+0000 7f76ca8dc740 10 client.0 _ll_register_callbacks cb 0x7f76b8009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b721ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b721ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:55858/0 (socket says 172.21.15.33:55858) 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 -- 172.21.15.33:0/1828392727 learned_addr learned my addr 172.21.15.33:0/1828392727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c6aa4640 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b71cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 -- 172.21.15.33:0/1828392727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70dda0 msgr2=0x56269b722480 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70dda0 0x56269b722480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 -- 172.21.15.33:0/1828392727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 msgr2=0x56269b71cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:26.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b71cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.957+0000 7f76c62a3640 1 -- 172.21.15.33:0/1828392727 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56269b71d490 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.958+0000 7f76c62a3640 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b721ee0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000c00 tx=0x7f76bc004540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e00a90b02b3ebcb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.958+0000 7f76a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.958+0000 7f76a9ffb640 1 -- 172.21.15.33:0/1828392727 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76bc035040 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.958+0000 7f76a9ffb640 1 -- 172.21.15.33:0/1828392727 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f76bc005590 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.958+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56269b681850 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.959+0000 7f76a9ffb640 1 -- 172.21.15.33:0/1828392727 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f76bc004770 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.959+0000 7f76ca8dc740 10 client.28541 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.959+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56269b797330 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.959+0000 7f76a9ffb640 1 -- 172.21.15.33:0/1828392727 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56269b681850 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.959+0000 7f76a9ffb640 1 -- 172.21.15.33:0/1828392727 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f76bc003c10 con 0x56269b70d480 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.959+0000 7f76ca8dc740 10 client.28541 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.959+0000 7f76ca8dc740 10 client.28541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.960+0000 7f76ca8dc740 20 client.28541 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.960+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b797ab0 0x56269b799ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.960+0000 7f76ca8dc740 4 client.28541 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.960+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56269b67ec80 con 0x56269b797ab0 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.960+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b797ab0 0x56269b799ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.961+0000 7f76c5aa2640 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b797ab0 0x56269b799ed0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e67a900657247248 server_cookie=4b0deb16d153cb96 in_seq=0 out_seq=0 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.961+0000 7f76a9ffb640 10 client.28541 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:26.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.966+0000 7f76a9ffb640 1 -- 172.21.15.33:0/1828392727 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7758 (crc 0 0 0) 0x56269b67ec80 con 0x56269b797ab0 2024-09-10T21:25:26.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76a9ffb640 10 client.28541 handle_command_reply: tid=0 2024-09-10T21:25:26.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 client.28541 shutdown 2024-09-10T21:25:26.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 msgr2=0x56269b721ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b721ee0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000c00 tx=0x7f76bc004540 comp rx=0 tx=0).stop 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 shutdown_connections 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56269b797ab0 0x56269b799ed0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56269b719d30 0x56269b71cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56269b70dda0 0x56269b722480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 --2- 172.21.15.33:0/1828392727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56269b70d480 0x56269b721ee0 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f76bc000c00 tx=0x7f76bc004540 comp rx=0 tx=0).stop 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.967+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 >> 172.21.15.33:0/1828392727 conn(0x56269b70b2b0 msgr2=0x56269b7188e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.968+0000 7f76a9ffb640 0 client.28541 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.968+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 shutdown_connections 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.968+0000 7f76ca8dc740 1 -- 172.21.15.33:0/1828392727 wait complete. 2024-09-10T21:25:26.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.968+0000 7f76ca8dc740 20 client.28541 trim_cache size 0 max 16384 2024-09-10T21:25:26.972 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:25:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:25:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:25:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:25:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:25:26.972 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.973 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.974 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:25:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:25:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.975 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:25:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:25:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.976 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:25:26.978 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.979 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:25:26.980 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.981 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:25:26.982 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.983 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.984 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:25:26.985 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:25:26.986 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.988 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:25:26.993 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:25:26.994 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:25:26.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f76ac0b52e0 msgr2=0x7f76ac0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f76ac0b52e0 0x7f76ac0b77a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009270 tx=0x7f76b8009200 comp rx=0 tx=0).stop 2024-09-10T21:25:26.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01087b0 msgr2=0x7f76c01997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01087b0 0x7f76c01997b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f76b000e9f0 tx=0x7f76b000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 shutdown_connections 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f76ac0b52e0 0x7f76ac0b77a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f76c0109130 0x7f76c019eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f76c01087b0 0x7f76c01997b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 --2- 172.21.15.33:0/2057560336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76c0102da0 0x7f76c0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.970+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 >> 172.21.15.33:0/2057560336 conn(0x7f76c00fe850 msgr2=0x7f76c0100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.971+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 shutdown_connections 2024-09-10T21:25:26.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:26.971+0000 7f76c8507640 1 -- 172.21.15.33:0/2057560336 wait complete. 2024-09-10T21:25:27.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 381232 38.83 MB/sec execute 387 sec latency 311.169 ms 2024-09-10T21:25:27.798 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:27 smithi033 ceph-mon[30860]: pgmap v990: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 758 op/s 2024-09-10T21:25:27.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:27 smithi104 ceph-mon[35325]: pgmap v990: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 758 op/s 2024-09-10T21:25:27.838 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:25:27.839 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:25:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:27 smithi174 ceph-mon[35635]: pgmap v990: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 758 op/s 2024-09-10T21:25:28.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 381790 38.79 MB/sec execute 388 sec latency 144.274 ms 2024-09-10T21:25:28.306 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:28.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.826+0000 7fe466814640 1 -- 172.21.15.33:0/1484904650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 msgr2=0x7fe4601053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:28.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.826+0000 7fe466814640 1 --2- 172.21.15.33:0/1484904650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 0x7fe4601053a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fe450009900 tx=0x7fe45002a410 comp rx=0 tx=0).stop 2024-09-10T21:25:28.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.827+0000 7fe466814640 1 -- 172.21.15.33:0/1484904650 shutdown_connections 2024-09-10T21:25:28.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.827+0000 7fe466814640 1 --2- 172.21.15.33:0/1484904650 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe46010b350 0x7fe460111b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.827+0000 7fe466814640 1 --2- 172.21.15.33:0/1484904650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 0x7fe46010ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.827+0000 7fe466814640 1 --2- 172.21.15.33:0/1484904650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 0x7fe4601053a0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.827+0000 7fe466814640 1 -- 172.21.15.33:0/1484904650 >> 172.21.15.33:0/1484904650 conn(0x7fe4601009f0 msgr2=0x7fe460102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.827+0000 7fe466814640 1 -- 172.21.15.33:0/1484904650 shutdown_connections 2024-09-10T21:25:28.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.828+0000 7fe466814640 1 -- 172.21.15.33:0/1484904650 wait complete. 2024-09-10T21:25:28.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.828+0000 7fe466814640 1 Processor -- start 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe466814640 1 -- start start 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe466814640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 0x7fe460074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe466814640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 0x7fe460074c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe466814640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe46010b350 0x7fe46006da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe466814640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe460114fd0 con 0x7fe460104fc0 2024-09-10T21:25:28.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe466814640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe460114e50 con 0x7fe46010b350 2024-09-10T21:25:28.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe466814640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe460115150 con 0x7fe46010a9d0 2024-09-10T21:25:28.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe45f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 0x7fe460074c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe45ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 0x7fe460074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe45f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 0x7fe460074c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:60406/0 (socket says 172.21.15.33:60406) 2024-09-10T21:25:28.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.830+0000 7fe45f7fe640 1 -- 172.21.15.33:0/4109593376 learned_addr learned my addr 172.21.15.33:0/4109593376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.831+0000 7fe464db1640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe46010b350 0x7fe46006da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.831+0000 7fe45f7fe640 1 -- 172.21.15.33:0/4109593376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe46010b350 msgr2=0x7fe46006da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.831+0000 7fe45f7fe640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe46010b350 0x7fe46006da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.831+0000 7fe45f7fe640 1 -- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 msgr2=0x7fe460074750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.831+0000 7fe45f7fe640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 0x7fe460074750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.831+0000 7fe45f7fe640 1 -- 172.21.15.33:0/4109593376 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46006e340 con 0x7fe46010a9d0 2024-09-10T21:25:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.831+0000 7fe45f7fe640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 0x7fe460074c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe45400d770 tx=0x7fe45400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.832+0000 7fe45d7fa640 1 -- 172.21.15.33:0/4109593376 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454017070 con 0x7fe46010a9d0 2024-09-10T21:25:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.832+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe46006e630 con 0x7fe46010a9d0 2024-09-10T21:25:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.832+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4600735e0 con 0x7fe46010a9d0 2024-09-10T21:25:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.832+0000 7fe45d7fa640 1 -- 172.21.15.33:0/4109593376 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4540054d0 con 0x7fe46010a9d0 2024-09-10T21:25:28.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.832+0000 7fe45d7fa640 1 -- 172.21.15.33:0/4109593376 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe454004c30 con 0x7fe46010a9d0 2024-09-10T21:25:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.833+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe44c0052c0 con 0x7fe46010a9d0 2024-09-10T21:25:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.835+0000 7fe45d7fa640 1 -- 172.21.15.33:0/4109593376 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe454009c30 con 0x7fe46010a9d0 2024-09-10T21:25:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.837+0000 7fe45d7fa640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4480b5360 0x7fe4480b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.837+0000 7fe45ffff640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4480b5360 0x7fe4480b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.837+0000 7fe45d7fa640 1 -- 172.21.15.33:0/4109593376 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe4540e5400 con 0x7fe46010a9d0 2024-09-10T21:25:28.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.837+0000 7fe45ffff640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4480b5360 0x7fe4480b7820 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe450005ab0 tx=0x7fe450005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:28.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:28.843+0000 7fe45d7fa640 1 -- 172.21.15.33:0/4109593376 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fe4540abc70 con 0x7fe46010a9d0 2024-09-10T21:25:29.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 382308 38.77 MB/sec execute 389 sec latency 187.537 ms 2024-09-10T21:25:29.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.214+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe44c0055b0 con 0x7fe46010a9d0 2024-09-10T21:25:29.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.214+0000 7fe45d7fa640 1 -- 172.21.15.33:0/4109593376 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+11980 (secure 0 0 0) 0x7fe4540b06a0 con 0x7fe46010a9d0 2024-09-10T21:25:29.217 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:25:29.217 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"btime":"2024-09-10T21:24:25:023764+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":84,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:24:24.175284+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:25:29.218 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 84 2024-09-10T21:25:29.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4480b5360 msgr2=0x7fe4480b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4480b5360 0x7fe4480b7820 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe450005ab0 tx=0x7fe450005a20 comp rx=0 tx=0).stop 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 msgr2=0x7fe460074c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 0x7fe460074c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe45400d770 tx=0x7fe45400dc30 comp rx=0 tx=0).stop 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 shutdown_connections 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe4480b5360 0x7fe4480b7820 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe46010b350 0x7fe46006da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe46010a9d0 0x7fe460074c90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 --2- 172.21.15.33:0/4109593376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe460104fc0 0x7fe460074750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 >> 172.21.15.33:0/4109593376 conn(0x7fe4601009f0 msgr2=0x7fe460102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 shutdown_connections 2024-09-10T21:25:29.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:29.217+0000 7fe466814640 1 -- 172.21.15.33:0/4109593376 wait complete. 2024-09-10T21:25:29.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:29 smithi104 ceph-mon[35325]: pgmap v991: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-09-10T21:25:29.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:29 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4109593376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:25:29.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:29 smithi174 ceph-mon[35635]: pgmap v991: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-09-10T21:25:29.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:29 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4109593376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:25:29.894 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:29 smithi033 ceph-mon[30860]: pgmap v991: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 770 op/s 2024-09-10T21:25:29.895 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:29 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4109593376' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:25:30.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 382938 38.76 MB/sec execute 390 sec latency 241.272 ms 2024-09-10T21:25:30.423 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:25:30.982 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:31.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 383654 38.79 MB/sec execute 391 sec latency 74.141 ms 2024-09-10T21:25:31.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.546+0000 7feef5dce640 1 -- 172.21.15.33:0/3932782689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feef011c0a0 msgr2=0x7feef011e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.546+0000 7feef5dce640 1 --2- 172.21.15.33:0/3932782689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feef011c0a0 0x7feef011e490 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7feee8009be0 tx=0x7feee802a6e0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.546+0000 7feef5dce640 1 -- 172.21.15.33:0/3932782689 shutdown_connections 2024-09-10T21:25:31.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.546+0000 7feef5dce640 1 --2- 172.21.15.33:0/3932782689 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feef011c0a0 0x7feef011e490 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.546+0000 7feef5dce640 1 --2- 172.21.15.33:0/3932782689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef010a530 0x7feef010a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.546+0000 7feef5dce640 1 --2- 172.21.15.33:0/3932782689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feef010a150 0x7feef01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.546+0000 7feef5dce640 1 -- 172.21.15.33:0/3932782689 >> 172.21.15.33:0/3932782689 conn(0x7feef006cb30 msgr2=0x7feef006cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.548+0000 7feef5dce640 1 -- 172.21.15.33:0/3932782689 shutdown_connections 2024-09-10T21:25:31.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.548+0000 7feef5dce640 1 -- 172.21.15.33:0/3932782689 wait complete. 2024-09-10T21:25:31.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 Processor -- start 2024-09-10T21:25:31.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 -- start start 2024-09-10T21:25:31.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feef010a150 0x7feef01a7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feef010a530 0x7feef01a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef011c0a0 0x7feef01ad740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feef0121a50 con 0x7feef010a150 2024-09-10T21:25:31.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feef01218d0 con 0x7feef010a530 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.549+0000 7feef5dce640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feef0121bd0 con 0x7feef011c0a0 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef011c0a0 0x7feef01ad740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef011c0a0 0x7feef01ad740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37282/0 (socket says 172.21.15.33:37282) 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 -- 172.21.15.33:0/3292598411 learned_addr learned my addr 172.21.15.33:0/3292598411 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 -- 172.21.15.33:0/3292598411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feef010a530 msgr2=0x7feef01a83a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feef010a530 0x7feef01a83a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 -- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feef010a150 msgr2=0x7feef01a7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feef010a150 0x7feef01a7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 -- 172.21.15.33:0/3292598411 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feef01adf10 con 0x7feef011c0a0 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.550+0000 7feef55cd640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef011c0a0 0x7feef01ad740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feee8030ea0 tx=0x7feee8030ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.551+0000 7feeedffb640 1 -- 172.21.15.33:0/3292598411 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8040020 con 0x7feef011c0a0 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.551+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feef01ae1a0 con 0x7feef011c0a0 2024-09-10T21:25:31.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.551+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feef01b48c0 con 0x7feef011c0a0 2024-09-10T21:25:31.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.551+0000 7feeedffb640 1 -- 172.21.15.33:0/3292598411 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee80318f0 con 0x7feef011c0a0 2024-09-10T21:25:31.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.551+0000 7feeedffb640 1 -- 172.21.15.33:0/3292598411 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8036040 con 0x7feef011c0a0 2024-09-10T21:25:31.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.557+0000 7feef9405740 1 Processor -- start 2024-09-10T21:25:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef9405740 1 -- start start 2024-09-10T21:25:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef9405740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 0x56104abf6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef9405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104ac02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef9405740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac05340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef9405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56104ab6cea0 con 0x56104abf6fc0 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef9405740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56104ab5c2d0 con 0x56104ac02f50 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef9405740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56104ab600e0 con 0x56104abf66a0 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feeedffb640 1 -- 172.21.15.33:0/3292598411 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7feee803c470 con 0x7feef011c0a0 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef4dcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104ac02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef4dcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104ac02a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46026/0 (socket says 172.21.15.33:46026) 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.558+0000 7feef4dcc640 1 -- 172.21.15.33:0/1677434459 learned_addr learned my addr 172.21.15.33:0/1677434459 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.559+0000 7feef4dcc640 1 -- 172.21.15.33:0/1677434459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 msgr2=0x56104abf6a80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.559+0000 7feef4dcc640 1 --2- 172.21.15.33:0/1677434459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 0x56104abf6a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.559+0000 7feef4dcc640 1 -- 172.21.15.33:0/1677434459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 msgr2=0x56104ac05340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.559+0000 7feef4dcc640 1 --2- 172.21.15.33:0/1677434459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac05340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.559+0000 7feef4dcc640 1 -- 172.21.15.33:0/1677434459 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104a8ba650 con 0x56104abf6fc0 2024-09-10T21:25:31.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.560+0000 7feeedffb640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feed80b55b0 0x7feed80b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.561+0000 7feeedffb640 1 -- 172.21.15.33:0/3292598411 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feee8106ad0 con 0x7feef011c0a0 2024-09-10T21:25:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.561+0000 7feef4dcc640 1 --2- 172.21.15.33:0/1677434459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104ac02a10 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7feee400bb10 tx=0x7feee400d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1d3f9341fffc60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.561+0000 7feed6ffd640 1 -- 172.21.15.33:0/1677434459 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4016070 con 0x56104abf6fc0 2024-09-10T21:25:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.562+0000 7feed6ffd640 1 -- 172.21.15.33:0/1677434459 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee400dce0 con 0x56104abf6fc0 2024-09-10T21:25:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.562+0000 7feed6ffd640 1 -- 172.21.15.33:0/1677434459 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4010460 con 0x56104abf6fc0 2024-09-10T21:25:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.562+0000 7feeeffff640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feed80b55b0 0x7feed80b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feef9405740 1 -- 172.21.15.33:0/1677434459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 msgr2=0x56104ac02a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feeeffff640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feed80b55b0 0x7feed80b7a70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feef011e770 tx=0x7feee0009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feef9405740 1 --2- 172.21.15.33:0/1677434459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104ac02a10 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7feee400bb10 tx=0x7feee400d2e0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feef9405740 1 -- 172.21.15.33:0/1677434459 shutdown_connections 2024-09-10T21:25:31.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feef9405740 1 --2- 172.21.15.33:0/1677434459 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac05340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feef9405740 1 --2- 172.21.15.33:0/1677434459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104ac02a10 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feef9405740 1 --2- 172.21.15.33:0/1677434459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 0x56104abf6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.563+0000 7feef9405740 1 -- 172.21.15.33:0/1677434459 >> 172.21.15.33:0/1677434459 conn(0x56104abf44d0 msgr2=0x56104abf5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.564+0000 7feef9405740 1 -- 172.21.15.33:0/1677434459 shutdown_connections 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.564+0000 7feef9405740 1 -- 172.21.15.33:0/1677434459 wait complete. 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.564+0000 7feef9405740 1 -- start start 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.564+0000 7feef9405740 1 Processor -- start 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef9405740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 0x56104ac02790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef9405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef9405740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac00140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef9405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56104a9bd700 con 0x56104abf6fc0 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef9405740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56104aaf7160 con 0x56104ac02f50 2024-09-10T21:25:31.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef9405740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56104ac16be0 con 0x56104abf66a0 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef9405740 10 client.0 _ll_register_callbacks cb 0x56104ab109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feeeffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feeeffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abffbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46036/0 (socket says 172.21.15.33:46036) 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feeeffff640 1 -- 172.21.15.33:0/2786774580 learned_addr learned my addr 172.21.15.33:0/2786774580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.565+0000 7feef55cd640 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac00140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feeeffff640 1 -- 172.21.15.33:0/2786774580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 msgr2=0x56104ac02790 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feeeffff640 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 0x56104ac02790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feeeffff640 1 -- 172.21.15.33:0/2786774580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 msgr2=0x56104ac00140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feeeffff640 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac00140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feeeffff640 1 -- 172.21.15.33:0/2786774580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104ab24c90 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feef55cd640 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac00140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feeeffff640 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abffbe0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7feee00106e0 tx=0x7feee0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69df8877fd9e5c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feecffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.566+0000 7feecffff640 1 -- 172.21.15.33:0/2786774580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee0020070 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.567+0000 7feecffff640 1 -- 172.21.15.33:0/2786774580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feee001c8f0 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.567+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee400dce0 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.567+0000 7feecffff640 1 -- 172.21.15.33:0/2786774580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7feee001caf0 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.567+0000 7feef9405740 10 client.19203 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.567+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56104ac22560 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.567+0000 7feecffff640 1 -- 172.21.15.33:0/2786774580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee400dce0 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.567+0000 7feecffff640 1 -- 172.21.15.33:0/2786774580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7feee000f3c0 con 0x56104abf6fc0 2024-09-10T21:25:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.568+0000 7feef9405740 10 client.19203 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.568+0000 7feef9405740 10 client.19203 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.568+0000 7feef9405740 20 client.19203 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.568+0000 7feef9405740 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac23b40 0x56104ac26000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.568+0000 7feef9405740 4 client.19203 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.568+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56104ab6edf0 con 0x56104ac23b40 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.568+0000 7feef4dcc640 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac23b40 0x56104ac26000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.569+0000 7feef4dcc640 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac23b40 0x56104ac26000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=aaf283bb838604f server_cookie=ffd18275a855f0a7 in_seq=0 out_seq=0 2024-09-10T21:25:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.569+0000 7feecffff640 10 client.19203 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:25:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.573+0000 7feecffff640 1 -- 172.21.15.33:0/2786774580 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56104ab6edf0 con 0x56104ac23b40 2024-09-10T21:25:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.573+0000 7feecffff640 10 client.19203 handle_command_reply: tid=0 2024-09-10T21:25:31.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.573+0000 7feef9405740 1 client.19203 shutdown 2024-09-10T21:25:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 msgr2=0x56104abffbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abffbe0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7feee00106e0 tx=0x7feee0004730 comp rx=0 tx=0).stop 2024-09-10T21:25:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 shutdown_connections 2024-09-10T21:25:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac23b40 0x56104ac26000 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104ac02f50 0x56104ac00140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abffbe0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 --2- 172.21.15.33:0/2786774580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104abf66a0 0x56104ac02790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 >> 172.21.15.33:0/2786774580 conn(0x56104abf44d0 msgr2=0x56104abfa040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feecffff640 0 client.19203 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:25:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 shutdown_connections 2024-09-10T21:25:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 1 -- 172.21.15.33:0/2786774580 wait complete. 2024-09-10T21:25:31.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.574+0000 7feef9405740 20 client.19203 trim_cache size 0 max 16384 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.597+0000 7feef9405740 1 Processor -- start 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.597+0000 7feef9405740 1 -- start start 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef9405740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf66a0 0x56104abf3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef9405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abf3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef9405740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104abfcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef9405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feedc00e4c0 con 0x56104abf6fc0 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef9405740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feedc0100c0 con 0x56104abf66a0 2024-09-10T21:25:31.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef9405740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feedc00ec50 con 0x56104ac02f50 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf66a0 0x56104abf3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf66a0 0x56104abf3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:45364/0 (socket says 172.21.15.33:45364) 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 -- 172.21.15.33:0/613213121 learned_addr learned my addr 172.21.15.33:0/613213121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feeeffff640 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abf3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef55cd640 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104abfcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 -- 172.21.15.33:0/613213121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 msgr2=0x56104abfcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104abfcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 -- 172.21.15.33:0/613213121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 msgr2=0x56104abf3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:31.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abf3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.598+0000 7feef4dcc640 1 -- 172.21.15.33:0/613213121 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104abf4f50 con 0x56104abf66a0 2024-09-10T21:25:31.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.599+0000 7feef4dcc640 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf66a0 0x56104abf3560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feee4005fd0 tx=0x7feee4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d96a25a82a25f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.599+0000 7feed67fc640 1 -- 172.21.15.33:0/613213121 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4016040 con 0x56104abf66a0 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.600+0000 7feed67fc640 1 -- 172.21.15.33:0/613213121 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee400dce0 con 0x56104abf66a0 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.600+0000 7feed67fc640 1 -- 172.21.15.33:0/613213121 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4003db0 con 0x56104abf66a0 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 -- 172.21.15.33:0/613213121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf66a0 msgr2=0x56104abf3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf66a0 0x56104abf3560 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feee4005fd0 tx=0x7feee4000bc0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 -- 172.21.15.33:0/613213121 shutdown_connections 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104abfcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf6fc0 0x56104abf3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 --2- 172.21.15.33:0/613213121 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf66a0 0x56104abf3560 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 -- 172.21.15.33:0/613213121 >> 172.21.15.33:0/613213121 conn(0x56104abf44d0 msgr2=0x56104abf2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 -- 172.21.15.33:0/613213121 shutdown_connections 2024-09-10T21:25:31.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 -- 172.21.15.33:0/613213121 wait complete. 2024-09-10T21:25:31.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 -- start start 2024-09-10T21:25:31.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.601+0000 7feef9405740 1 Processor -- start 2024-09-10T21:25:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef9405740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf66a0 0x56104ac1c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef9405740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf6fc0 0x56104ac18a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef9405740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104ac18fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef9405740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7feedc00f9f0 con 0x56104abf66a0 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef4dcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf66a0 0x56104ac1c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feeeffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf6fc0 0x56104ac18a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef9405740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7feedc00f320 con 0x56104abf6fc0 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef4dcc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf66a0 0x56104ac1c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46054/0 (socket says 172.21.15.33:46054) 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef4dcc640 1 -- 172.21.15.33:0/2243170607 learned_addr learned my addr 172.21.15.33:0/2243170607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feedc00c480 con 0x56104ac02f50 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef9405740 10 client.0 _ll_register_callbacks cb 0x7feee0014050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.602+0000 7feef55cd640 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104ac18fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feef4dcc640 1 -- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 msgr2=0x56104ac18fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feef4dcc640 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104ac18fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feef4dcc640 1 -- 172.21.15.33:0/2243170607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf6fc0 msgr2=0x56104ac18a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feef4dcc640 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf6fc0 0x56104ac18a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feef4dcc640 1 -- 172.21.15.33:0/2243170607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56104ac19710 con 0x56104abf66a0 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feef55cd640 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104ac18fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feef4dcc640 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf66a0 0x56104ac1c5b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7feee400bae0 tx=0x7feee4003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54331b607dc9a85c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.603+0000 7feecffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.604+0000 7feecffff640 1 -- 172.21.15.33:0/2243170607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee4034020 con 0x56104abf66a0 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.604+0000 7feecffff640 1 -- 172.21.15.33:0/2243170607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feee4003cb0 con 0x56104abf66a0 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.604+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feee400dce0 con 0x56104abf66a0 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.604+0000 7feecffff640 1 -- 172.21.15.33:0/2243170607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7feee4016040 con 0x56104abf66a0 2024-09-10T21:25:31.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.604+0000 7feef9405740 10 client.19209 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.604+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56104ac91a10 con 0x56104abf66a0 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.604+0000 7feecffff640 1 -- 172.21.15.33:0/2243170607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feee400dce0 con 0x56104abf66a0 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feecffff640 1 -- 172.21.15.33:0/2243170607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7feee4010770 con 0x56104abf66a0 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feef9405740 10 client.19209 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feef9405740 10 client.19209 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feef9405740 20 client.19209 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feef9405740 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac92010 0x56104ac94430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feef9405740 4 client.19209 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56104ab6a8a0 con 0x56104ac92010 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.605+0000 7feeeffff640 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac92010 0x56104ac94430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feecffff640 10 client.19209 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feeeffff640 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac92010 0x56104ac94430 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=815f4883a7a890c2 server_cookie=4c9906fa72cf8914 in_seq=0 out_seq=0 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feecffff640 1 -- 172.21.15.33:0/2243170607 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56104ab6a8a0 con 0x56104ac92010 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feecffff640 10 client.19209 handle_command_reply: tid=0 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 client.19209 shutdown 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf66a0 msgr2=0x56104ac1c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf66a0 0x56104ac1c5b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7feee400bae0 tx=0x7feee4003930 comp rx=0 tx=0).stop 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 shutdown_connections 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56104ac92010 0x56104ac94430 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56104ac02f50 0x56104ac18fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56104abf6fc0 0x56104ac18a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 --2- 172.21.15.33:0/2243170607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56104abf66a0 0x56104ac1c5b0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.615 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feecffff640 0 client.19209 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 >> 172.21.15.33:0/2243170607 conn(0x56104abf44d0 msgr2=0x56104abf3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 shutdown_connections 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.606+0000 7feef9405740 1 -- 172.21.15.33:0/2243170607 wait complete. 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.607+0000 7feef9405740 20 client.19209 trim_cache size 0 max 16384 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.608+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feed80b55b0 msgr2=0x7feed80b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.608+0000 7feef5dce640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feed80b55b0 0x7feed80b7a70 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feef011e770 tx=0x7feee0009260 comp rx=0 tx=0).stop 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.608+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef011c0a0 msgr2=0x7feef01ad740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef011c0a0 0x7feef01ad740 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feee8030ea0 tx=0x7feee8030ed0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 shutdown_connections 2024-09-10T21:25:31.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7feed80b55b0 0x7feed80b7a70 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feef011c0a0 0x7feef01ad740 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feef010a530 0x7feef01a83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 --2- 172.21.15.33:0/3292598411 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feef010a150 0x7feef01a7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 >> 172.21.15.33:0/3292598411 conn(0x7feef006cb30 msgr2=0x7feef011c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 shutdown_connections 2024-09-10T21:25:31.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:31.609+0000 7feef5dce640 1 -- 172.21.15.33:0/3292598411 wait complete. 2024-09-10T21:25:31.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:31 smithi033 ceph-mon[30860]: pgmap v992: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 893 op/s 2024-09-10T21:25:31.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:31 smithi104 ceph-mon[35325]: pgmap v992: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 893 op/s 2024-09-10T21:25:31.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:31 smithi174 ceph-mon[35635]: pgmap v992: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 15 MiB/s wr, 893 op/s 2024-09-10T21:25:32.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 384441 38.81 MB/sec execute 392 sec latency 83.151 ms 2024-09-10T21:25:32.446 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:25:32.792 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:33.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 385258 38.82 MB/sec execute 393 sec latency 58.168 ms 2024-09-10T21:25:33.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:33 smithi104 ceph-mon[35325]: pgmap v993: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-09-10T21:25:33.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:33 smithi174 ceph-mon[35635]: pgmap v993: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-09-10T21:25:33.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:33 smithi033 ceph-mon[30860]: pgmap v993: 97 pgs: 97 active+clean; 1.2 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-09-10T21:25:34.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 385685 38.78 MB/sec execute 394 sec latency 349.116 ms 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 -- 172.21.15.33:0/1454735751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2040104550 msgr2=0x7f204010cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 --2- 172.21.15.33:0/1454735751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2040104550 0x7f204010cb30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2030009900 tx=0x7f203002a410 comp rx=0 tx=0).stop 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 -- 172.21.15.33:0/1454735751 shutdown_connections 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 --2- 172.21.15.33:0/1454735751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f204010d100 0x7f204010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 --2- 172.21.15.33:0/1454735751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2040104550 0x7f204010cb30 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 --2- 172.21.15.33:0/1454735751 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2040103b50 0x7f2040103f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 -- 172.21.15.33:0/1454735751 >> 172.21.15.33:0/1454735751 conn(0x7f20400fd700 msgr2=0x7f20400ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:34.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 -- 172.21.15.33:0/1454735751 shutdown_connections 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.096+0000 7f20475ad640 1 -- 172.21.15.33:0/1454735751 wait complete. 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 Processor -- start 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 -- start start 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2040103b50 0x7f20401019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2040104550 0x7f2040101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204010d100 0x7f2040102450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112d60 con 0x7f204010d100 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112be0 con 0x7f2040103b50 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f20475ad640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2040112ee0 con 0x7f2040104550 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f2045b4a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204010d100 0x7f2040102450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f2045b4a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204010d100 0x7f2040102450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46068/0 (socket says 172.21.15.33:46068) 2024-09-10T21:25:34.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f2045349640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2040103b50 0x7f20401019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f2045b4a640 1 -- 172.21.15.33:0/2715523408 learned_addr learned my addr 172.21.15.33:0/2715523408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.097+0000 7f2044b48640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2040104550 0x7f2040101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f2045b4a640 1 -- 172.21.15.33:0/2715523408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2040104550 msgr2=0x7f2040101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f2045b4a640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2040104550 0x7f2040101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f2045b4a640 1 -- 172.21.15.33:0/2715523408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2040103b50 msgr2=0x7f20401019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f2045b4a640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2040103b50 0x7f20401019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f2045b4a640 1 -- 172.21.15.33:0/2715523408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20401a76d0 con 0x7f204010d100 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f2045b4a640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204010d100 0x7f2040102450 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f203800eb10 tx=0x7f203800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f20367fc640 1 -- 172.21.15.33:0/2715523408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038004b20 con 0x7f204010d100 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20401a79c0 con 0x7f204010d100 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f20367fc640 1 -- 172.21.15.33:0/2715523408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2038004cc0 con 0x7f204010d100 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.098+0000 7f20367fc640 1 -- 172.21.15.33:0/2715523408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038010670 con 0x7f204010d100 2024-09-10T21:25:34.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.099+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20401a7ed0 con 0x7f204010d100 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.101+0000 7f20367fc640 1 -- 172.21.15.33:0/2715523408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2038005000 con 0x7f204010d100 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.101+0000 7f2049982740 1 Processor -- start 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.101+0000 7f2049982740 1 -- start start 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2049982740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09e00f0 0x55c2a09e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2049982740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ec460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2049982740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09ec9a0 0x55c2a09eed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2049982740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c2a0956ea0 con 0x55c2a09ec9a0 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2049982740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c2a09462d0 con 0x55c2a09e0a10 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2049982740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2a094a0e0 con 0x55c2a09e00f0 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ec460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2044b48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09ec9a0 0x55c2a09eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f20367fc640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f202c0b5360 0x7f202c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ec460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:45410/0 (socket says 172.21.15.33:45410) 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 -- 172.21.15.33:0/1583209157 learned_addr learned my addr 172.21.15.33:0/1583209157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2044b48640 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09e00f0 0x55c2a09e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045b4a640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f202c0b5360 0x7f202c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 -- 172.21.15.33:0/1583209157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09e00f0 msgr2=0x55c2a09e04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09e00f0 0x55c2a09e04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 -- 172.21.15.33:0/1583209157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09ec9a0 msgr2=0x55c2a09eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09ec9a0 0x55c2a09eed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f20367fc640 1 -- 172.21.15.33:0/2715523408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f20380e6190 con 0x7f204010d100 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.102+0000 7f2045349640 1 -- 172.21.15.33:0/1583209157 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2a06a4650 con 0x55c2a09e0a10 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.103+0000 7f2045349640 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ec460 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f203c009cf0 tx=0x7f203c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e98809581f38d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.103+0000 7f2045b4a640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f202c0b5360 0x7f202c0b7820 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f20380a8ee0 tx=0x7f2038004e70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.103+0000 7f202b7fe640 1 -- 172.21.15.33:0/1583209157 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c01e070 con 0x55c2a09e0a10 2024-09-10T21:25:34.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.103+0000 7f202b7fe640 1 -- 172.21.15.33:0/1583209157 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203c009de0 con 0x55c2a09e0a10 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.103+0000 7f202b7fe640 1 -- 172.21.15.33:0/1583209157 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c019650 con 0x55c2a09e0a10 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.103+0000 7f2049982740 1 -- 172.21.15.33:0/1583209157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 msgr2=0x55c2a09ec460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.103+0000 7f2049982740 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ec460 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f203c009cf0 tx=0x7f203c011810 comp rx=0 tx=0).stop 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 -- 172.21.15.33:0/1583209157 shutdown_connections 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09ec9a0 0x55c2a09eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ec460 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 --2- 172.21.15.33:0/1583209157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09e00f0 0x55c2a09e04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 -- 172.21.15.33:0/1583209157 >> 172.21.15.33:0/1583209157 conn(0x55c2a09ddf20 msgr2=0x55c2a09df0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 -- 172.21.15.33:0/1583209157 shutdown_connections 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 -- 172.21.15.33:0/1583209157 wait complete. 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 -- start start 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.104+0000 7f2049982740 1 Processor -- start 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e00f0 0x55c2a09ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e0a10 0x55c2a09e9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09e9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c2a07a7700 con 0x55c2a09e0a10 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c2a08e1160 con 0x55c2a09e00f0 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2a09ef620 con 0x55c2a09ec9a0 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 10 client.0 _ll_register_callbacks cb 0x55c2a08fa9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e0a10 0x55c2a09e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e0a10 0x55c2a09e9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46088/0 (socket says 172.21.15.33:46088) 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2044b48640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e00f0 0x55c2a09ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 -- 172.21.15.33:0/905178595 learned_addr learned my addr 172.21.15.33:0/905178595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2044b48640 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09e9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 -- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 msgr2=0x55c2a09e9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09e9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 -- 172.21.15.33:0/905178595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e00f0 msgr2=0x55c2a09ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e00f0 0x55c2a09ec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 -- 172.21.15.33:0/905178595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2a090ec90 con 0x55c2a09e0a10 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2045349640 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e0a10 0x55c2a09e9690 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f203c0058c0 tx=0x7f203c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173451aa7aea343d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2028ff9640 1 -- 172.21.15.33:0/905178595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203c01e040 con 0x55c2a09e0a10 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2028ff9640 1 -- 172.21.15.33:0/905178595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f203c016070 con 0x55c2a09e0a10 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.105+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f203c009de0 con 0x55c2a09e0a10 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2028ff9640 1 -- 172.21.15.33:0/905178595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f203c005c50 con 0x55c2a09e0a10 2024-09-10T21:25:34.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 10 client.19221 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55c2a09faf10 con 0x55c2a09e0a10 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2028ff9640 1 -- 172.21.15.33:0/905178595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f203c009de0 con 0x55c2a09e0a10 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2028ff9640 1 -- 172.21.15.33:0/905178595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f203c02b030 con 0x55c2a09e0a10 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 10 client.19221 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 10 client.19221 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 20 client.19221 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a09fc310 0x55c2a09fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 4 client.19221 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2a0958df0 con 0x55c2a09fc310 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.106+0000 7f2044b48640 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a09fc310 0x55c2a09fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.107+0000 7f2044b48640 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a09fc310 0x55c2a09fe700 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=461923de20a046ac server_cookie=612d47aa42c46110 in_seq=0 out_seq=0 2024-09-10T21:25:34.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.107+0000 7f2028ff9640 10 client.19221 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2028ff9640 1 -- 172.21.15.33:0/905178595 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c2a0958df0 con 0x55c2a09fc310 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2028ff9640 10 client.19221 handle_command_reply: tid=0 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 client.19221 shutdown 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e0a10 msgr2=0x55c2a09e9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e0a10 0x55c2a09e9690 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f203c0058c0 tx=0x7f203c0052f0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 shutdown_connections 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a09fc310 0x55c2a09fe700 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09e9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e0a10 0x55c2a09e9690 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2028ff9640 0 client.19221 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 --2- 172.21.15.33:0/905178595 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e00f0 0x55c2a09ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 >> 172.21.15.33:0/905178595 conn(0x55c2a09ddf20 msgr2=0x55c2a09e3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:34.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 shutdown_connections 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 1 -- 172.21.15.33:0/905178595 wait complete. 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.108+0000 7f2049982740 20 client.19221 trim_cache size 0 max 16384 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.121+0000 7f2049982740 1 Processor -- start 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.121+0000 7f2049982740 1 -- start start 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.121+0000 7f2049982740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09df680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.121+0000 7f2049982740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2049982740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09edd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2049982740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b0610 con 0x55c2a09e00f0 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2049982740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b2210 con 0x55c2a09e0a10 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2049982740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b0da0 con 0x55c2a09ec9a0 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09df680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09df680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46100/0 (socket says 172.21.15.33:46100) 2024-09-10T21:25:34.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 -- 172.21.15.33:0/3403638458 learned_addr learned my addr 172.21.15.33:0/3403638458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2045349640 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 -- 172.21.15.33:0/3403638458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 msgr2=0x55c2a09edd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09edd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 -- 172.21.15.33:0/3403638458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 msgr2=0x55c2a09dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 -- 172.21.15.33:0/3403638458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2a09e0840 con 0x55c2a09e00f0 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f2044b48640 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09df680 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f203002a930 tx=0x7f2030041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762acaca8e25ff7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f202affd640 1 -- 172.21.15.33:0/3403638458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030035040 con 0x55c2a09e00f0 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f202affd640 1 -- 172.21.15.33:0/3403638458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2030004480 con 0x55c2a09e00f0 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.122+0000 7f202affd640 1 -- 172.21.15.33:0/3403638458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030003cc0 con 0x55c2a09e00f0 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 -- 172.21.15.33:0/3403638458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 msgr2=0x55c2a09df680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09df680 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f203002a930 tx=0x7f2030041040 comp rx=0 tx=0).stop 2024-09-10T21:25:34.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 -- 172.21.15.33:0/3403638458 shutdown_connections 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09edd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 --2- 172.21.15.33:0/3403638458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09df680 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 -- 172.21.15.33:0/3403638458 >> 172.21.15.33:0/3403638458 conn(0x55c2a09ddf20 msgr2=0x55c2a09def70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 -- 172.21.15.33:0/3403638458 shutdown_connections 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 -- 172.21.15.33:0/3403638458 wait complete. 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 -- start start 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.123+0000 7f2049982740 1 Processor -- start 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2049982740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09f5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2049982740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ef870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2049982740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2049982740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b1b40 con 0x55c2a09e00f0 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2049982740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f20401b1470 con 0x55c2a09e0a10 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2049982740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f20401ae5d0 con 0x55c2a09ec9a0 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2049982740 10 client.0 _ll_register_callbacks cb 0x7f203c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09f5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46110/0 (socket says 172.21.15.33:46110) 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 -- 172.21.15.33:0/769693012 learned_addr learned my addr 172.21.15.33:0/769693012 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2045349640 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 -- 172.21.15.33:0/769693012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 msgr2=0x55c2a09efdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 -- 172.21.15.33:0/769693012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 msgr2=0x55c2a09ef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ef870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 -- 172.21.15.33:0/769693012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2a09f05e0 con 0x55c2a09e00f0 2024-09-10T21:25:34.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2045349640 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2044b48640 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09f5060 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2030005b50 tx=0x7f2030031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ceb27bd1172212 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.124+0000 7f2028ff9640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2028ff9640 1 -- 172.21.15.33:0/769693012 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2030035040 con 0x55c2a09e00f0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2028ff9640 1 -- 172.21.15.33:0/769693012 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2030005240 con 0x55c2a09e00f0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2030004480 con 0x55c2a09e00f0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2028ff9640 1 -- 172.21.15.33:0/769693012 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f2030005440 con 0x55c2a09e00f0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 10 client.19227 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55c2a0a6a020 con 0x55c2a09e00f0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2028ff9640 1 -- 172.21.15.33:0/769693012 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2030004480 con 0x55c2a09e00f0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2028ff9640 1 -- 172.21.15.33:0/769693012 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f2030003ba0 con 0x55c2a09e00f0 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 10 client.19227 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:34.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 10 client.19227 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 20 client.19227 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a0a6a690 0x55c2a0a6cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 4 client.19227 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.125+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c2a09548a0 con 0x55c2a0a6a690 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2045349640 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a0a6a690 0x55c2a0a6cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2045349640 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a0a6a690 0x55c2a0a6cab0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=11821b9223b801d0 server_cookie=91e7001a61d5f85d in_seq=0 out_seq=0 2024-09-10T21:25:34.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2028ff9640 10 client.19227 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2028ff9640 1 -- 172.21.15.33:0/769693012 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c2a09548a0 con 0x55c2a0a6a690 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2028ff9640 10 client.19227 handle_command_reply: tid=0 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 client.19227 shutdown 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 msgr2=0x55c2a09f5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09f5060 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f2030005b50 tx=0x7f2030031400 comp rx=0 tx=0).stop 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 shutdown_connections 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55c2a0a6a690 0x55c2a0a6cab0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2a09ec9a0 0x55c2a09efdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c2a09e0a10 0x55c2a09ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2028ff9640 0 client.19227 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 --2- 172.21.15.33:0/769693012 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c2a09e00f0 0x55c2a09f5060 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 >> 172.21.15.33:0/769693012 conn(0x55c2a09ddf20 msgr2=0x55c2a09eacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:34.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 shutdown_connections 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.126+0000 7f2049982740 1 -- 172.21.15.33:0/769693012 wait complete. 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f2049982740 20 client.19227 trim_cache size 0 max 16384 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f202c0b5360 msgr2=0x7f202c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f202c0b5360 0x7f202c0b7820 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f20380a8ee0 tx=0x7f2038004e70 comp rx=0 tx=0).stop 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204010d100 msgr2=0x7f2040102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204010d100 0x7f2040102450 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f203800eb10 tx=0x7f203800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 shutdown_connections 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f202c0b5360 0x7f202c0b7820 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f204010d100 0x7f2040102450 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2040104550 0x7f2040101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 --2- 172.21.15.33:0/2715523408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2040103b50 0x7f20401019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:34.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 >> 172.21.15.33:0/2715523408 conn(0x7f20400fd700 msgr2=0x7f20400fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:34.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 shutdown_connections 2024-09-10T21:25:34.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:34.127+0000 7f20475ad640 1 -- 172.21.15.33:0/2715523408 wait complete. 2024-09-10T21:25:35.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 386216 38.75 MB/sec execute 395 sec latency 267.821 ms 2024-09-10T21:25:35.502 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:25:35.828 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:35.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:35 smithi174 ceph-mon[35635]: pgmap v994: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 856 op/s 2024-09-10T21:25:35.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:35 smithi033 ceph-mon[30860]: pgmap v994: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 856 op/s 2024-09-10T21:25:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:35 smithi104 ceph-mon[35325]: pgmap v994: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 856 op/s 2024-09-10T21:25:36.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 386832 38.73 MB/sec execute 396 sec latency 122.785 ms 2024-09-10T21:25:37.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 387174 38.69 MB/sec execute 397 sec latency 696.096 ms 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 -- 172.21.15.33:0/3986230089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd9810d140 msgr2=0x7fdd9810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 --2- 172.21.15.33:0/3986230089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd9810d140 0x7fdd9810f550 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fdd90009ca0 tx=0x7fdd9002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 -- 172.21.15.33:0/3986230089 shutdown_connections 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 --2- 172.21.15.33:0/3986230089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd9810d140 0x7fdd9810f550 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 --2- 172.21.15.33:0/3986230089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd98104590 0x7fdd9810cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 --2- 172.21.15.33:0/3986230089 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd98103b90 0x7fdd98103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 -- 172.21.15.33:0/3986230089 >> 172.21.15.33:0/3986230089 conn(0x7fdd980fd700 msgr2=0x7fdd980ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 -- 172.21.15.33:0/3986230089 shutdown_connections 2024-09-10T21:25:37.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.362+0000 7fdd9e907640 1 -- 172.21.15.33:0/3986230089 wait complete. 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 Processor -- start 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 -- start start 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd98103b90 0x7fdd98199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98104590 0x7fdd981997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9810d140 0x7fdd9819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98112d60 con 0x7fdd98104590 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98112be0 con 0x7fdd98103b90 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.363+0000 7fdd9e907640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd98112ee0 con 0x7fdd9810d140 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98104590 0x7fdd981997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98104590 0x7fdd981997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46136/0 (socket says 172.21.15.33:46136) 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 -- 172.21.15.33:0/1021268087 learned_addr learned my addr 172.21.15.33:0/1021268087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd97fff640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd98103b90 0x7fdd98199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 -- 172.21.15.33:0/1021268087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9810d140 msgr2=0x7fdd9819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9810d140 0x7fdd9819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 -- 172.21.15.33:0/1021268087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd98103b90 msgr2=0x7fdd98199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd98103b90 0x7fdd98199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 -- 172.21.15.33:0/1021268087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9819f320 con 0x7fdd98104590 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98104590 0x7fdd981997b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00d950 tx=0x7fdd8c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd957fa640 1 -- 172.21.15.33:0/1021268087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c017070 con 0x7fdd98104590 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9819f610 con 0x7fdd98104590 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd957fa640 1 -- 172.21.15.33:0/1021268087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd8c009c30 con 0x7fdd98104590 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd957fa640 1 -- 172.21.15.33:0/1021268087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c00f620 con 0x7fdd98104590 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.364+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd981a5ea0 con 0x7fdd98104590 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 Processor -- start 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdd957fa640 1 -- 172.21.15.33:0/1021268087 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdd8c010460 con 0x7fdd98104590 2024-09-10T21:25:37.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 -- start start 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31cb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31cbdf0 0x5615d31d7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31d7d80 0x5615d31da170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5615d3141ea0 con 0x5615d31d7d80 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5615d31312d0 con 0x5615d31cb4d0 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615d31350e0 con 0x5615d31cbdf0 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31d7d80 0x5615d31da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31d7d80 0x5615d31da170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46148/0 (socket says 172.21.15.33:46148) 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdd977fe640 1 -- 172.21.15.33:0/776219681 learned_addr learned my addr 172.21.15.33:0/776219681 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:37.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdd97fff640 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31cbdf0 0x5615d31d7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.367+0000 7fdd9cea4640 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31cb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd977fe640 1 -- 172.21.15.33:0/776219681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31cbdf0 msgr2=0x5615d31d7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd977fe640 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31cbdf0 0x5615d31d7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd977fe640 1 -- 172.21.15.33:0/776219681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 msgr2=0x5615d31cb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd977fe640 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31cb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd977fe640 1 -- 172.21.15.33:0/776219681 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d2e8f650 con 0x5615d31d7d80 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd957fa640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd800b52e0 0x7fdd800b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd977fe640 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31d7d80 0x5615d31da170 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ebc20 tx=0x7fdd8c0ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70f346df23cdee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd9cea4640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd800b52e0 0x7fdd800b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd957fa640 1 -- 172.21.15.33:0/1021268087 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdd8c0e57c0 con 0x7fdd98104590 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd7e7fc640 1 -- 172.21.15.33:0/776219681 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0a85d0 con 0x5615d31d7d80 2024-09-10T21:25:37.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd7e7fc640 1 -- 172.21.15.33:0/776219681 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd8c0aba90 con 0x5615d31d7d80 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.368+0000 7fdd7e7fc640 1 -- 172.21.15.33:0/776219681 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0ab6f0 con 0x5615d31d7d80 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 -- 172.21.15.33:0/776219681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31d7d80 msgr2=0x5615d31da170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31d7d80 0x5615d31da170 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ebc20 tx=0x7fdd8c0ed670 comp rx=0 tx=0).stop 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdd9cea4640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd800b52e0 0x7fdd800b77a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdd90004af0 tx=0x7fdd90005f70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 -- 172.21.15.33:0/776219681 shutdown_connections 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31d7d80 0x5615d31da170 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31cbdf0 0x5615d31d7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/776219681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31cb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 -- 172.21.15.33:0/776219681 >> 172.21.15.33:0/776219681 conn(0x5615d31c9300 msgr2=0x5615d31ca4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 -- 172.21.15.33:0/776219681 shutdown_connections 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.369+0000 7fdda0cdc740 1 -- 172.21.15.33:0/776219681 wait complete. 2024-09-10T21:25:37.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 -- start start 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 Processor -- start 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31d76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5615d2f92700 con 0x5615d31cbdf0 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5615d30cc160 con 0x5615d31cb4d0 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615d31daa00 con 0x5615d31d7d80 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdda0cdc740 10 client.0 _ll_register_callbacks cb 0x5615d30e59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46150/0 (socket says 172.21.15.33:46150) 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdd977fe640 1 -- 172.21.15.33:0/1288657028 learned_addr learned my addr 172.21.15.33:0/1288657028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdd97fff640 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31d76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.370+0000 7fdd9cea4640 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd977fe640 1 -- 172.21.15.33:0/1288657028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 msgr2=0x5615d31d5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd977fe640 1 -- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 msgr2=0x5615d31d76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31d76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd977fe640 1 -- 172.21.15.33:0/1288657028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d30f9c90 con 0x5615d31cbdf0 2024-09-10T21:25:37.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d4b00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ebbf0 tx=0x7fdd8c0cf110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b453bf987b4331be server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd777fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd777fe640 1 -- 172.21.15.33:0/1288657028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c020070 con 0x5615d31cbdf0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd777fe640 1 -- 172.21.15.33:0/1288657028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdd8c0971b0 con 0x5615d31cbdf0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8c0aba90 con 0x5615d31cbdf0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd777fe640 1 -- 172.21.15.33:0/1288657028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fdd8c0ab8d0 con 0x5615d31cbdf0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdda0cdc740 10 client.19239 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x5615d31e63a0 con 0x5615d31cbdf0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.371+0000 7fdd777fe640 1 -- 172.21.15.33:0/1288657028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd8c0aba90 con 0x5615d31cbdf0 2024-09-10T21:25:37.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdd777fe640 1 -- 172.21.15.33:0/1288657028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fdd8c0ac1e0 con 0x5615d31cbdf0 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdda0cdc740 10 client.19239 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdda0cdc740 10 client.19239 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdda0cdc740 20 client.19239 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d31e76f0 0x5615d31e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdda0cdc740 4 client.19239 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615d3143df0 con 0x5615d31e76f0 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdd97fff640 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d31e76f0 0x5615d31e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdd97fff640 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d31e76f0 0x5615d31e9ae0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=382678d8eaf6b69f server_cookie=4163a475e3f6f422 in_seq=0 out_seq=0 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.372+0000 7fdd777fe640 10 client.19239 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.373+0000 7fdd777fe640 1 -- 172.21.15.33:0/1288657028 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5615d3143df0 con 0x5615d31e76f0 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.373+0000 7fdd777fe640 10 client.19239 handle_command_reply: tid=0 2024-09-10T21:25:37.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.373+0000 7fdda0cdc740 1 client.19239 shutdown 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.373+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 msgr2=0x5615d31d4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.373+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d4b00 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ebbf0 tx=0x7fdd8c0cf110 comp rx=0 tx=0).stop 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 shutdown_connections 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d31e76f0 0x5615d31e9ae0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d4b00 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1288657028 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31d76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdd777fe640 0 client.19239 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 >> 172.21.15.33:0/1288657028 conn(0x5615d31c9300 msgr2=0x5615d31cee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 shutdown_connections 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1288657028 wait complete. 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.374+0000 7fdda0cdc740 20 client.19239 trim_cache size 0 max 16384 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.387+0000 7fdda0cdc740 1 Processor -- start 2024-09-10T21:25:37.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.387+0000 7fdda0cdc740 1 -- start start 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.387+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.387+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.387+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981ae410 con 0x5615d31cbdf0 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981b0010 con 0x5615d31cb4d0 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981aeba0 con 0x5615d31d7d80 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46162/0 (socket says 172.21.15.33:46162) 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 -- 172.21.15.33:0/1232694246 learned_addr learned my addr 172.21.15.33:0/1232694246 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd97fff640 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd9cea4640 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 -- 172.21.15.33:0/1232694246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 msgr2=0x5615d31d38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 -- 172.21.15.33:0/1232694246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 msgr2=0x5615d31e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 -- 172.21.15.33:0/1232694246 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d31cf7e0 con 0x5615d31cbdf0 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd977fe640 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d3380 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ebc20 tx=0x7fdd8c0ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75bff15041f9b3cc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd7dffb640 1 -- 172.21.15.33:0/1232694246 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c020020 con 0x5615d31cbdf0 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd7dffb640 1 -- 172.21.15.33:0/1232694246 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd8c0aba90 con 0x5615d31cbdf0 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.388+0000 7fdd7dffb640 1 -- 172.21.15.33:0/1232694246 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c0abd50 con 0x5615d31cbdf0 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1232694246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 msgr2=0x5615d31d3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d3380 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c0ebc20 tx=0x7fdd8c0ed670 comp rx=0 tx=0).stop 2024-09-10T21:25:37.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1232694246 shutdown_connections 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d31d38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31d3380 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/1232694246 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1232694246 >> 172.21.15.33:0/1232694246 conn(0x5615d31c9300 msgr2=0x5615d31cfd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1232694246 shutdown_connections 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 -- 172.21.15.33:0/1232694246 wait complete. 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.389+0000 7fdda0cdc740 1 -- start start 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 1 Processor -- start 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31e93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31e9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d324d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981af940 con 0x5615d31cbdf0 2024-09-10T21:25:37.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981af270 con 0x5615d31cb4d0 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd981ac3d0 con 0x5615d31d7d80 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdda0cdc740 10 client.0 _ll_register_callbacks cb 0x7fdd90009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31e9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31e9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46170/0 (socket says 172.21.15.33:46170) 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 -- 172.21.15.33:0/3543299879 learned_addr learned my addr 172.21.15.33:0/3543299879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd9cea4640 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d324d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 -- 172.21.15.33:0/3543299879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 msgr2=0x5615d324d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d324d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 -- 172.21.15.33:0/3543299879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 msgr2=0x5615d31e93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31e93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd977fe640 1 -- 172.21.15.33:0/3543299879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d324d930 con 0x5615d31cbdf0 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.390+0000 7fdd9cea4640 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d324d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:37.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdd977fe640 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31e9950 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009ed0 tx=0x7fdd8c0973e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bc3632c224d140 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdd777fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdd777fe640 1 -- 172.21.15.33:0/3543299879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c020070 con 0x5615d31cbdf0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdd777fe640 1 -- 172.21.15.33:0/3543299879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdd8c0b5080 con 0x5615d31cbdf0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd8c0aba90 con 0x5615d31cbdf0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdd777fe640 1 -- 172.21.15.33:0/3543299879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fdd8c05b030 con 0x5615d31cbdf0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 10 client.19245 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x5615d324d930 con 0x5615d31cbdf0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdd777fe640 1 -- 172.21.15.33:0/3543299879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd8c0aba90 con 0x5615d31cbdf0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdd777fe640 1 -- 172.21.15.33:0/3543299879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fdd8c0c3bd0 con 0x5615d31cbdf0 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 10 client.19245 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 10 client.19245 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:25:37.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 20 client.19245 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:37.092 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:25:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d3254f50 0x5615d3257370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 4 client.19245 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:25:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.391+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5615d313f8a0 con 0x5615d3254f50 2024-09-10T21:25:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdd97fff640 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d3254f50 0x5615d3257370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:37.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdd777fe640 10 client.19245 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdd97fff640 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d3254f50 0x5615d3257370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12564167707383 server_cookie=b2cbc361f3e464eb in_seq=0 out_seq=0 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdd777fe640 1 -- 172.21.15.33:0/3543299879 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5615d313f8a0 con 0x5615d3254f50 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdd777fe640 10 client.19245 handle_command_reply: tid=0 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdda0cdc740 1 client.19245 shutdown 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 msgr2=0x5615d31e9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.392+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31e9950 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009ed0 tx=0x7fdd8c0973e0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 shutdown_connections 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d31d7d80 0x5615d324d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5615d3254f50 0x5615d3257370 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5615d31cbdf0 0x5615d31e9950 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 --2- 172.21.15.33:0/3543299879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5615d31cb4d0 0x5615d31e93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd777fe640 0 client.19245 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:25:37.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 >> 172.21.15.33:0/3543299879 conn(0x5615d31c9300 msgr2=0x5615d31d3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 shutdown_connections 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 1 -- 172.21.15.33:0/3543299879 wait complete. 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdda0cdc740 20 client.19245 trim_cache size 0 max 16384 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd800b52e0 msgr2=0x7fdd800b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd800b52e0 0x7fdd800b77a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdd90004af0 tx=0x7fdd90005f70 comp rx=0 tx=0).stop 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98104590 msgr2=0x7fdd981997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98104590 0x7fdd981997b0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c00d950 tx=0x7fdd8c00de10 comp rx=0 tx=0).stop 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 shutdown_connections 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdd800b52e0 0x7fdd800b77a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd9810d140 0x7fdd9819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdd98104590 0x7fdd981997b0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 --2- 172.21.15.33:0/1021268087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdd98103b90 0x7fdd98199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:37.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 >> 172.21.15.33:0/1021268087 conn(0x7fdd980fd700 msgr2=0x7fdd980fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 shutdown_connections 2024-09-10T21:25:37.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:36.393+0000 7fdd9e907640 1 -- 172.21.15.33:0/1021268087 wait complete. 2024-09-10T21:25:37.862 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:37 smithi104 ceph-mon[35325]: pgmap v995: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 711 op/s 2024-09-10T21:25:37.873 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:37 smithi033 ceph-mon[30860]: pgmap v995: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 711 op/s 2024-09-10T21:25:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:37 smithi174 ceph-mon[35635]: pgmap v995: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 711 op/s 2024-09-10T21:25:37.910 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:25:38.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 387946 38.70 MB/sec execute 398 sec latency 143.506 ms 2024-09-10T21:25:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 -- 172.21.15.33:0/4043691446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e64103ca0 msgr2=0x7f0e64104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 --2- 172.21.15.33:0/4043691446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e64103ca0 0x7f0e64104080 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009900 tx=0x7f0e5402a410 comp rx=0 tx=0).stop 2024-09-10T21:25:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 -- 172.21.15.33:0/4043691446 shutdown_connections 2024-09-10T21:25:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 --2- 172.21.15.33:0/4043691446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 0x7f0e6410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 --2- 172.21.15.33:0/4043691446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e641045c0 0x7f0e6410cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 --2- 172.21.15.33:0/4043691446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e64103ca0 0x7f0e64104080 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 -- 172.21.15.33:0/4043691446 >> 172.21.15.33:0/4043691446 conn(0x7f0e640fd700 msgr2=0x7f0e640ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 -- 172.21.15.33:0/4043691446 shutdown_connections 2024-09-10T21:25:38.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.819+0000 7f0e69080640 1 -- 172.21.15.33:0/4043691446 wait complete. 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.820+0000 7f0e69080640 1 Processor -- start 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e69080640 1 -- start start 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e69080640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e64103ca0 0x7f0e64199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e69080640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e641045c0 0x7f0e641997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e69080640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 0x7f0e6419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e69080640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64112d70 con 0x7f0e641045c0 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e69080640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64112bf0 con 0x7f0e64103ca0 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e69080640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e64112ef0 con 0x7f0e6410d100 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e64103ca0 0x7f0e64199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e6359e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 0x7f0e6419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e64103ca0 0x7f0e64199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:45500/0 (socket says 172.21.15.33:45500) 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e6359e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 0x7f0e6419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37446/0 (socket says 172.21.15.33:37446) 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e62d9d640 1 -- 172.21.15.33:0/3436985870 learned_addr learned my addr 172.21.15.33:0/3436985870 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.821+0000 7f0e6259c640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e641045c0 0x7f0e641997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e6359e640 1 -- 172.21.15.33:0/3436985870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e64103ca0 msgr2=0x7f0e64199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e6359e640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e64103ca0 0x7f0e64199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e6359e640 1 -- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e641045c0 msgr2=0x7f0e641997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e6359e640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e641045c0 0x7f0e641997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e6359e640 1 -- 172.21.15.33:0/3436985870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6419f310 con 0x7f0e6410d100 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e6259c640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e641045c0 0x7f0e641997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e6359e640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 0x7f0e6419eb40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00e940 tx=0x7f0e5c00ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:38.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e4bfff640 1 -- 172.21.15.33:0/3436985870 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c004d20 con 0x7f0e6410d100 2024-09-10T21:25:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6419f600 con 0x7f0e6410d100 2024-09-10T21:25:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e4bfff640 1 -- 172.21.15.33:0/3436985870 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5c005020 con 0x7f0e6410d100 2024-09-10T21:25:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.822+0000 7f0e4bfff640 1 -- 172.21.15.33:0/3436985870 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0116d0 con 0x7f0e6410d100 2024-09-10T21:25:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.823+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e64101a00 con 0x7f0e6410d100 2024-09-10T21:25:38.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.825+0000 7f0e4bfff640 1 -- 172.21.15.33:0/3436985870 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0e5c00f040 con 0x7f0e6410d100 2024-09-10T21:25:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 Processor -- start 2024-09-10T21:25:38.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 -- start start 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e4bfff640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0e4c0b5360 0x7f0e4c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefdf68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe02840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefe02d80 0x55ffefe05170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffefd6cea0 con 0x55ffefdf64d0 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ffefd5c2d0 con 0x55ffefdf6df0 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e6b455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffefd600e0 con 0x55ffefe02d80 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefdf68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefdf68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46198/0 (socket says 172.21.15.33:46198) 2024-09-10T21:25:38.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.826+0000 7f0e62d9d640 1 -- 172.21.15.33:0/2956662323 learned_addr learned my addr 172.21.15.33:0/2956662323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e6359e640 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefe02d80 0x55ffefe05170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e4bfff640 1 -- 172.21.15.33:0/3436985870 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0e5c0e5be0 con 0x7f0e6410d100 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e6259c640 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe02840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e6259c640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0e4c0b5360 0x7f0e4c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e62d9d640 1 -- 172.21.15.33:0/2956662323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefe02d80 msgr2=0x55ffefe05170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefe02d80 0x55ffefe05170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e62d9d640 1 -- 172.21.15.33:0/2956662323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 msgr2=0x55ffefe02840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe02840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e62d9d640 1 -- 172.21.15.33:0/2956662323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffefaba650 con 0x55ffefdf64d0 2024-09-10T21:25:38.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e6259c640 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe02840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefdf68b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0e54000c00 tx=0x7f0e54031c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd08a23e0fc1e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e497fa640 1 -- 172.21.15.33:0/2956662323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54040070 con 0x55ffefdf64d0 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e497fa640 1 -- 172.21.15.33:0/2956662323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e540316a0 con 0x55ffefdf64d0 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.827+0000 7f0e497fa640 1 -- 172.21.15.33:0/2956662323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54035040 con 0x55ffefdf64d0 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 -- 172.21.15.33:0/2956662323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 msgr2=0x55ffefdf68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefdf68b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f0e54000c00 tx=0x7f0e54031c40 comp rx=0 tx=0).stop 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6259c640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0e4c0b5360 0x7f0e4c0b7820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ffefdf51e0 tx=0x7f0e58005cc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 -- 172.21.15.33:0/2956662323 shutdown_connections 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefe02d80 0x55ffefe05170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe02840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2956662323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefdf68b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 -- 172.21.15.33:0/2956662323 >> 172.21.15.33:0/2956662323 conn(0x55ffefdf4300 msgr2=0x55ffefdf54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 -- 172.21.15.33:0/2956662323 shutdown_connections 2024-09-10T21:25:38.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.828+0000 7f0e6b455740 1 -- 172.21.15.33:0/2956662323 wait complete. 2024-09-10T21:25:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 -- start start 2024-09-10T21:25:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 Processor -- start 2024-09-10T21:25:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefe02650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdf6df0 0x55ffefdffaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55ffefbbd700 con 0x55ffefdf64d0 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55ffefcf7160 con 0x55ffefe02d80 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e6b455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ffefe059a0 con 0x55ffefdf6df0 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefe02650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefe02650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46204/0 (socket says 172.21.15.33:46204) 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.829+0000 7f0e62d9d640 1 -- 172.21.15.33:0/220428987 learned_addr learned my addr 172.21.15.33:0/220428987 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e6b455740 10 client.0 _ll_register_callbacks cb 0x55ffefd109e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e6259c640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdf6df0 0x55ffefdffaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e6359e640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe00080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e62d9d640 1 -- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdf6df0 msgr2=0x55ffefdffaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdf6df0 0x55ffefdffaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e62d9d640 1 -- 172.21.15.33:0/220428987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 msgr2=0x55ffefe00080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe00080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e62d9d640 1 -- 172.21.15.33:0/220428987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffefd24c90 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e6359e640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefe02650 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0e54005eb0 tx=0x7f0e54009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63602316ebc3ed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e46ff5640 1 -- 172.21.15.33:0/220428987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54040030 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e540316a0 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e46ff5640 1 -- 172.21.15.33:0/220428987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0e540055e0 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e46ff5640 1 -- 172.21.15.33:0/220428987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f0e54035040 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e6b455740 10 client.19257 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.830+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55ffefe11200 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e46ff5640 1 -- 172.21.15.33:0/220428987 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e540316a0 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e46ff5640 1 -- 172.21.15.33:0/220428987 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f0e540035f0 con 0x55ffefdf64d0 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e6b455740 10 client.19257 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:38.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e6b455740 10 client.19257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e6b455740 20 client.19257 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e6b455740 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe12550 0x55ffefe14940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e6b455740 4 client.19257 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffefd6edf0 con 0x55ffefe12550 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.831+0000 7f0e6259c640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe12550 0x55ffefe14940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.832+0000 7f0e46ff5640 10 client.19257 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.832+0000 7f0e6259c640 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe12550 0x55ffefe14940 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73fa4c9e1b90bee1 server_cookie=d29113241abe5ce2 in_seq=0 out_seq=0 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.832+0000 7f0e46ff5640 1 -- 172.21.15.33:0/220428987 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ffefd6edf0 con 0x55ffefe12550 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.832+0000 7f0e46ff5640 10 client.19257 handle_command_reply: tid=0 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.832+0000 7f0e6b455740 1 client.19257 shutdown 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 msgr2=0x55ffefe02650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefe02650 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f0e54005eb0 tx=0x7f0e54009520 comp rx=0 tx=0).stop 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 shutdown_connections 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdf6df0 0x55ffefdffaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe12550 0x55ffefe14940 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 --2- 172.21.15.33:0/220428987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf64d0 0x55ffefe02650 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 >> 172.21.15.33:0/220428987 conn(0x55ffefdf4300 msgr2=0x55ffefdf9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e46ff5640 0 client.19257 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 shutdown_connections 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 1 -- 172.21.15.33:0/220428987 wait complete. 2024-09-10T21:25:38.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.833+0000 7f0e6b455740 20 client.19257 trim_cache size 0 max 16384 2024-09-10T21:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 Processor -- start 2024-09-10T21:25:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 -- start start 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe0ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefe02d80 0x55ffefdef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 0x55ffefe04170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641ac1e0 con 0x55ffefe02d80 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641adde0 con 0x55ffefdf6df0 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6b455740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641ac970 con 0x55ffefdfb440 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefe02d80 0x55ffefdef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefe02d80 0x55ffefdef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46210/0 (socket says 172.21.15.33:46210) 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 -- 172.21.15.33:0/2120971823 learned_addr learned my addr 172.21.15.33:0/2120971823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe0ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6359e640 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 0x55ffefe04170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 -- 172.21.15.33:0/2120971823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 msgr2=0x55ffefe04170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 0x55ffefe04170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 -- 172.21.15.33:0/2120971823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 msgr2=0x55ffefe0ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe0ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.849+0000 7f0e6259c640 1 -- 172.21.15.33:0/2120971823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffefdf4d80 con 0x55ffefe02d80 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.850+0000 7f0e6259c640 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefe02d80 0x55ffefdef320 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0e6410f660 tx=0x7f0e58011040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f4811922bfe201 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.850+0000 7f0e48ff9640 1 -- 172.21.15.33:0/2120971823 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5801b8b0 con 0x55ffefe02d80 2024-09-10T21:25:38.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.850+0000 7f0e48ff9640 1 -- 172.21.15.33:0/2120971823 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5802d030 con 0x55ffefe02d80 2024-09-10T21:25:38.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.850+0000 7f0e48ff9640 1 -- 172.21.15.33:0/2120971823 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58029670 con 0x55ffefe02d80 2024-09-10T21:25:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 -- 172.21.15.33:0/2120971823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefe02d80 msgr2=0x55ffefdef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefe02d80 0x55ffefdef320 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f0e6410f660 tx=0x7f0e58011040 comp rx=0 tx=0).stop 2024-09-10T21:25:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 -- 172.21.15.33:0/2120971823 shutdown_connections 2024-09-10T21:25:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 0x55ffefe04170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefe02d80 0x55ffefdef320 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 --2- 172.21.15.33:0/2120971823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefdf6df0 0x55ffefe0ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 -- 172.21.15.33:0/2120971823 >> 172.21.15.33:0/2120971823 conn(0x55ffefdf4300 msgr2=0x55ffefdfad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 -- 172.21.15.33:0/2120971823 shutdown_connections 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 -- 172.21.15.33:0/2120971823 wait complete. 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.851+0000 7f0e6b455740 1 -- start start 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 1 Processor -- start 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf6df0 0x55ffefe0b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 0x55ffefe05bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe06130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf6df0 0x55ffefe0b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e62d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf6df0 0x55ffefe0b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46216/0 (socket says 172.21.15.33:46216) 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e62d9d640 1 -- 172.21.15.33:0/3044457554 learned_addr learned my addr 172.21.15.33:0/3044457554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641ad710 con 0x55ffefdf6df0 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641ad040 con 0x55ffefe02d80 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e641aa1c0 con 0x55ffefdfb440 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6b455740 10 client.0 _ll_register_callbacks cb 0x7f0e5c005f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.852+0000 7f0e6359e640 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe06130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e62d9d640 1 -- 172.21.15.33:0/3044457554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 msgr2=0x55ffefe05bf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:25:38.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 0x55ffefe05bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e62d9d640 1 -- 172.21.15.33:0/3044457554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 msgr2=0x55ffefe06130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe06130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e62d9d640 1 -- 172.21.15.33:0/3044457554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffefe067a0 con 0x55ffefdf6df0 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e62d9d640 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf6df0 0x55ffefe0b200 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0e54041040 tx=0x7f0e54005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860cc332c7959924 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e46ff5640 1 -- 172.21.15.33:0/3044457554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e54004970 con 0x55ffefdf6df0 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e46ff5640 1 -- 172.21.15.33:0/3044457554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0e54004b10 con 0x55ffefdf6df0 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.853+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e5802d030 con 0x55ffefdf6df0 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e46ff5640 1 -- 172.21.15.33:0/3044457554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f0e54003800 con 0x55ffefdf6df0 2024-09-10T21:25:38.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 10 client.19263 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55ffefe80010 con 0x55ffefdf6df0 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e46ff5640 1 -- 172.21.15.33:0/3044457554 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5802d030 con 0x55ffefdf6df0 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e46ff5640 1 -- 172.21.15.33:0/3044457554 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f0e540042a0 con 0x55ffefdf6df0 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 10 client.19263 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 10 client.19263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 20 client.19263 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe80810 0x55ffefe82c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 4 client.19263 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.854+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffefd6a8a0 con 0x55ffefe80810 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.855+0000 7f0e6259c640 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe80810 0x55ffefe82c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.855+0000 7f0e6259c640 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe80810 0x55ffefe82c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=84b588f9c193984a server_cookie=2742bfe1017c459c in_seq=0 out_seq=0 2024-09-10T21:25:38.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.855+0000 7f0e46ff5640 10 client.19263 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e46ff5640 1 -- 172.21.15.33:0/3044457554 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f0e58034020 con 0x55ffefe80810 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e46ff5640 10 client.19263 handle_command_reply: tid=0 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 client.19263 shutdown 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf6df0 msgr2=0x55ffefe0b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf6df0 0x55ffefe0b200 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0e54041040 tx=0x7f0e54005e30 comp rx=0 tx=0).stop 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 shutdown_connections 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55ffefe02d80 0x55ffefe06130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ffefdfb440 0x55ffefe05bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55ffefe80810 0x55ffefe82c30 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 --2- 172.21.15.33:0/3044457554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55ffefdf6df0 0x55ffefe0b200 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 >> 172.21.15.33:0/3044457554 conn(0x55ffefdf4300 msgr2=0x55ffefdfe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:38.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e46ff5640 0 client.19263 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:25:38.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 shutdown_connections 2024-09-10T21:25:38.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 1 -- 172.21.15.33:0/3044457554 wait complete. 2024-09-10T21:25:38.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.858+0000 7f0e6b455740 20 client.19263 trim_cache size 0 max 16384 2024-09-10T21:25:38.865 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0e4c0b5360 msgr2=0x7f0e4c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0e4c0b5360 0x7f0e4c0b7820 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x55ffefdf51e0 tx=0x7f0e58005cc0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 msgr2=0x7f0e6419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 0x7f0e6419eb40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00e940 tx=0x7f0e5c00ee00 comp rx=0 tx=0).stop 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 shutdown_connections 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0e4c0b5360 0x7f0e4c0b7820 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e6410d100 0x7f0e6419eb40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e641045c0 0x7f0e641997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 --2- 172.21.15.33:0/3436985870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e64103ca0 0x7f0e64199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:38.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 >> 172.21.15.33:0/3436985870 conn(0x7f0e640fd700 msgr2=0x7f0e640fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:38.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 shutdown_connections 2024-09-10T21:25:38.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:38.859+0000 7f0e69080640 1 -- 172.21.15.33:0/3436985870 wait complete. 2024-09-10T21:25:39.074 INFO:tasks.workunit.client.0.smithi033.stdout: 10 388483 38.67 MB/sec execute 399 sec latency 147.594 ms 2024-09-10T21:25:39.609 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:25:39.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:39 smithi174 ceph-mon[35635]: pgmap v996: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 732 op/s 2024-09-10T21:25:39.911 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:39.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:39 smithi033 ceph-mon[30860]: pgmap v996: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 732 op/s 2024-09-10T21:25:40.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:39 smithi104 ceph-mon[35325]: pgmap v996: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 13 MiB/s wr, 732 op/s 2024-09-10T21:25:40.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 389189 38.66 MB/sec execute 400 sec latency 138.279 ms 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 -- 172.21.15.33:0/3015271635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45701076a0 msgr2=0x7f4570107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 --2- 172.21.15.33:0/3015271635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45701076a0 0x7f4570107ae0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f4564009900 tx=0x7f456402a410 comp rx=0 tx=0).stop 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 -- 172.21.15.33:0/3015271635 shutdown_connections 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 --2- 172.21.15.33:0/3015271635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4570108020 0x7f457010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 --2- 172.21.15.33:0/3015271635 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45701076a0 0x7f4570107ae0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 --2- 172.21.15.33:0/3015271635 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4570101c90 0x7f4570102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 -- 172.21.15.33:0/3015271635 >> 172.21.15.33:0/3015271635 conn(0x7f45700fd700 msgr2=0x7f45700ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 -- 172.21.15.33:0/3015271635 shutdown_connections 2024-09-10T21:25:40.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.426+0000 7f457643e640 1 -- 172.21.15.33:0/3015271635 wait complete. 2024-09-10T21:25:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 Processor -- start 2024-09-10T21:25:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 -- start start 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4570101c90 0x7f457019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701076a0 0x7f457019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570108020 0x7f45701a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4570111cf0 con 0x7f4570108020 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4570111b70 con 0x7f45701076a0 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f457643e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4570111e70 con 0x7f4570101c90 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f45749db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570108020 0x7f45701a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f45749db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570108020 0x7f45701a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52584/0 (socket says 172.21.15.33:52584) 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f45749db640 1 -- 172.21.15.33:0/2009271165 learned_addr learned my addr 172.21.15.33:0/2009271165 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.427+0000 7f456ffff640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4570101c90 0x7f457019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f45749db640 1 -- 172.21.15.33:0/2009271165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4570101c90 msgr2=0x7f457019b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f45749db640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4570101c90 0x7f457019b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f45749db640 1 -- 172.21.15.33:0/2009271165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701076a0 msgr2=0x7f457019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f45749db640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701076a0 0x7f457019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f45749db640 1 -- 172.21.15.33:0/2009271165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45701a1560 con 0x7f4570108020 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f45749db640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570108020 0x7f45701a0d90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f456800eb10 tx=0x7f456800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f456d7fa640 1 -- 172.21.15.33:0/2009271165 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568004b20 con 0x7f4570108020 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45701a1850 con 0x7f4570108020 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.428+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701a7f10 con 0x7f4570108020 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.429+0000 7f456d7fa640 1 -- 172.21.15.33:0/2009271165 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4568004cc0 con 0x7f4570108020 2024-09-10T21:25:40.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.429+0000 7f456d7fa640 1 -- 172.21.15.33:0/2009271165 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4568010690 con 0x7f4570108020 2024-09-10T21:25:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.431+0000 7f4578813740 1 Processor -- start 2024-09-10T21:25:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.431+0000 7f456d7fa640 1 -- 172.21.15.33:0/2009271165 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4568010830 con 0x7f4570108020 2024-09-10T21:25:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.431+0000 7f4578813740 1 -- start start 2024-09-10T21:25:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f4578813740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f4578813740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f4578813740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f4578813740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aad3447ea0 con 0x55aad34d14d0 2024-09-10T21:25:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f4578813740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55aad34372d0 con 0x55aad34e6590 2024-09-10T21:25:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f4578813740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aad343b0e0 con 0x55aad34d1df0 2024-09-10T21:25:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39462/0 (socket says 172.21.15.33:39462) 2024-09-10T21:25:40.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 -- 172.21.15.33:0/3910823489 learned_addr learned my addr 172.21.15.33:0/3910823489 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456f7fe640 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456d7fa640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45580b5360 0x7f45580b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 -- 172.21.15.33:0/3910823489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 msgr2=0x55aad34e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 -- 172.21.15.33:0/3910823489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 msgr2=0x55aad34d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456ffff640 1 -- 172.21.15.33:0/3910823489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad3195650 con 0x55aad34d1df0 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.432+0000 7f456f7fe640 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.433+0000 7f45749db640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45580b5360 0x7f45580b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.433+0000 7f456ffff640 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e6050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4560009cf0 tx=0x7f4560011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6821782b3a651cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.433+0000 7f456d7fa640 1 -- 172.21.15.33:0/2009271165 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f45680e5f90 con 0x7f4570108020 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.433+0000 7f45567fc640 1 -- 172.21.15.33:0/3910823489 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560004230 con 0x55aad34d1df0 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.433+0000 7f45749db640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45580b5360 0x7f45580b7820 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f45680049d0 tx=0x7f456807ad30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.433+0000 7f45567fc640 1 -- 172.21.15.33:0/3910823489 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45600043d0 con 0x55aad34d1df0 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.433+0000 7f45567fc640 1 -- 172.21.15.33:0/3910823489 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560019880 con 0x55aad34d1df0 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 -- 172.21.15.33:0/3910823489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 msgr2=0x55aad34e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e6050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4560009cf0 tx=0x7f4560011810 comp rx=0 tx=0).stop 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 -- 172.21.15.33:0/3910823489 shutdown_connections 2024-09-10T21:25:40.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e6050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 --2- 172.21.15.33:0/3910823489 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 -- 172.21.15.33:0/3910823489 >> 172.21.15.33:0/3910823489 conn(0x55aad34cf300 msgr2=0x55aad34d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:40.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 -- 172.21.15.33:0/3910823489 shutdown_connections 2024-09-10T21:25:40.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.434+0000 7f4578813740 1 -- 172.21.15.33:0/3910823489 wait complete. 2024-09-10T21:25:40.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 -- start start 2024-09-10T21:25:40.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 Processor -- start 2024-09-10T21:25:40.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34e5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55aad3298700 con 0x55aad34d14d0 2024-09-10T21:25:40.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55aad33d2160 con 0x55aad34e6590 2024-09-10T21:25:40.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aad34e9260 con 0x55aad34d1df0 2024-09-10T21:25:40.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f4578813740 10 client.0 _ll_register_callbacks cb 0x55aad33eb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34e5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34e5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52614/0 (socket says 172.21.15.33:52614) 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f456ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.435+0000 7f456f7fe640 1 -- 172.21.15.33:0/2500769909 learned_addr learned my addr 172.21.15.33:0/2500769909 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f456ffff640 1 -- 172.21.15.33:0/2500769909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 msgr2=0x55aad34e3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f456ffff640 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f456ffff640 1 -- 172.21.15.33:0/2500769909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 msgr2=0x55aad34e5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f456ffff640 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34e5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f456ffff640 1 -- 172.21.15.33:0/2500769909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad33ffc90 con 0x55aad34d1df0 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f456f7fe640 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f456ffff640 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e31f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4560019520 tx=0x7f45600038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd54ff75394ece75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f454f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:25:40.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f454f7fe640 1 -- 172.21.15.33:0/2500769909 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560027070 con 0x55aad34d1df0 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.436+0000 7f454f7fe640 1 -- 172.21.15.33:0/2500769909 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4560005240 con 0x55aad34d1df0 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45600043d0 con 0x55aad34d1df0 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f454f7fe640 1 -- 172.21.15.33:0/2500769909 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f4560033050 con 0x55aad34d1df0 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 10 client.28495 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55aad34f4b40 con 0x55aad34d1df0 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f454f7fe640 1 -- 172.21.15.33:0/2500769909 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45600043d0 con 0x55aad34d1df0 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f454f7fe640 1 -- 172.21.15.33:0/2500769909 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f4560004b70 con 0x55aad34d1df0 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 10 client.28495 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 10 client.28495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 20 client.28495 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad34f5fa0 0x55aad34f8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 4 client.28495 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:40.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.437+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aad3449df0 con 0x55aad34f5fa0 2024-09-10T21:25:40.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.438+0000 7f456f7fe640 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad34f5fa0 0x55aad34f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.438+0000 7f456f7fe640 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad34f5fa0 0x55aad34f8390 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf19fe068a83d43d server_cookie=effa614ffe2fd96b in_seq=0 out_seq=0 2024-09-10T21:25:40.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.438+0000 7f454f7fe640 10 client.28495 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:40.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f454f7fe640 1 -- 172.21.15.33:0/2500769909 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55aad3449df0 con 0x55aad34f5fa0 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f454f7fe640 10 client.28495 handle_command_reply: tid=0 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 client.28495 shutdown 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 msgr2=0x55aad34e31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e31f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4560019520 tx=0x7f45600038a0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 shutdown_connections 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad34f5fa0 0x55aad34f8390 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d1df0 0x55aad34e31f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 --2- 172.21.15.33:0/2500769909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34e5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f454f7fe640 0 client.28495 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:40.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.439+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 >> 172.21.15.33:0/2500769909 conn(0x55aad34cf300 msgr2=0x55aad34d4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:40.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.440+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 shutdown_connections 2024-09-10T21:25:40.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.440+0000 7f4578813740 1 -- 172.21.15.33:0/2500769909 wait complete. 2024-09-10T21:25:40.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.440+0000 7f4578813740 20 client.28495 trim_cache size 0 max 16384 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 Processor -- start 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 -- start start 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d14d0 0x55aad34cec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d1df0 0x55aad34ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f455c00e490 con 0x55aad34d1df0 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f455c010090 con 0x55aad34e6590 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f4578813740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f455c00ec20 con 0x55aad34d14d0 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f456ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d1df0 0x55aad34ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f456ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d1df0 0x55aad34ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52624/0 (socket says 172.21.15.33:52624) 2024-09-10T21:25:40.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f456ffff640 1 -- 172.21.15.33:0/1055507750 learned_addr learned my addr 172.21.15.33:0/1055507750 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.457+0000 7f456f7fe640 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d14d0 0x55aad34cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f456f7fe640 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f456ffff640 1 -- 172.21.15.33:0/1055507750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d14d0 msgr2=0x55aad34cec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f456ffff640 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d14d0 0x55aad34cec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f456ffff640 1 -- 172.21.15.33:0/1055507750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 msgr2=0x55aad34e2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f456ffff640 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f456ffff640 1 -- 172.21.15.33:0/1055507750 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aad34d1c20 con 0x55aad34d1df0 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f456ffff640 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d1df0 0x55aad34ca320 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f4560009250 tx=0x7f4560019550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34507ef6503d8fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f4555ffb640 1 -- 172.21.15.33:0/1055507750 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560027040 con 0x55aad34d1df0 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f4555ffb640 1 -- 172.21.15.33:0/1055507750 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4560009910 con 0x55aad34d1df0 2024-09-10T21:25:40.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.458+0000 7f4555ffb640 1 -- 172.21.15.33:0/1055507750 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560005320 con 0x55aad34d1df0 2024-09-10T21:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 -- 172.21.15.33:0/1055507750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d1df0 msgr2=0x55aad34ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d1df0 0x55aad34ca320 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f4560009250 tx=0x7f4560019550 comp rx=0 tx=0).stop 2024-09-10T21:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 -- 172.21.15.33:0/1055507750 shutdown_connections 2024-09-10T21:25:40.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34e6590 0x55aad34e2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d1df0 0x55aad34ca320 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 --2- 172.21.15.33:0/1055507750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34d14d0 0x55aad34cec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 -- 172.21.15.33:0/1055507750 >> 172.21.15.33:0/1055507750 conn(0x55aad34cf300 msgr2=0x55aad34d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 -- 172.21.15.33:0/1055507750 shutdown_connections 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.459+0000 7f4578813740 1 -- 172.21.15.33:0/1055507750 wait complete. 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 -- start start 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 Processor -- start 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34d1df0 0x55aad34ea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34e6590 0x55aad34eaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f455c00f9c0 con 0x55aad34d14d0 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f455c00f2f0 con 0x55aad34d1df0 2024-09-10T21:25:40.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.460+0000 7f4578813740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f455c00c450 con 0x55aad34e6590 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f4578813740 10 client.0 _ll_register_callbacks cb 0x7f4560009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34d1df0 0x55aad34ea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34eeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52626/0 (socket says 172.21.15.33:52626) 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 -- 172.21.15.33:0/92612383 learned_addr learned my addr 172.21.15.33:0/92612383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34e6590 0x55aad34eaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 -- 172.21.15.33:0/92612383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34e6590 msgr2=0x55aad34eaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34e6590 0x55aad34eaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 -- 172.21.15.33:0/92612383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34d1df0 msgr2=0x55aad34ea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34d1df0 0x55aad34ea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 -- 172.21.15.33:0/92612383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aad3561df0 con 0x55aad34d14d0 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f456f7fe640 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34eeea0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f456402a930 tx=0x7f45600045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508504d9d7f04e35 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:40.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f454f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f454f7fe640 1 -- 172.21.15.33:0/92612383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564035040 con 0x55aad34d14d0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f454f7fe640 1 -- 172.21.15.33:0/92612383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4564003e20 con 0x55aad34d14d0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.461+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4560009910 con 0x55aad34d14d0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f454f7fe640 1 -- 172.21.15.33:0/92612383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f45640314a0 con 0x55aad34d14d0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 10 client.19281 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55aad3562fc0 con 0x55aad34d14d0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f454f7fe640 1 -- 172.21.15.33:0/92612383 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4560009910 con 0x55aad34d14d0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f454f7fe640 1 -- 172.21.15.33:0/92612383 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7f456403a040 con 0x55aad34d14d0 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 10 client.19281 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 10 client.19281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 20 client.19281 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad3563720 0x55aad3565b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:40.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 4 client.19281 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aad34458a0 con 0x55aad3563720 2024-09-10T21:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.462+0000 7f456ffff640 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad3563720 0x55aad3565b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f456ffff640 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad3563720 0x55aad3565b40 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2435d52a03a7552 server_cookie=10353549e9a697ab in_seq=0 out_seq=0 2024-09-10T21:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f454f7fe640 10 client.19281 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f454f7fe640 1 -- 172.21.15.33:0/92612383 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aad34458a0 con 0x55aad3563720 2024-09-10T21:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f454f7fe640 10 client.19281 handle_command_reply: tid=0 2024-09-10T21:25:40.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 client.19281 shutdown 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 msgr2=0x55aad34eeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34eeea0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f456402a930 tx=0x7f45600045c0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 shutdown_connections 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55aad3563720 0x55aad3565b40 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aad34e6590 0x55aad34eaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55aad34d1df0 0x55aad34ea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f454f7fe640 0 client.19281 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 --2- 172.21.15.33:0/92612383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55aad34d14d0 0x55aad34eeea0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 >> 172.21.15.33:0/92612383 conn(0x55aad34cf300 msgr2=0x55aad34e2030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:40.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 shutdown_connections 2024-09-10T21:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.463+0000 7f4578813740 1 -- 172.21.15.33:0/92612383 wait complete. 2024-09-10T21:25:40.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f4578813740 20 client.19281 trim_cache size 0 max 16384 2024-09-10T21:25:40.471 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45580b5360 msgr2=0x7f45580b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45580b5360 0x7f45580b7820 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f45680049d0 tx=0x7f456807ad30 comp rx=0 tx=0).stop 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570108020 msgr2=0x7f45701a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570108020 0x7f45701a0d90 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f456800eb10 tx=0x7f456800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 shutdown_connections 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f45580b5360 0x7f45580b7820 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4570108020 0x7f45701a0d90 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45701076a0 0x7f457019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 --2- 172.21.15.33:0/2009271165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4570101c90 0x7f457019b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.464+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 >> 172.21.15.33:0/2009271165 conn(0x7f45700fd700 msgr2=0x7f457006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:40.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.465+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 shutdown_connections 2024-09-10T21:25:40.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:40.465+0000 7f457643e640 1 -- 172.21.15.33:0/2009271165 wait complete. 2024-09-10T21:25:41.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 390030 38.68 MB/sec execute 401 sec latency 116.486 ms 2024-09-10T21:25:41.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:41 smithi174 ceph-mon[35635]: pgmap v997: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 835 op/s 2024-09-10T21:25:41.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:41 smithi033 ceph-mon[30860]: pgmap v997: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 835 op/s 2024-09-10T21:25:42.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:41 smithi104 ceph-mon[35325]: pgmap v997: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 835 op/s 2024-09-10T21:25:42.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 390687 38.70 MB/sec execute 402 sec latency 238.246 ms 2024-09-10T21:25:42.214 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:25:42.214 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:25:42.537 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:43.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 391035 38.64 MB/sec execute 403 sec latency 293.622 ms 2024-09-10T21:25:43.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:43 smithi174 ceph-mon[35635]: pgmap v998: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.6 MiB/s wr, 599 op/s 2024-09-10T21:25:43.924 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:43 smithi033 ceph-mon[30860]: pgmap v998: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.6 MiB/s wr, 599 op/s 2024-09-10T21:25:44.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.037+0000 7fac56cc6640 1 -- 172.21.15.33:0/3709705310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 msgr2=0x7fac5010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.037+0000 7fac56cc6640 1 --2- 172.21.15.33:0/3709705310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5010f540 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fac48009ca0 tx=0x7fac4802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.037+0000 7fac56cc6640 1 -- 172.21.15.33:0/3709705310 shutdown_connections 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.037+0000 7fac56cc6640 1 --2- 172.21.15.33:0/3709705310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5010f540 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 --2- 172.21.15.33:0/3709705310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac50104510 0x7fac5010caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 --2- 172.21.15.33:0/3709705310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac50103b10 0x7fac50103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 -- 172.21.15.33:0/3709705310 >> 172.21.15.33:0/3709705310 conn(0x7fac500fd700 msgr2=0x7fac500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 -- 172.21.15.33:0/3709705310 shutdown_connections 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 -- 172.21.15.33:0/3709705310 wait complete. 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 Processor -- start 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 -- start start 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac50103b10 0x7fac50199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac50104510 0x7fac501997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac50112d90 con 0x7fac5010d0c0 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac50112c10 con 0x7fac50104510 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.038+0000 7fac56cc6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac50112f10 con 0x7fac50103b10 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52648/0 (socket says 172.21.15.33:52648) 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 -- 172.21.15.33:0/1598500193 learned_addr learned my addr 172.21.15.33:0/1598500193 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac54a62640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac50103b10 0x7fac50199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac4ffff640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac50104510 0x7fac501997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 -- 172.21.15.33:0/1598500193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac50103b10 msgr2=0x7fac50199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac50103b10 0x7fac50199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 -- 172.21.15.33:0/1598500193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac50104510 msgr2=0x7fac501997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac50104510 0x7fac501997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac55263640 1 -- 172.21.15.33:0/1598500193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5019f310 con 0x7fac5010d0c0 2024-09-10T21:25:44.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac4ffff640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac50104510 0x7fac501997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:44.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.039+0000 7fac54a62640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac50103b10 0x7fac50199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:44.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.040+0000 7fac55263640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5019eb40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fac480098d0 tx=0x7fac480317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:44.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.040+0000 7fac4dffb640 1 -- 172.21.15.33:0/1598500193 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48003b70 con 0x7fac5010d0c0 2024-09-10T21:25:44.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.040+0000 7fac4dffb640 1 -- 172.21.15.33:0/1598500193 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac48003d10 con 0x7fac5010d0c0 2024-09-10T21:25:44.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.040+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5019f5a0 con 0x7fac5010d0c0 2024-09-10T21:25:44.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.040+0000 7fac4dffb640 1 -- 172.21.15.33:0/1598500193 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac48003b70 con 0x7fac5010d0c0 2024-09-10T21:25:44.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.040+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac501a5d40 con 0x7fac5010d0c0 2024-09-10T21:25:44.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.043+0000 7fac4dffb640 1 -- 172.21.15.33:0/1598500193 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fac48004040 con 0x7fac5010d0c0 2024-09-10T21:25:44.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.043+0000 7fac5909b740 1 Processor -- start 2024-09-10T21:25:44.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.043+0000 7fac5909b740 1 -- start start 2024-09-10T21:25:44.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac5909b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a4492c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac5909b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 0x55bf1a459660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac5909b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac5909b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a3bfe60 con 0x55bf1a459ba0 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac5909b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a3af290 con 0x55bf1a448ee0 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac5909b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a3b30a0 con 0x55bf1a449800 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac4ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a4492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac4ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a4492c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49396/0 (socket says 172.21.15.33:49396) 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac4ffff640 1 -- 172.21.15.33:0/616667228 learned_addr learned my addr 172.21.15.33:0/616667228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac4dffb640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac400b54e0 0x7fac400b79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac4ffff640 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac4dffb640 1 -- 172.21.15.33:0/1598500193 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fac48106990 con 0x7fac5010d0c0 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.044+0000 7fac54a62640 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 0x55bf1a459660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac4ffff640 1 -- 172.21.15.33:0/616667228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 msgr2=0x55bf1a459660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac4ffff640 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 0x55bf1a459660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac4ffff640 1 -- 172.21.15.33:0/616667228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 msgr2=0x55bf1a45bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac4ffff640 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac4ffff640 1 -- 172.21.15.33:0/616667228 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1a10d930 con 0x55bf1a448ee0 2024-09-10T21:25:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac4ffff640 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a4492c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac3c015aa0 tx=0x7fac3c0098f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650de763536e3b84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:44.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac55263640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac400b54e0 0x7fac400b79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac3affd640 1 -- 172.21.15.33:0/616667228 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c00e8a0 con 0x55bf1a448ee0 2024-09-10T21:25:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac3affd640 1 -- 172.21.15.33:0/616667228 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3c01d070 con 0x55bf1a448ee0 2024-09-10T21:25:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.045+0000 7fac3affd640 1 -- 172.21.15.33:0/616667228 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c018ce0 con 0x55bf1a448ee0 2024-09-10T21:25:44.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.046+0000 7fac5909b740 1 -- 172.21.15.33:0/616667228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 msgr2=0x55bf1a4492c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.046+0000 7fac5909b740 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a4492c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac3c015aa0 tx=0x7fac3c0098f0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac55263640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac400b54e0 0x7fac400b79a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fac480cd0a0 tx=0x7fac480b4040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac5909b740 1 -- 172.21.15.33:0/616667228 shutdown_connections 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac5909b740 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac5909b740 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 0x55bf1a459660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac5909b740 1 --2- 172.21.15.33:0/616667228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a4492c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac5909b740 1 -- 172.21.15.33:0/616667228 >> 172.21.15.33:0/616667228 conn(0x55bf1a446d10 msgr2=0x55bf1a447eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac5909b740 1 -- 172.21.15.33:0/616667228 shutdown_connections 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.047+0000 7fac5909b740 1 -- 172.21.15.33:0/616667228 wait complete. 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.048+0000 7fac5909b740 1 -- start start 2024-09-10T21:25:44.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.048+0000 7fac5909b740 1 Processor -- start 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.048+0000 7fac5909b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a456b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac5909b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a457070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac5909b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a4575b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac5909b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a210890 con 0x55bf1a449800 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac5909b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a349d40 con 0x55bf1a448ee0 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac5909b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf1a45c7f0 con 0x55bf1a459ba0 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac5909b740 10 client.0 _ll_register_callbacks cb 0x55bf1a3adc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a456b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a456b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49406/0 (socket says 172.21.15.33:49406) 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 -- 172.21.15.33:0/2027234860 learned_addr learned my addr 172.21.15.33:0/2027234860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac54a62640 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a457070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a4575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 -- 172.21.15.33:0/2027234860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 msgr2=0x55bf1a4575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a4575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 -- 172.21.15.33:0/2027234860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 msgr2=0x55bf1a457070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a457070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.049+0000 7fac4ffff640 1 -- 172.21.15.33:0/2027234860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1a465b80 con 0x55bf1a448ee0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a456b30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009ad0 tx=0x7fac3c018820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c53d26c14eac4c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac2ffff640 1 -- 172.21.15.33:0/2027234860 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c029070 con 0x55bf1a448ee0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac3c01d070 con 0x55bf1a448ee0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac2ffff640 1 -- 172.21.15.33:0/2027234860 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fac3c00e8a0 con 0x55bf1a448ee0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac2ffff640 1 -- 172.21.15.33:0/2027234860 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fac3c033050 con 0x55bf1a448ee0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac5909b740 10 client.28628 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.051+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55bf1a468530 con 0x55bf1a448ee0 2024-09-10T21:25:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac2ffff640 1 -- 172.21.15.33:0/2027234860 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3c01d070 con 0x55bf1a448ee0 2024-09-10T21:25:44.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac2ffff640 1 -- 172.21.15.33:0/2027234860 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fac3c02ea10 con 0x55bf1a448ee0 2024-09-10T21:25:44.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac5909b740 10 client.28628 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:44.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac5909b740 10 client.28628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:44.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac5909b740 20 client.28628 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:44.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac5909b740 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a469940 0x55bf1a46bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac5909b740 4 client.28628 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:44.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.052+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf1a3c1db0 con 0x55bf1a469940 2024-09-10T21:25:44.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.056+0000 7fac54a62640 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a469940 0x55bf1a46bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.058+0000 7fac54a62640 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a469940 0x55bf1a46bd30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec459926e980fec4 server_cookie=1168e4133507fa00 in_seq=0 out_seq=0 2024-09-10T21:25:44.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.058+0000 7fac2ffff640 10 client.28628 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:44.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:43 smithi104 ceph-mon[35325]: pgmap v998: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.6 MiB/s wr, 599 op/s 2024-09-10T21:25:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.069+0000 7fac2ffff640 1 -- 172.21.15.33:0/2027234860 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bf1a3c1db0 con 0x55bf1a469940 2024-09-10T21:25:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.069+0000 7fac2ffff640 10 client.28628 handle_command_reply: tid=0 2024-09-10T21:25:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.069+0000 7fac5909b740 1 client.28628 shutdown 2024-09-10T21:25:44.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.069+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 msgr2=0x55bf1a456b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.069+0000 7fac5909b740 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a456b30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fac3c009ad0 tx=0x7fac3c018820 comp rx=0 tx=0).stop 2024-09-10T21:25:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 shutdown_connections 2024-09-10T21:25:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a469940 0x55bf1a46bd30 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac2ffff640 0 client.28628 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:44.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a4575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a457070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 --2- 172.21.15.33:0/2027234860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a456b30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 >> 172.21.15.33:0/2027234860 conn(0x55bf1a446d10 msgr2=0x55bf1a445f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 shutdown_connections 2024-09-10T21:25:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 1 -- 172.21.15.33:0/2027234860 wait complete. 2024-09-10T21:25:44.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.070+0000 7fac5909b740 20 client.28628 trim_cache size 0 max 16384 2024-09-10T21:25:44.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 391432 38.59 MB/sec execute 404 sec latency 389.071 ms 2024-09-10T21:25:44.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.101+0000 7fac5909b740 1 Processor -- start 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.101+0000 7fac5909b740 1 -- start start 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac5909b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a458820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac5909b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a456040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac5909b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a471360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac5909b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ac1e0 con 0x55bf1a449800 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac5909b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac501adde0 con 0x55bf1a448ee0 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac5909b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ac970 con 0x55bf1a459ba0 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac54a62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a456040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac54a62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a456040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52676/0 (socket says 172.21.15.33:52676) 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac54a62640 1 -- 172.21.15.33:0/2767005011 learned_addr learned my addr 172.21.15.33:0/2767005011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:44.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a458820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.102+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a471360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac4ffff640 1 -- 172.21.15.33:0/2767005011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 msgr2=0x55bf1a471360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a471360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac4ffff640 1 -- 172.21.15.33:0/2767005011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 msgr2=0x55bf1a456040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a456040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac4ffff640 1 -- 172.21.15.33:0/2767005011 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf1a454a70 con 0x55bf1a448ee0 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac54a62640 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a456040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac4ffff640 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a458820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fac3c02ee10 tx=0x7fac3c009110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5813f58b2c6cbdc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.103+0000 7fac3a7fc640 1 -- 172.21.15.33:0/2767005011 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c029070 con 0x55bf1a448ee0 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac3a7fc640 1 -- 172.21.15.33:0/2767005011 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac3c01d070 con 0x55bf1a448ee0 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac3a7fc640 1 -- 172.21.15.33:0/2767005011 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c000e60 con 0x55bf1a448ee0 2024-09-10T21:25:44.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac5909b740 1 -- 172.21.15.33:0/2767005011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 msgr2=0x55bf1a458820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac5909b740 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a458820 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fac3c02ee10 tx=0x7fac3c009110 comp rx=0 tx=0).stop 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac5909b740 1 -- 172.21.15.33:0/2767005011 shutdown_connections 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac5909b740 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a459ba0 0x55bf1a471360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac5909b740 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a449800 0x55bf1a456040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.104+0000 7fac5909b740 1 --2- 172.21.15.33:0/2767005011 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a458820 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.105+0000 7fac5909b740 1 -- 172.21.15.33:0/2767005011 >> 172.21.15.33:0/2767005011 conn(0x55bf1a446d10 msgr2=0x55bf1a458110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.105+0000 7fac5909b740 1 -- 172.21.15.33:0/2767005011 shutdown_connections 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.105+0000 7fac5909b740 1 -- 172.21.15.33:0/2767005011 wait complete. 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.106+0000 7fac5909b740 1 -- start start 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.106+0000 7fac5909b740 1 Processor -- start 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.106+0000 7fac5909b740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a461f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac5909b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 0x55bf1a4624e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac5909b740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac5909b740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ad710 con 0x55bf1a459ba0 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac5909b740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fac501ad040 con 0x55bf1a448ee0 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac4ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a461f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac5909b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac501aa1c0 con 0x55bf1a449800 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac4ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a461f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49422/0 (socket says 172.21.15.33:49422) 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac4ffff640 1 -- 172.21.15.33:0/1685050088 learned_addr learned my addr 172.21.15.33:0/1685050088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.107+0000 7fac4ffff640 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac5909b740 10 client.0 _ll_register_callbacks cb 0x7fac3c015400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac4ffff640 1 -- 172.21.15.33:0/1685050088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 msgr2=0x55bf1a4624e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac4ffff640 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 0x55bf1a4624e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac4ffff640 1 -- 172.21.15.33:0/1685050088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 msgr2=0x55bf1a461f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:44.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac4ffff640 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a461f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac4ffff640 1 -- 172.21.15.33:0/1685050088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf1a45eec0 con 0x55bf1a459ba0 2024-09-10T21:25:44.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac4ffff640 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45e690 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fac3c018930 tx=0x7fac3c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11737ded24d3010f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:44.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:44.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac3c01d070 con 0x55bf1a459ba0 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac2ffff640 1 -- 172.21.15.33:0/1685050088 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c029070 con 0x55bf1a459ba0 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.108+0000 7fac2ffff640 1 -- 172.21.15.33:0/1685050088 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fac3c0154b0 con 0x55bf1a459ba0 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac2ffff640 1 -- 172.21.15.33:0/1685050088 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fac3c009e60 con 0x55bf1a459ba0 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 10 client.19299 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=84}) -- 0x55bf1a4d71b0 con 0x55bf1a459ba0 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac2ffff640 1 -- 172.21.15.33:0/1685050088 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fac3c02e440 con 0x55bf1a459ba0 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 10 client.19299 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:44.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 10 client.19299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 20 client.19299 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a4d77c0 0x55bf1a4d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 4 client.19299 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:25:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bf1a3bd860 con 0x55bf1a4d77c0 2024-09-10T21:25:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.109+0000 7fac54a62640 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a4d77c0 0x55bf1a4d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.110+0000 7fac54a62640 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a4d77c0 0x55bf1a4d9be0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8396705c84fe22c2 server_cookie=223e950dec5da4d3 in_seq=0 out_seq=0 2024-09-10T21:25:44.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.110+0000 7fac2ffff640 10 client.19299 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.120+0000 7fac2ffff640 1 -- 172.21.15.33:0/1685050088 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bf1a3bd860 con 0x55bf1a4d77c0 2024-09-10T21:25:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.120+0000 7fac2ffff640 10 client.19299 handle_command_reply: tid=0 2024-09-10T21:25:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.120+0000 7fac5909b740 1 client.19299 shutdown 2024-09-10T21:25:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.120+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 msgr2=0x55bf1a45e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.120+0000 7fac5909b740 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45e690 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fac3c018930 tx=0x7fac3c01a040 comp rx=0 tx=0).stop 2024-09-10T21:25:44.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.121+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 shutdown_connections 2024-09-10T21:25:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.121+0000 7fac5909b740 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55bf1a4d77c0 0x55bf1a4d9be0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.121+0000 7fac5909b740 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55bf1a459ba0 0x55bf1a45e690 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.121+0000 7fac5909b740 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf1a449800 0x55bf1a4624e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.121+0000 7fac5909b740 1 --2- 172.21.15.33:0/1685050088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55bf1a448ee0 0x55bf1a461f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.122+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 >> 172.21.15.33:0/1685050088 conn(0x55bf1a446d10 msgr2=0x55bf1a458930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:44.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.122+0000 7fac2ffff640 0 client.19299 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.122+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 shutdown_connections 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.122+0000 7fac5909b740 1 -- 172.21.15.33:0/1685050088 wait complete. 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.122+0000 7fac5909b740 20 client.19299 trim_cache size 0 max 16384 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "59291033-0fe1-4840-9451-8996afd161e3", 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:25:44.126 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:25:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.123+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac400b54e0 msgr2=0x7fac400b79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.123+0000 7fac56cc6640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac400b54e0 0x7fac400b79a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fac480cd0a0 tx=0x7fac480b4040 comp rx=0 tx=0).stop 2024-09-10T21:25:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 msgr2=0x7fac5019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:44.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5019eb40 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fac480098d0 tx=0x7fac480317b0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 shutdown_connections 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fac400b54e0 0x7fac400b79a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac5010d0c0 0x7fac5019eb40 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac50104510 0x7fac501997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 --2- 172.21.15.33:0/1598500193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac50103b10 0x7fac50199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 >> 172.21.15.33:0/1598500193 conn(0x7fac500fd700 msgr2=0x7fac5010f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 shutdown_connections 2024-09-10T21:25:44.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:44.124+0000 7fac56cc6640 1 -- 172.21.15.33:0/1598500193 wait complete. 2024-09-10T21:25:44.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:44 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:25:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:44 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:44 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:25:44.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:44 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:25:44.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:44 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:44.915 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:44 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:25:44.952 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:25:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:44 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:25:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:44 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:44 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:25:45.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 391833 38.53 MB/sec execute 405 sec latency 313.892 ms 2024-09-10T21:25:45.288 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: pgmap v999: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 819 op/s 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:25:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.677 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:45 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 -- 172.21.15.33:0/3519080414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc109170 msgr2=0x7fabfc10f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 --2- 172.21.15.33:0/3519080414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc109170 0x7fabfc10f970 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fabf4009ca0 tx=0x7fabf402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 -- 172.21.15.33:0/3519080414 shutdown_connections 2024-09-10T21:25:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 --2- 172.21.15.33:0/3519080414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc109170 0x7fabfc10f970 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 --2- 172.21.15.33:0/3519080414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabfc1087f0 0x7fabfc108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 --2- 172.21.15.33:0/3519080414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 0x7fabfc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 -- 172.21.15.33:0/3519080414 >> 172.21.15.33:0/3519080414 conn(0x7fabfc0fe870 msgr2=0x7fabfc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 -- 172.21.15.33:0/3519080414 shutdown_connections 2024-09-10T21:25:45.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.838+0000 7fac019f3640 1 -- 172.21.15.33:0/3519080414 wait complete. 2024-09-10T21:25:45.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 Processor -- start 2024-09-10T21:25:45.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 -- start start 2024-09-10T21:25:45.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 0x7fabfc075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc1087f0 0x7fabfc075a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabfc109170 0x7fabfc075f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc112e30 con 0x7fabfc1087f0 2024-09-10T21:25:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc112cb0 con 0x7fabfc109170 2024-09-10T21:25:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fac019f3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc112fb0 con 0x7fabfc102de0 2024-09-10T21:25:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fabfa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc1087f0 0x7fabfc075a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fabfaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 0x7fabfc075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fabfaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 0x7fabfc075500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:39566/0 (socket says 172.21.15.33:39566) 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fabfa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc1087f0 0x7fabfc075a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52700/0 (socket says 172.21.15.33:52700) 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fabfaffd640 1 -- 172.21.15.33:0/60730055 learned_addr learned my addr 172.21.15.33:0/60730055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.839+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabfc109170 0x7fabfc075f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fabfaffd640 1 -- 172.21.15.33:0/60730055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabfc109170 msgr2=0x7fabfc075f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fabfaffd640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabfc109170 0x7fabfc075f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fabfaffd640 1 -- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc1087f0 msgr2=0x7fabfc075a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fabfaffd640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc1087f0 0x7fabfc075a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fabfaffd640 1 -- 172.21.15.33:0/60730055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc1a7650 con 0x7fabfc102de0 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc1087f0 0x7fabfc075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fabfaffd640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 0x7fabfc075500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fabec00e9f0 tx=0x7fabec00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:45.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.840+0000 7fac009f1640 1 -- 172.21.15.33:0/60730055 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec00cd80 con 0x7fabfc102de0 2024-09-10T21:25:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.841+0000 7fac009f1640 1 -- 172.21.15.33:0/60730055 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabec0054d0 con 0x7fabfc102de0 2024-09-10T21:25:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.841+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc1a7940 con 0x7fabfc102de0 2024-09-10T21:25:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.841+0000 7fac009f1640 1 -- 172.21.15.33:0/60730055 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec010690 con 0x7fabfc102de0 2024-09-10T21:25:45.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.841+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc1a7e30 con 0x7fabfc102de0 2024-09-10T21:25:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.844+0000 7fac03dc8740 1 Processor -- start 2024-09-10T21:25:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.844+0000 7fac03dc8740 1 -- start start 2024-09-10T21:25:45.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9480 0x56425c3e9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9da0 0x56425c3f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3f5d30 0x56425c3f8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fac03dc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56425c35fe50 con 0x56425c3f5d30 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fac03dc8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56425c34f280 con 0x56425c3e9480 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fac03dc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56425c353090 con 0x56425c3e9da0 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9480 0x56425c3e9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fac009f1640 1 -- 172.21.15.33:0/60730055 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fabec004e90 con 0x7fabfc102de0 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9480 0x56425c3e9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49458/0 (socket says 172.21.15.33:49458) 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 -- 172.21.15.33:0/1456261570 learned_addr learned my addr 172.21.15.33:0/1456261570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:45.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3f5d30 0x56425c3f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 -- 172.21.15.33:0/1456261570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9da0 msgr2=0x56425c3f57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfaffd640 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9da0 0x56425c3f57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9da0 0x56425c3f57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 -- 172.21.15.33:0/1456261570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3f5d30 msgr2=0x56425c3f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3f5d30 0x56425c3f8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfb7fe640 1 -- 172.21.15.33:0/1456261570 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425c0ad650 con 0x56425c3e9480 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3f5d30 0x56425c3f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.845+0000 7fabfaffd640 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9da0 0x56425c3f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:45.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.846+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9480 0x56425c3e9860 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fabf4005fa0 tx=0x7fabf4009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca6f75ef8dae691 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.846+0000 7fac009f1640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fabe40b52e0 0x7fabe40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.847+0000 7fabe1ffb640 1 -- 172.21.15.33:0/1456261570 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf4031cf0 con 0x56425c3e9480 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.847+0000 7fac009f1640 1 -- 172.21.15.33:0/60730055 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fabec0e5fe0 con 0x7fabfc102de0 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.847+0000 7fabfaffd640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fabe40b52e0 0x7fabe40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.847+0000 7fabe1ffb640 1 -- 172.21.15.33:0/1456261570 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf4036030 con 0x56425c3e9480 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.847+0000 7fabe1ffb640 1 -- 172.21.15.33:0/1456261570 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf40042a0 con 0x56425c3e9480 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 -- 172.21.15.33:0/1456261570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9480 msgr2=0x56425c3e9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fabfaffd640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fabe40b52e0 0x7fabe40b77a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fabec0a8d70 tx=0x7fabec004a90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:45.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9480 0x56425c3e9860 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fabf4005fa0 tx=0x7fabf4009800 comp rx=0 tx=0).stop 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 -- 172.21.15.33:0/1456261570 shutdown_connections 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3f5d30 0x56425c3f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9da0 0x56425c3f57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1456261570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9480 0x56425c3e9860 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 -- 172.21.15.33:0/1456261570 >> 172.21.15.33:0/1456261570 conn(0x56425c3e72b0 msgr2=0x56425c3e8450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 -- 172.21.15.33:0/1456261570 shutdown_connections 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.848+0000 7fac03dc8740 1 -- 172.21.15.33:0/1456261570 wait complete. 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 -- start start 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 Processor -- start 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9480 0x56425c3f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9da0 0x56425c3f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3f5d30 0x56425c3f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56425c1b0700 con 0x56425c3e9da0 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56425c2ea110 con 0x56425c3f5d30 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56425c3f87d0 con 0x56425c3e9480 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.849+0000 7fac03dc8740 10 client.0 _ll_register_callbacks cb 0x56425c303990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3f5d30 0x56425c3f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3f5d30 0x56425c3f31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:49468/0 (socket says 172.21.15.33:49468) 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfb7fe640 1 -- 172.21.15.33:0/1145207866 learned_addr learned my addr 172.21.15.33:0/1145207866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9da0 0x56425c3f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9480 0x56425c3f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfa7fc640 1 -- 172.21.15.33:0/1145207866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9480 msgr2=0x56425c3f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9480 0x56425c3f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfa7fc640 1 -- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3f5d30 msgr2=0x56425c3f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3f5d30 0x56425c3f31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfa7fc640 1 -- 172.21.15.33:0/1145207866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425c317c40 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3f5d30 0x56425c3f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9da0 0x56425c3f2b00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005d00 tx=0x7fabf0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fa3b2c049f69a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.850+0000 7fabdaffd640 1 -- 172.21.15.33:0/1145207866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf001f060 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fabdaffd640 1 -- 172.21.15.33:0/1145207866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fabf001a3f0 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf4036030 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fabdaffd640 1 -- 172.21.15.33:0/1145207866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fabf0010e50 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fac03dc8740 10 client.19311 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56425c404170 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fabdaffd640 1 -- 172.21.15.33:0/1145207866 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf4036030 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fabdaffd640 1 -- 172.21.15.33:0/1145207866 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fabf0024070 con 0x56425c3e9da0 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fac03dc8740 10 client.19311 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:45.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fac03dc8740 10 client.19311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fac03dc8740 20 client.19311 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.851+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c405680 0x56425c407aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.852+0000 7fac03dc8740 4 client.19311 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.852+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425c361da0 con 0x56425c405680 2024-09-10T21:25:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.852+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c405680 0x56425c407aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.852+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c405680 0x56425c407aa0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5726337dc12c00 server_cookie=ea6faac568ae3033 in_seq=0 out_seq=0 2024-09-10T21:25:45.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.852+0000 7fabdaffd640 10 client.19311 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.853+0000 7fabdaffd640 1 -- 172.21.15.33:0/1145207866 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56425c361da0 con 0x56425c405680 2024-09-10T21:25:45.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.853+0000 7fabdaffd640 10 client.19311 handle_command_reply: tid=0 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.853+0000 7fac03dc8740 1 client.19311 shutdown 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9da0 msgr2=0x56425c3f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9da0 0x56425c3f2b00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fabf0005d00 tx=0x7fabf0012bc0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 shutdown_connections 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c405680 0x56425c407aa0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3f5d30 0x56425c3f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9da0 0x56425c3f2b00 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 --2- 172.21.15.33:0/1145207866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3e9480 0x56425c3f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 >> 172.21.15.33:0/1145207866 conn(0x56425c3e72b0 msgr2=0x56425c3ece20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fabdaffd640 0 client.19311 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 shutdown_connections 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 1 -- 172.21.15.33:0/1145207866 wait complete. 2024-09-10T21:25:45.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.854+0000 7fac03dc8740 20 client.19311 trim_cache size 0 max 16384 2024-09-10T21:25:45.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 Processor -- start 2024-09-10T21:25:45.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 -- start start 2024-09-10T21:25:45.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c401a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1b04c0 con 0x56425c3e9480 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1b20c0 con 0x56425c3e9da0 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fac03dc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1b0c50 con 0x56425c3f5d30 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c401a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c401a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52718/0 (socket says 172.21.15.33:52718) 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fabfb7fe640 1 -- 172.21.15.33:0/3846550873 learned_addr learned my addr 172.21.15.33:0/3846550873 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.878+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabfa7fc640 1 -- 172.21.15.33:0/3846550873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 msgr2=0x56425c3f7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabfa7fc640 1 -- 172.21.15.33:0/3846550873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 msgr2=0x56425c401a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c401a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabfa7fc640 1 -- 172.21.15.33:0/3846550873 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56425c3f3540 con 0x56425c3e9da0 2024-09-10T21:25:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3f28b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011010 tx=0x7fabf0005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2585c257cb1d9d94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabe17fa640 1 -- 172.21.15.33:0/3846550873 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0000e60 con 0x56425c3e9da0 2024-09-10T21:25:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabe17fa640 1 -- 172.21.15.33:0/3846550873 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf002b070 con 0x56425c3e9da0 2024-09-10T21:25:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.879+0000 7fabe17fa640 1 -- 172.21.15.33:0/3846550873 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0029070 con 0x56425c3e9da0 2024-09-10T21:25:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.880+0000 7fac03dc8740 1 -- 172.21.15.33:0/3846550873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 msgr2=0x56425c3f28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.880+0000 7fac03dc8740 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3f28b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fabf0011010 tx=0x7fabf0005c50 comp rx=0 tx=0).stop 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.880+0000 7fac03dc8740 1 -- 172.21.15.33:0/3846550873 shutdown_connections 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.880+0000 7fac03dc8740 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.880+0000 7fac03dc8740 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3f28b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.880+0000 7fac03dc8740 1 --2- 172.21.15.33:0/3846550873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c401a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.880+0000 7fac03dc8740 1 -- 172.21.15.33:0/3846550873 >> 172.21.15.33:0/3846550873 conn(0x56425c3e72b0 msgr2=0x56425c3edd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.881+0000 7fac03dc8740 1 -- 172.21.15.33:0/3846550873 shutdown_connections 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.881+0000 7fac03dc8740 1 -- 172.21.15.33:0/3846550873 wait complete. 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.881+0000 7fac03dc8740 1 -- start start 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 1 Processor -- start 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c3fda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c3fda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1b19f0 con 0x56425c3e9480 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1b1320 con 0x56425c3e9da0 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1ae480 con 0x56425c3f5d30 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fabfb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c3fda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52734/0 (socket says 172.21.15.33:52734) 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fabfb7fe640 1 -- 172.21.15.33:0/672459093 learned_addr learned my addr 172.21.15.33:0/672459093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fac03dc8740 10 client.0 _ll_register_callbacks cb 0x7fabf000bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.882+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabfb7fe640 1 -- 172.21.15.33:0/672459093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 msgr2=0x56425c3f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabfb7fe640 1 -- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 msgr2=0x56425c3fdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:45.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3fdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabfb7fe640 1 -- 172.21.15.33:0/672459093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56425c3f8f60 con 0x56425c3e9480 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabfb7fe640 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c3fda70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000f80 tx=0x7fabf4003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d5d70c8adc4481 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabdaffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabdaffd640 1 -- 172.21.15.33:0/672459093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf405d030 con 0x56425c3e9480 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fabdaffd640 1 -- 172.21.15.33:0/672459093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fabf4004350 con 0x56425c3e9480 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.883+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf002b070 con 0x56425c3e9480 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fabdaffd640 1 -- 172.21.15.33:0/672459093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fabf4058020 con 0x56425c3e9480 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 10 client.19317 fetch_fsmap learned FSMap version 84 2024-09-10T21:25:45.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56425c472c80 con 0x56425c3e9480 2024-09-10T21:25:45.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fabdaffd640 1 -- 172.21.15.33:0/672459093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf4036070 con 0x56425c3e9480 2024-09-10T21:25:45.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fabdaffd640 1 -- 172.21.15.33:0/672459093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 84) ==== 8378+0+0 (secure 0 0 0) 0x7fabf4043070 con 0x56425c3e9480 2024-09-10T21:25:45.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 10 client.19317 fetch_fsmap finished waiting for FSMap version 84 2024-09-10T21:25:45.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 10 client.19317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:25:45.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 20 client.19317 populate_metadata read hostname 'smithi033' 2024-09-10T21:25:45.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c473170 0x56425c475560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 4 client.19317 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.884+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fabf002b070 con 0x56425c473170 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.885+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c473170 0x56425c475560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.886+0000 7fabfa7fc640 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c473170 0x56425c475560 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e78945e9ac7871a1 server_cookie=67a952631c1c2c31 in_seq=0 out_seq=0 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.886+0000 7fabdaffd640 10 client.19317 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.887+0000 7fabdaffd640 1 -- 172.21.15.33:0/672459093 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8238 (crc 0 0 0) 0x7fabf002b070 con 0x56425c473170 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.887+0000 7fabdaffd640 10 client.19317 handle_command_reply: tid=0 2024-09-10T21:25:45.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.887+0000 7fac03dc8740 1 client.19317 shutdown 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 msgr2=0x56425c3fda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c3fda70 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fabf4000f80 tx=0x7fabf4003d70 comp rx=0 tx=0).stop 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 shutdown_connections 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56425c473170 0x56425c475560 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56425c3f5d30 0x56425c3f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56425c3e9da0 0x56425c3fdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 --2- 172.21.15.33:0/672459093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56425c3e9480 0x56425c3fda70 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 >> 172.21.15.33:0/672459093 conn(0x56425c3e72b0 msgr2=0x56425c3f4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:45.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fabdaffd640 0 client.19317 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:25:45.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 shutdown_connections 2024-09-10T21:25:45.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 1 -- 172.21.15.33:0/672459093 wait complete. 2024-09-10T21:25:45.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.888+0000 7fac03dc8740 20 client.19317 trim_cache size 0 max 16384 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: pgmap v999: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 819 op/s 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:25:45.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:45 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:45.896 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:25:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:25:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:25:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:25:45.896 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "59291033-0fe1-4840-9451-8996afd161e3", 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.897 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.898 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.899 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.900 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.901 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:25:45.902 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.903 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.904 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.905 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.906 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.907 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.908 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.909 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.910 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:25:45.911 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.912 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:25:45.913 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:25:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:25:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:25:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:25:45.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:25:45.914 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fabe40b52e0 msgr2=0x7fabe40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fabe40b52e0 0x7fabe40b77a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fabec0a8d70 tx=0x7fabec004a90 comp rx=0 tx=0).stop 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 msgr2=0x7fabfc075500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 0x7fabfc075500 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fabec00e9f0 tx=0x7fabec00edf0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 shutdown_connections 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fabe40b52e0 0x7fabe40b77a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabfc109170 0x7fabfc075f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabfc1087f0 0x7fabfc075a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 --2- 172.21.15.33:0/60730055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabfc102de0 0x7fabfc075500 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 >> 172.21.15.33:0/60730055 conn(0x7fabfc0fe870 msgr2=0x7fabfc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 shutdown_connections 2024-09-10T21:25:45.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:25:45.890+0000 7fac019f3640 1 -- 172.21.15.33:0/60730055 wait complete. 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: pgmap v999: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 819 op/s 2024-09-10T21:25:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:25:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:45 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:25:46.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 392750 38.56 MB/sec execute 406 sec latency 87.280 ms 2024-09-10T21:25:46.701 INFO:tasks.cephfs.filesystem:scrub status for tag:59291033-0fe1-4840-9451-8996afd161e3 - {'path': '/', 'tag': '59291033-0fe1-4840-9451-8996afd161e3', 'options': 'recursive,force'} 2024-09-10T21:25:47.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 393549 38.59 MB/sec execute 407 sec latency 85.348 ms 2024-09-10T21:25:47.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:47 smithi174 ceph-mon[35635]: pgmap v1000: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 711 op/s 2024-09-10T21:25:47.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:47 smithi033 ceph-mon[30860]: pgmap v1000: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 711 op/s 2024-09-10T21:25:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:47 smithi104 ceph-mon[35325]: pgmap v1000: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 711 op/s 2024-09-10T21:25:48.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 394169 38.58 MB/sec execute 408 sec latency 81.024 ms 2024-09-10T21:25:48.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:48 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:25:48.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:48 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:25:49.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:48 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:25:49.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 394656 38.54 MB/sec execute 409 sec latency 189.644 ms 2024-09-10T21:25:49.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: pgmap v1001: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:25:49.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:25:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:49 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: pgmap v1001: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:25:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:25:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:49.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:49 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:50.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: pgmap v1001: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 771 op/s 2024-09-10T21:25:50.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:25:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:50.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:49 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:50.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 395026 38.50 MB/sec execute 410 sec latency 420.663 ms 2024-09-10T21:25:51.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 395802 38.53 MB/sec execute 411 sec latency 109.041 ms 2024-09-10T21:25:51.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.562 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:51 smithi104 ceph-mon[35325]: pgmap v1002: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 862 op/s 2024-09-10T21:25:51.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:51 smithi174 ceph-mon[35635]: pgmap v1002: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 862 op/s 2024-09-10T21:25:51.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:25:51.676 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:51 smithi033 ceph-mon[30860]: pgmap v1002: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 862 op/s 2024-09-10T21:25:52.075 INFO:tasks.workunit.client.0.smithi033.stdout: 10 396529 38.55 MB/sec execute 412 sec latency 91.777 ms 2024-09-10T21:25:52.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:52 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:25:52.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:52 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:25:52.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:52 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:25:53.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 397337 38.55 MB/sec execute 413 sec latency 68.014 ms 2024-09-10T21:25:53.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:53 smithi033 ceph-mon[30860]: pgmap v1003: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.5 MiB/s wr, 665 op/s 2024-09-10T21:25:53.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:53 smithi104 ceph-mon[35325]: pgmap v1003: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.5 MiB/s wr, 665 op/s 2024-09-10T21:25:53.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:53 smithi174 ceph-mon[35635]: pgmap v1003: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.5 MiB/s wr, 665 op/s 2024-09-10T21:25:54.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 398069 38.56 MB/sec execute 414 sec latency 182.089 ms 2024-09-10T21:25:55.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 398818 38.56 MB/sec execute 415 sec latency 94.828 ms 2024-09-10T21:25:55.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:55 smithi104 ceph-mon[35325]: pgmap v1004: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 879 op/s 2024-09-10T21:25:55.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:55 smithi174 ceph-mon[35635]: pgmap v1004: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 879 op/s 2024-09-10T21:25:55.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:55 smithi033 ceph-mon[30860]: pgmap v1004: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 879 op/s 2024-09-10T21:25:56.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 399527 38.56 MB/sec execute 416 sec latency 59.255 ms 2024-09-10T21:25:56.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:56 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:56.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:56 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:56.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:56 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:57.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 399926 38.53 MB/sec execute 417 sec latency 538.940 ms 2024-09-10T21:25:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:57 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:57 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:57 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:57 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:25:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:57 smithi104 ceph-mon[35325]: pgmap v1005: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 689 op/s 2024-09-10T21:25:57.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:57 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:57.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:57 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:57.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:57 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:57 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:57 smithi174 ceph-mon[35635]: pgmap v1005: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 689 op/s 2024-09-10T21:25:57.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:57 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:57.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:57 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:57.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:57 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:57.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:57 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:25:57.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:57 smithi033 ceph-mon[30860]: pgmap v1005: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.2 MiB/s wr, 689 op/s 2024-09-10T21:25:58.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 400513 38.51 MB/sec execute 418 sec latency 129.340 ms 2024-09-10T21:25:59.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 401222 38.51 MB/sec execute 419 sec latency 79.185 ms 2024-09-10T21:25:59.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:25:59 smithi104 ceph-mon[35325]: pgmap v1006: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2024-09-10T21:25:59.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:25:59 smithi174 ceph-mon[35635]: pgmap v1006: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2024-09-10T21:25:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:25:59 smithi033 ceph-mon[30860]: pgmap v1006: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.2 MiB/s wr, 753 op/s 2024-09-10T21:26:00.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 401808 38.49 MB/sec execute 420 sec latency 109.120 ms 2024-09-10T21:26:01.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 402390 38.45 MB/sec execute 421 sec latency 239.662 ms 2024-09-10T21:26:01.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:01 smithi104 ceph-mon[35325]: pgmap v1007: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 806 op/s 2024-09-10T21:26:01.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:01 smithi174 ceph-mon[35635]: pgmap v1007: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 806 op/s 2024-09-10T21:26:01.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:01 smithi033 ceph-mon[30860]: pgmap v1007: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 806 op/s 2024-09-10T21:26:02.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 403368 38.49 MB/sec execute 422 sec latency 278.781 ms 2024-09-10T21:26:03.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 404241 38.54 MB/sec execute 423 sec latency 80.064 ms 2024-09-10T21:26:03.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:03 smithi104 ceph-mon[35325]: pgmap v1008: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-10T21:26:03.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:03 smithi174 ceph-mon[35635]: pgmap v1008: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-10T21:26:03.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:03 smithi033 ceph-mon[30860]: pgmap v1008: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 10 MiB/s wr, 670 op/s 2024-09-10T21:26:04.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 404796 38.52 MB/sec execute 424 sec latency 87.979 ms 2024-09-10T21:26:05.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 405410 38.49 MB/sec execute 425 sec latency 97.279 ms 2024-09-10T21:26:05.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:05 smithi104 ceph-mon[35325]: pgmap v1009: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 891 op/s 2024-09-10T21:26:05.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:05 smithi174 ceph-mon[35635]: pgmap v1009: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 891 op/s 2024-09-10T21:26:05.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:05 smithi033 ceph-mon[30860]: pgmap v1009: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 12 MiB/s wr, 891 op/s 2024-09-10T21:26:06.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 406147 38.49 MB/sec execute 426 sec latency 82.599 ms 2024-09-10T21:26:07.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 407038 38.51 MB/sec execute 427 sec latency 94.646 ms 2024-09-10T21:26:07.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:07 smithi104 ceph-mon[35325]: pgmap v1010: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-09-10T21:26:07.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:07 smithi174 ceph-mon[35635]: pgmap v1010: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-09-10T21:26:07.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:07 smithi033 ceph-mon[30860]: pgmap v1010: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 12 MiB/s wr, 727 op/s 2024-09-10T21:26:08.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 407421 38.46 MB/sec execute 428 sec latency 502.792 ms 2024-09-10T21:26:09.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 407986 38.46 MB/sec execute 429 sec latency 510.869 ms 2024-09-10T21:26:09.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:09 smithi104 ceph-mon[35325]: pgmap v1011: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 775 op/s 2024-09-10T21:26:09.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:09 smithi174 ceph-mon[35635]: pgmap v1011: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 775 op/s 2024-09-10T21:26:09.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:09 smithi033 ceph-mon[30860]: pgmap v1011: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 13 MiB/s wr, 775 op/s 2024-09-10T21:26:10.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 408873 38.48 MB/sec execute 430 sec latency 105.272 ms 2024-09-10T21:26:11.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 409527 38.49 MB/sec execute 431 sec latency 118.952 ms 2024-09-10T21:26:11.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:11 smithi104 ceph-mon[35325]: pgmap v1012: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 884 op/s 2024-09-10T21:26:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:11 smithi174 ceph-mon[35635]: pgmap v1012: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 884 op/s 2024-09-10T21:26:11.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:11 smithi033 ceph-mon[30860]: pgmap v1012: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 14 MiB/s wr, 884 op/s 2024-09-10T21:26:12.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 410067 38.45 MB/sec execute 432 sec latency 89.262 ms 2024-09-10T21:26:13.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 410856 38.46 MB/sec execute 433 sec latency 68.404 ms 2024-09-10T21:26:13.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:13 smithi104 ceph-mon[35325]: pgmap v1013: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 726 op/s 2024-09-10T21:26:13.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:13 smithi174 ceph-mon[35635]: pgmap v1013: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 726 op/s 2024-09-10T21:26:13.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:13 smithi033 ceph-mon[30860]: pgmap v1013: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 726 op/s 2024-09-10T21:26:14.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 411759 38.47 MB/sec execute 434 sec latency 134.547 ms 2024-09-10T21:26:15.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 412443 38.49 MB/sec execute 435 sec latency 66.434 ms 2024-09-10T21:26:15.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:15 smithi104 ceph-mon[35325]: pgmap v1014: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 820 op/s 2024-09-10T21:26:15.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:15 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:26:15.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:15 smithi174 ceph-mon[35635]: pgmap v1014: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 820 op/s 2024-09-10T21:26:15.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:15 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:26:15.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:15 smithi033 ceph-mon[30860]: pgmap v1014: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 820 op/s 2024-09-10T21:26:15.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:15 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:26:16.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 413108 38.47 MB/sec execute 436 sec latency 66.627 ms 2024-09-10T21:26:16.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:26:17.021 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:17.076 INFO:tasks.workunit.client.0.smithi033.stdout: 10 413851 38.49 MB/sec execute 437 sec latency 113.487 ms 2024-09-10T21:26:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 -- 172.21.15.33:0/391996521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844102dc0 msgr2=0x7f08441031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 --2- 172.21.15.33:0/391996521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844102dc0 0x7f08441031a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f0834009900 tx=0x7f083402a410 comp rx=0 tx=0).stop 2024-09-10T21:26:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 -- 172.21.15.33:0/391996521 shutdown_connections 2024-09-10T21:26:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 --2- 172.21.15.33:0/391996521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0844109150 0x7f084410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 --2- 172.21.15.33:0/391996521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08441087d0 0x7f0844108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 --2- 172.21.15.33:0/391996521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844102dc0 0x7f08441031a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 -- 172.21.15.33:0/391996521 >> 172.21.15.33:0/391996521 conn(0x7f08440fe870 msgr2=0x7f0844100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 -- 172.21.15.33:0/391996521 shutdown_connections 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.553+0000 7f084c40f640 1 -- 172.21.15.33:0/391996521 wait complete. 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 Processor -- start 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 -- start start 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0844102dc0 0x7f0844199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08441087d0 0x7f08441997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844109150 0x7f084419eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0844112e60 con 0x7f0844109150 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0844112ce0 con 0x7f0844102dc0 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084c40f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0844112fe0 con 0x7f08441087d0 2024-09-10T21:26:17.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.554+0000 7f084a9ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844109150 0x7f084419eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844109150 0x7f084419eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43156/0 (socket says 172.21.15.33:43156) 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 -- 172.21.15.33:0/82305464 learned_addr learned my addr 172.21.15.33:0/82305464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a1ab640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0844102dc0 0x7f0844199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f08499aa640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08441087d0 0x7f08441997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 -- 172.21.15.33:0/82305464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08441087d0 msgr2=0x7f08441997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08441087d0 0x7f08441997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 -- 172.21.15.33:0/82305464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0844102dc0 msgr2=0x7f0844199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0844102dc0 0x7f0844199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 -- 172.21.15.33:0/82305464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084419f340 con 0x7f0844109150 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084a9ac640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844109150 0x7f084419eb70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f083c00eb10 tx=0x7f083c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f08437fe640 1 -- 172.21.15.33:0/82305464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c004b20 con 0x7f0844109150 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.555+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084419f630 con 0x7f0844109150 2024-09-10T21:26:17.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.556+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08441a5cf0 con 0x7f0844109150 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.558+0000 7f084e7e4740 1 Processor -- start 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.558+0000 7f084e7e4740 1 -- start start 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701480 0x56217d701860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701da0 0x56217d711c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d712140 0x56217d714530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f084e7e4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56217d677e50 con 0x56217d712140 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f084e7e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56217d667280 con 0x56217d701da0 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f084e7e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56217d66b090 con 0x56217d701480 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d712140 0x56217d714530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f084a1ab640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701da0 0x56217d711c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d712140 0x56217d714530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43168/0 (socket says 172.21.15.33:43168) 2024-09-10T21:26:17.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 -- 172.21.15.33:0/224721523 learned_addr learned my addr 172.21.15.33:0/224721523 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701480 0x56217d701860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 -- 172.21.15.33:0/224721523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701480 msgr2=0x56217d701860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701480 0x56217d701860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 -- 172.21.15.33:0/224721523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701da0 msgr2=0x56217d711c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701da0 0x56217d711c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.559+0000 7f08499aa640 1 -- 172.21.15.33:0/224721523 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217d3c5650 con 0x56217d712140 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.560+0000 7f08499aa640 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d712140 0x56217d714530 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0838011840 tx=0x7f08380097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bf64883bc7ff90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.560+0000 7f08437fe640 1 -- 172.21.15.33:0/82305464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c004cc0 con 0x7f0844109150 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.560+0000 7f08437fe640 1 -- 172.21.15.33:0/82305464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0216a0 con 0x7f0844109150 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.561+0000 7f0840de6640 1 -- 172.21.15.33:0/224721523 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838004030 con 0x56217d712140 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.561+0000 7f0840de6640 1 -- 172.21.15.33:0/224721523 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800f7c0 con 0x56217d712140 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.561+0000 7f0840de6640 1 -- 172.21.15.33:0/224721523 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0838003900 con 0x56217d712140 2024-09-10T21:26:17.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.562+0000 7f084e7e4740 1 -- 172.21.15.33:0/224721523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d712140 msgr2=0x56217d714530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.562+0000 7f084e7e4740 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d712140 0x56217d714530 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f0838011840 tx=0x7f08380097d0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f08437fe640 1 -- 172.21.15.33:0/82305464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f083c021840 con 0x7f0844109150 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f084e7e4740 1 -- 172.21.15.33:0/224721523 shutdown_connections 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f084e7e4740 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d712140 0x56217d714530 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f084e7e4740 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701da0 0x56217d711c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f084e7e4740 1 --2- 172.21.15.33:0/224721523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701480 0x56217d701860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f084e7e4740 1 -- 172.21.15.33:0/224721523 >> 172.21.15.33:0/224721523 conn(0x56217d6ff2b0 msgr2=0x56217d700450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f084e7e4740 1 -- 172.21.15.33:0/224721523 shutdown_connections 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.563+0000 7f084e7e4740 1 -- 172.21.15.33:0/224721523 wait complete. 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.564+0000 7f084e7e4740 1 -- start start 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.564+0000 7f084e7e4740 1 Processor -- start 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.564+0000 7f08437fe640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f082c0b51f0 0x7f082c0b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.564+0000 7f08437fe640 1 -- 172.21.15.33:0/82305464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f083c0e6280 con 0x7f0844109150 2024-09-10T21:26:17.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d711720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 0x56217d70ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d70f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084e7e4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56217d4c8700 con 0x56217d701480 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f08499aa640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f082c0b51f0 0x7f082c0b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084e7e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56217d602110 con 0x56217d712140 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084e7e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56217d714df0 con 0x56217d701da0 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084e7e4740 10 client.0 _ll_register_callbacks cb 0x56217d61b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084a1ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d711720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084a9ac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d70f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084a1ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d711720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43174/0 (socket says 172.21.15.33:43174) 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084a1ab640 1 -- 172.21.15.33:0/551777965 learned_addr learned my addr 172.21.15.33:0/551777965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:17.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084a9ac640 1 -- 172.21.15.33:0/551777965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 msgr2=0x56217d70ef20 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084a9ac640 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 0x56217d70ef20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.565+0000 7f084a9ac640 1 -- 172.21.15.33:0/551777965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 msgr2=0x56217d711720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f08499aa640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f082c0b51f0 0x7f082c0b76b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0838011840 tx=0x7f0838005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f084a9ac640 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d711720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f084a9ac640 1 -- 172.21.15.33:0/551777965 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56217d62fc40 con 0x56217d712140 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f084a1ab640 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d711720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f084a9ac640 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d70f460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f083c0a9cb0 tx=0x7f083c0ac1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82f6d314299530d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f0829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f0829ffb640 1 -- 172.21.15.33:0/551777965 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083c0d3020 con 0x56217d712140 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f0829ffb640 1 -- 172.21.15.33:0/551777965 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f083c0b5d50 con 0x56217d712140 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.566+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083800f7c0 con 0x56217d712140 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f0829ffb640 1 -- 172.21.15.33:0/551777965 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f083c0c7050 con 0x56217d712140 2024-09-10T21:26:17.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 10 client.28664 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x56217d724c00 con 0x56217d712140 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f0829ffb640 1 -- 172.21.15.33:0/551777965 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083c0ad140 con 0x56217d712140 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f0829ffb640 1 -- 172.21.15.33:0/551777965 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f083c0a8390 con 0x56217d712140 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 10 client.28664 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 10 client.28664 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 20 client.28664 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d726160 0x56217d728550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 4 client.28664 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f083800f7c0 con 0x56217d726160 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.567+0000 7f084a1ab640 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d726160 0x56217d728550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.568+0000 7f0829ffb640 10 client.28664 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:17.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.568+0000 7f084a1ab640 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d726160 0x56217d728550 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a17b4ca2c5f450c server_cookie=a08a1d3b14e455eb in_seq=0 out_seq=0 2024-09-10T21:26:17.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f0829ffb640 1 -- 172.21.15.33:0/551777965 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f083800f7c0 con 0x56217d726160 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f0829ffb640 10 client.28664 handle_command_reply: tid=0 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 client.28664 shutdown 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 msgr2=0x56217d70f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d70f460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f083c0a9cb0 tx=0x7f083c0ac1b0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 shutdown_connections 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d726160 0x56217d728550 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d70f460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 0x56217d70ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 --2- 172.21.15.33:0/551777965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d711720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 >> 172.21.15.33:0/551777965 conn(0x56217d6ff2b0 msgr2=0x56217d704e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f0829ffb640 0 client.28664 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 shutdown_connections 2024-09-10T21:26:17.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 1 -- 172.21.15.33:0/551777965 wait complete. 2024-09-10T21:26:17.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.573+0000 7f084e7e4740 20 client.28664 trim_cache size 0 max 16384 2024-09-10T21:26:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.596+0000 7f084e7e4740 1 Processor -- start 2024-09-10T21:26:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.596+0000 7f084e7e4740 1 -- start start 2024-09-10T21:26:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701480 0x56217d6febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701da0 0x56217d6fa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d712140 0x56217d713020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084e7e4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ac1b0 con 0x56217d701da0 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084e7e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08441add80 con 0x56217d701480 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084e7e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ac8e0 con 0x56217d712140 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701da0 0x56217d6fa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701da0 0x56217d6fa2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43186/0 (socket says 172.21.15.33:43186) 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 -- 172.21.15.33:0/1101088150 learned_addr learned my addr 172.21.15.33:0/1101088150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084a1ab640 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701480 0x56217d6febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084a9ac640 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d712140 0x56217d713020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 -- 172.21.15.33:0/1101088150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d712140 msgr2=0x56217d713020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d712140 0x56217d713020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 -- 172.21.15.33:0/1101088150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701480 msgr2=0x56217d6febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701480 0x56217d6febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 -- 172.21.15.33:0/1101088150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56217d701bd0 con 0x56217d701da0 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f084a1ab640 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701480 0x56217d6febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.597+0000 7f08499aa640 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701da0 0x56217d6fa2d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f08380057c0 tx=0x7f0838022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5318ebfb45f74509 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.598+0000 7f082bfff640 1 -- 172.21.15.33:0/1101088150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083800fab0 con 0x56217d701da0 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.598+0000 7f082bfff640 1 -- 172.21.15.33:0/1101088150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800fc50 con 0x56217d701da0 2024-09-10T21:26:17.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.598+0000 7f082bfff640 1 -- 172.21.15.33:0/1101088150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083802b670 con 0x56217d701da0 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.598+0000 7f084e7e4740 1 -- 172.21.15.33:0/1101088150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701da0 msgr2=0x56217d6fa2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.598+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701da0 0x56217d6fa2d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f08380057c0 tx=0x7f0838022000 comp rx=0 tx=0).stop 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- 172.21.15.33:0/1101088150 shutdown_connections 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d712140 0x56217d713020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701da0 0x56217d6fa2d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1101088150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d701480 0x56217d6febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- 172.21.15.33:0/1101088150 >> 172.21.15.33:0/1101088150 conn(0x56217d6ff2b0 msgr2=0x56217d70ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- 172.21.15.33:0/1101088150 shutdown_connections 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- 172.21.15.33:0/1101088150 wait complete. 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- start start 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 Processor -- start 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d723330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 0x56217d720820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d720f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f08441ad680 con 0x56217d701480 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f08441acfb0 con 0x56217d712140 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08441aa170 con 0x56217d701da0 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.599+0000 7f084e7e4740 10 client.0 _ll_register_callbacks cb 0x7f083c004e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d723330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d723330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43196/0 (socket says 172.21.15.33:43196) 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 -- 172.21.15.33:0/1744296059 learned_addr learned my addr 172.21.15.33:0/1744296059 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f08499aa640 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 0x56217d720820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a9ac640 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d720f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 -- 172.21.15.33:0/1744296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 msgr2=0x56217d720820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 0x56217d720820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 -- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 msgr2=0x56217d720f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d720f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 -- 172.21.15.33:0/1744296059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56217d792030 con 0x56217d701480 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084a1ab640 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d723330 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f083402a930 tx=0x7f08340045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb674beea7e4af9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f0829ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f0829ffb640 1 -- 172.21.15.33:0/1744296059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834035040 con 0x56217d701480 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f0829ffb640 1 -- 172.21.15.33:0/1744296059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0834005510 con 0x56217d701480 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.600+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083800fc50 con 0x56217d701480 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f0829ffb640 1 -- 172.21.15.33:0/1744296059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0834004770 con 0x56217d701480 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 10 client.19335 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:17.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x56217d793300 con 0x56217d701480 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f0829ffb640 1 -- 172.21.15.33:0/1744296059 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083800fc50 con 0x56217d701480 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f0829ffb640 1 -- 172.21.15.33:0/1744296059 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f0834003b50 con 0x56217d701480 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 10 client.19335 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 10 client.19335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 20 client.19335 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d793a60 0x56217d795e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 4 client.19335 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:26:17.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.601+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56217d679da0 con 0x56217d793a60 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.602+0000 7f08499aa640 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d793a60 0x56217d795e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.603+0000 7f08499aa640 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d793a60 0x56217d795e80 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89123d71f4f73e6 server_cookie=a2101291922885fc in_seq=0 out_seq=0 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.603+0000 7f0829ffb640 10 client.19335 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f0829ffb640 1 -- 172.21.15.33:0/1744296059 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8042 (crc 0 0 0) 0x56217d679da0 con 0x56217d793a60 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f0829ffb640 10 client.19335 handle_command_reply: tid=0 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 client.19335 shutdown 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 msgr2=0x56217d723330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d723330 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f083402a930 tx=0x7f08340045c0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 shutdown_connections 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56217d793a60 0x56217d795e80 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f0829ffb640 0 client.19335 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:17.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56217d712140 0x56217d720f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56217d701da0 0x56217d720820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 --2- 172.21.15.33:0/1744296059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56217d701480 0x56217d723330 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 >> 172.21.15.33:0/1744296059 conn(0x56217d6ff2b0 msgr2=0x56217d713020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 shutdown_connections 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 1 -- 172.21.15.33:0/1744296059 wait complete. 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.604+0000 7f084e7e4740 20 client.19335 trim_cache size 0 max 16384 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:26:17.611 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.612 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.613 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.614 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:26:17.615 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.617 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.618 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.619 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.620 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.622 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:26:17.623 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.624 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.625 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:26:17.626 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:17.628 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f082c0b51f0 msgr2=0x7f082c0b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f082c0b51f0 0x7f082c0b76b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0838011840 tx=0x7f0838005f90 comp rx=0 tx=0).stop 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844109150 msgr2=0x7f084419eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844109150 0x7f084419eb70 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f083c00eb10 tx=0x7f083c00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 shutdown_connections 2024-09-10T21:26:17.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f082c0b51f0 0x7f082c0b76b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0844109150 0x7f084419eb70 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08441087d0 0x7f08441997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 --2- 172.21.15.33:0/82305464 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0844102dc0 0x7f0844199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 >> 172.21.15.33:0/82305464 conn(0x7f08440fe870 msgr2=0x7f0844100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 shutdown_connections 2024-09-10T21:26:17.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:17.605+0000 7f084c40f640 1 -- 172.21.15.33:0/82305464 wait complete. 2024-09-10T21:26:17.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:17 smithi104 ceph-mon[35325]: pgmap v1015: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.6 MiB/s wr, 673 op/s 2024-09-10T21:26:17.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:17 smithi174 ceph-mon[35635]: pgmap v1015: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.6 MiB/s wr, 673 op/s 2024-09-10T21:26:17.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:17 smithi033 ceph-mon[30860]: pgmap v1015: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.6 MiB/s wr, 673 op/s 2024-09-10T21:26:18.077 INFO:tasks.workunit.client.0.smithi033.stdout: 10 414536 38.48 MB/sec execute 438 sec latency 73.479 ms 2024-09-10T21:26:18.530 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:26:18.530 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:26:18.842 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:19.077 INFO:tasks.workunit.client.0.smithi033.stdout: 10 415171 38.47 MB/sec execute 439 sec latency 79.815 ms 2024-09-10T21:26:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 -- 172.21.15.33:0/1426030575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d00101cb0 msgr2=0x7f6d00102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 --2- 172.21.15.33:0/1426030575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d00101cb0 0x7f6d00102090 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0009900 tx=0x7f6cf002a410 comp rx=0 tx=0).stop 2024-09-10T21:26:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 -- 172.21.15.33:0/1426030575 shutdown_connections 2024-09-10T21:26:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 --2- 172.21.15.33:0/1426030575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d00108040 0x7f6d0010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 --2- 172.21.15.33:0/1426030575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d001076c0 0x7f6d00107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 --2- 172.21.15.33:0/1426030575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d00101cb0 0x7f6d00102090 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 -- 172.21.15.33:0/1426030575 >> 172.21.15.33:0/1426030575 conn(0x7f6d000fd700 msgr2=0x7f6d000ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 -- 172.21.15.33:0/1426030575 shutdown_connections 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.400+0000 7f6d06409640 1 -- 172.21.15.33:0/1426030575 wait complete. 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 Processor -- start 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 -- start start 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d00101cb0 0x7f6d001981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d001076c0 0x7f6d001986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d00108040 0x7f6d0019da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00111d00 con 0x7f6d001076c0 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00111b80 con 0x7f6d00101cb0 2024-09-10T21:26:19.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.401+0000 7f6d06409640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00111e80 con 0x7f6d00108040 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d001076c0 0x7f6d001986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d001076c0 0x7f6d001986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43208/0 (socket says 172.21.15.33:43208) 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 -- 172.21.15.33:0/1179422861 learned_addr learned my addr 172.21.15.33:0/1179422861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cfffff640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d00101cb0 0x7f6d001981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6d049a6640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d00108040 0x7f6d0019da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 -- 172.21.15.33:0/1179422861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d00108040 msgr2=0x7f6d0019da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d00108040 0x7f6d0019da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 -- 172.21.15.33:0/1179422861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d00101cb0 msgr2=0x7f6d001981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d00101cb0 0x7f6d001981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 -- 172.21.15.33:0/1179422861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0019e260 con 0x7f6d001076c0 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cff7fe640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d001076c0 0x7f6d001986f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e9f0 tx=0x7f6cf400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.402+0000 7f6cfd7fa640 1 -- 172.21.15.33:0/1179422861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf400cd80 con 0x7f6d001076c0 2024-09-10T21:26:19.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.403+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0019e550 con 0x7f6d001076c0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.403+0000 7f6cfd7fa640 1 -- 172.21.15.33:0/1179422861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf40054d0 con 0x7f6d001076c0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.403+0000 7f6cfd7fa640 1 -- 172.21.15.33:0/1179422861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4010690 con 0x7f6d001076c0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.403+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0006b490 con 0x7f6d001076c0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.404+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cec0052c0 con 0x7f6d001076c0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.406+0000 7f6cfd7fa640 1 -- 172.21.15.33:0/1179422861 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f6cf4004e90 con 0x7f6d001076c0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.406+0000 7f6cfd7fa640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ce80b52e0 0x7f6ce80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.407+0000 7f6cfffff640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ce80b52e0 0x7f6ce80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.407+0000 7f6cfd7fa640 1 -- 172.21.15.33:0/1179422861 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f6cf40e5d80 con 0x7f6d001076c0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.407+0000 7f6cfffff640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ce80b52e0 0x7f6ce80b77a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0031440 tx=0x7f6cf0005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:19.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.412+0000 7f6cfd7fa640 1 -- 172.21.15.33:0/1179422861 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f6cf40acc40 con 0x7f6d001076c0 2024-09-10T21:26:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.681+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cec0055b0 con 0x7f6d001076c0 2024-09-10T21:26:19.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.682+0000 7f6cfd7fa640 1 -- 172.21.15.33:0/1179422861 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) ==== 94+0+11978 (secure 0 0 0) 0x7f6cf40ac540 con 0x7f6d001076c0 2024-09-10T21:26:19.686 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:26:19.686 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":87,"btime":"2024-09-10T21:26:14:541911+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":87,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:26:14.189138+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0,2],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:26:19.686 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 87 2024-09-10T21:26:19.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ce80b52e0 msgr2=0x7f6ce80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ce80b52e0 0x7f6ce80b77a0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0031440 tx=0x7f6cf0005b50 comp rx=0 tx=0).stop 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d001076c0 msgr2=0x7f6d001986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d001076c0 0x7f6d001986f0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6cf400e9f0 tx=0x7f6cf400edf0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 shutdown_connections 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f6ce80b52e0 0x7f6ce80b77a0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d00108040 0x7f6d0019da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d001076c0 0x7f6d001986f0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 --2- 172.21.15.33:0/1179422861 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d00101cb0 0x7f6d001981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 >> 172.21.15.33:0/1179422861 conn(0x7f6d000fd700 msgr2=0x7f6d000ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 shutdown_connections 2024-09-10T21:26:19.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:19.684+0000 7f6d06409640 1 -- 172.21.15.33:0/1179422861 wait complete. 2024-09-10T21:26:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:19 smithi174 ceph-mon[35635]: pgmap v1016: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.3 MiB/s wr, 728 op/s 2024-09-10T21:26:19.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:19 smithi033 ceph-mon[30860]: pgmap v1016: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.3 MiB/s wr, 728 op/s 2024-09-10T21:26:20.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:19 smithi104 ceph-mon[35325]: pgmap v1016: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 8.3 MiB/s wr, 728 op/s 2024-09-10T21:26:20.077 INFO:tasks.workunit.client.0.smithi033.stdout: 10 415721 38.44 MB/sec execute 440 sec latency 90.604 ms 2024-09-10T21:26:20.478 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:26:20.781 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:20.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:20 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1179422861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:26:20.903 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:20 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1179422861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:26:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:20 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1179422861' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:26:21.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 416496 38.44 MB/sec execute 441 sec latency 84.935 ms 2024-09-10T21:26:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/427506885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb541087f0 msgr2=0x7fdb54108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/427506885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb541087f0 0x7fdb54108c30 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009900 tx=0x7fdb4002a410 comp rx=0 tx=0).stop 2024-09-10T21:26:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/427506885 shutdown_connections 2024-09-10T21:26:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/427506885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb54109170 0x7fdb5410f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/427506885 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb541087f0 0x7fdb54108c30 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/427506885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb54102de0 0x7fdb541031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/427506885 >> 172.21.15.33:0/427506885 conn(0x7fdb540fe830 msgr2=0x7fdb54100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:21.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/427506885 shutdown_connections 2024-09-10T21:26:21.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.322+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/427506885 wait complete. 2024-09-10T21:26:21.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 Processor -- start 2024-09-10T21:26:21.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 -- start start 2024-09-10T21:26:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb54102de0 0x7fdb541992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb541087f0 0x7fdb54199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb54109170 0x7fdb5419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54112e30 con 0x7fdb54102de0 2024-09-10T21:26:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54112cb0 con 0x7fdb54109170 2024-09-10T21:26:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb5ac5b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54112fb0 con 0x7fdb541087f0 2024-09-10T21:26:21.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb589f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb54102de0 0x7fdb541992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb589f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb54102de0 0x7fdb541992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39852/0 (socket says 172.21.15.33:39852) 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb589f7640 1 -- 172.21.15.33:0/3357300107 learned_addr learned my addr 172.21.15.33:0/3357300107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb53fff640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb541087f0 0x7fdb54199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.323+0000 7fdb591f8640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb54109170 0x7fdb5419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb589f7640 1 -- 172.21.15.33:0/3357300107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb541087f0 msgr2=0x7fdb54199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb589f7640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb541087f0 0x7fdb54199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb589f7640 1 -- 172.21.15.33:0/3357300107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb54109170 msgr2=0x7fdb5419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb589f7640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb54109170 0x7fdb5419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb589f7640 1 -- 172.21.15.33:0/3357300107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5419f3a0 con 0x7fdb54102de0 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb591f8640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb54109170 0x7fdb5419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:21.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb589f7640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb54102de0 0x7fdb541992f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800d950 tx=0x7fdb4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb51ffb640 1 -- 172.21.15.33:0/3357300107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb48017030 con 0x7fdb54102de0 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5419f690 con 0x7fdb54102de0 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb51ffb640 1 -- 172.21.15.33:0/3357300107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb48004020 con 0x7fdb54102de0 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb51ffb640 1 -- 172.21.15.33:0/3357300107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb480047a0 con 0x7fdb54102de0 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.324+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb54075550 con 0x7fdb54102de0 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.327+0000 7fdb51ffb640 1 -- 172.21.15.33:0/3357300107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fdb48009a30 con 0x7fdb54102de0 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.327+0000 7fdb5d030740 1 Processor -- start 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.327+0000 7fdb5d030740 1 -- start start 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed84d0 0x56411fed88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed8df0 0x56411fee4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fee4d80 0x56411fee7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb5d030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56411fe4eea0 con 0x56411fed8df0 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb5d030740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56411fe3e2d0 con 0x56411fee4d80 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb5d030740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56411fe420e0 con 0x56411fed84d0 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb591f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed84d0 0x56411fed88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fee4d80 0x56411fee7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb591f8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed84d0 0x56411fed88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37576/0 (socket says 172.21.15.33:37576) 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb591f8640 1 -- 172.21.15.33:0/667070626 learned_addr learned my addr 172.21.15.33:0/667070626 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb589f7640 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed8df0 0x56411fee4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb51ffb640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb440b52e0 0x7fdb440b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb53fff640 1 -- 172.21.15.33:0/667070626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed84d0 msgr2=0x56411fed88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb53fff640 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed84d0 0x56411fed88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb53fff640 1 -- 172.21.15.33:0/667070626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed8df0 msgr2=0x56411fee4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb53fff640 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed8df0 0x56411fee4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb53fff640 1 -- 172.21.15.33:0/667070626 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56411fb9c650 con 0x56411fee4d80 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.328+0000 7fdb589f7640 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed8df0 0x56411fee4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.329+0000 7fdb591f8640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb440b52e0 0x7fdb440b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.329+0000 7fdb53fff640 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fee4d80 0x56411fee7170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009230 tx=0x7fdb40032e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d3f81ffb3ec567 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.329+0000 7fdb51ffb640 1 -- 172.21.15.33:0/3357300107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdb480e56b0 con 0x7fdb54102de0 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.329+0000 7fdb3effd640 1 -- 172.21.15.33:0/667070626 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40040070 con 0x56411fee4d80 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.329+0000 7fdb3effd640 1 -- 172.21.15.33:0/667070626 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb400316a0 con 0x56411fee4d80 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.329+0000 7fdb3effd640 1 -- 172.21.15.33:0/667070626 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40035040 con 0x56411fee4d80 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb591f8640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb440b52e0 0x7fdb440b77a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00a660 tx=0x7fdb4c00e040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 -- 172.21.15.33:0/667070626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fee4d80 msgr2=0x56411fee7170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fee4d80 0x56411fee7170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdb40009230 tx=0x7fdb40032e60 comp rx=0 tx=0).stop 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 -- 172.21.15.33:0/667070626 shutdown_connections 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fee4d80 0x56411fee7170 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed8df0 0x56411fee4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 --2- 172.21.15.33:0/667070626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed84d0 0x56411fed88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 -- 172.21.15.33:0/667070626 >> 172.21.15.33:0/667070626 conn(0x56411fed6300 msgr2=0x56411fed74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 -- 172.21.15.33:0/667070626 shutdown_connections 2024-09-10T21:26:21.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 -- 172.21.15.33:0/667070626 wait complete. 2024-09-10T21:26:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 -- start start 2024-09-10T21:26:21.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.330+0000 7fdb5d030740 1 Processor -- start 2024-09-10T21:26:21.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fee46b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb5d030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x56411fc9f700 con 0x56411fee4d80 2024-09-10T21:26:21.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb5d030740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56411fdd9160 con 0x56411fed84d0 2024-09-10T21:26:21.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb5d030740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56411fee79a0 con 0x56411fed8df0 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb5d030740 10 client.0 _ll_register_callbacks cb 0x56411fdf29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb589f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fee46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb589f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fee46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43542/0 (socket says 172.21.15.33:43542) 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb589f7640 1 -- 172.21.15.33:0/3810413096 learned_addr learned my addr 172.21.15.33:0/3810413096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb591f8640 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb53fff640 1 -- 172.21.15.33:0/3810413096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 msgr2=0x56411fee46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb53fff640 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fee46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb53fff640 1 -- 172.21.15.33:0/3810413096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 msgr2=0x56411fee2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb53fff640 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb53fff640 1 -- 172.21.15.33:0/3810413096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411fe06c90 con 0x56411fed8df0 2024-09-10T21:26:21.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.331+0000 7fdb589f7640 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fee46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb53fff640 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee1b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005ec0 tx=0x7fdb4002a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5464740f4c6ed825 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb33fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb33fff640 1 -- 172.21.15.33:0/3810413096 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4004d060 con 0x56411fed8df0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb33fff640 1 -- 172.21.15.33:0/3810413096 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdb40031d10 con 0x56411fed8df0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb400316a0 con 0x56411fed8df0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb33fff640 1 -- 172.21.15.33:0/3810413096 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fdb40040070 con 0x56411fed8df0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb5d030740 10 client.28558 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.332+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x56411fef3310 con 0x56411fed8df0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb33fff640 1 -- 172.21.15.33:0/3810413096 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb400035f0 con 0x56411fed8df0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb33fff640 1 -- 172.21.15.33:0/3810413096 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7fdb40052070 con 0x56411fed8df0 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb5d030740 10 client.28558 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:21.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb5d030740 10 client.28558 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb5d030740 20 client.28558 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb5d030740 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411fef4660 0x56411fef6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb5d030740 4 client.28558 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdb400316a0 con 0x56411fef4660 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.333+0000 7fdb589f7640 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411fef4660 0x56411fef6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.334+0000 7fdb589f7640 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411fef4660 0x56411fef6a50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1abe462718e6db1 server_cookie=e6c8c176710b4e9e in_seq=0 out_seq=0 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.334+0000 7fdb33fff640 10 client.28558 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.335+0000 7fdb33fff640 1 -- 172.21.15.33:0/3810413096 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fdb400316a0 con 0x56411fef4660 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.335+0000 7fdb33fff640 10 client.28558 handle_command_reply: tid=0 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.335+0000 7fdb5d030740 1 client.28558 shutdown 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 msgr2=0x56411fee1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee1b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdb40005ec0 tx=0x7fdb4002a410 comp rx=0 tx=0).stop 2024-09-10T21:26:21.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 shutdown_connections 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411fef4660 0x56411fef6a50 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee1b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 --2- 172.21.15.33:0/3810413096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fee46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 >> 172.21.15.33:0/3810413096 conn(0x56411fed6300 msgr2=0x56411fedbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb33fff640 0 client.28558 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 shutdown_connections 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 1 -- 172.21.15.33:0/3810413096 wait complete. 2024-09-10T21:26:21.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.336+0000 7fdb5d030740 20 client.28558 trim_cache size 0 max 16384 2024-09-10T21:26:21.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 Processor -- start 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 -- start start 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fedc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b05e0 con 0x56411fee4d80 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b21b0 con 0x56411fed84d0 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb5d030740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b0d10 con 0x56411fed8df0 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb591f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb589f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fedc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb589f7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fedc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43558/0 (socket says 172.21.15.33:43558) 2024-09-10T21:26:21.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb589f7640 1 -- 172.21.15.33:0/2264331451 learned_addr learned my addr 172.21.15.33:0/2264331451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37596/0 (socket says 172.21.15.33:37596) 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.351+0000 7fdb591f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39876/0 (socket says 172.21.15.33:39876) 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.352+0000 7fdb591f8640 1 -- 172.21.15.33:0/2264331451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 msgr2=0x56411fee0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.352+0000 7fdb591f8640 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.352+0000 7fdb591f8640 1 -- 172.21.15.33:0/2264331451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 msgr2=0x56411fedc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.352+0000 7fdb591f8640 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fedc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.352+0000 7fdb591f8640 1 -- 172.21.15.33:0/2264331451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56411fed8be0 con 0x56411fee4d80 2024-09-10T21:26:21.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.352+0000 7fdb53fff640 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.352+0000 7fdb591f8640 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee08c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009dc0 tx=0x7fdb4c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9e7d1469d36e86 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.353+0000 7fdb3e7fc640 1 -- 172.21.15.33:0/2264331451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c018840 con 0x56411fee4d80 2024-09-10T21:26:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.353+0000 7fdb3e7fc640 1 -- 172.21.15.33:0/2264331451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c029070 con 0x56411fee4d80 2024-09-10T21:26:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.353+0000 7fdb3e7fc640 1 -- 172.21.15.33:0/2264331451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c024d80 con 0x56411fee4d80 2024-09-10T21:26:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 -- 172.21.15.33:0/2264331451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 msgr2=0x56411fee08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee08c0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c009dc0 tx=0x7fdb4c014000 comp rx=0 tx=0).stop 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 -- 172.21.15.33:0/2264331451 shutdown_connections 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fee4d80 0x56411fee08c0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fed8df0 0x56411fee0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 --2- 172.21.15.33:0/2264331451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed84d0 0x56411fedc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 -- 172.21.15.33:0/2264331451 >> 172.21.15.33:0/2264331451 conn(0x56411fed6300 msgr2=0x56411fed7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 -- 172.21.15.33:0/2264331451 shutdown_connections 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 -- 172.21.15.33:0/2264331451 wait complete. 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.354+0000 7fdb5d030740 1 -- start start 2024-09-10T21:26:21.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.355+0000 7fdb5d030740 1 Processor -- start 2024-09-10T21:26:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.355+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed84d0 0x56411fef5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.355+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed8df0 0x56411fef5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.355+0000 7fdb5d030740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fee4d80 0x56411fef60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.355+0000 7fdb589f7640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed84d0 0x56411fef5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.355+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed8df0 0x56411fef5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb53fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed8df0 0x56411fef5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43570/0 (socket says 172.21.15.33:43570) 2024-09-10T21:26:21.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb53fff640 1 -- 172.21.15.33:0/692318942 learned_addr learned my addr 172.21.15.33:0/692318942 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b1ab0 con 0x56411fed84d0 2024-09-10T21:26:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b13e0 con 0x56411fed8df0 2024-09-10T21:26:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541ae5a0 con 0x56411fee4d80 2024-09-10T21:26:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb5d030740 10 client.0 _ll_register_callbacks cb 0x7fdb4c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:21.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb591f8640 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fee4d80 0x56411fef60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb53fff640 1 -- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fee4d80 msgr2=0x56411fef60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb53fff640 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fee4d80 0x56411fef60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb53fff640 1 -- 172.21.15.33:0/692318942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed84d0 msgr2=0x56411fef5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb53fff640 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed84d0 0x56411fef5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.356+0000 7fdb53fff640 1 -- 172.21.15.33:0/692318942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56411ff60210 con 0x56411fed8df0 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb53fff640 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed8df0 0x56411fef5b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56411fed5c50 tx=0x7fdb40005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33eda8b120060eba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb33fff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb33fff640 1 -- 172.21.15.33:0/692318942 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40058070 con 0x56411fed8df0 2024-09-10T21:26:21.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb33fff640 1 -- 172.21.15.33:0/692318942 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fdb40031d10 con 0x56411fed8df0 2024-09-10T21:26:21.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb4c029070 con 0x56411fed8df0 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb33fff640 1 -- 172.21.15.33:0/692318942 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7fdb40062050 con 0x56411fed8df0 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb5d030740 10 client.28694 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.357+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x56411ff614f0 con 0x56411fed8df0 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb33fff640 1 -- 172.21.15.33:0/692318942 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb4c029070 con 0x56411fed8df0 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb33fff640 1 -- 172.21.15.33:0/692318942 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7fdb40052040 con 0x56411fed8df0 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb5d030740 10 client.28694 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb5d030740 10 client.28694 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:26:21.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb5d030740 20 client.28694 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb5d030740 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411ff61d20 0x56411ff64140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb5d030740 4 client.28694 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56411fe50df0 con 0x56411ff61d20 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.358+0000 7fdb589f7640 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411ff61d20 0x56411ff64140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb33fff640 10 client.28694 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb589f7640 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411ff61d20 0x56411ff64140 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f72fececfa3f386 server_cookie=80310ddfa849b024 in_seq=0 out_seq=0 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb33fff640 1 -- 172.21.15.33:0/692318942 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56411fe50df0 con 0x56411ff61d20 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb33fff640 10 client.28694 handle_command_reply: tid=0 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 client.28694 shutdown 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed8df0 msgr2=0x56411fef5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed8df0 0x56411fef5b90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56411fed5c50 tx=0x7fdb40005b50 comp rx=0 tx=0).stop 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 shutdown_connections 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56411ff61d20 0x56411ff64140 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56411fee4d80 0x56411fef60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56411fed8df0 0x56411fef5b90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb33fff640 0 client.28694 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 --2- 172.21.15.33:0/692318942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56411fed84d0 0x56411fef5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.359+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 >> 172.21.15.33:0/692318942 conn(0x56411fed6300 msgr2=0x56411fee0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 shutdown_connections 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5d030740 1 -- 172.21.15.33:0/692318942 wait complete. 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5d030740 20 client.28694 trim_cache size 0 max 16384 2024-09-10T21:26:21.370 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:26:21.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb440b52e0 msgr2=0x7fdb440b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb440b52e0 0x7fdb440b77a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00a660 tx=0x7fdb4c00e040 comp rx=0 tx=0).stop 2024-09-10T21:26:21.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb54102de0 msgr2=0x7fdb541992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:21.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb54102de0 0x7fdb541992f0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fdb4800d950 tx=0x7fdb4800de10 comp rx=0 tx=0).stop 2024-09-10T21:26:21.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 shutdown_connections 2024-09-10T21:26:21.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fdb440b52e0 0x7fdb440b77a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb54109170 0x7fdb5419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.360+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb541087f0 0x7fdb54199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.361+0000 7fdb5ac5b640 1 --2- 172.21.15.33:0/3357300107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb54102de0 0x7fdb541992f0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:21.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.361+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 >> 172.21.15.33:0/3357300107 conn(0x7fdb540fe830 msgr2=0x7fdb54100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:21.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.361+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 shutdown_connections 2024-09-10T21:26:21.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:21.361+0000 7fdb5ac5b640 1 -- 172.21.15.33:0/3357300107 wait complete. 2024-09-10T21:26:21.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:21 smithi174 ceph-mon[35635]: pgmap v1017: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-09-10T21:26:21.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:21 smithi033 ceph-mon[30860]: pgmap v1017: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-09-10T21:26:22.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:21 smithi104 ceph-mon[35325]: pgmap v1017: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.9 MiB/s wr, 740 op/s 2024-09-10T21:26:22.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 417202 38.43 MB/sec execute 442 sec latency 97.239 ms 2024-09-10T21:26:22.220 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:26:22.532 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:23.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 417995 38.47 MB/sec execute 443 sec latency 127.983 ms 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.108+0000 7f231759e640 1 -- 172.21.15.33:0/4041239021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231811c0a0 msgr2=0x7f231811e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.108+0000 7f231759e640 1 --2- 172.21.15.33:0/4041239021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231811c0a0 0x7f231811e490 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2308009900 tx=0x7f230802a410 comp rx=0 tx=0).stop 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.110+0000 7f231759e640 1 -- 172.21.15.33:0/4041239021 shutdown_connections 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.110+0000 7f231759e640 1 --2- 172.21.15.33:0/4041239021 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231811c0a0 0x7f231811e490 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.110+0000 7f231759e640 1 --2- 172.21.15.33:0/4041239021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f231810a530 0x7f231810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.110+0000 7f231759e640 1 --2- 172.21.15.33:0/4041239021 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f231810a150 0x7f23181111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.110+0000 7f231759e640 1 -- 172.21.15.33:0/4041239021 >> 172.21.15.33:0/4041239021 conn(0x7f231806cb30 msgr2=0x7f231806cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.112+0000 7f231759e640 1 -- 172.21.15.33:0/4041239021 shutdown_connections 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.113+0000 7f231759e640 1 -- 172.21.15.33:0/4041239021 wait complete. 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 Processor -- start 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 -- start start 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f231810a150 0x7f2318112570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231810a530 0x7f2318112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2318117e50 0x7f2318118300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2318121b80 con 0x7f231810a530 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2318121a00 con 0x7f2318117e50 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.114+0000 7f231759e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2318121d00 con 0x7f231810a150 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.115+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231810a530 0x7f2318112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.116+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231810a530 0x7f2318112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39900/0 (socket says 172.21.15.33:39900) 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.116+0000 7f2315d9b640 1 -- 172.21.15.33:0/516702607 learned_addr learned my addr 172.21.15.33:0/516702607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.121+0000 7f231659c640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f231810a150 0x7f2318112570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.126+0000 7f2315d9b640 1 -- 172.21.15.33:0/516702607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f231810a150 msgr2=0x7f2318112570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.126+0000 7f2315d9b640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f231810a150 0x7f2318112570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.126+0000 7f2315d9b640 1 -- 172.21.15.33:0/516702607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2318117e50 msgr2=0x7f2318118300 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.126+0000 7f2315d9b640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2318117e50 0x7f2318118300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.126+0000 7f2315d9b640 1 -- 172.21.15.33:0/516702607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2318118a30 con 0x7f231810a530 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.132+0000 7f2315d9b640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231810a530 0x7f2318112ab0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f230c00b3c0 tx=0x7f230c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.134+0000 7f23077fe640 1 -- 172.21.15.33:0/516702607 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c011830 con 0x7f231810a530 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.134+0000 7f23077fe640 1 -- 172.21.15.33:0/516702607 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230c016070 con 0x7f231810a530 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.135+0000 7f23077fe640 1 -- 172.21.15.33:0/516702607 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c010dc0 con 0x7f231810a530 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.136+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2318118fc0 con 0x7f231810a530 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.136+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2318119420 con 0x7f231810a530 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.140+0000 7f231ed2d740 1 Processor -- start 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.141+0000 7f231ed2d740 1 -- start start 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.141+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e084142a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.141+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.141+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414ef50 0x55e084151340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.141+0000 7f231ed2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e0840b8ea0 con 0x55e084142fc0 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.141+0000 7f231ed2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e0840a82d0 con 0x55e0841426a0 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.141+0000 7f231ed2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e0840ac0e0 con 0x55e08414ef50 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.145+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e084142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.145+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e084142a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43596/0 (socket says 172.21.15.33:43596) 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.145+0000 7f2316d9d640 1 -- 172.21.15.33:0/1922042150 learned_addr learned my addr 172.21.15.33:0/1922042150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.145+0000 7f231659c640 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.146+0000 7f23077fe640 1 -- 172.21.15.33:0/516702607 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f230c010380 con 0x7f231810a530 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.147+0000 7f2316d9d640 1 -- 172.21.15.33:0/1922042150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414ef50 msgr2=0x55e084151340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.147+0000 7f2316d9d640 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414ef50 0x55e084151340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.147+0000 7f2316d9d640 1 -- 172.21.15.33:0/1922042150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 msgr2=0x55e08414ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.147+0000 7f2316d9d640 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.147+0000 7f2316d9d640 1 -- 172.21.15.33:0/1922042150 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e083e06650 con 0x55e0841426a0 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.147+0000 7f23077fe640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23000b5400 0x7f23000b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.147+0000 7f23077fe640 1 -- 172.21.15.33:0/516702607 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f230c0e4860 con 0x7f231810a530 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.150+0000 7f231659c640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23000b5400 0x7f23000b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.153+0000 7f2316d9d640 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e084142a80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2308009a20 tx=0x7f2308004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e308b34594441d9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.155+0000 7f2304ff9640 1 -- 172.21.15.33:0/1922042150 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230803f020 con 0x55e0841426a0 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.155+0000 7f2304ff9640 1 -- 172.21.15.33:0/1922042150 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23080054e0 con 0x55e0841426a0 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.156+0000 7f231ed2d740 1 -- 172.21.15.33:0/1922042150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 msgr2=0x55e084142a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.156+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e084142a80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2308009a20 tx=0x7f2308004040 comp rx=0 tx=0).stop 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.156+0000 7f231659c640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23000b5400 0x7f23000b78c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55e0841413b0 tx=0x7f2310007640 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.157+0000 7f231ed2d740 1 -- 172.21.15.33:0/1922042150 shutdown_connections 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.157+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414ef50 0x55e084151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.157+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.157+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1922042150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e084142a80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.157+0000 7f231ed2d740 1 -- 172.21.15.33:0/1922042150 >> 172.21.15.33:0/1922042150 conn(0x55e0841404d0 msgr2=0x55e084141670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.159+0000 7f231ed2d740 1 -- 172.21.15.33:0/1922042150 shutdown_connections 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.162+0000 7f231ed2d740 1 -- 172.21.15.33:0/1922042150 wait complete. 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.162+0000 7f231ed2d740 1 -- start start 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 1 Processor -- start 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e08414be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e08414c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e08414c910 0x55e08415af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e083f09700 con 0x55e08414c910 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e084043160 con 0x55e0841426a0 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e084151bc0 con 0x55e084142fc0 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.169+0000 7f231ed2d740 10 client.0 _ll_register_callbacks cb 0x55e08405c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.170+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e08414c910 0x55e08415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.170+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e08414c910 0x55e08415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39914/0 (socket says 172.21.15.33:39914) 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.170+0000 7f2316d9d640 1 -- 172.21.15.33:0/1479840755 learned_addr learned my addr 172.21.15.33:0/1479840755 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.172+0000 7f2315d9b640 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e08414c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.173+0000 7f231659c640 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e08414be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.177+0000 7f2316d9d640 1 -- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 msgr2=0x55e08414c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.177+0000 7f2316d9d640 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e08414c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.177+0000 7f2316d9d640 1 -- 172.21.15.33:0/1479840755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 msgr2=0x55e08414be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.177+0000 7f2316d9d640 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e08414be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.177+0000 7f2316d9d640 1 -- 172.21.15.33:0/1479840755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e084070c90 con 0x55e08414c910 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.181+0000 7f2316d9d640 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e08414c910 0x55e08415af40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f230802ac00 tx=0x7f23080048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff269f23849d4444 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.181+0000 7f22fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.182+0000 7f22fdffb640 1 -- 172.21.15.33:0/1479840755 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308005270 con 0x55e08414c910 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.182+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e0840badf0 con 0x55e08414c910 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.183+0000 7f22fdffb640 1 -- 172.21.15.33:0/1479840755 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2308005410 con 0x55e08414c910 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.184+0000 7f22fdffb640 1 -- 172.21.15.33:0/1479840755 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f2308054050 con 0x55e08414c910 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.184+0000 7f231ed2d740 10 client.19371 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.184+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x55e08415dbb0 con 0x55e08414c910 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f22fdffb640 1 -- 172.21.15.33:0/1479840755 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e0840badf0 con 0x55e08414c910 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f22fdffb640 1 -- 172.21.15.33:0/1479840755 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f23080314a0 con 0x55e08414c910 2024-09-10T21:26:23.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f231ed2d740 10 client.19371 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:23.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f231ed2d740 10 client.19371 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:26:23.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f231ed2d740 20 client.19371 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:23.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e08415ef00 0x55e0841612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f231ed2d740 4 client.19371 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:23.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.187+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0840b68a0 con 0x55e08415ef00 2024-09-10T21:26:23.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.189+0000 7f231659c640 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e08415ef00 0x55e0841612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.201+0000 7f231659c640 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e08415ef00 0x55e0841612f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=98fa48044b35875 server_cookie=c3dce4e0a6fc63ca in_seq=0 out_seq=0 2024-09-10T21:26:23.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.202+0000 7f22fdffb640 10 client.19371 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:26:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.203+0000 7f22fdffb640 1 -- 172.21.15.33:0/1479840755 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e0840b68a0 con 0x55e08415ef00 2024-09-10T21:26:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.203+0000 7f22fdffb640 10 client.19371 handle_command_reply: tid=0 2024-09-10T21:26:23.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.203+0000 7f231ed2d740 1 client.19371 shutdown 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e08414c910 msgr2=0x55e08415af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e08414c910 0x55e08415af40 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f230802ac00 tx=0x7f23080048f0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 shutdown_connections 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e08414c910 0x55e08415af40 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e08415ef00 0x55e0841612f0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e08414c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 --2- 172.21.15.33:0/1479840755 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e08414be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 >> 172.21.15.33:0/1479840755 conn(0x55e0841404d0 msgr2=0x55e084146070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f22fdffb640 0 client.19371 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:26:23.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 shutdown_connections 2024-09-10T21:26:23.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.207+0000 7f231ed2d740 1 -- 172.21.15.33:0/1479840755 wait complete. 2024-09-10T21:26:23.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.210+0000 7f231ed2d740 20 client.19371 trim_cache size 0 max 16384 2024-09-10T21:26:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 Processor -- start 2024-09-10T21:26:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 -- start start 2024-09-10T21:26:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e08413fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414c910 0x55e08414b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23181c58d0 con 0x55e084142fc0 2024-09-10T21:26:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23181c74a0 con 0x55e0841426a0 2024-09-10T21:26:23.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.240+0000 7f231ed2d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23181c6000 con 0x55e08414c910 2024-09-10T21:26:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.243+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.243+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414c910 0x55e08414b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.243+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39928/0 (socket says 172.21.15.33:39928) 2024-09-10T21:26:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.243+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414c910 0x55e08414b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37644/0 (socket says 172.21.15.33:37644) 2024-09-10T21:26:23.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.243+0000 7f2315d9b640 1 -- 172.21.15.33:0/4226045074 learned_addr learned my addr 172.21.15.33:0/4226045074 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:23.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.245+0000 7f2315d9b640 1 -- 172.21.15.33:0/4226045074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414c910 msgr2=0x55e08414b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.245+0000 7f2315d9b640 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414c910 0x55e08414b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.245+0000 7f2315d9b640 1 -- 172.21.15.33:0/4226045074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 msgr2=0x55e08413fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.245+0000 7f2315d9b640 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e08413fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.245+0000 7f2315d9b640 1 -- 172.21.15.33:0/4226045074 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e08414a390 con 0x55e084142fc0 2024-09-10T21:26:23.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.245+0000 7f2316d9d640 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414c910 0x55e08414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:23.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.247+0000 7f2315d9b640 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414acf0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f230c0ab000 tx=0x7f230c0aa7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12838fcaacd6794 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.255+0000 7f22fffff640 1 -- 172.21.15.33:0/4226045074 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c0aab80 con 0x55e084142fc0 2024-09-10T21:26:23.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.255+0000 7f22fffff640 1 -- 172.21.15.33:0/4226045074 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230c0ab770 con 0x55e084142fc0 2024-09-10T21:26:23.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.255+0000 7f22fffff640 1 -- 172.21.15.33:0/4226045074 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c0a6c20 con 0x55e084142fc0 2024-09-10T21:26:23.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.257+0000 7f231ed2d740 1 -- 172.21.15.33:0/4226045074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 msgr2=0x55e08414acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.257+0000 7f231ed2d740 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414acf0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f230c0ab000 tx=0x7f230c0aa7e0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.258+0000 7f231ed2d740 1 -- 172.21.15.33:0/4226045074 shutdown_connections 2024-09-10T21:26:23.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.258+0000 7f231ed2d740 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e08414c910 0x55e08414b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.258+0000 7f231ed2d740 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e084142fc0 0x55e08414acf0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.258+0000 7f231ed2d740 1 --2- 172.21.15.33:0/4226045074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e0841426a0 0x55e08413fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.258+0000 7f231ed2d740 1 -- 172.21.15.33:0/4226045074 >> 172.21.15.33:0/4226045074 conn(0x55e0841404d0 msgr2=0x55e08414a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.264+0000 7f231ed2d740 1 -- 172.21.15.33:0/4226045074 shutdown_connections 2024-09-10T21:26:23.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.266+0000 7f231ed2d740 1 -- 172.21.15.33:0/4226045074 wait complete. 2024-09-10T21:26:23.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.266+0000 7f231ed2d740 1 -- start start 2024-09-10T21:26:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.266+0000 7f231ed2d740 1 Processor -- start 2024-09-10T21:26:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.267+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0841426a0 0x55e084162860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.267+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e084162de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.267+0000 7f231ed2d740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e08414c910 0x55e0841c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.268+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e084162de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.268+0000 7f2315d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e084162de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37650/0 (socket says 172.21.15.33:37650) 2024-09-10T21:26:23.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.268+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e08414c910 0x55e0841c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.268+0000 7f2316d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e08414c910 0x55e0841c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43624/0 (socket says 172.21.15.33:43624) 2024-09-10T21:26:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.268+0000 7f2315d9b640 1 -- 172.21.15.33:0/369428457 learned_addr learned my addr 172.21.15.33:0/369428457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f23181c6da0 con 0x55e0841426a0 2024-09-10T21:26:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f23181c66d0 con 0x55e08414c910 2024-09-10T21:26:23.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23181c3890 con 0x55e084142fc0 2024-09-10T21:26:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f2316d9d640 1 -- 172.21.15.33:0/369428457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 msgr2=0x55e084162de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f2316d9d640 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e084162de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f2316d9d640 1 -- 172.21.15.33:0/369428457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0841426a0 msgr2=0x55e084162860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f2316d9d640 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0841426a0 0x55e084162860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.269+0000 7f2316d9d640 1 -- 172.21.15.33:0/369428457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0841c51c0 con 0x55e08414c910 2024-09-10T21:26:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.270+0000 7f2316d9d640 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e08414c910 0x55e0841c4ab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f230802a450 tx=0x7f230802a990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4240629f99156b29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.271+0000 7f231ed2d740 10 client.0 _ll_register_callbacks cb 0x7f2310008040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:23.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.271+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230c0ab770 con 0x55e08414c910 2024-09-10T21:26:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.273+0000 7f22fdffb640 10 client.28703 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.273+0000 7f22fdffb640 1 -- 172.21.15.33:0/369428457 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230803f040 con 0x55e08414c910 2024-09-10T21:26:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.273+0000 7f22fdffb640 1 -- 172.21.15.33:0/369428457 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f23080314a0 con 0x55e08414c910 2024-09-10T21:26:23.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.273+0000 7f22fdffb640 1 -- 172.21.15.33:0/369428457 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f23080316a0 con 0x55e08414c910 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.276+0000 7f231ed2d740 10 client.28703 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.276+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=87}) -- 0x55e0841cc120 con 0x55e08414c910 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.280+0000 7f22fdffb640 1 -- 172.21.15.33:0/369428457 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f2308031a90 con 0x55e08414c910 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.280+0000 7f231ed2d740 10 client.28703 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.280+0000 7f231ed2d740 10 client.28703 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.280+0000 7f231ed2d740 20 client.28703 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.280+0000 7f231ed2d740 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e0841cc8e0 0x55e0841ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.280+0000 7f231ed2d740 4 client.28703 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:26:23.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.280+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f230c0ab770 con 0x55e0841cc8e0 2024-09-10T21:26:23.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.285+0000 7f231659c640 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e0841cc8e0 0x55e0841ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.296+0000 7f231659c640 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e0841cc8e0 0x55e0841ceda0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c5dba9049915506 server_cookie=cc1337fb53374305 in_seq=0 out_seq=0 2024-09-10T21:26:23.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.297+0000 7f22fdffb640 10 client.28703 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:26:23.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.298+0000 7f22fdffb640 1 -- 172.21.15.33:0/369428457 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f230c0ab770 con 0x55e0841cc8e0 2024-09-10T21:26:23.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.298+0000 7f22fdffb640 10 client.28703 handle_command_reply: tid=0 2024-09-10T21:26:23.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.298+0000 7f231ed2d740 1 client.28703 shutdown 2024-09-10T21:26:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.303+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e08414c910 msgr2=0x55e0841c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.303+0000 7f231ed2d740 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e08414c910 0x55e0841c4ab0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f230802a450 tx=0x7f230802a990 comp rx=0 tx=0).stop 2024-09-10T21:26:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 shutdown_connections 2024-09-10T21:26:23.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e08414c910 0x55e0841c4ab0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55e0841cc8e0 0x55e0841ceda0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e084142fc0 0x55e084162de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 --2- 172.21.15.33:0/369428457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e0841426a0 0x55e084162860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 >> 172.21.15.33:0/369428457 conn(0x55e0841404d0 msgr2=0x55e08414ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f22fdffb640 0 client.28703 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:26:23.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 shutdown_connections 2024-09-10T21:26:23.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 1 -- 172.21.15.33:0/369428457 wait complete. 2024-09-10T21:26:23.313 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:26:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.307+0000 7f231ed2d740 20 client.28703 trim_cache size 0 max 16384 2024-09-10T21:26:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.308+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23000b5400 msgr2=0x7f23000b78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.308+0000 7f231759e640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23000b5400 0x7f23000b78c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55e0841413b0 tx=0x7f2310007640 comp rx=0 tx=0).stop 2024-09-10T21:26:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.308+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231810a530 msgr2=0x7f2318112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.309+0000 7f231759e640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231810a530 0x7f2318112ab0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f230c00b3c0 tx=0x7f230c00b880 comp rx=0 tx=0).stop 2024-09-10T21:26:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.310+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 shutdown_connections 2024-09-10T21:26:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.310+0000 7f231759e640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f23000b5400 0x7f23000b78c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.310+0000 7f231759e640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2318117e50 0x7f2318118300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.310+0000 7f231759e640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f231810a530 0x7f2318112ab0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.310+0000 7f231759e640 1 --2- 172.21.15.33:0/516702607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f231810a150 0x7f2318112570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.310+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 >> 172.21.15.33:0/516702607 conn(0x7f231806cb30 msgr2=0x7f231811d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.311+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 shutdown_connections 2024-09-10T21:26:23.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:23.311+0000 7f231759e640 1 -- 172.21.15.33:0/516702607 wait complete. 2024-09-10T21:26:23.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:23 smithi174 ceph-mon[35635]: pgmap v1018: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 554 op/s 2024-09-10T21:26:23.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:23 smithi033 ceph-mon[30860]: pgmap v1018: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 554 op/s 2024-09-10T21:26:24.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:23 smithi104 ceph-mon[35325]: pgmap v1018: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 554 op/s 2024-09-10T21:26:24.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 418039 38.40 MB/sec execute 444 sec latency 880.774 ms 2024-09-10T21:26:25.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 418377 38.36 MB/sec execute 445 sec latency 387.018 ms 2024-09-10T21:26:25.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:25 smithi174 ceph-mon[35635]: pgmap v1019: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.5 MiB/s wr, 680 op/s 2024-09-10T21:26:25.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:25 smithi033 ceph-mon[30860]: pgmap v1019: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.5 MiB/s wr, 680 op/s 2024-09-10T21:26:26.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:25 smithi104 ceph-mon[35325]: pgmap v1019: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.5 MiB/s wr, 680 op/s 2024-09-10T21:26:26.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 418865 38.32 MB/sec execute 446 sec latency 126.139 ms 2024-09-10T21:26:26.490 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:26:26.840 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:27.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 419487 38.30 MB/sec execute 447 sec latency 88.104 ms 2024-09-10T21:26:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 -- 172.21.15.33:0/694290937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 msgr2=0x7ff000103d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 --2- 172.21.15.33:0/694290937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 0x7ff000103d80 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7feff0009900 tx=0x7feff002a410 comp rx=0 tx=0).stop 2024-09-10T21:26:27.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 -- 172.21.15.33:0/694290937 shutdown_connections 2024-09-10T21:26:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 --2- 172.21.15.33:0/694290937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff00010cf50 0x7ff00010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 --2- 172.21.15.33:0/694290937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0001043a0 0x7ff00010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 --2- 172.21.15.33:0/694290937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 0x7ff000103d80 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 -- 172.21.15.33:0/694290937 >> 172.21.15.33:0/694290937 conn(0x7ff0000fd5b0 msgr2=0x7ff0000ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 -- 172.21.15.33:0/694290937 shutdown_connections 2024-09-10T21:26:27.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.385+0000 7ff0055d5640 1 -- 172.21.15.33:0/694290937 wait complete. 2024-09-10T21:26:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.386+0000 7ff0055d5640 1 Processor -- start 2024-09-10T21:26:27.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.387+0000 7ff0055d5640 1 -- start start 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.387+0000 7ff0055d5640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 0x7ff000199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7ff0055d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0001043a0 0x7ff0001995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7ff0055d5640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff00010cf50 0x7ff00019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7ff0055d5640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112c80 con 0x7ff0001039a0 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0001043a0 0x7ff0001995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0001043a0 0x7ff0001995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37664/0 (socket says 172.21.15.33:37664) 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7fefff7fe640 1 -- 172.21.15.33:0/1273293088 learned_addr learned my addr 172.21.15.33:0/1273293088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112b00 con 0x7ff00010cf50 2024-09-10T21:26:27.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff000112e00 con 0x7ff0001043a0 2024-09-10T21:26:27.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.388+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff00010cf50 0x7ff00019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7feffffff640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 0x7ff000199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7ff004dd4640 1 -- 172.21.15.33:0/1273293088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0001043a0 msgr2=0x7ff0001995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0001043a0 0x7ff0001995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7ff004dd4640 1 -- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 msgr2=0x7ff000199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 0x7ff000199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7ff004dd4640 1 -- 172.21.15.33:0/1273293088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00019f120 con 0x7ff00010cf50 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff00010cf50 0x7ff00019e950 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feff8009fd0 tx=0x7feff800ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.389+0000 7feffd7fa640 1 -- 172.21.15.33:0/1273293088 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff800ce20 con 0x7ff00010cf50 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.390+0000 7feffd7fa640 1 -- 172.21.15.33:0/1273293088 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff8005020 con 0x7ff00010cf50 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.390+0000 7feffd7fa640 1 -- 172.21.15.33:0/1273293088 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8010690 con 0x7ff00010cf50 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.390+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00019f410 con 0x7ff00010cf50 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.390+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff000101700 con 0x7ff00010cf50 2024-09-10T21:26:27.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.390+0000 7fefff7fe640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0001043a0 0x7ff0001995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:27.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.394+0000 7ff008c0c740 1 Processor -- start 2024-09-10T21:26:27.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.396+0000 7feffd7fa640 1 -- 172.21.15.33:0/1273293088 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7feff80054d0 con 0x7ff00010cf50 2024-09-10T21:26:27.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.397+0000 7ff008c0c740 1 -- start start 2024-09-10T21:26:27.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.397+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e7921f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.397+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f8fc0 0x55e792204a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.397+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e792207340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.397+0000 7ff008c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e79216eea0 con 0x55e7921f86a0 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.397+0000 7ff008c0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e79215e2d0 con 0x55e7921f8fc0 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.397+0000 7ff008c0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e7921620e0 con 0x55e792204f50 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f8fc0 0x55e792204a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f8fc0 0x55e792204a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:43658/0 (socket says 172.21.15.33:43658) 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 -- 172.21.15.33:0/449415354 learned_addr learned my addr 172.21.15.33:0/449415354 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:27.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7fefff7fe640 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e792207340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 -- 172.21.15.33:0/449415354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 msgr2=0x55e792207340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e792207340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 -- 172.21.15.33:0/449415354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 msgr2=0x55e7921f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e7921f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.398+0000 7feffffff640 1 -- 172.21.15.33:0/449415354 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e791ebc650 con 0x55e7921f8fc0 2024-09-10T21:26:27.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.399+0000 7feffffff640 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f8fc0 0x55e792204a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7feff0009230 tx=0x7feff00042e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95a76b136e325330 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.400+0000 7fefe67fc640 1 -- 172.21.15.33:0/449415354 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff0035030 con 0x55e7921f8fc0 2024-09-10T21:26:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.400+0000 7fefe67fc640 1 -- 172.21.15.33:0/449415354 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff00044d0 con 0x55e7921f8fc0 2024-09-10T21:26:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.400+0000 7fefe67fc640 1 -- 172.21.15.33:0/449415354 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff00035f0 con 0x55e7921f8fc0 2024-09-10T21:26:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.401+0000 7feffd7fa640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fefe80b9820 0x7fefe80bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.401+0000 7feffd7fa640 1 -- 172.21.15.33:0/1273293088 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feff80e6180 con 0x7ff00010cf50 2024-09-10T21:26:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.401+0000 7ff008c0c740 1 -- 172.21.15.33:0/449415354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f8fc0 msgr2=0x55e792204a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.401+0000 7ff008c0c740 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f8fc0 0x55e792204a10 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7feff0009230 tx=0x7feff00042e0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7ff008c0c740 1 -- 172.21.15.33:0/449415354 shutdown_connections 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7ff008c0c740 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e792207340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7ff008c0c740 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f8fc0 0x55e792204a10 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7ff008c0c740 1 --2- 172.21.15.33:0/449415354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e7921f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7fefff7fe640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fefe80b9820 0x7fefe80bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7ff008c0c740 1 -- 172.21.15.33:0/449415354 >> 172.21.15.33:0/449415354 conn(0x55e7921f64d0 msgr2=0x55e7921f7670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7ff008c0c740 1 -- 172.21.15.33:0/449415354 shutdown_connections 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7ff008c0c740 1 -- 172.21.15.33:0/449415354 wait complete. 2024-09-10T21:26:27.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.402+0000 7fefff7fe640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fefe80b9820 0x7fefe80bbce0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7feff4004810 tx=0x7feff4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.403+0000 7ff008c0c740 1 -- start start 2024-09-10T21:26:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.403+0000 7ff008c0c740 1 Processor -- start 2024-09-10T21:26:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f86a0 0x55e792204570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e7922019c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e792202380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7ff008c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e791fbf700 con 0x55e792201f00 2024-09-10T21:26:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7ff008c0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e7920f9160 con 0x55e7921f86a0 2024-09-10T21:26:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7ff008c0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e792207c10 con 0x55e792204f50 2024-09-10T21:26:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e7922019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e7922019c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37670/0 (socket says 172.21.15.33:37670) 2024-09-10T21:26:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7fefff7fe640 1 -- 172.21.15.33:0/1723380752 learned_addr learned my addr 172.21.15.33:0/1723380752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.404+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e792202380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7ff008c0c740 10 client.0 _ll_register_callbacks cb 0x55e7921129e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7ff004dd4640 1 -- 172.21.15.33:0/1723380752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 msgr2=0x55e7922019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e7922019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7ff004dd4640 1 -- 172.21.15.33:0/1723380752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f86a0 msgr2=0x55e792204570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f86a0 0x55e792204570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7ff004dd4640 1 -- 172.21.15.33:0/1723380752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e792126c90 con 0x55e792201f00 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7ff004dd4640 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e792202380 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7feff800c760 tx=0x7feff80a8170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcc34bec805bcd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.405+0000 7fefdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:26:27.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.406+0000 7fefdf7fe640 1 -- 172.21.15.33:0/1723380752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8096020 con 0x55e792201f00 2024-09-10T21:26:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.406+0000 7fefdf7fe640 1 -- 172.21.15.33:0/1723380752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feff80b5a40 con 0x55e792201f00 2024-09-10T21:26:27.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.407+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff00044d0 con 0x55e792201f00 2024-09-10T21:26:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.408+0000 7fefdf7fe640 1 -- 172.21.15.33:0/1723380752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7feff80b1420 con 0x55e792201f00 2024-09-10T21:26:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.408+0000 7ff008c0c740 10 client.19383 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:27.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.408+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x55e7922137c0 con 0x55e792201f00 2024-09-10T21:26:27.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.410+0000 7fefdf7fe640 1 -- 172.21.15.33:0/1723380752 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff80b5ed0 con 0x55e792201f00 2024-09-10T21:26:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.411+0000 7fefdf7fe640 1 -- 172.21.15.33:0/1723380752 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7feff80ab610 con 0x55e792201f00 2024-09-10T21:26:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.411+0000 7ff008c0c740 10 client.19383 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.411+0000 7ff008c0c740 10 client.19383 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:26:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.411+0000 7ff008c0c740 20 client.19383 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.412+0000 7ff008c0c740 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922150e0 0x55e792217560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.412+0000 7ff008c0c740 4 client.19383 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:27.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.412+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7feff00044d0 con 0x55e7922150e0 2024-09-10T21:26:27.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.412+0000 7feffffff640 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922150e0 0x55e792217560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.417+0000 7fefdf7fe640 10 client.19383 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:26:27.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.417+0000 7feffffff640 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922150e0 0x55e792217560 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5628a3d5b3c3586 server_cookie=2b048cc4c359edb9 in_seq=0 out_seq=0 2024-09-10T21:26:27.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.421+0000 7fefdf7fe640 1 -- 172.21.15.33:0/1723380752 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7feff00044d0 con 0x55e7922150e0 2024-09-10T21:26:27.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.422+0000 7fefdf7fe640 10 client.19383 handle_command_reply: tid=0 2024-09-10T21:26:27.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.422+0000 7ff008c0c740 1 client.19383 shutdown 2024-09-10T21:26:27.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 msgr2=0x55e792202380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e792202380 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7feff800c760 tx=0x7feff80a8170 comp rx=0 tx=0).stop 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 shutdown_connections 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922150e0 0x55e792217560 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e792202380 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792204f50 0x55e7922019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 --2- 172.21.15.33:0/1723380752 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e7921f86a0 0x55e792204570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 >> 172.21.15.33:0/1723380752 conn(0x55e7921f64d0 msgr2=0x55e7921f4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.423+0000 7fefdf7fe640 0 client.19383 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:26:27.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.424+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 shutdown_connections 2024-09-10T21:26:27.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.424+0000 7ff008c0c740 1 -- 172.21.15.33:0/1723380752 wait complete. 2024-09-10T21:26:27.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.424+0000 7ff008c0c740 20 client.19383 trim_cache size 0 max 16384 2024-09-10T21:26:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.444+0000 7ff008c0c740 1 Processor -- start 2024-09-10T21:26:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7ff008c0c740 1 -- start start 2024-09-10T21:26:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7921f86a0 0x55e792210ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e7921f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e7921f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7ff008c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001abe40 con 0x55e792201f00 2024-09-10T21:26:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7feffffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7921f86a0 0x55e792210ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7feffffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7921f86a0 0x55e792210ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37684/0 (socket says 172.21.15.33:37684) 2024-09-10T21:26:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.445+0000 7feffffff640 1 -- 172.21.15.33:0/2245406376 learned_addr learned my addr 172.21.15.33:0/2245406376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.446+0000 7ff008c0c740 1 -- 172.21.15.33:0/2245406376 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001ada10 con 0x55e792204f50 2024-09-10T21:26:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.446+0000 7ff004dd4640 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e7921f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.446+0000 7fefff7fe640 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e7921f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.446+0000 7ff008c0c740 1 -- 172.21.15.33:0/2245406376 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001ac570 con 0x55e7921f86a0 2024-09-10T21:26:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7ff004dd4640 1 -- 172.21.15.33:0/2245406376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7921f86a0 msgr2=0x55e792210ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7ff004dd4640 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7921f86a0 0x55e792210ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7ff004dd4640 1 -- 172.21.15.33:0/2245406376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 msgr2=0x55e7921f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7ff004dd4640 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e7921f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7ff004dd4640 1 -- 172.21.15.33:0/2245406376 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7922076b0 con 0x55e792204f50 2024-09-10T21:26:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7ff004dd4640 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e7921f7b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feff80a9bb0 tx=0x7feff80a90c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660c954e700c051d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7fefe5ffb640 1 -- 172.21.15.33:0/2245406376 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff8096070 con 0x55e792204f50 2024-09-10T21:26:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7fefe5ffb640 1 -- 172.21.15.33:0/2245406376 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff80a85e0 con 0x55e792204f50 2024-09-10T21:26:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.447+0000 7fefe5ffb640 1 -- 172.21.15.33:0/2245406376 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff80ab610 con 0x55e792204f50 2024-09-10T21:26:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.448+0000 7fefff7fe640 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e7921f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 -- 172.21.15.33:0/2245406376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 msgr2=0x55e7921f7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e7921f7b40 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feff80a9bb0 tx=0x7feff80a90c0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 -- 172.21.15.33:0/2245406376 shutdown_connections 2024-09-10T21:26:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e7921f7b40 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e792201f00 0x55e7921f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 --2- 172.21.15.33:0/2245406376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7921f86a0 0x55e792210ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 -- 172.21.15.33:0/2245406376 >> 172.21.15.33:0/2245406376 conn(0x55e7921f64d0 msgr2=0x55e792200240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 -- 172.21.15.33:0/2245406376 shutdown_connections 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.449+0000 7ff008c0c740 1 -- 172.21.15.33:0/2245406376 wait complete. 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.450+0000 7ff008c0c740 1 -- start start 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.450+0000 7ff008c0c740 1 Processor -- start 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e79220d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792201f00 0x55e792206320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7ff008c0c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e792206860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7ff008c0c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001ad310 con 0x55e7921f86a0 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7ff008c0c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001acc40 con 0x55e792204f50 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7ff008c0c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0001a9ed0 con 0x55e792201f00 2024-09-10T21:26:27.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7ff008c0c740 10 client.0 _ll_register_callbacks cb 0x55e792204ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792201f00 0x55e792206320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7fefff7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792201f00 0x55e792206320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:37686/0 (socket says 172.21.15.33:37686) 2024-09-10T21:26:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7fefff7fe640 1 -- 172.21.15.33:0/4095304298 learned_addr learned my addr 172.21.15.33:0/4095304298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.451+0000 7feffffff640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e79220d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.452+0000 7ff004dd4640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e792206860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.452+0000 7fefff7fe640 1 -- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 msgr2=0x55e792206860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.452+0000 7fefff7fe640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e792206860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.452+0000 7fefff7fe640 1 -- 172.21.15.33:0/4095304298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 msgr2=0x55e79220d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:27.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.452+0000 7fefff7fe640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e79220d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.452+0000 7fefff7fe640 1 -- 172.21.15.33:0/4095304298 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e792207070 con 0x55e792201f00 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.452+0000 7feffffff640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e79220d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.453+0000 7fefff7fe640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792201f00 0x55e792206320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feff4006e40 tx=0x7feff4006cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceab012fbf0d0ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.453+0000 7fefdf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.453+0000 7fefdf7fe640 1 -- 172.21.15.33:0/4095304298 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feff4025030 con 0x55e792201f00 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.453+0000 7fefdf7fe640 1 -- 172.21.15.33:0/4095304298 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7feff4021480 con 0x55e792201f00 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.453+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feff80a85e0 con 0x55e792201f00 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.453+0000 7fefdf7fe640 1 -- 172.21.15.33:0/4095304298 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7feff4021680 con 0x55e792201f00 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.454+0000 7ff008c0c740 10 client.28588 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:27.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.454+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x55e7922820d0 con 0x55e792201f00 2024-09-10T21:26:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.454+0000 7fefdf7fe640 1 -- 172.21.15.33:0/4095304298 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feff80a85e0 con 0x55e792201f00 2024-09-10T21:26:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.454+0000 7fefdf7fe640 1 -- 172.21.15.33:0/4095304298 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7feff401e050 con 0x55e792201f00 2024-09-10T21:26:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.455+0000 7ff008c0c740 10 client.28588 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.455+0000 7ff008c0c740 10 client.28588 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:26:27.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.455+0000 7ff008c0c740 20 client.28588 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.455+0000 7ff008c0c740 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922828d0 0x55e792284cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.455+0000 7ff008c0c740 4 client.28588 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:26:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.455+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e792170df0 con 0x55e7922828d0 2024-09-10T21:26:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.455+0000 7feffffff640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922828d0 0x55e792284cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.457+0000 7feffffff640 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922828d0 0x55e792284cc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=488f86926b5b2ee0 server_cookie=571fa99362eedf1a in_seq=0 out_seq=0 2024-09-10T21:26:27.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.457+0000 7fefdf7fe640 10 client.28588 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:26:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.460+0000 7fefdf7fe640 1 -- 172.21.15.33:0/4095304298 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e792170df0 con 0x55e7922828d0 2024-09-10T21:26:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.460+0000 7fefdf7fe640 10 client.28588 handle_command_reply: tid=0 2024-09-10T21:26:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.460+0000 7ff008c0c740 1 client.28588 shutdown 2024-09-10T21:26:27.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.460+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792201f00 msgr2=0x55e792206320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.460+0000 7ff008c0c740 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792201f00 0x55e792206320 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7feff4006e40 tx=0x7feff4006cc0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 shutdown_connections 2024-09-10T21:26:27.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e792204f50 0x55e792206860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e792201f00 0x55e792206320 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55e7922828d0 0x55e792284cc0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7fefdf7fe640 0 client.28588 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 --2- 172.21.15.33:0/4095304298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e7921f86a0 0x55e79220d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 >> 172.21.15.33:0/4095304298 conn(0x55e7921f64d0 msgr2=0x55e7921f73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 shutdown_connections 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.461+0000 7ff008c0c740 1 -- 172.21.15.33:0/4095304298 wait complete. 2024-09-10T21:26:27.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.462+0000 7ff008c0c740 20 client.28588 trim_cache size 0 max 16384 2024-09-10T21:26:27.466 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:26:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fefe80b9820 msgr2=0x7fefe80bbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fefe80b9820 0x7fefe80bbce0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7feff4004810 tx=0x7feff4008040 comp rx=0 tx=0).stop 2024-09-10T21:26:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff00010cf50 msgr2=0x7ff00019e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff00010cf50 0x7ff00019e950 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7feff8009fd0 tx=0x7feff800ef10 comp rx=0 tx=0).stop 2024-09-10T21:26:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 shutdown_connections 2024-09-10T21:26:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fefe80b9820 0x7fefe80bbce0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff00010cf50 0x7ff00019e950 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0001043a0 0x7ff0001995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 --2- 172.21.15.33:0/1273293088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0001039a0 0x7ff000199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 >> 172.21.15.33:0/1273293088 conn(0x7ff0000fd5b0 msgr2=0x7ff00010f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.464+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 shutdown_connections 2024-09-10T21:26:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:27.465+0000 7ff0055d5640 1 -- 172.21.15.33:0/1273293088 wait complete. 2024-09-10T21:26:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:27 smithi174 ceph-mon[35635]: pgmap v1020: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 673 op/s 2024-09-10T21:26:27.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:27 smithi033 ceph-mon[30860]: pgmap v1020: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 673 op/s 2024-09-10T21:26:28.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:27 smithi104 ceph-mon[35325]: pgmap v1020: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 10 MiB/s wr, 673 op/s 2024-09-10T21:26:28.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 420201 38.29 MB/sec execute 448 sec latency 91.476 ms 2024-09-10T21:26:29.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 420771 38.26 MB/sec execute 449 sec latency 299.580 ms 2024-09-10T21:26:29.232 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:26:29.607 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:29.875 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:29 smithi033 ceph-mon[30860]: pgmap v1021: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 686 op/s 2024-09-10T21:26:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:29 smithi174 ceph-mon[35635]: pgmap v1021: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 686 op/s 2024-09-10T21:26:30.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:29 smithi104 ceph-mon[35325]: pgmap v1021: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 686 op/s 2024-09-10T21:26:30.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 421494 38.27 MB/sec execute 450 sec latency 100.690 ms 2024-09-10T21:26:30.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.132+0000 7f39dd599640 1 -- 172.21.15.33:0/382243674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 msgr2=0x7f39d8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.132+0000 7f39dd599640 1 --2- 172.21.15.33:0/382243674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 0x7f39d8075920 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f39d001acb0 tx=0x7f39d003b920 comp rx=0 tx=0).stop 2024-09-10T21:26:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.132+0000 7f39dd599640 1 -- 172.21.15.33:0/382243674 shutdown_connections 2024-09-10T21:26:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.132+0000 7f39dd599640 1 --2- 172.21.15.33:0/382243674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d8113b40 0x7f39d8115f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.132+0000 7f39dd599640 1 --2- 172.21.15.33:0/382243674 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8075e60 0x7f39d8113600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.132+0000 7f39dd599640 1 --2- 172.21.15.33:0/382243674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 0x7f39d8075920 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.132+0000 7f39dd599640 1 -- 172.21.15.33:0/382243674 >> 172.21.15.33:0/382243674 conn(0x7f39d80fe830 msgr2=0x7f39d8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.133+0000 7f39dd599640 1 -- 172.21.15.33:0/382243674 shutdown_connections 2024-09-10T21:26:30.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.133+0000 7f39dd599640 1 -- 172.21.15.33:0/382243674 wait complete. 2024-09-10T21:26:30.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 Processor -- start 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 -- start start 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 0x7f39d810f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d8075e60 0x7f39d810f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8113b40 0x7f39d81be920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39d8109660 con 0x7f39d8075540 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81094e0 con 0x7f39d8113b40 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.134+0000 7f39dd599640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39d81097e0 con 0x7f39d8075e60 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8113b40 0x7f39d81be920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8113b40 0x7f39d81be920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58672/0 (socket says 172.21.15.33:58672) 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 -- 172.21.15.33:0/4114903612 learned_addr learned my addr 172.21.15.33:0/4114903612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:30.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d67fc640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d8075e60 0x7f39d810f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 0x7f39d810f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 -- 172.21.15.33:0/4114903612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d8075e60 msgr2=0x7f39d810f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d8075e60 0x7f39d810f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 -- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 msgr2=0x7f39d810f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 0x7f39d810f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 -- 172.21.15.33:0/4114903612 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d81bf0a0 con 0x7f39d8113b40 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 0x7f39d810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.135+0000 7f39d77fe640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8113b40 0x7f39d81be920 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39c800e970 tx=0x7f39c800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.136+0000 7f39bffff640 1 -- 172.21.15.33:0/4114903612 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c800ccc0 con 0x7f39d8113b40 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.136+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d81bf390 con 0x7f39d8113b40 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.136+0000 7f39bffff640 1 -- 172.21.15.33:0/4114903612 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39c80054d0 con 0x7f39d8113b40 2024-09-10T21:26:30.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.136+0000 7f39bffff640 1 -- 172.21.15.33:0/4114903612 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8010430 con 0x7f39d8113b40 2024-09-10T21:26:30.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.136+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d81c5ac0 con 0x7f39d8113b40 2024-09-10T21:26:30.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.140+0000 7f39df96e740 1 Processor -- start 2024-09-10T21:26:30.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.141+0000 7f39bffff640 1 -- 172.21.15.33:0/4114903612 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f39c8005020 con 0x7f39d8113b40 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.141+0000 7f39df96e740 1 -- start start 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.141+0000 7f39df96e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8d4d0 0x559166a8d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.141+0000 7f39df96e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a8ddf0 0x559166a99840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.141+0000 7f39df96e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a9c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.141+0000 7f39df96e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559166a03ea0 con 0x559166a99d80 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.142+0000 7f39df96e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591669f32d0 con 0x559166a8ddf0 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.142+0000 7f39df96e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5591669f70e0 con 0x559166a8d4d0 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.142+0000 7f39d67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8d4d0 0x559166a8d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.142+0000 7f39d67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8d4d0 0x559166a8d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54520/0 (socket says 172.21.15.33:54520) 2024-09-10T21:26:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.142+0000 7f39d67fc640 1 -- 172.21.15.33:0/2491292252 learned_addr learned my addr 172.21.15.33:0/2491292252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.142+0000 7f39d67fc640 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a9c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.142+0000 7f39bffff640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39c00b5390 0x7f39c00b7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.143+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a8ddf0 0x559166a99840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.143+0000 7f39bffff640 1 -- 172.21.15.33:0/4114903612 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f39c80e5dd0 con 0x7f39d8113b40 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.143+0000 7f39d67fc640 1 -- 172.21.15.33:0/2491292252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a8ddf0 msgr2=0x559166a99840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.143+0000 7f39d67fc640 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a8ddf0 0x559166a99840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.143+0000 7f39d67fc640 1 -- 172.21.15.33:0/2491292252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 msgr2=0x559166a9c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.143+0000 7f39d67fc640 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a9c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.143+0000 7f39d77fe640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39c00b5390 0x7f39c00b7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.144+0000 7f39d67fc640 1 -- 172.21.15.33:0/2491292252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559166751650 con 0x559166a8d4d0 2024-09-10T21:26:30.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.144+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a8ddf0 0x559166a99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.144+0000 7f39d77fe640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39c00b5390 0x7f39c00b7850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f39c800cf80 tx=0x7f39c80096c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.144+0000 7f39d67fc640 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8d4d0 0x559166a8d8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f39cc012770 tx=0x7f39cc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4d4aaa798f3352 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.145+0000 7f39bd7fa640 1 -- 172.21.15.33:0/2491292252 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc012cf0 con 0x559166a8d4d0 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.145+0000 7f39bd7fa640 1 -- 172.21.15.33:0/2491292252 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc010c20 con 0x559166a8d4d0 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.145+0000 7f39bd7fa640 1 -- 172.21.15.33:0/2491292252 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc01fad0 con 0x559166a8d4d0 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 -- 172.21.15.33:0/2491292252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8d4d0 msgr2=0x559166a8d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8d4d0 0x559166a8d8b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f39cc012770 tx=0x7f39cc0046c0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 -- 172.21.15.33:0/2491292252 shutdown_connections 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a9c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a8ddf0 0x559166a99840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 --2- 172.21.15.33:0/2491292252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8d4d0 0x559166a8d8b0 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f39cc012770 tx=0x7f39cc0046c0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 -- 172.21.15.33:0/2491292252 >> 172.21.15.33:0/2491292252 conn(0x559166a8b300 msgr2=0x559166a8c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.146+0000 7f39df96e740 1 -- 172.21.15.33:0/2491292252 shutdown_connections 2024-09-10T21:26:30.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.147+0000 7f39df96e740 1 -- 172.21.15.33:0/2491292252 wait complete. 2024-09-10T21:26:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.147+0000 7f39df96e740 1 -- start start 2024-09-10T21:26:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.147+0000 7f39df96e740 1 Processor -- start 2024-09-10T21:26:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.147+0000 7f39df96e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166a96cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39df96e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a97200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39df96e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39df96e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x559166854700 con 0x559166a99d80 2024-09-10T21:26:30.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39df96e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55916698e160 con 0x559166a97740 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39df96e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559166a9ca00 con 0x559166a8ddf0 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39df96e740 10 client.0 _ll_register_callbacks cb 0x5591669a79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166a96cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166a96cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54534/0 (socket says 172.21.15.33:54534) 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 -- 172.21.15.33:0/1487933763 learned_addr learned my addr 172.21.15.33:0/1487933763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a97200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 -- 172.21.15.33:0/1487933763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 msgr2=0x559166a96cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166a96cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 -- 172.21.15.33:0/1487933763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 msgr2=0x559166a97200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a97200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.148+0000 7f39d67fc640 1 -- 172.21.15.33:0/1487933763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591669bbc90 con 0x559166a97740 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.149+0000 7f39d67fc640 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39cc005780 tx=0x7f39cc00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a72ad5b73dc9dca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.149+0000 7f39baff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.149+0000 7f39baff5640 1 -- 172.21.15.33:0/1487933763 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc010980 con 0x559166a97740 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.149+0000 7f39baff5640 1 -- 172.21.15.33:0/1487933763 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f39cc005020 con 0x559166a97740 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.149+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39cc010c20 con 0x559166a97740 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39baff5640 1 -- 172.21.15.33:0/1487933763 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f39cc005220 con 0x559166a97740 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 10 client.28727 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x559166aa89a0 con 0x559166a97740 2024-09-10T21:26:30.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39baff5640 1 -- 172.21.15.33:0/1487933763 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39cc010c20 con 0x559166a97740 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39baff5640 1 -- 172.21.15.33:0/1487933763 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f39cc005450 con 0x559166a97740 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 10 client.28727 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 10 client.28727 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 20 client.28727 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166aa9cf0 0x559166aac0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 4 client.28727 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.150+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559166a05df0 con 0x559166aa9cf0 2024-09-10T21:26:30.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.151+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166aa9cf0 0x559166aac0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.151+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166aa9cf0 0x559166aac0e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8104eea7d18f02cd server_cookie=a8b8e4e8e298209c in_seq=0 out_seq=0 2024-09-10T21:26:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.151+0000 7f39baff5640 10 client.28727 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:26:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39baff5640 1 -- 172.21.15.33:0/1487933763 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559166a05df0 con 0x559166aa9cf0 2024-09-10T21:26:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39baff5640 10 client.28727 handle_command_reply: tid=0 2024-09-10T21:26:30.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 client.28727 shutdown 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 msgr2=0x559166aa5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5dc0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f39cc005780 tx=0x7f39cc00bfd0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 shutdown_connections 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5dc0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166aa9cf0 0x559166aac0e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a99d80 0x559166a97200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 --2- 172.21.15.33:0/1487933763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166a96cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 >> 172.21.15.33:0/1487933763 conn(0x559166a8b300 msgr2=0x559166a90e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.152+0000 7f39baff5640 0 client.28727 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.153+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 shutdown_connections 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.153+0000 7f39df96e740 1 -- 172.21.15.33:0/1487933763 wait complete. 2024-09-10T21:26:30.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.153+0000 7f39df96e740 20 client.28727 trim_cache size 0 max 16384 2024-09-10T21:26:30.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.170+0000 7f39df96e740 1 Processor -- start 2024-09-10T21:26:30.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.170+0000 7f39df96e740 1 -- start start 2024-09-10T21:26:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.170+0000 7f39df96e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a8ddf0 0x559166a98a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.170+0000 7f39df96e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.170+0000 7f39df96e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a99d80 0x559166aa6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.170+0000 7f39df96e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39c400e580 con 0x559166a8ddf0 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39df96e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39c4010150 con 0x559166a97740 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39df96e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39c400ecb0 con 0x559166a99d80 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39d6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39d6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58702/0 (socket says 172.21.15.33:58702) 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39d6ffd640 1 -- 172.21.15.33:0/2377792590 learned_addr learned my addr 172.21.15.33:0/2377792590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39d67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a8ddf0 0x559166a98a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39d67fc640 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a99d80 0x559166aa6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.171+0000 7f39d6ffd640 1 -- 172.21.15.33:0/2377792590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a99d80 msgr2=0x559166aa6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a99d80 0x559166aa6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39d6ffd640 1 -- 172.21.15.33:0/2377792590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a8ddf0 msgr2=0x559166a98a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a8ddf0 0x559166a98a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39d6ffd640 1 -- 172.21.15.33:0/2377792590 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559166a951c0 con 0x559166a97740 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f39d0000f80 tx=0x7f39d0040c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e75a10877681dd1d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39bcff9640 1 -- 172.21.15.33:0/2377792590 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0012070 con 0x559166a97740 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39bcff9640 1 -- 172.21.15.33:0/2377792590 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d0003e20 con 0x559166a97740 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.172+0000 7f39bcff9640 1 -- 172.21.15.33:0/2377792590 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d0045040 con 0x559166a97740 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.173+0000 7f39df96e740 1 -- 172.21.15.33:0/2377792590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 msgr2=0x559166aa5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.173+0000 7f39df96e740 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5cc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f39d0000f80 tx=0x7f39d0040c50 comp rx=0 tx=0).stop 2024-09-10T21:26:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.173+0000 7f39df96e740 1 -- 172.21.15.33:0/2377792590 shutdown_connections 2024-09-10T21:26:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.173+0000 7f39df96e740 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a99d80 0x559166aa6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.173+0000 7f39df96e740 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a97740 0x559166aa5cc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.173+0000 7f39df96e740 1 --2- 172.21.15.33:0/2377792590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a8ddf0 0x559166a98a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.173+0000 7f39df96e740 1 -- 172.21.15.33:0/2377792590 >> 172.21.15.33:0/2377792590 conn(0x559166a8b300 msgr2=0x559166a982f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.174+0000 7f39df96e740 1 -- 172.21.15.33:0/2377792590 shutdown_connections 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.174+0000 7f39df96e740 1 -- 172.21.15.33:0/2377792590 wait complete. 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.174+0000 7f39df96e740 1 -- start start 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.174+0000 7f39df96e740 1 Processor -- start 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39df96e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166aa1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39df96e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a97740 0x559166aa2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39df96e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a99d80 0x559166a9cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39d6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a97740 0x559166aa2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39d6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a97740 0x559166aa2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55644/0 (socket says 172.21.15.33:55644) 2024-09-10T21:26:30.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39d6ffd640 1 -- 172.21.15.33:0/3272462985 learned_addr learned my addr 172.21.15.33:0/3272462985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f39c400fa50 con 0x559166a97740 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f39c400f380 con 0x559166a99d80 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39d67fc640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166aa1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.175+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f39c400c540 con 0x559166a8ddf0 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39df96e740 10 client.0 _ll_register_callbacks cb 0x7f39cc012670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39d67fc640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a99d80 0x559166a9cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39d67fc640 1 -- 172.21.15.33:0/3272462985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a99d80 msgr2=0x559166a9cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39d67fc640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a99d80 0x559166a9cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39d67fc640 1 -- 172.21.15.33:0/3272462985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a97740 msgr2=0x559166aa2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39d67fc640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a97740 0x559166aa2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39d67fc640 1 -- 172.21.15.33:0/3272462985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559166a9d2f0 con 0x559166a8ddf0 2024-09-10T21:26:30.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.176+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a97740 0x559166aa2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.177+0000 7f39d67fc640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166aa1f10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39cc004730 tx=0x7f39cc021040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea08dc3a2f4518f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.177+0000 7f39baff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:26:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.177+0000 7f39baff5640 1 -- 172.21.15.33:0/3272462985 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39cc004920 con 0x559166a8ddf0 2024-09-10T21:26:30.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.177+0000 7f39baff5640 1 -- 172.21.15.33:0/3272462985 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f39cc004ac0 con 0x559166a8ddf0 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.177+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39d0003e20 con 0x559166a8ddf0 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.178+0000 7f39baff5640 1 -- 172.21.15.33:0/3272462985 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f39cc004d80 con 0x559166a8ddf0 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.178+0000 7f39df96e740 10 client.28606 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.178+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x559166b17430 con 0x559166a8ddf0 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.178+0000 7f39baff5640 1 -- 172.21.15.33:0/3272462985 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d0003e20 con 0x559166a8ddf0 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39baff5640 1 -- 172.21.15.33:0/3272462985 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f39cc01f900 con 0x559166a8ddf0 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39df96e740 10 client.28606 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39df96e740 10 client.28606 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39df96e740 20 client.28606 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39df96e740 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166b17b00 0x559166b19f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39df96e740 4 client.28606 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559166a018a0 con 0x559166b17b00 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.179+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166b17b00 0x559166b19f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39d6ffd640 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166b17b00 0x559166b19f20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d4cf716b2b03f4b5 server_cookie=f98d9c2e19dec30f in_seq=0 out_seq=0 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39baff5640 10 client.28606 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39baff5640 1 -- 172.21.15.33:0/3272462985 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559166a018a0 con 0x559166b17b00 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39baff5640 10 client.28606 handle_command_reply: tid=0 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 client.28606 shutdown 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 msgr2=0x559166aa1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166aa1f10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f39cc004730 tx=0x7f39cc021040 comp rx=0 tx=0).stop 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 shutdown_connections 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x559166a99d80 0x559166a9cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x559166b17b00 0x559166b19f20 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x559166a97740 0x559166aa2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 --2- 172.21.15.33:0/3272462985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559166a8ddf0 0x559166aa1f10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 >> 172.21.15.33:0/3272462985 conn(0x559166a8b300 msgr2=0x559166a98b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.180+0000 7f39baff5640 0 client.28606 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:26:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.181+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 shutdown_connections 2024-09-10T21:26:30.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.181+0000 7f39df96e740 1 -- 172.21.15.33:0/3272462985 wait complete. 2024-09-10T21:26:30.187 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:26:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.182+0000 7f39df96e740 20 client.28606 trim_cache size 0 max 16384 2024-09-10T21:26:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39c00b5390 msgr2=0x7f39c00b7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39c00b5390 0x7f39c00b7850 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f39c800cf80 tx=0x7f39c80096c0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8113b40 msgr2=0x7f39d81be920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8113b40 0x7f39d81be920 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f39c800e970 tx=0x7f39c800ee30 comp rx=0 tx=0).stop 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 shutdown_connections 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f39c00b5390 0x7f39c00b7850 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39d8113b40 0x7f39d81be920 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f39d8075e60 0x7f39d810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 --2- 172.21.15.33:0/4114903612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39d8075540 0x7f39d810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.183+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 >> 172.21.15.33:0/4114903612 conn(0x7f39d80fe830 msgr2=0x7f39d80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.184+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 shutdown_connections 2024-09-10T21:26:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:30.184+0000 7f39dd599640 1 -- 172.21.15.33:0/4114903612 wait complete. 2024-09-10T21:26:31.032 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:26:31.082 INFO:tasks.workunit.client.0.smithi033.stdout: 10 422171 38.28 MB/sec execute 451 sec latency 108.105 ms 2024-09-10T21:26:31.400 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:31.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:31 smithi033 ceph-mon[30860]: pgmap v1022: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-10T21:26:31.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:31 smithi174 ceph-mon[35635]: pgmap v1022: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-10T21:26:31.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.947+0000 7f9d7dcec640 1 -- 172.21.15.33:0/2711427598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78102230 msgr2=0x7f9d7810c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:31.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.947+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/2711427598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78102230 0x7f9d7810c9a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009860 tx=0x7f9d6c02a340 comp rx=0 tx=0).stop 2024-09-10T21:26:31.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.947+0000 7f9d7dcec640 1 -- 172.21.15.33:0/2711427598 shutdown_connections 2024-09-10T21:26:31.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.947+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/2711427598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d7810d0e0 0x7f9d7810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.947+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/2711427598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78102230 0x7f9d7810c9a0 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009860 tx=0x7f9d6c02a340 comp rx=0 tx=0).stop 2024-09-10T21:26:31.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.947+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/2711427598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78101880 0x7f9d78101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.947+0000 7f9d7dcec640 1 -- 172.21.15.33:0/2711427598 >> 172.21.15.33:0/2711427598 conn(0x7f9d78076e60 msgr2=0x7f9d78077270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:31.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.948+0000 7f9d7dcec640 1 -- 172.21.15.33:0/2711427598 shutdown_connections 2024-09-10T21:26:31.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.948+0000 7f9d7dcec640 1 -- 172.21.15.33:0/2711427598 wait complete. 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 Processor -- start 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 -- start start 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78101880 0x7f9d78104300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d7810d0e0 0x7f9d78102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78102de0 0x7f9d78103260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78112d90 con 0x7f9d78102de0 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78112c10 con 0x7f9d78101880 2024-09-10T21:26:31.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.949+0000 7f9d7dcec640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d78112f10 con 0x7f9d7810d0e0 2024-09-10T21:26:31.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.950+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78101880 0x7f9d78104300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.951+0000 7f9d77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78102de0 0x7f9d78103260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.951+0000 7f9d77fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78102de0 0x7f9d78103260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55664/0 (socket says 172.21.15.33:55664) 2024-09-10T21:26:31.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.951+0000 7f9d77fff640 1 -- 172.21.15.33:0/4228683313 learned_addr learned my addr 172.21.15.33:0/4228683313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:31.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.953+0000 7f9d77fff640 1 -- 172.21.15.33:0/4228683313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d7810d0e0 msgr2=0x7f9d78102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:31.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.953+0000 7f9d77fff640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d7810d0e0 0x7f9d78102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.953+0000 7f9d77fff640 1 -- 172.21.15.33:0/4228683313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78101880 msgr2=0x7f9d78104300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:31.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.953+0000 7f9d77fff640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78101880 0x7f9d78104300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.953+0000 7f9d77fff640 1 -- 172.21.15.33:0/4228683313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d781ac1b0 con 0x7f9d78102de0 2024-09-10T21:26:31.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.954+0000 7f9d77fff640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78102de0 0x7f9d78103260 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000eb10 tx=0x7f9d7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:31.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.954+0000 7f9d74ff9640 1 -- 172.21.15.33:0/4228683313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70004b20 con 0x7f9d78102de0 2024-09-10T21:26:31.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.955+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d781ac4a0 con 0x7f9d78102de0 2024-09-10T21:26:31.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.955+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d781ac9b0 con 0x7f9d78102de0 2024-09-10T21:26:31.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.955+0000 7f9d74ff9640 1 -- 172.21.15.33:0/4228683313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d70004cc0 con 0x7f9d78102de0 2024-09-10T21:26:31.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.956+0000 7f9d74ff9640 1 -- 172.21.15.33:0/4228683313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70010720 con 0x7f9d78102de0 2024-09-10T21:26:31.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.959+0000 7f9d800c1740 1 Processor -- start 2024-09-10T21:26:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.960+0000 7f9d74ff9640 1 -- 172.21.15.33:0/4228683313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9d70005020 con 0x7f9d78102de0 2024-09-10T21:26:31.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d800c1740 1 -- start start 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b91064d0 0x55c7b91068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b9112840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b9112d80 0x55c7b9115170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d800c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c7b907cea0 con 0x55c7b91064d0 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d800c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c7b906c2d0 con 0x55c7b9112d80 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d800c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7b90700e0 con 0x55c7b9106df0 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d74ff9640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9d600b5300 0x7f9d600b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.961+0000 7f9d77fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b9112d80 0x55c7b9115170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b9112d80 0x55c7b9115170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58738/0 (socket says 172.21.15.33:58738) 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 -- 172.21.15.33:0/3899086120 learned_addr learned my addr 172.21.15.33:0/3899086120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:31.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d74ff9640 1 -- 172.21.15.33:0/4228683313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9d700e6140 con 0x7f9d78102de0 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b9112840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b91064d0 0x55c7b91068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d777fe640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9d600b5300 0x7f9d600b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 -- 172.21.15.33:0/3899086120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 msgr2=0x55c7b9112840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b9112840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 -- 172.21.15.33:0/3899086120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b91064d0 msgr2=0x55c7b91068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b91064d0 0x55c7b91068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 -- 172.21.15.33:0/3899086120 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b8dca650 con 0x55c7b9112d80 2024-09-10T21:26:31.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.962+0000 7f9d77fff640 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b9112d80 0x55c7b9115170 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d700ac600 tx=0x7f9d70078bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f812378887b2b78d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.963+0000 7f9d5dffb640 1 -- 172.21.15.33:0/3899086120 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d700ac250 con 0x55c7b9112d80 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.963+0000 7f9d5dffb640 1 -- 172.21.15.33:0/3899086120 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d700ac410 con 0x55c7b9112d80 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.963+0000 7f9d777fe640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9d600b5300 0x7f9d600b77c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d680096c0 tx=0x7f9d68009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.963+0000 7f9d5dffb640 1 -- 172.21.15.33:0/3899086120 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d700a8070 con 0x55c7b9112d80 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 -- 172.21.15.33:0/3899086120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b9112d80 msgr2=0x55c7b9115170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b9112d80 0x55c7b9115170 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d700ac600 tx=0x7f9d70078bb0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 -- 172.21.15.33:0/3899086120 shutdown_connections 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b9112d80 0x55c7b9115170 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b9112840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3899086120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b91064d0 0x55c7b91068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 -- 172.21.15.33:0/3899086120 >> 172.21.15.33:0/3899086120 conn(0x55c7b9104300 msgr2=0x55c7b91054a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 -- 172.21.15.33:0/3899086120 shutdown_connections 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.964+0000 7f9d800c1740 1 -- 172.21.15.33:0/3899086120 wait complete. 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 -- start start 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 Processor -- start 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b91064d0 0x55c7b91125b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b910fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b910ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c7b8ecd700 con 0x55c7b9112d80 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c7b9007160 con 0x55c7b91064d0 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7b9115a00 con 0x55c7b9106df0 2024-09-10T21:26:31.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d800c1740 10 client.0 _ll_register_callbacks cb 0x55c7b90209e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b91064d0 0x55c7b91125b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b91064d0 0x55c7b91125b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58750/0 (socket says 172.21.15.33:58750) 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.965+0000 7f9d777fe640 1 -- 172.21.15.33:0/3420995508 learned_addr learned my addr 172.21.15.33:0/3420995508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b910fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d77fff640 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b910ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d777fe640 1 -- 172.21.15.33:0/3420995508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 msgr2=0x55c7b910fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d777fe640 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b910fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d777fe640 1 -- 172.21.15.33:0/3420995508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 msgr2=0x55c7b910ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d777fe640 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b910ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d777fe640 1 -- 172.21.15.33:0/3420995508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b9034c90 con 0x55c7b91064d0 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d777fe640 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b91064d0 0x55c7b91125b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d68010f10 tx=0x7f9d680046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c772bd2960a427 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:31.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d56ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3420995508 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68023070 con 0x55c7b91064d0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3420995508 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9d6801f650 con 0x55c7b91064d0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.966+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d700ac410 con 0x55c7b91064d0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3420995508 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9d6802f050 con 0x55c7b91064d0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 10 client.28742 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x55c7b9121330 con 0x55c7b91064d0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3420995508 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d700ac410 con 0x55c7b91064d0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3420995508 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f9d6800f040 con 0x55c7b91064d0 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 10 client.28742 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 10 client.28742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:31.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 20 client.28742 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b9122850 0x55c7b9124c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 4 client.28742 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:31.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.967+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7b907edf0 con 0x55c7b9122850 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.968+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b9122850 0x55c7b9124c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.968+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b9122850 0x55c7b9124c60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef3c48010049dfd server_cookie=ff6f4e75cb56e842 in_seq=0 out_seq=0 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.968+0000 7f9d56ffd640 10 client.28742 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3420995508 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c7b907edf0 con 0x55c7b9122850 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d56ffd640 10 client.28742 handle_command_reply: tid=0 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 client.28742 shutdown 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b91064d0 msgr2=0x55c7b91125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b91064d0 0x55c7b91125b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d68010f10 tx=0x7f9d680046c0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 shutdown_connections 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b9122850 0x55c7b9124c60 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b910ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b910fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3420995508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b91064d0 0x55c7b91125b0 secure :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d68010f10 tx=0x7f9d680046c0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 >> 172.21.15.33:0/3420995508 conn(0x55c7b9104300 msgr2=0x55c7b9109e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.969+0000 7f9d56ffd640 0 client.28742 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.970+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 shutdown_connections 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.970+0000 7f9d800c1740 1 -- 172.21.15.33:0/3420995508 wait complete. 2024-09-10T21:26:31.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.970+0000 7f9d800c1740 20 client.28742 trim_cache size 0 max 16384 2024-09-10T21:26:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.990+0000 7f9d800c1740 1 Processor -- start 2024-09-10T21:26:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.990+0000 7f9d800c1740 1 -- start start 2024-09-10T21:26:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9106df0 0x55c7b910fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9112d80 0x55c7b910cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b910d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d800c1740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6400e490 con 0x55c7b9106df0 2024-09-10T21:26:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d800c1740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64010060 con 0x55c7b911ec90 2024-09-10T21:26:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d800c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6400ebc0 con 0x55c7b9112d80 2024-09-10T21:26:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9106df0 0x55c7b910fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9106df0 0x55c7b910fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55700/0 (socket says 172.21.15.33:55700) 2024-09-10T21:26:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 -- 172.21.15.33:0/2158418406 learned_addr learned my addr 172.21.15.33:0/2158418406 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:31.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d77fff640 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b910d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 -- 172.21.15.33:0/2158418406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9112d80 msgr2=0x55c7b910cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9112d80 0x55c7b910cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9112d80 0x55c7b910cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 -- 172.21.15.33:0/2158418406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 msgr2=0x55c7b910d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b910d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.991+0000 7f9d777fe640 1 -- 172.21.15.33:0/2158418406 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7b9106c20 con 0x55c7b9106df0 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.992+0000 7f9d777fe640 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9106df0 0x55c7b910fec0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800a010 tx=0x7f9d68027560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c5195c90f76916 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:31.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.992+0000 7f9d5d7fa640 1 -- 172.21.15.33:0/2158418406 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68037070 con 0x55c7b9106df0 2024-09-10T21:26:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.992+0000 7f9d5d7fa640 1 -- 172.21.15.33:0/2158418406 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d6801f650 con 0x55c7b9106df0 2024-09-10T21:26:31.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.993+0000 7f9d5d7fa640 1 -- 172.21.15.33:0/2158418406 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68023070 con 0x55c7b9106df0 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 -- 172.21.15.33:0/2158418406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9106df0 msgr2=0x55c7b910fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9106df0 0x55c7b910fec0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f9d6800a010 tx=0x7f9d68027560 comp rx=0 tx=0).stop 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 -- 172.21.15.33:0/2158418406 shutdown_connections 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b910d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9112d80 0x55c7b910cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 --2- 172.21.15.33:0/2158418406 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9106df0 0x55c7b910fec0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 -- 172.21.15.33:0/2158418406 >> 172.21.15.33:0/2158418406 conn(0x55c7b9104300 msgr2=0x55c7b910f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 -- 172.21.15.33:0/2158418406 shutdown_connections 2024-09-10T21:26:31.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.994+0000 7f9d800c1740 1 -- 172.21.15.33:0/2158418406 wait complete. 2024-09-10T21:26:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.995+0000 7f9d800c1740 1 -- start start 2024-09-10T21:26:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.995+0000 7f9d800c1740 1 Processor -- start 2024-09-10T21:26:31.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.996+0000 7f9d800c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b911af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b911af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d777fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b911af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54602/0 (socket says 172.21.15.33:54602) 2024-09-10T21:26:31.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d777fe640 1 -- 172.21.15.33:0/3786529521 learned_addr learned my addr 172.21.15.33:0/3786529521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b911b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b9115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b911b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6400f960 con 0x55c7b9112d80 2024-09-10T21:26:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.997+0000 7f9d77fff640 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b9115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:32.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.998+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6400f290 con 0x55c7b911ec90 2024-09-10T21:26:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.998+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6400c450 con 0x55c7b9106df0 2024-09-10T21:26:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.998+0000 7f9d777fe640 1 -- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 msgr2=0x55c7b9115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:32.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.998+0000 7f9d777fe640 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b9115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.998+0000 7f9d777fe640 1 -- 172.21.15.33:0/3786529521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 msgr2=0x55c7b911b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.998+0000 7f9d777fe640 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b911b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.998+0000 7f9d777fe640 1 -- 172.21.15.33:0/3786529521 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7b9116520 con 0x55c7b9106df0 2024-09-10T21:26:32.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.999+0000 7f9d800c1740 10 client.0 _ll_register_callbacks cb 0x7f9d68010bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.999+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d6801f650 con 0x55c7b9106df0 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.999+0000 7f9d777fe640 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b911af70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d68005a10 tx=0x7f9d68005b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c6fce960f30a82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.999+0000 7f9d56ffd640 10 client.28618 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.999+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3786529521 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d68023070 con 0x55c7b9106df0 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.999+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3786529521 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9d68027910 con 0x55c7b9106df0 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:31.999+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3786529521 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f9d68037040 con 0x55c7b9106df0 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 10 client.28618 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:32.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=87}) -- 0x55c7b9190030 con 0x55c7b9106df0 2024-09-10T21:26:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3786529521 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f9d6803a070 con 0x55c7b9106df0 2024-09-10T21:26:32.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 10 client.28618 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 10 client.28618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 20 client.28618 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b91906d0 0x55c7b9192af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 4 client.28618 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:26:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7b907a8a0 con 0x55c7b91906d0 2024-09-10T21:26:32.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.000+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b91906d0 0x55c7b9192af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:32.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.001+0000 7f9d76ffd640 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b91906d0 0x55c7b9192af0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b01011569d87084a server_cookie=b95f0598b557b490 in_seq=0 out_seq=0 2024-09-10T21:26:32.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.001+0000 7f9d56ffd640 10 client.28618 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:32.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.001+0000 7f9d56ffd640 1 -- 172.21.15.33:0/3786529521 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7b907a8a0 con 0x55c7b91906d0 2024-09-10T21:26:32.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.001+0000 7f9d56ffd640 10 client.28618 handle_command_reply: tid=0 2024-09-10T21:26:32.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.001+0000 7f9d800c1740 1 client.28618 shutdown 2024-09-10T21:26:32.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.004+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 msgr2=0x55c7b911af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:32.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.004+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b911af70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9d68005a10 tx=0x7f9d68005b30 comp rx=0 tx=0).stop 2024-09-10T21:26:32.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.004+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 shutdown_connections 2024-09-10T21:26:32.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.004+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c7b91906d0 0x55c7b9192af0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d56ffd640 0 client.28618 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:32.008 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:26:32.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c7b911ec90 0x55c7b9115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c7b9112d80 0x55c7b911b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d800c1740 1 --2- 172.21.15.33:0/3786529521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7b9106df0 0x55c7b911af70 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 >> 172.21.15.33:0/3786529521 conn(0x55c7b9104300 msgr2=0x55c7b910e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:32.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 shutdown_connections 2024-09-10T21:26:32.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d800c1740 1 -- 172.21.15.33:0/3786529521 wait complete. 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.005+0000 7f9d800c1740 20 client.28618 trim_cache size 0 max 16384 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.006+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9d600b5300 msgr2=0x7f9d600b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.006+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9d600b5300 0x7f9d600b77c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d680096c0 tx=0x7f9d68009260 comp rx=0 tx=0).stop 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.006+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78102de0 msgr2=0x7f9d78103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.006+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78102de0 0x7f9d78103260 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000eb10 tx=0x7f9d7000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.006+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 shutdown_connections 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.007+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9d600b5300 0x7f9d600b77c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.007+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d78102de0 0x7f9d78103260 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.007+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d7810d0e0 0x7f9d78102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.007+0000 7f9d7dcec640 1 --2- 172.21.15.33:0/4228683313 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9d78101880 0x7f9d78104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.007+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 >> 172.21.15.33:0/4228683313 conn(0x7f9d78076e60 msgr2=0x7f9d780fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.007+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 shutdown_connections 2024-09-10T21:26:32.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:32.007+0000 7f9d7dcec640 1 -- 172.21.15.33:0/4228683313 wait complete. 2024-09-10T21:26:32.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:31 smithi104 ceph-mon[35325]: pgmap v1022: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 14 MiB/s wr, 843 op/s 2024-09-10T21:26:32.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 422953 38.31 MB/sec execute 452 sec latency 76.080 ms 2024-09-10T21:26:33.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 423631 38.29 MB/sec execute 453 sec latency 92.397 ms 2024-09-10T21:26:33.752 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:26:33.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:26:33.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:33 smithi174 ceph-mon[35635]: pgmap v1023: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 758 op/s 2024-09-10T21:26:33.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:33 smithi033 ceph-mon[30860]: pgmap v1023: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 758 op/s 2024-09-10T21:26:34.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:33 smithi104 ceph-mon[35325]: pgmap v1023: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 13 MiB/s wr, 758 op/s 2024-09-10T21:26:34.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 424287 38.28 MB/sec execute 454 sec latency 100.693 ms 2024-09-10T21:26:34.093 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:35.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 424875 38.26 MB/sec execute 455 sec latency 109.522 ms 2024-09-10T21:26:35.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 -- 172.21.15.33:0/727535195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84108020 msgr2=0x7f7b8410e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/727535195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84108020 0x7f7b8410e820 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c009ca0 tx=0x7f7b7c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 -- 172.21.15.33:0/727535195 shutdown_connections 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/727535195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84108020 0x7f7b8410e820 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/727535195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b841076a0 0x7f7b84107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/727535195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b84101c90 0x7f7b84102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 -- 172.21.15.33:0/727535195 >> 172.21.15.33:0/727535195 conn(0x7f7b840fd700 msgr2=0x7f7b840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 -- 172.21.15.33:0/727535195 shutdown_connections 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.486+0000 7f7b8a96e640 1 -- 172.21.15.33:0/727535195 wait complete. 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.487+0000 7f7b8a96e640 1 Processor -- start 2024-09-10T21:26:35.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.487+0000 7f7b8a96e640 1 -- start start 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b8a96e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101c90 0x7f7b84199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b8a96e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b841076a0 0x7f7b84199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101c90 0x7f7b84199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101c90 0x7f7b84199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55726/0 (socket says 172.21.15.33:55726) 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b83fff640 1 -- 172.21.15.33:0/2345313051 learned_addr learned my addr 172.21.15.33:0/2345313051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b84108020 0x7f7b8419eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111ca0 con 0x7f7b84101c90 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111b20 con 0x7f7b84108020 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b84111e20 con 0x7f7b841076a0 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.488+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b84108020 0x7f7b8419eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b841076a0 0x7f7b84199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b83fff640 1 -- 172.21.15.33:0/2345313051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b841076a0 msgr2=0x7f7b84199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b83fff640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b841076a0 0x7f7b84199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b83fff640 1 -- 172.21.15.33:0/2345313051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b84108020 msgr2=0x7f7b8419eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b83fff640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b84108020 0x7f7b8419eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b83fff640 1 -- 172.21.15.33:0/2345313051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b8419f300 con 0x7f7b84101c90 2024-09-10T21:26:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b84108020 0x7f7b8419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b83fff640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101c90 0x7f7b84199250 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400e9f0 tx=0x7f7b7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.489+0000 7f7b81ffb640 1 -- 172.21.15.33:0/2345313051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400cd80 con 0x7f7b84101c90 2024-09-10T21:26:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.490+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b8419f5f0 con 0x7f7b84101c90 2024-09-10T21:26:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.490+0000 7f7b81ffb640 1 -- 172.21.15.33:0/2345313051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b740108e0 con 0x7f7b84101c90 2024-09-10T21:26:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.490+0000 7f7b81ffb640 1 -- 172.21.15.33:0/2345313051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74018980 con 0x7f7b84101c90 2024-09-10T21:26:35.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.490+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b841a5e80 con 0x7f7b84101c90 2024-09-10T21:26:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.494+0000 7f7b8cd43740 1 Processor -- start 2024-09-10T21:26:35.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.494+0000 7f7b8cd43740 1 -- start start 2024-09-10T21:26:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b81ffb640 1 -- 172.21.15.33:0/2345313051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7b74010430 con 0x7f7b84101c90 2024-09-10T21:26:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363368d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x5643633786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x5643633825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5643632e1e60 con 0x564363378c00 2024-09-10T21:26:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5643632d1290 con 0x564363368960 2024-09-10T21:26:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643632d50a0 con 0x564363378280 2024-09-10T21:26:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b88f0b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363368d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b88f0b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363368d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58784/0 (socket says 172.21.15.33:58784) 2024-09-10T21:26:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b88f0b640 1 -- 172.21.15.33:0/3138188213 learned_addr learned my addr 172.21.15.33:0/3138188213 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x5643633825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.496+0000 7f7b83fff640 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x5643633786c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.497+0000 7f7b7bfff640 1 -- 172.21.15.33:0/3138188213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 msgr2=0x5643633786c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.497+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x5643633786c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.497+0000 7f7b7bfff640 1 -- 172.21.15.33:0/3138188213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 msgr2=0x564363368d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.497+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363368d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.497+0000 7f7b7bfff640 1 -- 172.21.15.33:0/3138188213 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56436302f930 con 0x564363378c00 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.497+0000 7f7b83fff640 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x5643633786c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.497+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363368d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:35.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.498+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x5643633825f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009b40 tx=0x7f7b6c0113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876228b75d37c04f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.498+0000 7f7b81ffb640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7b700b5360 0x7f7b700b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.498+0000 7f7b7a7fc640 1 -- 172.21.15.33:0/3138188213 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c011e10 con 0x564363378c00 2024-09-10T21:26:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.498+0000 7f7b7a7fc640 1 -- 172.21.15.33:0/3138188213 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c012900 con 0x564363378c00 2024-09-10T21:26:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.498+0000 7f7b7a7fc640 1 -- 172.21.15.33:0/3138188213 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c00fa50 con 0x564363378c00 2024-09-10T21:26:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.498+0000 7f7b81ffb640 1 -- 172.21.15.33:0/2345313051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7b740e6010 con 0x7f7b84101c90 2024-09-10T21:26:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7b700b5360 0x7f7b700b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3138188213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 msgr2=0x5643633825f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x5643633825f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c009b40 tx=0x7f7b6c0113e0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3138188213 shutdown_connections 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x5643633825f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x5643633786c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3138188213 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363368d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3138188213 >> 172.21.15.33:0/3138188213 conn(0x56436336ae80 msgr2=0x56436336b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7b700b5360 0x7f7b700b7820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c02f500 tx=0x7f7b7c02f310 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.499+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3138188213 shutdown_connections 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.500+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3138188213 wait complete. 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.500+0000 7f7b8cd43740 1 -- start start 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.500+0000 7f7b8cd43740 1 Processor -- start 2024-09-10T21:26:35.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363368960 0x56436337faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x564363380030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378c00 0x564363380570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x564363132890 con 0x564363368960 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56436326bd40 con 0x564363378c00 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564363382ce0 con 0x564363378280 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363368960 0x56436337faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363368960 0x56436337faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55732/0 (socket says 172.21.15.33:55732) 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b83fff640 1 -- 172.21.15.33:0/3276425681 learned_addr learned my addr 172.21.15.33:0/3276425681 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b8cd43740 10 client.0 _ll_register_callbacks cb 0x5643632cfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378c00 0x564363380570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.501+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x564363380030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b88f0b640 1 -- 172.21.15.33:0/3276425681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 msgr2=0x564363380030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x564363380030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b88f0b640 1 -- 172.21.15.33:0/3276425681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363368960 msgr2=0x56436337faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363368960 0x56436337faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b88f0b640 1 -- 172.21.15.33:0/3276425681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56436338c0e0 con 0x564363378c00 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378c00 0x564363380570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c02ade0 tx=0x7f7b7c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db48f0923633f345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b637fe640 1 -- 172.21.15.33:0/3276425681 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7c004250 con 0x564363378c00 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b6c012900 con 0x564363378c00 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.502+0000 7f7b637fe640 1 -- 172.21.15.33:0/3276425681 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7b7c0043f0 con 0x564363378c00 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b637fe640 1 -- 172.21.15.33:0/3276425681 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f7b7c0045f0 con 0x564363378c00 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b8cd43740 10 client.28760 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x56436338ea90 con 0x564363378c00 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b637fe640 1 -- 172.21.15.33:0/3276425681 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b6c012900 con 0x564363378c00 2024-09-10T21:26:35.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b637fe640 1 -- 172.21.15.33:0/3276425681 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f7b7c052070 con 0x564363378c00 2024-09-10T21:26:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b8cd43740 10 client.28760 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b8cd43740 10 client.28760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.503+0000 7f7b8cd43740 20 client.28760 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.504+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643633923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.504+0000 7f7b8cd43740 4 client.28760 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.504+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643632e3db0 con 0x56436338ffc0 2024-09-10T21:26:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.504+0000 7f7b83fff640 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643633923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.505+0000 7f7b83fff640 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643633923e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbbc9ad847c56070 server_cookie=a89d8272d74b06be in_seq=0 out_seq=0 2024-09-10T21:26:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.505+0000 7f7b637fe640 10 client.28760 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b637fe640 1 -- 172.21.15.33:0/3276425681 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5643632e3db0 con 0x56436338ffc0 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b637fe640 10 client.28760 handle_command_reply: tid=0 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 client.28760 shutdown 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378c00 msgr2=0x564363380570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378c00 0x564363380570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c02ade0 tx=0x7f7b7c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 shutdown_connections 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643633923e0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378c00 0x564363380570 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378280 0x564363380030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b637fe640 0 client.28760 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:35.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/3276425681 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363368960 0x56436337faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.506+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 >> 172.21.15.33:0/3276425681 conn(0x56436336ae80 msgr2=0x564363367f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.507+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 shutdown_connections 2024-09-10T21:26:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.507+0000 7f7b8cd43740 1 -- 172.21.15.33:0/3276425681 wait complete. 2024-09-10T21:26:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.507+0000 7f7b8cd43740 20 client.28760 trim_cache size 0 max 16384 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 Processor -- start 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 -- start start 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363368960 0x56436337a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378280 0x564363367750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x564363367c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841ac140 con 0x564363378c00 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841add10 con 0x564363378280 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841ac870 con 0x564363368960 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b88f0b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x564363367c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378280 0x564363367750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363368960 0x56436337a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b7bfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378280 0x564363367750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58796/0 (socket says 172.21.15.33:58796) 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b88f0b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x564363367c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55744/0 (socket says 172.21.15.33:55744) 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.535+0000 7f7b7bfff640 1 -- 172.21.15.33:0/12898482 learned_addr learned my addr 172.21.15.33:0/12898482 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:35.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b83fff640 1 -- 172.21.15.33:0/12898482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378280 msgr2=0x564363367750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b83fff640 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378280 0x564363367750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b83fff640 1 -- 172.21.15.33:0/12898482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 msgr2=0x564363367c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b83fff640 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x564363367c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b83fff640 1 -- 172.21.15.33:0/12898482 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643633690b0 con 0x564363368960 2024-09-10T21:26:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x564363367c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:35.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b83fff640 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363368960 0x56436337a910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b740ac720 tx=0x7f7b74018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278cc66d783c1005 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.536+0000 7f7b79ffb640 1 -- 172.21.15.33:0/12898482 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740cf040 con 0x564363368960 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b79ffb640 1 -- 172.21.15.33:0/12898482 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b740ac2a0 con 0x564363368960 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b79ffb640 1 -- 172.21.15.33:0/12898482 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740b5d20 con 0x564363368960 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 -- 172.21.15.33:0/12898482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363368960 msgr2=0x56436337a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363368960 0x56436337a910 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b740ac720 tx=0x7f7b74018420 comp rx=0 tx=0).stop 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 -- 172.21.15.33:0/12898482 shutdown_connections 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378c00 0x564363367c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363378280 0x564363367750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/12898482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363368960 0x56436337a910 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 -- 172.21.15.33:0/12898482 >> 172.21.15.33:0/12898482 conn(0x56436336ae80 msgr2=0x56436337e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.537+0000 7f7b8cd43740 1 -- 172.21.15.33:0/12898482 shutdown_connections 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.538+0000 7f7b8cd43740 1 -- 172.21.15.33:0/12898482 wait complete. 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.538+0000 7f7b8cd43740 1 -- start start 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.538+0000 7f7b8cd43740 1 Processor -- start 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.538+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363388170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.538+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378280 0x5643633886d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b8cd43740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378c00 0x564363396ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841ad610 con 0x564363378280 2024-09-10T21:26:35.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841acf40 con 0x564363368960 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b8cd43740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b841aa120 con 0x564363378c00 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b8cd43740 10 client.0 _ll_register_callbacks cb 0x7f7b7c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363388170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363388170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58806/0 (socket says 172.21.15.33:58806) 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 -- 172.21.15.33:0/2339003456 learned_addr learned my addr 172.21.15.33:0/2339003456 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b88f0b640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378c00 0x564363396ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 -- 172.21.15.33:0/2339003456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378c00 msgr2=0x564363396ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378280 0x5643633886d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378c00 0x564363396ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 -- 172.21.15.33:0/2339003456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378280 msgr2=0x5643633886d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378280 0x5643633886d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 -- 172.21.15.33:0/2339003456 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564363397470 con 0x564363368960 2024-09-10T21:26:35.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b83fff640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363388170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005c20 tx=0x7f7b74005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3508eaaae6eea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.539+0000 7f7b637fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.540+0000 7f7b637fe640 1 -- 172.21.15.33:0/2339003456 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b740cf070 con 0x564363368960 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.540+0000 7f7b637fe640 1 -- 172.21.15.33:0/2339003456 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7b740e90d0 con 0x564363368960 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.540+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378280 0x5643633886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.540+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b740ac2a0 con 0x564363368960 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.540+0000 7f7b637fe640 1 -- 172.21.15.33:0/2339003456 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f7b740ac790 con 0x564363368960 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.540+0000 7f7b8cd43740 10 client.28766 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:35.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.540+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x5643633987b0 con 0x564363368960 2024-09-10T21:26:35.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.541+0000 7f7b637fe640 1 -- 172.21.15.33:0/2339003456 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b740ac2a0 con 0x564363368960 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.541+0000 7f7b637fe640 1 -- 172.21.15.33:0/2339003456 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f7b74018b20 con 0x564363368960 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.541+0000 7f7b8cd43740 10 client.28766 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.542+0000 7f7b8cd43740 10 client.28766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.542+0000 7f7b8cd43740 20 client.28766 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.542+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643634001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.542+0000 7f7b8cd43740 4 client.28766 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.542+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5643632df860 con 0x56436338ffc0 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.542+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643634001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.544+0000 7f7b7bfff640 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643634001c0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10e9649751f0ed4b server_cookie=c4c2a0fe78a81f62 in_seq=0 out_seq=0 2024-09-10T21:26:35.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.544+0000 7f7b637fe640 10 client.28766 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:35.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.547+0000 7f7b637fe640 1 -- 172.21.15.33:0/2339003456 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5643632df860 con 0x56436338ffc0 2024-09-10T21:26:35.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.547+0000 7f7b637fe640 10 client.28766 handle_command_reply: tid=0 2024-09-10T21:26:35.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.547+0000 7f7b8cd43740 1 client.28766 shutdown 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 msgr2=0x564363388170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363388170 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7b74005c20 tx=0x7f7b74005c50 comp rx=0 tx=0).stop 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 shutdown_connections 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x56436338ffc0 0x5643634001c0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564363378c00 0x564363396ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x564363378280 0x5643633886d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 --2- 172.21.15.33:0/2339003456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x564363368960 0x564363388170 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 >> 172.21.15.33:0/2339003456 conn(0x56436336ae80 msgr2=0x564363367750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b637fe640 0 client.28766 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 shutdown_connections 2024-09-10T21:26:35.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 1 -- 172.21.15.33:0/2339003456 wait complete. 2024-09-10T21:26:35.552 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:26:35.552 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:26:35.552 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "05708340-c3c3-4430-9bae-81feded8d778", 2024-09-10T21:26:35.552 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:26:35.552 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.548+0000 7f7b8cd43740 20 client.28766 trim_cache size 0 max 16384 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7b700b5360 msgr2=0x7f7b700b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7b700b5360 0x7f7b700b7820 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c02f500 tx=0x7f7b7c02f310 comp rx=0 tx=0).stop 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101c90 msgr2=0x7f7b84199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101c90 0x7f7b84199250 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f7b7400e9f0 tx=0x7f7b7400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 shutdown_connections 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7b700b5360 0x7f7b700b7820 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b84108020 0x7f7b8419eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b841076a0 0x7f7b84199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 --2- 172.21.15.33:0/2345313051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b84101c90 0x7f7b84199250 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 >> 172.21.15.33:0/2345313051 conn(0x7f7b840fd700 msgr2=0x7f7b8406ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 shutdown_connections 2024-09-10T21:26:35.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:35.549+0000 7f7b8a96e640 1 -- 172.21.15.33:0/2345313051 wait complete. 2024-09-10T21:26:35.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:35 smithi033 ceph-mon[30860]: pgmap v1024: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 838 op/s 2024-09-10T21:26:35.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:35 smithi174 ceph-mon[35635]: pgmap v1024: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 838 op/s 2024-09-10T21:26:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:35 smithi104 ceph-mon[35325]: pgmap v1024: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 14 MiB/s wr, 838 op/s 2024-09-10T21:26:36.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 425311 38.22 MB/sec execute 456 sec latency 319.222 ms 2024-09-10T21:26:36.288 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:26:36.682 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:26:36.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:26:36.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:36.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:36.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:36 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:36.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:26:36.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:36.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:36.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:36.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:36.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:36 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:36 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 426007 38.21 MB/sec execute 457 sec latency 342.578 ms 2024-09-10T21:26:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2999479502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 msgr2=0x7f0aa410cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2999479502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 0x7f0aa410cab0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009900 tx=0x7f0a9802a410 comp rx=0 tx=0).stop 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2999479502 shutdown_connections 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2999479502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa410d100 0x7f0aa410f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2999479502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 0x7f0aa410cab0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2999479502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa4103ca0 0x7f0aa4104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2999479502 >> 172.21.15.33:0/2999479502 conn(0x7f0aa40fd700 msgr2=0x7f0aa40ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2999479502 shutdown_connections 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.234+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2999479502 wait complete. 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.235+0000 7f0aaa15a640 1 Processor -- start 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.235+0000 7f0aaa15a640 1 -- start start 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.235+0000 7f0aaa15a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa4103ca0 0x7f0aa410be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.235+0000 7f0aaa15a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 0x7f0aa410c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.235+0000 7f0aaa15a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa410d100 0x7f0aa419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aaa15a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4112db0 con 0x7f0aa41045c0 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aaa15a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4112c30 con 0x7f0aa410d100 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aaa15a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa4112f30 con 0x7f0aa4103ca0 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 0x7f0aa410c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa410d100 0x7f0aa419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa410d100 0x7f0aa419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58824/0 (socket says 172.21.15.33:58824) 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 -- 172.21.15.33:0/2559860598 learned_addr learned my addr 172.21.15.33:0/2559860598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 -- 172.21.15.33:0/2559860598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa4103ca0 msgr2=0x7f0aa410be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa4103ca0 0x7f0aa410be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 -- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 msgr2=0x7f0aa410c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 0x7f0aa410c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.236+0000 7f0aa3fff640 1 -- 172.21.15.33:0/2559860598 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa41a02d0 con 0x7f0aa410d100 2024-09-10T21:26:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.237+0000 7f0aa3fff640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa410d100 0x7f0aa419fb50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00eb10 tx=0x7f0a9c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.237+0000 7f0aa0ff9640 1 -- 172.21.15.33:0/2559860598 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c004b20 con 0x7f0aa410d100 2024-09-10T21:26:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.237+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa41a05c0 con 0x7f0aa410d100 2024-09-10T21:26:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.237+0000 7f0aa0ff9640 1 -- 172.21.15.33:0/2559860598 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a9c004cc0 con 0x7f0aa410d100 2024-09-10T21:26:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.237+0000 7f0aa0ff9640 1 -- 172.21.15.33:0/2559860598 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c010720 con 0x7f0aa410d100 2024-09-10T21:26:37.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.237+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa419cc20 con 0x7f0aa410d100 2024-09-10T21:26:37.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.240+0000 7f0aa0ff9640 1 -- 172.21.15.33:0/2559860598 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0a9c005020 con 0x7f0aa410d100 2024-09-10T21:26:37.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 Processor -- start 2024-09-10T21:26:37.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 -- start start 2024-09-10T21:26:37.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a71480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5622d39e7e50 con 0x5622d3a710a0 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5622d39d7280 con 0x5622d3a719c0 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.241+0000 7f0aac52f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622d39db090 con 0x5622d3a7d950 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a71480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a71480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55782/0 (socket says 172.21.15.33:55782) 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa2ffd640 1 -- 172.21.15.33:0/2739881823 learned_addr learned my addr 172.21.15.33:0/2739881823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa37fe640 1 -- 172.21.15.33:0/2739881823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 msgr2=0x5622d3a7fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa37fe640 1 -- 172.21.15.33:0/2739881823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 msgr2=0x5622d3a71480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a71480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa37fe640 1 -- 172.21.15.33:0/2739881823 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622d3735650 con 0x5622d3a719c0 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.242+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a71480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.243+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7d410 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009750 tx=0x7f0a94009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953a047fd315319d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.243+0000 7f0aa0ff9640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a8c0b5360 0x7f0a8c0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.243+0000 7f0a89ffb640 1 -- 172.21.15.33:0/2739881823 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94015020 con 0x5622d3a719c0 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.243+0000 7f0a89ffb640 1 -- 172.21.15.33:0/2739881823 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a94011420 con 0x5622d3a719c0 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.243+0000 7f0a89ffb640 1 -- 172.21.15.33:0/2739881823 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94009e60 con 0x5622d3a719c0 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aa0ff9640 1 -- 172.21.15.33:0/2559860598 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0a9c0e6210 con 0x7f0aa410d100 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a8c0b5360 0x7f0a8c0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 -- 172.21.15.33:0/2739881823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 msgr2=0x5622d3a7d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7d410 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0a94009750 tx=0x7f0a94009c10 comp rx=0 tx=0).stop 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a8c0b5360 0x7f0a8c0b7820 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0a980096c0 tx=0x7f0a98005780 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 -- 172.21.15.33:0/2739881823 shutdown_connections 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7d410 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 --2- 172.21.15.33:0/2739881823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a71480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 -- 172.21.15.33:0/2739881823 >> 172.21.15.33:0/2739881823 conn(0x5622d3a6eed0 msgr2=0x5622d3a70070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.244+0000 7f0aac52f740 1 -- 172.21.15.33:0/2739881823 shutdown_connections 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.245+0000 7f0aac52f740 1 -- 172.21.15.33:0/2739881823 wait complete. 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.245+0000 7f0aac52f740 1 -- start start 2024-09-10T21:26:37.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.245+0000 7f0aac52f740 1 Processor -- start 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.246+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a710a0 0x5622d3a7d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.246+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a719c0 0x5622d3a7a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.246+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a7acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.246+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a719c0 0x5622d3a7a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a719c0 0x5622d3a7a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54692/0 (socket says 172.21.15.33:54692) 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aac52f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5622d3838700 con 0x5622d3a7d950 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5622d3972110 con 0x5622d3a710a0 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa2ffd640 1 -- 172.21.15.33:0/3235762422 learned_addr learned my addr 172.21.15.33:0/3235762422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622d3a805d0 con 0x5622d3a719c0 2024-09-10T21:26:37.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aac52f740 10 client.0 _ll_register_callbacks cb 0x5622d398b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa3fff640 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a7acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a710a0 0x5622d3a7d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa37fe640 1 -- 172.21.15.33:0/3235762422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a719c0 msgr2=0x5622d3a7a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a719c0 0x5622d3a7a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa37fe640 1 -- 172.21.15.33:0/3235762422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 msgr2=0x5622d3a7acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a7acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa37fe640 1 -- 172.21.15.33:0/3235762422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622d399fc40 con 0x5622d3a710a0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.247+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a710a0 0x5622d3a7d230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0a94000c00 tx=0x7f0a94009490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aed307c286223f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.248+0000 7f0a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.248+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3235762422 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a940119e0 con 0x5622d3a710a0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.248+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a94011420 con 0x5622d3a710a0 2024-09-10T21:26:37.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.248+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3235762422 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0a9400b8d0 con 0x5622d3a710a0 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.248+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3235762422 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0a9400bb50 con 0x5622d3a710a0 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.249+0000 7f0aac52f740 10 client.28775 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.249+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x5622d3a8bf50 con 0x5622d3a710a0 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.249+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3235762422 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a9400bcf0 con 0x5622d3a710a0 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3235762422 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f0a9402b050 con 0x5622d3a710a0 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0aac52f740 10 client.28775 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0aac52f740 10 client.28775 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0aac52f740 20 client.28775 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3a8d2f0 0x5622d3a8f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0aac52f740 4 client.28775 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a94011420 con 0x5622d3a8d2f0 2024-09-10T21:26:37.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.250+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3a8d2f0 0x5622d3a8f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.251+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3a8d2f0 0x5622d3a8f6e0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95fa71cb82a26cb5 server_cookie=b46e9663beef6b2d in_seq=0 out_seq=0 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.252+0000 7f0a82ffd640 10 client.28775 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.253+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3235762422 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f0a94011420 con 0x5622d3a8d2f0 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.253+0000 7f0a82ffd640 10 client.28775 handle_command_reply: tid=0 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.253+0000 7f0aac52f740 1 client.28775 shutdown 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a710a0 msgr2=0x5622d3a7d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a710a0 0x5622d3a7d230 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0a94000c00 tx=0x7f0a94009490 comp rx=0 tx=0).stop 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 shutdown_connections 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3a8d2f0 0x5622d3a8f6e0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a7acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a719c0 0x5622d3a7a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3235762422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a710a0 0x5622d3a7d230 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 >> 172.21.15.33:0/3235762422 conn(0x5622d3a6eed0 msgr2=0x5622d3a74a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.254+0000 7f0a82ffd640 0 client.28775 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.255+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 shutdown_connections 2024-09-10T21:26:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.255+0000 7f0aac52f740 1 -- 172.21.15.33:0/3235762422 wait complete. 2024-09-10T21:26:37.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.255+0000 7f0aac52f740 20 client.28775 trim_cache size 0 max 16384 2024-09-10T21:26:37.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 Processor -- start 2024-09-10T21:26:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 -- start start 2024-09-10T21:26:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a89860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41bd460 con 0x5622d3a710a0 2024-09-10T21:26:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41bf030 con 0x5622d3a719c0 2024-09-10T21:26:37.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.278+0000 7f0aac52f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41bdb90 con 0x5622d3a7d950 2024-09-10T21:26:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a89860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a89860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55800/0 (socket says 172.21.15.33:55800) 2024-09-10T21:26:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 -- 172.21.15.33:0/284871907 learned_addr learned my addr 172.21.15.33:0/284871907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:58832/0 (socket says 172.21.15.33:58832) 2024-09-10T21:26:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa3fff640 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 -- 172.21.15.33:0/284871907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 msgr2=0x5622d3a7f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 -- 172.21.15.33:0/284871907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 msgr2=0x5622d3a7ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 -- 172.21.15.33:0/284871907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622d3a6f950 con 0x5622d3a710a0 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a89860 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f0a9401ae50 tx=0x7f0a9400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b73406ae94fb0e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.279+0000 7f0a897fa640 1 -- 172.21.15.33:0/284871907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94009e60 con 0x5622d3a710a0 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0a897fa640 1 -- 172.21.15.33:0/284871907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a940095a0 con 0x5622d3a710a0 2024-09-10T21:26:37.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0a897fa640 1 -- 172.21.15.33:0/284871907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94015040 con 0x5622d3a710a0 2024-09-10T21:26:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0aac52f740 1 -- 172.21.15.33:0/284871907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 msgr2=0x5622d3a89860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0aac52f740 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a89860 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f0a9401ae50 tx=0x7f0a9400b5e0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0aac52f740 1 -- 172.21.15.33:0/284871907 shutdown_connections 2024-09-10T21:26:37.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0aac52f740 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a7d950 0x5622d3a7f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0aac52f740 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a7ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0aac52f740 1 --2- 172.21.15.33:0/284871907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a710a0 0x5622d3a89860 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.280+0000 7f0aac52f740 1 -- 172.21.15.33:0/284871907 >> 172.21.15.33:0/284871907 conn(0x5622d3a6eed0 msgr2=0x5622d3a78f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.281+0000 7f0aac52f740 1 -- 172.21.15.33:0/284871907 shutdown_connections 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.281+0000 7f0aac52f740 1 -- 172.21.15.33:0/284871907 wait complete. 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.281+0000 7f0aac52f740 1 -- start start 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.281+0000 7f0aac52f740 1 Processor -- start 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a710a0 0x5622d3a85ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a80880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aac52f740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a80dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aac52f740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41be930 con 0x5622d3a7d950 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aac52f740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41be260 con 0x5622d3a719c0 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aac52f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa41bb420 con 0x5622d3a710a0 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aac52f740 10 client.0 _ll_register_callbacks cb 0x7f0a9c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:26:37.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a80880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a80dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a710a0 0x5622d3a85ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a710a0 0x5622d3a85ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54716/0 (socket says 172.21.15.33:54716) 2024-09-10T21:26:37.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa37fe640 1 -- 172.21.15.33:0/3305872006 learned_addr learned my addr 172.21.15.33:0/3305872006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa2ffd640 1 -- 172.21.15.33:0/3305872006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a710a0 msgr2=0x5622d3a85ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a710a0 0x5622d3a85ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa2ffd640 1 -- 172.21.15.33:0/3305872006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 msgr2=0x5622d3a80dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a80dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa2ffd640 1 -- 172.21.15.33:0/3305872006 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622d3a81460 con 0x5622d3a719c0 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a710a0 0x5622d3a85ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.282+0000 7f0aa3fff640 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a80dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:37.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0aa2ffd640 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a80880 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5622d3a8fd70 tx=0x7f0a9802a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2ceff741c90f822 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0a82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3305872006 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98046070 con 0x5622d3a719c0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a940095a0 con 0x5622d3a719c0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3305872006 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f0a98004450 con 0x5622d3a719c0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3305872006 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) ==== 24+0+0 (secure 0 0 0) 0x7f0a98035040 con 0x5622d3a719c0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0aac52f740 10 client.28781 fetch_fsmap learned FSMap version 87 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=4+}) -- 0x5622d3afaf60 con 0x5622d3a719c0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.283+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3305872006 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a940095a0 con 0x5622d3a719c0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.284+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3305872006 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 87) ==== 8378+0+0 (secure 0 0 0) 0x7f0a9805c070 con 0x5622d3a719c0 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.284+0000 7f0aac52f740 10 client.28781 fetch_fsmap finished waiting for FSMap version 87 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.284+0000 7f0aac52f740 10 client.28781 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.284+0000 7f0aac52f740 20 client.28781 populate_metadata read hostname 'smithi033' 2024-09-10T21:26:37.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.284+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3afb6c0 0x5622d3afdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.284+0000 7f0aac52f740 4 client.28781 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.284+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622d39e9da0 con 0x5622d3afb6c0 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.285+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3afb6c0 0x5622d3afdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.286+0000 7f0aa37fe640 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3afb6c0 0x5622d3afdae0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293d7cc6e2a09420 server_cookie=37b58db98201a090 in_seq=0 out_seq=0 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.286+0000 7f0a82ffd640 10 client.28781 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.288+0000 7f0a82ffd640 1 -- 172.21.15.33:0/3305872006 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8522 (crc 0 0 0) 0x5622d39e9da0 con 0x5622d3afb6c0 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.288+0000 7f0a82ffd640 10 client.28781 handle_command_reply: tid=0 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.288+0000 7f0aac52f740 1 client.28781 shutdown 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.288+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 msgr2=0x5622d3a80880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.288+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a80880 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x5622d3a8fd70 tx=0x7f0a9802a410 comp rx=0 tx=0).stop 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.288+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 shutdown_connections 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.288+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5622d3afb6c0 0x5622d3afdae0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5622d3a7d950 0x5622d3a80dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5622d3a719c0 0x5622d3a80880 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0aac52f740 1 --2- 172.21.15.33:0/3305872006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622d3a710a0 0x5622d3a85ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 >> 172.21.15.33:0/3305872006 conn(0x5622d3a6eed0 msgr2=0x5622d3a6df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0a82ffd640 0 client.28781 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:26:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 shutdown_connections 2024-09-10T21:26:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0aac52f740 1 -- 172.21.15.33:0/3305872006 wait complete. 2024-09-10T21:26:37.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.289+0000 7f0aac52f740 20 client.28781 trim_cache size 0 max 16384 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "05708340-c3c3-4430-9bae-81feded8d778": { 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "05708340-c3c3-4430-9bae-81feded8d778", 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:26:37.296 INFO:teuthology.orchestra.run.smithi033.stdout: "05708340-c3c3-4430-9bae-81feded8d778": { 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:26:35.545755+0000", 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.297 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:26:37.298 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:26:37.300 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.301 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.302 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:26:37.303 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.304 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:26:37.305 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:26:37.306 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.307 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.308 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.309 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.310 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:26:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.312 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:26:37.313 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:26:37.314 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a8c0b5360 msgr2=0x7f0a8c0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a8c0b5360 0x7f0a8c0b7820 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0a980096c0 tx=0x7f0a98005780 comp rx=0 tx=0).stop 2024-09-10T21:26:37.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa410d100 msgr2=0x7f0aa419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa410d100 0x7f0aa419fb50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00eb10 tx=0x7f0a9c00c2d0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 shutdown_connections 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f0a8c0b5360 0x7f0a8c0b7820 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0aa410d100 0x7f0aa419fb50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0aa41045c0 0x7f0aa410c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 --2- 172.21.15.33:0/2559860598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa4103ca0 0x7f0aa410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.290+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 >> 172.21.15.33:0/2559860598 conn(0x7f0aa40fd700 msgr2=0x7f0aa40fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.291+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 shutdown_connections 2024-09-10T21:26:37.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:26:37.291+0000 7f0aaa15a640 1 -- 172.21.15.33:0/2559860598 wait complete. 2024-09-10T21:26:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:26:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:37 smithi174 ceph-mon[35635]: pgmap v1025: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 761 op/s 2024-09-10T21:26:37.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:37 smithi033 ceph-mon[30860]: pgmap v1025: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 761 op/s 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:26:37.962 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:37 smithi104 ceph-mon[35325]: pgmap v1025: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 761 op/s 2024-09-10T21:26:38.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 426950 38.25 MB/sec execute 458 sec latency 93.388 ms 2024-09-10T21:26:38.092 INFO:tasks.cephfs.filesystem:scrub status for tag:05708340-c3c3-4430-9bae-81feded8d778 - {'path': '/', 'tag': '05708340-c3c3-4430-9bae-81feded8d778', 'options': 'recursive,force'} 2024-09-10T21:26:39.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 427717 38.27 MB/sec execute 459 sec latency 82.485 ms 2024-09-10T21:26:39.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:39 smithi174 ceph-mon[35635]: pgmap v1026: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.6 MiB/s wr, 725 op/s 2024-09-10T21:26:39.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:39 smithi033 ceph-mon[30860]: pgmap v1026: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.6 MiB/s wr, 725 op/s 2024-09-10T21:26:40.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:39 smithi104 ceph-mon[35325]: pgmap v1026: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 7.6 MiB/s wr, 725 op/s 2024-09-10T21:26:40.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 428261 38.25 MB/sec execute 460 sec latency 97.766 ms 2024-09-10T21:26:41.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 428813 38.22 MB/sec execute 461 sec latency 187.708 ms 2024-09-10T21:26:41.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:41 smithi174 ceph-mon[35635]: pgmap v1027: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 852 op/s 2024-09-10T21:26:41.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:41 smithi033 ceph-mon[30860]: pgmap v1027: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 852 op/s 2024-09-10T21:26:42.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:41 smithi104 ceph-mon[35325]: pgmap v1027: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.9 MiB/s wr, 852 op/s 2024-09-10T21:26:42.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 428933 38.16 MB/sec execute 462 sec latency 1069.762 ms 2024-09-10T21:26:43.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 429417 38.11 MB/sec execute 463 sec latency 117.035 ms 2024-09-10T21:26:43.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:43 smithi174 ceph-mon[35635]: pgmap v1028: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 597 op/s 2024-09-10T21:26:43.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:43 smithi033 ceph-mon[30860]: pgmap v1028: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 597 op/s 2024-09-10T21:26:44.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:43 smithi104 ceph-mon[35325]: pgmap v1028: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.6 MiB/s wr, 597 op/s 2024-09-10T21:26:44.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 430327 38.14 MB/sec execute 464 sec latency 52.327 ms 2024-09-10T21:26:45.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 431106 38.17 MB/sec execute 465 sec latency 133.263 ms 2024-09-10T21:26:45.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:45 smithi174 ceph-mon[35635]: pgmap v1029: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 834 op/s 2024-09-10T21:26:45.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:45 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:26:45.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:45 smithi033 ceph-mon[30860]: pgmap v1029: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 834 op/s 2024-09-10T21:26:45.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:45 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:26:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:45 smithi104 ceph-mon[35325]: pgmap v1029: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.5 MiB/s wr, 834 op/s 2024-09-10T21:26:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:45 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:26:46.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 431658 38.14 MB/sec execute 466 sec latency 288.038 ms 2024-09-10T21:26:46.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:46 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:26:47.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:46 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:26:47.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 432317 38.13 MB/sec execute 467 sec latency 90.309 ms 2024-09-10T21:26:47.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:46 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:26:47.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:47 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:26:47.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:47 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:26:47.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:47 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:26:47.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:47 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:26:47.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:47 smithi033 ceph-mon[30860]: pgmap v1030: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 818 op/s 2024-09-10T21:26:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:47 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:26:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:47 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:26:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:47 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:26:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:47 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:26:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:47 smithi104 ceph-mon[35325]: pgmap v1030: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 818 op/s 2024-09-10T21:26:48.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 432991 38.14 MB/sec execute 468 sec latency 109.358 ms 2024-09-10T21:26:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:47 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:26:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:47 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:26:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:47 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:26:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:47 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:26:48.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:47 smithi174 ceph-mon[35635]: pgmap v1030: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 10 MiB/s wr, 818 op/s 2024-09-10T21:26:49.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 433596 38.13 MB/sec execute 469 sec latency 126.078 ms 2024-09-10T21:26:49.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:49 smithi174 ceph-mon[35635]: pgmap v1031: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 783 op/s 2024-09-10T21:26:49.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:49 smithi033 ceph-mon[30860]: pgmap v1031: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 783 op/s 2024-09-10T21:26:49.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:49 smithi104 ceph-mon[35325]: pgmap v1031: 97 pgs: 97 active+clean; 1.2 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 783 op/s 2024-09-10T21:26:50.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 433987 38.08 MB/sec execute 470 sec latency 273.381 ms 2024-09-10T21:26:51.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 434734 38.09 MB/sec execute 471 sec latency 60.449 ms 2024-09-10T21:26:51.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:51 smithi104 ceph-mon[35325]: pgmap v1032: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 972 op/s 2024-09-10T21:26:51.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:26:51.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:26:51.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:26:51.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:26:51.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:51 smithi174 ceph-mon[35635]: pgmap v1032: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 972 op/s 2024-09-10T21:26:51.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:26:51.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:26:51.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:26:51.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:26:51.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:51 smithi033 ceph-mon[30860]: pgmap v1032: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 972 op/s 2024-09-10T21:26:51.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:26:51.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:26:51.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:26:51.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:26:52.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 435571 38.10 MB/sec execute 472 sec latency 136.504 ms 2024-09-10T21:26:53.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 436302 38.10 MB/sec execute 473 sec latency 49.322 ms 2024-09-10T21:26:53.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:53 smithi104 ceph-mon[35325]: pgmap v1033: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 760 op/s 2024-09-10T21:26:53.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:53 smithi174 ceph-mon[35635]: pgmap v1033: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 760 op/s 2024-09-10T21:26:53.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:53 smithi033 ceph-mon[30860]: pgmap v1033: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 14 MiB/s wr, 760 op/s 2024-09-10T21:26:54.083 INFO:tasks.workunit.client.0.smithi033.stdout: 10 436830 38.09 MB/sec execute 474 sec latency 75.329 ms 2024-09-10T21:26:55.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 437568 38.08 MB/sec execute 475 sec latency 88.826 ms 2024-09-10T21:26:55.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:55 smithi104 ceph-mon[35325]: pgmap v1034: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 899 op/s 2024-09-10T21:26:55.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:55 smithi174 ceph-mon[35635]: pgmap v1034: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 899 op/s 2024-09-10T21:26:55.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:55 smithi033 ceph-mon[30860]: pgmap v1034: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 899 op/s 2024-09-10T21:26:56.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 438252 38.08 MB/sec execute 476 sec latency 167.745 ms 2024-09-10T21:26:57.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 438954 38.10 MB/sec execute 477 sec latency 99.977 ms 2024-09-10T21:26:57.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:57 smithi104 ceph-mon[35325]: pgmap v1035: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 10 MiB/s wr, 731 op/s 2024-09-10T21:26:57.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:57 smithi174 ceph-mon[35635]: pgmap v1035: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 10 MiB/s wr, 731 op/s 2024-09-10T21:26:57.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:57 smithi033 ceph-mon[30860]: pgmap v1035: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 10 MiB/s wr, 731 op/s 2024-09-10T21:26:58.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 439238 38.06 MB/sec execute 478 sec latency 559.436 ms 2024-09-10T21:26:59.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 439600 38.00 MB/sec execute 479 sec latency 149.571 ms 2024-09-10T21:26:59.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:26:59 smithi104 ceph-mon[35325]: pgmap v1036: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 655 op/s 2024-09-10T21:26:59.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:26:59 smithi174 ceph-mon[35635]: pgmap v1036: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 655 op/s 2024-09-10T21:26:59.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:26:59 smithi033 ceph-mon[30860]: pgmap v1036: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 8.5 MiB/s wr, 655 op/s 2024-09-10T21:27:00.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 440244 37.99 MB/sec execute 480 sec latency 86.493 ms 2024-09-10T21:27:01.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 441017 37.98 MB/sec execute 481 sec latency 111.581 ms 2024-09-10T21:27:01.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:01 smithi104 ceph-mon[35325]: pgmap v1037: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-10T21:27:01.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:01 smithi174 ceph-mon[35635]: pgmap v1037: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-10T21:27:01.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:01 smithi033 ceph-mon[30860]: pgmap v1037: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 863 op/s 2024-09-10T21:27:02.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 441690 38.00 MB/sec execute 482 sec latency 293.343 ms 2024-09-10T21:27:03.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 442301 37.99 MB/sec execute 483 sec latency 109.935 ms 2024-09-10T21:27:03.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:03 smithi104 ceph-mon[35325]: pgmap v1038: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 626 op/s 2024-09-10T21:27:03.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:03 smithi174 ceph-mon[35635]: pgmap v1038: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 626 op/s 2024-09-10T21:27:03.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:03 smithi033 ceph-mon[30860]: pgmap v1038: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.1 MiB/s wr, 626 op/s 2024-09-10T21:27:04.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 442884 37.99 MB/sec execute 484 sec latency 159.711 ms 2024-09-10T21:27:05.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 443559 37.98 MB/sec execute 485 sec latency 107.168 ms 2024-09-10T21:27:05.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:05 smithi104 ceph-mon[35325]: pgmap v1039: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 790 op/s 2024-09-10T21:27:05.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:05 smithi174 ceph-mon[35635]: pgmap v1039: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 790 op/s 2024-09-10T21:27:05.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:05 smithi033 ceph-mon[30860]: pgmap v1039: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 11 MiB/s wr, 790 op/s 2024-09-10T21:27:06.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 444149 37.96 MB/sec execute 486 sec latency 126.301 ms 2024-09-10T21:27:07.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 444768 37.95 MB/sec execute 487 sec latency 98.638 ms 2024-09-10T21:27:07.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:07 smithi104 ceph-mon[35325]: pgmap v1040: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 732 op/s 2024-09-10T21:27:07.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:07 smithi174 ceph-mon[35635]: pgmap v1040: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 732 op/s 2024-09-10T21:27:07.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:07 smithi033 ceph-mon[30860]: pgmap v1040: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 10 MiB/s wr, 732 op/s 2024-09-10T21:27:08.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 445570 37.95 MB/sec execute 488 sec latency 81.040 ms 2024-09-10T21:27:08.093 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:27:08.419 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:08.703 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:08 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:08.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:08 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:09.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 -- 172.21.15.33:0/1288359238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c102d90 msgr2=0x7fb56c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 --2- 172.21.15.33:0/1288359238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c102d90 0x7fb56c103170 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb55c009900 tx=0x7fb55c02a410 comp rx=0 tx=0).stop 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 -- 172.21.15.33:0/1288359238 shutdown_connections 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 --2- 172.21.15.33:0/1288359238 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c109120 0x7fb56c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 --2- 172.21.15.33:0/1288359238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb56c1087a0 0x7fb56c108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 --2- 172.21.15.33:0/1288359238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c102d90 0x7fb56c103170 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 -- 172.21.15.33:0/1288359238 >> 172.21.15.33:0/1288359238 conn(0x7fb56c0fe820 msgr2=0x7fb56c100c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 -- 172.21.15.33:0/1288359238 shutdown_connections 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.998+0000 7fb5715a4640 1 -- 172.21.15.33:0/1288359238 wait complete. 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.999+0000 7fb5715a4640 1 Processor -- start 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.999+0000 7fb5715a4640 1 -- start start 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:08.999+0000 7fb5715a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb56c102d90 0x7fb56c0754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb5715a4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c1087a0 0x7fb56c0759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb5715a4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c109120 0x7fb56c075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb5715a4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c112de0 con 0x7fb56c109120 2024-09-10T21:27:09.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb5715a4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c112c60 con 0x7fb56c1087a0 2024-09-10T21:27:09.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb5715a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c112f60 con 0x7fb56c102d90 2024-09-10T21:27:09.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c1087a0 0x7fb56c0759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c1087a0 0x7fb56c0759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:46548/0 (socket says 172.21.15.33:46548) 2024-09-10T21:27:09.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 -- 172.21.15.33:0/350614978 learned_addr learned my addr 172.21.15.33:0/350614978 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:09.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c109120 0x7fb56c075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb56affd640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb56c102d90 0x7fb56c0754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 -- 172.21.15.33:0/350614978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb56c102d90 msgr2=0x7fb56c0754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb56c102d90 0x7fb56c0754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 -- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c109120 msgr2=0x7fb56c075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c109120 0x7fb56c075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb563fff640 1 -- 172.21.15.33:0/350614978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb56c1a7690 con 0x7fb56c1087a0 2024-09-10T21:27:09.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.000+0000 7fb56affd640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb56c102d90 0x7fb56c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:09.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.001+0000 7fb563fff640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c1087a0 0x7fb56c0759e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb55400e9f0 tx=0x7fb55400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.001+0000 7fb568ff9640 1 -- 172.21.15.33:0/350614978 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55400cd80 con 0x7fb56c1087a0 2024-09-10T21:27:09.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.001+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb56c1a7980 con 0x7fb56c1087a0 2024-09-10T21:27:09.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.001+0000 7fb568ff9640 1 -- 172.21.15.33:0/350614978 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5540054d0 con 0x7fb56c1087a0 2024-09-10T21:27:09.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.001+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb56c1a7ec0 con 0x7fb56c1087a0 2024-09-10T21:27:09.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.001+0000 7fb568ff9640 1 -- 172.21.15.33:0/350614978 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb554010690 con 0x7fb56c1087a0 2024-09-10T21:27:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 Processor -- start 2024-09-10T21:27:09.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 -- start start 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd045d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd0469d30 0x562dd046c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562dd03d3e50 con 0x562dd0469d30 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562dd03c3280 con 0x562dd045d480 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.004+0000 7fb573979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562dd03c7090 con 0x562dd045dda0 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd045d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd045d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:46562/0 (socket says 172.21.15.33:46562) 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 -- 172.21.15.33:0/3197205959 learned_addr learned my addr 172.21.15.33:0/3197205959 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56affd640 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 -- 172.21.15.33:0/3197205959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 msgr2=0x562dd04697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:09.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 -- 172.21.15.33:0/3197205959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd0469d30 msgr2=0x562dd046c120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd0469d30 0x562dd046c120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 -- 172.21.15.33:0/3197205959 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd0121650 con 0x562dd045d480 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.005+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd045d860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb564009f90 tx=0x7fb5640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a17e19f235dee27 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.006+0000 7fb5617fa640 1 -- 172.21.15.33:0/3197205959 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564004d20 con 0x562dd045d480 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.006+0000 7fb5617fa640 1 -- 172.21.15.33:0/3197205959 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb564004050 con 0x562dd045d480 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.006+0000 7fb5617fa640 1 -- 172.21.15.33:0/3197205959 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb564005220 con 0x562dd045d480 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.006+0000 7fb573979740 1 -- 172.21.15.33:0/3197205959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 msgr2=0x562dd045d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.006+0000 7fb573979740 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd045d860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb564009f90 tx=0x7fb5640118d0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.006+0000 7fb568ff9640 1 -- 172.21.15.33:0/350614978 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb554004e90 con 0x7fb56c1087a0 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 -- 172.21.15.33:0/3197205959 shutdown_connections 2024-09-10T21:27:09.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd0469d30 0x562dd046c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 --2- 172.21.15.33:0/3197205959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd045d860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 -- 172.21.15.33:0/3197205959 >> 172.21.15.33:0/3197205959 conn(0x562dd045b2b0 msgr2=0x562dd045c450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 -- 172.21.15.33:0/3197205959 shutdown_connections 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 -- 172.21.15.33:0/3197205959 wait complete. 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 -- start start 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb573979740 1 Processor -- start 2024-09-10T21:27:09.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.007+0000 7fb568ff9640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5580b51f0 0x7fb5580b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb573979740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd0469610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb573979740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045dda0 0x562dd0466a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb568ff9640 1 -- 172.21.15.33:0/350614978 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb5540e5ff0 con 0x7fb56c1087a0 2024-09-10T21:27:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb573979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd0467090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb573979740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562dd0224700 con 0x562dd045dda0 2024-09-10T21:27:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb573979740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562dd035e110 con 0x562dd045d480 2024-09-10T21:27:09.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb573979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562dd046ca00 con 0x562dd0469d30 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb573979740 10 client.0 _ll_register_callbacks cb 0x562dd0377990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5580b51f0 0x7fb5580b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd0469610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd0469610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:46570/0 (socket says 172.21.15.33:46570) 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 -- 172.21.15.33:0/2625613613 learned_addr learned my addr 172.21.15.33:0/2625613613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 -- 172.21.15.33:0/2625613613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 msgr2=0x562dd0467090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd0467090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd0467090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 -- 172.21.15.33:0/2625613613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045dda0 msgr2=0x562dd0466a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045dda0 0x562dd0466a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.008+0000 7fb56affd640 1 -- 172.21.15.33:0/2625613613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dd038bc40 con 0x562dd045d480 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb56affd640 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd0469610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb55c02a930 tx=0x7fb55c031730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3a1cac23789efd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb5527fc640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5580b51f0 0x7fb5580b76b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb564009da0 tx=0x7fb564005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb5527fc640 1 -- 172.21.15.33:0/2625613613 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c035070 con 0x562dd045d480 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb5527fc640 1 -- 172.21.15.33:0/2625613613 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb55c004240 con 0x562dd045d480 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb564004050 con 0x562dd045d480 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb5527fc640 1 -- 172.21.15.33:0/2625613613 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb55c005d50 con 0x562dd045d480 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb573979740 10 client.28793 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.009+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x562dd0478320 con 0x562dd045d480 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb5527fc640 1 -- 172.21.15.33:0/2625613613 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb564004050 con 0x562dd045d480 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb5527fc640 1 -- 172.21.15.33:0/2625613613 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7fb55c004a60 con 0x562dd045d480 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb573979740 10 client.28793 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb573979740 10 client.28793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb573979740 20 client.28793 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb573979740 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd047bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb573979740 4 client.28793 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dd03d5da0 con 0x562dd04797e0 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.010+0000 7fb563fff640 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd047bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.011+0000 7fb5527fc640 10 client.28793 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.011+0000 7fb563fff640 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd047bbf0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da58519bcf4ddb8d server_cookie=9c10763f0946d6d6 in_seq=0 out_seq=0 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.012+0000 7fb5527fc640 1 -- 172.21.15.33:0/2625613613 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562dd03d5da0 con 0x562dd04797e0 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.012+0000 7fb5527fc640 10 client.28793 handle_command_reply: tid=0 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.012+0000 7fb573979740 1 client.28793 shutdown 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.012+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 msgr2=0x562dd0469610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.012+0000 7fb573979740 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd0469610 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb55c02a930 tx=0x7fb55c031730 comp rx=0 tx=0).stop 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 shutdown_connections 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd047bbf0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb5527fc640 0 client.28793 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:09.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd0467090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045dda0 0x562dd0466a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 --2- 172.21.15.33:0/2625613613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045d480 0x562dd0469610 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 >> 172.21.15.33:0/2625613613 conn(0x562dd045b2b0 msgr2=0x562dd0460e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 shutdown_connections 2024-09-10T21:27:09.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 1 -- 172.21.15.33:0/2625613613 wait complete. 2024-09-10T21:27:09.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.013+0000 7fb573979740 20 client.28793 trim_cache size 0 max 16384 2024-09-10T21:27:09.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 Processor -- start 2024-09-10T21:27:09.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 -- start start 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0475c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045dda0 0x562dd04562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd04652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b05f0 con 0x562dd045d480 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b2190 con 0x562dd045dda0 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb573979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b0cc0 con 0x562dd0469d30 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb563fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045dda0 0x562dd04562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb56affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0475c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb563fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045dda0 0x562dd04562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:46582/0 (socket says 172.21.15.33:46582) 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb56affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0475c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42274/0 (socket says 172.21.15.33:42274) 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.035+0000 7fb563fff640 1 -- 172.21.15.33:0/4173132857 learned_addr learned my addr 172.21.15.33:0/4173132857 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd04652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb56affd640 1 -- 172.21.15.33:0/4173132857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 msgr2=0x562dd04652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb56affd640 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd04652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb56affd640 1 -- 172.21.15.33:0/4173132857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045dda0 msgr2=0x562dd04562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb56affd640 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045dda0 0x562dd04562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb56affd640 1 -- 172.21.15.33:0/4173132857 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd0464f60 con 0x562dd045d480 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb563fff640 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045dda0 0x562dd04562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb56affd640 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0475c30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005940 tx=0x7fb55c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af31d2c4a3443b32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.036+0000 7fb560ff9640 1 -- 172.21.15.33:0/4173132857 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c035030 con 0x562dd045d480 2024-09-10T21:27:09.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.037+0000 7fb560ff9640 1 -- 172.21.15.33:0/4173132857 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb55c003890 con 0x562dd045d480 2024-09-10T21:27:09.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.037+0000 7fb560ff9640 1 -- 172.21.15.33:0/4173132857 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb55c031920 con 0x562dd045d480 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 -- 172.21.15.33:0/4173132857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 msgr2=0x562dd0475c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0475c30 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fb55c005940 tx=0x7fb55c005b50 comp rx=0 tx=0).stop 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 -- 172.21.15.33:0/4173132857 shutdown_connections 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd0469d30 0x562dd04652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd045dda0 0x562dd04562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 --2- 172.21.15.33:0/4173132857 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0475c30 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 -- 172.21.15.33:0/4173132857 >> 172.21.15.33:0/4173132857 conn(0x562dd045b2b0 msgr2=0x562dd045c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 -- 172.21.15.33:0/4173132857 shutdown_connections 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 -- 172.21.15.33:0/4173132857 wait complete. 2024-09-10T21:27:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 -- start start 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.038+0000 7fb573979740 1 Processor -- start 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb573979740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0471ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb573979740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04721e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb573979740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd0469d30 0x562dd046cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb573979740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b1a90 con 0x562dd045d480 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb573979740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1b1390 con 0x562dd0469d30 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb573979740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb56c1ae5b0 con 0x562dd045dda0 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb573979740 10 client.0 _ll_register_callbacks cb 0x7fb5640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb563fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb563fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04721e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:36432/0 (socket says 172.21.15.33:36432) 2024-09-10T21:27:09.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb563fff640 1 -- 172.21.15.33:0/1475165087 learned_addr learned my addr 172.21.15.33:0/1475165087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb56b7fe640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd0469d30 0x562dd046cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.039+0000 7fb56affd640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0471ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb563fff640 1 -- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd0469d30 msgr2=0x562dd046cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb563fff640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd0469d30 0x562dd046cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb563fff640 1 -- 172.21.15.33:0/1475165087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 msgr2=0x562dd0471ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb563fff640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0471ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb563fff640 1 -- 172.21.15.33:0/1475165087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dd046d190 con 0x562dd045dda0 2024-09-10T21:27:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb56affd640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0471ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb563fff640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04721e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb5540ac7e0 tx=0x7fb554004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4f6dd6e02af08cb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb5527fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:27:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.040+0000 7fb5527fc640 1 -- 172.21.15.33:0/1475165087 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5540d3070 con 0x562dd045dda0 2024-09-10T21:27:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.041+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb55c003890 con 0x562dd045dda0 2024-09-10T21:27:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.041+0000 7fb5527fc640 1 -- 172.21.15.33:0/1475165087 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb5540a8ac0 con 0x562dd045dda0 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.041+0000 7fb5527fc640 1 -- 172.21.15.33:0/1475165087 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb5540abfc0 con 0x562dd045dda0 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.041+0000 7fb573979740 10 client.28666 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.041+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x562dd04ef7d0 con 0x562dd045dda0 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.041+0000 7fb5527fc640 1 -- 172.21.15.33:0/1475165087 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb55c003890 con 0x562dd045dda0 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.041+0000 7fb5527fc640 1 -- 172.21.15.33:0/1475165087 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7fb5540b5950 con 0x562dd045dda0 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb573979740 10 client.28666 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb573979740 10 client.28666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb573979740 20 client.28666 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb573979740 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd04f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb573979740 4 client.28666 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562dd03d1850 con 0x562dd04797e0 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb56affd640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd04f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb5527fc640 10 client.28666 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.042+0000 7fb56affd640 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd04f1fa0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8ea5da401b1f9b server_cookie=64c574212b822426 in_seq=0 out_seq=0 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb5527fc640 1 -- 172.21.15.33:0/1475165087 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8326 (crc 0 0 0) 0x562dd03d1850 con 0x562dd04797e0 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb5527fc640 10 client.28666 handle_command_reply: tid=0 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 client.28666 shutdown 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 msgr2=0x562dd04721e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04721e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb5540ac7e0 tx=0x7fb554004a90 comp rx=0 tx=0).stop 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 shutdown_connections 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x562dd04797e0 0x562dd04f1fa0 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb5527fc640 0 client.28666 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:09.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562dd0469d30 0x562dd046cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dd045dda0 0x562dd04721e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.043+0000 7fb573979740 1 --2- 172.21.15.33:0/1475165087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562dd045d480 0x562dd0471ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.044+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 >> 172.21.15.33:0/1475165087 conn(0x562dd045b2b0 msgr2=0x562dd045c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.044+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 shutdown_connections 2024-09-10T21:27:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.044+0000 7fb573979740 1 -- 172.21.15.33:0/1475165087 wait complete. 2024-09-10T21:27:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.044+0000 7fb573979740 20 client.28666 trim_cache size 0 max 16384 2024-09-10T21:27:09.050 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "05708340-c3c3-4430-9bae-81feded8d778": { 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:26:35.545755+0000", 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.051 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.053 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.056 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.058 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.061 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:27:09.063 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.065 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:27:09.066 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.067 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:09.068 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.044+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5580b51f0 msgr2=0x7fb5580b76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.044+0000 7fb5715a4640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5580b51f0 0x7fb5580b76b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb564009da0 tx=0x7fb564005bd0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c1087a0 msgr2=0x7fb56c0759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c1087a0 0x7fb56c0759e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fb55400e9f0 tx=0x7fb55400edf0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 shutdown_connections 2024-09-10T21:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5580b51f0 0x7fb5580b76b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c109120 0x7fb56c075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb56c1087a0 0x7fb56c0759e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 --2- 172.21.15.33:0/350614978 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb56c102d90 0x7fb56c0754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.045+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 >> 172.21.15.33:0/350614978 conn(0x7fb56c0fe820 msgr2=0x7fb56c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.046+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 shutdown_connections 2024-09-10T21:27:09.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:09.046+0000 7fb5715a4640 1 -- 172.21.15.33:0/350614978 wait complete. 2024-09-10T21:27:09.070 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:08 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:09.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 446455 37.98 MB/sec execute 489 sec latency 94.800 ms 2024-09-10T21:27:09.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:09 smithi174 ceph-mon[35635]: pgmap v1041: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.6 MiB/s wr, 659 op/s 2024-09-10T21:27:09.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:09 smithi033 ceph-mon[30860]: pgmap v1041: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.6 MiB/s wr, 659 op/s 2024-09-10T21:27:10.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:09 smithi104 ceph-mon[35325]: pgmap v1041: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 9.6 MiB/s wr, 659 op/s 2024-09-10T21:27:10.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 447121 37.98 MB/sec execute 490 sec latency 73.963 ms 2024-09-10T21:27:10.094 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:27:10.095 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:27:10.639 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:11.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 447249 37.92 MB/sec execute 491 sec latency 470.800 ms 2024-09-10T21:27:11.766 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:11 smithi033 ceph-mon[30860]: pgmap v1042: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 823 op/s 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 -- 172.21.15.33:0/3471845775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578103b30 msgr2=0x7fc578103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/3471845775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578103b30 0x7fc578103f10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fc568009900 tx=0x7fc56802a410 comp rx=0 tx=0).stop 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 -- 172.21.15.33:0/3471845775 shutdown_connections 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/3471845775 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc57810d0e0 0x7fc57810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/3471845775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578104530 0x7fc57810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/3471845775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc578103b30 0x7fc578103f10 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 -- 172.21.15.33:0/3471845775 >> 172.21.15.33:0/3471845775 conn(0x7fc5780fd700 msgr2=0x7fc5780ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 -- 172.21.15.33:0/3471845775 shutdown_connections 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.882+0000 7fc57d1d8640 1 -- 172.21.15.33:0/3471845775 wait complete. 2024-09-10T21:27:11.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.883+0000 7fc57d1d8640 1 Processor -- start 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.883+0000 7fc57d1d8640 1 -- start start 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57d1d8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57d1d8640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57d1d8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57d1d8640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc578112d60 con 0x7fc57810d0e0 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57d1d8640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc578112be0 con 0x7fc578104530 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57659c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53596/0 (socket says 172.21.15.33:53596) 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57659c640 1 -- 172.21.15.33:0/1973814919 learned_addr learned my addr 172.21.15.33:0/1973814919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57d1d8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc578112ee0 con 0x7fc578103b30 2024-09-10T21:27:11.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc57759e640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.884+0000 7fc576d9d640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57759e640 1 -- 172.21.15.33:0/1973814919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 msgr2=0x7fc5781019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57759e640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57759e640 1 -- 172.21.15.33:0/1973814919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 msgr2=0x7fc578101ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57759e640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57759e640 1 -- 172.21.15.33:0/1973814919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5781a76d0 con 0x7fc57810d0e0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57659c640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57759e640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc570009fd0 tx=0x7fc57000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc55ffff640 1 -- 172.21.15.33:0/1973814919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57000ce20 con 0x7fc57810d0e0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5781a79c0 con 0x7fc57810d0e0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc55ffff640 1 -- 172.21.15.33:0/1973814919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc570010430 con 0x7fc57810d0e0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc55ffff640 1 -- 172.21.15.33:0/1973814919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57000f620 con 0x7fc57810d0e0 2024-09-10T21:27:11.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.885+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5781a7ea0 con 0x7fc57810d0e0 2024-09-10T21:27:11.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.887+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc578103f10 con 0x7fc57810d0e0 2024-09-10T21:27:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:11 smithi174 ceph-mon[35635]: pgmap v1042: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 823 op/s 2024-09-10T21:27:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.889+0000 7fc55ffff640 1 -- 172.21.15.33:0/1973814919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc57001f020 con 0x7fc57810d0e0 2024-09-10T21:27:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.891+0000 7fc55ffff640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b52e0 0x7fc5600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.891+0000 7fc55ffff640 1 -- 172.21.15.33:0/1973814919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc5700e60a0 con 0x7fc57810d0e0 2024-09-10T21:27:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.891+0000 7fc576d9d640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b52e0 0x7fc5600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:11.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.892+0000 7fc576d9d640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b52e0 0x7fc5600b77a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc568031440 tx=0x7fc5680095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:11.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:11.895+0000 7fc55ffff640 1 -- 172.21.15.33:0/1973814919 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc5700acf60 con 0x7fc57810d0e0 2024-09-10T21:27:12.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:11 smithi104 ceph-mon[35325]: pgmap v1042: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 11 MiB/s wr, 823 op/s 2024-09-10T21:27:12.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 447728 37.89 MB/sec execute 492 sec latency 111.863 ms 2024-09-10T21:27:12.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.210+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc578102cb0 con 0x7fc57810d0e0 2024-09-10T21:27:12.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.210+0000 7fc55ffff640 1 -- 172.21.15.33:0/1973814919 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+11976 (secure 0 0 0) 0x7fc5700ac860 con 0x7fc57810d0e0 2024-09-10T21:27:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:27:12.213 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":89,"btime":"2024-09-10T21:27:07:545589+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":89,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:27:07.270493+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:27:12.213 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 89 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b52e0 msgr2=0x7fc5600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b52e0 0x7fc5600b77a0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc568031440 tx=0x7fc5680095f0 comp rx=0 tx=0).stop 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 msgr2=0x7fc578102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fc570009fd0 tx=0x7fc57000ef10 comp rx=0 tx=0).stop 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 shutdown_connections 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc5600b52e0 0x7fc5600b77a0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc57810d0e0 0x7fc578102430 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc578104530 0x7fc578101ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 --2- 172.21.15.33:0/1973814919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc578103b30 0x7fc5781019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.213+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 >> 172.21.15.33:0/1973814919 conn(0x7fc5780fd700 msgr2=0x7fc5780fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:12.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.214+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 shutdown_connections 2024-09-10T21:27:12.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:12.214+0000 7fc57d1d8640 1 -- 172.21.15.33:0/1973814919 wait complete. 2024-09-10T21:27:12.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:12 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1973814919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:27:12.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:12 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1973814919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:27:13.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:12 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1973814919' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:27:13.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 448610 37.90 MB/sec execute 493 sec latency 88.518 ms 2024-09-10T21:27:13.174 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:27:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:13.835 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:13 smithi033 ceph-mon[30860]: pgmap v1043: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.4 MiB/s wr, 592 op/s 2024-09-10T21:27:13.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:13 smithi174 ceph-mon[35635]: pgmap v1043: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.4 MiB/s wr, 592 op/s 2024-09-10T21:27:14.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:13 smithi104 ceph-mon[35325]: pgmap v1043: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 6.4 MiB/s wr, 592 op/s 2024-09-10T21:27:14.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 449417 37.92 MB/sec execute 494 sec latency 72.218 ms 2024-09-10T21:27:14.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 -- 172.21.15.33:0/2031877225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac103b10 msgr2=0x7f2cac103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2031877225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac103b10 0x7f2cac103ef0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009900 tx=0x7f2c9c02a410 comp rx=0 tx=0).stop 2024-09-10T21:27:14.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 -- 172.21.15.33:0/2031877225 shutdown_connections 2024-09-10T21:27:14.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2031877225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac10cf70 0x7f2cac10f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2031877225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac104430 0x7f2cac10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2031877225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac103b10 0x7f2cac103ef0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 -- 172.21.15.33:0/2031877225 >> 172.21.15.33:0/2031877225 conn(0x7f2cac0fd5b0 msgr2=0x7f2cac0ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:14.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.108+0000 7f2cabfff640 1 -- 172.21.15.33:0/2031877225 shutdown_connections 2024-09-10T21:27:14.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.109+0000 7f2cabfff640 1 -- 172.21.15.33:0/2031877225 wait complete. 2024-09-10T21:27:14.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.109+0000 7f2cabfff640 1 Processor -- start 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.109+0000 7f2cabfff640 1 -- start start 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac103b10 0x7f2cac199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac104430 0x7f2cac199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2cabfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac10cf70 0x7f2cac19e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2cabfff640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac112c30 con 0x7f2cac10cf70 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2cabfff640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac112ab0 con 0x7f2cac103b10 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2cabfff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac112db0 con 0x7f2cac104430 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2caaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac103b10 0x7f2cac199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2caaffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac103b10 0x7f2cac199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53628/0 (socket says 172.21.15.33:53628) 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2caaffd640 1 -- 172.21.15.33:0/2126135638 learned_addr learned my addr 172.21.15.33:0/2126135638 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac10cf70 0x7f2cac19e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.110+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac104430 0x7f2cac199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.111+0000 7f2caa7fc640 1 -- 172.21.15.33:0/2126135638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac103b10 msgr2=0x7f2cac199020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.111+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac103b10 0x7f2cac199020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.111+0000 7f2caa7fc640 1 -- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac10cf70 msgr2=0x7f2cac19e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.111+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac10cf70 0x7f2cac19e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.111+0000 7f2caa7fc640 1 -- 172.21.15.33:0/2126135638 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cac19f0d0 con 0x7f2cac104430 2024-09-10T21:27:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.111+0000 7f2caaffd640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac103b10 0x7f2cac199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.112+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac104430 0x7f2cac199560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000e770 tx=0x7f2ca000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:14.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.112+0000 7f2c93fff640 1 -- 172.21.15.33:0/2126135638 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca000ccc0 con 0x7f2cac104430 2024-09-10T21:27:14.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.112+0000 7f2c93fff640 1 -- 172.21.15.33:0/2126135638 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca000ee80 con 0x7f2cac104430 2024-09-10T21:27:14.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.112+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cac19f3c0 con 0x7f2cac104430 2024-09-10T21:27:14.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.112+0000 7f2c93fff640 1 -- 172.21.15.33:0/2126135638 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca0003e40 con 0x7f2cac104430 2024-09-10T21:27:14.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.112+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cac1a5a80 con 0x7f2cac104430 2024-09-10T21:27:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.116+0000 7f2c93fff640 1 -- 172.21.15.33:0/2126135638 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2ca0004000 con 0x7f2cac104430 2024-09-10T21:27:14.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.116+0000 7f2cb39f9740 1 Processor -- start 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.116+0000 7f2cb39f9740 1 -- start start 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621881692d0 0x5621881696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 0x562188180620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5621880dfea0 con 0x5621881692d0 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5621880cf2d0 con 0x562188173630 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621880d30e0 con 0x56218817e230 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621881692d0 0x5621881696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cab7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621881692d0 0x5621881696b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38440/0 (socket says 172.21.15.33:38440) 2024-09-10T21:27:14.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.117+0000 7f2cab7fe640 1 -- 172.21.15.33:0/1282024510 learned_addr learned my addr 172.21.15.33:0/1282024510 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2caaffd640 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2c93fff640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c940b5410 0x7f2c940b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2cab7fe640 1 -- 172.21.15.33:0/1282024510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 msgr2=0x562188180620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 0x562188180620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2cab7fe640 1 -- 172.21.15.33:0/1282024510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 msgr2=0x562188169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2cab7fe640 1 -- 172.21.15.33:0/1282024510 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562187e2d650 con 0x5621881692d0 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c940b5410 0x7f2c940b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2c93fff640 1 -- 172.21.15.33:0/2126135638 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2ca00e5ed0 con 0x7f2cac104430 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.118+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621881692d0 0x5621881696b0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40104f0 tx=0x7f2ca40109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a9134b9dc0e5d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.119+0000 7f2c917fa640 1 -- 172.21.15.33:0/1282024510 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4019070 con 0x5621881692d0 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.119+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c940b5410 0x7f2c940b78d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4021000 tx=0x7f2ca401e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:14.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.119+0000 7f2c917fa640 1 -- 172.21.15.33:0/1282024510 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca400edb0 con 0x5621881692d0 2024-09-10T21:27:14.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.119+0000 7f2c917fa640 1 -- 172.21.15.33:0/1282024510 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4011b80 con 0x5621881692d0 2024-09-10T21:27:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1282024510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621881692d0 msgr2=0x5621881696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621881692d0 0x5621881696b0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f2ca40104f0 tx=0x7f2ca40109b0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1282024510 shutdown_connections 2024-09-10T21:27:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 0x562188180620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1282024510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5621881692d0 0x5621881696b0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1282024510 >> 172.21.15.33:0/1282024510 conn(0x562188167100 msgr2=0x5621881682a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1282024510 shutdown_connections 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.120+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1282024510 wait complete. 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.121+0000 7f2cb39f9740 1 -- start start 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.121+0000 7f2cb39f9740 1 Processor -- start 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5621881692d0 0x562188172fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188170420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 0x562188170ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562187f30700 con 0x562188173630 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x56218806a160 con 0x5621881692d0 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562188181020 con 0x56218817e230 2024-09-10T21:27:14.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188170420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188170420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38450/0 (socket says 172.21.15.33:38450) 2024-09-10T21:27:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 -- 172.21.15.33:0/397014490 learned_addr learned my addr 172.21.15.33:0/397014490 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 0x562188170ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2cb39f9740 10 client.0 _ll_register_callbacks cb 0x5621880839e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:14.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 -- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 msgr2=0x562188170ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 0x562188170ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 -- 172.21.15.33:0/397014490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5621881692d0 msgr2=0x562188172fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caaffd640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5621881692d0 0x562188172fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5621881692d0 0x562188172fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.122+0000 7f2caa7fc640 1 -- 172.21.15.33:0/397014490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562188097c90 con 0x562188173630 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2caaffd640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5621881692d0 0x562188172fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188170420 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00ac5e0 tx=0x7f2ca00abdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9caf92c3230ea32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2c8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2c8eff5640 1 -- 172.21.15.33:0/397014490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00abab0 con 0x562188173630 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2c8eff5640 1 -- 172.21.15.33:0/397014490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2ca00a7fa0 con 0x562188173630 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca400edb0 con 0x562188173630 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2c8eff5640 1 -- 172.21.15.33:0/397014490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f2ca00ac160 con 0x562188173630 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2cb39f9740 10 client.19482 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:14.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.123+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x56218818c9c0 con 0x562188173630 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2c8eff5640 1 -- 172.21.15.33:0/397014490 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca400edb0 con 0x562188173630 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2c8eff5640 1 -- 172.21.15.33:0/397014490 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7f2ca00c6030 con 0x562188173630 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2cb39f9740 10 client.19482 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2cb39f9740 10 client.19482 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2cb39f9740 20 client.19482 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x562188190270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2cb39f9740 4 client.19482 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621880e1df0 con 0x56218818de80 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.124+0000 7f2caaffd640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x562188190270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.125+0000 7f2c8eff5640 10 client.19482 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.125+0000 7f2caaffd640 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x562188190270 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9742340e9a03564 server_cookie=cd0b070c69ced1ed in_seq=0 out_seq=0 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2c8eff5640 1 -- 172.21.15.33:0/397014490 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5621880e1df0 con 0x56218818de80 2024-09-10T21:27:14.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2c8eff5640 10 client.19482 handle_command_reply: tid=0 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 client.19482 shutdown 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 msgr2=0x562188170420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188170420 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f2ca00ac5e0 tx=0x7f2ca00abdc0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 shutdown_connections 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x562188190270 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2c8eff5640 0 client.19482 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56218817e230 0x562188170ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188170420 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/397014490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5621881692d0 0x562188172fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 >> 172.21.15.33:0/397014490 conn(0x562188167100 msgr2=0x562188165a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 shutdown_connections 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 1 -- 172.21.15.33:0/397014490 wait complete. 2024-09-10T21:27:14.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.126+0000 7f2cb39f9740 20 client.19482 trim_cache size 0 max 16384 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 Processor -- start 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 -- start start 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x56218818a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188165a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56218817e230 0x562188165f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1ac0a0 con 0x56218817e230 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1adc40 con 0x562188173630 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1ac770 con 0x5621881692d0 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2caaffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x56218818a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188165a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188165a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53654/0 (socket says 172.21.15.33:53654) 2024-09-10T21:27:14.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.144+0000 7f2caa7fc640 1 -- 172.21.15.33:0/737251637 learned_addr learned my addr 172.21.15.33:0/737251637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:14.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56218817e230 0x562188165f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2caa7fc640 1 -- 172.21.15.33:0/737251637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 msgr2=0x56218818a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x56218818a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2caa7fc640 1 -- 172.21.15.33:0/737251637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56218817e230 msgr2=0x562188165f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56218817e230 0x562188165f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2caa7fc640 1 -- 172.21.15.33:0/737251637 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562188168010 con 0x562188173630 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56218817e230 0x562188165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.145+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188165a00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000c00 tx=0x7f2ca00038e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1b9d30d56ef77b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.146+0000 7f2c90ff9640 1 -- 172.21.15.33:0/737251637 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00a8320 con 0x562188173630 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.146+0000 7f2c90ff9640 1 -- 172.21.15.33:0/737251637 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca00e3e50 con 0x562188173630 2024-09-10T21:27:14.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.146+0000 7f2c90ff9640 1 -- 172.21.15.33:0/737251637 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca00ab320 con 0x562188173630 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 -- 172.21.15.33:0/737251637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 msgr2=0x562188165a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188165a00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2ca0000c00 tx=0x7f2ca00038e0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 -- 172.21.15.33:0/737251637 shutdown_connections 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x56218817e230 0x562188165f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562188173630 0x562188165a00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/737251637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x56218818a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 -- 172.21.15.33:0/737251637 >> 172.21.15.33:0/737251637 conn(0x562188167100 msgr2=0x56218816eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 -- 172.21.15.33:0/737251637 shutdown_connections 2024-09-10T21:27:14.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.147+0000 7f2cb39f9740 1 -- 172.21.15.33:0/737251637 wait complete. 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 -- start start 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 Processor -- start 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x562188186670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188186bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56218817e230 0x562188181400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1ad540 con 0x562188173630 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1ace40 con 0x56218817e230 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2cac1aa060 con 0x5621881692d0 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2cb39f9740 10 client.0 _ll_register_callbacks cb 0x7f2ca00ac420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188186bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2caa7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188186bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38468/0 (socket says 172.21.15.33:38468) 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.148+0000 7f2caa7fc640 1 -- 172.21.15.33:0/1276500855 learned_addr learned my addr 172.21.15.33:0/1276500855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56218817e230 0x562188181400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2caaffd640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x562188186670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2cab7fe640 1 -- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 msgr2=0x562188186670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x562188186670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2cab7fe640 1 -- 172.21.15.33:0/1276500855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 msgr2=0x562188186bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188186bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2cab7fe640 1 -- 172.21.15.33:0/1276500855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562188181c30 con 0x56218817e230 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2caaffd640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x562188186670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2caa7fc640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188186bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2cab7fe640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56218817e230 0x562188181400 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4011730 tx=0x7f2ca40133d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e36368519938abd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.149+0000 7f2c8eff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.150+0000 7f2c8eff5640 1 -- 172.21.15.33:0/1276500855 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ca4019070 con 0x56218817e230 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.150+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ca00e3e50 con 0x56218817e230 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.150+0000 7f2c8eff5640 1 -- 172.21.15.33:0/1276500855 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2ca40119e0 con 0x56218817e230 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.150+0000 7f2c8eff5640 1 -- 172.21.15.33:0/1276500855 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f2ca4009cc0 con 0x56218817e230 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.150+0000 7f2cb39f9740 10 client.28820 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.150+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x5621881fb940 con 0x56218817e230 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2c8eff5640 1 -- 172.21.15.33:0/1276500855 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ca00e3e50 con 0x56218817e230 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2c8eff5640 1 -- 172.21.15.33:0/1276500855 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7f2ca4005030 con 0x56218817e230 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2cb39f9740 10 client.28820 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2cb39f9740 10 client.28820 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2cb39f9740 20 client.28820 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x5621881fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:14.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2cb39f9740 4 client.28820 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:27:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5621880dd8a0 con 0x56218818de80 2024-09-10T21:27:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.151+0000 7f2caaffd640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x5621881fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.152+0000 7f2caaffd640 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x5621881fe0b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58a804379abe559 server_cookie=af89bfc53af5ff31 in_seq=0 out_seq=0 2024-09-10T21:27:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.152+0000 7f2c8eff5640 10 client.28820 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.152+0000 7f2c8eff5640 1 -- 172.21.15.33:0/1276500855 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5621880dd8a0 con 0x56218818de80 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.152+0000 7f2c8eff5640 10 client.28820 handle_command_reply: tid=0 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.152+0000 7f2cb39f9740 1 client.28820 shutdown 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.152+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56218817e230 msgr2=0x562188181400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.152+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56218817e230 0x562188181400 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4011730 tx=0x7f2ca40133d0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 shutdown_connections 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x56218818de80 0x5621881fe0b0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x56218817e230 0x562188181400 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562188173630 0x562188186bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 --2- 172.21.15.33:0/1276500855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621881692d0 0x562188186670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 >> 172.21.15.33:0/1276500855 conn(0x562188167100 msgr2=0x56218817fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2c8eff5640 0 client.28820 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 shutdown_connections 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 1 -- 172.21.15.33:0/1276500855 wait complete. 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.153+0000 7f2cb39f9740 20 client.28820 trim_cache size 0 max 16384 2024-09-10T21:27:14.158 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:27:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c940b5410 msgr2=0x7f2c940b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c940b5410 0x7f2c940b78d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4021000 tx=0x7f2ca401e000 comp rx=0 tx=0).stop 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac104430 msgr2=0x7f2cac199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac104430 0x7f2cac199560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2ca000e770 tx=0x7f2ca000ec30 comp rx=0 tx=0).stop 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 shutdown_connections 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c940b5410 0x7f2c940b78d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2cac10cf70 0x7f2cac19e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2cac104430 0x7f2cac199560 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 --2- 172.21.15.33:0/2126135638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2cac103b10 0x7f2cac199020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.154+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 >> 172.21.15.33:0/2126135638 conn(0x7f2cac0fd5b0 msgr2=0x7f2cac10f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.155+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 shutdown_connections 2024-09-10T21:27:14.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:14.155+0000 7f2cabfff640 1 -- 172.21.15.33:0/2126135638 wait complete. 2024-09-10T21:27:14.986 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:27:15.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 450061 37.93 MB/sec execute 495 sec latency 83.836 ms 2024-09-10T21:27:15.318 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:15.769 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:15 smithi033 ceph-mon[30860]: pgmap v1044: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 770 op/s 2024-09-10T21:27:15.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:15 smithi174 ceph-mon[35635]: pgmap v1044: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 770 op/s 2024-09-10T21:27:16.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:15 smithi104 ceph-mon[35325]: pgmap v1044: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 770 op/s 2024-09-10T21:27:16.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 450603 37.89 MB/sec execute 496 sec latency 104.521 ms 2024-09-10T21:27:16.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 -- 172.21.15.33:0/1453142715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff554101c90 msgr2=0x7ff554102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 --2- 172.21.15.33:0/1453142715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff554101c90 0x7ff554102070 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7ff544009900 tx=0x7ff54402a410 comp rx=0 tx=0).stop 2024-09-10T21:27:16.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 -- 172.21.15.33:0/1453142715 shutdown_connections 2024-09-10T21:27:16.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 --2- 172.21.15.33:0/1453142715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff554108020 0x7ff55410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 --2- 172.21.15.33:0/1453142715 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff5541076a0 0x7ff554107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 --2- 172.21.15.33:0/1453142715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff554101c90 0x7ff554102070 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 -- 172.21.15.33:0/1453142715 >> 172.21.15.33:0/1453142715 conn(0x7ff5540fd700 msgr2=0x7ff5540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 -- 172.21.15.33:0/1453142715 shutdown_connections 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.254+0000 7ff559ba9640 1 -- 172.21.15.33:0/1453142715 wait complete. 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.255+0000 7ff559ba9640 1 Processor -- start 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff559ba9640 1 -- start start 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff559ba9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff554101c90 0x7ff5541992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff559ba9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5541076a0 0x7ff5541997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff559ba9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff554108020 0x7ff55419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff559ba9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff554112e00 con 0x7ff5541076a0 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff559ba9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff554112c80 con 0x7ff554101c90 2024-09-10T21:27:16.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff559ba9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff554112f80 con 0x7ff554108020 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5541076a0 0x7ff5541997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff552ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5541076a0 0x7ff5541997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38484/0 (socket says 172.21.15.33:38484) 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff552ffd640 1 -- 172.21.15.33:0/3832111209 learned_addr learned my addr 172.21.15.33:0/3832111209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.256+0000 7ff553fff640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff554108020 0x7ff55419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff552ffd640 1 -- 172.21.15.33:0/3832111209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff554108020 msgr2=0x7ff55419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff552ffd640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff554108020 0x7ff55419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff552ffd640 1 -- 172.21.15.33:0/3832111209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff554101c90 msgr2=0x7ff5541992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff552ffd640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff554101c90 0x7ff5541992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff552ffd640 1 -- 172.21.15.33:0/3832111209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55419f360 con 0x7ff5541076a0 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff553fff640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff554108020 0x7ff55419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff552ffd640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5541076a0 0x7ff5541997f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e9f0 tx=0x7ff54800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.257+0000 7ff550ff9640 1 -- 172.21.15.33:0/3832111209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54800cd80 con 0x7ff5541076a0 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.258+0000 7ff550ff9640 1 -- 172.21.15.33:0/3832111209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5480054d0 con 0x7ff5541076a0 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.258+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55419f650 con 0x7ff5541076a0 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.258+0000 7ff550ff9640 1 -- 172.21.15.33:0/3832111209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff548010690 con 0x7ff5541076a0 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.258+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5541a5d10 con 0x7ff5541076a0 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.261+0000 7ff550ff9640 1 -- 172.21.15.33:0/3832111209 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff548010830 con 0x7ff5541076a0 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.261+0000 7ff55bf7e740 1 Processor -- start 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.261+0000 7ff55bf7e740 1 -- start start 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.263+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.263+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.263+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581dbab050 0x55581dbad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.263+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55581db0cea0 con 0x55581dba0450 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.263+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55581dafc2d0 con 0x55581dbab050 2024-09-10T21:27:16.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.263+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55581db000e0 con 0x55581db960f0 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.264+0000 7ff553fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.264+0000 7ff553fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:50238/0 (socket says 172.21.15.33:50238) 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.264+0000 7ff553fff640 1 -- 172.21.15.33:0/1198815670 learned_addr learned my addr 172.21.15.33:0/1198815670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.264+0000 7ff552ffd640 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581dbab050 0x55581dbad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.264+0000 7ff553fff640 1 -- 172.21.15.33:0/1198815670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581dbab050 msgr2=0x55581dbad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff5537fe640 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff553fff640 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581dbab050 0x55581dbad440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff553fff640 1 -- 172.21.15.33:0/1198815670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dba0450 msgr2=0x55581db96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff553fff640 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff553fff640 1 -- 172.21.15.33:0/1198815670 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581d85a650 con 0x55581db960f0 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff552ffd640 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581dbab050 0x55581dbad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff5537fe640 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:16.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff553fff640 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581db960f0 0x55581db964d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0117e0 tx=0x7ff54c011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d282f301dc4ec16 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff550ff9640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff53c0b53e0 0x7ff53c0b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.265+0000 7ff539ffb640 1 -- 172.21.15.33:0/1198815670 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c011310 con 0x55581db960f0 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.266+0000 7ff539ffb640 1 -- 172.21.15.33:0/1198815670 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54c003e20 con 0x55581db960f0 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.266+0000 7ff539ffb640 1 -- 172.21.15.33:0/1198815670 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0044f0 con 0x55581db960f0 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.266+0000 7ff550ff9640 1 -- 172.21.15.33:0/3832111209 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff5480e5f20 con 0x7ff5541076a0 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.266+0000 7ff5537fe640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff53c0b53e0 0x7ff53c0b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1198815670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581db960f0 msgr2=0x55581db964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581db960f0 0x55581db964d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff54c0117e0 tx=0x7ff54c011ca0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1198815670 shutdown_connections 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff5537fe640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff53c0b53e0 0x7ff53c0b78a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff544009230 tx=0x7ff544031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581dbab050 0x55581dbad440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1198815670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1198815670 >> 172.21.15.33:0/1198815670 conn(0x55581db93f20 msgr2=0x55581db950c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1198815670 shutdown_connections 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.267+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1198815670 wait complete. 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 -- start start 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 Processor -- start 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db9f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db9f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbaa700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55581d95d700 con 0x55581dbab050 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55581da97160 con 0x55581db960f0 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55581dba0c80 con 0x55581dba0450 2024-09-10T21:27:16.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff55bf7e740 10 client.0 _ll_register_callbacks cb 0x55581dab09e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff553fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbaa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff553fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbaa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38494/0 (socket says 172.21.15.33:38494) 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db9f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff553fff640 1 -- 172.21.15.33:0/57448214 learned_addr learned my addr 172.21.15.33:0/57448214 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.268+0000 7ff552ffd640 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db9f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff553fff640 1 -- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 msgr2=0x55581db9f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff553fff640 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db9f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff553fff640 1 -- 172.21.15.33:0/57448214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 msgr2=0x55581db9f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff553fff640 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db9f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff553fff640 1 -- 172.21.15.33:0/57448214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581dac4c90 con 0x55581dbab050 2024-09-10T21:27:16.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff552ffd640 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db9f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff553fff640 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbaa700 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff54c004110 tx=0x7ff54c011480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ab8338a5bbdd80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff532ffd640 1 -- 172.21.15.33:0/57448214 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c00fdb0 con 0x55581dbab050 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54c003e20 con 0x55581dbab050 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.269+0000 7ff532ffd640 1 -- 172.21.15.33:0/57448214 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff54c004270 con 0x55581dbab050 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff532ffd640 1 -- 172.21.15.33:0/57448214 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff54c004470 con 0x55581dbab050 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 10 client.19497 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55581dbc20b0 con 0x55581dbab050 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff532ffd640 1 -- 172.21.15.33:0/57448214 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54c003e20 con 0x55581dbab050 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff532ffd640 1 -- 172.21.15.33:0/57448214 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7ff54c018020 con 0x55581dbab050 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 10 client.19497 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:16.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 10 client.19497 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 20 client.19497 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dbc57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 4 client.19497 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55581db0edf0 con 0x55581dbc3400 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.270+0000 7ff5537fe640 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dbc57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.271+0000 7ff5537fe640 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dbc57f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=93ccddbc8eb1f13 server_cookie=f8798f74878fe62f in_seq=0 out_seq=0 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.271+0000 7ff532ffd640 10 client.19497 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.275+0000 7ff532ffd640 1 -- 172.21.15.33:0/57448214 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55581db0edf0 con 0x55581dbc3400 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.275+0000 7ff532ffd640 10 client.19497 handle_command_reply: tid=0 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.275+0000 7ff55bf7e740 1 client.19497 shutdown 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 msgr2=0x55581dbaa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbaa700 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7ff54c004110 tx=0x7ff54c011480 comp rx=0 tx=0).stop 2024-09-10T21:27:16.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 shutdown_connections 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbaa700 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dbc57f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff532ffd640 0 client.19497 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db9f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/57448214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db9f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 >> 172.21.15.33:0/57448214 conn(0x55581db93f20 msgr2=0x55581db92750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 shutdown_connections 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 1 -- 172.21.15.33:0/57448214 wait complete. 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.276+0000 7ff55bf7e740 20 client.19497 trim_cache size 0 max 16384 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 Processor -- start 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 -- start start 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581db9d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff54000e4b0 con 0x55581dbab050 2024-09-10T21:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff540010050 con 0x55581db960f0 2024-09-10T21:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff54000eb80 con 0x55581dba0450 2024-09-10T21:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff553fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581db9d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff553fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581db9d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38496/0 (socket says 172.21.15.33:38496) 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff553fff640 1 -- 172.21.15.33:0/1295377000 learned_addr learned my addr 172.21.15.33:0/1295377000 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.296+0000 7ff552ffd640 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff553fff640 1 -- 172.21.15.33:0/1295377000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 msgr2=0x55581db96a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff553fff640 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff553fff640 1 -- 172.21.15.33:0/1295377000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 msgr2=0x55581db964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff553fff640 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff553fff640 1 -- 172.21.15.33:0/1295377000 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581db9fb00 con 0x55581dbab050 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff5537fe640 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff553fff640 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581db9d8f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff54c000f80 tx=0x7ff54c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43902c79ce705385 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff5397fa640 1 -- 172.21.15.33:0/1295377000 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c009e00 con 0x55581dbab050 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff5397fa640 1 -- 172.21.15.33:0/1295377000 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54c02a070 con 0x55581dbab050 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.297+0000 7ff5397fa640 1 -- 172.21.15.33:0/1295377000 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54c0042d0 con 0x55581dbab050 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1295377000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 msgr2=0x55581db9d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581db9d8f0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7ff54c000f80 tx=0x7ff54c005d30 comp rx=0 tx=0).stop 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1295377000 shutdown_connections 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581db9d8f0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581db96a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/1295377000 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581db964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1295377000 >> 172.21.15.33:0/1295377000 conn(0x55581db93f20 msgr2=0x55581db9a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1295377000 shutdown_connections 2024-09-10T21:27:16.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.298+0000 7ff55bf7e740 1 -- 172.21.15.33:0/1295377000 wait complete. 2024-09-10T21:27:16.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 -- start start 2024-09-10T21:27:16.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 Processor -- start 2024-09-10T21:27:16.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581dbd7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581dbd77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff54000f950 con 0x55581dbab050 2024-09-10T21:27:16.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff54000f250 con 0x55581db960f0 2024-09-10T21:27:16.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff54000c470 con 0x55581dba0450 2024-09-10T21:27:16.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581dbd7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff55bf7e740 10 client.0 _ll_register_callbacks cb 0x7ff54c0116e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff5537fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581dbd7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53716/0 (socket says 172.21.15.33:53716) 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff553fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.299+0000 7ff5537fe640 1 -- 172.21.15.33:0/295014889 learned_addr learned my addr 172.21.15.33:0/295014889 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff552ffd640 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581dbd77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff5537fe640 1 -- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 msgr2=0x55581dbd77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff5537fe640 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581dbd77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff5537fe640 1 -- 172.21.15.33:0/295014889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 msgr2=0x55581dbca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff5537fe640 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff5537fe640 1 -- 172.21.15.33:0/295014889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55581dbcb190 con 0x55581db960f0 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff5537fe640 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581dbd7260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff544009900 tx=0x7ff54403b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aacfabb6a4e8c2a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff532ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:27:16.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff532ffd640 1 -- 172.21.15.33:0/295014889 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff54405a060 con 0x55581db960f0 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.300+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff54c02a070 con 0x55581db960f0 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.301+0000 7ff532ffd640 1 -- 172.21.15.33:0/295014889 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff544004450 con 0x55581db960f0 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.301+0000 7ff532ffd640 1 -- 172.21.15.33:0/295014889 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff544035050 con 0x55581db960f0 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.301+0000 7ff55bf7e740 10 client.28832 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.301+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55581dbd5b60 con 0x55581db960f0 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.301+0000 7ff532ffd640 1 -- 172.21.15.33:0/295014889 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff54c02a070 con 0x55581db960f0 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.301+0000 7ff532ffd640 1 -- 172.21.15.33:0/295014889 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7ff54405f070 con 0x55581db960f0 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.301+0000 7ff55bf7e740 10 client.28832 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.302+0000 7ff55bf7e740 10 client.28832 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.302+0000 7ff55bf7e740 20 client.28832 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.302+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dc32570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:16.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.302+0000 7ff55bf7e740 4 client.28832 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.302+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55581db0a8a0 con 0x55581dbc3400 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.302+0000 7ff552ffd640 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dc32570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.302+0000 7ff532ffd640 10 client.28832 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff552ffd640 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dc32570 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2878dc198b9158af server_cookie=97664f561e2e57b3 in_seq=0 out_seq=0 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff532ffd640 1 -- 172.21.15.33:0/295014889 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55581db0a8a0 con 0x55581dbc3400 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff532ffd640 10 client.28832 handle_command_reply: tid=0 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 client.28832 shutdown 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 msgr2=0x55581dbd7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581dbd7260 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff544009900 tx=0x7ff54403b000 comp rx=0 tx=0).stop 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 shutdown_connections 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55581dbab050 0x55581dbca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x55581dbc3400 0x55581dc32570 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55581dba0450 0x55581dbd77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 --2- 172.21.15.33:0/295014889 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55581db960f0 0x55581dbd7260 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff532ffd640 0 client.28832 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 >> 172.21.15.33:0/295014889 conn(0x55581db93f20 msgr2=0x55581db9d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.303+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 shutdown_connections 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.304+0000 7ff55bf7e740 1 -- 172.21.15.33:0/295014889 wait complete. 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.304+0000 7ff55bf7e740 20 client.28832 trim_cache size 0 max 16384 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.304+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff53c0b53e0 msgr2=0x7ff53c0b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.304+0000 7ff559ba9640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff53c0b53e0 0x7ff53c0b78a0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff544009230 tx=0x7ff544031400 comp rx=0 tx=0).stop 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5541076a0 msgr2=0x7ff5541997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5541076a0 0x7ff5541997f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7ff54800e9f0 tx=0x7ff54800edf0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 shutdown_connections 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff53c0b53e0 0x7ff53c0b78a0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff554108020 0x7ff55419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff5541076a0 0x7ff5541997f0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 --2- 172.21.15.33:0/3832111209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff554101c90 0x7ff5541992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:16.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 >> 172.21.15.33:0/3832111209 conn(0x7ff5540fd700 msgr2=0x7ff5540ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:16.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 shutdown_connections 2024-09-10T21:27:16.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:16.305+0000 7ff559ba9640 1 -- 172.21.15.33:0/3832111209 wait complete. 2024-09-10T21:27:17.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 451366 37.88 MB/sec execute 497 sec latency 97.021 ms 2024-09-10T21:27:17.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:17 smithi174 ceph-mon[35635]: pgmap v1045: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.8 MiB/s wr, 661 op/s 2024-09-10T21:27:17.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:17 smithi033 ceph-mon[30860]: pgmap v1045: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.8 MiB/s wr, 661 op/s 2024-09-10T21:27:18.024 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:27:18.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:17 smithi104 ceph-mon[35325]: pgmap v1045: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 8.8 MiB/s wr, 661 op/s 2024-09-10T21:27:18.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 452209 37.92 MB/sec execute 498 sec latency 86.165 ms 2024-09-10T21:27:18.331 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.856+0000 7fb40eb44640 1 -- 172.21.15.33:0/558937092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb408101cb0 msgr2=0x7fb408102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.856+0000 7fb40eb44640 1 --2- 172.21.15.33:0/558937092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb408101cb0 0x7fb408102090 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009900 tx=0x7fb3fc02a410 comp rx=0 tx=0).stop 2024-09-10T21:27:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 -- 172.21.15.33:0/558937092 shutdown_connections 2024-09-10T21:27:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 --2- 172.21.15.33:0/558937092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb408108040 0x7fb40810e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 --2- 172.21.15.33:0/558937092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4081076c0 0x7fb408107b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 --2- 172.21.15.33:0/558937092 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb408101cb0 0x7fb408102090 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 -- 172.21.15.33:0/558937092 >> 172.21.15.33:0/558937092 conn(0x7fb4080fd700 msgr2=0x7fb4080ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 -- 172.21.15.33:0/558937092 shutdown_connections 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 -- 172.21.15.33:0/558937092 wait complete. 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.857+0000 7fb40eb44640 1 Processor -- start 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40eb44640 1 -- start start 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40eb44640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb408101cb0 0x7fb40819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40eb44640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4081076c0 0x7fb40819a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40eb44640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb408108040 0x7fb40819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40eb44640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb408111d50 con 0x7fb4081076c0 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40eb44640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb408111bd0 con 0x7fb408101cb0 2024-09-10T21:27:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40eb44640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb408111ed0 con 0x7fb408108040 2024-09-10T21:27:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4081076c0 0x7fb40819a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4081076c0 0x7fb40819a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38520/0 (socket says 172.21.15.33:38520) 2024-09-10T21:27:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb407fff640 1 -- 172.21.15.33:0/4195717041 learned_addr learned my addr 172.21.15.33:0/4195717041 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb408101cb0 0x7fb40819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.858+0000 7fb40d0e1640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb408108040 0x7fb40819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb407fff640 1 -- 172.21.15.33:0/4195717041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb408108040 msgr2=0x7fb40819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb407fff640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb408108040 0x7fb40819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb407fff640 1 -- 172.21.15.33:0/4195717041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb408101cb0 msgr2=0x7fb40819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb407fff640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb408101cb0 0x7fb40819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb407fff640 1 -- 172.21.15.33:0/4195717041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4081a5460 con 0x7fb4081076c0 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb407fff640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4081076c0 0x7fb40819a5c0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e9f0 tx=0x7fb3f400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb405ffb640 1 -- 172.21.15.33:0/4195717041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f400cd80 con 0x7fb4081076c0 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb405ffb640 1 -- 172.21.15.33:0/4195717041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40108e0 con 0x7fb4081076c0 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4081a5750 con 0x7fb4081076c0 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.859+0000 7fb405ffb640 1 -- 172.21.15.33:0/4195717041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4018980 con 0x7fb4081076c0 2024-09-10T21:27:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.860+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4081a5c90 con 0x7fb4081076c0 2024-09-10T21:27:18.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.862+0000 7fb405ffb640 1 -- 172.21.15.33:0/4195717041 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb3f4010430 con 0x7fb4081076c0 2024-09-10T21:27:18.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 Processor -- start 2024-09-10T21:27:18.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 -- start start 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec558b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec61840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec64170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55873ebcbea0 con 0x55873ec55df0 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55873ebbb2d0 con 0x55873ec554d0 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb410f19740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55873ebbf0e0 con 0x55873ec61d80 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb40d0e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb407fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec64170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb40d0e1640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec558b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:53738/0 (socket says 172.21.15.33:53738) 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.863+0000 7fb40d0e1640 1 -- 172.21.15.33:0/2580937876 learned_addr learned my addr 172.21.15.33:0/2580937876 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb405ffb640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3f80b52e0 0x7fb3f80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec61840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb407fff640 1 -- 172.21.15.33:0/2580937876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 msgr2=0x55873ec558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb40d0e1640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3f80b52e0 0x7fb3f80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb407fff640 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb407fff640 1 -- 172.21.15.33:0/2580937876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 msgr2=0x55873ec61840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb407fff640 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec61840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb407fff640 1 -- 172.21.15.33:0/2580937876 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873e919650 con 0x55873ec61d80 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec61840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb407fff640 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec64170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40e3990 tx=0x7fb3f40ac630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aab351cf1abe073 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb405ffb640 1 -- 172.21.15.33:0/4195717041 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb3f40e5ec0 con 0x7fb4081076c0 2024-09-10T21:27:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb3f2ffd640 1 -- 172.21.15.33:0/2580937876 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40abea0 con 0x55873ec61d80 2024-09-10T21:27:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb3f2ffd640 1 -- 172.21.15.33:0/2580937876 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40ac240 con 0x55873ec61d80 2024-09-10T21:27:18.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.864+0000 7fb3f2ffd640 1 -- 172.21.15.33:0/2580937876 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f401a0d0 con 0x55873ec61d80 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 -- 172.21.15.33:0/2580937876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 msgr2=0x55873ec64170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec64170 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40e3990 tx=0x7fb3f40ac630 comp rx=0 tx=0).stop 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb40d0e1640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3f80b52e0 0x7fb3f80b77a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb4000106d0 tx=0x7fb400004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 -- 172.21.15.33:0/2580937876 shutdown_connections 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec64170 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec61840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 --2- 172.21.15.33:0/2580937876 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 -- 172.21.15.33:0/2580937876 >> 172.21.15.33:0/2580937876 conn(0x55873ec53300 msgr2=0x55873ec544a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 -- 172.21.15.33:0/2580937876 shutdown_connections 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.865+0000 7fb410f19740 1 -- 172.21.15.33:0/2580937876 wait complete. 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.866+0000 7fb410f19740 1 -- start start 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.866+0000 7fb410f19740 1 Processor -- start 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.866+0000 7fb410f19740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec554d0 0x55873ec61680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.866+0000 7fb410f19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec55df0 0x55873ec5ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.866+0000 7fb40c8e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec554d0 0x55873ec61680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.866+0000 7fb40c8e0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec554d0 0x55873ec61680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38532/0 (socket says 172.21.15.33:38532) 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.866+0000 7fb40c8e0640 1 -- 172.21.15.33:0/152702117 learned_addr learned my addr 172.21.15.33:0/152702117 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb410f19740 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec61d80 0x55873ec5f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55873ea1c700 con 0x55873ec554d0 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55873eb56160 con 0x55873ec61d80 2024-09-10T21:27:18.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55873ec64a00 con 0x55873ec55df0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb410f19740 10 client.0 _ll_register_callbacks cb 0x55873eb6f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb40d0e1640 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec61d80 0x55873ec5f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb407fff640 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec55df0 0x55873ec5ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb40c8e0640 1 -- 172.21.15.33:0/152702117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec55df0 msgr2=0x55873ec5ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec55df0 0x55873ec5ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb40c8e0640 1 -- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec61d80 msgr2=0x55873ec5f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec61d80 0x55873ec5f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb40c8e0640 1 -- 172.21.15.33:0/152702117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873eb83c90 con 0x55873ec554d0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec554d0 0x55873ec61680 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc000c00 tx=0x7fb3fc0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7d75c5ef8ac94c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.867+0000 7fb3e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb3e7fff640 1 -- 172.21.15.33:0/152702117 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc035040 con 0x55873ec554d0 2024-09-10T21:27:18.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f40ac240 con 0x55873ec554d0 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb3e7fff640 1 -- 172.21.15.33:0/152702117 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb3fc005450 con 0x55873ec554d0 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb3e7fff640 1 -- 172.21.15.33:0/152702117 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb3fc005650 con 0x55873ec554d0 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb410f19740 10 client.19512 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55873ec702f0 con 0x55873ec554d0 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb3e7fff640 1 -- 172.21.15.33:0/152702117 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40ac240 con 0x55873ec554d0 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.868+0000 7fb3e7fff640 1 -- 172.21.15.33:0/152702117 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7fb3fc003ae0 con 0x55873ec554d0 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.869+0000 7fb410f19740 10 client.19512 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.869+0000 7fb410f19740 10 client.19512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.869+0000 7fb410f19740 20 client.19512 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.869+0000 7fb410f19740 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ec71800 0x55873ec73c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.869+0000 7fb410f19740 4 client.19512 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.869+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873ebcddf0 con 0x55873ec71800 2024-09-10T21:27:18.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.869+0000 7fb407fff640 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ec71800 0x55873ec73c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.870+0000 7fb407fff640 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ec71800 0x55873ec73c50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7fd0a8bab8d7a7 server_cookie=6b7a75d1bbb1f98c in_seq=0 out_seq=0 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.870+0000 7fb3e7fff640 10 client.19512 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb3e7fff640 1 -- 172.21.15.33:0/152702117 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55873ebcddf0 con 0x55873ec71800 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb3e7fff640 10 client.19512 handle_command_reply: tid=0 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 client.19512 shutdown 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec554d0 msgr2=0x55873ec61680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec554d0 0x55873ec61680 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc000c00 tx=0x7fb3fc0043b0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 shutdown_connections 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec61d80 0x55873ec5f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec55df0 0x55873ec5ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ec71800 0x55873ec73c50 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 --2- 172.21.15.33:0/152702117 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec554d0 0x55873ec61680 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb3e7fff640 0 client.19512 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:27:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 >> 172.21.15.33:0/152702117 conn(0x55873ec53300 msgr2=0x55873ec58ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 shutdown_connections 2024-09-10T21:27:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.871+0000 7fb410f19740 1 -- 172.21.15.33:0/152702117 wait complete. 2024-09-10T21:27:18.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.872+0000 7fb410f19740 20 client.19512 trim_cache size 0 max 16384 2024-09-10T21:27:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 Processor -- start 2024-09-10T21:27:18.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 -- start start 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec6dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec63170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec636b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081ac1e0 con 0x55873ec55df0 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081add80 con 0x55873ec554d0 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb410f19740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081ac8b0 con 0x55873ec61d80 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec63170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec63170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38542/0 (socket says 172.21.15.33:38542) 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb407fff640 1 -- 172.21.15.33:0/3778702441 learned_addr learned my addr 172.21.15.33:0/3778702441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec6dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.887+0000 7fb40d0e1640 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec636b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb407fff640 1 -- 172.21.15.33:0/3778702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 msgr2=0x55873ec636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb407fff640 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec636b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb407fff640 1 -- 172.21.15.33:0/3778702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 msgr2=0x55873ec6dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb407fff640 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec6dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb407fff640 1 -- 172.21.15.33:0/3778702441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873ec4e990 con 0x55873ec55df0 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec6dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb407fff640 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec63170 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c6a0 tx=0x7fb3f40e9940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7605814eeec48f24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb3f27fc640 1 -- 172.21.15.33:0/3778702441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40b5e60 con 0x55873ec55df0 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb3f27fc640 1 -- 172.21.15.33:0/3778702441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40ac240 con 0x55873ec55df0 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.888+0000 7fb3f27fc640 1 -- 172.21.15.33:0/3778702441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40b5c10 con 0x55873ec55df0 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.889+0000 7fb410f19740 1 -- 172.21.15.33:0/3778702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 msgr2=0x55873ec63170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.889+0000 7fb410f19740 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec63170 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400c6a0 tx=0x7fb3f40e9940 comp rx=0 tx=0).stop 2024-09-10T21:27:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.889+0000 7fb410f19740 1 -- 172.21.15.33:0/3778702441 shutdown_connections 2024-09-10T21:27:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.889+0000 7fb410f19740 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec636b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.889+0000 7fb410f19740 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec63170 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.889+0000 7fb410f19740 1 --2- 172.21.15.33:0/3778702441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec6dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.889+0000 7fb410f19740 1 -- 172.21.15.33:0/3778702441 >> 172.21.15.33:0/3778702441 conn(0x55873ec53300 msgr2=0x55873ec541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.890+0000 7fb410f19740 1 -- 172.21.15.33:0/3778702441 shutdown_connections 2024-09-10T21:27:18.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.890+0000 7fb410f19740 1 -- 172.21.15.33:0/3778702441 wait complete. 2024-09-10T21:27:18.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.890+0000 7fb410f19740 1 -- start start 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.890+0000 7fb410f19740 1 Processor -- start 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb410f19740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec79450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb410f19740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec79990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb410f19740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec79ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb410f19740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081ad680 con 0x55873ec55df0 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb410f19740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081acf80 con 0x55873ec554d0 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb410f19740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4081aa1a0 con 0x55873ec61d80 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb410f19740 10 client.0 _ll_register_callbacks cb 0x7fb4000093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec79990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb40c8e0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec79450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec79990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38548/0 (socket says 172.21.15.33:38548) 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 -- 172.21.15.33:0/3508453024 learned_addr learned my addr 172.21.15.33:0/3508453024 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb40d0e1640 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec79ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 -- 172.21.15.33:0/3508453024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 msgr2=0x55873ec79ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec79ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 -- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 msgr2=0x55873ec79450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec79450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb407fff640 1 -- 172.21.15.33:0/3508453024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873ec66210 con 0x55873ec55df0 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.891+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec79450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb407fff640 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec79990 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40a8010 tx=0x7fb3f400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302cf4f06e5693bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb3e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb3e7fff640 1 -- 172.21.15.33:0/3508453024 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40ac730 con 0x55873ec55df0 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb3e7fff640 1 -- 172.21.15.33:0/3508453024 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb3f4097830 con 0x55873ec55df0 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f40ac240 con 0x55873ec55df0 2024-09-10T21:27:18.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb3e7fff640 1 -- 172.21.15.33:0/3508453024 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fb3f40b5c10 con 0x55873ec55df0 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb410f19740 10 client.19518 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55873ece7260 con 0x55873ec55df0 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.892+0000 7fb3e7fff640 1 -- 172.21.15.33:0/3508453024 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40ac240 con 0x55873ec55df0 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb3e7fff640 1 -- 172.21.15.33:0/3508453024 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7fb3f40c5050 con 0x55873ec55df0 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb410f19740 10 client.19518 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb410f19740 10 client.19518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb410f19740 20 client.19518 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb410f19740 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ece72b0 0x55873ece96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb410f19740 4 client.19518 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55873ebc98a0 con 0x55873ece72b0 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ece72b0 0x55873ece96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb3e7fff640 10 client.19518 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb40c8e0640 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ece72b0 0x55873ece96a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a707ed5156bdc4ef server_cookie=f5412f30c00184df in_seq=0 out_seq=0 2024-09-10T21:27:18.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.893+0000 7fb3e7fff640 1 -- 172.21.15.33:0/3508453024 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55873ebc98a0 con 0x55873ece72b0 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb3e7fff640 10 client.19518 handle_command_reply: tid=0 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 client.19518 shutdown 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 msgr2=0x55873ec79990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec79990 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40a8010 tx=0x7fb3f400cf20 comp rx=0 tx=0).stop 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 shutdown_connections 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55873ec61d80 0x55873ec79ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x55873ece72b0 0x55873ece96a0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb3e7fff640 0 client.19518 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55873ec55df0 0x55873ec79990 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 --2- 172.21.15.33:0/3508453024 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55873ec554d0 0x55873ec79450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 >> 172.21.15.33:0/3508453024 conn(0x55873ec53300 msgr2=0x55873ec51c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 shutdown_connections 2024-09-10T21:27:18.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 1 -- 172.21.15.33:0/3508453024 wait complete. 2024-09-10T21:27:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.894+0000 7fb410f19740 20 client.19518 trim_cache size 0 max 16384 2024-09-10T21:27:18.901 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:27:18.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3f80b52e0 msgr2=0x7fb3f80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3f80b52e0 0x7fb3f80b77a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb4000106d0 tx=0x7fb400004ab0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4081076c0 msgr2=0x7fb40819a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4081076c0 0x7fb40819a5c0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fb3f400e9f0 tx=0x7fb3f400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 shutdown_connections 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb3f80b52e0 0x7fb3f80b77a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb408108040 0x7fb40819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb4081076c0 0x7fb40819a5c0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 --2- 172.21.15.33:0/4195717041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb408101cb0 0x7fb40819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.895+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 >> 172.21.15.33:0/4195717041 conn(0x7fb4080fd700 msgr2=0x7fb4080feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.896+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 shutdown_connections 2024-09-10T21:27:18.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:18.896+0000 7fb40eb44640 1 -- 172.21.15.33:0/4195717041 wait complete. 2024-09-10T21:27:19.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 452799 37.92 MB/sec execute 499 sec latency 94.226 ms 2024-09-10T21:27:19.751 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:19 smithi033 ceph-mon[30860]: pgmap v1046: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 701 op/s 2024-09-10T21:27:19.752 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:27:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:19 smithi174 ceph-mon[35635]: pgmap v1046: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 701 op/s 2024-09-10T21:27:20.057 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:20.064 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:19 smithi104 ceph-mon[35325]: pgmap v1046: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.9 MiB/s wr, 701 op/s 2024-09-10T21:27:20.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 453252 37.88 MB/sec execute 500 sec latency 101.213 ms 2024-09-10T21:27:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/3682773055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 msgr2=0x7fbdc810f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/3682773055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc810f590 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009ca0 tx=0x7fbdc002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/3682773055 shutdown_connections 2024-09-10T21:27:20.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/3682773055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc810f590 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/3682773055 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdc81045f0 0x7fbdc810cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/3682773055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc8103cd0 0x7fbdc81040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/3682773055 >> 172.21.15.33:0/3682773055 conn(0x7fbdc80fd770 msgr2=0x7fbdc80ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/3682773055 shutdown_connections 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.556+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/3682773055 wait complete. 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.557+0000 7fbdcfc8c640 1 Processor -- start 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.557+0000 7fbdcfc8c640 1 -- start start 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcfc8c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdc8103cd0 0x7fbdc8101980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcfc8c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc81045f0 0x7fbdc8101ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcfc8c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc8102400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcfc8c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112df0 con 0x7fbdc810d130 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcfc8c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112c70 con 0x7fbdc8103cd0 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcfc8c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc8112f70 con 0x7fbdc81045f0 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc8102400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc8102400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33356/0 (socket says 172.21.15.33:33356) 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 -- 172.21.15.33:0/2659462227 learned_addr learned my addr 172.21.15.33:0/2659462227 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcda28640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdc8103cd0 0x7fbdc8101980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdcd227640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc81045f0 0x7fbdc8101ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 -- 172.21.15.33:0/2659462227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc81045f0 msgr2=0x7fbdc8101ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc81045f0 0x7fbdc8101ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 -- 172.21.15.33:0/2659462227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdc8103cd0 msgr2=0x7fbdc8101980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdc8103cd0 0x7fbdc8101980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.558+0000 7fbdce229640 1 -- 172.21.15.33:0/2659462227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdc81a76c0 con 0x7fbdc810d130 2024-09-10T21:27:20.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.559+0000 7fbdce229640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc8102400 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fbdc002f540 tx=0x7fbdc00316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.559+0000 7fbdbeffd640 1 -- 172.21.15.33:0/2659462227 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003b70 con 0x7fbdc810d130 2024-09-10T21:27:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.559+0000 7fbdbeffd640 1 -- 172.21.15.33:0/2659462227 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc0003d10 con 0x7fbdc810d130 2024-09-10T21:27:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.559+0000 7fbdbeffd640 1 -- 172.21.15.33:0/2659462227 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc002f990 con 0x7fbdc810d130 2024-09-10T21:27:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.559+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdc81a7950 con 0x7fbdc810d130 2024-09-10T21:27:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.559+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdc81a7d80 con 0x7fbdc810d130 2024-09-10T21:27:20.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.564+0000 7fbdd2061740 1 Processor -- start 2024-09-10T21:27:20.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.566+0000 7fbdbeffd640 1 -- 172.21.15.33:0/2659462227 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fbdc002fb30 con 0x7fbdc810d130 2024-09-10T21:27:20.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.566+0000 7fbdd2061740 1 -- start start 2024-09-10T21:27:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af09278b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af093e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdd2061740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560af089dea0 con 0x560af093c590 2024-09-10T21:27:20.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdd2061740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560af088d2d0 con 0x560af09274d0 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdd2061740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560af08910e0 con 0x560af0927df0 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af09278b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af09278b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50142/0 (socket says 172.21.15.33:50142) 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdcd227640 1 -- 172.21.15.33:0/117166114 learned_addr learned my addr 172.21.15.33:0/117166114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.567+0000 7fbdcd227640 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af093e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.568+0000 7fbdcd227640 1 -- 172.21.15.33:0/117166114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 msgr2=0x560af093c050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.568+0000 7fbdcd227640 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093c050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.568+0000 7fbdcd227640 1 -- 172.21.15.33:0/117166114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 msgr2=0x560af093e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.568+0000 7fbdcd227640 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af093e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.568+0000 7fbdcd227640 1 -- 172.21.15.33:0/117166114 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af05eb650 con 0x560af09274d0 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.568+0000 7fbdbeffd640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdb40b5360 0x7fbdb40b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.568+0000 7fbdbeffd640 1 -- 172.21.15.33:0/2659462227 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbdc0106e60 con 0x7fbdc810d130 2024-09-10T21:27:20.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.569+0000 7fbdcd227640 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af09278b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8011410 tx=0x7fbdb80096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7680b78c1f60be4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.569+0000 7fbdcda28640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdb40b5360 0x7fbdb40b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.570+0000 7fbdb3fff640 1 -- 172.21.15.33:0/117166114 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8004030 con 0x560af09274d0 2024-09-10T21:27:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.570+0000 7fbdb3fff640 1 -- 172.21.15.33:0/117166114 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb8003610 con 0x560af09274d0 2024-09-10T21:27:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdd2061740 1 -- 172.21.15.33:0/117166114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 msgr2=0x560af09278b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdcda28640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdb40b5360 0x7fbdb40b7820 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x560af092a650 tx=0x7fbdc4005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdd2061740 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af09278b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8011410 tx=0x7fbdb80096b0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdd2061740 1 -- 172.21.15.33:0/117166114 shutdown_connections 2024-09-10T21:27:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdd2061740 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af093e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdd2061740 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdd2061740 1 --2- 172.21.15.33:0/117166114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af09278b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.571+0000 7fbdd2061740 1 -- 172.21.15.33:0/117166114 >> 172.21.15.33:0/117166114 conn(0x560af0925300 msgr2=0x560af09264a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.572+0000 7fbdd2061740 1 -- 172.21.15.33:0/117166114 shutdown_connections 2024-09-10T21:27:20.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.572+0000 7fbdd2061740 1 -- 172.21.15.33:0/117166114 wait complete. 2024-09-10T21:27:20.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.572+0000 7fbdd2061740 1 -- start start 2024-09-10T21:27:20.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 1 Processor -- start 2024-09-10T21:27:20.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af09274d0 0x560af093bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af0927df0 0x560af09391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af093c590 0x560af0939700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x560af06ee700 con 0x560af0927df0 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x560af0828160 con 0x560af093c590 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560af093f290 con 0x560af09274d0 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.573+0000 7fbdd2061740 10 client.0 _ll_register_callbacks cb 0x560af08419e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.574+0000 7fbdcda28640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af09274d0 0x560af093bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.574+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af0927df0 0x560af09391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.574+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af0927df0 0x560af09391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33388/0 (socket says 172.21.15.33:33388) 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.574+0000 7fbdcd227640 1 -- 172.21.15.33:0/3358030214 learned_addr learned my addr 172.21.15.33:0/3358030214 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.574+0000 7fbdce229640 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af093c590 0x560af0939700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdce229640 1 -- 172.21.15.33:0/3358030214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af09274d0 msgr2=0x560af093bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:20.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdce229640 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af09274d0 0x560af093bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdce229640 1 -- 172.21.15.33:0/3358030214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af0927df0 msgr2=0x560af09391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:20.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdce229640 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af0927df0 0x560af09391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdce229640 1 -- 172.21.15.33:0/3358030214 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af0855c90 con 0x560af093c590 2024-09-10T21:27:20.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdce229640 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af093c590 0x560af0939700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0104960 tx=0x7fbdc00cce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f66dde58f48d9fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:27:20.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdb17fa640 1 -- 172.21.15.33:0/3358030214 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00cda80 con 0x560af093c590 2024-09-10T21:27:20.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdb8003610 con 0x560af093c590 2024-09-10T21:27:20.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdb17fa640 1 -- 172.21.15.33:0/3358030214 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbdc00d6720 con 0x560af093c590 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.575+0000 7fbdb17fa640 1 -- 172.21.15.33:0/3358030214 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fbdc00d2100 con 0x560af093c590 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.576+0000 7fbdd2061740 10 client.28862 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.576+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x560af094aae0 con 0x560af093c590 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.576+0000 7fbdb17fa640 1 -- 172.21.15.33:0/3358030214 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdb8003610 con 0x560af093c590 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.576+0000 7fbdb17fa640 1 -- 172.21.15.33:0/3358030214 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7fbdc00e7020 con 0x560af093c590 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.576+0000 7fbdd2061740 10 client.28862 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.576+0000 7fbdd2061740 10 client.28862 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.577+0000 7fbdd2061740 20 client.28862 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.577+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af094bef0 0x560af094e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.577+0000 7fbdd2061740 4 client.28862 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.577+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af089fdf0 con 0x560af094bef0 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.577+0000 7fbdcda28640 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af094bef0 0x560af094e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.578+0000 7fbdcda28640 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af094bef0 0x560af094e350 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cd53fb95541ad88 server_cookie=ace21d33ae5089e9 in_seq=0 out_seq=0 2024-09-10T21:27:20.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.578+0000 7fbdb17fa640 10 client.28862 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:27:20.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdb17fa640 1 -- 172.21.15.33:0/3358030214 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560af089fdf0 con 0x560af094bef0 2024-09-10T21:27:20.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdb17fa640 10 client.28862 handle_command_reply: tid=0 2024-09-10T21:27:20.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 client.28862 shutdown 2024-09-10T21:27:20.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af093c590 msgr2=0x560af0939700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af093c590 0x560af0939700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0104960 tx=0x7fbdc00cce50 comp rx=0 tx=0).stop 2024-09-10T21:27:20.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 shutdown_connections 2024-09-10T21:27:20.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af093c590 0x560af0939700 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af094bef0 0x560af094e350 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af0927df0 0x560af09391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3358030214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af09274d0 0x560af093bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.580+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 >> 172.21.15.33:0/3358030214 conn(0x560af0925300 msgr2=0x560af092aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.581+0000 7fbdb17fa640 0 client.28862 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:27:20.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.581+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 shutdown_connections 2024-09-10T21:27:20.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.581+0000 7fbdd2061740 1 -- 172.21.15.33:0/3358030214 wait complete. 2024-09-10T21:27:20.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.581+0000 7fbdd2061740 20 client.28862 trim_cache size 0 max 16384 2024-09-10T21:27:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.604+0000 7fbdd2061740 1 Processor -- start 2024-09-10T21:27:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.604+0000 7fbdd2061740 1 -- start start 2024-09-10T21:27:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af094df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af093e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdd2061740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81ac110 con 0x560af093c590 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdd2061740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81adcb0 con 0x560af09274d0 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdd2061740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81ac7e0 con 0x560af0927df0 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdcda28640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af094df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdcda28640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af094df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50170/0 (socket says 172.21.15.33:50170) 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49530/0 (socket says 172.21.15.33:49530) 2024-09-10T21:27:20.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.605+0000 7fbdcda28640 1 -- 172.21.15.33:0/3796117779 learned_addr learned my addr 172.21.15.33:0/3796117779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.606+0000 7fbdcda28640 1 -- 172.21.15.33:0/3796117779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 msgr2=0x560af093e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.606+0000 7fbdcda28640 1 --2- 172.21.15.33:0/3796117779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.606+0000 7fbdcda28640 1 -- 172.21.15.33:0/3796117779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 msgr2=0x560af093e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.606+0000 7fbdcda28640 1 --2- 172.21.15.33:0/3796117779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af093e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.606+0000 7fbdcda28640 1 -- 172.21.15.33:0/3796117779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af0927c20 con 0x560af09274d0 2024-09-10T21:27:20.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.609+0000 7fbdcda28640 1 --2- 172.21.15.33:0/3796117779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af094df00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4011e40 tx=0x7fbdc4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01738b9c8c05595 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.613+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3796117779 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc4012e60 con 0x560af09274d0 2024-09-10T21:27:20.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.613+0000 7fbdb37fe640 1 -- 172.21.15.33:0/3796117779 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc4030af0 con 0x560af09274d0 2024-09-10T21:27:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 -- 172.21.15.33:0/3796117779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 msgr2=0x560af094df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3796117779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af094df00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4011e40 tx=0x7fbdc4014000 comp rx=0 tx=0).stop 2024-09-10T21:27:20.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 -- 172.21.15.33:0/3796117779 shutdown_connections 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3796117779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af093e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3796117779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af093e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 --2- 172.21.15.33:0/3796117779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af094df00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 -- 172.21.15.33:0/3796117779 >> 172.21.15.33:0/3796117779 conn(0x560af0925300 msgr2=0x560af0937a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 -- 172.21.15.33:0/3796117779 shutdown_connections 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.614+0000 7fbdd2061740 1 -- 172.21.15.33:0/3796117779 wait complete. 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.615+0000 7fbdd2061740 1 -- start start 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.615+0000 7fbdd2061740 1 Processor -- start 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.616+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af0944780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.616+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af0944cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.616+0000 7fbdd2061740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af0941570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.616+0000 7fbdd2061740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81ad5b0 con 0x560af093c590 2024-09-10T21:27:20.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.616+0000 7fbdd2061740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81aceb0 con 0x560af09274d0 2024-09-10T21:27:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.616+0000 7fbdd2061740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbdc81aa0f0 con 0x560af0927df0 2024-09-10T21:27:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.616+0000 7fbdd2061740 10 client.0 _ll_register_callbacks cb 0x7fbdc0009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af0944cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af0944cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49544/0 (socket says 172.21.15.33:49544) 2024-09-10T21:27:20.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 -- 172.21.15.33:0/959380515 learned_addr learned my addr 172.21.15.33:0/959380515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 -- 172.21.15.33:0/959380515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 msgr2=0x560af0944780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:27:20.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af0944780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 -- 172.21.15.33:0/959380515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 msgr2=0x560af0941570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af0941570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.617+0000 7fbdcd227640 1 -- 172.21.15.33:0/959380515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af0941c60 con 0x560af0927df0 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.618+0000 7fbdcd227640 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af0944cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8006010 tx=0x7fbdb8012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a7aae320e9ba5eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.618+0000 7fbdb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.618+0000 7fbdb17fa640 1 -- 172.21.15.33:0/959380515 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80041b0 con 0x560af0927df0 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.619+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560af089b8a0 con 0x560af0927df0 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.619+0000 7fbdb17fa640 1 -- 172.21.15.33:0/959380515 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fbdb8004350 con 0x560af0927df0 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.619+0000 7fbdb17fa640 1 -- 172.21.15.33:0/959380515 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fbdb8004550 con 0x560af0927df0 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.619+0000 7fbdd2061740 10 client.28732 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:20.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.619+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x560af092e2a0 con 0x560af0927df0 2024-09-10T21:27:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.622+0000 7fbdb17fa640 1 -- 172.21.15.33:0/959380515 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560af089b8a0 con 0x560af0927df0 2024-09-10T21:27:20.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.622+0000 7fbdb17fa640 1 -- 172.21.15.33:0/959380515 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7fbdb8028020 con 0x560af0927df0 2024-09-10T21:27:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.623+0000 7fbdd2061740 10 client.28732 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.623+0000 7fbdd2061740 10 client.28732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:27:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.623+0000 7fbdd2061740 20 client.28732 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.623+0000 7fbdd2061740 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af092e9a0 0x560af0930dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.623+0000 7fbdd2061740 4 client.28732 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:27:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.623+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560af0898cd0 con 0x560af092e9a0 2024-09-10T21:27:20.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.624+0000 7fbdcda28640 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af092e9a0 0x560af0930dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.629+0000 7fbdcda28640 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af092e9a0 0x560af0930dc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2aeda932068328f server_cookie=1f70489e0b9e7b38 in_seq=0 out_seq=0 2024-09-10T21:27:20.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.629+0000 7fbdb17fa640 10 client.28732 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:27:20.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.632+0000 7fbdb17fa640 1 -- 172.21.15.33:0/959380515 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560af0898cd0 con 0x560af092e9a0 2024-09-10T21:27:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.632+0000 7fbdb17fa640 10 client.28732 handle_command_reply: tid=0 2024-09-10T21:27:20.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.632+0000 7fbdd2061740 1 client.28732 shutdown 2024-09-10T21:27:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.638+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 msgr2=0x560af0944cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.638+0000 7fbdd2061740 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af0944cc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8006010 tx=0x7fbdb8012eb0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.642+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 shutdown_connections 2024-09-10T21:27:20.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.642+0000 7fbdd2061740 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x560af092e9a0 0x560af0930dc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.642+0000 7fbdd2061740 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x560af093c590 0x560af0941570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.642+0000 7fbdd2061740 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560af0927df0 0x560af0944cc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.642+0000 7fbdd2061740 1 --2- 172.21.15.33:0/959380515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x560af09274d0 0x560af0944780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.642+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 >> 172.21.15.33:0/959380515 conn(0x560af0925300 msgr2=0x560af093e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.642+0000 7fbdb17fa640 0 client.28732 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:27:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.645+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 shutdown_connections 2024-09-10T21:27:20.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.645+0000 7fbdd2061740 1 -- 172.21.15.33:0/959380515 wait complete. 2024-09-10T21:27:20.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.648+0000 7fbdd2061740 20 client.28732 trim_cache size 0 max 16384 2024-09-10T21:27:20.652 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:27:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.655+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdb40b5360 msgr2=0x7fbdb40b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.655+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdb40b5360 0x7fbdb40b7820 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x560af092a650 tx=0x7fbdc4005f90 comp rx=0 tx=0).stop 2024-09-10T21:27:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.655+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 msgr2=0x7fbdc8102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.655+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc8102400 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fbdc002f540 tx=0x7fbdc00316f0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 shutdown_connections 2024-09-10T21:27:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fbdb40b5360 0x7fbdb40b7820 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbdc810d130 0x7fbdc8102400 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbdc81045f0 0x7fbdc8101ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 --2- 172.21.15.33:0/2659462227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdc8103cd0 0x7fbdc8101980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 >> 172.21.15.33:0/2659462227 conn(0x7fbdc80fd770 msgr2=0x7fbdc80fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 shutdown_connections 2024-09-10T21:27:20.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:20.657+0000 7fbdcfc8c640 1 -- 172.21.15.33:0/2659462227 wait complete. 2024-09-10T21:27:21.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 453562 37.83 MB/sec execute 501 sec latency 439.132 ms 2024-09-10T21:27:21.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:21 smithi174 ceph-mon[35635]: pgmap v1047: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-09-10T21:27:21.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:21 smithi033 ceph-mon[30860]: pgmap v1047: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-09-10T21:27:22.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:21 smithi104 ceph-mon[35325]: pgmap v1047: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.4 MiB/s wr, 773 op/s 2024-09-10T21:27:22.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 453909 37.79 MB/sec execute 502 sec latency 648.254 ms 2024-09-10T21:27:22.569 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:27:22.901 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:23.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 454684 37.81 MB/sec execute 503 sec latency 80.923 ms 2024-09-10T21:27:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.430+0000 7ff548fff640 1 -- 172.21.15.33:0/3451704261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 msgr2=0x7ff5441072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.430+0000 7ff548fff640 1 --2- 172.21.15.33:0/3451704261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff5441072c0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7ff53400dc90 tx=0x7ff53402e370 comp rx=0 tx=0).stop 2024-09-10T21:27:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.430+0000 7ff548fff640 1 -- 172.21.15.33:0/3451704261 shutdown_connections 2024-09-10T21:27:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.430+0000 7ff548fff640 1 --2- 172.21.15.33:0/3451704261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff544107800 0x7ff54410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.430+0000 7ff548fff640 1 --2- 172.21.15.33:0/3451704261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff5441072c0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.431+0000 7ff548fff640 1 --2- 172.21.15.33:0/3451704261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff544101450 0x7ff544101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.431+0000 7ff548fff640 1 -- 172.21.15.33:0/3451704261 >> 172.21.15.33:0/3451704261 conn(0x7ff544078140 msgr2=0x7ff5440ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:23.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.432+0000 7ff548fff640 1 -- 172.21.15.33:0/3451704261 shutdown_connections 2024-09-10T21:27:23.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.432+0000 7ff548fff640 1 -- 172.21.15.33:0/3451704261 wait complete. 2024-09-10T21:27:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.434+0000 7ff548fff640 1 Processor -- start 2024-09-10T21:27:23.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.434+0000 7ff548fff640 1 -- start start 2024-09-10T21:27:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.435+0000 7ff548fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff544101450 0x7ff54419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.435+0000 7ff548fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff54419fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.435+0000 7ff53bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff54419fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.436+0000 7ff53bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff54419fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33424/0 (socket says 172.21.15.33:33424) 2024-09-10T21:27:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.436+0000 7ff53bfff640 1 -- 172.21.15.33:0/4269322268 learned_addr learned my addr 172.21.15.33:0/4269322268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.436+0000 7ff548fff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff544107800 0x7ff5441a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.436+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff544119420 con 0x7ff544106e60 2024-09-10T21:27:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.436+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441192a0 con 0x7ff544107800 2024-09-10T21:27:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.436+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441195a0 con 0x7ff544101450 2024-09-10T21:27:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.436+0000 7ff542d9d640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff544107800 0x7ff5441a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.437+0000 7ff53bfff640 1 -- 172.21.15.33:0/4269322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff544101450 msgr2=0x7ff54419f8d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:27:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.437+0000 7ff53bfff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff544101450 0x7ff54419f8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.437+0000 7ff53bfff640 1 -- 172.21.15.33:0/4269322268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff544107800 msgr2=0x7ff5441a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.437+0000 7ff53bfff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff544107800 0x7ff5441a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.437+0000 7ff53bfff640 1 -- 172.21.15.33:0/4269322268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5441a5980 con 0x7ff544106e60 2024-09-10T21:27:23.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.437+0000 7ff53bfff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff54419fe10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff53402e960 tx=0x7ff534004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:23.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.438+0000 7ff53b7fe640 1 -- 172.21.15.33:0/4269322268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53400d440 con 0x7ff544106e60 2024-09-10T21:27:23.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.438+0000 7ff53b7fe640 1 -- 172.21.15.33:0/4269322268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5340035f0 con 0x7ff544106e60 2024-09-10T21:27:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.438+0000 7ff53b7fe640 1 -- 172.21.15.33:0/4269322268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534036640 con 0x7ff544106e60 2024-09-10T21:27:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.438+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5441a5c10 con 0x7ff544106e60 2024-09-10T21:27:23.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.439+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5441a5fe0 con 0x7ff544106e60 2024-09-10T21:27:23.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.442+0000 7ff53b7fe640 1 -- 172.21.15.33:0/4269322268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff534005350 con 0x7ff544106e60 2024-09-10T21:27:23.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.445+0000 7ff54b3d4740 1 Processor -- start 2024-09-10T21:27:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.445+0000 7ff54b3d4740 1 -- start start 2024-09-10T21:27:23.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.446+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef134d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.446+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef134d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.446+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef134d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33436/0 (socket says 172.21.15.33:33436) 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.447+0000 7ff53b7fe640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff5300b5610 0x7ff5300b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.447+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef1f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.447+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef1f9a0 0x55e5eef21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.447+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e5eee89ea0 con 0x55e5eef130f0 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.447+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e5eee792d0 con 0x55e5eef1f9a0 2024-09-10T21:27:23.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.447+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5eee7d0e0 con 0x55e5eef13a10 2024-09-10T21:27:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.447+0000 7ff542d9d640 1 -- 172.21.15.33:0/332940632 learned_addr learned my addr 172.21.15.33:0/332940632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff54259c640 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef1f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff53bfff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff5300b5610 0x7ff5300b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff542d9d640 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef1f9a0 0x55e5eef21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff542d9d640 1 -- 172.21.15.33:0/332940632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 msgr2=0x55e5eef1f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff542d9d640 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef1f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff542d9d640 1 -- 172.21.15.33:0/332940632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef1f9a0 msgr2=0x55e5eef21d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff542d9d640 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef1f9a0 0x55e5eef21d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff542d9d640 1 -- 172.21.15.33:0/332940632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5eebd7650 con 0x55e5eef130f0 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff54259c640 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff53b7fe640 1 -- 172.21.15.33:0/4269322268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff534105b20 con 0x7ff544106e60 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.448+0000 7ff542d9d640 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef134d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009270 tx=0x7ff53c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bab8ed629c568f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.449+0000 7ff53bfff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff5300b5610 0x7ff5300b7ad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5340c94c0 tx=0x7ff5340c8980 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.449+0000 7ff538ff9640 1 -- 172.21.15.33:0/332940632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00e710 con 0x55e5eef130f0 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.449+0000 7ff538ff9640 1 -- 172.21.15.33:0/332940632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c00ed60 con 0x55e5eef130f0 2024-09-10T21:27:23.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff538ff9640 1 -- 172.21.15.33:0/332940632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c010cd0 con 0x55e5eef130f0 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff54b3d4740 1 -- 172.21.15.33:0/332940632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 msgr2=0x55e5eef134d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef134d0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009270 tx=0x7ff53c009b80 comp rx=0 tx=0).stop 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff54b3d4740 1 -- 172.21.15.33:0/332940632 shutdown_connections 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef1f9a0 0x55e5eef21d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef1f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/332940632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef134d0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.450+0000 7ff54b3d4740 1 -- 172.21.15.33:0/332940632 >> 172.21.15.33:0/332940632 conn(0x55e5eef10f20 msgr2=0x55e5eef120c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.451+0000 7ff54b3d4740 1 -- 172.21.15.33:0/332940632 shutdown_connections 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.451+0000 7ff54b3d4740 1 -- 172.21.15.33:0/332940632 wait complete. 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.451+0000 7ff54b3d4740 1 -- start start 2024-09-10T21:27:23.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.451+0000 7ff54b3d4740 1 Processor -- start 2024-09-10T21:27:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.452+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef130f0 0x55e5eef1f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.452+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 0x55e5eef1c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.452+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.452+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e5eecda700 con 0x55e5eef1f9a0 2024-09-10T21:27:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.452+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e5eee14160 con 0x55e5eef13a10 2024-09-10T21:27:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.452+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5eef22670 con 0x55e5eef130f0 2024-09-10T21:27:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.452+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef130f0 0x55e5eef1f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff54b3d4740 10 client.0 _ll_register_callbacks cb 0x55e5eee2d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef130f0 0x55e5eef1f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:49592/0 (socket says 172.21.15.33:49592) 2024-09-10T21:27:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 -- 172.21.15.33:0/2057557285 learned_addr learned my addr 172.21.15.33:0/2057557285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:23.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 -- 172.21.15.33:0/2057557285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef130f0 msgr2=0x55e5eef1f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef130f0 0x55e5eef1f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 -- 172.21.15.33:0/2057557285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 msgr2=0x55e5eef1c630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 0x55e5eef1c630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.453+0000 7ff542d9d640 1 -- 172.21.15.33:0/2057557285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5eee41c90 con 0x55e5eef1f9a0 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.455+0000 7ff542d9d640 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1cb90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff53c010850 tx=0x7ff53c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fffb3b898e200f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.455+0000 7ff529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.455+0000 7ff529ffb640 1 -- 172.21.15.33:0/2057557285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c010630 con 0x55e5eef1f9a0 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.455+0000 7ff529ffb640 1 -- 172.21.15.33:0/2057557285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff53c00e510 con 0x55e5eef1f9a0 2024-09-10T21:27:23.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.455+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff53c00ed60 con 0x55e5eef1f9a0 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.456+0000 7ff529ffb640 1 -- 172.21.15.33:0/2057557285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff53c00e710 con 0x55e5eef1f9a0 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.456+0000 7ff54b3d4740 10 client.19542 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.456+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55e5eef2df50 con 0x55e5eef1f9a0 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff529ffb640 1 -- 172.21.15.33:0/2057557285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c012450 con 0x55e5eef1f9a0 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff529ffb640 1 -- 172.21.15.33:0/2057557285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7ff53c01e430 con 0x55e5eef1f9a0 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff54b3d4740 10 client.19542 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff54b3d4740 10 client.19542 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff54b3d4740 20 client.19542 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef2f2a0 0x55e5eef31690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff54b3d4740 4 client.19542 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:23.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.457+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff53c00ed60 con 0x55e5eef2f2a0 2024-09-10T21:27:23.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.459+0000 7ff54259c640 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef2f2a0 0x55e5eef31690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.461+0000 7ff54259c640 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef2f2a0 0x55e5eef31690 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a310714e8cbea0a5 server_cookie=b0c5e443dd058a2c in_seq=0 out_seq=0 2024-09-10T21:27:23.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.461+0000 7ff529ffb640 10 client.19542 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.462+0000 7ff529ffb640 1 -- 172.21.15.33:0/2057557285 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff53c00ed60 con 0x55e5eef2f2a0 2024-09-10T21:27:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.462+0000 7ff529ffb640 10 client.19542 handle_command_reply: tid=0 2024-09-10T21:27:23.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.462+0000 7ff54b3d4740 1 client.19542 shutdown 2024-09-10T21:27:23.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 msgr2=0x55e5eef1cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1cb90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7ff53c010850 tx=0x7ff53c009da0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 shutdown_connections 2024-09-10T21:27:23.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef2f2a0 0x55e5eef31690 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1cb90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 0x55e5eef1c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/2057557285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef130f0 0x55e5eef1f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 >> 172.21.15.33:0/2057557285 conn(0x55e5eef10f20 msgr2=0x55e5eef16a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.463+0000 7ff529ffb640 0 client.19542 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.464+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 shutdown_connections 2024-09-10T21:27:23.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.464+0000 7ff54b3d4740 1 -- 172.21.15.33:0/2057557285 wait complete. 2024-09-10T21:27:23.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.464+0000 7ff54b3d4740 20 client.19542 trim_cache size 0 max 16384 2024-09-10T21:27:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.486+0000 7ff54b3d4740 1 Processor -- start 2024-09-10T21:27:23.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.486+0000 7ff54b3d4740 1 -- start start 2024-09-10T21:27:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.486+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef130f0 0x55e5eef16ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef0be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef130f0 0x55e5eef16ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef130f0 0x55e5eef16ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50218/0 (socket says 172.21.15.33:50218) 2024-09-10T21:27:23.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 -- 172.21.15.33:0/1478946862 learned_addr learned my addr 172.21.15.33:0/1478946862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441b4ad0 con 0x55e5eef1f9a0 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441b6670 con 0x55e5eef130f0 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441b51a0 con 0x55e5eef13a10 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 -- 172.21.15.33:0/1478946862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 msgr2=0x55e5eef0be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef0be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 -- 172.21.15.33:0/1478946862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 msgr2=0x55e5eef1aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.487+0000 7ff542d9d640 1 -- 172.21.15.33:0/1478946862 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5eef119a0 con 0x55e5eef130f0 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.488+0000 7ff542d9d640 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef130f0 0x55e5eef16ff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff53c01e830 tx=0x7ff53c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62b05ea4c76bb9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:23.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.488+0000 7ff52bfff640 1 -- 172.21.15.33:0/1478946862 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c010880 con 0x55e5eef130f0 2024-09-10T21:27:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.488+0000 7ff52bfff640 1 -- 172.21.15.33:0/1478946862 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c00ed60 con 0x55e5eef130f0 2024-09-10T21:27:23.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.488+0000 7ff52bfff640 1 -- 172.21.15.33:0/1478946862 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c0104d0 con 0x55e5eef130f0 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.489+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef130f0 msgr2=0x55e5eef16ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.489+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef130f0 0x55e5eef16ff0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff53c01e830 tx=0x7ff53c009f90 comp rx=0 tx=0).stop 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.489+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 shutdown_connections 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.489+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef1f9a0 0x55e5eef1aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.489+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef13a10 0x55e5eef0be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.489+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1478946862 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef130f0 0x55e5eef16ff0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.489+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 >> 172.21.15.33:0/1478946862 conn(0x55e5eef10f20 msgr2=0x55e5eef1c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:23.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.490+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 shutdown_connections 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.490+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1478946862 wait complete. 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.490+0000 7ff54b3d4740 1 -- start start 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.490+0000 7ff54b3d4740 1 Processor -- start 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef27b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 0x55e5eef28080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff54b3d4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef1f9a0 0x55e5eef22910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef27b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441b5f70 con 0x55e5eef130f0 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441b5870 con 0x55e5eef13a10 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff542d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef27b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33458/0 (socket says 172.21.15.33:33458) 2024-09-10T21:27:23.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff54b3d4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5441b2a90 con 0x55e5eef1f9a0 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff542d9d640 1 -- 172.21.15.33:0/1893234664 learned_addr learned my addr 172.21.15.33:0/1893234664 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.491+0000 7ff54b3d4740 10 client.0 _ll_register_callbacks cb 0x7ff53c00ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff542d9d640 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef1f9a0 0x55e5eef22910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff54259c640 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 0x55e5eef28080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff542d9d640 1 -- 172.21.15.33:0/1893234664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef1f9a0 msgr2=0x55e5eef22910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff542d9d640 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef1f9a0 0x55e5eef22910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff542d9d640 1 -- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 msgr2=0x55e5eef28080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff542d9d640 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 0x55e5eef28080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff542d9d640 1 -- 172.21.15.33:0/1893234664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5eef22f50 con 0x55e5eef130f0 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff542d9d640 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef27b40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff53c000f80 tx=0x7ff53c04a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c134e3c7e9ef808 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:23.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.492+0000 7ff529ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.493+0000 7ff529ffb640 1 -- 172.21.15.33:0/1893234664 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00eac0 con 0x55e5eef130f0 2024-09-10T21:27:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.493+0000 7ff529ffb640 1 -- 172.21.15.33:0/1893234664 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff53c051040 con 0x55e5eef130f0 2024-09-10T21:27:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.493+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff53c00ed60 con 0x55e5eef130f0 2024-09-10T21:27:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.493+0000 7ff529ffb640 1 -- 172.21.15.33:0/1893234664 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ff53c010460 con 0x55e5eef130f0 2024-09-10T21:27:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.493+0000 7ff54b3d4740 10 client.19545 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:23.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.493+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55e5eef9c9b0 con 0x55e5eef130f0 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff529ffb640 1 -- 172.21.15.33:0/1893234664 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff53c004e40 con 0x55e5eef130f0 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff529ffb640 1 -- 172.21.15.33:0/1893234664 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7ff53c0051d0 con 0x55e5eef130f0 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff54b3d4740 10 client.19545 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff54b3d4740 10 client.19545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff54b3d4740 20 client.19545 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef9d050 0x55e5eef9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff54b3d4740 4 client.19545 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:27:23.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.494+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff53c00ed60 con 0x55e5eef9d050 2024-09-10T21:27:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.498+0000 7ff54259c640 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef9d050 0x55e5eef9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff529ffb640 10 client.19545 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:23.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff54259c640 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef9d050 0x55e5eef9f470 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810617a7e9dfdcce server_cookie=c228f5da95a9384f in_seq=0 out_seq=0 2024-09-10T21:27:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff529ffb640 1 -- 172.21.15.33:0/1893234664 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff53c00ed60 con 0x55e5eef9d050 2024-09-10T21:27:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff529ffb640 10 client.19545 handle_command_reply: tid=0 2024-09-10T21:27:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff54b3d4740 1 client.19545 shutdown 2024-09-10T21:27:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 msgr2=0x55e5eef27b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef27b40 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7ff53c000f80 tx=0x7ff53c04a040 comp rx=0 tx=0).stop 2024-09-10T21:27:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.499+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 shutdown_connections 2024-09-10T21:27:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e5eef9d050 0x55e5eef9f470 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5eef1f9a0 0x55e5eef22910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e5eef13a10 0x55e5eef28080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff529ffb640 0 client.19545 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff54b3d4740 1 --2- 172.21.15.33:0/1893234664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e5eef130f0 0x55e5eef27b40 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 >> 172.21.15.33:0/1893234664 conn(0x55e5eef10f20 msgr2=0x55e5eef1dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 shutdown_connections 2024-09-10T21:27:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.500+0000 7ff54b3d4740 1 -- 172.21.15.33:0/1893234664 wait complete. 2024-09-10T21:27:23.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.501+0000 7ff54b3d4740 20 client.19545 trim_cache size 0 max 16384 2024-09-10T21:27:23.505 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff5300b5610 msgr2=0x7ff5300b7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff5300b5610 0x7ff5300b7ad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff5340c94c0 tx=0x7ff5340c8980 comp rx=0 tx=0).stop 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 msgr2=0x7ff54419fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff54419fe10 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7ff53402e960 tx=0x7ff534004da0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 shutdown_connections 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff5300b5610 0x7ff5300b7ad0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff544107800 0x7ff5441a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff544106e60 0x7ff54419fe10 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 --2- 172.21.15.33:0/4269322268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff544101450 0x7ff54419f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:23.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.502+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 >> 172.21.15.33:0/4269322268 conn(0x7ff544078140 msgr2=0x7ff5441036f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.503+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 shutdown_connections 2024-09-10T21:27:23.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:23.503+0000 7ff548fff640 1 -- 172.21.15.33:0/4269322268 wait complete. 2024-09-10T21:27:23.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:23 smithi174 ceph-mon[35635]: pgmap v1048: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.5 MiB/s wr, 637 op/s 2024-09-10T21:27:23.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:23 smithi033 ceph-mon[30860]: pgmap v1048: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.5 MiB/s wr, 637 op/s 2024-09-10T21:27:24.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:23 smithi104 ceph-mon[35325]: pgmap v1048: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 8.5 MiB/s wr, 637 op/s 2024-09-10T21:27:24.084 INFO:tasks.workunit.client.0.smithi033.stdout: 10 455362 37.81 MB/sec execute 504 sec latency 98.356 ms 2024-09-10T21:27:25.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 455901 37.77 MB/sec execute 505 sec latency 93.989 ms 2024-09-10T21:27:25.225 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:27:25.226 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:27:25.585 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:25.879 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:25 smithi033 ceph-mon[30860]: pgmap v1049: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 802 op/s 2024-09-10T21:27:25.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:25 smithi174 ceph-mon[35635]: pgmap v1049: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 802 op/s 2024-09-10T21:27:26.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:25 smithi104 ceph-mon[35325]: pgmap v1049: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 802 op/s 2024-09-10T21:27:26.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 456570 37.78 MB/sec execute 506 sec latency 69.880 ms 2024-09-10T21:27:26.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2278167456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64103b60 msgr2=0x7f2c64103f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2278167456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64103b60 0x7f2c64103f40 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009900 tx=0x7f2c6002a410 comp rx=0 tx=0).stop 2024-09-10T21:27:26.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2278167456 shutdown_connections 2024-09-10T21:27:26.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2278167456 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c6410d110 0x7f2c6410f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2278167456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c64104560 0x7f2c6410cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2278167456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64103b60 0x7f2c64103f40 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2278167456 >> 172.21.15.33:0/2278167456 conn(0x7f2c640fd750 msgr2=0x7f2c640ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2278167456 shutdown_connections 2024-09-10T21:27:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.137+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2278167456 wait complete. 2024-09-10T21:27:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 Processor -- start 2024-09-10T21:27:26.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 -- start start 2024-09-10T21:27:26.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c64103b60 0x7f2c64074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64104560 0x7f2c64074c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c6410d110 0x7f2c6406da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64112de0 con 0x7f2c64104560 2024-09-10T21:27:26.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64112c60 con 0x7f2c6410d110 2024-09-10T21:27:26.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6bf82640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c64112f60 con 0x7f2c64103b60 2024-09-10T21:27:26.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6951d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64104560 0x7f2c64074c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6951d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64104560 0x7f2c64074c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33466/0 (socket says 172.21.15.33:33466) 2024-09-10T21:27:26.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6951d640 1 -- 172.21.15.33:0/2729536229 learned_addr learned my addr 172.21.15.33:0/2729536229 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:26.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c64103b60 0x7f2c64074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.138+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c6410d110 0x7f2c6406da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6951d640 1 -- 172.21.15.33:0/2729536229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c64103b60 msgr2=0x7f2c64074750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6951d640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c64103b60 0x7f2c64074750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6951d640 1 -- 172.21.15.33:0/2729536229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c6410d110 msgr2=0x7f2c6406da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6951d640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c6410d110 0x7f2c6406da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6951d640 1 -- 172.21.15.33:0/2729536229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c6406e340 con 0x7f2c64104560 2024-09-10T21:27:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6951d640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64104560 0x7f2c64074c90 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800e9f0 tx=0x7f2c5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c56ffd640 1 -- 172.21.15.33:0/2729536229 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5800ccc0 con 0x7f2c64104560 2024-09-10T21:27:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6406e630 con 0x7f2c64104560 2024-09-10T21:27:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c56ffd640 1 -- 172.21.15.33:0/2729536229 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c580108e0 con 0x7f2c64104560 2024-09-10T21:27:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c56ffd640 1 -- 172.21.15.33:0/2729536229 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c58018880 con 0x7f2c64104560 2024-09-10T21:27:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.139+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c640735e0 con 0x7f2c64104560 2024-09-10T21:27:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 Processor -- start 2024-09-10T21:27:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c56ffd640 1 -- 172.21.15.33:0/2729536229 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f2c58010430 con 0x7f2c64104560 2024-09-10T21:27:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 -- start start 2024-09-10T21:27:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd44660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd46f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d90fcaae60 con 0x55d90fd34800 2024-09-10T21:27:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d90fc9a290 con 0x55d90fd33ee0 2024-09-10T21:27:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6e357740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d90fc9e0a0 con 0x55d90fd44ba0 2024-09-10T21:27:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6a51f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6a51f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd342c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50230/0 (socket says 172.21.15.33:50230) 2024-09-10T21:27:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.143+0000 7f2c6a51f640 1 -- 172.21.15.33:0/1812133111 learned_addr learned my addr 172.21.15.33:0/1812133111 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd44660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6951d640 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd46f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6a51f640 1 -- 172.21.15.33:0/1812133111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 msgr2=0x55d90fd46f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd46f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6a51f640 1 -- 172.21.15.33:0/1812133111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 msgr2=0x55d90fd44660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd44660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6a51f640 1 -- 172.21.15.33:0/1812133111 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90f9f8930 con 0x55d90fd33ee0 2024-09-10T21:27:26.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd44660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:26.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6951d640 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd46f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:26.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd342c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011440 tx=0x7f2c5c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e44790589b90bf60 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:26.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c4ffff640 1 -- 172.21.15.33:0/1812133111 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c003c30 con 0x55d90fd33ee0 2024-09-10T21:27:26.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c56ffd640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c500b52e0 0x7f2c500b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c4ffff640 1 -- 172.21.15.33:0/1812133111 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c5c003dd0 con 0x55d90fd33ee0 2024-09-10T21:27:26.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.144+0000 7f2c4ffff640 1 -- 172.21.15.33:0/1812133111 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c5c00fa60 con 0x55d90fd33ee0 2024-09-10T21:27:26.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c500b52e0 0x7f2c500b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c56ffd640 1 -- 172.21.15.33:0/2729536229 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2c580e5fb0 con 0x7f2c64104560 2024-09-10T21:27:26.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 -- 172.21.15.33:0/1812133111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 msgr2=0x55d90fd342c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd342c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c011440 tx=0x7f2c5c011900 comp rx=0 tx=0).stop 2024-09-10T21:27:26.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c500b52e0 0x7f2c500b77a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005ab0 tx=0x7f2c60005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:26.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 -- 172.21.15.33:0/1812133111 shutdown_connections 2024-09-10T21:27:26.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd46f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd44660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 --2- 172.21.15.33:0/1812133111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd342c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 -- 172.21.15.33:0/1812133111 >> 172.21.15.33:0/1812133111 conn(0x55d90fd31d10 msgr2=0x55d90fd32eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:26.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 -- 172.21.15.33:0/1812133111 shutdown_connections 2024-09-10T21:27:26.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.145+0000 7f2c6e357740 1 -- 172.21.15.33:0/1812133111 wait complete. 2024-09-10T21:27:26.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.146+0000 7f2c6e357740 1 -- start start 2024-09-10T21:27:26.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.146+0000 7f2c6e357740 1 Processor -- start 2024-09-10T21:27:26.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd41b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd42050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd42590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c69d1e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd41b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c69d1e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd41b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50244/0 (socket says 172.21.15.33:50244) 2024-09-10T21:27:26.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c69d1e640 1 -- 172.21.15.33:0/804390950 learned_addr learned my addr 172.21.15.33:0/804390950 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:26.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55d90fafb890 con 0x55d90fd34800 2024-09-10T21:27:26.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55d90fc34d40 con 0x55d90fd33ee0 2024-09-10T21:27:26.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d90fd47850 con 0x55d90fd44ba0 2024-09-10T21:27:26.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6951d640 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd42050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.147+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd42590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c69d1e640 1 -- 172.21.15.33:0/804390950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 msgr2=0x55d90fd42590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd42590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c69d1e640 1 -- 172.21.15.33:0/804390950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 msgr2=0x55d90fd42050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd42050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c69d1e640 1 -- 172.21.15.33:0/804390950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90fd50c00 con 0x55d90fd33ee0 2024-09-10T21:27:26.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd41b10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009900 tx=0x7f2c60009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d3fe21e856ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:27:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c6e357740 10 client.0 _ll_register_callbacks cb 0x55d90fc98c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/804390950 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6004d070 con 0x55d90fd33ee0 2024-09-10T21:27:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c5c003dd0 con 0x55d90fd33ee0 2024-09-10T21:27:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.148+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/804390950 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2c60004970 con 0x55d90fd33ee0 2024-09-10T21:27:26.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/804390950 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f2c60059050 con 0x55d90fd33ee0 2024-09-10T21:27:26.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 10 client.28889 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:26.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=89}) -- 0x55d90fd53380 con 0x55d90fd33ee0 2024-09-10T21:27:26.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/804390950 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7f2c60004b70 con 0x55d90fd33ee0 2024-09-10T21:27:26.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 10 client.28889 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:26.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 10 client.28889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:26.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 20 client.28889 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:26.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fd54730 0x55d90fd56bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 4 client.28889 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:26.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2c5c003dd0 con 0x55d90fd54730 2024-09-10T21:27:26.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.149+0000 7f2c6951d640 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fd54730 0x55d90fd56bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.150+0000 7f2c4d7fa640 10 client.28889 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.150+0000 7f2c6951d640 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fd54730 0x55d90fd56bc0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf790fe78c5eb4c server_cookie=ff64699435115a88 in_seq=0 out_seq=0 2024-09-10T21:27:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.150+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/804390950 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f2c5c003dd0 con 0x55d90fd54730 2024-09-10T21:27:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.150+0000 7f2c4d7fa640 10 client.28889 handle_command_reply: tid=0 2024-09-10T21:27:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.150+0000 7f2c6e357740 1 client.28889 shutdown 2024-09-10T21:27:26.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 msgr2=0x55d90fd41b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd41b10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009900 tx=0x7f2c60009260 comp rx=0 tx=0).stop 2024-09-10T21:27:26.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 shutdown_connections 2024-09-10T21:27:26.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fd54730 0x55d90fd56bc0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd42590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd42050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 --2- 172.21.15.33:0/804390950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd41b10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 >> 172.21.15.33:0/804390950 conn(0x55d90fd31d10 msgr2=0x55d90fd30f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:26.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c4d7fa640 0 client.28889 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:26.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 shutdown_connections 2024-09-10T21:27:26.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 1 -- 172.21.15.33:0/804390950 wait complete. 2024-09-10T21:27:26.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.151+0000 7f2c6e357740 20 client.28889 trim_cache size 0 max 16384 2024-09-10T21:27:26.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 Processor -- start 2024-09-10T21:27:26.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 -- start start 2024-09-10T21:27:26.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd33ee0 0x55d90fd43850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd34800 0x55d90fd5c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd5c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641b49a0 con 0x55d90fd33ee0 2024-09-10T21:27:26.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641b6540 con 0x55d90fd34800 2024-09-10T21:27:26.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6e357740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641b5070 con 0x55d90fd44ba0 2024-09-10T21:27:26.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd33ee0 0x55d90fd43850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6951d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd34800 0x55d90fd5c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd33ee0 0x55d90fd43850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33492/0 (socket says 172.21.15.33:33492) 2024-09-10T21:27:26.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 -- 172.21.15.33:0/3556837671 learned_addr learned my addr 172.21.15.33:0/3556837671 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:26.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd5c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 -- 172.21.15.33:0/3556837671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 msgr2=0x55d90fd5c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd5c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 -- 172.21.15.33:0/3556837671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd34800 msgr2=0x55d90fd5c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd34800 0x55d90fd5c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c69d1e640 1 -- 172.21.15.33:0/3556837671 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90fd34630 con 0x55d90fd33ee0 2024-09-10T21:27:26.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd5c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:26.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.171+0000 7f2c6951d640 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd34800 0x55d90fd5c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:26.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.172+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd33ee0 0x55d90fd43850 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005010 tx=0x7f2c60049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16974e4a74cd19dc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:26.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.172+0000 7f2c4f7fe640 1 -- 172.21.15.33:0/3556837671 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6004d040 con 0x55d90fd33ee0 2024-09-10T21:27:26.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.172+0000 7f2c4f7fe640 1 -- 172.21.15.33:0/3556837671 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c600036c0 con 0x55d90fd33ee0 2024-09-10T21:27:26.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.172+0000 7f2c4f7fe640 1 -- 172.21.15.33:0/3556837671 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60035040 con 0x55d90fd33ee0 2024-09-10T21:27:26.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.172+0000 7f2c6e357740 1 -- 172.21.15.33:0/3556837671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd33ee0 msgr2=0x55d90fd43850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.172+0000 7f2c6e357740 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd33ee0 0x55d90fd43850 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005010 tx=0x7f2c60049000 comp rx=0 tx=0).stop 2024-09-10T21:27:26.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 -- 172.21.15.33:0/3556837671 shutdown_connections 2024-09-10T21:27:26.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd5c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd34800 0x55d90fd5c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 --2- 172.21.15.33:0/3556837671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd33ee0 0x55d90fd43850 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 -- 172.21.15.33:0/3556837671 >> 172.21.15.33:0/3556837671 conn(0x55d90fd31d10 msgr2=0x55d90fd43140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 -- 172.21.15.33:0/3556837671 shutdown_connections 2024-09-10T21:27:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 -- 172.21.15.33:0/3556837671 wait complete. 2024-09-10T21:27:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 -- start start 2024-09-10T21:27:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 Processor -- start 2024-09-10T21:27:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd4d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd49ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.173+0000 7f2c6e357740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd4a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6e357740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641b5e40 con 0x55d90fd34800 2024-09-10T21:27:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6e357740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641b5740 con 0x55d90fd33ee0 2024-09-10T21:27:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6e357740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c641b2960 con 0x55d90fd44ba0 2024-09-10T21:27:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6e357740 10 client.0 _ll_register_callbacks cb 0x7f2c58005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd49ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c69d1e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd4d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd49ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33494/0 (socket says 172.21.15.33:33494) 2024-09-10T21:27:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 -- 172.21.15.33:0/2939830616 learned_addr learned my addr 172.21.15.33:0/2939830616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6a51f640 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd4a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 -- 172.21.15.33:0/2939830616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 msgr2=0x55d90fd4a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd4a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 -- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 msgr2=0x55d90fd4d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd4d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 -- 172.21.15.33:0/2939830616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90fd4a910 con 0x55d90fd34800 2024-09-10T21:27:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd4d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6951d640 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd49ae0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2c580ac700 tx=0x7f2c5801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb3421e9aa861b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:26.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/2939830616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580ac280 con 0x55d90fd34800 2024-09-10T21:27:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c600036c0 con 0x55d90fd34800 2024-09-10T21:27:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/2939830616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f2c580a89e0 con 0x55d90fd34800 2024-09-10T21:27:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/2939830616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f2c580ace70 con 0x55d90fd34800 2024-09-10T21:27:26.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6e357740 10 client.19563 fetch_fsmap learned FSMap version 89 2024-09-10T21:27:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.174+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=4+}) -- 0x55d90fdc2810 con 0x55d90fd34800 2024-09-10T21:27:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/2939830616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c600036c0 con 0x55d90fd34800 2024-09-10T21:27:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/2939830616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 89) ==== 8374+0+0 (secure 0 0 0) 0x7f2c580c3020 con 0x55d90fd34800 2024-09-10T21:27:26.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c6e357740 10 client.19563 fetch_fsmap finished waiting for FSMap version 89 2024-09-10T21:27:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c6e357740 10 client.19563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c6e357740 20 client.19563 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c6e357740 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fdc2f40 0x55d90fdc5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c6e357740 4 client.19563 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:27:26.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d90fcacdb0 con 0x55d90fdc2f40 2024-09-10T21:27:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fdc2f40 0x55d90fdc5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c69d1e640 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fdc2f40 0x55d90fdc5360 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d95576a58e81a77 server_cookie=9f0540df27b98dc4 in_seq=0 out_seq=0 2024-09-10T21:27:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.175+0000 7f2c4d7fa640 10 client.19563 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c4d7fa640 1 -- 172.21.15.33:0/2939830616 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d90fcacdb0 con 0x55d90fdc2f40 2024-09-10T21:27:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c4d7fa640 10 client.19563 handle_command_reply: tid=0 2024-09-10T21:27:26.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 client.19563 shutdown 2024-09-10T21:27:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 msgr2=0x55d90fd49ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd49ae0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f2c580ac700 tx=0x7f2c5801f000 comp rx=0 tx=0).stop 2024-09-10T21:27:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 shutdown_connections 2024-09-10T21:27:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55d90fdc2f40 0x55d90fdc5360 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d90fd44ba0 0x55d90fd4a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55d90fd34800 0x55d90fd49ae0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 --2- 172.21.15.33:0/2939830616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55d90fd33ee0 0x55d90fd4d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c4d7fa640 0 client.19563 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 >> 172.21.15.33:0/2939830616 conn(0x55d90fd31d10 msgr2=0x55d90fd43960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 shutdown_connections 2024-09-10T21:27:26.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 1 -- 172.21.15.33:0/2939830616 wait complete. 2024-09-10T21:27:26.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.177+0000 7f2c6e357740 20 client.19563 trim_cache size 0 max 16384 2024-09-10T21:27:26.385 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:27:26.385 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:27:26.386 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "b22ec403-44be-4b2f-9c71-e7053f2ad4b0", 2024-09-10T21:27:26.386 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:27:26.386 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:27:26.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c500b52e0 msgr2=0x7f2c500b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c500b52e0 0x7f2c500b77a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2c60005ab0 tx=0x7f2c60005a20 comp rx=0 tx=0).stop 2024-09-10T21:27:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64104560 msgr2=0x7f2c64074c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64104560 0x7f2c64074c90 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2c5800e9f0 tx=0x7f2c5800edf0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 shutdown_connections 2024-09-10T21:27:26.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f2c500b52e0 0x7f2c500b77a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2c6410d110 0x7f2c6406da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2c64104560 0x7f2c64074c90 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 --2- 172.21.15.33:0/2729536229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c64103b60 0x7f2c64074750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 >> 172.21.15.33:0/2729536229 conn(0x7f2c640fd750 msgr2=0x7f2c640fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 shutdown_connections 2024-09-10T21:27:26.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:26.178+0000 7f2c6bf82640 1 -- 172.21.15.33:0/2729536229 wait complete. 2024-09-10T21:27:26.884 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:26 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:27:26.884 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:26 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:26.884 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:26 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:27:26.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:26 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:27:26.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:26 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:26.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:26 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:27:27.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:26 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:27:27.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:26 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:27.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:26 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:27:27.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 456860 37.74 MB/sec execute 507 sec latency 899.868 ms 2024-09-10T21:27:27.237 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:27:27.586 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:27:27.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:27:27.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:27:27.871 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:27:27.872 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:27:27.872 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:27.872 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:27:27.872 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: pgmap v1050: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.1 MiB/s wr, 677 op/s 2024-09-10T21:27:27.872 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:27.872 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:27 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:27:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:27:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:27:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:27:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:27:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: pgmap v1050: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.1 MiB/s wr, 677 op/s 2024-09-10T21:27:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:27 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:27:28.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:27:28.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:27:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:27:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:27:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: pgmap v1050: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.1 MiB/s wr, 677 op/s 2024-09-10T21:27:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:28.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:27 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 457432 37.72 MB/sec execute 508 sec latency 1139.040 ms 2024-09-10T21:27:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 -- 172.21.15.33:0/3532865941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201043c0 msgr2=0x7f492010c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 --2- 172.21.15.33:0/3532865941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201043c0 0x7f492010c9a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f490c009900 tx=0x7f490c02a410 comp rx=0 tx=0).stop 2024-09-10T21:27:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 -- 172.21.15.33:0/3532865941 shutdown_connections 2024-09-10T21:27:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 --2- 172.21.15.33:0/3532865941 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f492010cf70 0x7f492010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 --2- 172.21.15.33:0/3532865941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201043c0 0x7f492010c9a0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 --2- 172.21.15.33:0/3532865941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49201039c0 0x7f4920103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 -- 172.21.15.33:0/3532865941 >> 172.21.15.33:0/3532865941 conn(0x7f49200fd5b0 msgr2=0x7f49200ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 -- 172.21.15.33:0/3532865941 shutdown_connections 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.102+0000 7f4925fa2640 1 -- 172.21.15.33:0/3532865941 wait complete. 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4925fa2640 1 Processor -- start 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4925fa2640 1 -- start start 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4925fa2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201039c0 0x7f492019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4925fa2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49201043c0 0x7f492019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4925fa2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f492010cf70 0x7f49201a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4924fa0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201039c0 0x7f492019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4925fa2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f4920101e40 con 0x7f49201039c0 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4924fa0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201039c0 0x7f492019ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33512/0 (socket says 172.21.15.33:33512) 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.103+0000 7f4924fa0640 1 -- 172.21.15.33:0/712807789 learned_addr learned my addr 172.21.15.33:0/712807789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:28.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f4920101cc0 con 0x7f492010cf70 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f491ffff640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49201043c0 0x7f492019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4920101fc0 con 0x7f49201043c0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f49257a1640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f492010cf70 0x7f49201a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4924fa0640 1 -- 172.21.15.33:0/712807789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49201043c0 msgr2=0x7f492019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4924fa0640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49201043c0 0x7f492019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4924fa0640 1 -- 172.21.15.33:0/712807789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f492010cf70 msgr2=0x7f49201a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4924fa0640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f492010cf70 0x7f49201a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4924fa0640 1 -- 172.21.15.33:0/712807789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49201a12d0 con 0x7f49201039c0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4924fa0640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201039c0 0x7f492019ca60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9f0 tx=0x7f491400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f491dffb640 1 -- 172.21.15.33:0/712807789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491400cd80 con 0x7f49201039c0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49201a15c0 con 0x7f49201039c0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f491dffb640 1 -- 172.21.15.33:0/712807789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49140054d0 con 0x7f49201039c0 2024-09-10T21:27:28.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.104+0000 7f491dffb640 1 -- 172.21.15.33:0/712807789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4914010690 con 0x7f49201039c0 2024-09-10T21:27:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.105+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49201a7c80 con 0x7f49201039c0 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.108+0000 7f491dffb640 1 -- 172.21.15.33:0/712807789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f4914004e90 con 0x7f49201039c0 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f491dffb640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49100b5410 0x7f49100b78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 Processor -- start 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 -- start start 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f491dffb640 1 -- 172.21.15.33:0/712807789 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f49140e5f60 con 0x7f49201039c0 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f491ffff640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49100b5410 0x7f49100b78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45eca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5611f4562e50 con 0x5611f45f8f00 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5611f4552280 con 0x5611f45ec650 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.109+0000 7f49295d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611f4556090 con 0x5611f45ecf70 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33528/0 (socket says 172.21.15.33:33528) 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f4924fa0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45eca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 -- 172.21.15.33:0/4089233723 learned_addr learned my addr 172.21.15.33:0/4089233723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f491ffff640 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 -- 172.21.15.33:0/4089233723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 msgr2=0x5611f45f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 -- 172.21.15.33:0/4089233723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 msgr2=0x5611f45eca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45eca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 -- 172.21.15.33:0/4089233723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f42b0650 con 0x5611f45f8f00 2024-09-10T21:27:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f491ffff640 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:27:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f491ffff640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49100b5410 0x7f49100b78d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f490c005ab0 tx=0x7f490c000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.110+0000 7f49257a1640 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fb2f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4918009fd0 tx=0x7f49180118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87be067d7a41ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f490affd640 1 -- 172.21.15.33:0/4089233723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918004d20 con 0x5611f45f8f00 2024-09-10T21:27:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f490affd640 1 -- 172.21.15.33:0/4089233723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4918004050 con 0x5611f45f8f00 2024-09-10T21:27:28.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f490affd640 1 -- 172.21.15.33:0/4089233723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918005280 con 0x5611f45f8f00 2024-09-10T21:27:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f49295d9740 1 -- 172.21.15.33:0/4089233723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 msgr2=0x5611f45fb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f49295d9740 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fb2f0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4918009fd0 tx=0x7f49180118d0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f49295d9740 1 -- 172.21.15.33:0/4089233723 shutdown_connections 2024-09-10T21:27:28.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f49295d9740 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fb2f0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f49295d9740 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f49295d9740 1 --2- 172.21.15.33:0/4089233723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45eca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.111+0000 7f49295d9740 1 -- 172.21.15.33:0/4089233723 >> 172.21.15.33:0/4089233723 conn(0x5611f45ea480 msgr2=0x5611f45eb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.112+0000 7f49295d9740 1 -- 172.21.15.33:0/4089233723 shutdown_connections 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.112+0000 7f49295d9740 1 -- 172.21.15.33:0/4089233723 wait complete. 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.112+0000 7f49295d9740 1 -- start start 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.112+0000 7f49295d9740 1 Processor -- start 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49295d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49295d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49295d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49295d9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5611f43b3700 con 0x5611f45f8f00 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49295d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5611f44ed110 con 0x5611f45ec650 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49295d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611f45fbb80 con 0x5611f45ecf70 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49295d9740 10 client.0 _ll_register_callbacks cb 0x5611f4506990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f49257a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50284/0 (socket says 172.21.15.33:50284) 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 -- 172.21.15.33:0/1569695420 learned_addr learned my addr 172.21.15.33:0/1569695420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f491ffff640 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 -- 172.21.15.33:0/1569695420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 msgr2=0x5611f45f5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f5c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 -- 172.21.15.33:0/1569695420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 msgr2=0x5611f45f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 -- 172.21.15.33:0/1569695420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f451ac40 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f4924fa0640 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45f87c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f49140e3a00 tx=0x7f49140048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95c97cb367104b93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.113+0000 7f48fffff640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f48fffff640 1 -- 172.21.15.33:0/1569695420 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49140d3020 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f48fffff640 1 -- 172.21.15.33:0/1569695420 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f49140b5a20 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4918004050 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f48fffff640 1 -- 172.21.15.33:0/1569695420 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f49140ac720 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 10 client.28907 fetch_fsmap learned FSMap version 90 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x5611f4607420 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f48fffff640 1 -- 172.21.15.33:0/1569695420 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4918004050 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f48fffff640 1 -- 172.21.15.33:0/1569695420 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 90) ==== 8382+0+0 (secure 0 0 0) 0x7f49140ab3f0 con 0x5611f45ec650 2024-09-10T21:27:28.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 10 client.28907 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 10 client.28907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 20 client.28907 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f460ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 4 client.28907 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.114+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611f4564da0 con 0x5611f46088f0 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.115+0000 7f491ffff640 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f460ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.115+0000 7f48fffff640 10 client.28907 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:28.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.115+0000 7f491ffff640 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f460ace0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5e7d1003522402 server_cookie=cd79a54f24c49917 in_seq=0 out_seq=0 2024-09-10T21:27:28.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f48fffff640 1 -- 172.21.15.33:0/1569695420 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5611f4564da0 con 0x5611f46088f0 2024-09-10T21:27:28.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f48fffff640 10 client.28907 handle_command_reply: tid=0 2024-09-10T21:27:28.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 client.28907 shutdown 2024-09-10T21:27:28.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 msgr2=0x5611f45f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45f87c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f49140e3a00 tx=0x7f49140048f0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 shutdown_connections 2024-09-10T21:27:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f460ace0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f48fffff640 0 client.28907 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45f5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 --2- 172.21.15.33:0/1569695420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45f87c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 >> 172.21.15.33:0/1569695420 conn(0x5611f45ea480 msgr2=0x5611f45efff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 shutdown_connections 2024-09-10T21:27:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 1 -- 172.21.15.33:0/1569695420 wait complete. 2024-09-10T21:27:28.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.118+0000 7f49295d9740 20 client.28907 trim_cache size 0 max 16384 2024-09-10T21:27:28.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.144+0000 7f49295d9740 1 Processor -- start 2024-09-10T21:27:28.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49295d9740 1 -- start start 2024-09-10T21:27:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49295d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ec650 0x5611f45e9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49295d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ecf70 0x5611f45f4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49295d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49295d9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49201b03d0 con 0x5611f45f8f00 2024-09-10T21:27:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49295d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49201b1fd0 con 0x5611f45ecf70 2024-09-10T21:27:28.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49295d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49201b0aa0 con 0x5611f45ec650 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49257a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f491ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ecf70 0x5611f45f4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49257a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33548/0 (socket says 172.21.15.33:33548) 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f49257a1640 1 -- 172.21.15.33:0/3063171114 learned_addr learned my addr 172.21.15.33:0/3063171114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f491ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ecf70 0x5611f45f4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:50290/0 (socket says 172.21.15.33:50290) 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.145+0000 7f4924fa0640 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ec650 0x5611f45e9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f49257a1640 1 -- 172.21.15.33:0/3063171114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ec650 msgr2=0x5611f45e9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f49257a1640 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ec650 0x5611f45e9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f49257a1640 1 -- 172.21.15.33:0/3063171114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ecf70 msgr2=0x5611f45f4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f49257a1640 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ecf70 0x5611f45f4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f49257a1640 1 -- 172.21.15.33:0/3063171114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611f45fb660 con 0x5611f45f8f00 2024-09-10T21:27:28.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f4924fa0640 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ec650 0x5611f45e9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f49257a1640 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f49d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4918006010 tx=0x7f4918009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afbe90b15145849 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f490a7fc640 1 -- 172.21.15.33:0/3063171114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918003a30 con 0x5611f45f8f00 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f490a7fc640 1 -- 172.21.15.33:0/3063171114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4918003bd0 con 0x5611f45f8f00 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.146+0000 7f490a7fc640 1 -- 172.21.15.33:0/3063171114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918004720 con 0x5611f45f8f00 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 -- 172.21.15.33:0/3063171114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 msgr2=0x5611f45f49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f49d0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4918006010 tx=0x7f4918009c90 comp rx=0 tx=0).stop 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 -- 172.21.15.33:0/3063171114 shutdown_connections 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45f49d0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ecf70 0x5611f45f4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 --2- 172.21.15.33:0/3063171114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ec650 0x5611f45e9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 -- 172.21.15.33:0/3063171114 >> 172.21.15.33:0/3063171114 conn(0x5611f45ea480 msgr2=0x5611f45f0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 -- 172.21.15.33:0/3063171114 shutdown_connections 2024-09-10T21:27:28.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.147+0000 7f49295d9740 1 -- 172.21.15.33:0/3063171114 wait complete. 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 -- start start 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 Processor -- start 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45fce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f49201b1870 con 0x5611f45f8f00 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f49201b1170 con 0x5611f45ec650 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f49201ae390 con 0x5611f45ecf70 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49295d9740 10 client.0 _ll_register_callbacks cb 0x7f49140ac580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f49257a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33564/0 (socket says 172.21.15.33:33564) 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 -- 172.21.15.33:0/4285234736 learned_addr learned my addr 172.21.15.33:0/4285234736 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:27:28.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.148+0000 7f4924fa0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f491ffff640 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 -- 172.21.15.33:0/4285234736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 msgr2=0x5611f45fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 -- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 msgr2=0x5611f45fce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45fce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 -- 172.21.15.33:0/4285234736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611f45f8b30 con 0x5611f45f8f00 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f4924fa0640 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:28.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49257a1640 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fd910 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4918014040 tx=0x7f4918004900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87530fa49f428e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:28.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f48fffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:27:28.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f48fffff640 1 -- 172.21.15.33:0/4285234736 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918034060 con 0x5611f45f8f00 2024-09-10T21:27:28.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f48fffff640 1 -- 172.21.15.33:0/4285234736 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f4918004df0 con 0x5611f45f8f00 2024-09-10T21:27:28.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.149+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4918003bd0 con 0x5611f45f8f00 2024-09-10T21:27:28.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f48fffff640 1 -- 172.21.15.33:0/4285234736 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f4918003a30 con 0x5611f45f8f00 2024-09-10T21:27:28.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 10 client.19581 fetch_fsmap learned FSMap version 90 2024-09-10T21:27:28.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=4+}) -- 0x5611f46759a0 con 0x5611f45f8f00 2024-09-10T21:27:28.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f48fffff640 1 -- 172.21.15.33:0/4285234736 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4918003bd0 con 0x5611f45f8f00 2024-09-10T21:27:28.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f48fffff640 1 -- 172.21.15.33:0/4285234736 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 90) ==== 8382+0+0 (secure 0 0 0) 0x7f4918039070 con 0x5611f45f8f00 2024-09-10T21:27:28.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 10 client.19581 fetch_fsmap finished waiting for FSMap version 90 2024-09-10T21:27:28.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 10 client.19581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:27:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 20 client.19581 populate_metadata read hostname 'smithi033' 2024-09-10T21:27:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f4677f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 4 client.19581 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:27:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.150+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5611f4560850 con 0x5611f46088f0 2024-09-10T21:27:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.151+0000 7f4924fa0640 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f4677f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:28.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.151+0000 7f48fffff640 10 client.19581 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:28.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.151+0000 7f4924fa0640 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f4677f40 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aab35db220cde7bd server_cookie=717fa99322aefa28 in_seq=0 out_seq=0 2024-09-10T21:27:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.152+0000 7f48fffff640 1 -- 172.21.15.33:0/4285234736 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8806 (crc 0 0 0) 0x5611f4560850 con 0x5611f46088f0 2024-09-10T21:27:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.152+0000 7f48fffff640 10 client.19581 handle_command_reply: tid=0 2024-09-10T21:27:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.152+0000 7f49295d9740 1 client.19581 shutdown 2024-09-10T21:27:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 msgr2=0x5611f45fd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fd910 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4918014040 tx=0x7f4918004900 comp rx=0 tx=0).stop 2024-09-10T21:27:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 shutdown_connections 2024-09-10T21:27:28.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5611f46088f0 0x5611f4677f40 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f48fffff640 0 client.19581 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5611f45f8f00 0x5611f45fd910 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611f45ecf70 0x5611f45fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 --2- 172.21.15.33:0/4285234736 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5611f45ec650 0x5611f45fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 >> 172.21.15.33:0/4285234736 conn(0x5611f45ea480 msgr2=0x5611f45f4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 shutdown_connections 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 1 -- 172.21.15.33:0/4285234736 wait complete. 2024-09-10T21:27:28.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.153+0000 7f49295d9740 20 client.19581 trim_cache size 0 max 16384 2024-09-10T21:27:28.169 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:27:28.169 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:27:28.169 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:27:28.169 INFO:teuthology.orchestra.run.smithi033.stdout: "b22ec403-44be-4b2f-9c71-e7053f2ad4b0": { 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "b22ec403-44be-4b2f-9c71-e7053f2ad4b0", 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.170 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:27:28.171 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.172 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "b22ec403-44be-4b2f-9c71-e7053f2ad4b0": { 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:27:26.176127+0000", 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.173 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.175 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.176 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.177 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:27:28.178 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.180 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:27:28.181 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.182 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.183 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:27:28.184 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.185 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.186 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.187 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "05708340-c3c3-4430-9bae-81feded8d778": { 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:26:35.545755+0000", 2024-09-10T21:27:28.188 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:27:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:27:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:27:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:27:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:27:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:27:28.189 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:27:28.189 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.154+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49100b5410 msgr2=0x7f49100b78d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.154+0000 7f4925fa2640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49100b5410 0x7f49100b78d0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f490c005ab0 tx=0x7f490c000bc0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.154+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201039c0 msgr2=0x7f492019ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.154+0000 7f4925fa2640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201039c0 0x7f492019ca60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f491400e9f0 tx=0x7f491400edf0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 shutdown_connections 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f49100b5410 0x7f49100b78d0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f492010cf70 0x7f49201a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f49201043c0 0x7f492019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 --2- 172.21.15.33:0/712807789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f49201039c0 0x7f492019ca60 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:28.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 >> 172.21.15.33:0/712807789 conn(0x7f49200fd5b0 msgr2=0x7f492010d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 shutdown_connections 2024-09-10T21:27:28.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:27:28.155+0000 7f4925fa2640 1 -- 172.21.15.33:0/712807789 wait complete. 2024-09-10T21:27:28.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:28 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:28 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:28 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:28 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:28 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:28 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:28 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:28.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:28 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:29.004 INFO:tasks.cephfs.filesystem:scrub status for tag:b22ec403-44be-4b2f-9c71-e7053f2ad4b0 - {'path': '/', 'tag': 'b22ec403-44be-4b2f-9c71-e7053f2ad4b0', 'options': 'recursive,force'} 2024-09-10T21:27:29.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:28 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:29.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:28 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:29.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:28 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:29.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:28 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:27:29.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 458149 37.73 MB/sec execute 509 sec latency 84.123 ms 2024-09-10T21:27:29.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:29 smithi174 ceph-mon[35635]: pgmap v1051: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 673 op/s 2024-09-10T21:27:29.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:29 smithi033 ceph-mon[30860]: pgmap v1051: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 673 op/s 2024-09-10T21:27:30.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:29 smithi104 ceph-mon[35325]: pgmap v1051: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 8.6 MiB/s wr, 673 op/s 2024-09-10T21:27:30.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 458870 37.73 MB/sec execute 510 sec latency 57.660 ms 2024-09-10T21:27:31.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 459577 37.73 MB/sec execute 511 sec latency 134.091 ms 2024-09-10T21:27:31.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:31 smithi174 ceph-mon[35635]: pgmap v1052: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 737 op/s 2024-09-10T21:27:31.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:31 smithi033 ceph-mon[30860]: pgmap v1052: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 737 op/s 2024-09-10T21:27:32.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:31 smithi104 ceph-mon[35325]: pgmap v1052: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 8.8 MiB/s wr, 737 op/s 2024-09-10T21:27:32.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 460049 37.70 MB/sec execute 512 sec latency 176.448 ms 2024-09-10T21:27:33.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 460856 37.72 MB/sec execute 513 sec latency 195.252 ms 2024-09-10T21:27:33.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:33 smithi174 ceph-mon[35635]: pgmap v1053: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 683 op/s 2024-09-10T21:27:33.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:33 smithi033 ceph-mon[30860]: pgmap v1053: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 683 op/s 2024-09-10T21:27:34.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:33 smithi104 ceph-mon[35325]: pgmap v1053: 97 pgs: 97 active+clean; 1.1 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.3 MiB/s wr, 683 op/s 2024-09-10T21:27:34.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 460996 37.66 MB/sec execute 514 sec latency 614.729 ms 2024-09-10T21:27:35.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 461381 37.62 MB/sec execute 515 sec latency 130.098 ms 2024-09-10T21:27:35.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:35 smithi033 ceph-mon[30860]: pgmap v1054: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-09-10T21:27:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:35 smithi104 ceph-mon[35325]: pgmap v1054: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-09-10T21:27:36.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 462021 37.62 MB/sec execute 516 sec latency 164.800 ms 2024-09-10T21:27:36.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:35 smithi174 ceph-mon[35635]: pgmap v1054: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 11 MiB/s wr, 782 op/s 2024-09-10T21:27:37.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 462733 37.62 MB/sec execute 517 sec latency 209.659 ms 2024-09-10T21:27:37.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:37 smithi033 ceph-mon[30860]: pgmap v1055: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 687 op/s 2024-09-10T21:27:38.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:37 smithi104 ceph-mon[35325]: pgmap v1055: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 687 op/s 2024-09-10T21:27:38.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 463339 37.61 MB/sec execute 518 sec latency 86.128 ms 2024-09-10T21:27:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:37 smithi174 ceph-mon[35635]: pgmap v1055: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 8.5 MiB/s wr, 687 op/s 2024-09-10T21:27:39.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 464069 37.61 MB/sec execute 519 sec latency 104.838 ms 2024-09-10T21:27:39.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:39 smithi033 ceph-mon[30860]: pgmap v1056: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.1 MiB/s wr, 707 op/s 2024-09-10T21:27:40.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:39 smithi104 ceph-mon[35325]: pgmap v1056: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.1 MiB/s wr, 707 op/s 2024-09-10T21:27:40.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 464669 37.61 MB/sec execute 520 sec latency 347.317 ms 2024-09-10T21:27:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:39 smithi174 ceph-mon[35635]: pgmap v1056: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 9.1 MiB/s wr, 707 op/s 2024-09-10T21:27:41.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 465431 37.62 MB/sec execute 521 sec latency 76.803 ms 2024-09-10T21:27:41.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:41 smithi033 ceph-mon[30860]: pgmap v1057: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-10T21:27:41.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:41 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:27:42.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:41 smithi104 ceph-mon[35325]: pgmap v1057: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-10T21:27:42.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:41 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:27:42.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 466053 37.59 MB/sec execute 522 sec latency 73.029 ms 2024-09-10T21:27:42.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:41 smithi174 ceph-mon[35635]: pgmap v1057: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 11 MiB/s wr, 853 op/s 2024-09-10T21:27:42.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:41 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:27:42.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:27:42.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:27:42.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:27:42.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:42 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:27:43.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:27:43.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:27:43.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:27:43.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:42 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:27:43.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 466839 37.61 MB/sec execute 523 sec latency 103.205 ms 2024-09-10T21:27:43.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:27:43.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:27:43.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:27:43.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:42 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:27:43.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:43 smithi033 ceph-mon[30860]: pgmap v1058: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 705 op/s 2024-09-10T21:27:44.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:43 smithi104 ceph-mon[35325]: pgmap v1058: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 705 op/s 2024-09-10T21:27:44.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 467601 37.61 MB/sec execute 524 sec latency 83.365 ms 2024-09-10T21:27:44.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:43 smithi174 ceph-mon[35635]: pgmap v1058: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 11 MiB/s wr, 705 op/s 2024-09-10T21:27:45.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 468489 37.65 MB/sec execute 525 sec latency 51.837 ms 2024-09-10T21:27:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:45 smithi104 ceph-mon[35325]: pgmap v1059: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-09-10T21:27:46.088 INFO:tasks.workunit.client.0.smithi033.stdout: 10 469124 37.64 MB/sec execute 526 sec latency 102.735 ms 2024-09-10T21:27:46.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:45 smithi174 ceph-mon[35635]: pgmap v1059: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-09-10T21:27:46.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:45 smithi033 ceph-mon[30860]: pgmap v1059: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 13 MiB/s wr, 783 op/s 2024-09-10T21:27:47.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 469773 37.61 MB/sec execute 527 sec latency 90.581 ms 2024-09-10T21:27:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:47 smithi104 ceph-mon[35325]: pgmap v1060: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 802 op/s 2024-09-10T21:27:48.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 470032 37.57 MB/sec execute 528 sec latency 575.735 ms 2024-09-10T21:27:48.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:47 smithi174 ceph-mon[35635]: pgmap v1060: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 802 op/s 2024-09-10T21:27:48.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:47 smithi033 ceph-mon[30860]: pgmap v1060: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 802 op/s 2024-09-10T21:27:49.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 470622 37.57 MB/sec execute 529 sec latency 103.262 ms 2024-09-10T21:27:50.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:49 smithi104 ceph-mon[35325]: pgmap v1061: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 717 op/s 2024-09-10T21:27:50.085 INFO:tasks.workunit.client.0.smithi033.stdout: 10 471409 37.58 MB/sec execute 530 sec latency 153.185 ms 2024-09-10T21:27:50.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:49 smithi174 ceph-mon[35635]: pgmap v1061: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 717 op/s 2024-09-10T21:27:50.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:49 smithi033 ceph-mon[30860]: pgmap v1061: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 10 MiB/s wr, 717 op/s 2024-09-10T21:27:51.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 472073 37.58 MB/sec execute 531 sec latency 91.000 ms 2024-09-10T21:27:52.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:51 smithi104 ceph-mon[35325]: pgmap v1062: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 838 op/s 2024-09-10T21:27:52.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:51 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:27:52.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 472639 37.58 MB/sec execute 532 sec latency 157.490 ms 2024-09-10T21:27:52.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:51 smithi174 ceph-mon[35635]: pgmap v1062: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 838 op/s 2024-09-10T21:27:52.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:51 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:27:52.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:51 smithi033 ceph-mon[30860]: pgmap v1062: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 838 op/s 2024-09-10T21:27:52.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:51 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:27:53.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:27:53.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:27:53.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:27:53.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 473298 37.55 MB/sec execute 533 sec latency 112.533 ms 2024-09-10T21:27:53.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:27:53.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:27:53.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:27:53.143 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:27:53.143 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:27:53.143 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:27:54.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 474041 37.56 MB/sec execute 534 sec latency 72.178 ms 2024-09-10T21:27:54.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:54 smithi104 ceph-mon[35325]: pgmap v1063: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 693 op/s 2024-09-10T21:27:54.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:54 smithi174 ceph-mon[35635]: pgmap v1063: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 693 op/s 2024-09-10T21:27:54.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:54 smithi033 ceph-mon[30860]: pgmap v1063: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 693 op/s 2024-09-10T21:27:55.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 474698 37.57 MB/sec execute 535 sec latency 64.834 ms 2024-09-10T21:27:56.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 475388 37.55 MB/sec execute 536 sec latency 78.497 ms 2024-09-10T21:27:56.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:56 smithi104 ceph-mon[35325]: pgmap v1064: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-10T21:27:56.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:56 smithi174 ceph-mon[35635]: pgmap v1064: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-10T21:27:56.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:56 smithi033 ceph-mon[30860]: pgmap v1064: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-09-10T21:27:57.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 476060 37.56 MB/sec execute 537 sec latency 101.874 ms 2024-09-10T21:27:58.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 476888 37.57 MB/sec execute 538 sec latency 63.228 ms 2024-09-10T21:27:58.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:58 smithi104 ceph-mon[35325]: pgmap v1065: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.5 MiB/s wr, 738 op/s 2024-09-10T21:27:58.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:58 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:58.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:58 smithi174 ceph-mon[35635]: pgmap v1065: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.5 MiB/s wr, 738 op/s 2024-09-10T21:27:58.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:58 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:58.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:58 smithi033 ceph-mon[30860]: pgmap v1065: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.5 MiB/s wr, 738 op/s 2024-09-10T21:27:58.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:58 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:59.006 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:27:59.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 477540 37.58 MB/sec execute 539 sec latency 130.791 ms 2024-09-10T21:27:59.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:27:59 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:59.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:27:59 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:59.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:27:59 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:27:59.547 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:00.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 477623 37.51 MB/sec execute 540 sec latency 604.566 ms 2024-09-10T21:28:00.310 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:00 smithi033 ceph-mon[30860]: pgmap v1066: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.7 MiB/s wr, 643 op/s 2024-09-10T21:28:00.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:00 smithi104 ceph-mon[35325]: pgmap v1066: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.7 MiB/s wr, 643 op/s 2024-09-10T21:28:00.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:00 smithi174 ceph-mon[35635]: pgmap v1066: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.7 MiB/s wr, 643 op/s 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/1014876283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48103b30 msgr2=0x7fcd48103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/1014876283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48103b30 0x7fcd48103f10 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fcd40009ca0 tx=0x7fcd4002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/1014876283 shutdown_connections 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/1014876283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd4810d0e0 0x7fcd4810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/1014876283 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48104530 0x7fcd4810cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/1014876283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd48103b30 0x7fcd48103f10 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/1014876283 >> 172.21.15.33:0/1014876283 conn(0x7fcd480fd700 msgr2=0x7fcd480ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.421+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/1014876283 shutdown_connections 2024-09-10T21:28:00.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.422+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/1014876283 wait complete. 2024-09-10T21:28:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.422+0000 7fcd4d5d4640 1 Processor -- start 2024-09-10T21:28:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.422+0000 7fcd4d5d4640 1 -- start start 2024-09-10T21:28:00.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd4d5d4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48103b30 0x7fcd48199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd4d5d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd48104530 0x7fcd481997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd4d5d4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd4810d0e0 0x7fcd4819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd4d5d4640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48112d60 con 0x7fcd4810d0e0 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd4d5d4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48112be0 con 0x7fcd48103b30 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd4d5d4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd48112ee0 con 0x7fcd48104530 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48103b30 0x7fcd48199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48103b30 0x7fcd48199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:48444/0 (socket says 172.21.15.33:48444) 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd46ffd640 1 -- 172.21.15.33:0/3205450441 learned_addr learned my addr 172.21.15.33:0/3205450441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd467fc640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd48104530 0x7fcd481997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.423+0000 7fcd477fe640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd4810d0e0 0x7fcd4819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd46ffd640 1 -- 172.21.15.33:0/3205450441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd48104530 msgr2=0x7fcd481997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd48104530 0x7fcd481997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd46ffd640 1 -- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd4810d0e0 msgr2=0x7fcd4819eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd4810d0e0 0x7fcd4819eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd46ffd640 1 -- 172.21.15.33:0/3205450441 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4819f340 con 0x7fcd48103b30 2024-09-10T21:28:00.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48103b30 0x7fcd48199290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcd4002ade0 tx=0x7fcd40031770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd2ffff640 1 -- 172.21.15.33:0/3205450441 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40003d80 con 0x7fcd48103b30 2024-09-10T21:28:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4819f5d0 con 0x7fcd48103b30 2024-09-10T21:28:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd2ffff640 1 -- 172.21.15.33:0/3205450441 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd4003ec50 con 0x7fcd48103b30 2024-09-10T21:28:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.424+0000 7fcd2ffff640 1 -- 172.21.15.33:0/3205450441 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40004790 con 0x7fcd48103b30 2024-09-10T21:28:00.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.425+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd48101890 con 0x7fcd48103b30 2024-09-10T21:28:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.428+0000 7fcd2ffff640 1 -- 172.21.15.33:0/3205450441 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fcd40046070 con 0x7fcd48103b30 2024-09-10T21:28:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.428+0000 7fcd4f9a9740 1 Processor -- start 2024-09-10T21:28:00.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.428+0000 7fcd4f9a9740 1 -- start start 2024-09-10T21:28:00.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c415070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c421000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c423930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e80c382e50 con 0x55e80c4155b0 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e80c372280 con 0x55e80c414c90 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e80c376090 con 0x55e80c421540 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c421000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd467fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c423930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd2ffff640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcd300b5360 0x7fcd300b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c421000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60598/0 (socket says 172.21.15.33:60598) 2024-09-10T21:28:00.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.429+0000 7fcd46ffd640 1 -- 172.21.15.33:0/4259608472 learned_addr learned my addr 172.21.15.33:0/4259608472 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd2ffff640 1 -- 172.21.15.33:0/3205450441 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcd4010aa50 con 0x7fcd48103b30 2024-09-10T21:28:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd46ffd640 1 -- 172.21.15.33:0/4259608472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 msgr2=0x55e80c423930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/4259608472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c423930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd46ffd640 1 -- 172.21.15.33:0/4259608472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 msgr2=0x55e80c415070 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/4259608472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c415070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd46ffd640 1 -- 172.21.15.33:0/4259608472 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80c0d0650 con 0x55e80c4155b0 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/4259608472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c421000 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcd400d1160 tx=0x7fcd400b87d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbee6ba0d4014d80 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd477fe640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcd300b5360 0x7fcd300b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd2d7fa640 1 -- 172.21.15.33:0/4259608472 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400d0b20 con 0x55e80c4155b0 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd2d7fa640 1 -- 172.21.15.33:0/4259608472 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd40036070 con 0x55e80c4155b0 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.430+0000 7fcd2d7fa640 1 -- 172.21.15.33:0/4259608472 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400d0940 con 0x55e80c4155b0 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/4259608472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 msgr2=0x55e80c421000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd477fe640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcd300b5360 0x7fcd300b7820 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009b70 tx=0x7fcd38005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:00.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/4259608472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c421000 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fcd400d1160 tx=0x7fcd400b87d0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/4259608472 shutdown_connections 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/4259608472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c423930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/4259608472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c421000 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/4259608472 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c415070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/4259608472 >> 172.21.15.33:0/4259608472 conn(0x55e80c412ac0 msgr2=0x55e80c413c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.431+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/4259608472 shutdown_connections 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.432+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/4259608472 wait complete. 2024-09-10T21:28:00.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.432+0000 7fcd4f9a9740 1 -- start start 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.432+0000 7fcd4f9a9740 1 Processor -- start 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.432+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c414c90 0x55e80c420e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.432+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c41e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c421540 0x55e80c41e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e80c1d3700 con 0x55e80c421540 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e80c30d110 con 0x55e80c4155b0 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e80c382e50 con 0x55e80c414c90 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c414c90 0x55e80c420e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c414c90 0x55e80c420e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34286/0 (socket says 172.21.15.33:34286) 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1762156410 learned_addr learned my addr 172.21.15.33:0/1762156410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd467fc640 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c41e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd477fe640 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c421540 0x55e80c41e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd4f9a9740 10 client.0 _ll_register_callbacks cb 0x55e80c326990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 msgr2=0x55e80c41e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c41e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1762156410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c421540 msgr2=0x55e80c41e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c421540 0x55e80c41e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1762156410 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80c33ac40 con 0x55e80c414c90 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c414c90 0x55e80c420e30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd4002f550 tx=0x7fcd400095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4f2d2d250161345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.433+0000 7fcd2aff5640 1 -- 172.21.15.33:0/1762156410 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400b8620 con 0x55e80c414c90 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.434+0000 7fcd2aff5640 1 -- 172.21.15.33:0/1762156410 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcd400da2d0 con 0x55e80c414c90 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.434+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd40036070 con 0x55e80c414c90 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.434+0000 7fcd2aff5640 1 -- 172.21.15.33:0/1762156410 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fcd400d18d0 con 0x55e80c414c90 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.434+0000 7fcd4f9a9740 10 client.28786 fetch_fsmap learned FSMap version 92 2024-09-10T21:28:00.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.434+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55e80c42f620 con 0x55e80c414c90 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.434+0000 7fcd2aff5640 1 -- 172.21.15.33:0/1762156410 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd4003ee00 con 0x55e80c414c90 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.435+0000 7fcd2aff5640 1 -- 172.21.15.33:0/1762156410 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 92) ==== 8382+0+0 (secure 0 0 0) 0x7fcd400d1430 con 0x55e80c414c90 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.435+0000 7fcd4f9a9740 10 client.28786 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.435+0000 7fcd4f9a9740 10 client.28786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.435+0000 7fcd4f9a9740 20 client.28786 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.435+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c430b20 0x55e80c432f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.435+0000 7fcd4f9a9740 4 client.28786 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:00.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.435+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd40036070 con 0x55e80c430b20 2024-09-10T21:28:00.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.452+0000 7fcd467fc640 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c430b20 0x55e80c432f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.453+0000 7fcd467fc640 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c430b20 0x55e80c432f40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1a231649830782 server_cookie=dac84eabbe2c32a0 in_seq=0 out_seq=0 2024-09-10T21:28:00.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.453+0000 7fcd2aff5640 10 client.28786 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:00.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.456+0000 7fcd2aff5640 1 -- 172.21.15.33:0/1762156410 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fcd40036070 con 0x55e80c430b20 2024-09-10T21:28:00.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.456+0000 7fcd2aff5640 10 client.28786 handle_command_reply: tid=0 2024-09-10T21:28:00.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.456+0000 7fcd4f9a9740 1 client.28786 shutdown 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c414c90 msgr2=0x55e80c420e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c414c90 0x55e80c420e30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcd4002f550 tx=0x7fcd400095c0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 shutdown_connections 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c430b20 0x55e80c432f40 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c421540 0x55e80c41e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c41e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1762156410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c414c90 0x55e80c420e30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 >> 172.21.15.33:0/1762156410 conn(0x55e80c412ac0 msgr2=0x55e80c4093d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd2aff5640 0 client.28786 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 shutdown_connections 2024-09-10T21:28:00.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.457+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1762156410 wait complete. 2024-09-10T21:28:00.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.458+0000 7fcd4f9a9740 20 client.28786 trim_cache size 0 max 16384 2024-09-10T21:28:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.489+0000 7fcd4f9a9740 1 Processor -- start 2024-09-10T21:28:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd4f9a9740 1 -- start start 2024-09-10T21:28:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c414c90 0x55e80c42cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c422930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c422e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c414c90 0x55e80c42cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c414c90 0x55e80c42cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60614/0 (socket says 172.21.15.33:60614) 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1292813591 learned_addr learned my addr 172.21.15.33:0/1292813591 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481abf70 con 0x55e80c414c90 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481adb40 con 0x55e80c4155b0 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.490+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481ac640 con 0x55e80c421540 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.491+0000 7fcd467fc640 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c422930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.491+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1292813591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 msgr2=0x55e80c422e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.491+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c422e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.491+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1292813591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 msgr2=0x55e80c422930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.491+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c422930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.491+0000 7fcd46ffd640 1 -- 172.21.15.33:0/1292813591 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e80c4335c0 con 0x55e80c414c90 2024-09-10T21:28:00.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.491+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c414c90 0x55e80c42cf00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcd400da040 tx=0x7fcd400d12e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16edc0d6ca681c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.492+0000 7fcd2cff9640 1 -- 172.21.15.33:0/1292813591 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400d0940 con 0x55e80c414c90 2024-09-10T21:28:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.492+0000 7fcd2cff9640 1 -- 172.21.15.33:0/1292813591 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd4003ee00 con 0x55e80c414c90 2024-09-10T21:28:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.492+0000 7fcd2cff9640 1 -- 172.21.15.33:0/1292813591 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400cd820 con 0x55e80c414c90 2024-09-10T21:28:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.493+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c414c90 msgr2=0x55e80c42cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.493+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c414c90 0x55e80c42cf00 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fcd400da040 tx=0x7fcd400d12e0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.493+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 shutdown_connections 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.494+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c422e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.494+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c4155b0 0x55e80c422930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.494+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/1292813591 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c414c90 0x55e80c42cf00 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.494+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 >> 172.21.15.33:0/1292813591 conn(0x55e80c412ac0 msgr2=0x55e80c413b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.494+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 shutdown_connections 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.494+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/1292813591 wait complete. 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.494+0000 7fcd4f9a9740 1 -- start start 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 1 Processor -- start 2024-09-10T21:28:00.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c4293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c429930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c424170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481ad410 con 0x55e80c4155b0 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481acd10 con 0x55e80c414c90 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd481a9f50 con 0x55e80c421540 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd4f9a9740 10 client.0 _ll_register_callbacks cb 0x7fcd40004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c4293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd46ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c4293f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:48504/0 (socket says 172.21.15.33:48504) 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.495+0000 7fcd46ffd640 1 -- 172.21.15.33:0/927800945 learned_addr learned my addr 172.21.15.33:0/927800945 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd467fc640 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c429930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd46ffd640 1 -- 172.21.15.33:0/927800945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 msgr2=0x55e80c424170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:00.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c424170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd46ffd640 1 -- 172.21.15.33:0/927800945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 msgr2=0x55e80c429930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c429930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd46ffd640 1 -- 172.21.15.33:0/927800945 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e80c41f150 con 0x55e80c414c90 2024-09-10T21:28:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd46ffd640 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c4293f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcd400cd750 tx=0x7fcd400095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0867dd27962600 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:00.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd2aff5640 1 -- 172.21.15.33:0/927800945 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd400cd820 con 0x55e80c414c90 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.496+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd4003ee00 con 0x55e80c414c90 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.498+0000 7fcd2aff5640 1 -- 172.21.15.33:0/927800945 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fcd40004b40 con 0x55e80c414c90 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.498+0000 7fcd2aff5640 1 -- 172.21.15.33:0/927800945 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=92) ==== 24+0+0 (secure 0 0 0) 0x7fcd400d0940 con 0x55e80c414c90 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.498+0000 7fcd4f9a9740 10 client.28928 fetch_fsmap learned FSMap version 92 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.498+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=92,monmap=4+}) -- 0x55e80c413540 con 0x55e80c414c90 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.498+0000 7fcd2aff5640 1 -- 172.21.15.33:0/927800945 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd4003ee00 con 0x55e80c414c90 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.499+0000 7fcd2aff5640 1 -- 172.21.15.33:0/927800945 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 92) ==== 8382+0+0 (secure 0 0 0) 0x7fcd400ccb20 con 0x55e80c414c90 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.499+0000 7fcd4f9a9740 10 client.28928 fetch_fsmap finished waiting for FSMap version 92 2024-09-10T21:28:00.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.499+0000 7fcd4f9a9740 10 client.28928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:00.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.499+0000 7fcd4f9a9740 20 client.28928 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:00.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.499+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c49e700 0x55e80c4a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:00.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.499+0000 7fcd4f9a9740 4 client.28928 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:28:00.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.499+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e80c384da0 con 0x55e80c49e700 2024-09-10T21:28:00.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.501+0000 7fcd467fc640 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c49e700 0x55e80c4a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:00.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.502+0000 7fcd467fc640 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c49e700 0x55e80c4a0af0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86298f768c6f70ea server_cookie=4e80ccdd1a11a601 in_seq=0 out_seq=0 2024-09-10T21:28:00.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.502+0000 7fcd2aff5640 10 client.28928 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.504+0000 7fcd2aff5640 1 -- 172.21.15.33:0/927800945 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8610 (crc 0 0 0) 0x55e80c384da0 con 0x55e80c49e700 2024-09-10T21:28:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.504+0000 7fcd2aff5640 10 client.28928 handle_command_reply: tid=0 2024-09-10T21:28:00.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.504+0000 7fcd4f9a9740 1 client.28928 shutdown 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 msgr2=0x55e80c4293f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c4293f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcd400cd750 tx=0x7fcd400095c0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 shutdown_connections 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e80c49e700 0x55e80c4a0af0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e80c421540 0x55e80c424170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e80c4155b0 0x55e80c429930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 --2- 172.21.15.33:0/927800945 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e80c414c90 0x55e80c4293f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 >> 172.21.15.33:0/927800945 conn(0x55e80c412ac0 msgr2=0x55e80c408bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:00.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.505+0000 7fcd2aff5640 0 client.28928 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:00.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.506+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 shutdown_connections 2024-09-10T21:28:00.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.506+0000 7fcd4f9a9740 1 -- 172.21.15.33:0/927800945 wait complete. 2024-09-10T21:28:00.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.506+0000 7fcd4f9a9740 20 client.28928 trim_cache size 0 max 16384 2024-09-10T21:28:00.509 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.511 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.512 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "b22ec403-44be-4b2f-9c71-e7053f2ad4b0": { 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:27:26.176127+0000", 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.513 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.514 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.515 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:28:00.516 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.517 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:28:00.518 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.519 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.520 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.521 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.522 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:28:00.523 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.525 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.526 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.527 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "05708340-c3c3-4430-9bae-81feded8d778": { 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:26:35.545755+0000", 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:00.528 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcd300b5360 msgr2=0x7fcd300b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcd300b5360 0x7fcd300b7820 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcd38009b70 tx=0x7fcd38005f90 comp rx=0 tx=0).stop 2024-09-10T21:28:00.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48103b30 msgr2=0x7fcd48199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48103b30 0x7fcd48199290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fcd4002ade0 tx=0x7fcd40031770 comp rx=0 tx=0).stop 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 shutdown_connections 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fcd300b5360 0x7fcd300b7820 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd4810d0e0 0x7fcd4819eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd48104530 0x7fcd481997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 --2- 172.21.15.33:0/3205450441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcd48103b30 0x7fcd48199290 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 >> 172.21.15.33:0/3205450441 conn(0x7fcd480fd700 msgr2=0x7fcd480fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.507+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 shutdown_connections 2024-09-10T21:28:00.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:00.508+0000 7fcd4d5d4640 1 -- 172.21.15.33:0/3205450441 wait complete. 2024-09-10T21:28:01.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 478003 37.48 MB/sec execute 541 sec latency 353.683 ms 2024-09-10T21:28:01.466 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:28:01.466 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:28:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:02.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 478496 37.46 MB/sec execute 542 sec latency 141.292 ms 2024-09-10T21:28:02.310 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:02 smithi104 ceph-mon[35325]: pgmap v1067: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.0 MiB/s wr, 753 op/s 2024-09-10T21:28:02.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:02 smithi174 ceph-mon[35635]: pgmap v1067: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.0 MiB/s wr, 753 op/s 2024-09-10T21:28:02.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:02 smithi033 ceph-mon[30860]: pgmap v1067: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.0 MiB/s wr, 753 op/s 2024-09-10T21:28:03.086 INFO:tasks.workunit.client.0.smithi033.stdout: 10 478958 37.41 MB/sec execute 543 sec latency 547.311 ms 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 -- 172.21.15.33:0/2494883516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a011c0a0 msgr2=0x7ff9a011e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 --2- 172.21.15.33:0/2494883516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a011c0a0 0x7ff9a011e490 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff998009be0 tx=0x7ff99802a750 comp rx=0 tx=0).stop 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 -- 172.21.15.33:0/2494883516 shutdown_connections 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 --2- 172.21.15.33:0/2494883516 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a011c0a0 0x7ff9a011e490 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 --2- 172.21.15.33:0/2494883516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9a010a530 0x7ff9a010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 --2- 172.21.15.33:0/2494883516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a010a150 0x7ff9a01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 -- 172.21.15.33:0/2494883516 >> 172.21.15.33:0/2494883516 conn(0x7ff9a006dfc0 msgr2=0x7ff9a006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 -- 172.21.15.33:0/2494883516 shutdown_connections 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.615+0000 7ff9a696c640 1 -- 172.21.15.33:0/2494883516 wait complete. 2024-09-10T21:28:03.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.616+0000 7ff9a696c640 1 Processor -- start 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.616+0000 7ff9a696c640 1 -- start start 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a696c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a010a150 0x7ff9a01a8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a696c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9a010a530 0x7ff9a01a85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a696c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a011c0a0 0x7ff9a01ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a696c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0121b60 con 0x7ff9a010a530 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a696c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01219e0 con 0x7ff9a010a150 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a696c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0121ce0 con 0x7ff9a011c0a0 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a4f09640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a011c0a0 0x7ff9a01ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff997fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9a010a530 0x7ff9a01a85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a010a150 0x7ff9a01a8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff997fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9a010a530 0x7ff9a01a85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60646/0 (socket says 172.21.15.33:60646) 2024-09-10T21:28:03.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff997fff640 1 -- 172.21.15.33:0/3510274181 learned_addr learned my addr 172.21.15.33:0/3510274181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a4f09640 1 -- 172.21.15.33:0/3510274181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a010a150 msgr2=0x7ff9a01a8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.617+0000 7ff9a4f09640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a010a150 0x7ff9a01a8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.618+0000 7ff9a4f09640 1 -- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9a010a530 msgr2=0x7ff9a01a85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.618+0000 7ff9a4f09640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9a010a530 0x7ff9a01a85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.618+0000 7ff9a4f09640 1 -- 172.21.15.33:0/3510274181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a01ae140 con 0x7ff9a011c0a0 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.618+0000 7ff9a4f09640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a011c0a0 0x7ff9a01ad970 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff998034040 tx=0x7ff998003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.618+0000 7ff99dffb640 1 -- 172.21.15.33:0/3510274181 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998041020 con 0x7ff9a011c0a0 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.619+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a01ae3d0 con 0x7ff9a011c0a0 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.619+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01b4af0 con 0x7ff9a011c0a0 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.619+0000 7ff99dffb640 1 -- 172.21.15.33:0/3510274181 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9980318f0 con 0x7ff9a011c0a0 2024-09-10T21:28:03.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.619+0000 7ff99dffb640 1 -- 172.21.15.33:0/3510274181 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998036040 con 0x7ff9a011c0a0 2024-09-10T21:28:03.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.621+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff984005290 con 0x7ff9a011c0a0 2024-09-10T21:28:03.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.624+0000 7ff99dffb640 1 -- 172.21.15.33:0/3510274181 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7ff998031440 con 0x7ff9a011c0a0 2024-09-10T21:28:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.625+0000 7ff99dffb640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9800b5420 0x7ff9800b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:03.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.625+0000 7ff99dffb640 1 -- 172.21.15.33:0/3510274181 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7ff998106770 con 0x7ff9a011c0a0 2024-09-10T21:28:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.625+0000 7ff99ffff640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9800b5420 0x7ff9800b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:03.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.626+0000 7ff99ffff640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9800b5420 0x7ff9800b78e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff990009bd0 tx=0x7ff990005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:03.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.629+0000 7ff99dffb640 1 -- 172.21.15.33:0/3510274181 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7ff99804b020 con 0x7ff9a011c0a0 2024-09-10T21:28:04.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:03.999+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff984005580 con 0x7ff9a011c0a0 2024-09-10T21:28:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.001+0000 7ff99dffb640 1 -- 172.21.15.33:0/3510274181 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+11980 (secure 0 0 0) 0x7ff9980cd5f0 con 0x7ff9a011c0a0 2024-09-10T21:28:04.003 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:28:04.004 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":93,"btime":"2024-09-10T21:28:03:053726+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":93,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:28:02.240738+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:28:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 93 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.004+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9800b5420 msgr2=0x7ff9800b78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.004+0000 7ff9a696c640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9800b5420 0x7ff9800b78e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff990009bd0 tx=0x7ff990005f90 comp rx=0 tx=0).stop 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.004+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a011c0a0 msgr2=0x7ff9a01ad970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.004+0000 7ff9a696c640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a011c0a0 0x7ff9a01ad970 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff998034040 tx=0x7ff998003c10 comp rx=0 tx=0).stop 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 shutdown_connections 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7ff9800b5420 0x7ff9800b78e0 secure :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff990009bd0 tx=0x7ff990005f90 comp rx=0 tx=0).stop 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a011c0a0 0x7ff9a01ad970 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:04.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9a010a530 0x7ff9a01a85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:04.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 --2- 172.21.15.33:0/3510274181 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9a010a150 0x7ff9a01a8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:04.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 >> 172.21.15.33:0/3510274181 conn(0x7ff9a006dfc0 msgr2=0x7ff9a011c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:04.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 shutdown_connections 2024-09-10T21:28:04.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:04.005+0000 7ff9a696c640 1 -- 172.21.15.33:0/3510274181 wait complete. 2024-09-10T21:28:04.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 479749 37.43 MB/sec execute 544 sec latency 99.455 ms 2024-09-10T21:28:04.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:04 smithi033 ceph-mon[30860]: pgmap v1068: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.7 MiB/s wr, 579 op/s 2024-09-10T21:28:04.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:04 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:04.337 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:04 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3510274181' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:28:04.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:04 smithi174 ceph-mon[35635]: pgmap v1068: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.7 MiB/s wr, 579 op/s 2024-09-10T21:28:04.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:04 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:04.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:04 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3510274181' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:28:04.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:04 smithi104 ceph-mon[35325]: pgmap v1068: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 6.7 MiB/s wr, 579 op/s 2024-09-10T21:28:04.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:04 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:04.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:04 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3510274181' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:28:04.884 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:28:05.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 480612 37.46 MB/sec execute 545 sec latency 67.506 ms 2024-09-10T21:28:05.249 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:06.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 481116 37.45 MB/sec execute 546 sec latency 312.474 ms 2024-09-10T21:28:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.833+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/2834494575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7810a530 msgr2=0x7f8d7810a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.833+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/2834494575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7810a530 0x7f8d7810a930 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009860 tx=0x7f8d6002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.838+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/2834494575 shutdown_connections 2024-09-10T21:28:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.838+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/2834494575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d7811c0a0 0x7f8d7811e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.838+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/2834494575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7810a530 0x7f8d7810a930 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.838+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/2834494575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7810a150 0x7f8d781111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.838+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/2834494575 >> 172.21.15.33:0/2834494575 conn(0x7f8d7806dfc0 msgr2=0x7f8d7806e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.841+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/2834494575 shutdown_connections 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.842+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/2834494575 wait complete. 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.843+0000 7f8d7d4e3640 1 Processor -- start 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.844+0000 7f8d7d4e3640 1 -- start start 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.844+0000 7f8d7d4e3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7810a150 0x7f8d781a8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.844+0000 7f8d7d4e3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d7810a530 0x7f8d781a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.844+0000 7f8d7d4e3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7811c0a0 0x7f8d781ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.844+0000 7f8d7d4e3640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78121c10 con 0x7f8d7811c0a0 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.844+0000 7f8d7d4e3640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78121a90 con 0x7f8d7810a530 2024-09-10T21:28:06.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.844+0000 7f8d7d4e3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d78121d90 con 0x7f8d7810a150 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.845+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7811c0a0 0x7f8d781ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.845+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7811c0a0 0x7f8d781ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60662/0 (socket says 172.21.15.33:60662) 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.845+0000 7f8d777fe640 1 -- 172.21.15.33:0/197006144 learned_addr learned my addr 172.21.15.33:0/197006144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.848+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d7810a530 0x7f8d781a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.850+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7810a150 0x7f8d781a8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.855+0000 7f8d76ffd640 1 -- 172.21.15.33:0/197006144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d7810a530 msgr2=0x7f8d781a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.855+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d7810a530 0x7f8d781a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.855+0000 7f8d76ffd640 1 -- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7811c0a0 msgr2=0x7f8d781ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.855+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7811c0a0 0x7f8d781ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.855+0000 7f8d76ffd640 1 -- 172.21.15.33:0/197006144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d781ae1d0 con 0x7f8d7810a150 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.857+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7810a150 0x7f8d781a8120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800d6d0 tx=0x7f8d6800db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.858+0000 7f8d74ff9640 1 -- 172.21.15.33:0/197006144 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68017070 con 0x7f8d7810a150 2024-09-10T21:28:06.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.858+0000 7f8d74ff9640 1 -- 172.21.15.33:0/197006144 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d68010460 con 0x7f8d7810a150 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.858+0000 7f8d74ff9640 1 -- 172.21.15.33:0/197006144 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6800f600 con 0x7f8d7810a150 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.859+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d781ae4c0 con 0x7f8d7810a150 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.859+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d781b4b80 con 0x7f8d7810a150 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.861+0000 7f8d7f8b8740 1 Processor -- start 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.861+0000 7f8d7f8b8740 1 -- start start 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be704d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be7c9a0 0x562b6be7ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d7f8b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b6bde6ea0 con 0x562b6be7c9a0 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d7f8b8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b6bdd62d0 con 0x562b6be70a10 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d7f8b8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b6bdda0e0 con 0x562b6be700f0 2024-09-10T21:28:06.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d6e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be7c9a0 0x562b6be7ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d777fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34342/0 (socket says 172.21.15.33:34342) 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d777fe640 1 -- 172.21.15.33:0/2930513661 learned_addr learned my addr 172.21.15.33:0/2930513661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d74ff9640 1 -- 172.21.15.33:0/197006144 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8d68009a20 con 0x7f8d7810a150 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d6e7fc640 1 -- 172.21.15.33:0/2930513661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 msgr2=0x562b6be704d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be704d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.862+0000 7f8d6e7fc640 1 -- 172.21.15.33:0/2930513661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 msgr2=0x562b6be7c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d6e7fc640 1 -- 172.21.15.33:0/2930513661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6bb34650 con 0x562b6be7c9a0 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be7c9a0 0x562b6be7ed90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009830 tx=0x7f8d60030670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bfd80572ffef30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d6d7fa640 1 -- 172.21.15.33:0/2930513661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60004250 con 0x562b6be7c9a0 2024-09-10T21:28:06.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d6d7fa640 1 -- 172.21.15.33:0/2930513661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d60040030 con 0x562b6be7c9a0 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d6d7fa640 1 -- 172.21.15.33:0/2930513661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d600055a0 con 0x562b6be7c9a0 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d74ff9640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8d640b54f0 0x7f8d640b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d74ff9640 1 -- 172.21.15.33:0/197006144 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8d680e5970 con 0x7f8d7810a150 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.863+0000 7f8d777fe640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8d640b54f0 0x7f8d640b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2930513661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be7c9a0 msgr2=0x562b6be7ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d777fe640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8d640b54f0 0x7f8d640b79b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009500 tx=0x7f8d7000b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be7c9a0 0x562b6be7ed90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009830 tx=0x7f8d60030670 comp rx=0 tx=0).stop 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2930513661 shutdown_connections 2024-09-10T21:28:06.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be7c9a0 0x562b6be7ed90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2930513661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be704d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2930513661 >> 172.21.15.33:0/2930513661 conn(0x562b6be6df20 msgr2=0x562b6be6f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2930513661 shutdown_connections 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2930513661 wait complete. 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.864+0000 7f8d7f8b8740 1 -- start start 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 1 Processor -- start 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be7c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be70a10 0x562b6be796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be70a10 0x562b6be796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be70a10 0x562b6be796c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60678/0 (socket says 172.21.15.33:60678) 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be79ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 -- 172.21.15.33:0/1514390093 learned_addr learned my addr 172.21.15.33:0/1514390093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x562b6bc37700 con 0x562b6be70a10 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x562b6bd71160 con 0x562b6be7c9a0 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b6be7f5c0 con 0x562b6be700f0 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be7c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d7f8b8740 10 client.0 _ll_register_callbacks cb 0x562b6bd8a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d777fe640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be79ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 -- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 msgr2=0x562b6be7c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be7c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 -- 172.21.15.33:0/1514390093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 msgr2=0x562b6be79ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be79ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d6e7fc640 1 -- 172.21.15.33:0/1514390093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6bd9ec90 con 0x562b6be70a10 2024-09-10T21:28:06.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.865+0000 7f8d777fe640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be79ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be70a10 0x562b6be796c0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8d6002a430 tx=0x7f8d600094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f56ae07721c59545 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1514390093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60005c50 con 0x562b6be70a10 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1514390093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8d60004260 con 0x562b6be70a10 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d60040030 con 0x562b6be70a10 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1514390093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f8d60004460 con 0x562b6be70a10 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 10 client.19617 fetch_fsmap learned FSMap version 93 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) -- 0x562b6be8aeb0 con 0x562b6be70a10 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1514390093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d60040030 con 0x562b6be70a10 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1514390093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 93) ==== 8382+0+0 (secure 0 0 0) 0x7f8d60003b70 con 0x562b6be70a10 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 10 client.19617 fetch_fsmap finished waiting for FSMap version 93 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 10 client.19617 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:28:06.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 20 client.19617 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6be8c200 0x562b6be8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 4 client.19617 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.866+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b6bde8df0 con 0x562b6be8c200 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.867+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6be8c200 0x562b6be8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.867+0000 7f8d5e7fc640 10 client.19617 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.867+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6be8c200 0x562b6be8e5f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ae8d69f4d21bc00f server_cookie=d5f0d12d12a9e9f2 in_seq=0 out_seq=0 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1514390093 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b6bde8df0 con 0x562b6be8c200 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d5e7fc640 10 client.19617 handle_command_reply: tid=0 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 client.19617 shutdown 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be70a10 msgr2=0x562b6be796c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be70a10 0x562b6be796c0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f8d6002a430 tx=0x7f8d600094e0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 shutdown_connections 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6be8c200 0x562b6be8e5f0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be79ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be70a10 0x562b6be796c0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1514390093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be700f0 0x562b6be7c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 >> 172.21.15.33:0/1514390093 conn(0x562b6be6df20 msgr2=0x562b6be73ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d5e7fc640 0 client.19617 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 shutdown_connections 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1514390093 wait complete. 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.868+0000 7f8d7f8b8740 20 client.19617 trim_cache size 0 max 16384 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.882+0000 7f8d7f8b8740 1 Processor -- start 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.882+0000 7f8d7f8b8740 1 -- start start 2024-09-10T21:28:06.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.882+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be6d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.882+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be6d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be7c9a0 0x562b6be7e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be6d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60686/0 (socket says 172.21.15.33:60686) 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781bd2a0 con 0x562b6be700f0 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 -- 172.21.15.33:0/2850488515 learned_addr learned my addr 172.21.15.33:0/2850488515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781bee70 con 0x562b6be70a10 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781bd970 con 0x562b6be7c9a0 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d777fe640 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be7c9a0 0x562b6be7e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 -- 172.21.15.33:0/2850488515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be7c9a0 msgr2=0x562b6be7e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be7c9a0 0x562b6be7e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 -- 172.21.15.33:0/2850488515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 msgr2=0x562b6be7dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 -- 172.21.15.33:0/2850488515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b6be7a010 con 0x562b6be700f0 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be6d760 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800d6a0 tx=0x7f8d68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c71cb02d3d0800 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d6cff9640 1 -- 172.21.15.33:0/2850488515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68021040 con 0x562b6be700f0 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d6cff9640 1 -- 172.21.15.33:0/2850488515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d680abc00 con 0x562b6be700f0 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.883+0000 7f8d6cff9640 1 -- 172.21.15.33:0/2850488515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68010910 con 0x562b6be700f0 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 msgr2=0x562b6be6d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be6d760 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800d6a0 tx=0x7f8d68000bc0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 shutdown_connections 2024-09-10T21:28:06.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be7c9a0 0x562b6be7e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be70a10 0x562b6be7dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/2850488515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be6d760 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 >> 172.21.15.33:0/2850488515 conn(0x562b6be6df20 msgr2=0x562b6be7b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 shutdown_connections 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.884+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/2850488515 wait complete. 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 -- start start 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 Processor -- start 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be84a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be70a10 0x562b6be84fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be7f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781be740 con 0x562b6be700f0 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781be040 con 0x562b6be7c9a0 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d781bb260 con 0x562b6be70a10 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be84a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d7f8b8740 10 client.0 _ll_register_callbacks cb 0x7f8d70009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be84a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60692/0 (socket says 172.21.15.33:60692) 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 -- 172.21.15.33:0/1503521277 learned_addr learned my addr 172.21.15.33:0/1503521277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be70a10 0x562b6be84fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d777fe640 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be7f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 -- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be70a10 msgr2=0x562b6be84fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be70a10 0x562b6be84fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 -- 172.21.15.33:0/1503521277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 msgr2=0x562b6be7f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:06.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be7f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.885+0000 7f8d76ffd640 1 -- 172.21.15.33:0/1503521277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b6be80040 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d76ffd640 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be84a40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f8d680058c0 tx=0x7f8d68005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb10e191365175a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d5e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1503521277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68021070 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1503521277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8d68009d40 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d680abc00 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1503521277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f8d680b0bd0 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 10 client.19623 fetch_fsmap learned FSMap version 93 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) -- 0x562b6bef9850 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1503521277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d680abc00 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1503521277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 93) ==== 8382+0+0 (secure 0 0 0) 0x7f8d68097170 con 0x562b6be700f0 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 10 client.19623 fetch_fsmap finished waiting for FSMap version 93 2024-09-10T21:28:06.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 10 client.19623 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 20 client.19623 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6befa030 0x562b6befc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 4 client.19623 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.886+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b6bde48a0 con 0x562b6befa030 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.887+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6befa030 0x562b6befc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.887+0000 7f8d6e7fc640 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6befa030 0x562b6befc450 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=78dbc21863da68cc server_cookie=521487773b42a9d6 in_seq=0 out_seq=0 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.887+0000 7f8d5e7fc640 10 client.19623 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.887+0000 7f8d5e7fc640 1 -- 172.21.15.33:0/1503521277 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b6bde48a0 con 0x562b6befa030 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.887+0000 7f8d5e7fc640 10 client.19623 handle_command_reply: tid=0 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.887+0000 7f8d7f8b8740 1 client.19623 shutdown 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 msgr2=0x562b6be84a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be84a40 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f8d680058c0 tx=0x7f8d68005d20 comp rx=0 tx=0).stop 2024-09-10T21:28:06.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 shutdown_connections 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x562b6befa030 0x562b6befc450 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x562b6be7c9a0 0x562b6be7f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b6be70a10 0x562b6be84fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 --2- 172.21.15.33:0/1503521277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x562b6be700f0 0x562b6be84a40 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d5e7fc640 0 client.19623 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 >> 172.21.15.33:0/1503521277 conn(0x562b6be6df20 msgr2=0x562b6be73150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 shutdown_connections 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 1 -- 172.21.15.33:0/1503521277 wait complete. 2024-09-10T21:28:06.324 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7f8b8740 20 client.19623 trim_cache size 0 max 16384 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8d640b54f0 msgr2=0x7f8d640b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8d640b54f0 0x7f8d640b79b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009500 tx=0x7f8d7000b040 comp rx=0 tx=0).stop 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7810a150 msgr2=0x7f8d781a8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7810a150 0x7f8d781a8120 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800d6d0 tx=0x7f8d6800db90 comp rx=0 tx=0).stop 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.888+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 shutdown_connections 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.889+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f8d640b54f0 0x7f8d640b79b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.889+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8d7811c0a0 0x7f8d781ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.889+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d7810a530 0x7f8d781a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.889+0000 7f8d7d4e3640 1 --2- 172.21.15.33:0/197006144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d7810a150 0x7f8d781a8120 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.889+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 >> 172.21.15.33:0/197006144 conn(0x7f8d7806dfc0 msgr2=0x7f8d7811c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.889+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 shutdown_connections 2024-09-10T21:28:06.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:05.889+0000 7f8d7d4e3640 1 -- 172.21.15.33:0/197006144 wait complete. 2024-09-10T21:28:06.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:06 smithi174 ceph-mon[35635]: pgmap v1069: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.9 MiB/s wr, 742 op/s 2024-09-10T21:28:06.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:06 smithi033 ceph-mon[30860]: pgmap v1069: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.9 MiB/s wr, 742 op/s 2024-09-10T21:28:06.560 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:06 smithi104 ceph-mon[35325]: pgmap v1069: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.9 MiB/s wr, 742 op/s 2024-09-10T21:28:07.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 481481 37.42 MB/sec execute 547 sec latency 348.200 ms 2024-09-10T21:28:08.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 482027 37.40 MB/sec execute 548 sec latency 150.329 ms 2024-09-10T21:28:08.160 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:28:08.362 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:08 smithi104 ceph-mon[35325]: pgmap v1070: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 650 op/s 2024-09-10T21:28:08.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:08 smithi174 ceph-mon[35635]: pgmap v1070: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 650 op/s 2024-09-10T21:28:08.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:08 smithi033 ceph-mon[30860]: pgmap v1070: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 10 MiB/s wr, 650 op/s 2024-09-10T21:28:08.543 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 -- 172.21.15.33:0/1452064470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 msgr2=0x7f819010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 --2- 172.21.15.33:0/1452064470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819010e820 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f8188009ca0 tx=0x7f818802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 -- 172.21.15.33:0/1452064470 shutdown_connections 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 --2- 172.21.15.33:0/1452064470 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819010e820 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 --2- 172.21.15.33:0/1452064470 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81901076a0 0x7f8190107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 --2- 172.21.15.33:0/1452064470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8190101c90 0x7f8190102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 -- 172.21.15.33:0/1452064470 >> 172.21.15.33:0/1452064470 conn(0x7f81900fd700 msgr2=0x7f81900ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:09.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 -- 172.21.15.33:0/1452064470 shutdown_connections 2024-09-10T21:28:09.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.084+0000 7f8195887640 1 -- 172.21.15.33:0/1452064470 wait complete. 2024-09-10T21:28:09.088 INFO:tasks.workunit.client.0.smithi033.stdout: 10 482682 37.41 MB/sec execute 549 sec latency 118.046 ms 2024-09-10T21:28:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.086+0000 7f8195887640 1 Processor -- start 2024-09-10T21:28:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.086+0000 7f8195887640 1 -- start start 2024-09-10T21:28:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.086+0000 7f8195887640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8190101c90 0x7f8190199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.086+0000 7f8195887640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81901076a0 0x7f8190199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.086+0000 7f8195887640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f8195887640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f8190112e30 con 0x7f8190108020 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f8195887640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f8190112cb0 con 0x7f8190101c90 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f8195887640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8190112fb0 con 0x7f81901076a0 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60712/0 (socket says 172.21.15.33:60712) 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 -- 172.21.15.33:0/3570828492 learned_addr learned my addr 172.21.15.33:0/3570828492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818e7fc640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81901076a0 0x7f8190199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818effd640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8190101c90 0x7f8190199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 -- 172.21.15.33:0/3570828492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81901076a0 msgr2=0x7f8190199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81901076a0 0x7f8190199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 -- 172.21.15.33:0/3570828492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8190101c90 msgr2=0x7f8190199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8190101c90 0x7f8190199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.087+0000 7f818f7fe640 1 -- 172.21.15.33:0/3570828492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819019f0f0 con 0x7f8190108020 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.088+0000 7f818f7fe640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819019e920 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f8188009bf0 tx=0x7f8188031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.088+0000 7f8194885640 1 -- 172.21.15.33:0/3570828492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8188003bf0 con 0x7f8190108020 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.088+0000 7f8194885640 1 -- 172.21.15.33:0/3570828492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8188003d90 con 0x7f8190108020 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.088+0000 7f8194885640 1 -- 172.21.15.33:0/3570828492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818802fc90 con 0x7f8190108020 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.088+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819019f380 con 0x7f8190108020 2024-09-10T21:28:09.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.089+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81901a5aa0 con 0x7f8190108020 2024-09-10T21:28:09.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.092+0000 7f8197c5c740 1 Processor -- start 2024-09-10T21:28:09.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.093+0000 7f8194885640 1 -- 172.21.15.33:0/3570828492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f8188004040 con 0x7f8190108020 2024-09-10T21:28:09.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.093+0000 7f8197c5c740 1 -- start start 2024-09-10T21:28:09.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea266b70 0x5653ea266f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea276490 0x5653ea2768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea280800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f8197c5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653ea1dfea0 con 0x5653ea266b70 2024-09-10T21:28:09.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f8197c5c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5653ea1cf2d0 con 0x5653ea276e10 2024-09-10T21:28:09.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f8197c5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653ea1d30e0 con 0x5653ea276490 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f818e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea266b70 0x5653ea266f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f818effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea276490 0x5653ea2768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f818effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea276490 0x5653ea2768d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:34380/0 (socket says 172.21.15.33:34380) 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f818effd640 1 -- 172.21.15.33:0/144108478 learned_addr learned my addr 172.21.15.33:0/144108478 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f8194885640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81780b5430 0x7f81780b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f818e7fc640 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea280800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.094+0000 7f818f7fe640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81780b5430 0x7f81780b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818effd640 1 -- 172.21.15.33:0/144108478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 msgr2=0x5653ea280800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818effd640 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea280800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818effd640 1 -- 172.21.15.33:0/144108478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea266b70 msgr2=0x5653ea266f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818effd640 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea266b70 0x5653ea266f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818effd640 1 -- 172.21.15.33:0/144108478 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653e9f2d650 con 0x5653ea276490 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818e7fc640 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea266b70 0x5653ea266f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f8194885640 1 -- 172.21.15.33:0/3570828492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8188106d10 con 0x7f8190108020 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818effd640 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea276490 0x5653ea2768d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8180009d20 tx=0x7f8180011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28c0965454991d61 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f8175ffb640 1 -- 172.21.15.33:0/144108478 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818001e070 con 0x5653ea276490 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f818f7fe640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81780b5430 0x7f81780b78f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f81880ca7d0 tx=0x7f81880fd940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f8175ffb640 1 -- 172.21.15.33:0/144108478 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8180019ae0 con 0x5653ea276490 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.095+0000 7f8175ffb640 1 -- 172.21.15.33:0/144108478 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818000e9f0 con 0x5653ea276490 2024-09-10T21:28:09.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.096+0000 7f8197c5c740 1 -- 172.21.15.33:0/144108478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea276490 msgr2=0x5653ea2768d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.096+0000 7f8197c5c740 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea276490 0x5653ea2768d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8180009d20 tx=0x7f8180011810 comp rx=0 tx=0).stop 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.096+0000 7f8197c5c740 1 -- 172.21.15.33:0/144108478 shutdown_connections 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.096+0000 7f8197c5c740 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea280800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.096+0000 7f8197c5c740 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea276490 0x5653ea2768d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.096+0000 7f8197c5c740 1 --2- 172.21.15.33:0/144108478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea266b70 0x5653ea266f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.096+0000 7f8197c5c740 1 -- 172.21.15.33:0/144108478 >> 172.21.15.33:0/144108478 conn(0x5653ea269090 msgr2=0x5653ea269470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.097+0000 7f8197c5c740 1 -- 172.21.15.33:0/144108478 shutdown_connections 2024-09-10T21:28:09.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.097+0000 7f8197c5c740 1 -- 172.21.15.33:0/144108478 wait complete. 2024-09-10T21:28:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.097+0000 7f8197c5c740 1 -- start start 2024-09-10T21:28:09.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 1 Processor -- start 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea280600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea27daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea27e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5653ea030700 con 0x5653ea276490 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5653ea16a160 con 0x5653ea276e10 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653ea280ef0 con 0x5653ea266b70 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.098+0000 7f8197c5c740 10 client.0 _ll_register_callbacks cb 0x5653ea1839e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea27daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea27daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60730/0 (socket says 172.21.15.33:60730) 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 -- 172.21.15.33:0/2972246199 learned_addr learned my addr 172.21.15.33:0/2972246199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818e7fc640 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea280600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818e7fc640 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea27e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 -- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 msgr2=0x5653ea280600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea280600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 -- 172.21.15.33:0/2972246199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 msgr2=0x5653ea27e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea27e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.099+0000 7f818effd640 1 -- 172.21.15.33:0/2972246199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ea197c90 con 0x5653ea276490 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.100+0000 7f818effd640 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea27daa0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f818000a010 tx=0x7f8180004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9811760de477b193 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.100+0000 7f816effd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.100+0000 7f816effd640 1 -- 172.21.15.33:0/2972246199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180027020 con 0x5653ea276490 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.100+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8180019ae0 con 0x5653ea276490 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.100+0000 7f816effd640 1 -- 172.21.15.33:0/2972246199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f8180019630 con 0x5653ea276490 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.101+0000 7f816effd640 1 -- 172.21.15.33:0/2972246199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f818001e070 con 0x5653ea276490 2024-09-10T21:28:09.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.101+0000 7f8197c5c740 10 client.19635 fetch_fsmap learned FSMap version 93 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.101+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) -- 0x5653ea28c7b0 con 0x5653ea276490 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.101+0000 7f816effd640 1 -- 172.21.15.33:0/2972246199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8180019ae0 con 0x5653ea276490 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.101+0000 7f816effd640 1 -- 172.21.15.33:0/2972246199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 93) ==== 8382+0+0 (secure 0 0 0) 0x7f8180004a60 con 0x5653ea276490 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.102+0000 7f8197c5c740 10 client.19635 fetch_fsmap finished waiting for FSMap version 93 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.102+0000 7f8197c5c740 10 client.19635 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.102+0000 7f8197c5c740 20 client.19635 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.102+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea28e000 0x5653ea290420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.102+0000 7f8197c5c740 4 client.19635 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.102+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ea1e1df0 con 0x5653ea28e000 2024-09-10T21:28:09.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.102+0000 7f818e7fc640 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea28e000 0x5653ea290420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.104+0000 7f818e7fc640 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea28e000 0x5653ea290420 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=debe1a45aa68df34 server_cookie=b77ee532f9faa68 in_seq=0 out_seq=0 2024-09-10T21:28:09.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.104+0000 7f816effd640 10 client.19635 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:28:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f816effd640 1 -- 172.21.15.33:0/2972246199 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5653ea1e1df0 con 0x5653ea28e000 2024-09-10T21:28:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f816effd640 10 client.19635 handle_command_reply: tid=0 2024-09-10T21:28:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 client.19635 shutdown 2024-09-10T21:28:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 msgr2=0x5653ea27daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea27daa0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f818000a010 tx=0x7f8180004270 comp rx=0 tx=0).stop 2024-09-10T21:28:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 shutdown_connections 2024-09-10T21:28:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea27e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea27daa0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea28e000 0x5653ea290420 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2972246199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea280600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.107+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 >> 172.21.15.33:0/2972246199 conn(0x5653ea269090 msgr2=0x5653ea278da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.108+0000 7f816effd640 0 client.19635 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:28:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.108+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 shutdown_connections 2024-09-10T21:28:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.108+0000 7f8197c5c740 1 -- 172.21.15.33:0/2972246199 wait complete. 2024-09-10T21:28:09.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.108+0000 7f8197c5c740 20 client.19635 trim_cache size 0 max 16384 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.123+0000 7f8197c5c740 1 Processor -- start 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.123+0000 7f8197c5c740 1 -- start start 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.123+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea268b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276490 0x5653ea28a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276e10 0x5653ea27b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f8197c5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ac060 con 0x5653ea276e10 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f8197c5c740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81901adc30 con 0x5653ea276490 2024-09-10T21:28:09.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f8197c5c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ac730 con 0x5653ea266b70 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276490 0x5653ea28a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276490 0x5653ea28a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:48628/0 (socket says 172.21.15.33:48628) 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 -- 172.21.15.33:0/2850026916 learned_addr learned my addr 172.21.15.33:0/2850026916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818e7fc640 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276e10 0x5653ea27b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 -- 172.21.15.33:0/2850026916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 msgr2=0x5653ea268b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea268b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 -- 172.21.15.33:0/2850026916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276e10 msgr2=0x5653ea27b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276e10 0x5653ea27b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.124+0000 7f818effd640 1 -- 172.21.15.33:0/2850026916 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ea27d850 con 0x5653ea276490 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.125+0000 7f818effd640 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276490 0x5653ea28a0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8180009800 tx=0x7f818000e520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97eee743ab690d0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.125+0000 7f81757fa640 1 -- 172.21.15.33:0/2850026916 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180027070 con 0x5653ea276490 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.125+0000 7f81757fa640 1 -- 172.21.15.33:0/2850026916 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818001f040 con 0x5653ea276490 2024-09-10T21:28:09.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.125+0000 7f81757fa640 1 -- 172.21.15.33:0/2850026916 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8180019d30 con 0x5653ea276490 2024-09-10T21:28:09.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 -- 172.21.15.33:0/2850026916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276490 msgr2=0x5653ea28a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276490 0x5653ea28a0e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8180009800 tx=0x7f818000e520 comp rx=0 tx=0).stop 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 -- 172.21.15.33:0/2850026916 shutdown_connections 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276e10 0x5653ea27b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276490 0x5653ea28a0e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 --2- 172.21.15.33:0/2850026916 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea268b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 -- 172.21.15.33:0/2850026916 >> 172.21.15.33:0/2850026916 conn(0x5653ea269090 msgr2=0x5653ea268400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 -- 172.21.15.33:0/2850026916 shutdown_connections 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 -- 172.21.15.33:0/2850026916 wait complete. 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.126+0000 7f8197c5c740 1 -- start start 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f8197c5c740 1 Processor -- start 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea2862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea286830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f8197c5c740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea294ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f8197c5c740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ad500 con 0x5653ea276490 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f818e7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea2862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f818effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea286830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f818effd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea286830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60748/0 (socket says 172.21.15.33:60748) 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f818effd640 1 -- 172.21.15.33:0/1850875215 learned_addr learned my addr 172.21.15.33:0/1850875215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f81901ace00 con 0x5653ea276e10 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.127+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81901aa040 con 0x5653ea266b70 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f818e7fc640 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea294ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f8197c5c740 10 client.0 _ll_register_callbacks cb 0x7f818001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f818e7fc640 1 -- 172.21.15.33:0/1850875215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 msgr2=0x5653ea294ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f818e7fc640 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea294ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f818e7fc640 1 -- 172.21.15.33:0/1850875215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 msgr2=0x5653ea286830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f818e7fc640 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea286830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f818e7fc640 1 -- 172.21.15.33:0/1850875215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ea295320 con 0x5653ea266b70 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f818e7fc640 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea2862f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8184009fd0 tx=0x7f8184005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=436b0636411eb95c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.128+0000 7f816effd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.129+0000 7f816effd640 1 -- 172.21.15.33:0/1850875215 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8184003e50 con 0x5653ea266b70 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.129+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f818001f040 con 0x5653ea266b70 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.129+0000 7f816effd640 1 -- 172.21.15.33:0/1850875215 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f818402e430 con 0x5653ea266b70 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.129+0000 7f816effd640 1 -- 172.21.15.33:0/1850875215 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f818402e650 con 0x5653ea266b70 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.129+0000 7f8197c5c740 10 client.28825 fetch_fsmap learned FSMap version 93 2024-09-10T21:28:09.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.129+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=93,monmap=4+}) -- 0x5653ea296630 con 0x5653ea266b70 2024-09-10T21:28:09.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f816effd640 1 -- 172.21.15.33:0/1850875215 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818001f040 con 0x5653ea266b70 2024-09-10T21:28:09.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f816effd640 1 -- 172.21.15.33:0/1850875215 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 93) ==== 8382+0+0 (secure 0 0 0) 0x7f818402d3e0 con 0x5653ea266b70 2024-09-10T21:28:09.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f8197c5c740 10 client.28825 fetch_fsmap finished waiting for FSMap version 93 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f8197c5c740 10 client.28825 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f8197c5c740 20 client.28825 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f8197c5c740 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea2fc110 0x5653ea2fe530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f8197c5c740 4 client.28825 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5653ea1dd8a0 con 0x5653ea2fc110 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.130+0000 7f818effd640 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea2fc110 0x5653ea2fe530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.131+0000 7f816effd640 10 client.28825 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.131+0000 7f818effd640 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea2fc110 0x5653ea2fe530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=288f856d2ac50c1 server_cookie=c324067357ba0ee5 in_seq=0 out_seq=0 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.133+0000 7f816effd640 1 -- 172.21.15.33:0/1850875215 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5653ea1dd8a0 con 0x5653ea2fc110 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.133+0000 7f816effd640 10 client.28825 handle_command_reply: tid=0 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.133+0000 7f8197c5c740 1 client.28825 shutdown 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.133+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 msgr2=0x5653ea2862f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea2862f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8184009fd0 tx=0x7f8184005f90 comp rx=0 tx=0).stop 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 shutdown_connections 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5653ea276e10 0x5653ea294ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5653ea276490 0x5653ea286830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x5653ea2fc110 0x5653ea2fe530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 --2- 172.21.15.33:0/1850875215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653ea266b70 0x5653ea2862f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f816effd640 0 client.28825 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 >> 172.21.15.33:0/1850875215 conn(0x5653ea269090 msgr2=0x5653ea27f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 shutdown_connections 2024-09-10T21:28:09.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.134+0000 7f8197c5c740 1 -- 172.21.15.33:0/1850875215 wait complete. 2024-09-10T21:28:09.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.136+0000 7f8197c5c740 20 client.28825 trim_cache size 0 max 16384 2024-09-10T21:28:09.139 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:28:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81780b5430 msgr2=0x7f81780b78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81780b5430 0x7f81780b78f0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f81880ca7d0 tx=0x7f81880fd940 comp rx=0 tx=0).stop 2024-09-10T21:28:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 msgr2=0x7f819019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:09.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819019e920 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f8188009bf0 tx=0x7f8188031a10 comp rx=0 tx=0).stop 2024-09-10T21:28:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 shutdown_connections 2024-09-10T21:28:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f81780b5430 0x7f81780b78f0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8190108020 0x7f819019e920 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81901076a0 0x7f8190199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 --2- 172.21.15.33:0/3570828492 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8190101c90 0x7f8190199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.138+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 >> 172.21.15.33:0/3570828492 conn(0x7f81900fd700 msgr2=0x7f81900fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:09.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.139+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 shutdown_connections 2024-09-10T21:28:09.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:09.139+0000 7f8195887640 1 -- 172.21.15.33:0/3570828492 wait complete. 2024-09-10T21:28:10.004 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:28:10.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 483490 37.42 MB/sec execute 550 sec latency 200.251 ms 2024-09-10T21:28:10.389 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:10.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:10 smithi174 ceph-mon[35635]: pgmap v1071: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 739 op/s 2024-09-10T21:28:10.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:10 smithi033 ceph-mon[30860]: pgmap v1071: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 739 op/s 2024-09-10T21:28:10.560 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:10 smithi104 ceph-mon[35325]: pgmap v1071: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 11 MiB/s wr, 739 op/s 2024-09-10T21:28:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.909+0000 7f42c3d24640 1 -- 172.21.15.33:0/2376988375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc103ce0 msgr2=0x7f42bc1040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.909+0000 7f42c3d24640 1 --2- 172.21.15.33:0/2376988375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc103ce0 0x7f42bc1040c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.910+0000 7f42c3d24640 1 -- 172.21.15.33:0/2376988375 shutdown_connections 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.910+0000 7f42c3d24640 1 --2- 172.21.15.33:0/2376988375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc10d140 0x7f42bc10f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.910+0000 7f42c3d24640 1 --2- 172.21.15.33:0/2376988375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42bc104600 0x7f42bc10caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.910+0000 7f42c3d24640 1 --2- 172.21.15.33:0/2376988375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc103ce0 0x7f42bc1040c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.910+0000 7f42c3d24640 1 -- 172.21.15.33:0/2376988375 >> 172.21.15.33:0/2376988375 conn(0x7f42bc0fd740 msgr2=0x7f42bc0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.910+0000 7f42c3d24640 1 -- 172.21.15.33:0/2376988375 shutdown_connections 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.910+0000 7f42c3d24640 1 -- 172.21.15.33:0/2376988375 wait complete. 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 Processor -- start 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 -- start start 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42bc103ce0 0x7f42bc199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc104600 0x7f42bc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc10d140 0x7f42bc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112e00 con 0x7f42bc103ce0 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112c80 con 0x7f42bc104600 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c3d24640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc112f80 con 0x7f42bc10d140 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c12bf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc104600 0x7f42bc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c12bf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc104600 0x7f42bc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54522/0 (socket says 172.21.15.33:54522) 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c12bf640 1 -- 172.21.15.33:0/3985811924 learned_addr learned my addr 172.21.15.33:0/3985811924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42bc103ce0 0x7f42bc199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.911+0000 7f42c22c1640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc10d140 0x7f42bc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42c12bf640 1 -- 172.21.15.33:0/3985811924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc10d140 msgr2=0x7f42bc19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42c12bf640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc10d140 0x7f42bc19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42c12bf640 1 -- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42bc103ce0 msgr2=0x7f42bc199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42c12bf640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42bc103ce0 0x7f42bc199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42c12bf640 1 -- 172.21.15.33:0/3985811924 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42bc19f400 con 0x7f42bc104600 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42c12bf640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc104600 0x7f42bc199890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e9c0 tx=0x7f42ac00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42b2ffd640 1 -- 172.21.15.33:0/3985811924 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00ccc0 con 0x7f42bc104600 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42b2ffd640 1 -- 172.21.15.33:0/3985811924 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac0035f0 con 0x7f42bc104600 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42b2ffd640 1 -- 172.21.15.33:0/3985811924 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac00ce60 con 0x7f42bc104600 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.912+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42bc19f6f0 con 0x7f42bc104600 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.913+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42bc101a40 con 0x7f42bc104600 2024-09-10T21:28:10.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 Processor -- start 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 -- start start 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b058b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b11840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 0x5591a8b14170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591a8a7bea0 con 0x5591a8b054d0 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591a8a6b2d0 con 0x5591a8b11d80 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.916+0000 7f42c60f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5591a8a6f0e0 con 0x5591a8b05df0 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b058b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b058b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44630/0 (socket says 172.21.15.33:44630) 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 -- 172.21.15.33:0/403415813 learned_addr learned my addr 172.21.15.33:0/403415813 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 -- 172.21.15.33:0/403415813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 msgr2=0x5591a8b11840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:28:10.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 --2- 172.21.15.33:0/403415813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b11840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 -- 172.21.15.33:0/403415813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 msgr2=0x5591a8b14170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 --2- 172.21.15.33:0/403415813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 0x5591a8b14170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.917+0000 7f42c22c1640 1 -- 172.21.15.33:0/403415813 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a87c9650 con 0x5591a8b054d0 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.918+0000 7f42c22c1640 1 --2- 172.21.15.33:0/403415813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b058b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009f50 tx=0x7f42b40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342a03ff94319f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.918+0000 7f42b2ffd640 1 -- 172.21.15.33:0/3985811924 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f42ac004000 con 0x7f42bc104600 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.919+0000 7f42a7fff640 1 -- 172.21.15.33:0/403415813 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4004b20 con 0x5591a8b054d0 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.919+0000 7f42a7fff640 1 -- 172.21.15.33:0/403415813 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4004cc0 con 0x5591a8b054d0 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.919+0000 7f42b2ffd640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f42a80b53e0 0x7f42a80b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.919+0000 7f42a7fff640 1 -- 172.21.15.33:0/403415813 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4005420 con 0x5591a8b054d0 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.919+0000 7f42b2ffd640 1 -- 172.21.15.33:0/3985811924 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f42ac0e5e30 con 0x7f42bc104600 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f42a80b53e0 0x7f42a80b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 -- 172.21.15.33:0/403415813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 msgr2=0x5591a8b058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 --2- 172.21.15.33:0/403415813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b058b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f42b4009f50 tx=0x7f42b40118d0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 -- 172.21.15.33:0/403415813 shutdown_connections 2024-09-10T21:28:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 --2- 172.21.15.33:0/403415813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 0x5591a8b14170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 --2- 172.21.15.33:0/403415813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b11840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 --2- 172.21.15.33:0/403415813 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b058b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 -- 172.21.15.33:0/403415813 >> 172.21.15.33:0/403415813 conn(0x5591a8b03300 msgr2=0x5591a8b044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:10.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.920+0000 7f42c60f9740 1 -- 172.21.15.33:0/403415813 shutdown_connections 2024-09-10T21:28:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.921+0000 7f42c60f9740 1 -- 172.21.15.33:0/403415813 wait complete. 2024-09-10T21:28:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.921+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f42a80b53e0 0x7f42a80b78a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005ab0 tx=0x7f42b803c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.921+0000 7f42c60f9740 1 -- start start 2024-09-10T21:28:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.921+0000 7f42c60f9740 1 Processor -- start 2024-09-10T21:28:10.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b11610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b0ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 0x5591a8b0f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c60f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5591a88cc700 con 0x5591a8b054d0 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c60f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5591a8a06160 con 0x5591a8b11d80 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c60f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5591a8b1d260 con 0x5591a8b05df0 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c60f9740 10 client.0 _ll_register_callbacks cb 0x5591a8a1f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c1ac0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b11610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c1ac0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b11610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44644/0 (socket says 172.21.15.33:44644) 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c1ac0640 1 -- 172.21.15.33:0/3624151454 learned_addr learned my addr 172.21.15.33:0/3624151454 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c22c1640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 0x5591a8b0f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.922+0000 7f42c12bf640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b0ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.923+0000 7f42c1ac0640 1 -- 172.21.15.33:0/3624151454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 msgr2=0x5591a8b0ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:10.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.923+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b0ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.923+0000 7f42c1ac0640 1 -- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 msgr2=0x5591a8b0f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.923+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 0x5591a8b0f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.923+0000 7f42c1ac0640 1 -- 172.21.15.33:0/3624151454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a8a33c90 con 0x5591a8b054d0 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.923+0000 7f42c12bf640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.924+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b11610 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ac55e515c86bc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.924+0000 7f42a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.924+0000 7f42a57fa640 1 -- 172.21.15.33:0/3624151454 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b804c070 con 0x5591a8b054d0 2024-09-10T21:28:10.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.924+0000 7f42a57fa640 1 -- 172.21.15.33:0/3624151454 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f42b8004b80 con 0x5591a8b054d0 2024-09-10T21:28:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.926+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b4004cc0 con 0x5591a8b054d0 2024-09-10T21:28:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.926+0000 7f42a57fa640 1 -- 172.21.15.33:0/3624151454 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f42b8056050 con 0x5591a8b054d0 2024-09-10T21:28:10.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.926+0000 7f42c60f9740 10 client.19653 fetch_fsmap learned FSMap version 94 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.926+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) -- 0x5591a8b28b80 con 0x5591a8b054d0 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42a57fa640 1 -- 172.21.15.33:0/3624151454 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b4004cc0 con 0x5591a8b054d0 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42a57fa640 1 -- 172.21.15.33:0/3624151454 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 94) ==== 8390+0+0 (secure 0 0 0) 0x7f42b80324b0 con 0x5591a8b054d0 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42c60f9740 10 client.19653 fetch_fsmap finished waiting for FSMap version 94 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42c60f9740 10 client.19653 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42c60f9740 20 client.19653 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42c60f9740 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b2a4a0 0x5591a8b2c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42c60f9740 4 client.19653 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.927+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591a8a7ddf0 con 0x5591a8b2a4a0 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.928+0000 7f42c12bf640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b2a4a0 0x5591a8b2c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.928+0000 7f42c12bf640 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b2a4a0 0x5591a8b2c8e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33705ef3df130698 server_cookie=d93ff4f706daa3bb in_seq=0 out_seq=0 2024-09-10T21:28:10.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.928+0000 7f42a57fa640 10 client.19653 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.929+0000 7f42a57fa640 1 -- 172.21.15.33:0/3624151454 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5591a8a7ddf0 con 0x5591a8b2a4a0 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.929+0000 7f42a57fa640 10 client.19653 handle_command_reply: tid=0 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.929+0000 7f42c60f9740 1 client.19653 shutdown 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.929+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 msgr2=0x5591a8b11610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.929+0000 7f42c60f9740 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b11610 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b8009260 comp rx=0 tx=0).stop 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 shutdown_connections 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b11d80 0x5591a8b0f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b2a4a0 0x5591a8b2c8e0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 --2- 172.21.15.33:0/3624151454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b11610 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 >> 172.21.15.33:0/3624151454 conn(0x5591a8b03300 msgr2=0x5591a8b08e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42a57fa640 0 client.19653 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 shutdown_connections 2024-09-10T21:28:10.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 1 -- 172.21.15.33:0/3624151454 wait complete. 2024-09-10T21:28:10.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.930+0000 7f42c60f9740 20 client.19653 trim_cache size 0 max 16384 2024-09-10T21:28:10.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.957+0000 7f42c60f9740 1 Processor -- start 2024-09-10T21:28:10.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.957+0000 7f42c60f9740 1 -- start start 2024-09-10T21:28:10.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.957+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b04ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.957+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b05df0 0x5591a8b02c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b11d80 0x5591a8b13170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c60f9740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc073130 con 0x5591a8b054d0 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c60f9740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc074d60 con 0x5591a8b05df0 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c60f9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc073830 con 0x5591a8b11d80 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b05df0 0x5591a8b02c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b05df0 0x5591a8b02c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54548/0 (socket says 172.21.15.33:54548) 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 -- 172.21.15.33:0/2133703218 learned_addr learned my addr 172.21.15.33:0/2133703218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b04ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 -- 172.21.15.33:0/2133703218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b11d80 msgr2=0x5591a8b13170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:10.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b11d80 0x5591a8b13170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 -- 172.21.15.33:0/2133703218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 msgr2=0x5591a8b04ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:10.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b04ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 -- 172.21.15.33:0/2133703218 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a8b0f8d0 con 0x5591a8b05df0 2024-09-10T21:28:10.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b04ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:10.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.958+0000 7f42c12bf640 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b05df0 0x5591a8b02c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00c6a0 tx=0x7f42ac003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a8275a66c06c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.959+0000 7f42a77fe640 1 -- 172.21.15.33:0/2133703218 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0d3070 con 0x5591a8b05df0 2024-09-10T21:28:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.959+0000 7f42a77fe640 1 -- 172.21.15.33:0/2133703218 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42ac0accb0 con 0x5591a8b05df0 2024-09-10T21:28:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.959+0000 7f42a77fe640 1 -- 172.21.15.33:0/2133703218 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0a8820 con 0x5591a8b05df0 2024-09-10T21:28:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 -- 172.21.15.33:0/2133703218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b05df0 msgr2=0x5591a8b02c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b05df0 0x5591a8b02c10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00c6a0 tx=0x7f42ac003e20 comp rx=0 tx=0).stop 2024-09-10T21:28:10.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 -- 172.21.15.33:0/2133703218 shutdown_connections 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b11d80 0x5591a8b13170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b05df0 0x5591a8b02c10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2133703218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b054d0 0x5591a8b04ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 -- 172.21.15.33:0/2133703218 >> 172.21.15.33:0/2133703218 conn(0x5591a8b03300 msgr2=0x5591a8b043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 -- 172.21.15.33:0/2133703218 shutdown_connections 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 -- 172.21.15.33:0/2133703218 wait complete. 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.960+0000 7f42c60f9740 1 -- start start 2024-09-10T21:28:10.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.961+0000 7f42c60f9740 1 Processor -- start 2024-09-10T21:28:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.961+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b054d0 0x5591a8b22550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.962+0000 7f42c60f9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b22ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.962+0000 7f42c1ac0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b054d0 0x5591a8b22550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.962+0000 7f42c1ac0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b054d0 0x5591a8b22550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54560/0 (socket says 172.21.15.33:54560) 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.962+0000 7f42c1ac0640 1 -- 172.21.15.33:0/2135154043 learned_addr learned my addr 172.21.15.33:0/2135154043 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.962+0000 7f42c12bf640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b22ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.962+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b11d80 0x5591a8b1d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc074630 con 0x5591a8b11d80 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc073f30 con 0x5591a8b054d0 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42bc0710f0 con 0x5591a8b05df0 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c22c1640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b11d80 0x5591a8b1d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c60f9740 10 client.0 _ll_register_callbacks cb 0x7f42b40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c1ac0640 1 -- 172.21.15.33:0/2135154043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 msgr2=0x5591a8b22ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b22ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c1ac0640 1 -- 172.21.15.33:0/2135154043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b11d80 msgr2=0x5591a8b1d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:10.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b11d80 0x5591a8b1d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c1ac0640 1 -- 172.21.15.33:0/2135154043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a8b1dac0 con 0x5591a8b054d0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.963+0000 7f42c22c1640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b11d80 0x5591a8b1d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42c1ac0640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b054d0 0x5591a8b22550 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b593aa990d00f33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42a57fa640 1 -- 172.21.15.33:0/2135154043 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b804c040 con 0x5591a8b054d0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42ac0accb0 con 0x5591a8b054d0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42a57fa640 1 -- 172.21.15.33:0/2135154043 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f42b80035f0 con 0x5591a8b054d0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42a57fa640 1 -- 172.21.15.33:0/2135154043 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7f42b8041070 con 0x5591a8b054d0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42c60f9740 10 client.28976 fetch_fsmap learned FSMap version 94 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.964+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=4+}) -- 0x5591a8b97c70 con 0x5591a8b054d0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.965+0000 7f42a57fa640 1 -- 172.21.15.33:0/2135154043 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b8003ca0 con 0x5591a8b054d0 2024-09-10T21:28:10.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.965+0000 7f42a57fa640 1 -- 172.21.15.33:0/2135154043 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 94) ==== 8390+0+0 (secure 0 0 0) 0x7f42b805b640 con 0x5591a8b054d0 2024-09-10T21:28:10.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.965+0000 7f42c60f9740 10 client.28976 fetch_fsmap finished waiting for FSMap version 94 2024-09-10T21:28:10.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.965+0000 7f42c60f9740 10 client.28976 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:28:10.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.965+0000 7f42c60f9740 20 client.28976 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:10.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.965+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b984d0 0x5591a8b9a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:10.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.965+0000 7f42c60f9740 4 client.28976 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:28:10.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.966+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f42ac0accb0 con 0x5591a8b984d0 2024-09-10T21:28:10.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.966+0000 7f42c12bf640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b984d0 0x5591a8b9a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:10.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.966+0000 7f42c12bf640 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b984d0 0x5591a8b9a8f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=adf4d1650fedf5d2 server_cookie=e41bb0219b25ca4d in_seq=0 out_seq=0 2024-09-10T21:28:10.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.966+0000 7f42a57fa640 10 client.28976 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:28:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.969+0000 7f42a57fa640 1 -- 172.21.15.33:0/2135154043 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f42ac0accb0 con 0x5591a8b984d0 2024-09-10T21:28:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.969+0000 7f42a57fa640 10 client.28976 handle_command_reply: tid=0 2024-09-10T21:28:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.969+0000 7f42c60f9740 1 client.28976 shutdown 2024-09-10T21:28:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.970+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b054d0 msgr2=0x5591a8b22550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.970+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b054d0 0x5591a8b22550 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b8000bc0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.970+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 shutdown_connections 2024-09-10T21:28:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.970+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5591a8b984d0 0x5591a8b9a8f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.970+0000 7f42a57fa640 0 client.28976 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:28:10.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.970+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5591a8b11d80 0x5591a8b1d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.970+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5591a8b05df0 0x5591a8b22ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.971+0000 7f42c60f9740 1 --2- 172.21.15.33:0/2135154043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5591a8b054d0 0x5591a8b22550 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f42b8009900 tx=0x7f42b8000bc0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.971+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 >> 172.21.15.33:0/2135154043 conn(0x5591a8b03300 msgr2=0x5591a8b10090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:10.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.971+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 shutdown_connections 2024-09-10T21:28:10.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.971+0000 7f42c60f9740 1 -- 172.21.15.33:0/2135154043 wait complete. 2024-09-10T21:28:10.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.971+0000 7f42c60f9740 20 client.28976 trim_cache size 0 max 16384 2024-09-10T21:28:10.974 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.972+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f42a80b53e0 msgr2=0x7f42a80b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.972+0000 7f42c3d24640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f42a80b53e0 0x7f42a80b78a0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f42b8005ab0 tx=0x7f42b803c040 comp rx=0 tx=0).stop 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.972+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc104600 msgr2=0x7f42bc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.972+0000 7f42c3d24640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc104600 0x7f42bc199890 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e9c0 tx=0x7f42ac00ee80 comp rx=0 tx=0).stop 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.973+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 shutdown_connections 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.973+0000 7f42c3d24640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f42a80b53e0 0x7f42a80b78a0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.973+0000 7f42c3d24640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f42bc10d140 0x7f42bc19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.973+0000 7f42c3d24640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f42bc104600 0x7f42bc199890 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.973+0000 7f42c3d24640 1 --2- 172.21.15.33:0/3985811924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f42bc103ce0 0x7f42bc199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.973+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 >> 172.21.15.33:0/3985811924 conn(0x7f42bc0fd740 msgr2=0x7f42bc0fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:10.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.974+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 shutdown_connections 2024-09-10T21:28:10.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:10.974+0000 7f42c3d24640 1 -- 172.21.15.33:0/3985811924 wait complete. 2024-09-10T21:28:11.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 484391 37.45 MB/sec execute 551 sec latency 88.600 ms 2024-09-10T21:28:11.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:11 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:11.392 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:11 smithi174 ceph-mon[35635]: pgmap v1072: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 803 op/s 2024-09-10T21:28:11.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:11 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:11.425 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:11 smithi033 ceph-mon[30860]: pgmap v1072: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 803 op/s 2024-09-10T21:28:11.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:11 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:11.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:11 smithi104 ceph-mon[35325]: pgmap v1072: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 13 MiB/s wr, 803 op/s 2024-09-10T21:28:12.067 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:28:12.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 485144 37.46 MB/sec execute 552 sec latency 77.553 ms 2024-09-10T21:28:12.411 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 -- 172.21.15.33:0/3192348310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba081045e0 msgr2=0x7fba0810cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 --2- 172.21.15.33:0/3192348310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba081045e0 0x7fba0810cad0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009900 tx=0x7fb9fc02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 -- 172.21.15.33:0/3192348310 shutdown_connections 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 --2- 172.21.15.33:0/3192348310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba0810d120 0x7fba0810f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 --2- 172.21.15.33:0/3192348310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba081045e0 0x7fba0810cad0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 --2- 172.21.15.33:0/3192348310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba08103cc0 0x7fba081040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 -- 172.21.15.33:0/3192348310 >> 172.21.15.33:0/3192348310 conn(0x7fba080fd740 msgr2=0x7fba080ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:12.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 -- 172.21.15.33:0/3192348310 shutdown_connections 2024-09-10T21:28:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.982+0000 7fba0e521640 1 -- 172.21.15.33:0/3192348310 wait complete. 2024-09-10T21:28:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.983+0000 7fba0e521640 1 Processor -- start 2024-09-10T21:28:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.983+0000 7fba0e521640 1 -- start start 2024-09-10T21:28:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba0e521640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba08103cc0 0x7fba08101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba0e521640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba081045e0 0x7fba08101eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:12.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba0e521640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba0810d120 0x7fba081023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba0e521640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fba08112d90 con 0x7fba08103cc0 2024-09-10T21:28:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba0e521640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fba08112c10 con 0x7fba081045e0 2024-09-10T21:28:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba0e521640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba08112f10 con 0x7fba0810d120 2024-09-10T21:28:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba08103cc0 0x7fba08101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba08103cc0 0x7fba08101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44662/0 (socket says 172.21.15.33:44662) 2024-09-10T21:28:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba07fff640 1 -- 172.21.15.33:0/3399252601 learned_addr learned my addr 172.21.15.33:0/3399252601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:12.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba077fe640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba081045e0 0x7fba08101eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.984+0000 7fba0cabe640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba0810d120 0x7fba081023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba077fe640 1 -- 172.21.15.33:0/3399252601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba0810d120 msgr2=0x7fba081023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba077fe640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba0810d120 0x7fba081023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba077fe640 1 -- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba08103cc0 msgr2=0x7fba08101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:12.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba077fe640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba08103cc0 0x7fba08101970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba077fe640 1 -- 172.21.15.33:0/3399252601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba081a76e0 con 0x7fba081045e0 2024-09-10T21:28:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba077fe640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba081045e0 0x7fba08101eb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc006010 tx=0x7fb9fc004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba057fa640 1 -- 172.21.15.33:0/3399252601 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc03f070 con 0x7fba081045e0 2024-09-10T21:28:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba081a7970 con 0x7fba081045e0 2024-09-10T21:28:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba081a7ea0 con 0x7fba081045e0 2024-09-10T21:28:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba057fa640 1 -- 172.21.15.33:0/3399252601 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9fc003ca0 con 0x7fba081045e0 2024-09-10T21:28:12.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.985+0000 7fba057fa640 1 -- 172.21.15.33:0/3399252601 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc035070 con 0x7fba081045e0 2024-09-10T21:28:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.989+0000 7fba108f6740 1 Processor -- start 2024-09-10T21:28:12.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.990+0000 7fba108f6740 1 -- start start 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.990+0000 7fba108f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 0x557ea7204ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.990+0000 7fba108f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7205220 0x557ea7210c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.990+0000 7fba108f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea72135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.990+0000 7fba108f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ea7172ea0 con 0x557ea7205220 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.990+0000 7fba108f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ea71622d0 con 0x557ea72111b0 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.990+0000 7fba108f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ea71660e0 con 0x557ea7204900 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7205220 0x557ea7210c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7205220 0x557ea7210c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44674/0 (socket says 172.21.15.33:44674) 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba077fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea72135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:12.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 -- 172.21.15.33:0/657385242 learned_addr learned my addr 172.21.15.33:0/657385242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba057fa640 1 -- 172.21.15.33:0/3399252601 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb9fc04b060 con 0x7fba081045e0 2024-09-10T21:28:12.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba0cabe640 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 0x557ea7204ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 -- 172.21.15.33:0/657385242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 msgr2=0x557ea7204ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 0x557ea7204ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 -- 172.21.15.33:0/657385242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 msgr2=0x557ea72135a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea72135a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 -- 172.21.15.33:0/657385242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea6ec0650 con 0x557ea7205220 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.991+0000 7fba07fff640 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7205220 0x557ea7210c70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b890 tx=0x7fb9f800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8023d8c5e24c0892 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.992+0000 7fb9ee7fc640 1 -- 172.21.15.33:0/657385242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8003dd0 con 0x557ea7205220 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.992+0000 7fb9ee7fc640 1 -- 172.21.15.33:0/657385242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f8015070 con 0x557ea7205220 2024-09-10T21:28:12.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.992+0000 7fb9ee7fc640 1 -- 172.21.15.33:0/657385242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8011840 con 0x557ea7205220 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba108f6740 1 -- 172.21.15.33:0/657385242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7205220 msgr2=0x557ea7210c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba108f6740 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7205220 0x557ea7210c70 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b890 tx=0x7fb9f800bd50 comp rx=0 tx=0).stop 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba108f6740 1 -- 172.21.15.33:0/657385242 shutdown_connections 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba108f6740 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea72135a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba108f6740 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7205220 0x557ea7210c70 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba057fa640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9f00b53e0 0x7fb9f00b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba108f6740 1 --2- 172.21.15.33:0/657385242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 0x557ea7204ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.993+0000 7fba108f6740 1 -- 172.21.15.33:0/657385242 >> 172.21.15.33:0/657385242 conn(0x557ea7202730 msgr2=0x557ea72038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.994+0000 7fba0cabe640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9f00b53e0 0x7fb9f00b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:12.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.994+0000 7fba057fa640 1 -- 172.21.15.33:0/3399252601 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb9fc105a80 con 0x7fba081045e0 2024-09-10T21:28:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.994+0000 7fba108f6740 1 -- 172.21.15.33:0/657385242 shutdown_connections 2024-09-10T21:28:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.994+0000 7fba108f6740 1 -- 172.21.15.33:0/657385242 wait complete. 2024-09-10T21:28:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.994+0000 7fba108f6740 1 -- start start 2024-09-10T21:28:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba0cabe640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9f00b53e0 0x7fb9f00b78a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fba00005c40 tx=0x7fba00005bd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:12.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 1 Processor -- start 2024-09-10T21:28:13.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 0x557ea7210940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7205220 0x557ea720dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x557ea6fc3700 con 0x557ea72111b0 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x557ea70fd160 con 0x557ea7205220 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ea7213ed0 con 0x557ea7204900 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba108f6740 10 client.0 _ll_register_callbacks cb 0x557ea71169e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.995+0000 7fba0cabe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44682/0 (socket says 172.21.15.33:44682) 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 -- 172.21.15.33:0/116045637 learned_addr learned my addr 172.21.15.33:0/116045637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba077fe640 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7205220 0x557ea720dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 -- 172.21.15.33:0/116045637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 msgr2=0x557ea7210940 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:28:13.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 0x557ea7210940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 -- 172.21.15.33:0/116045637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7205220 msgr2=0x557ea720dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7205220 0x557ea720dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 -- 172.21.15.33:0/116045637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea712ac90 con 0x557ea72111b0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fba0cabe640 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720e2d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fba00012bb0 tx=0x7fba00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46b2e337fa27ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fb9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.996+0000 7fb9e77fe640 1 -- 172.21.15.33:0/116045637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0002ece0 con 0x557ea72111b0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fb9e77fe640 1 -- 172.21.15.33:0/116045637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fba00030430 con 0x557ea72111b0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f8015070 con 0x557ea72111b0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fb9e77fe640 1 -- 172.21.15.33:0/116045637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fba00030630 con 0x557ea72111b0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 10 client.19671 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x557ea721f7e0 con 0x557ea72111b0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fb9e77fe640 1 -- 172.21.15.33:0/116045637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f8015070 con 0x557ea72111b0 2024-09-10T21:28:13.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fb9e77fe640 1 -- 172.21.15.33:0/116045637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7fba0000f050 con 0x557ea72111b0 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 10 client.19671 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 10 client.19671 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 20 client.19671 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea72211a0 0x557ea7223660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 4 client.19671 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.997+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ea7174df0 con 0x557ea72211a0 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.999+0000 7fba07fff640 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea72211a0 0x557ea7223660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.999+0000 7fba07fff640 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea72211a0 0x557ea7223660 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac8b913ccbacaf2a server_cookie=f862b632cfc894a9 in_seq=0 out_seq=0 2024-09-10T21:28:13.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:12.999+0000 7fb9e77fe640 10 client.19671 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.000+0000 7fb9e77fe640 1 -- 172.21.15.33:0/116045637 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ea7174df0 con 0x557ea72211a0 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.000+0000 7fb9e77fe640 10 client.19671 handle_command_reply: tid=0 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.000+0000 7fba108f6740 1 client.19671 shutdown 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 msgr2=0x557ea720e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720e2d0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fba00012bb0 tx=0x7fba00004a40 comp rx=0 tx=0).stop 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 shutdown_connections 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea72211a0 0x557ea7223660 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720e2d0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7205220 0x557ea720dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 --2- 172.21.15.33:0/116045637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7204900 0x557ea7210940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 >> 172.21.15.33:0/116045637 conn(0x557ea7202730 msgr2=0x557ea72082a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fb9e77fe640 0 client.19671 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:28:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.001+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 shutdown_connections 2024-09-10T21:28:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.002+0000 7fba108f6740 1 -- 172.21.15.33:0/116045637 wait complete. 2024-09-10T21:28:13.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.002+0000 7fba108f6740 20 client.19671 trim_cache size 0 max 16384 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 Processor -- start 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 -- start start 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7204900 0x557ea72231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 0x557ea7203640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7204900 0x557ea72231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea7203b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f40304d0 con 0x557ea7204900 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4032100 con 0x557ea72111b0 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba108f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f4030bd0 con 0x557ea7205220 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba07fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7204900 0x557ea72231b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44696/0 (socket says 172.21.15.33:44696) 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.024+0000 7fba07fff640 1 -- 172.21.15.33:0/813165402 learned_addr learned my addr 172.21.15.33:0/813165402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:13.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fba0cabe640 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea7203b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fba07fff640 1 -- 172.21.15.33:0/813165402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 msgr2=0x557ea7203640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fba07fff640 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 0x557ea7203640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fba07fff640 1 -- 172.21.15.33:0/813165402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 msgr2=0x557ea7203b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fba07fff640 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea7203b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fba07fff640 1 -- 172.21.15.33:0/813165402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea7210cb0 con 0x557ea7204900 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fba07fff640 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7204900 0x557ea72231b0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b310 tx=0x7fb9f8003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2052c9a3b550a05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fb9edffb640 1 -- 172.21.15.33:0/813165402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8005040 con 0x557ea7204900 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fb9edffb640 1 -- 172.21.15.33:0/813165402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f8015040 con 0x557ea7204900 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.025+0000 7fb9edffb640 1 -- 172.21.15.33:0/813165402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004c80 con 0x557ea7204900 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 -- 172.21.15.33:0/813165402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7204900 msgr2=0x557ea72231b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7204900 0x557ea72231b0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800b310 tx=0x7fb9f8003d60 comp rx=0 tx=0).stop 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 -- 172.21.15.33:0/813165402 shutdown_connections 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea72111b0 0x557ea7203b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 0x557ea7203640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 --2- 172.21.15.33:0/813165402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea7204900 0x557ea72231b0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 -- 172.21.15.33:0/813165402 >> 172.21.15.33:0/813165402 conn(0x557ea7202730 msgr2=0x557ea720c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 -- 172.21.15.33:0/813165402 shutdown_connections 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.026+0000 7fba108f6740 1 -- 172.21.15.33:0/813165402 wait complete. 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.027+0000 7fba108f6740 1 -- start start 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.027+0000 7fba108f6740 1 Processor -- start 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.027+0000 7fba108f6740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7204900 0x557ea72199a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.027+0000 7fba108f6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 0x557ea720ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba108f6740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba108f6740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f40319d0 con 0x557ea72111b0 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba108f6740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f40312d0 con 0x557ea7204900 2024-09-10T21:28:13.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba108f6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f402e490 con 0x557ea7205220 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7204900 0x557ea72199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba077fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 0x557ea720ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7204900 0x557ea72199a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54620/0 (socket says 172.21.15.33:54620) 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 -- 172.21.15.33:0/3687576358 learned_addr learned my addr 172.21.15.33:0/3687576358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba108f6740 10 client.0 _ll_register_callbacks cb 0x7fba000094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba0cabe640 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 -- 172.21.15.33:0/3687576358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 msgr2=0x557ea720ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 0x557ea720ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 -- 172.21.15.33:0/3687576358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 msgr2=0x557ea720b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 -- 172.21.15.33:0/3687576358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea720b950 con 0x557ea7204900 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.028+0000 7fba07fff640 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7204900 0x557ea72199a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800bd90 tx=0x7fb9f8003af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d63fc7d291b3e69a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:13.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.029+0000 7fb9e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.029+0000 7fb9e77fe640 1 -- 172.21.15.33:0/3687576358 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8036070 con 0x557ea7204900 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.029+0000 7fb9e77fe640 1 -- 172.21.15.33:0/3687576358 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb9f801a670 con 0x557ea7204900 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.029+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f8015040 con 0x557ea7204900 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.029+0000 7fb9e77fe640 1 -- 172.21.15.33:0/3687576358 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fb9f801a8b0 con 0x557ea7204900 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.029+0000 7fba108f6740 10 client.28994 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.029+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x557ea728e180 con 0x557ea7204900 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.030+0000 7fb9e77fe640 1 -- 172.21.15.33:0/3687576358 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f802e460 con 0x557ea7204900 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.030+0000 7fb9e77fe640 1 -- 172.21.15.33:0/3687576358 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7fb9f8026420 con 0x557ea7204900 2024-09-10T21:28:13.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.030+0000 7fba108f6740 10 client.28994 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.030+0000 7fba108f6740 10 client.28994 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:28:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.030+0000 7fba108f6740 20 client.28994 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.031+0000 7fba108f6740 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea728e650 0x557ea7290a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.031+0000 7fba108f6740 4 client.28994 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:28:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.031+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb9f8015040 con 0x557ea728e650 2024-09-10T21:28:13.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.032+0000 7fba077fe640 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea728e650 0x557ea7290a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.035+0000 7fb9e77fe640 10 client.28994 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:28:13.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.035+0000 7fba077fe640 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea728e650 0x557ea7290a70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bc6ad0901f9d09bb server_cookie=cc2c61143a84adb3 in_seq=0 out_seq=0 2024-09-10T21:28:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fb9e77fe640 1 -- 172.21.15.33:0/3687576358 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb9f8015040 con 0x557ea728e650 2024-09-10T21:28:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fb9e77fe640 10 client.28994 handle_command_reply: tid=0 2024-09-10T21:28:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 client.28994 shutdown 2024-09-10T21:28:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7204900 msgr2=0x557ea72199a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7204900 0x557ea72199a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800bd90 tx=0x7fb9f8003af0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 shutdown_connections 2024-09-10T21:28:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x557ea728e650 0x557ea7290a70 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x557ea72111b0 0x557ea720b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ea7205220 0x557ea720ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 --2- 172.21.15.33:0/3687576358 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x557ea7204900 0x557ea72199a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.036+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 >> 172.21.15.33:0/3687576358 conn(0x557ea7202730 msgr2=0x557ea720f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.037+0000 7fb9e77fe640 0 client.28994 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:28:13.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.037+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 shutdown_connections 2024-09-10T21:28:13.040 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:28:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.037+0000 7fba108f6740 1 -- 172.21.15.33:0/3687576358 wait complete. 2024-09-10T21:28:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.037+0000 7fba108f6740 20 client.28994 trim_cache size 0 max 16384 2024-09-10T21:28:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.038+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9f00b53e0 msgr2=0x7fb9f00b78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.038+0000 7fba0e521640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9f00b53e0 0x7fb9f00b78a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fba00005c40 tx=0x7fba00005bd0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.038+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba081045e0 msgr2=0x7fba08101eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.038+0000 7fba0e521640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba081045e0 0x7fba08101eb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc006010 tx=0x7fb9fc004060 comp rx=0 tx=0).stop 2024-09-10T21:28:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.039+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 shutdown_connections 2024-09-10T21:28:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.039+0000 7fba0e521640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb9f00b53e0 0x7fb9f00b78a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.039+0000 7fba0e521640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba0810d120 0x7fba081023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.039+0000 7fba0e521640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fba081045e0 0x7fba08101eb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.039+0000 7fba0e521640 1 --2- 172.21.15.33:0/3399252601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fba08103cc0 0x7fba08101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.039+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 >> 172.21.15.33:0/3399252601 conn(0x7fba080fd740 msgr2=0x7fba080fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.040+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 shutdown_connections 2024-09-10T21:28:13.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:13.040+0000 7fba0e521640 1 -- 172.21.15.33:0/3399252601 wait complete. 2024-09-10T21:28:13.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 485785 37.47 MB/sec execute 553 sec latency 86.570 ms 2024-09-10T21:28:13.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:13 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:13.561 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:13 smithi104 ceph-mon[35325]: pgmap v1073: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 678 op/s 2024-09-10T21:28:13.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:13 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:13.575 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:13 smithi033 ceph-mon[30860]: pgmap v1073: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 678 op/s 2024-09-10T21:28:13.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:13 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:13.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:13 smithi174 ceph-mon[35635]: pgmap v1073: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 11 MiB/s wr, 678 op/s 2024-09-10T21:28:13.762 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:28:14.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 486426 37.45 MB/sec execute 554 sec latency 71.602 ms 2024-09-10T21:28:14.171 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.750+0000 7f9bc0227640 1 -- 172.21.15.33:0/1158807922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8109150 msgr2=0x7f9bb810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.750+0000 7f9bc0227640 1 --2- 172.21.15.33:0/1158807922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8109150 0x7f9bb810f950 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009ca0 tx=0x7f9bb002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.750+0000 7f9bc0227640 1 -- 172.21.15.33:0/1158807922 shutdown_connections 2024-09-10T21:28:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.750+0000 7f9bc0227640 1 --2- 172.21.15.33:0/1158807922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8109150 0x7f9bb810f950 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.750+0000 7f9bc0227640 1 --2- 172.21.15.33:0/1158807922 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb81087d0 0x7f9bb8108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.750+0000 7f9bc0227640 1 --2- 172.21.15.33:0/1158807922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb8102dc0 0x7f9bb81031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.750+0000 7f9bc0227640 1 -- 172.21.15.33:0/1158807922 >> 172.21.15.33:0/1158807922 conn(0x7f9bb80fe830 msgr2=0x7f9bb8100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.752+0000 7f9bc0227640 1 -- 172.21.15.33:0/1158807922 shutdown_connections 2024-09-10T21:28:14.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.752+0000 7f9bc0227640 1 -- 172.21.15.33:0/1158807922 wait complete. 2024-09-10T21:28:14.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.753+0000 7f9bc0227640 1 Processor -- start 2024-09-10T21:28:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.754+0000 7f9bc0227640 1 -- start start 2024-09-10T21:28:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bc0227640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8102dc0 0x7f9bb810e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bc0227640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb81087d0 0x7f9bb810ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bc0227640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8109150 0x7f9bb81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bc0227640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8112e20 con 0x7f9bb8102dc0 2024-09-10T21:28:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bc0227640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8112ca0 con 0x7f9bb8109150 2024-09-10T21:28:14.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bc0227640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb8112fa0 con 0x7f9bb81087d0 2024-09-10T21:28:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bbd7c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb81087d0 0x7f9bb810ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bbd7c2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb81087d0 0x7f9bb810ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46248/0 (socket says 172.21.15.33:46248) 2024-09-10T21:28:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bbd7c2640 1 -- 172.21.15.33:0/2385855237 learned_addr learned my addr 172.21.15.33:0/2385855237 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.755+0000 7f9bbe7c4640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8109150 0x7f9bb81995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.756+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8102dc0 0x7f9bb810e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.756+0000 7f9bbd7c2640 1 -- 172.21.15.33:0/2385855237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8109150 msgr2=0x7f9bb81995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.756+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8109150 0x7f9bb81995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.756+0000 7f9bbd7c2640 1 -- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8102dc0 msgr2=0x7f9bb810e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.756+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8102dc0 0x7f9bb810e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.756+0000 7f9bbd7c2640 1 -- 172.21.15.33:0/2385855237 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bb8199da0 con 0x7f9bb81087d0 2024-09-10T21:28:14.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.757+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb81087d0 0x7f9bb810ee10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800e9f0 tx=0x7f9ba800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:14.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.757+0000 7f9baeffd640 1 -- 172.21.15.33:0/2385855237 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba800cd80 con 0x7f9bb81087d0 2024-09-10T21:28:14.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.757+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bb819a090 con 0x7f9bb81087d0 2024-09-10T21:28:14.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.758+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bb819fb40 con 0x7f9bb81087d0 2024-09-10T21:28:14.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.758+0000 7f9baeffd640 1 -- 172.21.15.33:0/2385855237 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ba80108e0 con 0x7f9bb81087d0 2024-09-10T21:28:14.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.758+0000 7f9baeffd640 1 -- 172.21.15.33:0/2385855237 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba8026060 con 0x7f9bb81087d0 2024-09-10T21:28:14.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.763+0000 7f9baeffd640 1 -- 172.21.15.33:0/2385855237 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9ba8010430 con 0x7f9bb81087d0 2024-09-10T21:28:14.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.764+0000 7f9baeffd640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ba40b52e0 0x7f9ba40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.765+0000 7f9baeffd640 1 -- 172.21.15.33:0/2385855237 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9ba80e5d70 con 0x7f9bb81087d0 2024-09-10T21:28:14.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.765+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ba40b52e0 0x7f9ba40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.766+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ba40b52e0 0x7f9ba40b77a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005d00 tx=0x7f9bb400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:14.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.766+0000 7f9bc25fc740 1 Processor -- start 2024-09-10T21:28:14.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.767+0000 7f9bc25fc740 1 -- start start 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e05238b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e0532170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607e0499ea0 con 0x5607e052fd80 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5607e04892d0 con 0x5607e0523df0 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607e048d0e0 con 0x5607e05234d0 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bbdfc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e05238b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bbdfc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e05238b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46252/0 (socket says 172.21.15.33:46252) 2024-09-10T21:28:14.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2654987602 learned_addr learned my addr 172.21.15.33:0/2654987602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.768+0000 7f9bbe7c4640 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e0532170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2654987602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 msgr2=0x5607e052f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2654987602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 msgr2=0x5607e0532170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e0532170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2654987602 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e01e7650 con 0x5607e05234d0 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e05238b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4011010 tx=0x7f9bb4013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3da714c251ec4a43 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.769+0000 7f9ba3fff640 1 -- 172.21.15.33:0/2654987602 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb401e620 con 0x5607e05234d0 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.770+0000 7f9ba3fff640 1 -- 172.21.15.33:0/2654987602 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb401ec70 con 0x5607e05234d0 2024-09-10T21:28:14.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.770+0000 7f9ba3fff640 1 -- 172.21.15.33:0/2654987602 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4026c40 con 0x5607e05234d0 2024-09-10T21:28:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2654987602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 msgr2=0x5607e05238b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e05238b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4011010 tx=0x7f9bb4013000 comp rx=0 tx=0).stop 2024-09-10T21:28:14.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2654987602 shutdown_connections 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e0532170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2654987602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e05238b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2654987602 >> 172.21.15.33:0/2654987602 conn(0x5607e0521300 msgr2=0x5607e05224a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2654987602 shutdown_connections 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.771+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2654987602 wait complete. 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.772+0000 7f9bc25fc740 1 -- start start 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.772+0000 7f9bc25fc740 1 Processor -- start 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.772+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e052f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e052cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5607e02ea700 con 0x5607e052fd80 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5607e0424160 con 0x5607e0523df0 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5607e0532a00 con 0x5607e05234d0 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bc25fc740 10 client.0 _ll_register_callbacks cb 0x5607e043d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bbdfc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e052f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bbd7c2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bbe7c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e052cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bbe7c4640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e052cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44738/0 (socket says 172.21.15.33:44738) 2024-09-10T21:28:14.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.773+0000 7f9bbe7c4640 1 -- 172.21.15.33:0/3982624678 learned_addr learned my addr 172.21.15.33:0/3982624678 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.774+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/3982624678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 msgr2=0x5607e052ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.774+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.774+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/3982624678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 msgr2=0x5607e052cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.774+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e052cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.774+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/3982624678 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e0451c90 con 0x5607e05234d0 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.774+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e052f5e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4025000 tx=0x7f9bb4026630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4b8d7beb3dba92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.774+0000 7f9ba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.775+0000 7f9ba17fa640 1 -- 172.21.15.33:0/3982624678 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4036060 con 0x5607e05234d0 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.775+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb401ec70 con 0x5607e05234d0 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.775+0000 7f9ba17fa640 1 -- 172.21.15.33:0/3982624678 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9bb401e420 con 0x5607e05234d0 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.775+0000 7f9ba17fa640 1 -- 172.21.15.33:0/3982624678 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f9bb401e660 con 0x5607e05234d0 2024-09-10T21:28:14.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.775+0000 7f9bc25fc740 10 client.28858 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.775+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x5607e053e340 con 0x5607e05234d0 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9ba17fa640 1 -- 172.21.15.33:0/3982624678 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb401ec70 con 0x5607e05234d0 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9ba17fa640 1 -- 172.21.15.33:0/3982624678 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7f9bb403acb0 con 0x5607e05234d0 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9bc25fc740 10 client.28858 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9bc25fc740 10 client.28858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9bc25fc740 20 client.28858 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e053f6f0 0x5607e0541ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9bc25fc740 4 client.28858 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:14.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.776+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e049bdf0 con 0x5607e053f6f0 2024-09-10T21:28:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.777+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e053f6f0 0x5607e0541ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.778+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e053f6f0 0x5607e0541ae0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9cdfde3fe7ad1de server_cookie=638dbcd2ce99fab5 in_seq=0 out_seq=0 2024-09-10T21:28:14.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.778+0000 7f9ba17fa640 10 client.28858 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.781+0000 7f9ba17fa640 1 -- 172.21.15.33:0/3982624678 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5607e049bdf0 con 0x5607e053f6f0 2024-09-10T21:28:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.781+0000 7f9ba17fa640 10 client.28858 handle_command_reply: tid=0 2024-09-10T21:28:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 client.28858 shutdown 2024-09-10T21:28:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 msgr2=0x5607e052f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e052f5e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4025000 tx=0x7f9bb4026630 comp rx=0 tx=0).stop 2024-09-10T21:28:14.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 shutdown_connections 2024-09-10T21:28:14.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e053f6f0 0x5607e0541ae0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e052cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e052ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/3982624678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e05234d0 0x5607e052f5e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 >> 172.21.15.33:0/3982624678 conn(0x5607e0521300 msgr2=0x5607e0526e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:14.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9ba17fa640 0 client.28858 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:14.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 shutdown_connections 2024-09-10T21:28:14.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.782+0000 7f9bc25fc740 1 -- 172.21.15.33:0/3982624678 wait complete. 2024-09-10T21:28:14.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.783+0000 7f9bc25fc740 20 client.28858 trim_cache size 0 max 16384 2024-09-10T21:28:14.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 Processor -- start 2024-09-10T21:28:14.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 -- start start 2024-09-10T21:28:14.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e05234d0 0x5607e0522ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e0520c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bbdfc3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e05234d0 0x5607e0522ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e052fd80 0x5607e0529c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bbdfc3640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e05234d0 0x5607e0522ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44742/0 (socket says 172.21.15.33:44742) 2024-09-10T21:28:14.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2103802063 learned_addr learned my addr 172.21.15.33:0/2103802063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:14.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c00e490 con 0x5607e05234d0 2024-09-10T21:28:14.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c0100c0 con 0x5607e0523df0 2024-09-10T21:28:14.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.803+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c00eb90 con 0x5607e052fd80 2024-09-10T21:28:14.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e0520c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbe7c4640 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e052fd80 0x5607e0529c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2103802063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e052fd80 msgr2=0x5607e0529c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e052fd80 0x5607e0529c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2103802063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 msgr2=0x5607e0520c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e0520c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2103802063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e0521fe0 con 0x5607e05234d0 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.804+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e0520c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.805+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e05234d0 0x5607e0522ad0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4021040 tx=0x7f9bb400b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedf3a97c7867f19 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.805+0000 7f9ba37fe640 1 -- 172.21.15.33:0/2103802063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb403f020 con 0x5607e05234d0 2024-09-10T21:28:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.805+0000 7f9ba37fe640 1 -- 172.21.15.33:0/2103802063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb400bc40 con 0x5607e05234d0 2024-09-10T21:28:14.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.805+0000 7f9ba37fe640 1 -- 172.21.15.33:0/2103802063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb402a070 con 0x5607e05234d0 2024-09-10T21:28:14.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.807+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e05234d0 msgr2=0x5607e0522ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.807+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e05234d0 0x5607e0522ad0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4021040 tx=0x7f9bb400b9b0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.807+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 shutdown_connections 2024-09-10T21:28:14.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.807+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e052fd80 0x5607e0529c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.807+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e0523df0 0x5607e0520c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.807+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2103802063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e05234d0 0x5607e0522ad0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.807+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 >> 172.21.15.33:0/2103802063 conn(0x5607e0521300 msgr2=0x5607e05223c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.808+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 shutdown_connections 2024-09-10T21:28:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.808+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2103802063 wait complete. 2024-09-10T21:28:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.809+0000 7f9bc25fc740 1 -- start start 2024-09-10T21:28:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.809+0000 7f9bc25fc740 1 Processor -- start 2024-09-10T21:28:14.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e05234d0 0x5607e05385b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e0523df0 0x5607e0533a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bc25fc740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e05340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c00f990 con 0x5607e052fd80 2024-09-10T21:28:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c00f290 con 0x5607e05234d0 2024-09-10T21:28:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bc25fc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b9c00c450 con 0x5607e0523df0 2024-09-10T21:28:14.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bc25fc740 10 client.0 _ll_register_callbacks cb 0x7f9bb400bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbdfc3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e05234d0 0x5607e05385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbdfc3640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e05234d0 0x5607e05385b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:54674/0 (socket says 172.21.15.33:54674) 2024-09-10T21:28:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2296124059 learned_addr learned my addr 172.21.15.33:0/2296124059 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:14.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e0523df0 0x5607e0533a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbe7c4640 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e05340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2296124059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e0523df0 msgr2=0x5607e0533a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e0523df0 0x5607e0533a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2296124059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 msgr2=0x5607e05340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.810+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e05340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9bbdfc3640 1 -- 172.21.15.33:0/2296124059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e05ab370 con 0x5607e05234d0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9bbdfc3640 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e05234d0 0x5607e05385b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9bb401a590 tx=0x7f9bb4011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c44e89986f20296f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9ba17fa640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9ba17fa640 1 -- 172.21.15.33:0/2296124059 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4036020 con 0x5607e05234d0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9ba17fa640 1 -- 172.21.15.33:0/2296124059 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9bb402d050 con 0x5607e05234d0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bb400bc40 con 0x5607e05234d0 2024-09-10T21:28:14.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9ba17fa640 1 -- 172.21.15.33:0/2296124059 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f9bb402a040 con 0x5607e05234d0 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.811+0000 7f9bc25fc740 10 client.29009 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x5607e05ac4c0 con 0x5607e05234d0 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9ba17fa640 1 -- 172.21.15.33:0/2296124059 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9bb401aac0 con 0x5607e05234d0 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9ba17fa640 1 -- 172.21.15.33:0/2296124059 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7f9bb404d020 con 0x5607e05234d0 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9bc25fc740 10 client.29009 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9bc25fc740 10 client.29009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9bc25fc740 20 client.29009 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e05acc30 0x5607e05af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9bc25fc740 4 client.29009 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.812+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f9bb400bc40 con 0x5607e05acc30 2024-09-10T21:28:14.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.813+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e05acc30 0x5607e05af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:14.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.813+0000 7f9ba17fa640 10 client.29009 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:14.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.813+0000 7f9bbd7c2640 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e05acc30 0x5607e05af050 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e4c87f62bb43a8d server_cookie=580554042b8abcb in_seq=0 out_seq=0 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9ba17fa640 1 -- 172.21.15.33:0/2296124059 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f9bb400bc40 con 0x5607e05acc30 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9ba17fa640 10 client.29009 handle_command_reply: tid=0 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 client.29009 shutdown 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e05234d0 msgr2=0x5607e05385b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e05234d0 0x5607e05385b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9bb401a590 tx=0x7f9bb4011ca0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 shutdown_connections 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x5607e05acc30 0x5607e05af050 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5607e052fd80 0x5607e05340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9ba17fa640 0 client.29009 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5607e0523df0 0x5607e0533a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 --2- 172.21.15.33:0/2296124059 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5607e05234d0 0x5607e05385b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.817+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 >> 172.21.15.33:0/2296124059 conn(0x5607e0521300 msgr2=0x5607e052e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.818+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 shutdown_connections 2024-09-10T21:28:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.818+0000 7f9bc25fc740 1 -- 172.21.15.33:0/2296124059 wait complete. 2024-09-10T21:28:14.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.818+0000 7f9bc25fc740 20 client.29009 trim_cache size 0 max 16384 2024-09-10T21:28:14.821 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:28:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ba40b52e0 msgr2=0x7f9ba40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ba40b52e0 0x7f9ba40b77a0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4005d00 tx=0x7f9bb400b410 comp rx=0 tx=0).stop 2024-09-10T21:28:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb81087d0 msgr2=0x7f9bb810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb81087d0 0x7f9bb810ee10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ba800e9f0 tx=0x7f9ba800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 shutdown_connections 2024-09-10T21:28:14.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9ba40b52e0 0x7f9ba40b77a0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bb8109150 0x7f9bb81995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bb81087d0 0x7f9bb810ee10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.820+0000 7f9bc0227640 1 --2- 172.21.15.33:0/2385855237 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9bb8102dc0 0x7f9bb810e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.821+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 >> 172.21.15.33:0/2385855237 conn(0x7f9bb80fe830 msgr2=0x7f9bb8075410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.821+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 shutdown_connections 2024-09-10T21:28:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:14.821+0000 7f9bc0227640 1 -- 172.21.15.33:0/2385855237 wait complete. 2024-09-10T21:28:15.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 487025 37.43 MB/sec execute 555 sec latency 99.648 ms 2024-09-10T21:28:15.811 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:15 smithi104 ceph-mon[35325]: pgmap v1074: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 811 op/s 2024-09-10T21:28:15.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:15 smithi174 ceph-mon[35635]: pgmap v1074: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 811 op/s 2024-09-10T21:28:15.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:15 smithi033 ceph-mon[30860]: pgmap v1074: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 12 MiB/s wr, 811 op/s 2024-09-10T21:28:16.087 INFO:tasks.workunit.client.0.smithi033.stdout: 10 487954 37.45 MB/sec execute 556 sec latency 79.445 ms 2024-09-10T21:28:16.558 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-10T21:28:16.559 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-10T21:28:17.037 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:17.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 488679 37.46 MB/sec execute 557 sec latency 292.177 ms 2024-09-10T21:28:17.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:17 smithi104 ceph-mon[35325]: pgmap v1075: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.8 MiB/s wr, 648 op/s 2024-09-10T21:28:17.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:17 smithi174 ceph-mon[35635]: pgmap v1075: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.8 MiB/s wr, 648 op/s 2024-09-10T21:28:17.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:17 smithi033 ceph-mon[30860]: pgmap v1075: 97 pgs: 97 active+clean; 1.2 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 7.8 MiB/s wr, 648 op/s 2024-09-10T21:28:18.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 488806 37.40 MB/sec execute 558 sec latency 752.044 ms 2024-09-10T21:28:19.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 489257 37.39 MB/sec execute 559 sec latency 106.316 ms 2024-09-10T21:28:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.158+0000 7fb654ede640 1 -- 172.21.15.33:0/1373280937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb650103cc0 msgr2=0x7fb6501040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.158+0000 7fb654ede640 1 --2- 172.21.15.33:0/1373280937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb650103cc0 0x7fb6501040a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb640009900 tx=0x7fb64002a400 comp rx=0 tx=0).stop 2024-09-10T21:28:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.159+0000 7fb654ede640 1 -- 172.21.15.33:0/1373280937 shutdown_connections 2024-09-10T21:28:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.159+0000 7fb654ede640 1 --2- 172.21.15.33:0/1373280937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb65010d120 0x7fb65010f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.159+0000 7fb654ede640 1 --2- 172.21.15.33:0/1373280937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6501045e0 0x7fb65010cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.159+0000 7fb654ede640 1 --2- 172.21.15.33:0/1373280937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb650103cc0 0x7fb6501040a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.159+0000 7fb654ede640 1 -- 172.21.15.33:0/1373280937 >> 172.21.15.33:0/1373280937 conn(0x7fb6500fd700 msgr2=0x7fb6500ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.159+0000 7fb654ede640 1 -- 172.21.15.33:0/1373280937 shutdown_connections 2024-09-10T21:28:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.159+0000 7fb654ede640 1 -- 172.21.15.33:0/1373280937 wait complete. 2024-09-10T21:28:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.160+0000 7fb654ede640 1 Processor -- start 2024-09-10T21:28:19.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.160+0000 7fb654ede640 1 -- start start 2024-09-10T21:28:19.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb654ede640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb650103cc0 0x7fb6501990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb654ede640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6501045e0 0x7fb6501995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb654ede640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb65010d120 0x7fb65019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb654ede640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb650112de0 con 0x7fb6501045e0 2024-09-10T21:28:19.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb64e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb650103cc0 0x7fb6501990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb64e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb650103cc0 0x7fb6501990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46306/0 (socket says 172.21.15.33:46306) 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb64e59c640 1 -- 172.21.15.33:0/3338313976 learned_addr learned my addr 172.21.15.33:0/3338313976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb650112c60 con 0x7fb65010d120 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb650112f60 con 0x7fb650103cc0 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6501045e0 0x7fb6501995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.161+0000 7fb64ed9d640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb65010d120 0x7fb65019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb64e59c640 1 -- 172.21.15.33:0/3338313976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb65010d120 msgr2=0x7fb65019e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb64e59c640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb65010d120 0x7fb65019e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb64e59c640 1 -- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6501045e0 msgr2=0x7fb6501995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb64e59c640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6501045e0 0x7fb6501995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb64e59c640 1 -- 172.21.15.33:0/3338313976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb65019f160 con 0x7fb650103cc0 2024-09-10T21:28:19.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb64e59c640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb650103cc0 0x7fb6501990b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb640009850 tx=0x7fb640030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb63f7fe640 1 -- 172.21.15.33:0/3338313976 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64003f070 con 0x7fb650103cc0 2024-09-10T21:28:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb65019f3f0 con 0x7fb650103cc0 2024-09-10T21:28:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb63f7fe640 1 -- 172.21.15.33:0/3338313976 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb640004460 con 0x7fb650103cc0 2024-09-10T21:28:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb63f7fe640 1 -- 172.21.15.33:0/3338313976 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640035030 con 0x7fb650103cc0 2024-09-10T21:28:19.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.162+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6501a5b10 con 0x7fb650103cc0 2024-09-10T21:28:19.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 Processor -- start 2024-09-10T21:28:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 -- start start 2024-09-10T21:28:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbba07e0 0x55c3fbbb0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbb2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3fbb16e60 con 0x55c3fbb9fec0 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c3fbb06290 con 0x55c3fbba07e0 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.166+0000 7fb6572b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3fbb0a0a0 con 0x55c3fbbb0b80 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbb2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbb2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46318/0 (socket says 172.21.15.33:46318) 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb63f7fe640 1 -- 172.21.15.33:0/3338313976 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb640030840 con 0x7fb650103cc0 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 -- 172.21.15.33:0/4231073157 learned_addr learned my addr 172.21.15.33:0/4231073157 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64ed9d640 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 -- 172.21.15.33:0/4231073157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbba07e0 msgr2=0x55c3fbbb0640 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:28:19.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbba07e0 0x55c3fbbb0640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 -- 172.21.15.33:0/4231073157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 msgr2=0x55c3fbba02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.167+0000 7fb64dd9b640 1 -- 172.21.15.33:0/4231073157 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3fb864930 con 0x55c3fbbb0b80 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.168+0000 7fb64ed9d640 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.168+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbb2f70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb64400bc90 tx=0x7fb6440107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4fbb0b27c30281d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.168+0000 7fb63f7fe640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb6380b5360 0x7fb6380b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.168+0000 7fb63f7fe640 1 -- 172.21.15.33:0/3338313976 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb640106760 con 0x7fb650103cc0 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.169+0000 7fb63cff9640 1 -- 172.21.15.33:0/4231073157 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64400be40 con 0x55c3fbbb0b80 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.169+0000 7fb63cff9640 1 -- 172.21.15.33:0/4231073157 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb644005020 con 0x55c3fbbb0b80 2024-09-10T21:28:19.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.169+0000 7fb64ed9d640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb6380b5360 0x7fb6380b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.169+0000 7fb63cff9640 1 -- 172.21.15.33:0/4231073157 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644004ad0 con 0x55c3fbbb0b80 2024-09-10T21:28:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 -- 172.21.15.33:0/4231073157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 msgr2=0x55c3fbbb2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb64ed9d640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb6380b5360 0x7fb6380b7820 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb648009fd0 tx=0x7fb648009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbb2f70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb64400bc90 tx=0x7fb6440107a0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 -- 172.21.15.33:0/4231073157 shutdown_connections 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbb2f70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbba07e0 0x55c3fbbb0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 --2- 172.21.15.33:0/4231073157 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 -- 172.21.15.33:0/4231073157 >> 172.21.15.33:0/4231073157 conn(0x55c3fbb9dcf0 msgr2=0x55c3fbb9ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 -- 172.21.15.33:0/4231073157 shutdown_connections 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.170+0000 7fb6572b3740 1 -- 172.21.15.33:0/4231073157 wait complete. 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.171+0000 7fb6572b3740 1 -- start start 2024-09-10T21:28:19.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.171+0000 7fb6572b3740 1 Processor -- start 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbaa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb6572b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c3fb967890 con 0x55c3fbba07e0 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb6572b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c3fbaa0d40 con 0x55c3fbb9fec0 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb6572b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3fbba6430 con 0x55c3fbbb0b80 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbaa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbaa1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44792/0 (socket says 172.21.15.33:44792) 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64dd9b640 1 -- 172.21.15.33:0/608244088 learned_addr learned my addr 172.21.15.33:0/608244088 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64e59c640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64ed9d640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb6572b3740 10 client.0 _ll_register_callbacks cb 0x55c3fbb04c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64e59c640 1 -- 172.21.15.33:0/608244088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 msgr2=0x55c3fbbaa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64e59c640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64e59c640 1 -- 172.21.15.33:0/608244088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 msgr2=0x55c3fbbaa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64e59c640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbaa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64e59c640 1 -- 172.21.15.33:0/608244088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3fbbaf730 con 0x55c3fbb9fec0 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.172+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbaa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb64e59c640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55c3fbb9f9a0 tx=0x7fb6400c8ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62fd9a2a6333bbec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:19.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb635ffb640 10 client.29018 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb644005020 con 0x55c3fbb9fec0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb635ffb640 1 -- 172.21.15.33:0/608244088 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb640049070 con 0x55c3fbb9fec0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb635ffb640 1 -- 172.21.15.33:0/608244088 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb640030b60 con 0x55c3fbb9fec0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb635ffb640 1 -- 172.21.15.33:0/608244088 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fb6400ccaa0 con 0x55c3fbb9fec0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb6572b3740 10 client.29018 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.173+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({fsmap=95}) -- 0x55c3fbbbb0c0 con 0x55c3fbb9fec0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb635ffb640 1 -- 172.21.15.33:0/608244088 <== mon.2 v2:172.21.15.104:3300/0 4 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7fb6400c9200 con 0x55c3fbb9fec0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb6572b3740 10 client.29018 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb6572b3740 10 client.29018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb6572b3740 20 client.29018 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb6572b3740 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbbbe950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb6572b3740 4 client.29018 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb644005020 con 0x55c3fbbbc4d0 2024-09-10T21:28:19.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbbbe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbbbe950 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af921191ff398b7c server_cookie=48533e82010de85d in_seq=0 out_seq=0 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.174+0000 7fb635ffb640 10 client.29018 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb635ffb640 1 -- 172.21.15.33:0/608244088 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fb644005020 con 0x55c3fbbbc4d0 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb635ffb640 10 client.29018 handle_command_reply: tid=0 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb6572b3740 1 client.29018 shutdown 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbb9fec0 msgr2=0x55c3fbba9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb6572b3740 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9cb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x55c3fbb9f9a0 tx=0x7fb6400c8ef0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 shutdown_connections 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb6572b3740 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbbbe950 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.175+0000 7fb635ffb640 0 client.29018 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.176+0000 7fb6572b3740 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.176+0000 7fb6572b3740 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbaa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.176+0000 7fb6572b3740 1 --2- 172.21.15.33:0/608244088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9cb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.176+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 >> 172.21.15.33:0/608244088 conn(0x55c3fbb9dcf0 msgr2=0x55c3fbb9cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.176+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 shutdown_connections 2024-09-10T21:28:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.176+0000 7fb6572b3740 1 -- 172.21.15.33:0/608244088 wait complete. 2024-09-10T21:28:19.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.176+0000 7fb6572b3740 20 client.29018 trim_cache size 0 max 16384 2024-09-10T21:28:19.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 Processor -- start 2024-09-10T21:28:19.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 -- start start 2024-09-10T21:28:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbba07e0 0x55c3fbba76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501ac1c0 con 0x55c3fbb9fec0 2024-09-10T21:28:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501addf0 con 0x55c3fbbb0b80 2024-09-10T21:28:19.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb6572b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501ac8c0 con 0x55c3fbba07e0 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb64dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbba07e0 0x55c3fbba76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb64dd9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbba07e0 0x55c3fbba76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46338/0 (socket says 172.21.15.33:46338) 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.200+0000 7fb64dd9b640 1 -- 172.21.15.33:0/662364559 learned_addr learned my addr 172.21.15.33:0/662364559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64ed9d640 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64e59c640 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64e59c640 1 -- 172.21.15.33:0/662364559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbba07e0 msgr2=0x55c3fbba76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64e59c640 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbba07e0 0x55c3fbba76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64e59c640 1 -- 172.21.15.33:0/662364559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 msgr2=0x55c3fbbaca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64e59c640 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64e59c640 1 -- 172.21.15.33:0/662364559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3fbbb1730 con 0x55c3fbb9fec0 2024-09-10T21:28:19.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb64e59c640 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9560 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb64002a9c0 tx=0x7fb64003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c221e9097826e05 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.201+0000 7fb637fff640 1 -- 172.21.15.33:0/662364559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400d6470 con 0x55c3fbb9fec0 2024-09-10T21:28:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb637fff640 1 -- 172.21.15.33:0/662364559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6400d19c0 con 0x55c3fbb9fec0 2024-09-10T21:28:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb637fff640 1 -- 172.21.15.33:0/662364559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6400d5fe0 con 0x55c3fbb9fec0 2024-09-10T21:28:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb6572b3740 1 -- 172.21.15.33:0/662364559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 msgr2=0x55c3fbba9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb6572b3740 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9560 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb64002a9c0 tx=0x7fb64003c000 comp rx=0 tx=0).stop 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb6572b3740 1 -- 172.21.15.33:0/662364559 shutdown_connections 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb6572b3740 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbaca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb6572b3740 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbba07e0 0x55c3fbba76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb6572b3740 1 --2- 172.21.15.33:0/662364559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbb9fec0 0x55c3fbba9560 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.202+0000 7fb6572b3740 1 -- 172.21.15.33:0/662364559 >> 172.21.15.33:0/662364559 conn(0x55c3fbb9dcf0 msgr2=0x55c3fbbac310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 -- 172.21.15.33:0/662364559 shutdown_connections 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 -- 172.21.15.33:0/662364559 wait complete. 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 -- start start 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 Processor -- start 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbb9fec0 0x55c3fbbc4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbc3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbc3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501ad6c0 con 0x55c3fbba07e0 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501acfc0 con 0x55c3fbbb0b80 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.203+0000 7fb6572b3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6501aa180 con 0x55c3fbb9fec0 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb6572b3740 10 client.0 _ll_register_callbacks cb 0x7fb640005780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbb9fec0 0x55c3fbbc4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64dd9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbc3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64e59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbb9fec0 0x55c3fbbc4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:46352/0 (socket says 172.21.15.33:46352) 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64e59c640 1 -- 172.21.15.33:0/3226713275 learned_addr learned my addr 172.21.15.33:0/3226713275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64dd9b640 1 -- 172.21.15.33:0/3226713275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbb9fec0 msgr2=0x55c3fbbc4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbb9fec0 0x55c3fbbc4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64dd9b640 1 -- 172.21.15.33:0/3226713275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 msgr2=0x55c3fbbc3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbc3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64dd9b640 1 -- 172.21.15.33:0/3226713275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3fbba6020 con 0x55c3fbba07e0 2024-09-10T21:28:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64e59c640 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbb9fec0 0x55c3fbbc4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb64dd9b640 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbc3210 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb64400bb80 tx=0x7fb64400b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850cd3f4862db632 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.204+0000 7fb635ffb640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb635ffb640 1 -- 172.21.15.33:0/3226713275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644034060 con 0x55c3fbba07e0 2024-09-10T21:28:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb635ffb640 1 -- 172.21.15.33:0/3226713275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb644029f90 con 0x55c3fbba07e0 2024-09-10T21:28:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb6400d19c0 con 0x55c3fbba07e0 2024-09-10T21:28:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb635ffb640 1 -- 172.21.15.33:0/3226713275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7fb64400b820 con 0x55c3fbba07e0 2024-09-10T21:28:19.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 10 client.19710 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x55c3fbc2a1c0 con 0x55c3fbba07e0 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb635ffb640 1 -- 172.21.15.33:0/3226713275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6440044f0 con 0x55c3fbba07e0 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb635ffb640 1 -- 172.21.15.33:0/3226713275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7fb64401f420 con 0x55c3fbba07e0 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 10 client.19710 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 10 client.19710 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 20 client.19710 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbc2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 4 client.19710 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.205+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb6400d19c0 con 0x55c3fbbbc4d0 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.206+0000 7fb64e59c640 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbc2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.206+0000 7fb64e59c640 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbc2c650 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=284f769ad0aa04ce server_cookie=29c26bcb8f5cf975 in_seq=0 out_seq=0 2024-09-10T21:28:19.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.206+0000 7fb635ffb640 10 client.19710 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.209+0000 7fb635ffb640 1 -- 172.21.15.33:0/3226713275 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb6400d19c0 con 0x55c3fbbbc4d0 2024-09-10T21:28:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.209+0000 7fb635ffb640 10 client.19710 handle_command_reply: tid=0 2024-09-10T21:28:19.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.209+0000 7fb6572b3740 1 client.19710 shutdown 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.209+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 msgr2=0x55c3fbbc3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.209+0000 7fb6572b3740 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbc3210 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb64400bb80 tx=0x7fb64400b4c0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 shutdown_connections 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55c3fbbbc4d0 0x55c3fbc2c650 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c3fbbb0b80 0x55c3fbbc3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c3fbba07e0 0x55c3fbbc3210 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 --2- 172.21.15.33:0/3226713275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c3fbb9fec0 0x55c3fbbc4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 >> 172.21.15.33:0/3226713275 conn(0x55c3fbb9dcf0 msgr2=0x55c3fbba5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb635ffb640 0 client.19710 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 shutdown_connections 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 1 -- 172.21.15.33:0/3226713275 wait complete. 2024-09-10T21:28:19.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.210+0000 7fb6572b3740 20 client.19710 trim_cache size 0 max 16384 2024-09-10T21:28:19.214 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:28:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: "return_code": 0, 2024-09-10T21:28:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_tag": "abb3152f-b4a8-4eb5-baa6-ac4d8cbb70b0", 2024-09-10T21:28:19.214 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": "asynchronous" 2024-09-10T21:28:19.214 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb6380b5360 msgr2=0x7fb6380b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb6380b5360 0x7fb6380b7820 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb648009fd0 tx=0x7fb648009500 comp rx=0 tx=0).stop 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb650103cc0 msgr2=0x7fb6501990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb650103cc0 0x7fb6501990b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb640009850 tx=0x7fb640030650 comp rx=0 tx=0).stop 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 shutdown_connections 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb6380b5360 0x7fb6380b7820 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb65010d120 0x7fb65019e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6501045e0 0x7fb6501995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 --2- 172.21.15.33:0/3338313976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb650103cc0 0x7fb6501990b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:19.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 >> 172.21.15.33:0/3338313976 conn(0x7fb6500fd700 msgr2=0x7fb650106bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:19.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 shutdown_connections 2024-09-10T21:28:19.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:19.211+0000 7fb654ede640 1 -- 172.21.15.33:0/3338313976 wait complete. 2024-09-10T21:28:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:19 smithi174 ceph-mon[35635]: pgmap v1076: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.1 MiB/s wr, 653 op/s 2024-09-10T21:28:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:19 smithi174 ceph-mon[35635]: scrub queued for path: / 2024-09-10T21:28:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:19 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:19 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:28:19.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:19 smithi033 ceph-mon[30860]: pgmap v1076: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.1 MiB/s wr, 653 op/s 2024-09-10T21:28:19.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:19 smithi033 ceph-mon[30860]: scrub queued for path: / 2024-09-10T21:28:19.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:19 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:19.898 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:19 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:28:19.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:28:20.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:19 smithi104 ceph-mon[35325]: pgmap v1076: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.1 MiB/s wr, 653 op/s 2024-09-10T21:28:20.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:19 smithi104 ceph-mon[35325]: scrub queued for path: / 2024-09-10T21:28:20.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:19 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:20.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:19 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:28:20.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 489568 37.35 MB/sec execute 560 sec latency 392.925 ms 2024-09-10T21:28:20.304 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:20.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:28:20.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:28:20.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:28:20.712 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:20 smithi033 ceph-mon[30860]: scrub summary: active paths [/] 2024-09-10T21:28:20.713 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:20.713 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:20.713 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:20 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.816+0000 7f7efdd71640 1 -- 172.21.15.33:0/3175253088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef810d100 msgr2=0x7f7ef810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.816+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3175253088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef810d100 0x7f7ef810f560 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ca0 tx=0x7f7ef002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.817+0000 7f7efdd71640 1 -- 172.21.15.33:0/3175253088 shutdown_connections 2024-09-10T21:28:20.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.817+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3175253088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef810d100 0x7f7ef810f560 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.817+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3175253088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ef81045c0 0x7f7ef810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.817+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3175253088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef8103ca0 0x7f7ef8104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.817+0000 7f7efdd71640 1 -- 172.21.15.33:0/3175253088 >> 172.21.15.33:0/3175253088 conn(0x7f7ef80fd700 msgr2=0x7f7ef80ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.817+0000 7f7efdd71640 1 -- 172.21.15.33:0/3175253088 shutdown_connections 2024-09-10T21:28:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.817+0000 7f7efdd71640 1 -- 172.21.15.33:0/3175253088 wait complete. 2024-09-10T21:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.818+0000 7f7efdd71640 1 Processor -- start 2024-09-10T21:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.818+0000 7f7efdd71640 1 -- start start 2024-09-10T21:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.818+0000 7f7efdd71640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef8103ca0 0x7f7ef8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.818+0000 7f7efdd71640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ef81045c0 0x7f7ef81997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7efdd71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef810d100 0x7f7ef819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7efdd71640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112d70 con 0x7f7ef8103ca0 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7efdd71640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112bf0 con 0x7f7ef81045c0 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7efdd71640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef8112ef0 con 0x7f7ef810d100 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef8103ca0 0x7f7ef8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef8103ca0 0x7f7ef8199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48634/0 (socket says 172.21.15.33:48634) 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef77fe640 1 -- 172.21.15.33:0/3359032979 learned_addr learned my addr 172.21.15.33:0/3359032979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef810d100 0x7f7ef819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef6ffd640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ef81045c0 0x7f7ef81997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef7fff640 1 -- 172.21.15.33:0/3359032979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ef81045c0 msgr2=0x7f7ef81997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ef81045c0 0x7f7ef81997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef7fff640 1 -- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef8103ca0 msgr2=0x7f7ef8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef8103ca0 0x7f7ef8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef7fff640 1 -- 172.21.15.33:0/3359032979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef819f310 con 0x7f7ef810d100 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef8103ca0 0x7f7ef8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.819+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef810d100 0x7f7ef819eb40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7ef002ade0 tx=0x7f7ef0031660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.820+0000 7f7ef4ff9640 1 -- 172.21.15.33:0/3359032979 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef0003b70 con 0x7f7ef810d100 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.820+0000 7f7ef4ff9640 1 -- 172.21.15.33:0/3359032979 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef0031910 con 0x7f7ef810d100 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.820+0000 7f7ef4ff9640 1 -- 172.21.15.33:0/3359032979 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef002f950 con 0x7f7ef810d100 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.820+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef819f5a0 con 0x7f7ef810d100 2024-09-10T21:28:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.820+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef8101830 con 0x7f7ef810d100 2024-09-10T21:28:20.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.823+0000 7f7f00146740 1 Processor -- start 2024-09-10T21:28:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7ef4ff9640 1 -- 172.21.15.33:0/3359032979 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7ef002faf0 con 0x7f7ef810d100 2024-09-10T21:28:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7f00146740 1 -- start start 2024-09-10T21:28:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7f00146740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7f00146740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7f00146740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7f00146740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558eced9de50 con 0x558ecee3c160 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7f00146740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558eced8d280 con 0x558ecee2f8b0 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7f00146740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558eced91090 con 0x558ecee301d0 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:33082/0 (socket says 172.21.15.33:33082) 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.824+0000 7f7ef6ffd640 1 -- 172.21.15.33:0/3919991865 learned_addr learned my addr 172.21.15.33:0/3919991865 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.825+0000 7f7ef77fe640 1 -- 172.21.15.33:0/3919991865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 msgr2=0x558ecee2fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.825+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.825+0000 7f7ef77fe640 1 -- 172.21.15.33:0/3919991865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 msgr2=0x558ecee3e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.825+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.825+0000 7f7ef77fe640 1 -- 172.21.15.33:0/3919991865 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558eceaeb650 con 0x558ecee301d0 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.825+0000 7f7ef6ffd640 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.825+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3bc20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8011510 tx=0x7f7ee80119d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56cedd2cf2fb55a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7eddffb640 1 -- 172.21.15.33:0/3919991865 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800f7c0 con 0x558ecee301d0 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7eddffb640 1 -- 172.21.15.33:0/3919991865 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee800fe10 con 0x558ecee301d0 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7eddffb640 1 -- 172.21.15.33:0/3919991865 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee803be10 con 0x558ecee301d0 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7ef4ff9640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ee00b5360 0x7f7ee00b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7ef4ff9640 1 -- 172.21.15.33:0/3359032979 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7ef0106c70 con 0x7f7ef810d100 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7ef6ffd640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ee00b5360 0x7f7ee00b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7f00146740 1 -- 172.21.15.33:0/3919991865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 msgr2=0x558ecee3bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.826+0000 7f7f00146740 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3bc20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8011510 tx=0x7f7ee80119d0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 -- 172.21.15.33:0/3919991865 shutdown_connections 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3bc20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 --2- 172.21.15.33:0/3919991865 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 -- 172.21.15.33:0/3919991865 >> 172.21.15.33:0/3919991865 conn(0x558ecee2d6e0 msgr2=0x558ecee2e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 -- 172.21.15.33:0/3919991865 shutdown_connections 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 -- 172.21.15.33:0/3919991865 wait complete. 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7ef6ffd640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ee00b5360 0x7f7ee00b7820 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005b90 tx=0x7f7eec004730 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 -- start start 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.827+0000 7f7f00146740 1 Processor -- start 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7f00146740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee2f8b0 0x558ecee33db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7f00146740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee301d0 0x558ecee342f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7f00146740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7f00146740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x558ecebee700 con 0x558ecee3c160 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7f00146740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558eced28110 con 0x558ecee301d0 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7f00146740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ecee24b50 con 0x558ecee2f8b0 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee301d0 0x558ecee342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef6ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee301d0 0x558ecee342f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:33088/0 (socket says 172.21.15.33:33088) 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef6ffd640 1 -- 172.21.15.33:0/3441811638 learned_addr learned my addr 172.21.15.33:0/3441811638 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee2f8b0 0x558ecee33db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7f00146740 10 client.0 _ll_register_callbacks cb 0x558eced41990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef7fff640 1 -- 172.21.15.33:0/3441811638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee2f8b0 msgr2=0x558ecee33db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee2f8b0 0x558ecee33db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef7fff640 1 -- 172.21.15.33:0/3441811638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee301d0 msgr2=0x558ecee342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee301d0 0x558ecee342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.828+0000 7f7ef7fff640 1 -- 172.21.15.33:0/3441811638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558eced55c40 con 0x558ecee3c160 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.829+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3ac30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00ca660 tx=0x7f7ef00ccb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3b613e7fed534c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.829+0000 7f7ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.829+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/3441811638 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00cdaf0 con 0x558ecee3c160 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.829+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/3441811638 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7ef00cd650 con 0x558ecee3c160 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.829+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ee800fe10 con 0x558ecee3c160 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.829+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/3441811638 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f7ef0003d10 con 0x558ecee3c160 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 10 client.19722 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x558ecee41ee0 con 0x558ecee3c160 2024-09-10T21:28:20.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/3441811638 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ee800fe10 con 0x558ecee3c160 2024-09-10T21:28:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/3441811638 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7f7ef00d64f0 con 0x558ecee3c160 2024-09-10T21:28:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 10 client.19722 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 10 client.19722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 20 client.19722 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558ecee458a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 4 client.19722 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:20.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.830+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558eced9fda0 con 0x558ecee43440 2024-09-10T21:28:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.831+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558ecee458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.832+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558ecee458a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6ff6d163aab26f8 server_cookie=24f8038b1981d066 in_seq=0 out_seq=0 2024-09-10T21:28:20.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.832+0000 7f7ed6ffd640 10 client.19722 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/3441811638 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558eced9fda0 con 0x558ecee43440 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7ed6ffd640 10 client.19722 handle_command_reply: tid=0 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 client.19722 shutdown 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 msgr2=0x558ecee3ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3ac30 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00ca660 tx=0x7f7ef00ccb60 comp rx=0 tx=0).stop 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 shutdown_connections 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558ecee458a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee3ac30 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee301d0 0x558ecee342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7ed6ffd640 0 client.19722 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 --2- 172.21.15.33:0/3441811638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee2f8b0 0x558ecee33db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 >> 172.21.15.33:0/3441811638 conn(0x558ecee2d6e0 msgr2=0x558ecee33250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:20.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 shutdown_connections 2024-09-10T21:28:20.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.833+0000 7f7f00146740 1 -- 172.21.15.33:0/3441811638 wait complete. 2024-09-10T21:28:20.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.834+0000 7f7f00146740 20 client.19722 trim_cache size 0 max 16384 2024-09-10T21:28:20.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 Processor -- start 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 -- start start 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee25de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ac260 con 0x558ecee3c160 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ade90 con 0x558ecee2f8b0 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7f00146740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ac960 con 0x558ecee301d0 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee25de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:33096/0 (socket says 172.21.15.33:33096) 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.858+0000 7f7ef77fe640 1 -- 172.21.15.33:0/2955727383 learned_addr learned my addr 172.21.15.33:0/2955727383 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:20.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef6ffd640 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef7fff640 1 -- 172.21.15.33:0/2955727383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 msgr2=0x558ecee3ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef7fff640 1 -- 172.21.15.33:0/2955727383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 msgr2=0x558ecee2ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef7fff640 1 -- 172.21.15.33:0/2955727383 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ecee30000 con 0x558ecee3c160 2024-09-10T21:28:20.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee25de0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00cd380 tx=0x7f7ef002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c646a0b774ce560 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7edd7fa640 1 -- 172.21.15.33:0/2955727383 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00c9a40 con 0x558ecee3c160 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7edd7fa640 1 -- 172.21.15.33:0/2955727383 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef0036070 con 0x558ecee3c160 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.859+0000 7f7edd7fa640 1 -- 172.21.15.33:0/2955727383 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00cd3f0 con 0x558ecee3c160 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.860+0000 7f7f00146740 1 -- 172.21.15.33:0/2955727383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 msgr2=0x558ecee25de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.860+0000 7f7f00146740 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee25de0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7ef00cd380 tx=0x7f7ef002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.860+0000 7f7f00146740 1 -- 172.21.15.33:0/2955727383 shutdown_connections 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee3c160 0x558ecee25de0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee3ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 --2- 172.21.15.33:0/2955727383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee2f8b0 0x558ecee2ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 -- 172.21.15.33:0/2955727383 >> 172.21.15.33:0/2955727383 conn(0x558ecee2d6e0 msgr2=0x558ecee2e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 -- 172.21.15.33:0/2955727383 shutdown_connections 2024-09-10T21:28:20.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 -- 172.21.15.33:0/2955727383 wait complete. 2024-09-10T21:28:20.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 -- start start 2024-09-10T21:28:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.861+0000 7f7f00146740 1 Processor -- start 2024-09-10T21:28:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7f00146740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee2f8b0 0x558ecee3a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7f00146740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee36ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7f00146740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee3c160 0x558ecee370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7f00146740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ad760 con 0x558ecee2f8b0 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7f00146740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81ad060 con 0x558ecee3c160 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7f00146740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef81aa220 con 0x558ecee301d0 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7f00146740 10 client.0 _ll_register_callbacks cb 0x7f7ef0005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee3c160 0x558ecee370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee2f8b0 0x558ecee3a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee3c160 0x558ecee370e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:33112/0 (socket says 172.21.15.33:33112) 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef77fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee2f8b0 0x558ecee3a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48660/0 (socket says 172.21.15.33:48660) 2024-09-10T21:28:20.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 -- 172.21.15.33:0/468833359 learned_addr learned my addr 172.21.15.33:0/468833359 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef6ffd640 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee36ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 -- 172.21.15.33:0/468833359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 msgr2=0x558ecee36ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee36ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 -- 172.21.15.33:0/468833359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee2f8b0 msgr2=0x558ecee3a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee2f8b0 0x558ecee3a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.862+0000 7f7ef7fff640 1 -- 172.21.15.33:0/468833359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ecee37650 con 0x558ecee3c160 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.863+0000 7f7ef7fff640 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee3c160 0x558ecee370e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0030040 tx=0x7f7ef00cd980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efdb1d8f757ac226 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.863+0000 7f7ed6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.863+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/468833359 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00e6070 con 0x558ecee3c160 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.863+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/468833359 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7ef0004d40 con 0x558ecee3c160 2024-09-10T21:28:20.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.863+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef0036070 con 0x558ecee3c160 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.863+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/468833359 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) ==== 24+0+0 (secure 0 0 0) 0x7f7ef00c9a40 con 0x558ecee3c160 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 10 client.29039 fetch_fsmap learned FSMap version 95 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=4+}) -- 0x558ecee37650 con 0x558ecee3c160 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/468833359 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef0036070 con 0x558ecee3c160 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/468833359 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 95) ==== 8394+0+0 (secure 0 0 0) 0x7f7ef00c8d40 con 0x558ecee3c160 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 10 client.29039 fetch_fsmap finished waiting for FSMap version 95 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 10 client.29039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 20 client.29039 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558eceeb34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 4 client.29039 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:28:20.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558eced9b850 con 0x558ecee43440 2024-09-10T21:28:20.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.864+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558eceeb34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:20.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.868+0000 7f7ef77fe640 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558eceeb34d0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff86c982d618211c server_cookie=1eb3e13b6c40eed8 in_seq=0 out_seq=0 2024-09-10T21:28:20.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.868+0000 7f7ed6ffd640 10 client.29039 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.869+0000 7f7ed6ffd640 1 -- 172.21.15.33:0/468833359 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9090 (crc 0 0 0) 0x558eced9b850 con 0x558ecee43440 2024-09-10T21:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.869+0000 7f7ed6ffd640 10 client.29039 handle_command_reply: tid=0 2024-09-10T21:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.869+0000 7f7f00146740 1 client.29039 shutdown 2024-09-10T21:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.869+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee3c160 msgr2=0x558ecee370e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.869+0000 7f7f00146740 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee3c160 0x558ecee370e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0030040 tx=0x7f7ef00cd980 comp rx=0 tx=0).stop 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 shutdown_connections 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x558ecee43440 0x558eceeb34d0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558ecee3c160 0x558ecee370e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ecee301d0 0x558ecee36ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 --2- 172.21.15.33:0/468833359 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558ecee2f8b0 0x558ecee3a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 >> 172.21.15.33:0/468833359 conn(0x558ecee2d6e0 msgr2=0x558ecee23700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7ed6ffd640 0 client.29039 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 shutdown_connections 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 1 -- 172.21.15.33:0/468833359 wait complete. 2024-09-10T21:28:20.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.870+0000 7f7f00146740 20 client.29039 trim_cache size 0 max 16384 2024-09-10T21:28:20.874 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:28:20.874 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:28:20.874 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": { 2024-09-10T21:28:20.874 INFO:teuthology.orchestra.run.smithi033.stdout: "abb3152f-b4a8-4eb5-baa6-ac4d8cbb70b0": { 2024-09-10T21:28:20.874 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tag": "abb3152f-b4a8-4eb5-baa6-ac4d8cbb70b0", 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "options": "recursive,force" 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.876 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.877 INFO:teuthology.orchestra.run.smithi033.stdout: "b22ec403-44be-4b2f-9c71-e7053f2ad4b0": { 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:27:26.176127+0000", 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.878 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.879 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.880 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.881 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:28:20.882 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.883 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.884 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.885 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.886 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "abb3152f-b4a8-4eb5-baa6-ac4d8cbb70b0": { 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:28:19.207486+0000", 2024-09-10T21:28:20.887 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.888 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.889 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:28:20.890 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:28:20.891 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.892 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "05708340-c3c3-4430-9bae-81feded8d778": { 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:26:35.545755+0000", 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:20.893 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ee00b5360 msgr2=0x7f7ee00b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ee00b5360 0x7f7ee00b7820 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f7eec005b90 tx=0x7f7eec004730 comp rx=0 tx=0).stop 2024-09-10T21:28:20.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef810d100 msgr2=0x7f7ef819eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef810d100 0x7f7ef819eb40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7ef002ade0 tx=0x7f7ef0031660 comp rx=0 tx=0).stop 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 shutdown_connections 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7ee00b5360 0x7f7ee00b7820 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ef810d100 0x7f7ef819eb40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7ef81045c0 0x7f7ef81997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 --2- 172.21.15.33:0/3359032979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7ef8103ca0 0x7f7ef8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 >> 172.21.15.33:0/3359032979 conn(0x7f7ef80fd700 msgr2=0x7f7ef810f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 shutdown_connections 2024-09-10T21:28:20.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:20.871+0000 7f7efdd71640 1 -- 172.21.15.33:0/3359032979 wait complete. 2024-09-10T21:28:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:28:20.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:28:20.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:28:20.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:20 smithi174 ceph-mon[35635]: scrub summary: active paths [/] 2024-09-10T21:28:20.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:20.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:20.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:20 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:28:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:28:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:28:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:20 smithi104 ceph-mon[35325]: scrub summary: active paths [/] 2024-09-10T21:28:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:20 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 490096 37.33 MB/sec execute 561 sec latency 132.074 ms 2024-09-10T21:28:21.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:21 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:21 smithi033 ceph-mon[30860]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.669 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:21 smithi033 ceph-mon[30860]: pgmap v1077: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-09-10T21:28:21.697 INFO:tasks.cephfs.filesystem:scrub status for tag:abb3152f-b4a8-4eb5-baa6-ac4d8cbb70b0 - {'path': '/', 'tag': 'abb3152f-b4a8-4eb5-baa6-ac4d8cbb70b0', 'options': 'recursive,force'} 2024-09-10T21:28:21.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:21 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:21 smithi174 ceph-mon[35635]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:21.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:21 smithi174 ceph-mon[35635]: pgmap v1077: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-09-10T21:28:22.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:21 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:22.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:21 smithi104 ceph-mon[35325]: scrub summary: idle+waiting paths [/] 2024-09-10T21:28:22.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:21 smithi104 ceph-mon[35325]: pgmap v1077: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 12 MiB/s wr, 743 op/s 2024-09-10T21:28:22.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 490693 37.31 MB/sec execute 562 sec latency 241.193 ms 2024-09-10T21:28:23.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 491467 37.33 MB/sec execute 563 sec latency 79.661 ms 2024-09-10T21:28:23.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:23 smithi174 ceph-mon[35635]: pgmap v1078: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 661 op/s 2024-09-10T21:28:23.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:23 smithi033 ceph-mon[30860]: pgmap v1078: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 661 op/s 2024-09-10T21:28:24.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:23 smithi104 ceph-mon[35325]: pgmap v1078: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 11 MiB/s wr, 661 op/s 2024-09-10T21:28:24.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 492194 37.33 MB/sec execute 564 sec latency 95.597 ms 2024-09-10T21:28:25.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 492892 37.34 MB/sec execute 565 sec latency 71.147 ms 2024-09-10T21:28:25.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:25 smithi174 ceph-mon[35635]: pgmap v1079: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 849 op/s 2024-09-10T21:28:25.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:25 smithi033 ceph-mon[30860]: pgmap v1079: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 849 op/s 2024-09-10T21:28:26.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:25 smithi104 ceph-mon[35325]: pgmap v1079: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 849 op/s 2024-09-10T21:28:26.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 493557 37.34 MB/sec execute 566 sec latency 70.749 ms 2024-09-10T21:28:27.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 494007 37.30 MB/sec execute 567 sec latency 320.332 ms 2024-09-10T21:28:27.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:27 smithi174 ceph-mon[35635]: pgmap v1080: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T21:28:27.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:27 smithi033 ceph-mon[30860]: pgmap v1080: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T21:28:28.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:27 smithi104 ceph-mon[35325]: pgmap v1080: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 10 MiB/s wr, 728 op/s 2024-09-10T21:28:28.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 494589 37.29 MB/sec execute 568 sec latency 545.165 ms 2024-09-10T21:28:28.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:28 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:28.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:28 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:29.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:28 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:29.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 495427 37.31 MB/sec execute 569 sec latency 79.470 ms 2024-09-10T21:28:29.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:29 smithi174 ceph-mon[35635]: pgmap v1081: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 695 op/s 2024-09-10T21:28:29.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:29 smithi033 ceph-mon[30860]: pgmap v1081: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 695 op/s 2024-09-10T21:28:30.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:29 smithi104 ceph-mon[35325]: pgmap v1081: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 10 MiB/s wr, 695 op/s 2024-09-10T21:28:30.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 496286 37.32 MB/sec execute 570 sec latency 46.629 ms 2024-09-10T21:28:31.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 497043 37.34 MB/sec execute 571 sec latency 81.389 ms 2024-09-10T21:28:31.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:31 smithi174 ceph-mon[35635]: pgmap v1082: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 733 op/s 2024-09-10T21:28:31.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:31 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:28:31.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:31 smithi033 ceph-mon[30860]: pgmap v1082: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 733 op/s 2024-09-10T21:28:31.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:31 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:28:32.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:31 smithi104 ceph-mon[35325]: pgmap v1082: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.4 MiB/s wr, 733 op/s 2024-09-10T21:28:32.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:31 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:28:32.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 497750 37.32 MB/sec execute 572 sec latency 71.564 ms 2024-09-10T21:28:32.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:32 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:28:32.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:32 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:28:32.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:32 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:28:32.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:32 smithi174 ceph-mon[35635]: scrub summary: idle 2024-09-10T21:28:32.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:32 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:28:32.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:32 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:28:32.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:32 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:28:32.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:32 smithi033 ceph-mon[30860]: scrub summary: idle 2024-09-10T21:28:33.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:32 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:28:33.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:32 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:28:33.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:32 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:28:33.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:32 smithi104 ceph-mon[35325]: scrub summary: idle 2024-09-10T21:28:33.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 498482 37.36 MB/sec execute 573 sec latency 112.498 ms 2024-09-10T21:28:33.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:33 smithi174 ceph-mon[35635]: pgmap v1083: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 550 op/s 2024-09-10T21:28:33.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:33 smithi033 ceph-mon[30860]: pgmap v1083: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 550 op/s 2024-09-10T21:28:34.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:33 smithi104 ceph-mon[35325]: pgmap v1083: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 4.1 MiB/s wr, 550 op/s 2024-09-10T21:28:34.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 499137 37.35 MB/sec execute 574 sec latency 57.636 ms 2024-09-10T21:28:35.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 500031 37.38 MB/sec execute 575 sec latency 104.685 ms 2024-09-10T21:28:35.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:35 smithi174 ceph-mon[35635]: pgmap v1084: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 627 op/s 2024-09-10T21:28:35.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:35 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:35.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:35 smithi033 ceph-mon[30860]: pgmap v1084: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 627 op/s 2024-09-10T21:28:35.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:35 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:35 smithi104 ceph-mon[35325]: pgmap v1084: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 4.1 MiB/s wr, 627 op/s 2024-09-10T21:28:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:35 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:36.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 500144 37.32 MB/sec execute 576 sec latency 826.570 ms 2024-09-10T21:28:37.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 500369 37.29 MB/sec execute 577 sec latency 1009.104 ms 2024-09-10T21:28:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:37 smithi174 ceph-mon[35635]: pgmap v1085: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.4 MiB/s wr, 534 op/s 2024-09-10T21:28:37.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:37 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:37.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:37 smithi033 ceph-mon[30860]: pgmap v1085: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.4 MiB/s wr, 534 op/s 2024-09-10T21:28:37.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:37 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:38.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:37 smithi104 ceph-mon[35325]: pgmap v1085: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 4.4 MiB/s wr, 534 op/s 2024-09-10T21:28:38.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:37 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:28:38.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 500845 37.26 MB/sec execute 578 sec latency 122.261 ms 2024-09-10T21:28:39.089 INFO:tasks.workunit.client.0.smithi033.stdout: 10 501590 37.28 MB/sec execute 579 sec latency 245.848 ms 2024-09-10T21:28:39.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:39 smithi033 ceph-mon[30860]: pgmap v1086: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 675 op/s 2024-09-10T21:28:39.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:39 smithi104 ceph-mon[35325]: pgmap v1086: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 675 op/s 2024-09-10T21:28:39.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:39 smithi174 ceph-mon[35635]: pgmap v1086: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 9.2 MiB/s wr, 675 op/s 2024-09-10T21:28:40.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 502103 37.26 MB/sec execute 580 sec latency 253.844 ms 2024-09-10T21:28:41.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 502668 37.23 MB/sec execute 581 sec latency 79.738 ms 2024-09-10T21:28:41.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:41 smithi104 ceph-mon[35325]: pgmap v1087: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 793 op/s 2024-09-10T21:28:41.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:41 smithi174 ceph-mon[35635]: pgmap v1087: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 793 op/s 2024-09-10T21:28:41.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:41 smithi033 ceph-mon[30860]: pgmap v1087: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 13 MiB/s wr, 793 op/s 2024-09-10T21:28:42.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 503471 37.25 MB/sec execute 582 sec latency 96.579 ms 2024-09-10T21:28:43.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 504342 37.27 MB/sec execute 583 sec latency 66.614 ms 2024-09-10T21:28:43.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:43 smithi104 ceph-mon[35325]: pgmap v1088: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 715 op/s 2024-09-10T21:28:43.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:43 smithi174 ceph-mon[35635]: pgmap v1088: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 715 op/s 2024-09-10T21:28:43.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:43 smithi033 ceph-mon[30860]: pgmap v1088: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 13 MiB/s wr, 715 op/s 2024-09-10T21:28:44.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 505008 37.27 MB/sec execute 584 sec latency 53.686 ms 2024-09-10T21:28:45.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 505545 37.25 MB/sec execute 585 sec latency 123.500 ms 2024-09-10T21:28:45.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:45 smithi174 ceph-mon[35635]: pgmap v1089: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 860 op/s 2024-09-10T21:28:45.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:45 smithi033 ceph-mon[30860]: pgmap v1089: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 860 op/s 2024-09-10T21:28:46.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:45 smithi104 ceph-mon[35325]: pgmap v1089: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 14 MiB/s wr, 860 op/s 2024-09-10T21:28:46.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 506107 37.26 MB/sec execute 586 sec latency 93.648 ms 2024-09-10T21:28:47.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 506874 37.26 MB/sec execute 587 sec latency 77.367 ms 2024-09-10T21:28:47.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:47 smithi174 ceph-mon[35635]: pgmap v1090: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 777 op/s 2024-09-10T21:28:47.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:47 smithi033 ceph-mon[30860]: pgmap v1090: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 777 op/s 2024-09-10T21:28:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:47 smithi104 ceph-mon[35325]: pgmap v1090: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 14 MiB/s wr, 777 op/s 2024-09-10T21:28:48.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 507701 37.29 MB/sec execute 588 sec latency 85.173 ms 2024-09-10T21:28:49.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 508465 37.30 MB/sec execute 589 sec latency 75.108 ms 2024-09-10T21:28:49.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:49 smithi174 ceph-mon[35635]: pgmap v1091: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 764 op/s 2024-09-10T21:28:49.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:49 smithi033 ceph-mon[30860]: pgmap v1091: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 764 op/s 2024-09-10T21:28:50.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:49 smithi104 ceph-mon[35325]: pgmap v1091: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 764 op/s 2024-09-10T21:28:50.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 509065 37.30 MB/sec execute 590 sec latency 69.350 ms 2024-09-10T21:28:51.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 509692 37.28 MB/sec execute 591 sec latency 59.710 ms 2024-09-10T21:28:51.698 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-10T21:28:51.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:51 smithi174 ceph-mon[35635]: pgmap v1092: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 682 op/s 2024-09-10T21:28:51.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:51 smithi033 ceph-mon[30860]: pgmap v1092: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 682 op/s 2024-09-10T21:28:52.037 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:52.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:51 smithi104 ceph-mon[35325]: pgmap v1092: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.6 MiB/s wr, 682 op/s 2024-09-10T21:28:52.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 510369 37.28 MB/sec execute 592 sec latency 100.533 ms 2024-09-10T21:28:52.758 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:52 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:28:52.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:52 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:28:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 -- 172.21.15.33:0/3567037502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8410d140 msgr2=0x7f7c8410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 --2- 172.21.15.33:0/3567037502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8410d140 0x7f7c8410f550 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009ca0 tx=0x7f7c7c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 -- 172.21.15.33:0/3567037502 shutdown_connections 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 --2- 172.21.15.33:0/3567037502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8410d140 0x7f7c8410f550 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 --2- 172.21.15.33:0/3567037502 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c84104590 0x7f7c8410cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 --2- 172.21.15.33:0/3567037502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c84103b90 0x7f7c84103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 -- 172.21.15.33:0/3567037502 >> 172.21.15.33:0/3567037502 conn(0x7f7c840fd700 msgr2=0x7f7c840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 -- 172.21.15.33:0/3567037502 shutdown_connections 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.980+0000 7f7c8b294640 1 -- 172.21.15.33:0/3567037502 wait complete. 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.981+0000 7f7c8b294640 1 Processor -- start 2024-09-10T21:28:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.981+0000 7f7c8b294640 1 -- start start 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c8b294640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c84103b90 0x7f7c84199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c8b294640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c84104590 0x7f7c84199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c89030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c84103b90 0x7f7c84199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c8b294640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8410d140 0x7f7c8419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c89030640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c84103b90 0x7f7c84199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42564/0 (socket says 172.21.15.33:42564) 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c89030640 1 -- 172.21.15.33:0/1532574294 learned_addr learned my addr 172.21.15.33:0/1532574294 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84112d60 con 0x7f7c84103b90 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84112be0 con 0x7f7c84104590 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.982+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84112ee0 con 0x7f7c8410d140 2024-09-10T21:28:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.983+0000 7f7c8882f640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c84104590 0x7f7c84199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.983+0000 7f7c89831640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8410d140 0x7f7c8419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.983+0000 7f7c89030640 1 -- 172.21.15.33:0/1532574294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8410d140 msgr2=0x7f7c8419e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.983+0000 7f7c89030640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8410d140 0x7f7c8419e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.983+0000 7f7c89030640 1 -- 172.21.15.33:0/1532574294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c84104590 msgr2=0x7f7c84199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.983+0000 7f7c89030640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c84104590 0x7f7c84199590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.983+0000 7f7c89030640 1 -- 172.21.15.33:0/1532574294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8419f100 con 0x7f7c84103b90 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.984+0000 7f7c89030640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c84103b90 0x7f7c84199050 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000e9f0 tx=0x7f7c8000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.984+0000 7f7c7a7fc640 1 -- 172.21.15.33:0/1532574294 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8000cd80 con 0x7f7c84103b90 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.984+0000 7f7c7a7fc640 1 -- 172.21.15.33:0/1532574294 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c800054d0 con 0x7f7c84103b90 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.984+0000 7f7c7a7fc640 1 -- 172.21.15.33:0/1532574294 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80010690 con 0x7f7c84103b90 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.984+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8419f3f0 con 0x7f7c84103b90 2024-09-10T21:28:52.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.984+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c841018f0 con 0x7f7c84103b90 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.987+0000 7f7c8d669740 1 Processor -- start 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8d669740 1 -- start start 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90378b0 0x55e6e9037c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90381d0 0x55e6e9032840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e9032d80 0x55e6e90331c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8d669740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e6e8fa5e50 con 0x55e6e90378b0 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8d669740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e6e8f95280 con 0x55e6e9032d80 2024-09-10T21:28:52.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8d669740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6e8f99090 con 0x55e6e90381d0 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c89831640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90378b0 0x55e6e9037c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c7a7fc640 1 -- 172.21.15.33:0/1532574294 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7c80004e90 con 0x7f7c84103b90 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c89831640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90378b0 0x55e6e9037c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42580/0 (socket says 172.21.15.33:42580) 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c89831640 1 -- 172.21.15.33:0/1501975209 learned_addr learned my addr 172.21.15.33:0/1501975209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.988+0000 7f7c8882f640 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e9032d80 0x55e6e90331c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c89030640 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90381d0 0x55e6e9032840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c8882f640 1 -- 172.21.15.33:0/1501975209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90381d0 msgr2=0x55e6e9032840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c8882f640 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90381d0 0x55e6e9032840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c8882f640 1 -- 172.21.15.33:0/1501975209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90378b0 msgr2=0x55e6e9037c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c8882f640 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90378b0 0x55e6e9037c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c8882f640 1 -- 172.21.15.33:0/1501975209 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e8cf3650 con 0x55e6e9032d80 2024-09-10T21:28:52.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c89831640 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90378b0 0x55e6e9037c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c8882f640 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e9032d80 0x55e6e90331c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7c74012a00 tx=0x7f7c74012ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c18f3ba43eff66b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1501975209 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74004050 con 0x55e6e9032d80 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1501975209 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c740041f0 con 0x55e6e9032d80 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.989+0000 7f7c6f7fe640 1 -- 172.21.15.33:0/1501975209 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7401b730 con 0x55e6e9032d80 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.990+0000 7f7c7a7fc640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7c700b52e0 0x7f7c700b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.990+0000 7f7c7a7fc640 1 -- 172.21.15.33:0/1532574294 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7c800e5fe0 con 0x7f7c84103b90 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.990+0000 7f7c8d669740 1 -- 172.21.15.33:0/1501975209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e9032d80 msgr2=0x55e6e90331c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.990+0000 7f7c8d669740 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e9032d80 0x55e6e90331c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7c74012a00 tx=0x7f7c74012ec0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.991+0000 7f7c8d669740 1 -- 172.21.15.33:0/1501975209 shutdown_connections 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.991+0000 7f7c8d669740 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e9032d80 0x55e6e90331c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.991+0000 7f7c8d669740 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90381d0 0x55e6e9032840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.991+0000 7f7c8d669740 1 --2- 172.21.15.33:0/1501975209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90378b0 0x55e6e9037c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.991+0000 7f7c8d669740 1 -- 172.21.15.33:0/1501975209 >> 172.21.15.33:0/1501975209 conn(0x55e6e90356e0 msgr2=0x55e6e9036880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:52.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.991+0000 7f7c8d669740 1 -- 172.21.15.33:0/1501975209 shutdown_connections 2024-09-10T21:28:52.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.991+0000 7f7c8d669740 1 -- 172.21.15.33:0/1501975209 wait complete. 2024-09-10T21:28:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.992+0000 7f7c8d669740 1 -- start start 2024-09-10T21:28:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.992+0000 7f7c89831640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7c700b52e0 0x7f7c700b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.992+0000 7f7c8d669740 1 Processor -- start 2024-09-10T21:28:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e903f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e903f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e903e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8d669740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55e6e8df6700 con 0x55e6e90381d0 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8d669740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55e6e8f30110 con 0x55e6e90378b0 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8d669740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6e9033a00 con 0x55e6e903de50 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8d669740 10 client.0 _ll_register_callbacks cb 0x55e6e8f49990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89030640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e903f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89030640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e903f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38656/0 (socket says 172.21.15.33:38656) 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89030640 1 -- 172.21.15.33:0/2607446144 learned_addr learned my addr 172.21.15.33:0/2607446144 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c8882f640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e903f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89831640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e903e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89030640 1 -- 172.21.15.33:0/2607446144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 msgr2=0x55e6e903e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89030640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e903e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:52.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89030640 1 -- 172.21.15.33:0/2607446144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 msgr2=0x55e6e903f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.993+0000 7f7c89030640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e903f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c89030640 1 -- 172.21.15.33:0/2607446144 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e8f5dc40 con 0x55e6e90378b0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c89831640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e903e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c89831640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7c700b52e0 0x7f7c700b77a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009c70 tx=0x7f7c7c005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c6cff9640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c89030640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e903f080 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000e9c0 tx=0x7f7c800048f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe62006f2e9105cd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c6cff9640 1 -- 172.21.15.33:0/2607446144 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800abf10 con 0x55e6e90378b0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c6cff9640 1 -- 172.21.15.33:0/2607446144 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7c800a8a10 con 0x55e6e90378b0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.994+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c740041f0 con 0x55e6e90378b0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c6cff9640 1 -- 172.21.15.33:0/2607446144 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f7c800abf10 con 0x55e6e90378b0 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 10 client.29051 fetch_fsmap learned FSMap version 98 2024-09-10T21:28:53.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) -- 0x55e6e904a560 con 0x55e6e90378b0 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c6cff9640 1 -- 172.21.15.33:0/2607446144 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c740041f0 con 0x55e6e90378b0 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c6cff9640 1 -- 172.21.15.33:0/2607446144 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 98) ==== 8410+0+0 (secure 0 0 0) 0x7f7c800b58a0 con 0x55e6e90378b0 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 10 client.29051 fetch_fsmap finished waiting for FSMap version 98 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 10 client.29051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 20 client.29051 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e904de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 4 client.29051 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.995+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6e8fa7da0 con 0x55e6e904ba20 2024-09-10T21:28:53.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.996+0000 7f7c8882f640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e904de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.997+0000 7f7c6cff9640 10 client.29051 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:53.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:52.997+0000 7f7c8882f640 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e904de10 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bffb722d2a22714 server_cookie=8a31c0fbe8251624 in_seq=0 out_seq=0 2024-09-10T21:28:53.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.001+0000 7f7c6cff9640 1 -- 172.21.15.33:0/2607446144 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e6e8fa7da0 con 0x55e6e904ba20 2024-09-10T21:28:53.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.001+0000 7f7c6cff9640 10 client.29051 handle_command_reply: tid=0 2024-09-10T21:28:53.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.001+0000 7f7c8d669740 1 client.29051 shutdown 2024-09-10T21:28:53.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 msgr2=0x55e6e903f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:53.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e903f080 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000e9c0 tx=0x7f7c800048f0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 shutdown_connections 2024-09-10T21:28:53.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e904de10 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c6cff9640 0 client.29051 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:53.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e903e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e903f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2607446144 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e903f080 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 >> 172.21.15.33:0/2607446144 conn(0x55e6e90356e0 msgr2=0x55e6e903b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 shutdown_connections 2024-09-10T21:28:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 1 -- 172.21.15.33:0/2607446144 wait complete. 2024-09-10T21:28:53.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.002+0000 7f7c8d669740 20 client.29051 trim_cache size 0 max 16384 2024-09-10T21:28:53.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.027+0000 7f7c8d669740 1 Processor -- start 2024-09-10T21:28:53.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.027+0000 7f7c8d669740 1 -- start start 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.027+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e9031ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.027+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e9034c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.027+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e9031230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c8d669740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ac470 con 0x55e6e90381d0 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c8d669740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ae130 con 0x55e6e90378b0 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c8d669740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841acba0 con 0x55e6e903de50 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89831640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e9031230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e9031ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e9031ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38672/0 (socket says 172.21.15.33:38672) 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 -- 172.21.15.33:0/2597778914 learned_addr learned my addr 172.21.15.33:0/2597778914 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 -- 172.21.15.33:0/2597778914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 msgr2=0x55e6e9031230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c8882f640 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e9034c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e9031230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 -- 172.21.15.33:0/2597778914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 msgr2=0x55e6e9034c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e9034c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89030640 1 -- 172.21.15.33:0/2597778914 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6e9041ec0 con 0x55e6e90378b0 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.028+0000 7f7c89831640 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e9031230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.029+0000 7f7c8882f640 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e9034c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.029+0000 7f7c89030640 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e9031ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000cfd0 tx=0x7f7c800a8df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7650cdeb8cfa55d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.029+0000 7f7c6effd640 1 -- 172.21.15.33:0/2597778914 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c800abf10 con 0x55e6e90378b0 2024-09-10T21:28:53.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.029+0000 7f7c6effd640 1 -- 172.21.15.33:0/2597778914 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c800cf040 con 0x55e6e90378b0 2024-09-10T21:28:53.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.029+0000 7f7c6effd640 1 -- 172.21.15.33:0/2597778914 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c80005680 con 0x55e6e90378b0 2024-09-10T21:28:53.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 -- 172.21.15.33:0/2597778914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 msgr2=0x55e6e9031ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:53.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e9031ea0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000cfd0 tx=0x7f7c800a8df0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 -- 172.21.15.33:0/2597778914 shutdown_connections 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e903de50 0x55e6e9031230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e90381d0 0x55e6e9034c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 --2- 172.21.15.33:0/2597778914 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90378b0 0x55e6e9031ea0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 -- 172.21.15.33:0/2597778914 >> 172.21.15.33:0/2597778914 conn(0x55e6e90356e0 msgr2=0x55e6e902dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 -- 172.21.15.33:0/2597778914 shutdown_connections 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.030+0000 7f7c8d669740 1 -- 172.21.15.33:0/2597778914 wait complete. 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.031+0000 7f7c8d669740 1 -- start start 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.031+0000 7f7c8d669740 1 Processor -- start 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.031+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90378b0 0x55e6e9053630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.031+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90381d0 0x55e6e9053b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.031+0000 7f7c8d669740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e903de50 0x55e6e90540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8d669740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ad9d0 con 0x55e6e903de50 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8882f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90381d0 0x55e6e9053b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8882f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90381d0 0x55e6e9053b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38682/0 (socket says 172.21.15.33:38682) 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8882f640 1 -- 172.21.15.33:0/3802767451 learned_addr learned my addr 172.21.15.33:0/3802767451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89030640 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90378b0 0x55e6e9053630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841ad2d0 con 0x55e6e90381d0 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89831640 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e903de50 0x55e6e90540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841aa3f0 con 0x55e6e90378b0 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8d669740 10 client.0 _ll_register_callbacks cb 0x7f7c8000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89831640 1 -- 172.21.15.33:0/3802767451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90378b0 msgr2=0x55e6e9053630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89831640 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90378b0 0x55e6e9053630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89831640 1 -- 172.21.15.33:0/3802767451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90381d0 msgr2=0x55e6e9053b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89831640 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90381d0 0x55e6e9053b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89831640 1 -- 172.21.15.33:0/3802767451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6e9046890 con 0x55e6e903de50 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c800cf040 con 0x55e6e903de50 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c89831640 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e903de50 0x55e6e90540b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009a60 tx=0x7f7c7c063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51a45c20b4ddb28 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.032+0000 7f7c6cff9640 10 client.19743 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.033+0000 7f7c6cff9640 1 -- 172.21.15.33:0/3802767451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7c07b060 con 0x55e6e903de50 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.033+0000 7f7c6cff9640 1 -- 172.21.15.33:0/3802767451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7c7c004040 con 0x55e6e903de50 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.033+0000 7f7c6cff9640 1 -- 172.21.15.33:0/3802767451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f7c7c004240 con 0x55e6e903de50 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.033+0000 7f7c8d669740 10 client.19743 fetch_fsmap learned FSMap version 98 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.033+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=98}) -- 0x55e6e90b8ab0 con 0x55e6e903de50 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c6cff9640 1 -- 172.21.15.33:0/3802767451 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 98) ==== 8410+0+0 (secure 0 0 0) 0x7f7c7c0043e0 con 0x55e6e903de50 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c8d669740 10 client.19743 fetch_fsmap finished waiting for FSMap version 98 2024-09-10T21:28:53.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c8d669740 10 client.19743 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:28:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c8d669740 20 client.19743 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c8d669740 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e90bb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c8d669740 4 client.19743 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:28:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e6e8fa3850 con 0x55e6e904ba20 2024-09-10T21:28:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c89030640 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e90bb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c6cff9640 10 client.19743 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:53.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.034+0000 7f7c89030640 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e90bb1f0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f70fe5f38d47e272 server_cookie=56c241effbe63ef7 in_seq=0 out_seq=0 2024-09-10T21:28:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.037+0000 7f7c6cff9640 1 -- 172.21.15.33:0/3802767451 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+8894 (crc 0 0 0) 0x55e6e8fa3850 con 0x55e6e904ba20 2024-09-10T21:28:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.037+0000 7f7c6cff9640 10 client.19743 handle_command_reply: tid=0 2024-09-10T21:28:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.037+0000 7f7c8d669740 1 client.19743 shutdown 2024-09-10T21:28:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e903de50 msgr2=0x55e6e90540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e903de50 0x55e6e90540b0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009a60 tx=0x7f7c7c063000 comp rx=0 tx=0).stop 2024-09-10T21:28:53.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 shutdown_connections 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55e6e904ba20 0x55e6e90bb1f0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55e6e903de50 0x55e6e90540b0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55e6e90381d0 0x55e6e9053b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 --2- 172.21.15.33:0/3802767451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6e90378b0 0x55e6e9053630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 >> 172.21.15.33:0/3802767451 conn(0x55e6e90356e0 msgr2=0x55e6e90311b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c6cff9640 0 client.19743 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 shutdown_connections 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 1 -- 172.21.15.33:0/3802767451 wait complete. 2024-09-10T21:28:53.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.038+0000 7f7c8d669740 20 client.19743 trim_cache size 0 max 16384 2024-09-10T21:28:53.043 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:28:53.043 INFO:teuthology.orchestra.run.smithi033.stdout: "status": "no active scrubs running", 2024-09-10T21:28:53.043 INFO:teuthology.orchestra.run.smithi033.stdout: "scrubs": {}, 2024-09-10T21:28:53.043 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_stats": { 2024-09-10T21:28:53.043 INFO:teuthology.orchestra.run.smithi033.stdout: "e69efbf4-61ef-4ef6-be19-b7823dae502f": { 2024-09-10T21:28:53.043 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:12:34.693268+0000", 2024-09-10T21:28:53.043 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "1ad13a76-9513-4068-b340-0c8aa514c166": { 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:13:19.951702+0000", 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.044 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "13d23824-38f6-4822-b207-44e9d8d4829e": { 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:22:18.005980+0000", 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "0038ceab-3214-49d2-b703-dad66105eb96": { 2024-09-10T21:28:53.045 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:03.770404+0000", 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "b22ec403-44be-4b2f-9c71-e7053f2ad4b0": { 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:27:26.176127+0000", 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "ee25ff54-3f06-4448-8815-95355590c2e1": { 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:10:18.440971+0000", 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "38b6b1ae-0098-4918-bd6a-8bcb16725aa7": { 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:08:47.529109+0000", 2024-09-10T21:28:53.047 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "0512e745-916d-4819-9e47-d7b895d38518": { 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:59.633710+0000", 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "107d5f4a-5704-4eb9-a25b-e2fdce4273a9": { 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:07:13.236292+0000", 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "13db7707-2d44-4d9b-bcfc-6a0f10383258": { 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:11:49.355834+0000", 2024-09-10T21:28:53.049 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "232d6926-c99d-4411-a1a9-c37d6c417870": { 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:09:33.175537+0000", 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.050 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "59291033-0fe1-4840-9451-8996afd161e3": { 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:25:44.117399+0000", 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "238d99b0-b1dc-4b1f-9885-f601eee0e81e": { 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:05:40.492456+0000", 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.051 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "f9774723-3a34-4b63-a0e3-38b64011cc24": { 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:51.935065+0000", 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.052 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "7e38eb25-b519-4c3e-a4b5-2dbcdd302f1e": { 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:06:26.328881+0000", 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "3d0a5da9-0921-4663-ac39-3add4adc1499": { 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:04:05.672645+0000", 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.053 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "7c5a98b8-eef5-45b6-8fa8-b9cd5976f591": { 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:01.026191+0000", 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.054 INFO:teuthology.orchestra.run.smithi033.stdout: "df90b8ee-36ca-4b4b-b236-67440e7f292e": { 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:17:11.704807+0000", 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "43b2cb50-7571-4216-b724-5dd1d60c9297": { 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:00.672694+0000", 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.055 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "6023db06-103a-4538-877a-f434e14bd948": { 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:19:43.903905+0000", 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.056 INFO:teuthology.orchestra.run.smithi033.stdout: "abb3152f-b4a8-4eb5-baa6-ac4d8cbb70b0": { 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:28:19.207486+0000", 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "bac5b008-4545-4044-8f53-cc00a8b4c23e": { 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:53.084538+0000", 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.057 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "e13c6b7a-1a1f-4f76-b41a-20e6bd77bd2e": { 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:15:39.541958+0000", 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "0d738eee-48a0-490a-83f5-59006db2e660": { 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:16:25.253387+0000", 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.058 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "d1ed5978-a0e0-405b-a593-858fe0b8c98d": { 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:18:54.312525+0000", 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.059 INFO:teuthology.orchestra.run.smithi033.stdout: "941658e5-2c5c-43b5-aa24-7d3c6632ea39": { 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:20:35.193264+0000", 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "1737cca1-c03b-4eef-8451-596ad722fe8c": { 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:21:27.091616+0000", 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.060 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "3d46f877-fb1e-4f1a-80f8-e6908aafdfbb": { 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:23:09.357337+0000", 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "4c915ed5-38f4-4ec0-94f7-4fe10ec6ac07": { 2024-09-10T21:28:53.061 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:24:51.166062+0000", 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "279c3483-bbc8-45ec-93b3-a95fbd081ae3": { 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:14:06.443121+0000", 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.062 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: "05708340-c3c3-4430-9bae-81feded8d778": { 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: "start_time": "2024-09-10T21:26:35.545755+0000", 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: "path": "/", 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_passed": 0, 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_failed": 0, 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_skipped": 0 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:28:53.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7c700b52e0 msgr2=0x7f7c700b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7c700b52e0 0x7f7c700b77a0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c009c70 tx=0x7f7c7c005c50 comp rx=0 tx=0).stop 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c84103b90 msgr2=0x7f7c84199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c84103b90 0x7f7c84199050 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000e9f0 tx=0x7f7c8000edf0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 shutdown_connections 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7c700b52e0 0x7f7c700b77a0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7c8410d140 0x7f7c8419e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c84104590 0x7f7c84199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 --2- 172.21.15.33:0/1532574294 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c84103b90 0x7f7c84199050 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.039+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 >> 172.21.15.33:0/1532574294 conn(0x7f7c840fd700 msgr2=0x7f7c84104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.040+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 shutdown_connections 2024-09-10T21:28:53.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:53.040+0000 7f7c8b294640 1 -- 172.21.15.33:0/1532574294 wait complete. 2024-09-10T21:28:53.065 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:52 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T21:28:53.092 INFO:tasks.workunit.client.0.smithi033.stdout: 10 510905 37.26 MB/sec execute 593 sec latency 231.034 ms 2024-09-10T21:28:53.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:53 smithi174 ceph-mon[35635]: pgmap v1093: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-09-10T21:28:53.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:28:53.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:28:53.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:53 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:28:53.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:53 smithi033 ceph-mon[30860]: pgmap v1093: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-09-10T21:28:53.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:28:53.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:28:53.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:53 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:28:53.994 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:28:53.995 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:28:54.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:53 smithi104 ceph-mon[35325]: pgmap v1093: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.8 MiB/s wr, 539 op/s 2024-09-10T21:28:54.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T21:28:54.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T21:28:54.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:53 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' 2024-09-10T21:28:54.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 511674 37.26 MB/sec execute 594 sec latency 102.491 ms 2024-09-10T21:28:54.322 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.849+0000 7fc77baf2640 1 -- 172.21.15.33:0/2331355800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77410d1a0 msgr2=0x7fc77410f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:54.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.849+0000 7fc77baf2640 1 --2- 172.21.15.33:0/2331355800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77410d1a0 0x7fc77410f600 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc76c009ca0 tx=0x7fc76c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.849+0000 7fc77baf2640 1 -- 172.21.15.33:0/2331355800 shutdown_connections 2024-09-10T21:28:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.849+0000 7fc77baf2640 1 --2- 172.21.15.33:0/2331355800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77410d1a0 0x7fc77410f600 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.849+0000 7fc77baf2640 1 --2- 172.21.15.33:0/2331355800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc774104660 0x7fc77410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.849+0000 7fc77baf2640 1 --2- 172.21.15.33:0/2331355800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 0x7fc774104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.849+0000 7fc77baf2640 1 -- 172.21.15.33:0/2331355800 >> 172.21.15.33:0/2331355800 conn(0x7fc7740fd7e0 msgr2=0x7fc7740ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.850+0000 7fc77baf2640 1 -- 172.21.15.33:0/2331355800 shutdown_connections 2024-09-10T21:28:54.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.850+0000 7fc77baf2640 1 -- 172.21.15.33:0/2331355800 wait complete. 2024-09-10T21:28:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.851+0000 7fc77baf2640 1 Processor -- start 2024-09-10T21:28:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.851+0000 7fc77baf2640 1 -- start start 2024-09-10T21:28:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77baf2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 0x7fc7741992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:54.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77baf2640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc774104660 0x7fc7741997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77baf2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc77410d1a0 0x7fc77419eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77baf2640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fc774112e50 con 0x7fc774104660 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77baf2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fc774112cd0 con 0x7fc77410d1a0 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77baf2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc774112fd0 con 0x7fc774103d40 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77a08f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc77410d1a0 0x7fc77419eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77988e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 0x7fc7741992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77a08f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc77410d1a0 0x7fc77419eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38696/0 (socket says 172.21.15.33:38696) 2024-09-10T21:28:54.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77988e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 0x7fc7741992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:54754/0 (socket says 172.21.15.33:54754) 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77a08f640 1 -- 172.21.15.33:0/163594481 learned_addr learned my addr 172.21.15.33:0/163594481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77908d640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc774104660 0x7fc7741997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77988e640 1 -- 172.21.15.33:0/163594481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc77410d1a0 msgr2=0x7fc77419eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.852+0000 7fc77988e640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc77410d1a0 0x7fc77419eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc77988e640 1 -- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc774104660 msgr2=0x7fc7741997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc77988e640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc774104660 0x7fc7741997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc77988e640 1 -- 172.21.15.33:0/163594481 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc77419f350 con 0x7fc774103d40 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc77908d640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc774104660 0x7fc7741997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc77988e640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 0x7fc7741992a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b3f0 tx=0x7fc77000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc76affd640 1 -- 172.21.15.33:0/163594481 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc770015020 con 0x7fc774103d40 2024-09-10T21:28:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc77419f640 con 0x7fc774103d40 2024-09-10T21:28:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741a5ed0 con 0x7fc774103d40 2024-09-10T21:28:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.853+0000 7fc76affd640 1 -- 172.21.15.33:0/163594481 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc770011ce0 con 0x7fc774103d40 2024-09-10T21:28:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.854+0000 7fc76affd640 1 -- 172.21.15.33:0/163594481 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77001aca0 con 0x7fc774103d40 2024-09-10T21:28:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.855+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc774104120 con 0x7fc774103d40 2024-09-10T21:28:54.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.857+0000 7fc76affd640 1 -- 172.21.15.33:0/163594481 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fc770011830 con 0x7fc774103d40 2024-09-10T21:28:54.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.858+0000 7fc76affd640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7600b52e0 0x7fc7600b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:54.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.858+0000 7fc76affd640 1 -- 172.21.15.33:0/163594481 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fc7700e5ad0 con 0x7fc774103d40 2024-09-10T21:28:54.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.859+0000 7fc77908d640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7600b52e0 0x7fc7600b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:54.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.859+0000 7fc77908d640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7600b52e0 0x7fc7600b77a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc774100010 tx=0x7fc764005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:54.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:54.865+0000 7fc76affd640 1 -- 172.21.15.33:0/163594481 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fc77001f020 con 0x7fc774103d40 2024-09-10T21:28:55.090 INFO:tasks.workunit.client.0.smithi033.stdout: 10 512453 37.27 MB/sec execute 595 sec latency 80.018 ms 2024-09-10T21:28:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.206+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7741a60a0 con 0x7fc774103d40 2024-09-10T21:28:55.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.207+0000 7fc76affd640 1 -- 172.21.15.33:0/163594481 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 98 v98) ==== 94+0+11992 (secure 0 0 0) 0x7fc7700ac990 con 0x7fc774103d40 2024-09-10T21:28:55.209 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:28:55.209 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":98,"btime":"2024-09-10T21:28:36:619783+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":98,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:28:36.191330+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[1,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24526,24539,24532,24547,24566]},"id":1}]} 2024-09-10T21:28:55.209 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 98 2024-09-10T21:28:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7600b52e0 msgr2=0x7fc7600b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7600b52e0 0x7fc7600b77a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc774100010 tx=0x7fc764005f90 comp rx=0 tx=0).stop 2024-09-10T21:28:55.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 msgr2=0x7fc7741992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 0x7fc7741992a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc77000b3f0 tx=0x7fc77000b8b0 comp rx=0 tx=0).stop 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 shutdown_connections 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fc7600b52e0 0x7fc7600b77a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc77410d1a0 0x7fc77419eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc774104660 0x7fc7741997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 --2- 172.21.15.33:0/163594481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc774103d40 0x7fc7741992a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 >> 172.21.15.33:0/163594481 conn(0x7fc7740fd7e0 msgr2=0x7fc77410f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 shutdown_connections 2024-09-10T21:28:55.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:55.209+0000 7fc77baf2640 1 -- 172.21.15.33:0/163594481 wait complete. 2024-09-10T21:28:55.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:55 smithi174 ceph-mon[35635]: pgmap v1094: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.8 MiB/s wr, 677 op/s 2024-09-10T21:28:55.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:55 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/163594481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:28:55.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:55 smithi033 ceph-mon[30860]: pgmap v1094: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.8 MiB/s wr, 677 op/s 2024-09-10T21:28:55.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:55 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/163594481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:28:56.011 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-10T21:28:56.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:55 smithi104 ceph-mon[35325]: pgmap v1094: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 4.8 MiB/s wr, 677 op/s 2024-09-10T21:28:56.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:55 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/163594481' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:28:56.091 INFO:tasks.workunit.client.0.smithi033.stdout: 10 513255 37.29 MB/sec execute 596 sec latency 58.305 ms 2024-09-10T21:28:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:57.091 INFO:tasks.workunit.client.0.smithi033.stdout: 10 513898 37.30 MB/sec execute 597 sec latency 76.020 ms 2024-09-10T21:28:57.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.136+0000 7f926a37e640 1 -- 172.21.15.33:0/2066507381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926410b350 msgr2=0x7f9264111b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.136+0000 7f926a37e640 1 --2- 172.21.15.33:0/2066507381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926410b350 0x7f9264111b50 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f925c009be0 tx=0x7f925c02a6c0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.138+0000 7f926a37e640 1 -- 172.21.15.33:0/2066507381 shutdown_connections 2024-09-10T21:28:57.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.138+0000 7f926a37e640 1 --2- 172.21.15.33:0/2066507381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f926410b350 0x7f9264111b50 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.138+0000 7f926a37e640 1 --2- 172.21.15.33:0/2066507381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926410a9d0 0x7f926410ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.138+0000 7f926a37e640 1 --2- 172.21.15.33:0/2066507381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9264104fc0 0x7f92641053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.138+0000 7f926a37e640 1 -- 172.21.15.33:0/2066507381 >> 172.21.15.33:0/2066507381 conn(0x7f92641009f0 msgr2=0x7f9264102e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:57.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.139+0000 7f926a37e640 1 -- 172.21.15.33:0/2066507381 shutdown_connections 2024-09-10T21:28:57.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.139+0000 7f926a37e640 1 -- 172.21.15.33:0/2066507381 wait complete. 2024-09-10T21:28:57.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.140+0000 7f926a37e640 1 Processor -- start 2024-09-10T21:28:57.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.140+0000 7f926a37e640 1 -- start start 2024-09-10T21:28:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.140+0000 7f926a37e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9264104fc0 0x7f92641105c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9264104fc0 0x7f92641105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f926a37e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f926410a9d0 0x7f9264110b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9264104fc0 0x7f92641105c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42650/0 (socket says 172.21.15.33:42650) 2024-09-10T21:28:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 -- 172.21.15.33:0/2420774019 learned_addr learned my addr 172.21.15.33:0/2420774019 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f926a37e640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926410b350 0x7f92641ab370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9264078730 con 0x7f9264104fc0 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f92640785b0 con 0x7f926410a9d0 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92640788b0 con 0x7f926410b350 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f925b7fe640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f926410a9d0 0x7f9264110b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 -- 172.21.15.33:0/2420774019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926410b350 msgr2=0x7f92641ab370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926410b350 0x7f92641ab370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 -- 172.21.15.33:0/2420774019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f926410a9d0 msgr2=0x7f9264110b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f926410a9d0 0x7f9264110b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.141+0000 7f9263fff640 1 -- 172.21.15.33:0/2420774019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92641aba40 con 0x7f9264104fc0 2024-09-10T21:28:57.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.142+0000 7f9263fff640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9264104fc0 0x7f92641105c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9f0 tx=0x7f924c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:57.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.142+0000 7f9261ffb640 1 -- 172.21.15.33:0/2420774019 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c00cc40 con 0x7f9264104fc0 2024-09-10T21:28:57.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.142+0000 7f9261ffb640 1 -- 172.21.15.33:0/2420774019 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f924c00cde0 con 0x7f9264104fc0 2024-09-10T21:28:57.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.142+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92641abd30 con 0x7f9264104fc0 2024-09-10T21:28:57.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.143+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92641ac1b0 con 0x7f9264104fc0 2024-09-10T21:28:57.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.143+0000 7f9261ffb640 1 -- 172.21.15.33:0/2420774019 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0187e0 con 0x7f9264104fc0 2024-09-10T21:28:57.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.147+0000 7f9261ffb640 1 -- 172.21.15.33:0/2420774019 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f924c018980 con 0x7f9264104fc0 2024-09-10T21:28:57.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 Processor -- start 2024-09-10T21:28:57.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 -- start start 2024-09-10T21:28:57.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f9261ffb640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92440b54f0 0x7f92440b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x5589117568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911762840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x558911765170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5589116ccea0 con 0x5589117564d0 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5589116bc2d0 con 0x558911756df0 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.149+0000 7f926c753740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589116c00e0 con 0x558911762d80 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f9263fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x5589117568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f9263fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x5589117568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42664/0 (socket says 172.21.15.33:42664) 2024-09-10T21:28:57.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f9263fff640 1 -- 172.21.15.33:0/1236754974 learned_addr learned my addr 172.21.15.33:0/1236754974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f925b7fe640 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911762840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f926891b640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92440b54f0 0x7f92440b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f925b7fe640 1 -- 172.21.15.33:0/1236754974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 msgr2=0x558911765170 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f925b7fe640 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x558911765170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f925b7fe640 1 -- 172.21.15.33:0/1236754974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 msgr2=0x5589117568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f925b7fe640 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x5589117568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f925b7fe640 1 -- 172.21.15.33:0/1236754974 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55891141a650 con 0x558911756df0 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f9261ffb640 1 -- 172.21.15.33:0/2420774019 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f924c0e61f0 con 0x7f9264104fc0 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.150+0000 7f925b7fe640 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911762840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f926410a7e0 tx=0x7f9254011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ab11098d2c8a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.151+0000 7f925a7fc640 1 -- 172.21.15.33:0/1236754974 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254009db0 con 0x558911756df0 2024-09-10T21:28:57.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.151+0000 7f925a7fc640 1 -- 172.21.15.33:0/1236754974 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92540035f0 con 0x558911756df0 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.152+0000 7f926c753740 1 -- 172.21.15.33:0/1236754974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 msgr2=0x558911762840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.152+0000 7f926c753740 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911762840 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f926410a7e0 tx=0x7f9254011810 comp rx=0 tx=0).stop 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.152+0000 7f926c753740 1 -- 172.21.15.33:0/1236754974 shutdown_connections 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.152+0000 7f926c753740 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x558911765170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.152+0000 7f926c753740 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911762840 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.152+0000 7f926c753740 1 --2- 172.21.15.33:0/1236754974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x5589117568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.152+0000 7f926c753740 1 -- 172.21.15.33:0/1236754974 >> 172.21.15.33:0/1236754974 conn(0x558911754300 msgr2=0x5589117554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.153+0000 7f926891b640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92440b54f0 0x7f92440b79b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f925c005ee0 tx=0x7f925c009840 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.153+0000 7f926c753740 1 -- 172.21.15.33:0/1236754974 shutdown_connections 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.153+0000 7f926c753740 1 -- 172.21.15.33:0/1236754974 wait complete. 2024-09-10T21:28:57.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.154+0000 7f926c753740 1 -- start start 2024-09-10T21:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.154+0000 7f926c753740 1 Processor -- start 2024-09-10T21:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.155+0000 7f926c753740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589117564d0 0x5589117625c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.155+0000 7f926c753740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558911756df0 0x55891175fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.155+0000 7f9263fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589117564d0 0x5589117625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.155+0000 7f9263fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589117564d0 0x5589117625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38720/0 (socket says 172.21.15.33:38720) 2024-09-10T21:28:57.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.155+0000 7f9263fff640 1 -- 172.21.15.33:0/2826824408 learned_addr learned my addr 172.21.15.33:0/2826824408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.155+0000 7f925b7fe640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558911756df0 0x55891175fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.156+0000 7f926c753740 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x55891175ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.156+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55891151d700 con 0x558911756df0 2024-09-10T21:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.156+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x558911657160 con 0x5589117564d0 2024-09-10T21:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.156+0000 7f926891b640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x55891175ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.156+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558911765a50 con 0x558911762d80 2024-09-10T21:28:57.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.156+0000 7f926c753740 10 client.0 _ll_register_callbacks cb 0x5589116709e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:57.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f925b7fe640 1 -- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 msgr2=0x55891175ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:57.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f925b7fe640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x55891175ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f925b7fe640 1 -- 172.21.15.33:0/2826824408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589117564d0 msgr2=0x5589117625c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f925b7fe640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589117564d0 0x5589117625c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f925b7fe640 1 -- 172.21.15.33:0/2826824408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558911684c90 con 0x558911756df0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f9263fff640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589117564d0 0x5589117625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f925b7fe640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558911756df0 0x55891175fa10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f925401b040 tx=0x7f9254003ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af1d29608ab053e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f92537fe640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.157+0000 7f92537fe640 1 -- 172.21.15.33:0/2826824408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254027020 con 0x558911756df0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.158+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92540035f0 con 0x558911756df0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.158+0000 7f92537fe640 1 -- 172.21.15.33:0/2826824408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9254019ae0 con 0x558911756df0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.158+0000 7f92537fe640 1 -- 172.21.15.33:0/2826824408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f9254019ce0 con 0x558911756df0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.158+0000 7f926c753740 10 client.19761 fetch_fsmap learned FSMap version 98 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.158+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) -- 0x5589117713a0 con 0x558911756df0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.158+0000 7f92537fe640 1 -- 172.21.15.33:0/2826824408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92540035f0 con 0x558911756df0 2024-09-10T21:28:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f92537fe640 1 -- 172.21.15.33:0/2826824408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 98) ==== 8410+0+0 (secure 0 0 0) 0x7f9254022710 con 0x558911756df0 2024-09-10T21:28:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f926c753740 10 client.19761 fetch_fsmap finished waiting for FSMap version 98 2024-09-10T21:28:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f926c753740 10 client.19761 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:28:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f926c753740 20 client.19761 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f926c753740 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x558911772750 0x558911774b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f926c753740 4 client.19761 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:28:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589116cedf0 con 0x558911772750 2024-09-10T21:28:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.159+0000 7f9263fff640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x558911772750 0x558911774b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.160+0000 7f9263fff640 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x558911772750 0x558911774b40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c5740ae97540760 server_cookie=444c25c735590d2d in_seq=0 out_seq=0 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.161+0000 7f92537fe640 10 client.19761 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.161+0000 7f92537fe640 1 -- 172.21.15.33:0/2826824408 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5589116cedf0 con 0x558911772750 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.161+0000 7f92537fe640 10 client.19761 handle_command_reply: tid=0 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.161+0000 7f926c753740 1 client.19761 shutdown 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.161+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558911756df0 msgr2=0x55891175fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.161+0000 7f926c753740 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558911756df0 0x55891175fa10 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f925401b040 tx=0x7f9254003ca0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 shutdown_connections 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x558911772750 0x558911774b40 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x55891175ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x558911756df0 0x55891175fa10 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f92537fe640 0 client.19761 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 --2- 172.21.15.33:0/2826824408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5589117564d0 0x5589117625c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 >> 172.21.15.33:0/2826824408 conn(0x558911754300 msgr2=0x558911759e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 shutdown_connections 2024-09-10T21:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 1 -- 172.21.15.33:0/2826824408 wait complete. 2024-09-10T21:28:57.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.162+0000 7f926c753740 20 client.19761 trim_cache size 0 max 16384 2024-09-10T21:28:57.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.180+0000 7f926c753740 1 Processor -- start 2024-09-10T21:28:57.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.180+0000 7f926c753740 1 -- start start 2024-09-10T21:28:57.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926c753740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x55891175e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926c753740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911756df0 0x55891175e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926c753740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911762d80 0x558911755430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926c753740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f924800e5d0 con 0x5589117564d0 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926c753740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9248010290 con 0x558911762d80 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926c753740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f924800ed00 con 0x558911756df0 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926891b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911762d80 0x558911755430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926891b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911762d80 0x558911755430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:38736/0 (socket says 172.21.15.33:38736) 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.181+0000 7f926891b640 1 -- 172.21.15.33:0/2709145607 learned_addr learned my addr 172.21.15.33:0/2709145607 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f9263fff640 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x55891175e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f925b7fe640 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911756df0 0x55891175e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f926891b640 1 -- 172.21.15.33:0/2709145607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911756df0 msgr2=0x55891175e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:57.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f926891b640 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911756df0 0x55891175e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f926891b640 1 -- 172.21.15.33:0/2709145607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 msgr2=0x55891175e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f926891b640 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x55891175e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f926891b640 1 -- 172.21.15.33:0/2709145607 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589117602a0 con 0x558911762d80 2024-09-10T21:28:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f926891b640 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911762d80 0x558911755430 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f925c033000 tx=0x7f925c03a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64868f776a510cc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.182+0000 7f9259ffb640 1 -- 172.21.15.33:0/2709145607 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c043030 con 0x558911762d80 2024-09-10T21:28:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.183+0000 7f9259ffb640 1 -- 172.21.15.33:0/2709145607 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f925c04e070 con 0x558911762d80 2024-09-10T21:28:57.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.183+0000 7f9259ffb640 1 -- 172.21.15.33:0/2709145607 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c036040 con 0x558911762d80 2024-09-10T21:28:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 -- 172.21.15.33:0/2709145607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911762d80 msgr2=0x558911755430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911762d80 0x558911755430 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f925c033000 tx=0x7f925c03a000 comp rx=0 tx=0).stop 2024-09-10T21:28:57.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 -- 172.21.15.33:0/2709145607 shutdown_connections 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911762d80 0x558911755430 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911756df0 0x55891175e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 --2- 172.21.15.33:0/2709145607 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x55891175e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 -- 172.21.15.33:0/2709145607 >> 172.21.15.33:0/2709145607 conn(0x558911754300 msgr2=0x558911774760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 -- 172.21.15.33:0/2709145607 shutdown_connections 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.184+0000 7f926c753740 1 -- 172.21.15.33:0/2709145607 wait complete. 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.185+0000 7f926c753740 1 -- start start 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.185+0000 7f926c753740 1 Processor -- start 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926c753740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x558911766e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926c753740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911767340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926c753740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x558911767880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f9263fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x558911766e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f9263fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x558911766e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42686/0 (socket says 172.21.15.33:42686) 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f925b7fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911767340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f9263fff640 1 -- 172.21.15.33:0/408886432 learned_addr learned my addr 172.21.15.33:0/408886432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f924800fb30 con 0x5589117564d0 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f924800f430 con 0x558911756df0 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f924800c590 con 0x558911762d80 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926c753740 10 client.0 _ll_register_callbacks cb 0x7f924c0ac740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.186+0000 7f926891b640 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x558911767880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f925b7fe640 1 -- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 msgr2=0x558911767880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f925b7fe640 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x558911767880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f925b7fe640 1 -- 172.21.15.33:0/408886432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 msgr2=0x558911766e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f925b7fe640 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x558911766e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f925b7fe640 1 -- 172.21.15.33:0/408886432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589117de650 con 0x558911756df0 2024-09-10T21:28:57.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f925b7fe640 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911767340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9254022c80 tx=0x7f9254023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0b80079dd88fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f92537fe640 10 client.0 ms_handle_connect on v2:172.21.15.104:3300/0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.187+0000 7f92537fe640 1 -- 172.21.15.33:0/408886432 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9254027040 con 0x558911756df0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.188+0000 7f92537fe640 1 -- 172.21.15.33:0/408886432 <== mon.2 v2:172.21.15.104:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f92540050b0 con 0x558911756df0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.188+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f925c04e070 con 0x558911756df0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.188+0000 7f92537fe640 1 -- 172.21.15.33:0/408886432 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7f9254005b70 con 0x558911756df0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.188+0000 7f926c753740 10 client.29081 fetch_fsmap learned FSMap version 98 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.188+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) -- 0x5589117df930 con 0x558911756df0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.188+0000 7f92537fe640 1 -- 172.21.15.33:0/408886432 <== mon.2 v2:172.21.15.104:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f925c04e070 con 0x558911756df0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f92537fe640 1 -- 172.21.15.33:0/408886432 <== mon.2 v2:172.21.15.104:3300/0 5 ==== fsmap(e 98) ==== 8410+0+0 (secure 0 0 0) 0x7f9254035070 con 0x558911756df0 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f926c753740 10 client.29081 fetch_fsmap finished waiting for FSMap version 98 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f926c753740 10 client.29081 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f926c753740 20 client.29081 populate_metadata read hostname 'smithi033' 2024-09-10T21:28:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f926c753740 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5589117e00a0 0x5589117e24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f926c753740 4 client.29081 mds_command: new command op to 24526 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 --> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5589116ca8a0 con 0x5589117e00a0 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.189+0000 7f9263fff640 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5589117e00a0 0x5589117e24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.190+0000 7f9263fff640 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5589117e00a0 0x5589117e24c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2da9dff920b7db1d server_cookie=e189da2ebc82966c in_seq=0 out_seq=0 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.190+0000 7f92537fe640 10 client.29081 ms_handle_connect on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.190+0000 7f92537fe640 1 -- 172.21.15.33:0/408886432 <== mds.4 v2:172.21.15.174:6834/982304250 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5589116ca8a0 con 0x5589117e00a0 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.190+0000 7f92537fe640 10 client.29081 handle_command_reply: tid=0 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.190+0000 7f926c753740 1 client.29081 shutdown 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.190+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 msgr2=0x558911767340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.190+0000 7f926c753740 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911767340 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9254022c80 tx=0x7f9254023000 comp rx=0 tx=0).stop 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 shutdown_connections 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:6834/982304250,v1:172.21.15.174:6835/982304250] conn(0x5589117e00a0 0x5589117e24c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f92537fe640 0 client.29081 ms_handle_reset on v2:172.21.15.174:6834/982304250 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558911762d80 0x558911767880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x558911756df0 0x558911767340 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 --2- 172.21.15.33:0/408886432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5589117564d0 0x558911766e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 >> 172.21.15.33:0/408886432 conn(0x558911754300 msgr2=0x55891175e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 shutdown_connections 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 1 -- 172.21.15.33:0/408886432 wait complete. 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.191+0000 7f926c753740 20 client.29081 trim_cache size 0 max 16384 2024-09-10T21:28:57.198 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:28:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.192+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92440b54f0 msgr2=0x7f92440b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.192+0000 7f926a37e640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92440b54f0 0x7f92440b79b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f925c005ee0 tx=0x7f925c009840 comp rx=0 tx=0).stop 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.192+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9264104fc0 msgr2=0x7f92641105c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.192+0000 7f926a37e640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9264104fc0 0x7f92641105c0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e9f0 tx=0x7f924c00edf0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 shutdown_connections 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f92440b54f0 0x7f92440b79b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f926410b350 0x7f92641ab370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f926410a9d0 0x7f9264110b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 --2- 172.21.15.33:0/2420774019 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9264104fc0 0x7f92641105c0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 >> 172.21.15.33:0/2420774019 conn(0x7f92641009f0 msgr2=0x7f9264102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 shutdown_connections 2024-09-10T21:28:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:57.193+0000 7f926a37e640 1 -- 172.21.15.33:0/2420774019 wait complete. 2024-09-10T21:28:57.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:57 smithi174 ceph-mon[35635]: pgmap v1095: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 535 op/s 2024-09-10T21:28:57.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:57 smithi033 ceph-mon[30860]: pgmap v1095: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 535 op/s 2024-09-10T21:28:58.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:57 smithi104 ceph-mon[35325]: pgmap v1095: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.0 MiB/s wr, 535 op/s 2024-09-10T21:28:58.091 INFO:tasks.workunit.client.0.smithi033.stdout: 10 514491 37.28 MB/sec execute 598 sec latency 113.825 ms 2024-09-10T21:28:58.723 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-10T21:28:59.091 INFO:tasks.workunit.client.0.smithi033.stdout: 10 515278 37.27 MB/sec execute 599 sec latency 91.393 ms 2024-09-10T21:28:59.108 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:28:59.864 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:28:59 smithi033 ceph-mon[30860]: pgmap v1096: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-09-10T21:28:59.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:28:59 smithi174 ceph-mon[35635]: pgmap v1096: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-09-10T21:29:00.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:28:59 smithi104 ceph-mon[35325]: pgmap v1096: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.0 MiB/s wr, 531 op/s 2024-09-10T21:29:00.091 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 600 sec 2024-09-10T21:29:00.901 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:29:00.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.730+0000 7fe5a93a9640 1 -- 172.21.15.33:0/109991989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a411c0a0 msgr2=0x7fe5a411e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.730+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/109991989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a411c0a0 0x7fe5a411e490 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009ca0 tx=0x7fe59c02a950 comp rx=0 tx=0).stop 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.730+0000 7fe5a93a9640 1 -- 172.21.15.33:0/109991989 shutdown_connections 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.730+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/109991989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a411c0a0 0x7fe5a411e490 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.730+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/109991989 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a530 0x7fe5a410a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.730+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/109991989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5a410a150 0x7fe5a41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.730+0000 7fe5a93a9640 1 -- 172.21.15.33:0/109991989 >> 172.21.15.33:0/109991989 conn(0x7fe5a406dfc0 msgr2=0x7fe5a406e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.734+0000 7fe5a93a9640 1 -- 172.21.15.33:0/109991989 shutdown_connections 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.734+0000 7fe5a93a9640 1 -- 172.21.15.33:0/109991989 wait complete. 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 Processor -- start 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 -- start start 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a150 0x7fe5a411b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a410a530 0x7fe5a411b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5a411c0a0 0x7fe5a41157c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4112140 con 0x7fe5a410a530 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4111fc0 con 0x7fe5a410a150 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.735+0000 7fe5a93a9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41122c0 con 0x7fe5a411c0a0 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.736+0000 7fe5a27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a410a530 0x7fe5a411b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.736+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a150 0x7fe5a411b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.736+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5a411c0a0 0x7fe5a41157c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.736+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a150 0x7fe5a411b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51904/0 (socket says 172.21.15.33:51904) 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.736+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3289112228 learned_addr learned my addr 172.21.15.33:0/3289112228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.737+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3289112228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5a411c0a0 msgr2=0x7fe5a41157c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.737+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5a411c0a0 0x7fe5a41157c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.737+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a410a530 msgr2=0x7fe5a411b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.737+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a410a530 0x7fe5a411b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.737+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3289112228 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a4116080 con 0x7fe5a410a150 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.738+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a150 0x7fe5a411b4a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe59400e9f0 tx=0x7fe59400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.739+0000 7fe58bfff640 1 -- 172.21.15.33:0/3289112228 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59400cd80 con 0x7fe5a410a150 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.739+0000 7fe58bfff640 1 -- 172.21.15.33:0/3289112228 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940108e0 con 0x7fe5a410a150 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.739+0000 7fe58bfff640 1 -- 172.21.15.33:0/3289112228 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594018880 con 0x7fe5a410a150 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.742+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a4116370 con 0x7fe5a410a150 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.742+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a41bd1c0 con 0x7fe5a410a150 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.746+0000 7fe5ab77e740 1 Processor -- start 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.746+0000 7fe5ab77e740 1 -- start start 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448db4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f44851ea0 con 0x556f448dba10 2024-09-10T21:29:00.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f448412d0 con 0x556f448e79a0 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f448450e0 con 0x556f448db0f0 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55374/0 (socket says 172.21.15.33:55374) 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/1017705724 learned_addr learned my addr 172.21.15.33:0/1017705724 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5a27fc640 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.747+0000 7fe5a37fe640 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448db4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.748+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/1017705724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 msgr2=0x556f448db4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.748+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448db4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.748+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/1017705724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 msgr2=0x556f448e9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.748+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.748+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/1017705724 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f4459f650 con 0x556f448dba10 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.748+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e7460 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe5940095f0 tx=0x7fe594018df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3160816fa8222ead server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:00.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.751+0000 7fe58bfff640 1 -- 172.21.15.33:0/3289112228 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fe59404f9e0 con 0x7fe5a410a150 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.755+0000 7fe58bfff640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe58c0b54f0 0x7fe58c0b79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.756+0000 7fe58bfff640 1 -- 172.21.15.33:0/3289112228 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe5940e7760 con 0x7fe5a410a150 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.757+0000 7fe5897fa640 1 -- 172.21.15.33:0/1017705724 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940aa6d0 con 0x556f448dba10 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.757+0000 7fe5a37fe640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe58c0b54f0 0x7fe58c0b79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.757+0000 7fe5897fa640 1 -- 172.21.15.33:0/1017705724 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940adb90 con 0x556f448dba10 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.758+0000 7fe5ab77e740 1 -- 172.21.15.33:0/1017705724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 msgr2=0x556f448e7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.758+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e7460 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fe5940095f0 tx=0x7fe594018df0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.759+0000 7fe5a37fe640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe58c0b54f0 0x7fe58c0b79b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe59c006010 tx=0x7fe59c02f3e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.764+0000 7fe5ab77e740 1 -- 172.21.15.33:0/1017705724 shutdown_connections 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.764+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.764+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e7460 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.764+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/1017705724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448db4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.764+0000 7fe5ab77e740 1 -- 172.21.15.33:0/1017705724 >> 172.21.15.33:0/1017705724 conn(0x556f448d8f20 msgr2=0x556f448da0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.771+0000 7fe5ab77e740 1 -- 172.21.15.33:0/1017705724 shutdown_connections 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.773+0000 7fe5ab77e740 1 -- 172.21.15.33:0/1017705724 wait complete. 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.774+0000 7fe5ab77e740 1 -- start start 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.774+0000 7fe5ab77e740 1 Processor -- start 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.775+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.775+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.775+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.775+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x556f446a2700 con 0x556f448dba10 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.775+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x556f447dc160 con 0x556f448e79a0 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.775+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556f448ea620 con 0x556f448db0f0 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.775+0000 7fe5ab77e740 10 client.0 _ll_register_callbacks cb 0x556f447f59e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.777+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.778+0000 7fe5a37fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51926/0 (socket says 172.21.15.33:51926) 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.778+0000 7fe5a37fe640 1 -- 172.21.15.33:0/317951025 learned_addr learned my addr 172.21.15.33:0/317951025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.782+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.782+0000 7fe5a27fc640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.789+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/317951025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 msgr2=0x556f448e4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.789+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.789+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/317951025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 msgr2=0x556f448e4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.789+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.789+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/317951025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f44809c90 con 0x556f448db0f0 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.789+0000 7fe5a37fe640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.790+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e70f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe594000c00 tx=0x7fe594010cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6311ae1d2f36d93f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.790+0000 7fe586ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.790+0000 7fe586ff5640 1 -- 172.21.15.33:0/317951025 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594028040 con 0x556f448db0f0 2024-09-10T21:29:00.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.791+0000 7fe586ff5640 1 -- 172.21.15.33:0/317951025 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe5940ae4e0 con 0x556f448db0f0 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.792+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5940adb90 con 0x556f448db0f0 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.793+0000 7fe586ff5640 1 -- 172.21.15.33:0/317951025 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fe5940ad9d0 con 0x556f448db0f0 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.794+0000 7fe5ab77e740 10 client.28942 fetch_fsmap learned FSMap version 98 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.794+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) -- 0x556f448f5de0 con 0x556f448db0f0 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.794+0000 7fe586ff5640 1 -- 172.21.15.33:0/317951025 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940adb90 con 0x556f448db0f0 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.795+0000 7fe586ff5640 1 -- 172.21.15.33:0/317951025 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 98) ==== 8410+0+0 (secure 0 0 0) 0x7fe5940a9ad0 con 0x556f448db0f0 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.796+0000 7fe5ab77e740 10 client.28942 fetch_fsmap finished waiting for FSMap version 98 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.796+0000 7fe5ab77e740 10 client.28942 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.796+0000 7fe5ab77e740 20 client.28942 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.796+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f448f7130 0x556f448f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.796+0000 7fe5ab77e740 4 client.28942 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.796+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f44853df0 con 0x556f448f7130 2024-09-10T21:29:00.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.799+0000 7fe5a27fc640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f448f7130 0x556f448f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.806+0000 7fe5a27fc640 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f448f7130 0x556f448f9520 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5510b3512297a644 server_cookie=911d2592dca975e3 in_seq=0 out_seq=0 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.806+0000 7fe586ff5640 10 client.28942 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.811+0000 7fe586ff5640 1 -- 172.21.15.33:0/317951025 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556f44853df0 con 0x556f448f7130 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.812+0000 7fe586ff5640 10 client.28942 handle_command_reply: tid=0 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.813+0000 7fe5ab77e740 1 client.28942 shutdown 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.818+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 msgr2=0x556f448e70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.818+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e70f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe594000c00 tx=0x7fe594010cb0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.820+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 shutdown_connections 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.820+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448e4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.820+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448e4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.820+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f448f7130 0x556f448f9520 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.820+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/317951025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e70f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.820+0000 7fe586ff5640 0 client.28942 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.820+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 >> 172.21.15.33:0/317951025 conn(0x556f448d8f20 msgr2=0x556f448deac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.824+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 shutdown_connections 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.827+0000 7fe5ab77e740 1 -- 172.21.15.33:0/317951025 wait complete. 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.827+0000 7fe5ab77e740 20 client.28942 trim_cache size 0 max 16384 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.859+0000 7fe5ab77e740 1 Processor -- start 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.859+0000 7fe5ab77e740 1 -- start start 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448dba10 0x556f448e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448e79a0 0x556f448e1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41c59f0 con 0x556f448e79a0 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41c76b0 con 0x556f448dba10 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41c6120 con 0x556f448db0f0 2024-09-10T21:29:00.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:53684/0 (socket says 172.21.15.33:53684) 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/758676420 learned_addr learned my addr 172.21.15.33:0/758676420 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.860+0000 7fe5a37fe640 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448e79a0 0x556f448e1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.861+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/758676420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448dba10 msgr2=0x556f448e1750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.861+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448dba10 0x556f448e1750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.861+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/758676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448e79a0 msgr2=0x556f448e1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.861+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448e79a0 0x556f448e1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.861+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/758676420 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f448d8d50 con 0x556f448db0f0 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.861+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e4a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe5940ae010 tx=0x7fe5940ae200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=508507d95979ea90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.867+0000 7fe588ff9640 1 -- 172.21.15.33:0/758676420 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5940c7070 con 0x556f448db0f0 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.868+0000 7fe588ff9640 1 -- 172.21.15.33:0/758676420 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940b5e50 con 0x556f448db0f0 2024-09-10T21:29:00.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.868+0000 7fe588ff9640 1 -- 172.21.15.33:0/758676420 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594028070 con 0x556f448db0f0 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.872+0000 7fe5ab77e740 1 -- 172.21.15.33:0/758676420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 msgr2=0x556f448e4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.872+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e4a00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe5940ae010 tx=0x7fe5940ae200 comp rx=0 tx=0).stop 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.875+0000 7fe5ab77e740 1 -- 172.21.15.33:0/758676420 shutdown_connections 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.875+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448e79a0 0x556f448e1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.875+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448dba10 0x556f448e1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.875+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/758676420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448e4a00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.875+0000 7fe5ab77e740 1 -- 172.21.15.33:0/758676420 >> 172.21.15.33:0/758676420 conn(0x556f448d8f20 msgr2=0x556f448e42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.876+0000 7fe5ab77e740 1 -- 172.21.15.33:0/758676420 shutdown_connections 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.876+0000 7fe5ab77e740 1 -- 172.21.15.33:0/758676420 wait complete. 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.876+0000 7fe5ab77e740 1 -- start start 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.876+0000 7fe5ab77e740 1 Processor -- start 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448eba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5ab77e740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448ec500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41c6f50 con 0x556f448dba10 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41c6850 con 0x556f448e79a0 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5ab77e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41c39b0 con 0x556f448db0f0 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5ab77e740 10 client.0 _ll_register_callbacks cb 0x7fe5940aa6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448eba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.877+0000 7fe5a2ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448eba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:53700/0 (socket says 172.21.15.33:53700) 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.878+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3096230152 learned_addr learned my addr 172.21.15.33:0/3096230152 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.878+0000 7fe5a37fe640 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448ec500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.881+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3096230152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 msgr2=0x556f448ec500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.881+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448ec500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.881+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3096230152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 msgr2=0x556f448ebfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.881+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.881+0000 7fe5a2ffd640 1 -- 172.21.15.33:0/3096230152 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f44963020 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.881+0000 7fe5a2ffd640 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448eba80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe5940e7040 tx=0x7fe594010b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4778d99f80899e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.881+0000 7fe586ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.884+0000 7fe586ff5640 1 -- 172.21.15.33:0/3096230152 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe594028070 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.884+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5940b5e50 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.884+0000 7fe586ff5640 1 -- 172.21.15.33:0/3096230152 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fe5940a99d0 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.888+0000 7fe586ff5640 1 -- 172.21.15.33:0/3096230152 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=98) ==== 24+0+0 (secure 0 0 0) 0x7fe5940c7070 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.889+0000 7fe5ab77e740 10 client.28948 fetch_fsmap learned FSMap version 98 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.889+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=98,monmap=4+}) -- 0x556f44963020 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.890+0000 7fe586ff5640 1 -- 172.21.15.33:0/3096230152 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5940b5e50 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.890+0000 7fe586ff5640 1 -- 172.21.15.33:0/3096230152 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 98) ==== 8410+0+0 (secure 0 0 0) 0x7fe5940c3b70 con 0x556f448db0f0 2024-09-10T21:29:00.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.891+0000 7fe5ab77e740 10 client.28948 fetch_fsmap finished waiting for FSMap version 98 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.891+0000 7fe5ab77e740 10 client.28948 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.891+0000 7fe5ab77e740 20 client.28948 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.891+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f44964600 0x556f44966a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.891+0000 7fe5ab77e740 4 client.28948 mds_command: new command op to 24532 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.891+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 --> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f4484f8a0 con 0x556f44964600 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.891+0000 7fe5a27fc640 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f44964600 0x556f44966a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5a27fc640 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f44964600 0x556f44966a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2ed204830144a4 server_cookie=933192606a3e13ec in_seq=0 out_seq=0 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe586ff5640 10 client.28948 ms_handle_connect on v2:172.21.15.174:6836/1592763439 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe586ff5640 1 -- 172.21.15.33:0/3096230152 <== mds.3 v2:172.21.15.174:6836/1592763439 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f4484f8a0 con 0x556f44964600 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe586ff5640 10 client.28948 handle_command_reply: tid=0 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 client.28948 shutdown 2024-09-10T21:29:00.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 msgr2=0x556f448eba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448eba80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe5940e7040 tx=0x7fe594010b90 comp rx=0 tx=0).stop 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 shutdown_connections 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x556f448e79a0 0x556f448ec500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x556f448dba10 0x556f448ebfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:6836/1592763439,v1:172.21.15.174:6837/1592763439] conn(0x556f44964600 0x556f44966a20 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 --2- 172.21.15.33:0/3096230152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556f448db0f0 0x556f448eba80 secure :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe5940e7040 tx=0x7fe594010b90 comp rx=0 tx=0).stop 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.893+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 >> 172.21.15.33:0/3096230152 conn(0x556f448d8f20 msgr2=0x556f448e5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.894+0000 7fe586ff5640 0 client.28948 ms_handle_reset on v2:172.21.15.174:6836/1592763439 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.894+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 shutdown_connections 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.895+0000 7fe5ab77e740 1 -- 172.21.15.33:0/3096230152 wait complete. 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.895+0000 7fe5ab77e740 20 client.28948 trim_cache size 0 max 16384 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe58c0b54f0 msgr2=0x7fe58c0b79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe58c0b54f0 0x7fe58c0b79b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe59c006010 tx=0x7fe59c02f3e0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a150 msgr2=0x7fe5a411b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a150 0x7fe5a411b4a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe59400e9f0 tx=0x7fe59400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 shutdown_connections 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fe58c0b54f0 0x7fe58c0b79b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe5a411c0a0 0x7fe5a41157c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe5a410a530 0x7fe5a411b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 --2- 172.21.15.33:0/3289112228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5a410a150 0x7fe5a411b4a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.901+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 >> 172.21.15.33:0/3289112228 conn(0x7fe5a406dfc0 msgr2=0x7fe5a4071730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.903+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 shutdown_connections 2024-09-10T21:29:00.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:28:59.903+0000 7fe5a93a9640 1 -- 172.21.15.33:0/3289112228 wait complete. 2024-09-10T21:29:01.091 INFO:tasks.workunit.client.0.smithi033.stdout: 10 cleanup 601 sec 2024-09-10T21:29:01.774 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-10T21:29:01.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:01 smithi174 ceph-mon[35635]: pgmap v1097: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 508 op/s 2024-09-10T21:29:01.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:01 smithi033 ceph-mon[30860]: pgmap v1097: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 508 op/s 2024-09-10T21:29:02.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:01 smithi104 ceph-mon[35325]: pgmap v1097: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.2 MiB/s wr, 508 op/s 2024-09-10T21:29:02.094 INFO:tasks.workunit.client.0.smithi033.stdout: 2 cleanup 602 sec 2024-09-10T21:29:02.187 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:02.302 INFO:tasks.workunit.client.0.smithi033.stdout: 0 cleanup 602 sec 2024-09-10T21:29:02.302 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-09-10T21:29:02.302 INFO:tasks.workunit.client.0.smithi033.stdout: Operation Count AvgLat MaxLat 2024-09-10T21:29:02.302 INFO:tasks.workunit.client.0.smithi033.stdout: ---------------------------------------- 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: NTCreateX 713461 3.423 1617.413 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Close 523966 0.003 0.314 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Rename 30210 15.878 994.494 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Unlink 144189 3.351 318.378 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Deltree 14 143.888 275.652 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Mkdir 7 0.007 0.008 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Qpathinfo 646759 2.800 1381.378 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Qfileinfo 113169 0.003 124.591 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Qfsinfo 118581 0.284 231.024 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Sfileinfo 58131 0.102 377.757 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: Find 249978 0.221 384.884 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: WriteX 354914 0.077 1009.098 2024-09-10T21:29:02.303 INFO:tasks.workunit.client.0.smithi033.stdout: ReadX 1118060 0.035 983.958 2024-09-10T21:29:02.304 INFO:tasks.workunit.client.0.smithi033.stdout: LockX 2322 1.982 67.100 2024-09-10T21:29:02.304 INFO:tasks.workunit.client.0.smithi033.stdout: UnlockX 2322 1.648 105.001 2024-09-10T21:29:02.304 INFO:tasks.workunit.client.0.smithi033.stdout: Flush 50014 12.194 1355.240 2024-09-10T21:29:02.304 INFO:tasks.workunit.client.0.smithi033.stdout: 2024-09-10T21:29:02.304 INFO:tasks.workunit.client.0.smithi033.stdout:Throughput 37.2736 MB/sec 10 clients 10 procs max_latency=1617.422 ms 2024-09-10T21:29:02.304 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T21:29:02.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T21:29:02.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.806+0000 7f79c717c640 1 -- 172.21.15.33:0/1478780363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c011c0a0 msgr2=0x7f79c011e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.806+0000 7f79c717c640 1 --2- 172.21.15.33:0/1478780363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c011c0a0 0x7f79c011e490 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f79b401a910 tx=0x7f79b403b4c0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.807+0000 7f79c717c640 1 -- 172.21.15.33:0/1478780363 shutdown_connections 2024-09-10T21:29:02.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.807+0000 7f79c717c640 1 --2- 172.21.15.33:0/1478780363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c011c0a0 0x7f79c011e490 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.807+0000 7f79c717c640 1 --2- 172.21.15.33:0/1478780363 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79c010a530 0x7f79c010a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.807+0000 7f79c717c640 1 --2- 172.21.15.33:0/1478780363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a150 0x7f79c01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.807+0000 7f79c717c640 1 -- 172.21.15.33:0/1478780363 >> 172.21.15.33:0/1478780363 conn(0x7f79c006dfc0 msgr2=0x7f79c006e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:02.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.810+0000 7f79c717c640 1 -- 172.21.15.33:0/1478780363 shutdown_connections 2024-09-10T21:29:02.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.810+0000 7f79c717c640 1 -- 172.21.15.33:0/1478780363 wait complete. 2024-09-10T21:29:02.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.811+0000 7f79c717c640 1 Processor -- start 2024-09-10T21:29:02.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.811+0000 7f79c717c640 1 -- start start 2024-09-10T21:29:02.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.811+0000 7f79c717c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c010a150 0x7f79c0117870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.812+0000 7f79c717c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a530 0x7f79c0117db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.812+0000 7f79c4f18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c010a150 0x7f79c0117870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.812+0000 7f79c4f18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c010a150 0x7f79c0117870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55424/0 (socket says 172.21.15.33:55424) 2024-09-10T21:29:02.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.812+0000 7f79bffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a530 0x7f79c0117db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.812+0000 7f79bffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a530 0x7f79c0117db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:53730/0 (socket says 172.21.15.33:53730) 2024-09-10T21:29:02.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c717c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79c011c0a0 0x7f79c01a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c717c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79c01140c0 con 0x7f79c010a150 2024-09-10T21:29:02.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c717c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0113f40 con 0x7f79c011c0a0 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c717c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79c0114240 con 0x7f79c010a530 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c4f18640 1 -- 172.21.15.33:0/229074794 learned_addr learned my addr 172.21.15.33:0/229074794 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c4f18640 1 -- 172.21.15.33:0/229074794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a530 msgr2=0x7f79c0117db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c4f18640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a530 0x7f79c0117db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c4f18640 1 -- 172.21.15.33:0/229074794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79c011c0a0 msgr2=0x7f79c01a50b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c4f18640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79c011c0a0 0x7f79c01a50b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.814+0000 7f79c4f18640 1 -- 172.21.15.33:0/229074794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c01a5880 con 0x7f79c010a150 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.815+0000 7f79c4f18640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c010a150 0x7f79c0117870 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f79b800eab0 tx=0x7f79b800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.815+0000 7f79bdffb640 1 -- 172.21.15.33:0/229074794 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b800cdc0 con 0x7f79c010a150 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.815+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79c01a5b70 con 0x7f79c010a150 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.815+0000 7f79bffff640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a530 0x7f79c0117db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.816+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79c01ac210 con 0x7f79c010a150 2024-09-10T21:29:02.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.816+0000 7f79bdffb640 1 -- 172.21.15.33:0/229074794 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b8011920 con 0x7f79c010a150 2024-09-10T21:29:02.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.817+0000 7f79bdffb640 1 -- 172.21.15.33:0/229074794 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b8010970 con 0x7f79c010a150 2024-09-10T21:29:02.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.820+0000 7f79c9551740 1 Processor -- start 2024-09-10T21:29:02.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.820+0000 7f79c9551740 1 -- start start 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.821+0000 7f79c9551740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dc0f0 0x5593cd5dc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.821+0000 7f79c9551740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5e8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.821+0000 7f79c9551740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5e89a0 0x5593cd5ead90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.821+0000 7f79c9551740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593cd552ea0 con 0x5593cd5e89a0 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.821+0000 7f79c9551740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5593cd5422d0 con 0x5593cd5dc0f0 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.821+0000 7f79c9551740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593cd5460e0 con 0x5593cd5dca10 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.821+0000 7f79c5719640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dc0f0 0x5593cd5dc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.822+0000 7f79bdffb640 1 -- 172.21.15.33:0/229074794 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f79b800f040 con 0x7f79c010a150 2024-09-10T21:29:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.823+0000 7f79bdffb640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f79a80b53c0 0x7f79a80b7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.824+0000 7f79bdffb640 1 -- 172.21.15.33:0/229074794 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f79b80e5de0 con 0x7f79c010a150 2024-09-10T21:29:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dc0f0 0x5593cd5dc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51950/0 (socket says 172.21.15.33:51950) 2024-09-10T21:29:02.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 -- 172.21.15.33:0/2188878225 learned_addr learned my addr 172.21.15.33:0/2188878225 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79bffff640 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5e89a0 0x5593cd5ead90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f79a80b53c0 0x7f79a80b7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 -- 172.21.15.33:0/2188878225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 msgr2=0x5593cd5e8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5e8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 -- 172.21.15.33:0/2188878225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5e89a0 msgr2=0x5593cd5ead90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5e89a0 0x5593cd5ead90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.825+0000 7f79c5719640 1 -- 172.21.15.33:0/2188878225 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593cd2a0650 con 0x5593cd5dc0f0 2024-09-10T21:29:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.826+0000 7f79c5719640 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dc0f0 0x5593cd5dc4d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f79b403ba30 tx=0x7f79b4005240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc13239448c06699 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:02.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.826+0000 7f79c5719640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f79a80b53c0 0x7f79a80b7880 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005c20 tx=0x7f79b4019000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:02.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.826+0000 7f79a6ffd640 1 -- 172.21.15.33:0/2188878225 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4046040 con 0x5593cd5dc0f0 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.826+0000 7f79a6ffd640 1 -- 172.21.15.33:0/2188878225 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4004890 con 0x5593cd5dc0f0 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.827+0000 7f79c9551740 1 -- 172.21.15.33:0/2188878225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dc0f0 msgr2=0x5593cd5dc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.827+0000 7f79c9551740 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dc0f0 0x5593cd5dc4d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f79b403ba30 tx=0x7f79b4005240 comp rx=0 tx=0).stop 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.827+0000 7f79c9551740 1 -- 172.21.15.33:0/2188878225 shutdown_connections 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.827+0000 7f79c9551740 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5e89a0 0x5593cd5ead90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.827+0000 7f79c9551740 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5e8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.827+0000 7f79c9551740 1 --2- 172.21.15.33:0/2188878225 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dc0f0 0x5593cd5dc4d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.827+0000 7f79c9551740 1 -- 172.21.15.33:0/2188878225 >> 172.21.15.33:0/2188878225 conn(0x5593cd5d9f20 msgr2=0x5593cd5db0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.830+0000 7f79c9551740 1 -- 172.21.15.33:0/2188878225 shutdown_connections 2024-09-10T21:29:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.831+0000 7f79c9551740 1 -- 172.21.15.33:0/2188878225 wait complete. 2024-09-10T21:29:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.831+0000 7f79c9551740 1 -- start start 2024-09-10T21:29:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.831+0000 7f79c9551740 1 Processor -- start 2024-09-10T21:29:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c9551740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c9551740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dca10 0x5593cd5e5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c9551740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5e89a0 0x5593cd5e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c9551740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x5593cd3a3700 con 0x5593cd5dc0f0 2024-09-10T21:29:02.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c9551740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x5593cd4dd160 con 0x5593cd5dca10 2024-09-10T21:29:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c9551740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593cd5eb600 con 0x5593cd5e89a0 2024-09-10T21:29:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c9551740 10 client.0 _ll_register_callbacks cb 0x5593cd4f69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c5719640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5e89a0 0x5593cd5e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c5719640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5e89a0 0x5593cd5e5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:53746/0 (socket says 172.21.15.33:53746) 2024-09-10T21:29:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.832+0000 7f79c5719640 1 -- 172.21.15.33:0/2437735160 learned_addr learned my addr 172.21.15.33:0/2437735160 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.833+0000 7f79c5719640 1 -- 172.21.15.33:0/2437735160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dca10 msgr2=0x5593cd5e5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.833+0000 7f79c5719640 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dca10 0x5593cd5e5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.833+0000 7f79c5719640 1 -- 172.21.15.33:0/2437735160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 msgr2=0x5593cd5e81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.833+0000 7f79c5719640 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.833+0000 7f79c5719640 1 -- 172.21.15.33:0/2437735160 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593cd50ac90 con 0x5593cd5e89a0 2024-09-10T21:29:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.833+0000 7f79c5719640 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5e89a0 0x5593cd5e5ba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f79b403ba00 tx=0x7f79b40045d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f98936ff58d27674 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.834+0000 7f799ffff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-09-10T21:29:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.835+0000 7f799ffff640 1 -- 172.21.15.33:0/2437735160 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4046070 con 0x5593cd5e89a0 2024-09-10T21:29:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.835+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4004890 con 0x5593cd5e89a0 2024-09-10T21:29:02.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.835+0000 7f799ffff640 1 -- 172.21.15.33:0/2437735160 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f79b4040510 con 0x5593cd5e89a0 2024-09-10T21:29:02.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.836+0000 7f799ffff640 1 -- 172.21.15.33:0/2437735160 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f79b4005d50 con 0x5593cd5e89a0 2024-09-10T21:29:02.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.838+0000 7f79c9551740 10 client.28954 fetch_fsmap learned FSMap version 99 2024-09-10T21:29:02.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.838+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=4+}) -- 0x5593cd5f6f70 con 0x5593cd5e89a0 2024-09-10T21:29:02.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.839+0000 7f799ffff640 1 -- 172.21.15.33:0/2437735160 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4004890 con 0x5593cd5e89a0 2024-09-10T21:29:02.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.840+0000 7f799ffff640 1 -- 172.21.15.33:0/2437735160 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 99) ==== 8414+0+0 (secure 0 0 0) 0x7f79b4058070 con 0x5593cd5e89a0 2024-09-10T21:29:02.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.840+0000 7f79c9551740 10 client.28954 fetch_fsmap finished waiting for FSMap version 99 2024-09-10T21:29:02.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.840+0000 7f79c9551740 10 client.28954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:29:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.840+0000 7f79c9551740 20 client.28954 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.840+0000 7f79c9551740 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd5f8600 0x5593cd5faa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.840+0000 7f79c9551740 4 client.28954 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:29:02.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.840+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593cd554df0 con 0x5593cd5f8600 2024-09-10T21:29:02.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.844+0000 7f79c4f18640 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd5f8600 0x5593cd5faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.854+0000 7f79c4f18640 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd5f8600 0x5593cd5faa80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7cb194faea9349b4 server_cookie=6e0f9cc7cdbf62f4 in_seq=0 out_seq=0 2024-09-10T21:29:02.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.855+0000 7f799ffff640 10 client.28954 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:29:02.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.856+0000 7f799ffff640 1 -- 172.21.15.33:0/2437735160 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5593cd554df0 con 0x5593cd5f8600 2024-09-10T21:29:02.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.856+0000 7f799ffff640 10 client.28954 handle_command_reply: tid=0 2024-09-10T21:29:02.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.856+0000 7f79c9551740 1 client.28954 shutdown 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.858+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5e89a0 msgr2=0x5593cd5e5ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.858+0000 7f79c9551740 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5e89a0 0x5593cd5e5ba0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f79b403ba00 tx=0x7f79b40045d0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.858+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 shutdown_connections 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.858+0000 7f79c9551740 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5e89a0 0x5593cd5e5ba0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.858+0000 7f79c9551740 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5dca10 0x5593cd5e5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.858+0000 7f79c9551740 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd5f8600 0x5593cd5faa80 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.859+0000 7f79c9551740 1 --2- 172.21.15.33:0/2437735160 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5e81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.859+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 >> 172.21.15.33:0/2437735160 conn(0x5593cd5d9f20 msgr2=0x5593cd5dfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:02.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.860+0000 7f799ffff640 0 client.28954 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:29:02.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.860+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 shutdown_connections 2024-09-10T21:29:02.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.863+0000 7f79c9551740 1 -- 172.21.15.33:0/2437735160 wait complete. 2024-09-10T21:29:02.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.864+0000 7f79c9551740 20 client.28954 trim_cache size 0 max 16384 2024-09-10T21:29:02.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.890+0000 7f79c9551740 1 Processor -- start 2024-09-10T21:29:02.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.890+0000 7f79c9551740 1 -- start start 2024-09-10T21:29:02.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.890+0000 7f79c9551740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.890+0000 7f79c9551740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5e2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.892 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.890+0000 7f79c9551740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79c9551740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac00e480 con 0x5593cd5dc0f0 2024-09-10T21:29:02.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79c5719640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79c5719640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5e2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:51968/0 (socket says 172.21.15.33:51968) 2024-09-10T21:29:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79c5719640 1 -- 172.21.15.33:0/1521972398 learned_addr learned my addr 172.21.15.33:0/1521972398 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79bffff640 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79c9551740 1 -- 172.21.15.33:0/1521972398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac010170 con 0x5593cd5e89a0 2024-09-10T21:29:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79c9551740 1 -- 172.21.15.33:0/1521972398 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac00ebb0 con 0x5593cd5dca10 2024-09-10T21:29:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.891+0000 7f79c4f18640 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.892+0000 7f79c5719640 1 -- 172.21.15.33:0/1521972398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 msgr2=0x5593cd5e2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.892+0000 7f79c5719640 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5e2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.892+0000 7f79c5719640 1 -- 172.21.15.33:0/1521972398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 msgr2=0x5593cd5d9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.892+0000 7f79c5719640 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5d9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.892+0000 7f79c5719640 1 -- 172.21.15.33:0/1521972398 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593cd5da9a0 con 0x5593cd5e89a0 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.892+0000 7f79c5719640 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5e2d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f79b403b500 tx=0x7f79b4003ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ce9edce70871c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.893+0000 7f79a67fc640 1 -- 172.21.15.33:0/1521972398 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4046030 con 0x5593cd5e89a0 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.893+0000 7f79a67fc640 1 -- 172.21.15.33:0/1521972398 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4067070 con 0x5593cd5e89a0 2024-09-10T21:29:02.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.893+0000 7f79a67fc640 1 -- 172.21.15.33:0/1521972398 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b4040770 con 0x5593cd5e89a0 2024-09-10T21:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.894+0000 7f79c9551740 1 -- 172.21.15.33:0/1521972398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 msgr2=0x5593cd5e2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.894+0000 7f79c9551740 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5e2d90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f79b403b500 tx=0x7f79b4003ca0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.894+0000 7f79c9551740 1 -- 172.21.15.33:0/1521972398 shutdown_connections 2024-09-10T21:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.894+0000 7f79c9551740 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5e2d90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.894+0000 7f79c9551740 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5e2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.894+0000 7f79c9551740 1 --2- 172.21.15.33:0/1521972398 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5d9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.894+0000 7f79c9551740 1 -- 172.21.15.33:0/1521972398 >> 172.21.15.33:0/1521972398 conn(0x5593cd5d9f20 msgr2=0x5593cd5e0870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:02.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.896+0000 7f79c9551740 1 -- 172.21.15.33:0/1521972398 shutdown_connections 2024-09-10T21:29:02.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.896+0000 7f79c9551740 1 -- 172.21.15.33:0/1521972398 wait complete. 2024-09-10T21:29:02.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 -- start start 2024-09-10T21:29:02.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 Processor -- start 2024-09-10T21:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5f0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5f12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5ec390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac00fa10 con 0x5593cd5dc0f0 2024-09-10T21:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac00f2e0 con 0x5593cd5e89a0 2024-09-10T21:29:02.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.897+0000 7f79c9551740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79ac00c510 con 0x5593cd5dca10 2024-09-10T21:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c9551740 10 client.0 _ll_register_callbacks cb 0x7f79b4005040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c5719640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5ec390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5f0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5f0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55466/0 (socket says 172.21.15.33:55466) 2024-09-10T21:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 -- 172.21.15.33:0/1254694888 learned_addr learned my addr 172.21.15.33:0/1254694888 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79bffff640 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5f12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 -- 172.21.15.33:0/1254694888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 msgr2=0x5593cd5f12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5f12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 -- 172.21.15.33:0/1254694888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 msgr2=0x5593cd5ec390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5ec390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.898+0000 7f79c4f18640 1 -- 172.21.15.33:0/1254694888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593cd5ecbc0 con 0x5593cd5dc0f0 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.899+0000 7f79c4f18640 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5f0d50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f79b80a8570 tx=0x7f79b80ac0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d01f8b824d910b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.899+0000 7f799ffff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.900+0000 7f799ffff640 1 -- 172.21.15.33:0/1254694888 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79b80b5860 con 0x5593cd5dc0f0 2024-09-10T21:29:02.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.900+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79b4067070 con 0x5593cd5dc0f0 2024-09-10T21:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.900+0000 7f799ffff640 1 -- 172.21.15.33:0/1254694888 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f79b8011470 con 0x5593cd5dc0f0 2024-09-10T21:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.900+0000 7f799ffff640 1 -- 172.21.15.33:0/1254694888 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7f79b80b1040 con 0x5593cd5dc0f0 2024-09-10T21:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.900+0000 7f79c9551740 10 client.19797 fetch_fsmap learned FSMap version 99 2024-09-10T21:29:02.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.901+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=4+}) -- 0x5593cd6651e0 con 0x5593cd5dc0f0 2024-09-10T21:29:02.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.901+0000 7f799ffff640 1 -- 172.21.15.33:0/1254694888 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79b4067070 con 0x5593cd5dc0f0 2024-09-10T21:29:02.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.901+0000 7f799ffff640 1 -- 172.21.15.33:0/1254694888 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 99) ==== 8414+0+0 (secure 0 0 0) 0x7f79b80d31b0 con 0x5593cd5dc0f0 2024-09-10T21:29:02.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.901+0000 7f79c9551740 10 client.19797 fetch_fsmap finished waiting for FSMap version 99 2024-09-10T21:29:02.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.902+0000 7f79c9551740 10 client.19797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-09-10T21:29:02.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.902+0000 7f79c9551740 20 client.19797 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.902+0000 7f79c9551740 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd665920 0x5593cd667d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.902+0000 7f79c9551740 4 client.19797 mds_command: new command op to 24539 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:29:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.902+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 --> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593cd5508a0 con 0x5593cd665920 2024-09-10T21:29:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.902+0000 7f79bffff640 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd665920 0x5593cd667d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.903+0000 7f79bffff640 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd665920 0x5593cd667d40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=307800e0eb24190e server_cookie=e07f1a29ef2484e8 in_seq=0 out_seq=0 2024-09-10T21:29:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.903+0000 7f799ffff640 10 client.19797 ms_handle_connect on v2:172.21.15.104:6836/2894143081 2024-09-10T21:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.903+0000 7f799ffff640 1 -- 172.21.15.33:0/1254694888 <== mds.2 v2:172.21.15.104:6836/2894143081 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593cd5508a0 con 0x5593cd665920 2024-09-10T21:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.903+0000 7f799ffff640 10 client.19797 handle_command_reply: tid=0 2024-09-10T21:29:02.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.903+0000 7f79c9551740 1 client.19797 shutdown 2024-09-10T21:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.905+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 msgr2=0x5593cd5f0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.905+0000 7f79c9551740 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5f0d50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f79b80a8570 tx=0x7f79b80ac0e0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 shutdown_connections 2024-09-10T21:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x5593cd5e89a0 0x5593cd5ec390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593cd5dca10 0x5593cd5f12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.104:6836/2894143081,v1:172.21.15.104:6837/2894143081] conn(0x5593cd665920 0x5593cd667d40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 --2- 172.21.15.33:0/1254694888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x5593cd5dc0f0 0x5593cd5f0d50 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f799ffff640 0 client.19797 ms_handle_reset on v2:172.21.15.104:6836/2894143081 2024-09-10T21:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 >> 172.21.15.33:0/1254694888 conn(0x5593cd5d9f20 msgr2=0x5593cd5e6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 shutdown_connections 2024-09-10T21:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.907+0000 7f79c9551740 1 -- 172.21.15.33:0/1254694888 wait complete. 2024-09-10T21:29:02.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.909+0000 7f79c9551740 20 client.19797 trim_cache size 0 max 16384 2024-09-10T21:29:02.911 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:29:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f79a80b53c0 msgr2=0x7f79a80b7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f79a80b53c0 0x7f79a80b7880 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f79b4005c20 tx=0x7f79b4019000 comp rx=0 tx=0).stop 2024-09-10T21:29:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c010a150 msgr2=0x7f79c0117870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c010a150 0x7f79c0117870 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f79b800eab0 tx=0x7f79b800ef70 comp rx=0 tx=0).stop 2024-09-10T21:29:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 shutdown_connections 2024-09-10T21:29:02.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f79a80b53c0 0x7f79a80b7880 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79c011c0a0 0x7f79c01a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79c010a530 0x7f79c0117db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 --2- 172.21.15.33:0/229074794 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f79c010a150 0x7f79c0117870 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 >> 172.21.15.33:0/229074794 conn(0x7f79c006dfc0 msgr2=0x7f79c0071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.910+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 shutdown_connections 2024-09-10T21:29:02.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:02.911+0000 7f79c717c640 1 -- 172.21.15.33:0/229074794 wait complete. 2024-09-10T21:29:03.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:03 smithi033 ceph-mon[30860]: pgmap v1098: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-09-10T21:29:03.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:03 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:03.742 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-10T21:29:03.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:03 smithi174 ceph-mon[35635]: pgmap v1098: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-09-10T21:29:03.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:03 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:04.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:03 smithi104 ceph-mon[35325]: pgmap v1098: 97 pgs: 97 active+clean; 1.2 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.3 MiB/s wr, 493 op/s 2024-09-10T21:29:04.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:03 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:04.124 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-10T21:29:04.124 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T21:29:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 -- 172.21.15.33:0/1054834508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 msgr2=0x7f1f3810f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/1054834508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 0x7f1f3810f560 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009ca0 tx=0x7f1f3002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 -- 172.21.15.33:0/1054834508 shutdown_connections 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/1054834508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 0x7f1f3810f560 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/1054834508 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 0x7f1f3810cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/1054834508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38103ca0 0x7f1f38104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 -- 172.21.15.33:0/1054834508 >> 172.21.15.33:0/1054834508 conn(0x7f1f380fd700 msgr2=0x7f1f380ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 -- 172.21.15.33:0/1054834508 shutdown_connections 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.594+0000 7f1f3e16f640 1 -- 172.21.15.33:0/1054834508 wait complete. 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.596+0000 7f1f3e16f640 1 Processor -- start 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.596+0000 7f1f3e16f640 1 -- start start 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f3e16f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38103ca0 0x7f1f38199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f3e16f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 0x7f1f38199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f3e16f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 0x7f1f3819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f3e16f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38112dc0 con 0x7f1f3810d100 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f3e16f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38112c40 con 0x7f1f381045c0 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f3e16f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f38112f40 con 0x7f1f38103ca0 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38103ca0 0x7f1f38199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f36ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 0x7f1f38199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f36ffd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 0x7f1f38199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52006/0 (socket says 172.21.15.33:52006) 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38103ca0 0x7f1f38199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:53772/0 (socket says 172.21.15.33:53772) 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f36ffd640 1 -- 172.21.15.33:0/4274656617 learned_addr learned my addr 172.21.15.33:0/4274656617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.597+0000 7f1f37fff640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 0x7f1f3819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f36ffd640 1 -- 172.21.15.33:0/4274656617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38103ca0 msgr2=0x7f1f38199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38103ca0 0x7f1f38199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f36ffd640 1 -- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 msgr2=0x7f1f3819eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 0x7f1f3819eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f36ffd640 1 -- 172.21.15.33:0/4274656617 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3819f300 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 0x7f1f38199790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009d00 tx=0x7f1f2c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f34ff9640 1 -- 172.21.15.33:0/4274656617 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c004220 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f34ff9640 1 -- 172.21.15.33:0/4274656617 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f2c0043c0 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3819f5f0 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f34ff9640 1 -- 172.21.15.33:0/4274656617 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c011db0 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.598+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f38101a00 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.601+0000 7f1f34ff9640 1 -- 172.21.15.33:0/4274656617 <== mon.2 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f1f2c012050 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.602+0000 7f1f34ff9640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f200b5360 0x7f1f200b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.602+0000 7f1f34ff9640 1 -- 172.21.15.33:0/4274656617 <== mon.2 v2:172.21.15.104:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1f2c0e9d50 con 0x7f1f381045c0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.602+0000 7f1f40544740 1 Processor -- start 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.602+0000 7f1f377fe640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f200b5360 0x7f1f200b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.602+0000 7f1f40544740 1 -- start start 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f40544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71fef24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f40544740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff06c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f40544740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff095a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f40544740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c71fe68ea0 con 0x55c71fef2a10 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f40544740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c71fe582d0 con 0x55c71ff071b0 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f40544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c71fe5c0e0 con 0x55c71fef20f0 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71fef24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71fef24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.33:53782/0 (socket says 172.21.15.33:53782) 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f377fe640 1 -- 172.21.15.33:0/2280663184 learned_addr learned my addr 172.21.15.33:0/2280663184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff06c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f37fff640 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff095a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f377fe640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f200b5360 0x7f1f200b7820 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009b60 tx=0x7f1f28005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:04.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f37fff640 1 -- 172.21.15.33:0/2280663184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 msgr2=0x55c71fef24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.603+0000 7f1f37fff640 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71fef24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f37fff640 1 -- 172.21.15.33:0/2280663184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 msgr2=0x55c71ff06c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f37fff640 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff06c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f37fff640 1 -- 172.21.15.33:0/2280663184 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71fbb6650 con 0x55c71ff071b0 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff06c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f37fff640 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff095a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1f3002ade0 tx=0x7f1f30009870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9035cd66b7ab3298 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f1dffb640 1 -- 172.21.15.33:0/2280663184 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30031cf0 con 0x55c71ff071b0 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f1dffb640 1 -- 172.21.15.33:0/2280663184 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f30036040 con 0x55c71ff071b0 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.604+0000 7f1f1dffb640 1 -- 172.21.15.33:0/2280663184 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30004240 con 0x55c71ff071b0 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.605+0000 7f1f40544740 1 -- 172.21.15.33:0/2280663184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 msgr2=0x55c71ff095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.605+0000 7f1f40544740 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff095a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1f3002ade0 tx=0x7f1f30009870 comp rx=0 tx=0).stop 2024-09-10T21:29:04.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 -- 172.21.15.33:0/2280663184 shutdown_connections 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff095a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff06c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 --2- 172.21.15.33:0/2280663184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71fef24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 -- 172.21.15.33:0/2280663184 >> 172.21.15.33:0/2280663184 conn(0x55c71feeff20 msgr2=0x55c71fef10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 -- 172.21.15.33:0/2280663184 shutdown_connections 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 -- 172.21.15.33:0/2280663184 wait complete. 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.606+0000 7f1f40544740 1 -- start start 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 1 Processor -- start 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef20f0 0x55c71ff06ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef2a10 0x55c71ff03f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef20f0 0x55c71ff06ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef20f0 0x55c71ff06ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55502/0 (socket says 172.21.15.33:55502) 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f377fe640 1 -- 172.21.15.33:0/877498608 learned_addr learned my addr 172.21.15.33:0/877498608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff04510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55c71fcb9700 con 0x55c71fef20f0 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55c71fdf3160 con 0x55c71ff071b0 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c71ff09e00 con 0x55c71fef2a10 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef2a10 0x55c71ff03f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f37fff640 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff04510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.607+0000 7f1f40544740 10 client.0 _ll_register_callbacks cb 0x55c71fe0c9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f377fe640 1 -- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef2a10 msgr2=0x55c71ff03f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f377fe640 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef2a10 0x55c71ff03f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f377fe640 1 -- 172.21.15.33:0/877498608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 msgr2=0x55c71ff04510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f377fe640 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff04510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f377fe640 1 -- 172.21.15.33:0/877498608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71fe20c90 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f377fe640 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef20f0 0x55c71ff06ab0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1f28011c10 tx=0x7f1f28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666348c266f1caf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.608+0000 7f1f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.609+0000 7f1f16ffd640 1 -- 172.21.15.33:0/877498608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2801fc00 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.609+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f30036040 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.609+0000 7f1f16ffd640 1 -- 172.21.15.33:0/877498608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1f2801fda0 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.609+0000 7f1f16ffd640 1 -- 172.21.15.33:0/877498608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f1f28020430 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.609+0000 7f1f40544740 10 client.19809 fetch_fsmap learned FSMap version 100 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.609+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) -- 0x55c71ff157c0 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f16ffd640 1 -- 172.21.15.33:0/877498608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f30036040 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f16ffd640 1 -- 172.21.15.33:0/877498608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 100) ==== 8426+0+0 (secure 0 0 0) 0x7f1f28032610 con 0x55c71fef20f0 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f40544740 10 client.19809 fetch_fsmap finished waiting for FSMap version 100 2024-09-10T21:29:04.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f40544740 10 client.19809 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f40544740 20 client.19809 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f40544740 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff191c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f40544740 4 client.19809 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c71fe6adf0 con 0x55c71ff16d40 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.610+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff191c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.611+0000 7f1f16ffd640 10 client.19809 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.611+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff191c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d327b4c3dafdf5bd server_cookie=46005e2aec4c87d5 in_seq=0 out_seq=0 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.611+0000 7f1f16ffd640 1 -- 172.21.15.33:0/877498608 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c71fe6adf0 con 0x55c71ff16d40 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.611+0000 7f1f16ffd640 10 client.19809 handle_command_reply: tid=0 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.611+0000 7f1f40544740 1 client.19809 shutdown 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef20f0 msgr2=0x55c71ff06ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef20f0 0x55c71ff06ab0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1f28011c10 tx=0x7f1f28004730 comp rx=0 tx=0).stop 2024-09-10T21:29:04.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 shutdown_connections 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff04510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef2a10 0x55c71ff03f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff191c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f16ffd640 0 client.19809 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 --2- 172.21.15.33:0/877498608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef20f0 0x55c71ff06ab0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 >> 172.21.15.33:0/877498608 conn(0x55c71feeff20 msgr2=0x55c71fef5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 shutdown_connections 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 1 -- 172.21.15.33:0/877498608 wait complete. 2024-09-10T21:29:04.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.612+0000 7f1f40544740 20 client.19809 trim_cache size 0 max 16384 2024-09-10T21:29:04.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 Processor -- start 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 -- start start 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71ff13030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff02780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff02cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381ac2c0 con 0x55c71fef2a10 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381ae040 con 0x55c71ff071b0 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.635+0000 7f1f40544740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381aca20 con 0x55c71fef20f0 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff02cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff02cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52032/0 (socket says 172.21.15.33:52032) 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 -- 172.21.15.33:0/4010382638 learned_addr learned my addr 172.21.15.33:0/4010382638 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 -- 172.21.15.33:0/4010382638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 msgr2=0x55c71ff13030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 --2- 172.21.15.33:0/4010382638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71ff13030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 -- 172.21.15.33:0/4010382638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 msgr2=0x55c71ff02780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 --2- 172.21.15.33:0/4010382638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff02780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.636+0000 7f1f37fff640 1 -- 172.21.15.33:0/4010382638 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c71ff066e0 con 0x55c71ff071b0 2024-09-10T21:29:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.637+0000 7f1f37fff640 1 --2- 172.21.15.33:0/4010382638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff02cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1f3002ade0 tx=0x7f1f300099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba52f4b7cf37b180 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.637+0000 7f1f1d7fa640 1 -- 172.21.15.33:0/4010382638 <== mon.2 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3004a040 con 0x55c71ff071b0 2024-09-10T21:29:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.637+0000 7f1f1d7fa640 1 -- 172.21.15.33:0/4010382638 <== mon.2 v2:172.21.15.104:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f30036040 con 0x55c71ff071b0 2024-09-10T21:29:04.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.638+0000 7f1f1d7fa640 1 -- 172.21.15.33:0/4010382638 <== mon.2 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f30004030 con 0x55c71ff071b0 2024-09-10T21:29:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.639+0000 7f1f40544740 1 -- 172.21.15.33:0/4010382638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 msgr2=0x55c71ff02cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.639+0000 7f1f40544740 1 --2- 172.21.15.33:0/4010382638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff02cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1f3002ade0 tx=0x7f1f300099e0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.640+0000 7f1f40544740 1 -- 172.21.15.33:0/4010382638 shutdown_connections 2024-09-10T21:29:04.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.640+0000 7f1f40544740 1 --2- 172.21.15.33:0/4010382638 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71ff071b0 0x55c71ff02cc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.640+0000 7f1f40544740 1 --2- 172.21.15.33:0/4010382638 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff02780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.640+0000 7f1f40544740 1 --2- 172.21.15.33:0/4010382638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71fef20f0 0x55c71ff13030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.640+0000 7f1f40544740 1 -- 172.21.15.33:0/4010382638 >> 172.21.15.33:0/4010382638 conn(0x55c71feeff20 msgr2=0x55c71ff08000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.641+0000 7f1f40544740 1 -- 172.21.15.33:0/4010382638 shutdown_connections 2024-09-10T21:29:04.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.641+0000 7f1f40544740 1 -- 172.21.15.33:0/4010382638 wait complete. 2024-09-10T21:29:04.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.641+0000 7f1f40544740 1 -- start start 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.641+0000 7f1f40544740 1 Processor -- start 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f40544740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71fef20f0 0x55c71ff0f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f40544740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff0f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71fef20f0 0x55c71ff0f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f377fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71fef20f0 0x55c71ff0f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.33:52040/0 (socket says 172.21.15.33:52040) 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f377fe640 1 -- 172.21.15.33:0/2194986096 learned_addr learned my addr 172.21.15.33:0/2194986096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f40544740 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71ff071b0 0x55c71ff0a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff0f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.642+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381ad8e0 con 0x55c71fef2a10 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381ad1b0 con 0x55c71fef20f0 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f381aa280 con 0x55c71ff071b0 2024-09-10T21:29:04.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f37fff640 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71ff071b0 0x55c71ff0a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f40544740 10 client.0 _ll_register_callbacks cb 0x7f1f28011ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f36ffd640 1 -- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71ff071b0 msgr2=0x55c71ff0a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71ff071b0 0x55c71ff0a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f36ffd640 1 -- 172.21.15.33:0/2194986096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71fef20f0 msgr2=0x55c71ff0f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71fef20f0 0x55c71ff0f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f36ffd640 1 -- 172.21.15.33:0/2194986096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c71ff0a590 con 0x55c71fef2a10 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f36ffd640 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff0f5e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c012850 tx=0x7f1f2c011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dea6559379e891f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:04.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.643+0000 7f1f16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:29:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.644+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f30036040 con 0x55c71fef2a10 2024-09-10T21:29:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.644+0000 7f1f16ffd640 1 -- 172.21.15.33:0/2194986096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0b9aa0 con 0x55c71fef2a10 2024-09-10T21:29:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.644+0000 7f1f16ffd640 1 -- 172.21.15.33:0/2194986096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f1f2c0ac780 con 0x55c71fef2a10 2024-09-10T21:29:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.644+0000 7f1f16ffd640 1 -- 172.21.15.33:0/2194986096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f1f2c0b9610 con 0x55c71fef2a10 2024-09-10T21:29:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.644+0000 7f1f40544740 10 client.19812 fetch_fsmap learned FSMap version 100 2024-09-10T21:29:04.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.644+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({fsmap=100}) -- 0x55c71ff84260 con 0x55c71fef2a10 2024-09-10T21:29:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f16ffd640 1 -- 172.21.15.33:0/2194986096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== fsmap(e 100) ==== 8426+0+0 (secure 0 0 0) 0x7f1f2c0d71e0 con 0x55c71fef2a10 2024-09-10T21:29:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f40544740 10 client.19812 fetch_fsmap finished waiting for FSMap version 100 2024-09-10T21:29:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f40544740 10 client.19812 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-09-10T21:29:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f40544740 20 client.19812 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f40544740 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff86970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:04.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f40544740 4 client.19812 mds_command: new command op to 24547 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:29:04.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 --> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1f30036040 con 0x55c71ff16d40 2024-09-10T21:29:04.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.649+0000 7f1f377fe640 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff86970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.651+0000 7f1f377fe640 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff86970 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ea47b287e918f5c3 server_cookie=7ca5c9d15e345329 in_seq=0 out_seq=0 2024-09-10T21:29:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.651+0000 7f1f16ffd640 10 client.19812 ms_handle_connect on v2:172.21.15.174:6838/3620064873 2024-09-10T21:29:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.651+0000 7f1f16ffd640 1 -- 172.21.15.33:0/2194986096 <== mds.1 v2:172.21.15.174:6838/3620064873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1f30036040 con 0x55c71ff16d40 2024-09-10T21:29:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.651+0000 7f1f16ffd640 10 client.19812 handle_command_reply: tid=0 2024-09-10T21:29:04.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.651+0000 7f1f40544740 1 client.19812 shutdown 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 msgr2=0x55c71ff0f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff0f5e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c012850 tx=0x7f1f2c011850 comp rx=0 tx=0).stop 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 shutdown_connections 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c71ff071b0 0x55c71ff0a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.174:6838/3620064873,v1:172.21.15.174:6839/3620064873] conn(0x55c71ff16d40 0x55c71ff86970 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55c71fef2a10 0x55c71ff0f5e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 --2- 172.21.15.33:0/2194986096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55c71fef20f0 0x55c71ff0f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.653+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 >> 172.21.15.33:0/2194986096 conn(0x55c71feeff20 msgr2=0x55c71ff05530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.655+0000 7f1f16ffd640 0 client.19812 ms_handle_reset on v2:172.21.15.174:6838/3620064873 2024-09-10T21:29:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.655+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 shutdown_connections 2024-09-10T21:29:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.655+0000 7f1f40544740 1 -- 172.21.15.33:0/2194986096 wait complete. 2024-09-10T21:29:04.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.655+0000 7f1f40544740 20 client.19812 trim_cache size 0 max 16384 2024-09-10T21:29:04.658 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:29:04.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.656+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f200b5360 msgr2=0x7f1f200b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.656+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f200b5360 0x7f1f200b7820 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1f28009b60 tx=0x7f1f28005f90 comp rx=0 tx=0).stop 2024-09-10T21:29:04.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.656+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 msgr2=0x7f1f38199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:04.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.656+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 0x7f1f38199790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009d00 tx=0x7f1f2c00c6a0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.657+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 shutdown_connections 2024-09-10T21:29:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.657+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f1f200b5360 0x7f1f200b7820 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.657+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f3810d100 0x7f1f3819eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.657+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1f381045c0 0x7f1f38199790 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.657+0000 7f1f3e16f640 1 --2- 172.21.15.33:0/4274656617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f38103ca0 0x7f1f38199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.657+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 >> 172.21.15.33:0/4274656617 conn(0x7f1f380fd700 msgr2=0x7f1f3810f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.658+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 shutdown_connections 2024-09-10T21:29:04.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:04.658+0000 7f1f3e16f640 1 -- 172.21.15.33:0/4274656617 wait complete. 2024-09-10T21:29:04.678 DEBUG:teuthology.parallel:result is None 2024-09-10T21:29:04.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T21:29:04.718 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T21:29:04.718 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-10T21:29:04.732 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-10T21:29:05.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-10T21:29:05.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:05 smithi033 ceph-mon[30860]: pgmap v1099: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 759 op/s 2024-09-10T21:29:05.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:05 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:05.715 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:05.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:05 smithi174 ceph-mon[35635]: pgmap v1099: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 759 op/s 2024-09-10T21:29:05.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:05 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:06.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:05 smithi104 ceph-mon[35325]: pgmap v1099: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 13 MiB/s wr, 759 op/s 2024-09-10T21:29:06.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:05 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:07.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:07 smithi174 ceph-mon[35635]: pgmap v1100: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 724 op/s 2024-09-10T21:29:07.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:07 smithi033 ceph-mon[30860]: pgmap v1100: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 724 op/s 2024-09-10T21:29:08.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:07 smithi104 ceph-mon[35325]: pgmap v1100: 97 pgs: 97 active+clean; 1.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 724 op/s 2024-09-10T21:29:08.231 INFO:teuthology.orchestra.run.smithi033.stdout:[] 2024-09-10T21:29:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 -- 172.21.15.33:0/1651568015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087d0 msgr2=0x7f7d7c108c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 --2- 172.21.15.33:0/1651568015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087d0 0x7f7d7c108c10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f7d68009900 tx=0x7f7d6802a410 comp rx=0 tx=0).stop 2024-09-10T21:29:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 -- 172.21.15.33:0/1651568015 shutdown_connections 2024-09-10T21:29:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 --2- 172.21.15.33:0/1651568015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109150 0x7f7d7c10f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 --2- 172.21.15.33:0/1651568015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c1087d0 0x7f7d7c108c10 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 --2- 172.21.15.33:0/1651568015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c102dc0 0x7f7d7c1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 -- 172.21.15.33:0/1651568015 >> 172.21.15.33:0/1651568015 conn(0x7f7d7c0fe830 msgr2=0x7f7d7c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 -- 172.21.15.33:0/1651568015 shutdown_connections 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.217+0000 7f7d82b62640 1 -- 172.21.15.33:0/1651568015 wait complete. 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 Processor -- start 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 -- start start 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c102dc0 0x7f7d7c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c1087d0 0x7f7d7c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109150 0x7f7d7c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112e10 con 0x7f7d7c102dc0 2024-09-10T21:29:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112c90 con 0x7f7d7c1087d0 2024-09-10T21:29:08.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d82b62640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112f90 con 0x7f7d7c109150 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c102dc0 0x7f7d7c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c102dc0 0x7f7d7c1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55540/0 (socket says 172.21.15.33:55540) 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d808fe640 1 -- 172.21.15.33:0/3054103332 learned_addr learned my addr 172.21.15.33:0/3054103332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.218+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c1087d0 0x7f7d7c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d810ff640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109150 0x7f7d7c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d808fe640 1 -- 172.21.15.33:0/3054103332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109150 msgr2=0x7f7d7c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d808fe640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109150 0x7f7d7c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d808fe640 1 -- 172.21.15.33:0/3054103332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c1087d0 msgr2=0x7f7d7c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d808fe640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c1087d0 0x7f7d7c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d808fe640 1 -- 172.21.15.33:0/3054103332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c19f390 con 0x7f7d7c102dc0 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d810ff640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109150 0x7f7d7c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d808fe640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c102dc0 0x7f7d7c1992e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000e9f0 tx=0x7f7d7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d79ffb640 1 -- 172.21.15.33:0/3054103332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7000cd80 con 0x7f7d7c102dc0 2024-09-10T21:29:08.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d79ffb640 1 -- 172.21.15.33:0/3054103332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d700108e0 con 0x7f7d7c102dc0 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c19f680 con 0x7f7d7c102dc0 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d79ffb640 1 -- 172.21.15.33:0/3054103332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d70018980 con 0x7f7d7c102dc0 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.219+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1a5dc0 con 0x7f7d7c102dc0 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.222+0000 7f7d84f37740 1 Processor -- start 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.222+0000 7f7d79ffb640 1 -- 172.21.15.33:0/3054103332 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f7d70010430 con 0x7f7d7c102dc0 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.222+0000 7f7d84f37740 1 -- start start 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.222+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be124138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.222+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12413df0 0x55be12423c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.222+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be12426580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d84f37740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be12389ea0 con 0x55be12413df0 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d84f37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55be123792d0 con 0x55be12424190 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d84f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55be1237d0e0 con 0x55be124134d0 2024-09-10T21:29:08.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12413df0 0x55be12423c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12413df0 0x55be12423c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55554/0 (socket says 172.21.15.33:55554) 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 -- 172.21.15.33:0/2566937384 learned_addr learned my addr 172.21.15.33:0/2566937384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d810ff640 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be124138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be12426580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 -- 172.21.15.33:0/2566937384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 msgr2=0x55be124138b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be124138b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 -- 172.21.15.33:0/2566937384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 msgr2=0x55be12426580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be12426580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 -- 172.21.15.33:0/2566937384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be120d7650 con 0x55be12413df0 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d79ffb640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b52e0 0x7f7d6c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.223+0000 7f7d808fe640 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12413df0 0x55be12423c50 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ee190 tx=0x7f7d700ee650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462dd0cebb1955c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d810ff640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b52e0 0x7f7d6c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d79ffb640 1 -- 172.21.15.33:0/3054103332 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d700e5fd0 con 0x7f7d7c102dc0 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d66ffd640 1 -- 172.21.15.33:0/2566937384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700cf070 con 0x55be12413df0 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d66ffd640 1 -- 172.21.15.33:0/2566937384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d700ac2a0 con 0x55be12413df0 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d66ffd640 1 -- 172.21.15.33:0/2566937384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700a8630 con 0x55be12413df0 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d84f37740 1 -- 172.21.15.33:0/2566937384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12413df0 msgr2=0x55be12423c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d810ff640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b52e0 0x7f7d6c0b77a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400f4c0 tx=0x7f7d74007740 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d84f37740 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12413df0 0x55be12423c50 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f7d700ee190 tx=0x7f7d700ee650 comp rx=0 tx=0).stop 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d84f37740 1 -- 172.21.15.33:0/2566937384 shutdown_connections 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d84f37740 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be12426580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d84f37740 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12413df0 0x55be12423c50 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d84f37740 1 --2- 172.21.15.33:0/2566937384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be124138b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.224+0000 7f7d84f37740 1 -- 172.21.15.33:0/2566937384 >> 172.21.15.33:0/2566937384 conn(0x55be12411300 msgr2=0x55be124124a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.225+0000 7f7d84f37740 1 -- 172.21.15.33:0/2566937384 shutdown_connections 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.225+0000 7f7d84f37740 1 -- 172.21.15.33:0/2566937384 wait complete. 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.225+0000 7f7d84f37740 1 -- start start 2024-09-10T21:29:08.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.225+0000 7f7d84f37740 1 Processor -- start 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be12423a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be12420eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12424190 0x55be12421490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d84f37740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x55be121da700 con 0x55be12424190 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d84f37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x55be12314160 con 0x55be12413df0 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d84f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55be12426de0 con 0x55be124134d0 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d84f37740 10 client.0 _ll_register_callbacks cb 0x55be1232d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12424190 0x55be12421490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12424190 0x55be12421490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55564/0 (socket says 172.21.15.33:55564) 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 -- 172.21.15.33:0/4196891570 learned_addr learned my addr 172.21.15.33:0/4196891570 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be12420eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d808fe640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be12423a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 -- 172.21.15.33:0/4196891570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 msgr2=0x55be12423a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be12423a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 -- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 msgr2=0x55be12420eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be12420eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 -- 172.21.15.33:0/4196891570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be12341c90 con 0x55be12424190 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be12420eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d810ff640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12424190 0x55be12421490 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7d74010ee0 tx=0x7f7d74004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3469c2ce4542b441 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.226+0000 7f7d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d5bfff640 1 -- 172.21.15.33:0/4196891570 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74026680 con 0x55be12424190 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d5bfff640 1 -- 172.21.15.33:0/4196891570 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d7401d420 con 0x55be12424190 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d700ac2a0 con 0x55be12424190 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d5bfff640 1 -- 172.21.15.33:0/4196891570 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f7d74026820 con 0x55be12424190 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 10 client.19824 fetch_fsmap learned FSMap version 100 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) -- 0x55be12432780 con 0x55be12424190 2024-09-10T21:29:08.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d5bfff640 1 -- 172.21.15.33:0/4196891570 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d700ac2a0 con 0x55be12424190 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d5bfff640 1 -- 172.21.15.33:0/4196891570 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 100) ==== 8426+0+0 (secure 0 0 0) 0x7f7d7401dc30 con 0x55be12424190 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 10 client.19824 fetch_fsmap finished waiting for FSMap version 100 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 10 client.19824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 20 client.19824 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be12433ce0 0x55be12436120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 4 client.19824 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.227+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be1238bdf0 con 0x55be12433ce0 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.228+0000 7f7d808fe640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be12433ce0 0x55be12436120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.228+0000 7f7d808fe640 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be12433ce0 0x55be12436120 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f34a79b16fe7e1e server_cookie=c15460a58fdbf723 in_seq=0 out_seq=0 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.228+0000 7f7d5bfff640 10 client.19824 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d5bfff640 1 -- 172.21.15.33:0/4196891570 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55be1238bdf0 con 0x55be12433ce0 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d5bfff640 10 client.19824 handle_command_reply: tid=0 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 client.19824 shutdown 2024-09-10T21:29:08.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12424190 msgr2=0x55be12421490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12424190 0x55be12421490 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f7d74010ee0 tx=0x7f7d74004b50 comp rx=0 tx=0).stop 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 shutdown_connections 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be12433ce0 0x55be12436120 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be12424190 0x55be12421490 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d5bfff640 0 client.19824 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be12420eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 --2- 172.21.15.33:0/4196891570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be124134d0 0x55be12423a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 >> 172.21.15.33:0/4196891570 conn(0x55be12411300 msgr2=0x55be12416e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 shutdown_connections 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 1 -- 172.21.15.33:0/4196891570 wait complete. 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.229+0000 7f7d84f37740 20 client.19824 trim_cache size 0 max 16384 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 Processor -- start 2024-09-10T21:29:08.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 -- start start 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be12410c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12413df0 0x55be12430040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be1243ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ac2c0 con 0x55be124134d0 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ae040 con 0x55be12424190 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d84f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1aca20 con 0x55be12413df0 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be12410c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be12410c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55570/0 (socket says 172.21.15.33:55570) 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 -- 172.21.15.33:0/1408341907 learned_addr learned my addr 172.21.15.33:0/1408341907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d810ff640 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be1243ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12413df0 0x55be12430040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 -- 172.21.15.33:0/1408341907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12413df0 msgr2=0x55be12430040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12413df0 0x55be12430040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 -- 172.21.15.33:0/1408341907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 msgr2=0x55be1243ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be1243ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.243+0000 7f7d808fe640 1 -- 172.21.15.33:0/1408341907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be1241f3b0 con 0x55be124134d0 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d808fe640 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be12410c10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7d700a9a00 tx=0x7f7d700a8c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37e98b402bc4fecc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d667fc640 1 -- 172.21.15.33:0/1408341907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700cf040 con 0x55be124134d0 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d667fc640 1 -- 172.21.15.33:0/1408341907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d700eee80 con 0x55be124134d0 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d667fc640 1 -- 172.21.15.33:0/1408341907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700e9b40 con 0x55be124134d0 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d84f37740 1 -- 172.21.15.33:0/1408341907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 msgr2=0x55be12410c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be12410c10 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7d700a9a00 tx=0x7f7d700a8c10 comp rx=0 tx=0).stop 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d84f37740 1 -- 172.21.15.33:0/1408341907 shutdown_connections 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12424190 0x55be1243ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12413df0 0x55be12430040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1408341907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be12410c10 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.244+0000 7f7d84f37740 1 -- 172.21.15.33:0/1408341907 >> 172.21.15.33:0/1408341907 conn(0x55be12411300 msgr2=0x55be12425f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:08.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 -- 172.21.15.33:0/1408341907 shutdown_connections 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 -- 172.21.15.33:0/1408341907 wait complete. 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 -- start start 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 Processor -- start 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be1242c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be1242c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12424190 0x55be12429070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ad8e0 con 0x55be124134d0 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ad1b0 con 0x55be12413df0 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1aa280 con 0x55be12424190 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d84f37740 10 client.0 _ll_register_callbacks cb 0x7f7d74010b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.245+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be1242c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be1242c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55584/0 (socket says 172.21.15.33:55584) 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 -- 172.21.15.33:0/1859442938 learned_addr learned my addr 172.21.15.33:0/1859442938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:08.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be1242c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d810ff640 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12424190 0x55be12429070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 -- 172.21.15.33:0/1859442938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12424190 msgr2=0x55be12429070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12424190 0x55be12429070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 -- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 msgr2=0x55be1242c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be1242c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 -- 172.21.15.33:0/1859442938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be124297e0 con 0x55be124134d0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d808fe640 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be1242c280 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7d700097a0 tx=0x7f7d7000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1a725d522cc09 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.33:3300/0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d5bfff640 1 -- 172.21.15.33:0/1859442938 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700cf070 con 0x55be124134d0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d5bfff640 1 -- 172.21.15.33:0/1859442938 <== mon.0 v2:172.21.15.33:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f7d700ac9f0 con 0x55be124134d0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be123878a0 con 0x55be124134d0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d5bfff640 1 -- 172.21.15.33:0/1859442938 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f7d700abf00 con 0x55be124134d0 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d84f37740 10 client.19830 fetch_fsmap learned FSMap version 100 2024-09-10T21:29:08.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.246+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=4+}) -- 0x55be124a16a0 con 0x55be124134d0 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d5bfff640 1 -- 172.21.15.33:0/1859442938 <== mon.0 v2:172.21.15.33:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55be123878a0 con 0x55be124134d0 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d5bfff640 1 -- 172.21.15.33:0/1859442938 <== mon.0 v2:172.21.15.33:3300/0 5 ==== fsmap(e 100) ==== 8426+0+0 (secure 0 0 0) 0x7f7d700b5890 con 0x55be124134d0 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d84f37740 10 client.19830 fetch_fsmap finished waiting for FSMap version 100 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d84f37740 10 client.19830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d84f37740 20 client.19830 populate_metadata read hostname 'smithi033' 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be124a19d0 0x55be124a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d84f37740 4 client.19830 mds_command: new command op to 24566 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 --> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be12384cd0 con 0x55be124a19d0 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.247+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be124a19d0 0x55be124a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d5bfff640 10 client.19830 ms_handle_connect on v2:172.21.15.104:6838/2034440949 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d7bfff640 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be124a19d0 0x55be124a3df0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=973fca6bdd082de8 server_cookie=ed6eb2b3c7754811 in_seq=0 out_seq=0 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d5bfff640 1 -- 172.21.15.33:0/1859442938 <== mds.0 v2:172.21.15.104:6838/2034440949 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be12384cd0 con 0x55be124a19d0 2024-09-10T21:29:08.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d5bfff640 10 client.19830 handle_command_reply: tid=0 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 client.19830 shutdown 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 msgr2=0x55be1242c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be1242c280 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7d700097a0 tx=0x7f7d7000cf20 comp rx=0 tx=0).stop 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 shutdown_connections 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:6838/2034440949,v1:172.21.15.104:6839/2034440949] conn(0x55be124a19d0 0x55be124a3df0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55be12424190 0x55be12429070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x55be12413df0 0x55be1242c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d5bfff640 0 client.19830 ms_handle_reset on v2:172.21.15.104:6838/2034440949 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 --2- 172.21.15.33:0/1859442938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x55be124134d0 0x55be1242c280 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 >> 172.21.15.33:0/1859442938 conn(0x55be12411300 msgr2=0x55be124224e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 shutdown_connections 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.248+0000 7f7d84f37740 1 -- 172.21.15.33:0/1859442938 wait complete. 2024-09-10T21:29:08.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d84f37740 20 client.19830 trim_cache size 0 max 16384 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b52e0 msgr2=0x7f7d6c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b52e0 0x7f7d6c0b77a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400f4c0 tx=0x7f7d74007740 comp rx=0 tx=0).stop 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c102dc0 msgr2=0x7f7d7c1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c102dc0 0x7f7d7c1992e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000e9f0 tx=0x7f7d7000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 shutdown_connections 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f7d6c0b52e0 0x7f7d6c0b77a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d7c109150 0x7f7d7c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7d7c1087d0 0x7f7d7c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 --2- 172.21.15.33:0/3054103332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d7c102dc0 0x7f7d7c1992e0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.249+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 >> 172.21.15.33:0/3054103332 conn(0x7f7d7c0fe830 msgr2=0x7f7d7c100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.250+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 shutdown_connections 2024-09-10T21:29:08.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:07.250+0000 7f7d82b62640 1 -- 172.21.15.33:0/3054103332 wait complete. 2024-09-10T21:29:09.302 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-10T21:29:09.303 INFO:tasks.fwd_scrub:done joining 2024-09-10T21:29:09.303 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-10T21:29:09.313 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 -- 172.21.15.33:0/975049356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740103390 msgr2=0x7f3740109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 --2- 172.21.15.33:0/975049356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740103390 0x7f3740109ed0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f3738009ca0 tx=0x7f373802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 -- 172.21.15.33:0/975049356 shutdown_connections 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 --2- 172.21.15.33:0/975049356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740103390 0x7f3740109ed0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 --2- 172.21.15.33:0/975049356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3740102820 0x7f3740102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 --2- 172.21.15.33:0/975049356 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3740101ee0 0x7f37401022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 -- 172.21.15.33:0/975049356 >> 172.21.15.33:0/975049356 conn(0x7f37400fc8d0 msgr2=0x7f37400fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 -- 172.21.15.33:0/975049356 shutdown_connections 2024-09-10T21:29:09.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 -- 172.21.15.33:0/975049356 wait complete. 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 Processor -- start 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.533+0000 7f3748591640 1 -- start start 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3748591640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740101ee0 0x7f37401014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3748591640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3740102820 0x7f374019d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3748591640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3740103390 0x7f374019ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3748591640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f374010d4f0 con 0x7f3740101ee0 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3748591640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f374010d370 con 0x7f3740103390 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3748591640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f374010d670 con 0x7f3740102820 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740101ee0 0x7f37401014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740101ee0 0x7f37401014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48654/0 (socket says 172.21.15.33:48654) 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 -- 172.21.15.33:0/1089208868 learned_addr learned my addr 172.21.15.33:0/1089208868 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3745b2c640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3740102820 0x7f374019d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3746b2e640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3740103390 0x7f374019ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 -- 172.21.15.33:0/1089208868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3740102820 msgr2=0x7f374019d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3740102820 0x7f374019d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 -- 172.21.15.33:0/1089208868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3740103390 msgr2=0x7f374019ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3740103390 0x7f374019ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 -- 172.21.15.33:0/1089208868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37401a07e0 con 0x7f3740101ee0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3746b2e640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3740103390 0x7f374019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f374632d640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740101ee0 0x7f37401014b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f373c00d950 tx=0x7f373c00dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f37377fe640 1 -- 172.21.15.33:0/1089208868 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c017070 con 0x7f3740101ee0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37401a0ad0 con 0x7f3740101ee0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.534+0000 7f37377fe640 1 -- 172.21.15.33:0/1089208868 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f373c00de80 con 0x7f3740101ee0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.535+0000 7f37377fe640 1 -- 172.21.15.33:0/1089208868 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00f620 con 0x7f3740101ee0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.535+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37401a1020 con 0x7f3740101ee0 2024-09-10T21:29:09.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.536+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3740108ce0 con 0x7f3740101ee0 2024-09-10T21:29:09.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.538+0000 7f37377fe640 1 -- 172.21.15.33:0/1089208868 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f373c005030 con 0x7f3740101ee0 2024-09-10T21:29:09.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.539+0000 7f37377fe640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f372c0b52e0 0x7f372c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.539+0000 7f3745b2c640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f372c0b52e0 0x7f372c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.539+0000 7f37377fe640 1 -- 172.21.15.33:0/1089208868 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f373c0e57d0 con 0x7f3740101ee0 2024-09-10T21:29:09.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.539+0000 7f3745b2c640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f372c0b52e0 0x7f372c0b77a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f373000abe0 tx=0x7f373000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:09.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.541+0000 7f37377fe640 1 -- 172.21.15.33:0/1089208868 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f373c0ac690 con 0x7f3740101ee0 2024-09-10T21:29:09.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.735+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f374002e870 con 0x7f3740101ee0 2024-09-10T21:29:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.737+0000 7f37377fe640 1 -- 172.21.15.33:0/1089208868 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22763 (secure 0 0 0) 0x7f373c0abf90 con 0x7f3740101ee0 2024-09-10T21:29:09.739 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:29:09.739 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:41.855317+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:09.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f372c0b52e0 msgr2=0x7f372c0b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f372c0b52e0 0x7f372c0b77a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f373000abe0 tx=0x7f373000a2d0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740101ee0 msgr2=0x7f37401014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740101ee0 0x7f37401014b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f373c00d950 tx=0x7f373c00dd50 comp rx=0 tx=0).stop 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 shutdown_connections 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f372c0b52e0 0x7f372c0b77a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3740103390 0x7f374019ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3740102820 0x7f374019d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 --2- 172.21.15.33:0/1089208868 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3740101ee0 0x7f37401014b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 >> 172.21.15.33:0/1089208868 conn(0x7f37400fc8d0 msgr2=0x7f3740105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 shutdown_connections 2024-09-10T21:29:09.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.739+0000 7f3748591640 1 -- 172.21.15.33:0/1089208868 wait complete. 2024-09-10T21:29:09.753 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T20:57:24.470902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T21:01:57.440910+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T21:01:59.340474+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:29:09.753 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:29:09.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 -- 172.21.15.33:0/2600481190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 msgr2=0x7f537c107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 --2- 172.21.15.33:0/2600481190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c107f30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f5364009900 tx=0x7f536402a410 comp rx=0 tx=0).stop 2024-09-10T21:29:09.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 -- 172.21.15.33:0/2600481190 shutdown_connections 2024-09-10T21:29:09.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 --2- 172.21.15.33:0/2600481190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f537c108670 0x7f537c10ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 --2- 172.21.15.33:0/2600481190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c107f30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 --2- 172.21.15.33:0/2600481190 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c06afc0 0x7f537c105530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 -- 172.21.15.33:0/2600481190 >> 172.21.15.33:0/2600481190 conn(0x7f537c0fd140 msgr2=0x7f537c0ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 -- 172.21.15.33:0/2600481190 shutdown_connections 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.862+0000 7f5382956640 1 -- 172.21.15.33:0/2600481190 wait complete. 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 Processor -- start 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 -- start start 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c06afc0 0x7f537c198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f537c108670 0x7f537c199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10e5a0 con 0x7f537c105ac0 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10e420 con 0x7f537c06afc0 2024-09-10T21:29:09.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5382956640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f537c10e720 con 0x7f537c108670 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48670/0 (socket says 172.21.15.33:48670) 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 -- 172.21.15.33:0/3027204907 learned_addr learned my addr 172.21.15.33:0/3027204907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5380ef3640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f537c108670 0x7f537c199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f537bfff640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c06afc0 0x7f537c198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 -- 172.21.15.33:0/3027204907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f537c108670 msgr2=0x7f537c199fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f537c108670 0x7f537c199fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 -- 172.21.15.33:0/3027204907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c06afc0 msgr2=0x7f537c198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c06afc0 0x7f537c198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.863+0000 7f5373fff640 1 -- 172.21.15.33:0/3027204907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f537c19a880 con 0x7f537c105ac0 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.864+0000 7f5373fff640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c199400 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f5364005940 tx=0x7f5364004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:09.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.864+0000 7f5379ffb640 1 -- 172.21.15.33:0/3027204907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536403f030 con 0x7f537c105ac0 2024-09-10T21:29:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.864+0000 7f5379ffb640 1 -- 172.21.15.33:0/3027204907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5364003cb0 con 0x7f537c105ac0 2024-09-10T21:29:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.864+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f537c19ab10 con 0x7f537c105ac0 2024-09-10T21:29:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.864+0000 7f5379ffb640 1 -- 172.21.15.33:0/3027204907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364035070 con 0x7f537c105ac0 2024-09-10T21:29:09.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.864+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f537c1a1240 con 0x7f537c105ac0 2024-09-10T21:29:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.865+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f537c105f20 con 0x7f537c105ac0 2024-09-10T21:29:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.866+0000 7f5379ffb640 1 -- 172.21.15.33:0/3027204907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f536403b460 con 0x7f537c105ac0 2024-09-10T21:29:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.867+0000 7f5379ffb640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f53680b5360 0x7f53680b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.867+0000 7f537bfff640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f53680b5360 0x7f53680b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:09.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.867+0000 7f5379ffb640 1 -- 172.21.15.33:0/3027204907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f53641069c0 con 0x7f537c105ac0 2024-09-10T21:29:09.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.867+0000 7f537bfff640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f53680b5360 0x7f53680b7820 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f536c00bbe0 tx=0x7f536c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:09.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:09.870+0000 7f5379ffb640 1 -- 172.21.15.33:0/3027204907 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5364003e60 con 0x7f537c105ac0 2024-09-10T21:29:09.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:09 smithi174 ceph-mon[35635]: pgmap v1101: 97 pgs: 97 active+clean; 981 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 737 op/s 2024-09-10T21:29:09.908 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:09 smithi033 ceph-mon[30860]: pgmap v1101: 97 pgs: 97 active+clean; 981 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 737 op/s 2024-09-10T21:29:10.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:09 smithi104 ceph-mon[35325]: pgmap v1101: 97 pgs: 97 active+clean; 981 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 15 MiB/s wr, 737 op/s 2024-09-10T21:29:10.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.059+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f537c02e5c0 con 0x7f537c105ac0 2024-09-10T21:29:10.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.059+0000 7f5379ffb640 1 -- 172.21.15.33:0/3027204907 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f53640cd1f0 con 0x7f537c105ac0 2024-09-10T21:29:10.062 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f53680b5360 msgr2=0x7f53680b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f53680b5360 0x7f53680b7820 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f536c00bbe0 tx=0x7f536c00b390 comp rx=0 tx=0).stop 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 msgr2=0x7f537c199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c199400 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f5364005940 tx=0x7f5364004430 comp rx=0 tx=0).stop 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 shutdown_connections 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f53680b5360 0x7f53680b7820 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f537c108670 0x7f537c199fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c105ac0 0x7f537c199400 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 --2- 172.21.15.33:0/3027204907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f537c06afc0 0x7f537c198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 >> 172.21.15.33:0/3027204907 conn(0x7f537c0fd140 msgr2=0x7f537c103da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 shutdown_connections 2024-09-10T21:29:10.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.062+0000 7f5382956640 1 -- 172.21.15.33:0/3027204907 wait complete. 2024-09-10T21:29:10.076 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T21:29:10.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.183+0000 7f9c300bd640 1 -- 172.21.15.33:0/692987337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c28103370 msgr2=0x7f9c28109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.183+0000 7f9c300bd640 1 --2- 172.21.15.33:0/692987337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c28103370 0x7f9c28109cf0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009cd0 tx=0x7f9c1802a8e0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.183+0000 7f9c300bd640 1 -- 172.21.15.33:0/692987337 shutdown_connections 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 --2- 172.21.15.33:0/692987337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c28103370 0x7f9c28109cf0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 --2- 172.21.15.33:0/692987337 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c281029b0 0x7f9c28102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 --2- 172.21.15.33:0/692987337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c28101860 0x7f9c28101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 -- 172.21.15.33:0/692987337 >> 172.21.15.33:0/692987337 conn(0x7f9c280fd140 msgr2=0x7f9c280ff560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 -- 172.21.15.33:0/692987337 shutdown_connections 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 -- 172.21.15.33:0/692987337 wait complete. 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 Processor -- start 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 -- start start 2024-09-10T21:29:10.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c28101860 0x7f9c28105690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.184+0000 7f9c300bd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c281029b0 0x7f9c28103ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c300bd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28103370 0x7f9c28104430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c300bd640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2810d4b0 con 0x7f9c281029b0 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c300bd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2810d330 con 0x7f9c28103370 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c300bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2810d630 con 0x7f9c28101860 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c281029b0 0x7f9c28103ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c281029b0 0x7f9c28103ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48696/0 (socket says 172.21.15.33:48696) 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 -- 172.21.15.33:0/1159109518 learned_addr learned my addr 172.21.15.33:0/1159109518 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2de59640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c28101860 0x7f9c28105690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2e65a640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28103370 0x7f9c28104430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 -- 172.21.15.33:0/1159109518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c28101860 msgr2=0x7f9c28105690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c28101860 0x7f9c28105690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 -- 172.21.15.33:0/1159109518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28103370 msgr2=0x7f9c28104430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28103370 0x7f9c28104430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 -- 172.21.15.33:0/1159109518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c28104ca0 con 0x7f9c281029b0 2024-09-10T21:29:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2e65a640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28103370 0x7f9c28104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c2d658640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c281029b0 0x7f9c28103ef0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d950 tx=0x7f9c1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c26ffd640 1 -- 172.21.15.33:0/1159109518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c017030 con 0x7f9c281029b0 2024-09-10T21:29:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c281a2e00 con 0x7f9c281029b0 2024-09-10T21:29:10.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c26ffd640 1 -- 172.21.15.33:0/1159109518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c1c004020 con 0x7f9c281029b0 2024-09-10T21:29:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c26ffd640 1 -- 172.21.15.33:0/1159109518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c0047a0 con 0x7f9c281029b0 2024-09-10T21:29:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.185+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281a32c0 con 0x7f9c281029b0 2024-09-10T21:29:10.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.186+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c281a3550 con 0x7f9c281029b0 2024-09-10T21:29:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.188+0000 7f9c26ffd640 1 -- 172.21.15.33:0/1159109518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9c1c009a30 con 0x7f9c281029b0 2024-09-10T21:29:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.188+0000 7f9c26ffd640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c100b5360 0x7f9c100b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.189+0000 7f9c2de59640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c100b5360 0x7f9c100b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.189+0000 7f9c26ffd640 1 -- 172.21.15.33:0/1159109518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9c1c0e5640 con 0x7f9c281029b0 2024-09-10T21:29:10.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.189+0000 7f9c2de59640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c100b5360 0x7f9c100b7820 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009bd0 tx=0x7f9c20005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:10.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.196+0000 7f9c26ffd640 1 -- 172.21.15.33:0/1159109518 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9c1c0ac4c0 con 0x7f9c281029b0 2024-09-10T21:29:10.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.382+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9c281a38e0 con 0x7f9c281029b0 2024-09-10T21:29:10.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.383+0000 7f9c26ffd640 1 -- 172.21.15.33:0/1159109518 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) ==== 104+0+11 (secure 0 0 0) 0x7f9c1c0abdc0 con 0x7f9c281029b0 2024-09-10T21:29:10.385 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-09-10T21:29:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c100b5360 msgr2=0x7f9c100b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c100b5360 0x7f9c100b7820 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009bd0 tx=0x7f9c20005f90 comp rx=0 tx=0).stop 2024-09-10T21:29:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c281029b0 msgr2=0x7f9c28103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c281029b0 0x7f9c28103ef0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00d950 tx=0x7f9c1c00de10 comp rx=0 tx=0).stop 2024-09-10T21:29:10.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 shutdown_connections 2024-09-10T21:29:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c100b5360 0x7f9c100b7820 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c28103370 0x7f9c28104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c281029b0 0x7f9c28103ef0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 --2- 172.21.15.33:0/1159109518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c28101860 0x7f9c28105690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 >> 172.21.15.33:0/1159109518 conn(0x7f9c280fd140 msgr2=0x7f9c2806a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 shutdown_connections 2024-09-10T21:29:10.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.385+0000 7f9c300bd640 1 -- 172.21.15.33:0/1159109518 wait complete. 2024-09-10T21:29:10.398 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T21:29:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 -- 172.21.15.33:0/1945335034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c103ae0 msgr2=0x7fb57c103f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 --2- 172.21.15.33:0/1945335034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c103ae0 0x7fb57c103f60 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fb564009930 tx=0x7fb56402a420 comp rx=0 tx=0).stop 2024-09-10T21:29:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 -- 172.21.15.33:0/1945335034 shutdown_connections 2024-09-10T21:29:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 --2- 172.21.15.33:0/1945335034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb57c1044a0 0x7fb57c10ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 --2- 172.21.15.33:0/1945335034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c103ae0 0x7fb57c103f60 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 --2- 172.21.15.33:0/1945335034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c102990 0x7fb57c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 -- 172.21.15.33:0/1945335034 >> 172.21.15.33:0/1945335034 conn(0x7fb57c0fe250 msgr2=0x7fb57c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 -- 172.21.15.33:0/1945335034 shutdown_connections 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.504+0000 7fb5814fe640 1 -- 172.21.15.33:0/1945335034 wait complete. 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 Processor -- start 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 -- start start 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c102990 0x7fb57c076ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c103ae0 0x7fb57c075540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb57c1044a0 0x7fb57c075a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c10e630 con 0x7fb57c102990 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c10e4b0 con 0x7fb57c103ae0 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb5814fe640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c10e7b0 con 0x7fb57c1044a0 2024-09-10T21:29:10.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c102990 0x7fb57c076ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c102990 0x7fb57c076ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48714/0 (socket says 172.21.15.33:48714) 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 -- 172.21.15.33:0/3455782195 learned_addr learned my addr 172.21.15.33:0/3455782195 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb573fff640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c103ae0 0x7fb57c075540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57b7fe640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb57c1044a0 0x7fb57c075a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 -- 172.21.15.33:0/3455782195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb57c1044a0 msgr2=0x7fb57c075a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb57c1044a0 0x7fb57c075a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 -- 172.21.15.33:0/3455782195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c103ae0 msgr2=0x7fb57c075540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c103ae0 0x7fb57c075540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.505+0000 7fb57affd640 1 -- 172.21.15.33:0/3455782195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c0762f0 con 0x7fb57c102990 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.506+0000 7fb57affd640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c102990 0x7fb57c076ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00d950 tx=0x7fb56c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.506+0000 7fb578ff9640 1 -- 172.21.15.33:0/3455782195 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c017070 con 0x7fb57c102990 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.506+0000 7fb578ff9640 1 -- 172.21.15.33:0/3455782195 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb56c009c30 con 0x7fb57c102990 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.506+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c1a2ed0 con 0x7fb57c102990 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.506+0000 7fb578ff9640 1 -- 172.21.15.33:0/3455782195 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56c00f620 con 0x7fb57c102990 2024-09-10T21:29:10.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.506+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c1a3330 con 0x7fb57c102990 2024-09-10T21:29:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.507+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57c109ca0 con 0x7fb57c102990 2024-09-10T21:29:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.508+0000 7fb578ff9640 1 -- 172.21.15.33:0/3455782195 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fb56c005030 con 0x7fb57c102990 2024-09-10T21:29:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.509+0000 7fb578ff9640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5680b52e0 0x7fb5680b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.510+0000 7fb573fff640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5680b52e0 0x7fb5680b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.510+0000 7fb578ff9640 1 -- 172.21.15.33:0/3455782195 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fb56c0e57f0 con 0x7fb57c102990 2024-09-10T21:29:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.510+0000 7fb573fff640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5680b52e0 0x7fb5680b77a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb564031440 tx=0x7fb5640314a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:10.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.513+0000 7fb578ff9640 1 -- 172.21.15.33:0/3455782195 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fb56c0ac6b0 con 0x7fb57c102990 2024-09-10T21:29:10.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:10 smithi033 ceph-mon[30860]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:10.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:10 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1089208868' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:29:10.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:10 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3027204907' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:29:10.675 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:10 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1159109518' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T21:29:10.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.702+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb57c02e830 con 0x7fb57c102990 2024-09-10T21:29:10.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.702+0000 7fb578ff9640 1 -- 172.21.15.33:0/3455782195 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) ==== 100+0+11 (secure 0 0 0) 0x7fb56c009de0 con 0x7fb57c102990 2024-09-10T21:29:10.704 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 64 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5680b52e0 msgr2=0x7fb5680b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5680b52e0 0x7fb5680b77a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb564031440 tx=0x7fb5640314a0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c102990 msgr2=0x7fb57c076ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c102990 0x7fb57c076ce0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fb56c00d950 tx=0x7fb56c00de10 comp rx=0 tx=0).stop 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 shutdown_connections 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fb5680b52e0 0x7fb5680b77a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb57c1044a0 0x7fb57c075a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb57c103ae0 0x7fb57c075540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 --2- 172.21.15.33:0/3455782195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb57c102990 0x7fb57c076ce0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 >> 172.21.15.33:0/3455782195 conn(0x7fb57c0fe250 msgr2=0x7fb57c106e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:10.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 shutdown_connections 2024-09-10T21:29:10.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.705+0000 7fb5814fe640 1 -- 172.21.15.33:0/3455782195 wait complete. 2024-09-10T21:29:10.719 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-10T21:29:10.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.820+0000 7f5271f5a640 1 -- 172.21.15.33:0/1850606177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c102a00 msgr2=0x7f526c102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.820+0000 7f5271f5a640 1 --2- 172.21.15.33:0/1850606177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c102a00 0x7f526c102e80 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f5260009930 tx=0x7f526002a420 comp rx=0 tx=0).stop 2024-09-10T21:29:10.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 -- 172.21.15.33:0/1850606177 shutdown_connections 2024-09-10T21:29:10.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 --2- 172.21.15.33:0/1850606177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526c1033c0 0x7f526c109d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 --2- 172.21.15.33:0/1850606177 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c102a00 0x7f526c102e80 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 --2- 172.21.15.33:0/1850606177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f526c1018b0 0x7f526c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 -- 172.21.15.33:0/1850606177 >> 172.21.15.33:0/1850606177 conn(0x7f526c0fd190 msgr2=0x7f526c0ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 -- 172.21.15.33:0/1850606177 shutdown_connections 2024-09-10T21:29:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 -- 172.21.15.33:0/1850606177 wait complete. 2024-09-10T21:29:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 Processor -- start 2024-09-10T21:29:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.821+0000 7f5271f5a640 1 -- start start 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f526c1018b0 0x7f526c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526c102a00 0x7f526c199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c1033c0 0x7f526c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f526c10e610 con 0x7f526c1033c0 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f526c10e490 con 0x7f526c102a00 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f526c10e790 con 0x7f526c1018b0 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c1033c0 0x7f526c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c1033c0 0x7f526c19a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48728/0 (socket says 172.21.15.33:48728) 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 -- 172.21.15.33:0/843212426 learned_addr learned my addr 172.21.15.33:0/843212426 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526b7fe640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f526c1018b0 0x7f526c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 -- 172.21.15.33:0/843212426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f526c1018b0 msgr2=0x7f526c198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f526c1018b0 0x7f526c198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526affd640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526c102a00 0x7f526c199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 -- 172.21.15.33:0/843212426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526c102a00 msgr2=0x7f526c199440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526c102a00 0x7f526c199440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 -- 172.21.15.33:0/843212426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c19a930 con 0x7f526c1033c0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526affd640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526c102a00 0x7f526c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f526bfff640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c1033c0 0x7f526c19a070 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f526400d750 tx=0x7f526400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5268ff9640 1 -- 172.21.15.33:0/843212426 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264017070 con 0x7f526c1033c0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5268ff9640 1 -- 172.21.15.33:0/843212426 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52640054d0 con 0x7f526c1033c0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c19ac20 con 0x7f526c1033c0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5268ff9640 1 -- 172.21.15.33:0/843212426 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264004c60 con 0x7f526c1033c0 2024-09-10T21:29:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.822+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a12f0 con 0x7f526c1033c0 2024-09-10T21:29:10.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.824+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526c108c20 con 0x7f526c1033c0 2024-09-10T21:29:10.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.825+0000 7f5268ff9640 1 -- 172.21.15.33:0/843212426 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5264004e00 con 0x7f526c1033c0 2024-09-10T21:29:10.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.826+0000 7f5268ff9640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52540b52e0 0x7f52540b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:10.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.826+0000 7f526b7fe640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52540b52e0 0x7f52540b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:10.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.826+0000 7f5268ff9640 1 -- 172.21.15.33:0/843212426 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f52640e55f0 con 0x7f526c1033c0 2024-09-10T21:29:10.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.826+0000 7f526b7fe640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52540b52e0 0x7f52540b77a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f525c005fd0 tx=0x7f525c009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:10.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:10.830+0000 7f5268ff9640 1 -- 172.21.15.33:0/843212426 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f52640ac4b0 con 0x7f526c1033c0 2024-09-10T21:29:10.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:10 smithi174 ceph-mon[35635]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:10 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1089208868' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:29:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:10 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3027204907' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:29:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:10 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1159109518' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T21:29:11.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:10 smithi104 ceph-mon[35325]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-10T21:29:11.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:10 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1089208868' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:29:11.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:10 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3027204907' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T21:29:11.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:10 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1159109518' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-10T21:29:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.081+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f526c063d50 con 0x7f526c1033c0 2024-09-10T21:29:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.090+0000 7f5268ff9640 1 -- 172.21.15.33:0/843212426 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+827944 (secure 0 0 0) 0x7f52640abdb0 con 0x7f526c1033c0 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "epoch": 18, 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "active_gid": 14148, 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "active_name": "x", 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addrs": { 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-10T21:29:11.103 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6800", 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3102318098 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:6801", 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 3102318098 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "active_addr": "172.21.15.33:6801/3102318098", 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "active_change": "2024-09-10T20:54:39.992688+0000", 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "active_mgr_features": 4540719139924082687, 2024-09-10T21:29:11.104 INFO:teuthology.orchestra.run.smithi033.stdout: "available": true, 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "standbys": [ 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24136, 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "z", 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540719139924082687, 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:11.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.107 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.110 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.113 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.114 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-09-10T21:29:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.115 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.116 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-09-10T21:29:11.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-09-10T21:29:11.116 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.117 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-09-10T21:29:11.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.118 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-09-10T21:29:11.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-09-10T21:29:11.118 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-09-10T21:29:11.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.119 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-09-10T21:29:11.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.120 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-09-10T21:29:11.120 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-09-10T21:29:11.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.121 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.121 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.121 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-09-10T21:29:11.122 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.122 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.123 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-09-10T21:29:11.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-09-10T21:29:11.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.123 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.123 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.124 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.125 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.126 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-09-10T21:29:11.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.128 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.131 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.131 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-09-10T21:29:11.131 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-09-10T21:29:11.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T21:29:11.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T21:29:11.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.132 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.132 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.132 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-09-10T21:29:11.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-09-10T21:29:11.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.133 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-09-10T21:29:11.133 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:11.133 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-09-10T21:29:11.133 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.133 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T21:29:11.133 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.134 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T21:29:11.136 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T21:29:11.137 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T21:29:11.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.139 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.140 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.141 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.142 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-09-10T21:29:11.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.147 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.149 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T21:29:11.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.152 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-09-10T21:29:11.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-09-10T21:29:11.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.155 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T21:29:11.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.159 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-09-10T21:29:11.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-09-10T21:29:11.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.162 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.163 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-09-10T21:29:11.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.170 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.172 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.173 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nginx": { 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nginx", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nginx container image", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.176 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_oauth2_proxy": { 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "oauth2-proxy container image", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-09-10T21:29:11.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-09-10T21:29:11.180 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba_metrics": { 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba_metrics", 2024-09-10T21:29:11.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-09-10T21:29:11.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.186 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T21:29:11.187 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-09-10T21:29:11.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T21:29:11.189 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.190 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.192 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.193 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.194 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.195 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.197 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.200 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.201 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.204 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T21:29:11.207 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.208 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T21:29:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T21:29:11.209 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-09-10T21:29:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-09-10T21:29:11.210 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-09-10T21:29:11.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-09-10T21:29:11.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-09-10T21:29:11.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.216 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-09-10T21:29:11.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-09-10T21:29:11.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-09-10T21:29:11.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.220 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.221 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.222 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.223 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.225 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.228 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.230 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-09-10T21:29:11.232 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.233 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.236 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.238 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.240 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.245 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.250 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T21:29:11.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T21:29:11.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T21:29:11.254 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.256 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T21:29:11.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.258 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.262 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.264 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.264 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.264 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.265 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.267 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.268 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.270 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.271 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.272 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.273 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.274 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.275 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.277 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.279 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-09-10T21:29:11.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.286 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.287 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.288 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.289 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.290 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.292 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.293 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.298 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:11.302 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.304 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.305 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.306 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-09-10T21:29:11.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.311 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.312 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-09-10T21:29:11.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-09-10T21:29:11.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.316 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.318 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.319 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.321 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.322 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.323 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.324 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.325 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.326 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.328 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-09-10T21:29:11.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.331 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.332 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.336 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.338 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-09-10T21:29:11.339 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.340 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.342 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.344 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-09-10T21:29:11.345 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.346 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.349 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.351 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.354 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.355 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.358 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.359 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.362 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.363 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.365 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.372 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-09-10T21:29:11.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-09-10T21:29:11.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.376 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.377 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.379 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.382 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.389 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.390 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.392 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.393 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.395 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.396 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.400 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.401 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.402 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.404 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.405 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.406 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.408 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.409 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.410 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.411 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.412 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.420 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.422 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.423 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.424 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.425 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.427 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.429 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.433 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.435 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.437 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.439 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.440 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.442 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.443 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.445 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.448 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.451 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.453 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.454 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.455 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.456 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.460 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.461 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.464 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.465 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.466 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.467 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.468 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.469 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.470 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.471 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.472 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.476 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.482 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max_requests": { 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_requests", 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:11.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.487 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.488 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.491 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.495 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.502 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.505 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.506 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.507 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.509 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.510 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-09-10T21:29:11.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.513 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.515 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.517 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.521 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.523 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "internal_store_backend": { 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "internal_store_backend", 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.526 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.527 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.528 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.529 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.530 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.532 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.533 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.536 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.539 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-09-10T21:29:11.542 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.543 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.546 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.549 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.551 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.553 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-09-10T21:29:11.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-09-10T21:29:11.565 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.566 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.570 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.575 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.578 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.581 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.583 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.585 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.586 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-09-10T21:29:11.587 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-09-10T21:29:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T21:29:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.588 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.589 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-09-10T21:29:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.590 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.591 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.592 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.596 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.599 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.600 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.601 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.602 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-09-10T21:29:11.608 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.614 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.615 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.617 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.621 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "gid": 24152, 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "y", 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr_features": 4540719139924082687, 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:11.622 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.626 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.629 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-09-10T21:29:11.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-09-10T21:29:11.631 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.634 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T21:29:11.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.636 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.637 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.638 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.639 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.641 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-09-10T21:29:11.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-09-10T21:29:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T21:29:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-09-10T21:29:11.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.646 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.648 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.653 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-09-10T21:29:11.654 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:11.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.659 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-09-10T21:29:11.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-09-10T21:29:11.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.663 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T21:29:11.665 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T21:29:11.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T21:29:11.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T21:29:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-09-10T21:29:11.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-09-10T21:29:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T21:29:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.674 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-09-10T21:29:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-09-10T21:29:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T21:29:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-09-10T21:29:11.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-09-10T21:29:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T21:29:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-09-10T21:29:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-09-10T21:29:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-09-10T21:29:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T21:29:11.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-09-10T21:29:11.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T21:29:11.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nginx": { 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nginx", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T21:29:11.687 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nginx container image", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T21:29:11.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.689 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_oauth2_proxy": { 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "oauth2-proxy container image", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T21:29:11.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T21:29:11.692 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba_metrics": { 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba_metrics", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-09-10T21:29:11.696 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-09-10T21:29:11.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T21:29:11.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-09-10T21:29:11.699 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-09-10T21:29:11.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-09-10T21:29:11.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-09-10T21:29:11.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.704 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-09-10T21:29:11.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-09-10T21:29:11.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.707 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.708 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.709 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.711 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T21:29:11.714 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T21:29:11.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-09-10T21:29:11.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-09-10T21:29:11.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.721 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.722 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.723 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.724 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.725 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.726 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.727 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.728 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.729 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.730 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.731 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.732 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.733 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.734 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.735 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.736 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.737 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.738 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.739 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.740 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.741 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T21:29:11.742 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.743 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-09-10T21:29:11.744 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.745 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.746 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.747 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.748 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.749 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.750 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.751 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.752 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.753 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.754 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.755 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.756 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.757 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.758 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.759 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.760 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.761 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.762 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.763 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.764 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.765 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.766 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.767 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.768 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.769 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.770 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.771 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.772 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.773 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.774 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.775 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.776 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.777 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.778 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.779 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.780 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T21:29:11.781 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.782 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T21:29:11.783 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T21:29:11.784 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T21:29:11.785 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.786 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.787 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.788 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.789 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.790 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.791 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.792 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.793 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.794 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.795 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.796 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.797 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.798 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.799 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.800 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.801 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.802 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.803 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.804 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.805 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.806 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.807 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-09-10T21:29:11.808 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-09-10T21:29:11.809 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.810 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-09-10T21:29:11.811 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-09-10T21:29:11.812 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.814 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.815 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.816 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.817 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.818 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.819 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.820 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.821 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.822 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.823 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.824 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.825 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.826 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.827 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.828 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.829 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.830 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.831 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.832 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.833 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.836 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.837 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.838 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-09-10T21:29:11.839 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.840 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.841 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.842 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.843 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.845 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.847 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.848 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.849 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.850 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.851 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.852 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-09-10T21:29:11.854 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.855 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.856 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.857 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.858 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.859 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.860 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.861 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.862 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.863 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.864 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.865 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.866 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.867 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.868 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-09-10T21:29:11.869 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-09-10T21:29:11.870 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.871 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.872 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.873 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.874 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.875 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.876 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.877 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.878 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.879 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.880 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.881 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.882 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.883 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.884 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-09-10T21:29:11.885 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.886 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.887 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.888 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.889 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.890 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-09-10T21:29:11.891 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.892 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.893 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.894 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.895 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.896 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.897 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.898 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.899 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.900 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.901 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.902 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.903 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.904 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.905 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.906 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.907 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.908 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.909 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.910 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.911 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.912 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.913 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.914 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.915 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.916 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.917 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.918 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.919 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.920 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.921 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.922 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.923 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.924 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.925 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.926 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.927 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.928 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.929 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.930 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.931 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.932 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.933 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.934 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.935 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.936 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.937 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.938 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.939 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.940 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.941 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.942 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.943 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.944 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.945 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.946 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.947 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.948 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.949 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:11.950 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.951 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-09-10T21:29:11.952 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.953 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.954 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.955 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.956 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.957 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.958 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.959 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.960 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.961 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.962 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-09-10T21:29:11.963 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.964 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.965 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.966 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.967 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.968 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T21:29:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T21:29:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.969 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.970 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.971 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.972 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.973 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.974 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.975 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.976 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-09-10T21:29:11.977 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-09-10T21:29:11.978 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.979 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.980 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.981 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.982 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.983 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.984 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-09-10T21:29:11.985 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.986 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.987 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.988 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.989 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:11.990 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.991 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.992 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.993 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.994 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:11.995 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.996 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.997 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:11.998 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:11.999 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.000 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.001 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.002 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.003 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.004 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.005 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.006 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max_requests": { 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_requests", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.007 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.008 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.009 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.010 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.011 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.012 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.013 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.014 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.015 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.016 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.017 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.018 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.019 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.020 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.021 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.022 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.023 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.024 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.025 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.026 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-09-10T21:29:12.027 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.028 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.029 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.030 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.031 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.032 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.033 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.034 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.035 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.036 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-09-10T21:29:12.037 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.038 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.039 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.040 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.041 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.042 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.043 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.044 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-09-10T21:29:12.045 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.046 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.047 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.048 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "internal_store_backend": { 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "internal_store_backend", 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.049 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.050 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.051 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.052 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.053 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.054 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.055 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.056 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.057 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.058 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.059 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.060 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.061 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.062 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.063 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.064 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.065 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.066 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.067 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.068 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.069 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.070 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.071 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.072 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.073 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.074 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.075 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.076 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.077 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.078 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.079 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.080 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.083 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.084 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.085 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.086 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.087 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.088 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.089 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.090 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.091 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.092 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.093 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.094 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.095 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.096 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-09-10T21:29:12.097 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-09-10T21:29:12.098 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.099 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.100 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.101 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.102 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.103 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.104 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.105 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.106 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-09-10T21:29:12.107 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-09-10T21:29:12.108 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.109 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.110 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.111 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.112 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.113 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.116 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.117 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.118 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-09-10T21:29:12.119 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.120 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.121 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.122 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T21:29:12.123 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-09-10T21:29:12.124 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.125 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.126 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.127 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.128 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.129 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.130 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.131 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.132 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.133 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.134 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.135 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.136 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.137 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.138 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.139 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.140 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-09-10T21:29:12.141 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T21:29:12.142 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.143 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.144 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.145 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.146 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.147 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.148 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.149 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.150 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.151 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.152 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.153 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.155 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.156 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.157 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.158 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "modules": [ 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard", 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "iostat", 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "nfs", 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "restful" 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "available_modules": [ 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "alerts", 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.159 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.160 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How frequently to reexamine health status", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.161 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.162 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.163 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.164 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.165 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_destination": { 2024-09-10T21:29:12.166 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_destination", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.167 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_from_name": { 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_from_name", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "Ceph", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Email From: name", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.168 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_host": { 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_host", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP server", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.169 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_password": { 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_password", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Password to authenticate with", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.170 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_port": { 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_port", 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "465", 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP port", 2024-09-10T21:29:12.171 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_sender": { 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_sender", 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.172 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SMTP envelope sender", 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_ssl": { 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_ssl", 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.173 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "smtp_user": { 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smtp_user", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.174 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "User to authenticate as", 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.175 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "balancer", 2024-09-10T21:29:12.176 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "active": { 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "active", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically balance PGs across cluster", 2024-09-10T21:29:12.177 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_time": { 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_time", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0000", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "beginning time of day to automatically balance", 2024-09-10T21:29:12.178 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "begin_weekday": { 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "begin_weekday", 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.179 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_max_iterations": { 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_max_iterations", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "25", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "250", 2024-09-10T21:29:12.180 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_metrics": { 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_metrics", 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "pgs,objects,bytes", 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.181 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "crush_compat_step": { 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crush_compat_step", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.5", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0.001", 2024-09-10T21:29:12.182 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "0.999", 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "aggressiveness of optimization", 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "end_time": { 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_time", 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2359", 2024-09-10T21:29:12.183 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ending time of day to automatically balance", 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "end_weekday": { 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "end_weekday", 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.184 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "0", 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "6", 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.187 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.188 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.189 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.190 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.191 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.192 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.193 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.194 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.195 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.196 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "min_score": { 2024-09-10T21:29:12.197 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_score", 2024-09-10T21:29:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.198 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-10T21:29:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.199 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "upmap", 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.200 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "crush-compat", 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "none", 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "read", 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap", 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap-read" 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Balancer mode", 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.201 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_ids": { 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_ids", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:12.202 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.203 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_deviation": { 2024-09-10T21:29:12.204 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_deviation", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.205 INFO:teuthology.orchestra.run.smithi033.stdout: "upmap_max_optimizations": { 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "upmap_max_optimizations", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.206 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm", 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_down_multiplier": { 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_down_multiplier", 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.207 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_refresh_rate": { 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_refresh_rate", 2024-09-10T21:29:12.208 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "20", 2024-09-10T21:29:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.209 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-10T21:29:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "agent_starting_port": { 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "agent_starting_port", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4721", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.211 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_ptrace": { 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_ptrace", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-10T21:29:12.212 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_interval": { 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_interval", 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.213 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "autotune_memory_target_ratio": { 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "autotune_memory_target_ratio", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0.7", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.214 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm_log_destination": { 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cephadm_log_destination", 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.215 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "file", 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "file,syslog", 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "syslog" 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.216 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "cgroups_split": { 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cgroups_split", 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.217 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "config_checks_enabled": { 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_checks_enabled", 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.218 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "config_dashboard": { 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_dashboard", 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.219 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_alertmanager": { 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_alertmanager", 2024-09-10T21:29:12.220 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_base": { 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_base", 2024-09-10T21:29:12.221 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Container image name, without the tag", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_elasticsearch": { 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_elasticsearch", 2024-09-10T21:29:12.222 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "elasticsearch container image", 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_grafana": { 2024-09-10T21:29:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_grafana", 2024-09-10T21:29:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-10T21:29:12.224 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_haproxy": { 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_haproxy", 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.225 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "HAproxy container image", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_agent": { 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_agent", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-10T21:29:12.226 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger agent container image", 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_collector": { 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_collector", 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.227 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger collector container image", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_jaeger_query": { 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_jaeger_query", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.228 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Jaeger query container image", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_keepalived": { 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_keepalived", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.229 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Keepalived container image", 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_loki": { 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_loki", 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.230 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Loki container image", 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.231 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nginx": { 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nginx", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nginx container image", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.232 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_node_exporter": { 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_node_exporter", 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:12.233 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_nvmeof": { 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_nvmeof", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Nvme-of container image", 2024-09-10T21:29:12.234 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_oauth2_proxy": { 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_oauth2_proxy", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "oauth2-proxy container image", 2024-09-10T21:29:12.235 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_prometheus": { 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_prometheus", 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.236 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Prometheus container image", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_promtail": { 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_promtail", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.237 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Promtail container image", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba": { 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.238 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB container image", 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_samba_metrics": { 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_samba_metrics", 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-10T21:29:12.239 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "container_image_snmp_gateway": { 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_image_snmp_gateway", 2024-09-10T21:29:12.240 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "SNMP Gateway container image", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "container_init": { 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "container_init", 2024-09-10T21:29:12.241 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Run podman/docker with `--init`", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "daemon_cache_timeout": { 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "daemon_cache_timeout", 2024-09-10T21:29:12.242 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.243 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_cephadm_command_timeout": { 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_cephadm_command_timeout", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "900", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.244 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_registry": { 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "default_registry", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "docker.io", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.245 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "device_cache_timeout": { 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_cache_timeout", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1800", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache device inventory", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.246 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "device_enhanced_scan": { 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_enhanced_scan", 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.247 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "facts_cache_timeout": { 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "facts_cache_timeout", 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.248 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "seconds to cache host facts data", 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "grafana_dashboards_path": { 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "grafana_dashboards_path", 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-10T21:29:12.249 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "host_check_interval": { 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "host_check_interval", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:12.250 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to perform a host check", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "hw_monitoring": { 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hw_monitoring", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.251 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "inventory_list_all": { 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "inventory_list_all", 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.252 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.253 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.254 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "log_refresh_metadata": { 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_refresh_metadata", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.255 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.256 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.257 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.258 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.259 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.260 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "*", 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.261 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "max_count_per_host": { 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_count_per_host", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.262 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of daemons per service per host", 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "max_osd_draining_count": { 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_osd_draining_count", 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.263 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-10T21:29:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.264 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "migration_current": { 2024-09-10T21:29:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "migration_current", 2024-09-10T21:29:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.265 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "internal - do not modify", 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.266 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "mode": { 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mode", 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "root", 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.267 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm-package", 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "root" 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "mode for remote execution of cephadm", 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.268 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "oob_default_addr": { 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "oob_default_addr", 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "169.254.1.1", 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.269 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "prometheus_alerts_path": { 2024-09-10T21:29:12.270 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus_alerts_path", 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.271 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_insecure": { 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_insecure", 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.272 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.273 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_password": { 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_password", 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.274 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.275 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_url": { 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_url", 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.276 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-10T21:29:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "registry_username": { 2024-09-10T21:29:12.277 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "registry_username", 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-10T21:29:12.278 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "secure_monitoring_stack": { 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secure_monitoring_stack", 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.279 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "service_discovery_port": { 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "service_discovery_port", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.280 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8765", 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "cephadm service discovery port", 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.281 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "ssh_config_file": { 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssh_config_file", 2024-09-10T21:29:12.282 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "use_agent": { 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_agent", 2024-09-10T21:29:12.283 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "use_repo_digest": { 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "use_repo_digest", 2024-09-10T21:29:12.284 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_failed_host_check": { 2024-09-10T21:29:12.285 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_failed_host_check", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if the host check fails", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.286 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_daemons": { 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_daemons", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.287 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_on_stray_hosts": { 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_on_stray_hosts", 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-10T21:29:12.288 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crash", 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.289 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.290 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.291 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.292 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.293 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.294 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "retain_interval": { 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retain_interval", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "31536000", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.295 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.296 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_recent_interval": { 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_recent_interval", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1209600", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.297 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dashboard", 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.298 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_HOST": { 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.299 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.300 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_ENABLED": { 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_ENABLED", 2024-09-10T21:29:12.301 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-10T21:29:12.302 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.303 INFO:teuthology.orchestra.run.smithi033.stdout: "ENABLE_BROWSABLE_API": { 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.304 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.305 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.306 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.307 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.308 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.309 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.310 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.311 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.312 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_PASSWORD": { 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.313 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.314 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_URL": { 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_URL", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.315 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_API_USERNAME": { 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:12.316 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.317 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.320 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.321 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.322 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.323 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.324 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "[]", 2024-09-10T21:29:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.325 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "MULTICLUSTER_CONFIG": { 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "{}", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.326 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_HOST": { 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.327 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.328 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.329 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.330 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.331 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.332 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.333 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.334 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.335 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.336 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_ENABLED": { 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.337 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.338 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-10T21:29:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-10T21:29:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.339 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.340 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.341 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.342 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "45", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.343 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ACCESS_KEY": { 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.344 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "admin", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.345 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SECRET_KEY": { 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.346 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "RGW_API_SSL_VERIFY": { 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.347 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "UNSAFE_TLS_v1_2": { 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.348 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.349 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.350 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.351 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "cross_origin_url": { 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cross_origin_url", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.352 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "crt_file": { 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "crt_file", 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.353 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.354 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "debug": { 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "debug", 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.355 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Enable/disable debug options", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "jwt_token_ttl": { 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "jwt_token_ttl", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "28800", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.356 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.357 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.358 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.359 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.360 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.361 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.362 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.363 INFO:teuthology.orchestra.run.smithi033.stdout: "motd": { 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "motd", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "The message of the day", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.364 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect_resolve_ip_addr": { 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "redirect_resolve_ip_addr", 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.365 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.366 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8080", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.367 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.368 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.369 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl_server_port": { 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl_server_port", 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8443", 2024-09-10T21:29:12.370 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.371 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "redirect", 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "redirect" 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.372 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.373 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "url_prefix": { 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url_prefix", 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.374 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_monitoring": { 2024-09-10T21:29:12.375 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_monitoring", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "monitor device health metrics", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.376 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.377 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.378 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.379 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.380 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.381 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "mark_out_threshold": { 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mark_out_threshold", 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "2419200", 2024-09-10T21:29:12.382 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "pool_name": { 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pool_name", 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.383 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "device_health_metrics", 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "retention_period": { 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "retention_period", 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.384 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15552000", 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long to retain device health metrics", 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_frequency": { 2024-09-10T21:29:12.385 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_frequency", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.386 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "self_heal": { 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "self_heal", 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.387 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.388 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how frequently to wake up and check device health", 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.389 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "warn_threshold": { 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "warn_threshold", 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.390 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7257600", 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "diskprediction_local", 2024-09-10T21:29:12.391 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.392 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.393 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.394 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "predict_interval": { 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predict_interval", 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.397 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "86400", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "predictor_model": { 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "predictor_model", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.398 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "prophetstor", 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.399 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "600", 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.400 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.401 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "influx", 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": false, 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "influxdb python module not found", 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "batch_size": { 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "batch_size", 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.402 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5000", 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "database": { 2024-09-10T21:29:12.403 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "database", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "ceph", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": { 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "hostname", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server hostname", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.405 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "30", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "5", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.406 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.407 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.408 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.409 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.410 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.411 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.412 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "password": { 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "password", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "password of InfluxDB server user", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.413 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "port": { 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "port", 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "8086", 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "InfluxDB server port", 2024-09-10T21:29:12.414 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.415 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "ssl": { 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ssl", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "false", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T21:29:12.416 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "threads": { 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threads", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "32", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-10T21:29:12.417 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "username": { 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "username", 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.418 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "username of InfluxDB server user", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "verify_ssl": { 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "verify_ssl", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "true", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.419 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "insights", 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.420 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.421 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.422 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.423 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.424 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.425 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.426 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.427 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "iostat", 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.428 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.429 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.430 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.431 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.432 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.433 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.434 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "k8sevents", 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.435 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_event_retention_days": { 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "ceph_event_retention_days", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "7", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.436 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "config_check_secs": { 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "config_check_secs", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "10", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.437 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.438 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.439 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.440 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.441 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.442 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.443 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.444 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.445 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "localpool", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "failure_domain": { 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "failure_domain", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "host", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "failure domain for any created local pool", 2024-09-10T21:29:12.446 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.447 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.448 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.449 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.450 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.452 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min_size": { 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "min_size", 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.453 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default min_size for any created local pool", 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "num_rep": { 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "num_rep", 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3", 2024-09-10T21:29:12.454 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default replica count for any created local pool", 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_num": { 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_num", 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "128", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "default pg_num for any created local pool", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "prefix": { 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prefix", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.456 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "name prefix for any created local pool", 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "subtree": { 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "subtree", 2024-09-10T21:29:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "rack", 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mds_autoscaler", 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.460 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.461 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.462 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.463 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.464 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.465 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.466 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirroring", 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.467 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.468 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.469 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.470 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.471 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.472 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.473 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.474 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "nfs", 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.475 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.476 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.477 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.478 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.479 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.480 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.481 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "fail_fs": { 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "fail_fs", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.482 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.483 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.484 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.485 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.486 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator": { 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "orchestrator", 2024-09-10T21:29:12.487 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "cephadm", 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "rook", 2024-09-10T21:29:12.488 INFO:teuthology.orchestra.run.smithi033.stdout: "test_orchestrator" 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Orchestrator backend", 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.489 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_perf_query", 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.490 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.491 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.492 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.493 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.494 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.495 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.496 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.497 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "osd_support", 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.498 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.499 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.500 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.501 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.502 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.504 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "pg_autoscaler", 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.505 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.506 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.507 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.508 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.509 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.510 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.511 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.512 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.513 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.514 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.515 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.516 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "threshold": { 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "threshold", 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "3.0", 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1.0", 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.517 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "scaling threshold", 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "progress", 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_pg_recovery_event": { 2024-09-10T21:29:12.518 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_pg_recovery_event", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.519 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.520 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.521 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.522 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.523 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.524 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.525 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.526 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.528 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.529 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.530 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "max_completed_events": { 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_completed_events", 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "50", 2024-09-10T21:29:12.531 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "number of past completed events to remember", 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.532 INFO:teuthology.orchestra.run.smithi033.stdout: "sleep_interval": { 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sleep_interval", 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.533 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "how long the module is going to sleep", 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.534 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.535 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "prometheus", 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "cache": { 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "cache", 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.536 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "exclude_perf_counters": { 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "exclude_perf_counters", 2024-09-10T21:29:12.537 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.538 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.539 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.540 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.541 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.542 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.543 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools": { 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools", 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.544 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "300", 2024-09-10T21:29:12.545 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "scrape_interval": { 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "scrape_interval", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15.0", 2024-09-10T21:29:12.546 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "::", 2024-09-10T21:29:12.547 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "9283", 2024-09-10T21:29:12.548 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.549 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "stale_cache_strategy": { 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stale_cache_strategy", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.550 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "log", 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.551 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_behaviour": { 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_behaviour", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "default", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "default", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "error" 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.552 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "standby_error_status_code": { 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "standby_error_status_code", 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "400", 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "599", 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.553 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.554 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.555 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.556 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.557 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.558 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_snap_create": { 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_snap_create", 2024-09-10T21:29:12.559 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10", 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "mirror_snapshot_schedule": { 2024-09-10T21:29:12.560 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "mirror_snapshot_schedule", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.561 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.562 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "trash_purge_schedule": { 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "trash_purge_schedule", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.563 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "restful", 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "enable_auth": { 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enable_auth", 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.564 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.565 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "key_file": { 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "key_file", 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.566 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.567 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.568 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.569 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.570 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.571 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "max_requests": { 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_requests", 2024-09-10T21:29:12.572 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "500", 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "server_addr": { 2024-09-10T21:29:12.573 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_addr", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "server_port": { 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "server_port", 2024-09-10T21:29:12.574 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.575 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rgw", 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.577 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.579 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.580 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.581 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.582 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "secondary_zone_period_retry_limit": { 2024-09-10T21:29:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-10T21:29:12.583 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "5", 2024-09-10T21:29:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.584 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-10T21:29:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.585 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.586 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.587 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rook", 2024-09-10T21:29:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.588 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.589 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.590 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.591 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.592 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.594 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.595 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "storage_class": { 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "storage_class", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.596 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "local", 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.597 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "selftest", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.598 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.599 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.600 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.601 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.602 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "roption1": { 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption1", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.603 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "roption2": { 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "roption2", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.604 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "xyz", 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption1": { 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption1", 2024-09-10T21:29:12.605 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption2": { 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption2", 2024-09-10T21:29:12.606 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption3": { 2024-09-10T21:29:12.607 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption3", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "float", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.608 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption4": { 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption4", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.609 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption5": { 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption5", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.610 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption6": { 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption6", 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.611 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "rwoption7": { 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rwoption7", 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "1", 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "42", 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.612 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.613 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "testkey": { 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testkey", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.614 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "testlkey": { 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testlkey", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.615 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "testnewline": { 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "testnewline", 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.616 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "smb", 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.617 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "internal_store_backend": { 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "internal_store_backend", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.618 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.619 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.620 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.621 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.622 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.623 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.624 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "update_orchestration": { 2024-09-10T21:29:12.625 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "update_orchestration", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.626 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snap_schedule", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "allow_m_granularity": { 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "allow_m_granularity", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.627 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "allow minute scheduled snapshots", 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "dump_on_update": { 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "dump_on_update", 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.628 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "dump database to debug log on update", 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.629 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.630 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.631 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.632 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.633 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.634 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.635 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.636 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "stats", 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.637 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.638 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.639 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.640 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.641 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.642 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.643 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.644 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.645 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "status", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.646 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.647 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.648 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.649 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.650 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.651 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.652 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telegraf", 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "address": { 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "address", 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.653 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.654 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "15", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.655 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.656 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.657 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.658 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.659 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.660 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.661 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "telemetry", 2024-09-10T21:29:12.662 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_basic": { 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_basic", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-10T21:29:12.663 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_crash": { 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_crash", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-10T21:29:12.664 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_device": { 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_device", 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.665 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_ident": { 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_ident", 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.666 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "channel_perf": { 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "channel_perf", 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.667 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "contact": { 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "contact", 2024-09-10T21:29:12.668 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "description": { 2024-09-10T21:29:12.669 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "description", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "device_url": { 2024-09-10T21:29:12.670 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "device_url", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.671 INFO:teuthology.orchestra.run.smithi033.stdout: "enabled": { 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "enabled", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.672 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "24", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "8", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.673 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "last_opt_revision": { 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "last_opt_revision", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "1", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.674 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard": { 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.675 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "leaderboard_description": { 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "leaderboard_description", 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.676 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.677 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.678 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.679 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.680 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.681 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.682 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "organization": { 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "organization", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.683 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "proxy": { 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "proxy", 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.684 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.685 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "url": { 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "url", 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.686 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "test_orchestrator", 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.687 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.688 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.689 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.690 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.691 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.692 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.693 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.694 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.695 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.696 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.697 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.698 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.699 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.700 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max_concurrent_clones": { 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "max_concurrent_clones", 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "4", 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.701 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "periodic_async_work": { 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "periodic_async_work", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Periodically check for async work", 2024-09-10T21:29:12.702 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_delay": { 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_delay", 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.703 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "snapshot_clone_no_wait": { 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "snapshot_clone_no_wait", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "True", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.704 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.705 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix", 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "can_run": true, 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "error_string": "", 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "module_options": { 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "discovery_interval": { 2024-09-10T21:29:12.706 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "discovery_interval", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "uint", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "100", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "identifier": { 2024-09-10T21:29:12.707 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "identifier", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.708 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "interval": { 2024-09-10T21:29:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "interval", 2024-09-10T21:29:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "secs", 2024-09-10T21:29:12.709 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "60", 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "log_level": { 2024-09-10T21:29:12.710 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_level", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.711 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster": { 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster", 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.712 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_cluster_level": { 2024-09-10T21:29:12.713 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_cluster_level", 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "info", 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [ 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "", 2024-09-10T21:29:12.714 INFO:teuthology.orchestra.run.smithi033.stdout: "critical", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "debug", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "error", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "info", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "warning" 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "log_to_file": { 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "log_to_file", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "bool", 2024-09-10T21:29:12.715 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "False", 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "sqlite3_killpoint": { 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "sqlite3_killpoint", 2024-09-10T21:29:12.716 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "dev", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 1, 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "0", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_host": { 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_host", 2024-09-10T21:29:12.717 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "", 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_port": { 2024-09-10T21:29:12.718 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_port", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "int", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "10051", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "zabbix_sender": { 2024-09-10T21:29:12.719 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "zabbix_sender", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "str", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "level": "advanced", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "flags": 0, 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "min": "", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "max": "", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "enum_allowed": [], 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "desc": "", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "long_desc": "", 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "tags": [], 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: "see_also": [] 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.720 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "services": { 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "dashboard": "https://172.21.15.33:8443/" 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "always_on_modules": { 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "octopus": [ 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-09-10T21:29:12.721 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "pacific": [ 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-09-10T21:29:12.722 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "quincy": [ 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-09-10T21:29:12.723 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "reef": [ 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-09-10T21:29:12.724 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "squid": [ 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "balancer", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "devicehealth", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "orchestrator", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "pg_autoscaler", 2024-09-10T21:29:12.725 INFO:teuthology.orchestra.run.smithi033.stdout: "progress", 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "rbd_support", 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "status", 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "telemetry", 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "volumes" 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "force_disabled_modules": {}, 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "last_failure_osd_epoch": 3, 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "active_clients": [ 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "devicehealth", 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.726 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2582306509 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "libcephsqlite", 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 4010722974 2024-09-10T21:29:12.727 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "rbd_support", 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 1011618733 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:12.728 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "name": "volumes", 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "addrvec": [ 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: { 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v2", 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "172.21.15.33:0", 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 2607078036 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.729 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:12.730 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-09-10T21:29:12.730 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:29:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52540b52e0 msgr2=0x7f52540b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52540b52e0 0x7f52540b77a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f525c005fd0 tx=0x7f525c009680 comp rx=0 tx=0).stop 2024-09-10T21:29:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c1033c0 msgr2=0x7f526c19a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c1033c0 0x7f526c19a070 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f526400d750 tx=0x7f526400dc10 comp rx=0 tx=0).stop 2024-09-10T21:29:12.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 shutdown_connections 2024-09-10T21:29:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f52540b52e0 0x7f52540b77a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f526c1033c0 0x7f526c19a070 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f526c102a00 0x7f526c199440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 --2- 172.21.15.33:0/843212426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f526c1018b0 0x7f526c198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 >> 172.21.15.33:0/843212426 conn(0x7f526c0fd190 msgr2=0x7f526c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 shutdown_connections 2024-09-10T21:29:12.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:11.096+0000 7f5271f5a640 1 -- 172.21.15.33:0/843212426 wait complete. 2024-09-10T21:29:12.743 DEBUG:tasks.check_counter:Getting stats from a 2024-09-10T21:29:12.744 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-10T21:29:12.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:11 smithi033 ceph-mon[30860]: pgmap v1102: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 15 MiB/s wr, 692 op/s 2024-09-10T21:29:12.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:11 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3455782195' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T21:29:12.744 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:11 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/843212426' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T21:29:12.745 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:11 smithi104 ceph-mon[35325]: pgmap v1102: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 15 MiB/s wr, 692 op/s 2024-09-10T21:29:12.745 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:11 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3455782195' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T21:29:12.745 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:11 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/843212426' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T21:29:12.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:11 smithi174 ceph-mon[35635]: pgmap v1102: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 15 MiB/s wr, 692 op/s 2024-09-10T21:29:12.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:11 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3455782195' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-10T21:29:12.746 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:11 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/843212426' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-10T21:29:13.017 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:13.549 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 3108, 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 3110, 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 177082317, 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 988345, 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 8, 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.434015942, 2024-09-10T21:29:13.552 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.127811288, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.198628352, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.083304000, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 3108, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.108451842, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000034894 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.553 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 1974, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2303, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 167260476, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 1398916, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 8, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.386778416, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.089125855, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.178623711, 2024-09-10T21:29:13.554 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.102688634, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2302, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.090050583, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000039118 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 773024 2024-09-10T21:29:13.555 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 1537, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 1460, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 118803078, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 594624, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.225204559, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.071703108, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.126728021, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.026197733, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1459, 2024-09-10T21:29:13.556 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.052537958, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000036009 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1104672, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 271536 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-09-10T21:29:13.557 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 10331, 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.274118167, 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000026533 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-10T21:29:13.558 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.559 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 44873, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 15677, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 29196, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-09-10T21:29:13.560 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 22476, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-09-10T21:29:13.561 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T21:29:13.562 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:13.563 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-09-10T21:29:13.564 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:13.565 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_write_failed": 0 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-09-10T21:29:13.566 INFO:teuthology.orchestra.run.smithi033.stdout: "evlrg": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 235834, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 136, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "segmjr": 21, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-09-10T21:29:13.567 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 230543972, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 464553696, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 464552558, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 235834 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-09-10T21:29:13.568 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 27, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 44843, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 44816, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 24, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 159, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 135, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 44873, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 45564, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 691, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-09-10T21:29:13.569 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 90948, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 296321024 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.570 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-09-10T21:29:13.571 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.572 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:13.573 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.574 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.575 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-09-10T21:29:13.576 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:13.577 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.578 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.579 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-09-10T21:29:13.580 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.581 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-09-10T21:29:13.582 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 2, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 1, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 1, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 2556, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:13.583 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:13.584 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:13.585 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 241517, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 96, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:13.586 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 156, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 30608474, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 181152, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-10T21:29:13.587 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 6092, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 6092, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 6092, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 22.391649737, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.003675582 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-09-10T21:29:13.588 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 6092, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 6093 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 6092, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 6092, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 3216, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 2874, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-09-10T21:29:13.589 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-09-10T21:29:13.590 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-09-10T21:29:13.591 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 77, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 2, 2024-09-10T21:29:13.592 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 11, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 11, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 0, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-09-10T21:29:13.593 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed_ops": 0, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:13.594 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 6615, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 462521014, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 6615, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 6615, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 462521014, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.595 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 6092, 2024-09-10T21:29:13.596 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 460359859, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 6092, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 460359859, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:13.597 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 6092, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 6092, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 6092, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 6092, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.598 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-09-10T21:29:13.599 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x55af09cf5040": { 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:13.600 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:13.601 INFO:teuthology.orchestra.run.smithi033.stdout: "trackedop": { 2024-09-10T21:29:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-09-10T21:29:13.602 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:13.602 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:29:13.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:13 smithi174 ceph-mon[35635]: pgmap v1103: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 606 op/s 2024-09-10T21:29:13.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:13 smithi033 ceph-mon[30860]: pgmap v1103: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 606 op/s 2024-09-10T21:29:14.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:13 smithi104 ceph-mon[35325]: pgmap v1103: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 14 MiB/s wr, 606 op/s 2024-09-10T21:29:14.426 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-10T21:29:14.426 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-10T21:29:14.426 DEBUG:tasks.check_counter:Getting stats from d 2024-09-10T21:29:14.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-10T21:29:14.686 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:15.199 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:29:15.199 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:15.199 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-09-10T21:29:15.199 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-09-10T21:29:15.199 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001408555, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000252818, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:15.200 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-09-10T21:29:15.201 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 338, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 220224, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.028028661, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.023479911, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001533221, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 336, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.015119248, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000044997 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.202 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 220224 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 449, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 444, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 592384, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 274000, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-09-10T21:29:15.203 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.061299812, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.032988191, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.031308844, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 444, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.022729880, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000051193 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:15.204 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 592384, 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 274000 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-10T21:29:15.205 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:15.206 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:15.207 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148738, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 31, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:15.208 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 968, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-09-10T21:29:15.209 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 447, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 526490, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 447, 2024-09-10T21:29:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 447, 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 526490, 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:15.211 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:29:15.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:15 smithi174 ceph-mon[35635]: pgmap v1104: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 12 MiB/s wr, 509 op/s 2024-09-10T21:29:15.909 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:15 smithi033 ceph-mon[30860]: pgmap v1104: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 12 MiB/s wr, 509 op/s 2024-09-10T21:29:15.943 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-10T21:29:15.943 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-10T21:29:15.943 DEBUG:tasks.check_counter:Getting stats from g 2024-09-10T21:29:15.943 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-10T21:29:16.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:15 smithi104 ceph-mon[35325]: pgmap v1104: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 12 MiB/s wr, 509 op/s 2024-09-10T21:29:16.208 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:16.789 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:29:16.789 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:16.789 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2680, 2024-09-10T21:29:16.789 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2680, 2024-09-10T21:29:16.789 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 101478815, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 851417, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 9, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.337692789, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.130470786, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.118439794, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.067780550, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2679, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.105250276, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000039287 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:16.790 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2177, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2510, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 116791728, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 1376249, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 7, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:16.791 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.285218642, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.088886571, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.132139470, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.049059891, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 2508, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.077145966, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000030759 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.792 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 684224 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 1636, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 1559, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 118830300, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 626439, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 5, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 4, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.241129128, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.071929549, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.139618505, 2024-09-10T21:29:16.793 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.028247455, 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 1558, 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.056818074, 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000036468 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.794 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 1104944, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 272176 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 1, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-MDSRank": { 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 10228, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.256329130, 2024-09-10T21:29:16.795 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000025061 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "queue_len": 0, 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "complete_latency": { 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "request": 0, 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "reply": 0, 2024-09-10T21:29:16.796 INFO:teuthology.orchestra.run.smithi033.stdout: "reply_latency": { 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_reply": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "forward": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_commit": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_split": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_merge": 0, 2024-09-10T21:29:16.797 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes": 38104, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_top": 12639, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_bottom": 25465, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pin_tail": 0, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_pinned": 20048, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_expired": 0, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "inodes_with_caps": 0, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "caps": 0, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "subtrees": 1, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse": 0, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_hit": 0, 2024-09-10T21:29:16.798 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_forward": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_discover": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "traverse_lock": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "load_cent": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "q": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "exported": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "exported_inodes": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "imported": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "imported_inodes": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_backtrace_fetch": 0, 2024-09-10T21:29:16.799 INFO:teuthology.orchestra.run.smithi033.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rfiles": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rbytes": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "root_rsnaps": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_inodes": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "scrub_file_inodes": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_inode_file_caps": 0, 2024-09-10T21:29:16.800 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_grant": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_caps_dirty": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_cap_release": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "process_request_cap_release": 0 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_cache": { 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_update_receipt": 0, 2024-09-10T21:29:16.801 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_try_discover": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_send_discover": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "dir_handle_discover": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_created": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_enqueued": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "strays_migrated": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_started": 0, 2024-09-10T21:29:16.802 INFO:teuthology.orchestra.run.smithi033.stdout: "recovery_completed": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_exportdir": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_flush": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragmentdir": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_started": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "uninline_write_failed": 0 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_log": { 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "evlrg": 0, 2024-09-10T21:29:16.803 INFO:teuthology.orchestra.run.smithi033.stdout: "evadd": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "evex": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "evtrm": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "ev": 201450, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "evexg": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "evexd": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "segadd": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "segex": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "segtrm": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "seg": 133, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "segmjr": 18, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "segexg": 0, 2024-09-10T21:29:16.804 INFO:teuthology.orchestra.run.smithi033.stdout: "segexd": 0, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "expos": 156972059, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "wrpos": 338537264, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "rdpos": 338534980, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "jlat": { 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "replayed": 201450 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_mem": { 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "ino": 16, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "ino+": 38144, 2024-09-10T21:29:16.805 INFO:teuthology.orchestra.run.smithi033.stdout: "ino-": 38128, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dir": 14, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dir+": 138, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dir-": 124, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dn": 38104, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dn+": 38601, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dn-": 497, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "cap": 0, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "cap+": 0, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "cap-": 0, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "rss": 83520, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "heap": 286810112 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_server": { 2024-09-10T21:29:16.806 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_client_request": 0, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "dispatch_server_request": 0, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_request": 0, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_client_session": 0, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "handle_peer_request": 0, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "req_create_latency": { 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getattr_latency": { 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.807 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "req_link_latency": { 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookup_latency": { 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.808 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupino_latency": { 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.809 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupname_latency": { 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.810 INFO:teuthology.orchestra.run.smithi033.stdout: "req_lssnap_latency": { 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mkdir_latency": { 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mknod_latency": { 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.811 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "req_mksnap_latency": { 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "req_open_latency": { 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "req_readdir_latency": { 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.812 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rename_latency": { 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:16.813 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmdir_latency": { 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.814 INFO:teuthology.orchestra.run.smithi033.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setattr_latency": { 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.815 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setlayout_latency": { 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "req_setxattr_latency": { 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.816 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "req_symlink_latency": { 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "req_unlink_latency": { 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.817 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_sessions": { 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "session_count": 1, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "session_add": 2, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "session_remove": 1, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_open": 1, 2024-09-10T21:29:16.818 INFO:teuthology.orchestra.run.smithi033.stdout: "sessions_stale": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "total_load": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "average_load": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "avg_session_uptime": 2558, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:16.819 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:16.820 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-10T21:29:16.821 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 240326, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 83, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:16.822 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 156, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 25966360, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 152912, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "objecter": { 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "op_active": 0, 2024-09-10T21:29:16.823 INFO:teuthology.orchestra.run.smithi033.stdout: "op_laggy": 0, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send": 5966, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "op_send_bytes": 21, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "op_resend": 3, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "op_reply": 5966, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "op_latency": { 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5966, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 18.904704382, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.003168740 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "op_inflight": 0, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "oplen_avg": { 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 5966, 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 5967 2024-09-10T21:29:16.824 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "op": 5966, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "op_r": 5966, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "op_w": 0, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "op_rmw": 0, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "op_pg": 0, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_stat": 3139, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_create": 0, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_read": 2825, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_write": 0, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writefull": 0, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_writesame": 0, 2024-09-10T21:29:16.825 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_append": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_zero": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_truncate": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_delete": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_mapext": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_getxattr": 1, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_setxattr": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:16.826 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_call": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_watch": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_notify": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "osdop_other": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_active": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_send": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_resend": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "linger_ping": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_active": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_send": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "poolop_resend": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_active": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_send": 0, 2024-09-10T21:29:16.827 INFO:teuthology.orchestra.run.smithi033.stdout: "poolstat_resend": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_active": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_send": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "statfs_resend": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "command_active": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "command_send": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "command_resend": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "map_epoch": 77, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "map_full": 0, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "map_inc": 2, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_sessions": 11, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_open": 12, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_session_close": 1, 2024-09-10T21:29:16.828 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_laggy": 0, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_wr": 0, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_rd": 2, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_del": 0 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "oft": { 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_objs": 0, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_updates": 0, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "omap_total_removes": 0 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "purge_queue": { 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T21:29:16.829 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing": 0, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executing_high_water": 0, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed_ops": 0, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_executed": 0, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 6489, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 336487317, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 6489, 2024-09-10T21:29:16.830 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 6489, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 336487317, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:16.831 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 5966, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 334346741, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 5966, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 334346741, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.832 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 1024, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 5966, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 5966, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 5966, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 5966, 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:16.833 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:16.834 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-write_buf_throttle-0x55fc779d9040": { 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 3758096384, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 0, 2024-09-10T21:29:16.835 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:16.836 INFO:teuthology.orchestra.run.smithi033.stdout: "trackedop": { 2024-09-10T21:29:16.837 INFO:teuthology.orchestra.run.smithi033.stdout: "slow_ops_count": 0 2024-09-10T21:29:16.837 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:16.837 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:29:17.626 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-10T21:29:17.626 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-10T21:29:17.626 DEBUG:tasks.check_counter:Getting stats from j 2024-09-10T21:29:17.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-10T21:29:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:17.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:17 smithi174 ceph-mon[35635]: pgmap v1105: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-10T21:29:17.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:17 smithi033 ceph-mon[30860]: pgmap v1105: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-10T21:29:18.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:17 smithi104 ceph-mon[35325]: pgmap v1105: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-10T21:29:18.424 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 0, 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 2, 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 0, 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 0, 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 0, 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.001637146, 2024-09-10T21:29:18.425 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.000000000, 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.000299350, 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:18.426 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 2, 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 339, 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 320, 2024-09-10T21:29:18.427 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 220400, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 4, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 2, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.028161278, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.023153428, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.001976218, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 337, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.017602000, 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000052231 2024-09-10T21:29:18.428 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 220400 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:18.429 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_messages": 451, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages": 446, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_bytes": 592928, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_bytes": 275280, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_created_connections": 2, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_active_connections": 1, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_total_time": 0.062277305, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_send_time": 0.033281907, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_recv_time": 0.032271087, 2024-09-10T21:29:18.430 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 446, 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.023027095, 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000051630 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:18.431 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_recv_encrypted_bytes": 592928, 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "msgr_send_encrypted_bytes": 275280 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "cct": { 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "total_workers": 0, 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "unhealthy_workers": 0 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "mempool": { 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:18.432 INFO:teuthology.orchestra.run.smithi033.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:18.433 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:18.434 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_items": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_bytes": 148744, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_anon_items": 30, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_bytes": 0, 2024-09-10T21:29:18.435 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_items": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_bytes": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_items": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "pgmap_items": 0, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_bytes": 968, 2024-09-10T21:29:18.436 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_co_items": 2, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_1_items": 0, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "unittest_2_items": 0 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "val": 0, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "max": 104857600, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "get_started": 0, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "get": 449, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "get_sum": 526750, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "get_or_fail_success": 449, 2024-09-10T21:29:18.437 INFO:teuthology.orchestra.run.smithi033.stdout: "take": 0, 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "take_sum": 0, 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "put": 449, 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "put_sum": 526750, 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "wait": { 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "avgcount": 0, 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "sum": 0.000000000, 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: "avgtime": 0.000000000 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-09-10T21:29:18.438 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-09-10T21:29:19.128 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-10T21:29:19.128 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-10T21:29:19.128 DEBUG:tasks.check_counter:Getting stats from b 2024-09-10T21:29:19.128 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-10T21:29:19.431 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:29:19.810 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:19 smithi104 ceph-mon[35325]: pgmap v1106: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 48 KiB/s wr, 78 op/s 2024-09-10T21:29:19.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:19 smithi174 ceph-mon[35635]: pgmap v1106: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 48 KiB/s wr, 78 op/s 2024-09-10T21:29:19.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:19 smithi033 ceph-mon[30860]: pgmap v1106: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 48 KiB/s wr, 78 op/s 2024-09-10T21:29:20.000 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-10T21:29:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 810831, 2024-09-10T21:29:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1006718, 2024-09-10T21:29:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 240195546, 2024-09-10T21:29:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 877556016, 2024-09-10T21:29:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 15, 2024-09-10T21:29:20.001 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 58.092075299, 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 31.026108101, 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 28.044831668, 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 2.923178432, 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1006714, 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 56.498090559, 2024-09-10T21:29:20.002 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000056121 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1558115, 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.270445186, 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000173 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 58784, 2024-09-10T21:29:20.003 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 30192 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 245799, 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 272675, 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 63368376, 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 269860461, 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 11, 2024-09-10T21:29:20.004 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 22.088140673, 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 11.210858778, 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 8.763004522, 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 3.085329208, 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 272676, 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 11.899353949, 2024-09-10T21:29:20.005 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000043639 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 380858, 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.059748576, 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000156 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 695888 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:20.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 380122, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 466423, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 89648421, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 258001493, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 255, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 8, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 26.639583102, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 12.890322518, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 12.967329714, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 2.449116221, 2024-09-10T21:29:20.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 466422, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 20.988838810, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000044999 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 755840, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.109870769, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000145 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1150608, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 318912 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.008 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 208821, 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 194.480720706, 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000931327 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.009 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 83835, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 9.425370116, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000112427 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 338345, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 305916, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 305916, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1665.024890726, 2024-09-10T21:29:20.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.005442751 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 32429, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 1, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 98, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 6, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 6, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 170, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 69, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 75, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 26, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 70, 2024-09-10T21:29:20.011 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 2785, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 1, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 1, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 6, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 564438, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 329972, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 32420, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 169, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 4575, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 41, 2024-09-10T21:29:20.012 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 30, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 1471, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 30, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 493, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 3488, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 3488, 2024-09-10T21:29:20.013 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 881, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 31, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 850, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 2607, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 63, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 62379, 2024-09-10T21:29:20.014 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 176584, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 177067, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 39279, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 16592, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 189842 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 38, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 226, 2024-09-10T21:29:20.015 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 121, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 364, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 519, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 15, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 44987, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 44972, 2024-09-10T21:29:20.016 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-10T21:29:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-10T21:29:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-10T21:29:20.017 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 31, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 31, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 12, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_started": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_write_failed": 0 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-10T21:29:20.018 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 270328, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 176356, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 172061, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 98267, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 4295, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 351, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 223, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 217, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 134, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 8, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 6, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 352143986, 2024-09-10T21:29:20.019 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 539614046, 2024-09-10T21:29:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-09-10T21:29:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-10T21:29:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 113945, 2024-09-10T21:29:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1162.455172931, 2024-09-10T21:29:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.010201897 2024-09-10T21:29:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.020 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-09-10T21:29:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-10T21:29:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 62, 2024-09-10T21:29:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 46283, 2024-09-10T21:29:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 46221, 2024-09-10T21:29:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 57, 2024-09-10T21:29:20.021 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 390, 2024-09-10T21:29:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 333, 2024-09-10T21:29:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 196, 2024-09-10T21:29:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 88267, 2024-09-10T21:29:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 88071, 2024-09-10T21:29:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 1, 2024-09-10T21:29:20.022 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 51074, 2024-09-10T21:29:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 51073, 2024-09-10T21:29:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 87812, 2024-09-10T21:29:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 307089408 2024-09-10T21:29:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-10T21:29:20.023 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 500801, 2024-09-10T21:29:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 44222, 2024-09-10T21:29:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 338345, 2024-09-10T21:29:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 6385, 2024-09-10T21:29:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 99398, 2024-09-10T21:29:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-10T21:29:20.024 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 45395, 2024-09-10T21:29:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 180.168612306, 2024-09-10T21:29:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003968908 2024-09-10T21:29:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-10T21:29:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 53830, 2024-09-10T21:29:20.025 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 556.238759161, 2024-09-10T21:29:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.010333248 2024-09-10T21:29:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-10T21:29:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-10T21:29:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 85135, 2024-09-10T21:29:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 362.709062003, 2024-09-10T21:29:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004260398 2024-09-10T21:29:20.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.029 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-10T21:29:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-10T21:29:20.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:20.031 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.032 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.033 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-10T21:29:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-10T21:29:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 113, 2024-09-10T21:29:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.395946273, 2024-09-10T21:29:20.035 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003503949 2024-09-10T21:29:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-10T21:29:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-10T21:29:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.037 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-10T21:29:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 10, 2024-09-10T21:29:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.012135632, 2024-09-10T21:29:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001213563 2024-09-10T21:29:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-10T21:29:20.038 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 67335, 2024-09-10T21:29:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 414.009464523, 2024-09-10T21:29:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.006148503 2024-09-10T21:29:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-10T21:29:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7849, 2024-09-10T21:29:20.039 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 30.623049880, 2024-09-10T21:29:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003901522 2024-09-10T21:29:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.040 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-10T21:29:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 44, 2024-09-10T21:29:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.929576793, 2024-09-10T21:29:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.021126745 2024-09-10T21:29:20.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.043 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.044 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-10T21:29:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 44, 2024-09-10T21:29:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.837256575, 2024-09-10T21:29:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.019028558 2024-09-10T21:29:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:20.045 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:20.046 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1200, 2024-09-10T21:29:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.965911442, 2024-09-10T21:29:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001638259 2024-09-10T21:29:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-10T21:29:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.047 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-10T21:29:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 7, 2024-09-10T21:29:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.019304973, 2024-09-10T21:29:20.048 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002757853 2024-09-10T21:29:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-10T21:29:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.049 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-10T21:29:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 44954, 2024-09-10T21:29:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 117.115811165, 2024-09-10T21:29:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002605236 2024-09-10T21:29:20.050 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.051 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-10T21:29:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-09-10T21:29:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-09-10T21:29:20.052 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 1, 2024-09-10T21:29:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-09-10T21:29:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-10T21:29:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 21707, 2024-09-10T21:29:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 21707, 2024-09-10T21:29:20.053 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 1520, 2024-09-10T21:29:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-10T21:29:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:20.054 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:20.055 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:20.056 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:20.057 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:20.058 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:20.059 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 307272, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 145, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 352, 2024-09-10T21:29:20.060 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 4, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 156, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:20.061 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 380860, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 5408, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-10T21:29:20.062 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 168626, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 545317289, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 37, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 168626, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 168626, 2024-09-10T21:29:20.063 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1651.169474473, 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.009791903 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 168626, 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 171179 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 168626, 2024-09-10T21:29:20.064 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 5047, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 163579, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 0, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 192, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 1527, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 115441, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 844, 2024-09-10T21:29:20.065 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 6, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 45743, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:20.066 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 3520, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 384, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-10T21:29:20.067 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-10T21:29:20.068 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-10T21:29:20.069 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 77, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 2629, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-09-10T21:29:20.070 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 891 2024-09-10T21:29:20.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.071 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-10T21:29:20.071 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-09-10T21:29:20.071 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:20.071 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-10T21:29:20.071 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-10T21:29:20.071 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.072 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-10T21:29:20.072 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:20.072 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 6, 2024-09-10T21:29:20.072 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-10T21:29:20.072 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 6, 2024-09-10T21:29:20.072 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 44972, 2024-09-10T21:29:20.073 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 44972, 2024-09-10T21:29:20.073 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:20.073 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.073 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:20.073 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:20.073 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:20.074 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:20.074 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 1436748, 2024-09-10T21:29:20.074 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 263864143, 2024-09-10T21:29:20.074 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:20.074 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 1436748, 2024-09-10T21:29:20.075 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:20.075 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:20.075 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1436748, 2024-09-10T21:29:20.075 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 263864143, 2024-09-10T21:29:20.075 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:20.075 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.076 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.076 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.076 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.076 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.076 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:20.077 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:20.077 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:20.077 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:20.077 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:20.077 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:20.077 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:20.078 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:20.078 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 168626, 2024-09-10T21:29:20.078 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 549859445, 2024-09-10T21:29:20.078 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 168626, 2024-09-10T21:29:20.078 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 549859445, 2024-09-10T21:29:20.079 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:20.079 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.079 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.079 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.079 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.079 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 168626, 2024-09-10T21:29:20.080 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 168626, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 168626, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 168626, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 44972, 2024-09-10T21:29:20.081 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 4542172, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 44972, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1496, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 4542172, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55f0fbb76dc0": { 2024-09-10T21:29:20.082 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 270328, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 535419742, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 270328, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 113945, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 535419742, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:20.083 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:20.084 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.084 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:20.084 INFO:teuthology.orchestra.run.smithi104.stdout: "trackedop": { 2024-09-10T21:29:20.084 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-10T21:29:20.084 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:20.084 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-10T21:29:21.021 INFO:tasks.check_counter:Daemon mds.b mds.exported=30 2024-09-10T21:29:21.021 INFO:tasks.check_counter:Daemon mds.b mds.imported=30 2024-09-10T21:29:21.021 DEBUG:tasks.check_counter:Getting stats from e 2024-09-10T21:29:21.024 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-10T21:29:21.288 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:29:21.696 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:21 smithi104 ceph-mon[35325]: pgmap v1107: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-09-10T21:29:21.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:21 smithi174 ceph-mon[35635]: pgmap v1107: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-09-10T21:29:21.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:21 smithi033 ceph-mon[30860]: pgmap v1107: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.8 KiB/s wr, 22 op/s 2024-09-10T21:29:22.005 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-10T21:29:22.005 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:22.005 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 3653, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 3576, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 159964626, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 1269290, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.429123311, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.158926013, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.188712362, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.071472731, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3575, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.131084395, 2024-09-10T21:29:22.006 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000036666 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1106464, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 275536 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1539, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1875, 2024-09-10T21:29:22.007 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 176552790, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 1263952, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 8, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.238413455, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.066438094, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.126297366, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.034641422, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1873, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.067151771, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000035852 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.008 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 775120 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1685, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1684, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 201756116, 2024-09-10T21:29:22.009 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 534695, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.274248830, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.052874016, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.168707460, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.041503550, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1683, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.060137137, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000035732 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.010 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:22.011 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 10738, 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.272371265, 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000025365 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.012 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:22.013 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 45191, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 17718, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 27473, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 19879, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 2, 2024-09-10T21:29:22.014 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-09-10T21:29:22.015 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 4, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 234, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-10T21:29:22.016 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-09-10T21:29:22.017 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-09-10T21:29:22.018 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-10T21:29:22.019 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_started": 0, 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_write_failed": 0 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-10T21:29:22.020 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 270328, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 134, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 23, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-10T21:29:22.021 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 352143986, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 539614046, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 539609409, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 270328 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-10T21:29:22.022 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 38, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 45689, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 45651, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 33, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 198, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 165, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 45191, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 46706, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 1515, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-09-10T21:29:22.023 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 94124, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 298270720 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.024 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.025 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.026 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.027 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.028 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.029 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-10T21:29:22.030 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.031 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.032 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:22.033 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.034 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:22.035 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 1, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-09-10T21:29:22.036 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 2789, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:22.037 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:22.038 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:22.039 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 241494, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 96, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 156, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:22.040 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 30862177, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 182788, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-10T21:29:22.041 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 6342, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 26, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 4, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 6342, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6342, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 20.561744606, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003242154 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6342, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 6343 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.042 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 6342, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 6342, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 3304, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 3035, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-10T21:29:22.043 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 2, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-10T21:29:22.044 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-10T21:29:22.045 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 77, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 11, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 11, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-10T21:29:22.046 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:22.047 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 6873, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 537625097, 2024-09-10T21:29:22.048 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 6873, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 6873, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 537625097, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:22.049 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 6342, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 535419752, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 6342, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 535419752, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.050 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 6342, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 6342, 2024-09-10T21:29:22.051 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 6342, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 6342, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:22.052 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x5579cc21b040": { 2024-09-10T21:29:22.053 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:22.054 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:22.055 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:22.055 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.055 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:22.055 INFO:teuthology.orchestra.run.smithi104.stdout: "trackedop": { 2024-09-10T21:29:22.055 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-10T21:29:22.055 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:22.055 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-10T21:29:23.824 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-10T21:29:23.824 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-10T21:29:23.824 DEBUG:tasks.check_counter:Getting stats from h 2024-09-10T21:29:23.824 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-10T21:29:23.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:23 smithi033 ceph-mon[30860]: pgmap v1108: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T21:29:24.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:23 smithi104 ceph-mon[35325]: pgmap v1108: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T21:29:24.081 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:29:24.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:23 smithi174 ceph-mon[35635]: pgmap v1108: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-09-10T21:29:24.628 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-10T21:29:24.628 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:24.628 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 532819, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 746302, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 163253249, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 602197528, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 14, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 42.848527069, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 23.161432550, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 20.596351565, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 1.320523579, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 746301, 2024-09-10T21:29:24.629 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 45.895942012, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000061497 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1137310, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.208279607, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000183 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1140800, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 313424 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 186162, 2024-09-10T21:29:24.630 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 184536, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 46578325, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 245922814, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 12, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 14.148228075, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 6.366109816, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 6.010497164, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 2.429964477, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 184537, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 12.719759907, 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000068927 2024-09-10T21:29:24.631 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 261731, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.043341621, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000165 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 778864 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 257991, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 231449, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 70353996, 2024-09-10T21:29:24.632 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 202966113, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 72, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 10, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 19.990285514, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 10.013188517, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 9.270366046, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 2.177137916, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 231445, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 9.552491919, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000041273 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.633 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 380980, 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.056754265, 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000148 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-10T21:29:24.634 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 140608, 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 154.530145464, 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001099013 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 81382, 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 7.956421254, 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000097766 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.635 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 284831, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 262287, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 262287, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1184.560983490, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.004516277 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 22544, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 1, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:24.636 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 99, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 7, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 10, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 872, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 541, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 260, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 71, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 97, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 608, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 3, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 3, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 6, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 473642, 2024-09-10T21:29:24.637 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 264639, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 22539, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 110, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 5875, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 15, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 233, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 15, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 558, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 2344, 2024-09-10T21:29:24.638 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 661, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 190757445, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 2344, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 280, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 280, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 2064, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 12, 2024-09-10T21:29:24.639 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 25792, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 174929, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 101170, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 39507, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 13652, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 184374 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 69, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 433, 2024-09-10T21:29:24.640 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 248, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 482, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 196, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 4, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 43468, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 43464, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:24.641 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 15, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 17, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_started": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_write_failed": 0 2024-09-10T21:29:24.642 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 231623, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 138438, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 135191, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 96432, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 3247, 2024-09-10T21:29:24.643 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 303, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 175, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 171, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 132, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 8, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 4, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 245061805, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 427943922, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 0, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 78383, 2024-09-10T21:29:24.644 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 996.215442263, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.012709585 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 0 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 46, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 43787, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 43741, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 45, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 440, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 395, 2024-09-10T21:29:24.645 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 882, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 78050, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 77168, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 3, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 47238, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 47235, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 83432, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 294510592 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 408906, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 10084, 2024-09-10T21:29:24.646 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 284831, 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 7916, 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 37128, 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 43344, 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 122.398650310, 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002823889 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 10416, 2024-09-10T21:29:24.647 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 324.130885213, 2024-09-10T21:29:24.648 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.031118556 2024-09-10T21:29:24.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.648 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:24.648 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.648 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.648 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.648 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 85163, 2024-09-10T21:29:24.649 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 118.579153209, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.001392378 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000386857, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000193428 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.650 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-10T21:29:24.651 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.651 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.651 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.651 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.651 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:24.651 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-10T21:29:24.652 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.653 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-10T21:29:24.654 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 66624, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 485.565283991, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.007288143 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 10957, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 34.256701658, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.003126467 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.655 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 20, 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.299515162, 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.014975758 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:24.656 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 41, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.895620641, 2024-09-10T21:29:24.657 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.021844405 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 2272, 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1.668594766, 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000734416 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-10T21:29:24.658 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.659 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 43448, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 96.766191683, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002227172 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.660 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 1, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 25171, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 25171, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 1529, 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:24.661 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:24.662 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:24.663 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 305695, 2024-09-10T21:29:24.664 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 123, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 88, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 1, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 156, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:24.665 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 815535, 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 10724, 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-10T21:29:24.666 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 129732, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 433364365, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 29, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 129732, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 129732, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 1459.452891499, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.011249752 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-10T21:29:24.667 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 129732, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 132180 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 129732, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 3838, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 125894, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 2, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 136, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 1493, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 79877, 2024-09-10T21:29:24.668 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 842, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 7, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 43695, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 2345, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 272, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:24.669 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-10T21:29:24.670 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 77, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 12, 2024-09-10T21:29:24.671 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 12, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 2608, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 901 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 1, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-10T21:29:24.672 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 6, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 6, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 43464, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 43464, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:24.673 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 976968, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 192224390, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 976968, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 976968, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 192224390, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.674 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 129732, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 437754213, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 129732, 2024-09-10T21:29:24.675 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 437754213, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:24.676 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 129732, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 129732, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 129732, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 129732, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:24.677 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 43464, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 4389864, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 43464, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 1494, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 4389864, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.678 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x55d2aea4edc0": { 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 231623, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 423749618, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 231623, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 78383, 2024-09-10T21:29:24.679 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 423749618, 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: "trackedop": { 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:24.680 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-10T21:29:25.358 INFO:tasks.check_counter:Daemon mds.h mds.exported=15 2024-09-10T21:29:25.358 INFO:tasks.check_counter:Daemon mds.h mds.imported=15 2024-09-10T21:29:25.358 DEBUG:tasks.check_counter:Getting stats from k 2024-09-10T21:29:25.358 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-10T21:29:25.608 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T21:29:25.906 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:25 smithi104 ceph-mon[35325]: pgmap v1109: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-09-10T21:29:25.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:25 smithi033 ceph-mon[30860]: pgmap v1109: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-09-10T21:29:26.118 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-09-10T21:29:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1579, 2024-09-10T21:29:26.118 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1502, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 144105972, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 610884, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 7, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 3, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.232594293, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.063952453, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.144332731, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.025262108, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1501, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.073107131, 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000048705 2024-09-10T21:29:26.119 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 1107120, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 277296 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 1422, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 1761, 2024-09-10T21:29:26.120 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 101023150, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 1228861, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 8, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.206329708, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.063814366, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.097993033, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.033122859, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 1759, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.065788129, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000037400 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.121 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 777296 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_messages": 3666, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages": 3666, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_bytes": 181420635, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_bytes": 1165022, 2024-09-10T21:29:26.122 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_created_connections": 6, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_active_connections": 5, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_total_time": 0.431687925, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_send_time": 0.138150836, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_recv_time": 0.181541324, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_running_fast_dispatch_time": 0.088834896, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 3665, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.135698848, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000037025 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.123 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "msgr_send_encrypted_bytes": 448 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "cct": { 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "total_workers": 1, 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "unhealthy_workers": 0 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-MDSRank": { 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:26.124 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 10435, 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.266320583, 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000025521 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "queue_len": 0, 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "complete_latency": { 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.125 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "mds": { 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "request": 0, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "reply": 0, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "reply_latency": { 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_reply": 0, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "forward": 0, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:26.126 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_commit": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_split": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_merge": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes": 43576, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_top": 16830, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_bottom": 26746, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pin_tail": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_pinned": 19532, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_expired": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "inodes_with_caps": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "caps": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "subtrees": 1, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse": 0, 2024-09-10T21:29:26.127 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_hit": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_forward": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_discover": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "traverse_lock": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "load_cent": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "q": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "exported": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "exported_inodes": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "imported": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "imported_inodes": 0, 2024-09-10T21:29:26.128 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_backtrace_fetch": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rfiles": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rbytes": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "root_rsnaps": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_inodes": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T21:29:26.129 INFO:teuthology.orchestra.run.smithi104.stdout: "scrub_file_inodes": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_inode_file_caps": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_grant": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_caps_dirty": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_cap_release": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "process_request_cap_release": 0 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_cache": { 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update": 0, 2024-09-10T21:29:26.130 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_update_receipt": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_try_discover": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_send_discover": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "dir_handle_discover": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_created": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_enqueued": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "strays_migrated": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:26.131 INFO:teuthology.orchestra.run.smithi104.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_started": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "recovery_completed": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_exportdir": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_flush": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragmentdir": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_started": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:26.132 INFO:teuthology.orchestra.run.smithi104.stdout: "uninline_write_failed": 0 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_log": { 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "evlrg": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "evadd": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "evex": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "evtrm": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "ev": 231623, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "evexg": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "evexd": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "segadd": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "segex": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "segtrm": 0, 2024-09-10T21:29:26.133 INFO:teuthology.orchestra.run.smithi104.stdout: "seg": 132, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "segmjr": 20, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "segexg": 0, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "segexd": 0, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "expos": 245061805, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "wrpos": 427943922, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "rdpos": 427942784, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "jlat": { 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "replayed": 231623 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.134 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_mem": { 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "ino": 17, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "ino+": 43609, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "ino-": 43592, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "dir": 15, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "dir+": 138, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "dir-": 123, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "dn": 43576, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "dn+": 44079, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "dn-": 503, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "cap": 0, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "cap+": 0, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "cap-": 0, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "rss": 91268, 2024-09-10T21:29:26.135 INFO:teuthology.orchestra.run.smithi104.stdout: "heap": 295780352 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_server": { 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_client_request": 0, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "dispatch_server_request": 0, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_request": 0, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_client_session": 0, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "handle_peer_request": 0, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "req_create_latency": { 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.136 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getattr_latency": { 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "req_link_latency": { 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.137 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookup_latency": { 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupino_latency": { 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.138 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupname_latency": { 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:26.139 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: "req_lssnap_latency": { 2024-09-10T21:29:26.140 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mkdir_latency": { 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mknod_latency": { 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.141 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "req_mksnap_latency": { 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "req_open_latency": { 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "req_readdir_latency": { 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.142 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rename_latency": { 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:26.143 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmdir_latency": { 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.144 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setattr_latency": { 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.145 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setlayout_latency": { 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "req_setxattr_latency": { 2024-09-10T21:29:26.146 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "req_symlink_latency": { 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "req_unlink_latency": { 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.147 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_sessions": { 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "session_count": 1, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "session_add": 2, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "session_remove": 1, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_open": 1, 2024-09-10T21:29:26.148 INFO:teuthology.orchestra.run.smithi104.stdout: "sessions_stale": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "total_load": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "average_load": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "avg_session_uptime": 2789, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "mempool": { 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:26.149 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:26.150 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_items": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:26.151 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_bytes": 242739, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_anon_items": 90, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_bytes": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_items": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:26.152 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_items": 156, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "pgmap_items": 0, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_bytes": 29690436, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "mds_co_items": 174830, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_1_items": 0, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "unittest_2_items": 0 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "objecter": { 2024-09-10T21:29:26.153 INFO:teuthology.orchestra.run.smithi104.stdout: "op_active": 0, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "op_laggy": 0, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send": 6131, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "op_send_bytes": 21, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "op_resend": 3, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "op_reply": 6131, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "op_latency": { 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6131, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 17.314785197, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.002824137 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "op_inflight": 0, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "oplen_avg": { 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 6131, 2024-09-10T21:29:26.154 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 6132 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "op": 6131, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "op_r": 6131, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "op_w": 0, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "op_rmw": 0, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "op_pg": 0, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_stat": 3238, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_create": 0, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_read": 2891, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_write": 0, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writefull": 0, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_writesame": 0, 2024-09-10T21:29:26.155 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_append": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_zero": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_truncate": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_delete": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_mapext": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_getxattr": 1, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_setxattr": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_call": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_watch": 0, 2024-09-10T21:29:26.156 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_notify": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "osdop_other": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_active": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_send": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_resend": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "linger_ping": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_active": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_send": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "poolop_resend": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_active": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_send": 0, 2024-09-10T21:29:26.157 INFO:teuthology.orchestra.run.smithi104.stdout: "poolstat_resend": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_active": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_send": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "statfs_resend": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "command_active": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "command_send": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "command_resend": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "map_epoch": 77, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "map_full": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "map_inc": 2, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_sessions": 11, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_open": 11, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_session_close": 0, 2024-09-10T21:29:26.158 INFO:teuthology.orchestra.run.smithi104.stdout: "osd_laggy": 0, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_wr": 0, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_rd": 2, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_del": 0 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "oft": { 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_objs": 0, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_updates": 0, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "omap_total_removes": 0 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "purge_queue": { 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T21:29:26.159 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing": 0, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executing_high_water": 0, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed_ops": 0, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_executed": 0, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 6663, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 425920112, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:26.160 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 6663, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 6663, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 425920112, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 104857600, 2024-09-10T21:29:26.161 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 6131, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 423749623, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 6131, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 423749623, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.162 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 1024, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 6131, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 6131, 2024-09-10T21:29:26.163 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 6131, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 6131, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:26.164 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "throttle-write_buf_throttle-0x5600bcb4f040": { 2024-09-10T21:29:26.165 INFO:teuthology.orchestra.run.smithi104.stdout: "val": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "max": 3758096384, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "get_started": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "get": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "get_sum": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "take": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "take_sum": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "put": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "put_sum": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "wait": { 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "avgcount": 0, 2024-09-10T21:29:26.166 INFO:teuthology.orchestra.run.smithi104.stdout: "sum": 0.000000000, 2024-09-10T21:29:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: "avgtime": 0.000000000 2024-09-10T21:29:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: }, 2024-09-10T21:29:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: "trackedop": { 2024-09-10T21:29:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: "slow_ops_count": 0 2024-09-10T21:29:26.167 INFO:teuthology.orchestra.run.smithi104.stdout: } 2024-09-10T21:29:26.167 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-09-10T21:29:26.167 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:25 smithi174 ceph-mon[35635]: pgmap v1109: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-09-10T21:29:26.832 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-10T21:29:26.832 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-10T21:29:26.832 DEBUG:tasks.check_counter:Getting stats from c 2024-09-10T21:29:26.833 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-10T21:29:27.168 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:29:27.788 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T21:29:27.788 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:27.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 565843, 2024-09-10T21:29:27.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 795388, 2024-09-10T21:29:27.788 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 176639283, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 674856114, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 15, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 54.034145483, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 29.402801226, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 23.788367429, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 2.616258357, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 795388, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 87.759998614, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000110336 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.789 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1157369, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.249249325, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000215 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1140352, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 314992 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 128404, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 135336, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 31784718, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 227494612, 2024-09-10T21:29:27.790 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 14, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 15.232733741, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 7.811144050, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 4.872216391, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 2.754129473, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 135334, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 15.438809820, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000114079 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 161948, 2024-09-10T21:29:27.791 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.031927001, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000197 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 689760 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 321893, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 305918, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 80623105, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 213749065, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 67, 2024-09-10T21:29:27.792 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 8, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 23.068525478, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 10.635249119, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 11.804464653, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 2.313221072, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 305916, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 31.959399785, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000104471 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 537458, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.088046290, 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000163 2024-09-10T21:29:27.793 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 128689, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 195.637944570, 2024-09-10T21:29:27.794 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001520238 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 80342, 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 11.495454406, 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000143081 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 301527, 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 272884, 2024-09-10T21:29:27.795 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 272884, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1642.870397439, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.006020398 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 28643, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 137, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 8, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 15, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 931, 2024-09-10T21:29:27.796 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 521, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 302, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 108, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 181, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 889, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 1, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 1, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 6, 2024-09-10T21:29:27.797 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 479056, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 267747, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 28635, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 94, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 5304, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 22, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 474, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 22, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 951, 2024-09-10T21:29:27.798 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 3037, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 661, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 190757445, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 3037, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 430, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 430, 2024-09-10T21:29:27.799 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 2607, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 48, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 43404, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 194098, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 125532, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 38325, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 15356, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 179822 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.800 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 81, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 269, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 162, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 309, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 379, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 14, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 44572, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 44558, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:27.801 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 23, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 23, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_started": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:27.802 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_write_failed": 0 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 235834, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 130058, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 122901, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 112933, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 7157, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 284, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 156, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 148, 2024-09-10T21:29:27.803 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 136, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 10, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 8, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 230543972, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 464553696, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 75513, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1079.974140573, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.014301830 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.804 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 51, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 44996, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 44945, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 49, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 336, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 287, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 940, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 85092, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 84152, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 1, 2024-09-10T21:29:27.805 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 49492, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 49491, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 88668, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 300802048 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 421132, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 13619, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 301527, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 6425, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 36946, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44466, 2024-09-10T21:29:27.806 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 138.735668898, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003120039 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 17510, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 509.399093069, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.029091895 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.807 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 88811, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 175.860203003, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001980162 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.808 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:27.809 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.810 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.810 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.810 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.810 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:27.810 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.810 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.811 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.812 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 66181, 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 626.174139648, 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.009461539 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10148, 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 36.101049251, 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003557454 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:27.813 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 30, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.149792068, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004993068 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.814 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 60, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.275579338, 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.021259655 2024-09-10T21:29:27.815 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1136, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.123295647, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001869098 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.816 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.817 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44542, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 153.051576517, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003436118 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.818 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 25010, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 25010, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1466, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:27.819 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:27.820 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:27.821 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 315198, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 142, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 264, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 3, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T21:29:27.822 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 156, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 876411, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 12161, 2024-09-10T21:29:27.823 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 129113, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 470513165, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 129113, 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T21:29:27.824 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 129113, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1516.907880873, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.011748684 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 129113, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 131736 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 129113, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 4684, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 124429, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T21:29:27.825 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 19, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 180, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1647, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 77160, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 910, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 8, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 44805, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:27.826 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 3037, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 360, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T21:29:27.827 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T21:29:27.828 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 77, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 2678, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 932 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T21:29:27.829 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 6, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 6, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 44558, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 44558, 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:27.830 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 1016136, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 197560410, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 1016136, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1016136, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 197560410, 2024-09-10T21:29:27.831 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:27.832 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:27 smithi174 ceph-mon[35635]: pgmap v1110: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-10T21:29:27.832 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.832 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.832 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.832 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.832 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.832 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:27.832 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 129113, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 475013523, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 129113, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 475013523, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.833 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 129113, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 129113, 2024-09-10T21:29:27.834 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 129113, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 129113, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:27.835 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 44558, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 4500358, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 44558, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1647, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4500358, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.836 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55aa28134dc0": { 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 235834, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 460359392, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 235834, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:27.837 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 75513, 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 460359392, 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: "trackedop": { 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:27.838 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T21:29:27.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:27 smithi033 ceph-mon[30860]: pgmap v1110: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-10T21:29:28.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:27 smithi104 ceph-mon[35325]: pgmap v1110: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 511 B/s wr, 13 op/s 2024-09-10T21:29:28.742 INFO:tasks.check_counter:Daemon mds.c mds.exported=22 2024-09-10T21:29:28.742 INFO:tasks.check_counter:Daemon mds.c mds.imported=22 2024-09-10T21:29:28.742 DEBUG:tasks.check_counter:Getting stats from f 2024-09-10T21:29:28.742 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-10T21:29:28.996 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:29:29.546 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T21:29:29.546 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:29.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 256733, 2024-09-10T21:29:29.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 247696, 2024-09-10T21:29:29.546 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 68995026, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 176726211, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 14, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 21.257862897, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 10.400296733, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 10.277473845, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 1.631116484, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 247699, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 23.783568451, 2024-09-10T21:29:29.547 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000096018 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 431306, 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.078051560, 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000180 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3872, 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 1184 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:29.548 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 250372, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 207367, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 65464948, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 147179693, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 10, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 17.089631248, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 7.806630912, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 8.612811665, 2024-09-10T21:29:29.549 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 2.003334399, 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 207365, 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 16.077203600, 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000077530 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 381686, 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.059106292, 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000154 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.550 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 689952 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 380338, 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 545605, 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 120126094, 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 562406559, 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 73, 2024-09-10T21:29:29.551 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 11, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 36.069322209, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 19.028339233, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 15.195600402, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 1.890155868, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 545600, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 59.033212663, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000108198 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:29.552 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 771954, 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.163858672, 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000212 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1129376, 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 315568 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.553 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 105138, 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 127.030144139, 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001208222 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:29.554 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 69619, 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.735251530, 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000096744 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 215039, 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 212917, 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 212917, 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 794.833046085, 2024-09-10T21:29:29.555 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003733065 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 2122, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 100, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 7, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 11, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 500, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 318, 2024-09-10T21:29:29.556 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 146, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 36, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 42, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 936, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 6, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 418980, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 215306, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 2112, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 91, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:29.557 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 6100, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 16, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 338, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 16, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 503, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 1973, 2024-09-10T21:29:29.558 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 661, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 190757445, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 1973, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 130, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 130, 2024-09-10T21:29:29.559 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 1843, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 12, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 14656, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 152106, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 95665, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 27225, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 12869, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 172417 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.560 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 266, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 294, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 167, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 336, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 364, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 3, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 38039, 2024-09-10T21:29:29.561 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 38036, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 19, 2024-09-10T21:29:29.562 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 18, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_started": 0, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_write_failed": 0 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 201450, 2024-09-10T21:29:29.563 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 102017, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 98321, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 103129, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 3696, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 259, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 131, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 126, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 133, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 9, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T21:29:29.564 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 5, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 156972059, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 338537264, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 57308, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 681.830039932, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.011897641 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T21:29:29.565 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 52, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 38359, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 38307, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 51, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 294, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 243, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 572, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 69685, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 69113, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 42267, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 42267, 2024-09-10T21:29:29.566 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 79280, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 289263616 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 364693, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 1313, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 215039, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 4094, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 28915, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 37873, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 174.086499553, 2024-09-10T21:29:29.567 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004596585 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7654, 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 118.066789689, 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.015425501 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.568 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 65354, 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 108.402453604, 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001658696 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.569 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.570 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.571 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.572 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.573 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 55472, 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 230.396572397, 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004153384 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7750, 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 36.399903695, 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004696761 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:29.574 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T21:29:29.575 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10, 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.044280185, 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004428018 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.576 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 35, 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.004169318, 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.028690551 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.577 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 740, 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.952349942, 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001286959 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T21:29:29.578 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.579 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 38029, 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 125.480027702, 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003299587 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.580 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 17683, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 17683, 2024-09-10T21:29:29.581 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2802, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:29.582 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:29.583 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:29.584 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 283114, 2024-09-10T21:29:29.585 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 126, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 440, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 5, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:29.586 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 156, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 601405, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 7502, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:29.587 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 102576, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 342947138, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 102576, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 102576, 2024-09-10T21:29:29.588 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1099.040848116, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.010714405 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 102576, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 104927 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 102576, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3387, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 99189, 2024-09-10T21:29:29.589 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 4, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 143, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1414, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 58722, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 796, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 7, 2024-09-10T21:29:29.590 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 38223, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1973, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 286, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T21:29:29.591 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T21:29:29.592 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 77, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-09-10T21:29:29.593 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 2538, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 821 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:29.594 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 6, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 6, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 38036, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 38036, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.595 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 887439, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 174682388, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 887439, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 887439, 2024-09-10T21:29:29.596 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 174682388, 2024-09-10T21:29:29.597 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:29.597 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.597 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.597 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.597 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:29.598 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 102576, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 346788774, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 102576, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 346788774, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:29.599 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 102576, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 102576, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 102576, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 102576, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.600 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 38036, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 3841636, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 38036, 2024-09-10T21:29:29.601 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1414, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3841636, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x559e0bd80dc0": { 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:29.602 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 201450, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 334342960, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 201450, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 57308, 2024-09-10T21:29:29.603 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 334342960, 2024-09-10T21:29:29.604 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:29.604 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:29.604 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:29.604 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:29.604 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.604 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:29.604 INFO:teuthology.orchestra.run.smithi174.stdout: "trackedop": { 2024-09-10T21:29:29.605 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-10T21:29:29.605 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:29.605 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T21:29:29.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:29 smithi174 ceph-mon[35635]: pgmap v1111: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-09-10T21:29:29.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:29 smithi033 ceph-mon[30860]: pgmap v1111: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-09-10T21:29:30.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:29 smithi104 ceph-mon[35325]: pgmap v1111: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 12 op/s 2024-09-10T21:29:30.543 INFO:tasks.check_counter:Daemon mds.f mds.exported=16 2024-09-10T21:29:30.543 INFO:tasks.check_counter:Daemon mds.f mds.imported=16 2024-09-10T21:29:30.543 DEBUG:tasks.check_counter:Getting stats from i 2024-09-10T21:29:30.544 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-10T21:29:30.794 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:29:31.333 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 679368, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 880899, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 200927165, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 706379699, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 14, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 55.560423607, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 28.846327774, 2024-09-10T21:29:31.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 26.309854238, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 2.239080735, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 880895, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 87.242722194, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000099038 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1355229, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.276763534, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000204 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T21:29:31.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 188530, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 180211, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 47011132, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 190440091, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 15, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 17.087525793, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 8.291034761, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 7.311278267, 2024-09-10T21:29:31.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 2.216557596, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 180213, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 19.721606115, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000109434 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 285627, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.053376527, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000186 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 782528 2024-09-10T21:29:31.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 92344, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 102151, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 27795939, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 176495567, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 68, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 9, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 11.546410704, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 5.685546826, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 3.405753873, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 2.519265021, 2024-09-10T21:29:31.338 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 102149, 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 12.544837799, 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000122809 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 112252, 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.024965848, 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000222 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1147200, 2024-09-10T21:29:31.339 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 308832 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 115836, 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 176.223509695, 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001521319 2024-09-10T21:29:31.340 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 81976, 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 9.785922220, 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000119375 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 291512, 2024-09-10T21:29:31.341 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 268363, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 268363, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1187.210614508, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004423898 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 23149, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 146, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 12, 2024-09-10T21:29:31.342 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 14, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 1567, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 975, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 440, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 152, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 171, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 797, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 6, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 453965, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 244080, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 23141, 2024-09-10T21:29:31.343 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 112, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 3571, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 1000, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 1, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 16, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 389, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 16, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 400, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 2362, 2024-09-10T21:29:31.344 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 661, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 190757445, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 2362, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 290, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 290, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 2072, 2024-09-10T21:29:31.345 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 16, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 44183, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 200219, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 110360, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 30705, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 13212, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 179115 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 201, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 317, 2024-09-10T21:29:31.346 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 186, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 364, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 397, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 6, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 44912, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 44906, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T21:29:31.347 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 16, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 26, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_started": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_write_failed": 0 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.348 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 218491, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 117855, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 110611, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 107880, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 7244, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 279, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 151, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 141, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 138, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 9, 2024-09-10T21:29:31.349 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 10, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 207642121, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 424376674, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 66165, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 810.774809443, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.012253832 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T21:29:31.350 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 35, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 45288, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 45253, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 34, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 366, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 332, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 1562, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 90012, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 88450, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 49612, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 49612, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 95076, 2024-09-10T21:29:31.351 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 307093504 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 393664, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 6649, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 291512, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 5629, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 24699, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44791, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 123.447879555, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002756086 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.352 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6474, 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 184.510478688, 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.028500228 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T21:29:31.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 90110, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 150.210848777, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001666972 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:31.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.356 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.356 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.356 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:31.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.018680106, 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003113351 2024-09-10T21:29:31.357 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-09-10T21:29:31.358 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000871195, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000435597 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 72102, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 570.906368308, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.007918037 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 8367, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 30.202933378, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003609768 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.359 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 20, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.151207457, 2024-09-10T21:29:31.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.007560372 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.361 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 19, 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.556851454, 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.029307971 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1580, 2024-09-10T21:29:31.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.454117322, 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001553238 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.363 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44892, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 124.750378268, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002778899 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:31.364 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 23702, 2024-09-10T21:29:31.365 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 23702, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1466, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:31.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:31.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:31.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 293080, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 130, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:31.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 156, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 1297686, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 18379, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T21:29:31.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 119358, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 430638039, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 28, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 119358, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 119358, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1293.659528861, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.010838481 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T21:29:31.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 119358, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 121869 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 119358, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3990, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 115368, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 4, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 139, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1628, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 67793, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 879, 2024-09-10T21:29:31.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 12, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 45166, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2362, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 278, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:31.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T21:29:31.374 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 77, 2024-09-10T21:29:31.375 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 2702, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 906 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:31.376 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 6, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 6, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 44906, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 44906, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.377 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 960238, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 189278497, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 960238, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 960238, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 189278497, 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:31.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:31.379 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 119358, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 435173545, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 119358, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 435173545, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:31.380 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 119358, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 119358, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 119358, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 119358, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.381 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 44906, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 4535506, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 44906, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1628, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4535506, 2024-09-10T21:29:31.382 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55bbc4f4cdc0": { 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 218491, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 420182370, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 218491, 2024-09-10T21:29:31.383 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 66165, 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 420182370, 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "trackedop": { 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-10T21:29:31.384 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:31.385 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T21:29:31.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:31 smithi033 ceph-mon[30860]: pgmap v1112: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-10T21:29:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:31 smithi174 ceph-mon[35635]: pgmap v1112: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-10T21:29:32.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:31 smithi104 ceph-mon[35325]: pgmap v1112: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 13 op/s 2024-09-10T21:29:32.234 INFO:tasks.check_counter:Daemon mds.i mds.exported=16 2024-09-10T21:29:32.234 INFO:tasks.check_counter:Daemon mds.i mds.imported=16 2024-09-10T21:29:32.234 DEBUG:tasks.check_counter:Getting stats from l 2024-09-10T21:29:32.234 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-10T21:29:32.481 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.c/config 2024-09-10T21:29:33.201 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-09-10T21:29:33.201 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-09-10T21:29:33.201 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3659, 2024-09-10T21:29:33.201 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3660, 2024-09-10T21:29:33.201 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 168833846, 2024-09-10T21:29:33.201 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1163251, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.418489799, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.133195400, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.170644853, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.092024114, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3658, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.157342342, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000043013 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.202 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1798, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1721, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 123068246, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 681548, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.255719426, 2024-09-10T21:29:33.203 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.076055334, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.148582807, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.032140717, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1720, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.086582031, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000050338 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.204 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1107936, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 279216 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1205, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1544, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 131087194, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1162658, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.227877051, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.059176819, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.128518139, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.029684533, 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-09-10T21:29:33.205 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1543, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.088307380, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000057230 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 781376 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-09-10T21:29:33.206 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10358, 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.276077217, 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000026653 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-09-10T21:29:33.207 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.208 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 45008, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 15647, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 29361, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 22654, 2024-09-10T21:29:33.209 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-09-10T21:29:33.210 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-09-10T21:29:33.211 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-09-10T21:29:33.212 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-09-10T21:29:33.213 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-09-10T21:29:33.214 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_started": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_succeeded": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "uninline_write_failed": 0 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 218491, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-09-10T21:29:33.215 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 138, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 19, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 207642121, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 424376674, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 424375536, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.216 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 218491 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 19, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 45100, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 45081, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 17, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 195, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 178, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 45008, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 45646, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 638, 2024-09-10T21:29:33.217 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 92168, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 297254912 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.218 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-09-10T21:29:33.219 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.220 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.221 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.222 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-09-10T21:29:33.223 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.224 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.225 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.226 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.227 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.228 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.229 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2803, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-09-10T21:29:33.230 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-09-10T21:29:33.231 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-10T21:29:33.232 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 241221, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 88, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-09-10T21:29:33.233 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7592, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 156, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 30672436, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 180876, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-09-10T21:29:33.234 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 6122, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 6122, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6122, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 20.600752561, 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003365036 2024-09-10T21:29:33.235 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6122, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6123 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 6122, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 6122, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 3242, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 2878, 2024-09-10T21:29:33.236 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-09-10T21:29:33.237 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-09-10T21:29:33.238 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 77, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 11, 2024-09-10T21:29:33.239 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 1, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.240 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 6658, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 422359935, 2024-09-10T21:29:33.241 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 6658, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6658, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 422359935, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:33.242 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 6122, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 420190962, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6122, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 420190962, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.243 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 6122, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 6122, 2024-09-10T21:29:33.244 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6122, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 6122, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:33.245 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x564780f05040": { 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-09-10T21:29:33.246 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-09-10T21:29:33.247 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.248 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-09-10T21:29:33.248 INFO:teuthology.orchestra.run.smithi174.stdout: "trackedop": { 2024-09-10T21:29:33.248 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-09-10T21:29:33.248 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-09-10T21:29:33.248 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-09-10T21:29:33.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:33 smithi033 ceph-mon[30860]: pgmap v1113: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2024-09-10T21:29:34.015 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:33 smithi174 ceph-mon[35635]: pgmap v1113: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2024-09-10T21:29:34.043 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-10T21:29:34.043 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-10T21:29:34.043 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T21:29:34.054 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T21:29:34.054 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T21:29:34.054 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:34.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:33 smithi104 ceph-mon[35325]: pgmap v1113: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 11 op/s 2024-09-10T21:29:34.093 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-09-10T21:29:34.093 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:29:34.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T21:29:34.468 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 -- 172.21.15.33:0/2148434368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc109150 msgr2=0x7f94fc10f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 --2- 172.21.15.33:0/2148434368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc109150 0x7f94fc10f950 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f94f4009ca0 tx=0x7f94f402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 -- 172.21.15.33:0/2148434368 shutdown_connections 2024-09-10T21:29:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 --2- 172.21.15.33:0/2148434368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc109150 0x7f94fc10f950 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 --2- 172.21.15.33:0/2148434368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc1087d0 0x7f94fc108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 --2- 172.21.15.33:0/2148434368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94fc102dc0 0x7f94fc1031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 -- 172.21.15.33:0/2148434368 >> 172.21.15.33:0/2148434368 conn(0x7f94fc0fe830 msgr2=0x7f94fc100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 -- 172.21.15.33:0/2148434368 shutdown_connections 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.092+0000 7f9501d4e640 1 -- 172.21.15.33:0/2148434368 wait complete. 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 Processor -- start 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 -- start start 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc102dc0 0x7f94fc199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94fc1087d0 0x7f94fc199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc109150 0x7f94fc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc112e10 con 0x7f94fc102dc0 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc112c90 con 0x7f94fc109150 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f9501d4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94fc112f90 con 0x7f94fc1087d0 2024-09-10T21:29:35.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f94fb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc102dc0 0x7f94fc199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f94fb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc102dc0 0x7f94fc199340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54914/0 (socket says 172.21.15.33:54914) 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.093+0000 7f94fb7fe640 1 -- 172.21.15.33:0/3250093574 learned_addr learned my addr 172.21.15.33:0/3250093574 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94fbfff640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc109150 0x7f94fc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94faffd640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94fc1087d0 0x7f94fc199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94fb7fe640 1 -- 172.21.15.33:0/3250093574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94fc1087d0 msgr2=0x7f94fc199880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94fb7fe640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94fc1087d0 0x7f94fc199880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94fb7fe640 1 -- 172.21.15.33:0/3250093574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc109150 msgr2=0x7f94fc19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94fb7fe640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc109150 0x7f94fc19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94fb7fe640 1 -- 172.21.15.33:0/3250093574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94fc19f3f0 con 0x7f94fc102dc0 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94fb7fe640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc102dc0 0x7f94fc199340 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b4a0 tx=0x7f94ec00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94f8ff9640 1 -- 172.21.15.33:0/3250093574 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec014070 con 0x7f94fc102dc0 2024-09-10T21:29:35.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94fc19f6e0 con 0x7f94fc102dc0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94f8ff9640 1 -- 172.21.15.33:0/3250093574 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94ec010ac0 con 0x7f94fc102dc0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f94f8ff9640 1 -- 172.21.15.33:0/3250093574 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ec017c30 con 0x7f94fc102dc0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.094+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94fc1a5da0 con 0x7f94fc102dc0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.095+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94fc1031a0 con 0x7f94fc102dc0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.097+0000 7f94f8ff9640 1 -- 172.21.15.33:0/3250093574 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f94ec010610 con 0x7f94fc102dc0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.097+0000 7f94f8ff9640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94e40b52e0 0x7f94e40b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.097+0000 7f94faffd640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94e40b52e0 0x7f94e40b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.097+0000 7f94f8ff9640 1 -- 172.21.15.33:0/3250093574 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f94ec0e5280 con 0x7f94fc102dc0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.098+0000 7f94faffd640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94e40b52e0 0x7f94e40b77a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f94fc107230 tx=0x7f94f000a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:35.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.101+0000 7f94f8ff9640 1 -- 172.21.15.33:0/3250093574 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f94ec0ac100 con 0x7f94fc102dc0 2024-09-10T21:29:35.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.301+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f94fc02e5c0 con 0x7f94fc102dc0 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.301+0000 7f94f8ff9640 1 -- 172.21.15.33:0/3250093574 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v77) ==== 81+0+592 (secure 0 0 0) 0x7f94ec0aba00 con 0x7f94fc102dc0 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2096552792 2024-09-11T20:54:39.992584+0000 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/2468168068 2024-09-11T20:54:39.992584+0000 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/2468168068 2024-09-11T20:54:39.992584+0000 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2596082174 2024-09-11T20:54:14.879196+0000 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2930837239 2024-09-11T20:54:39.992584+0000 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1151081438 2024-09-11T20:54:14.879196+0000 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2648657406 2024-09-11T20:54:39.992584+0000 2024-09-10T21:29:35.305 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1151081438 2024-09-11T20:54:14.879196+0000 2024-09-10T21:29:35.306 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4287278386 2024-09-11T20:54:14.879196+0000 2024-09-10T21:29:35.306 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2461327124 2024-09-11T20:54:14.879196+0000 2024-09-10T21:29:35.306 INFO:teuthology.orchestra.run.smithi033.stderr:listed 10 entries 2024-09-10T21:29:35.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94e40b52e0 msgr2=0x7f94e40b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94e40b52e0 0x7f94e40b77a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f94fc107230 tx=0x7f94f000a4c0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc102dc0 msgr2=0x7f94fc199340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:35.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc102dc0 0x7f94fc199340 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f94ec00b4a0 tx=0x7f94ec00b960 comp rx=0 tx=0).stop 2024-09-10T21:29:35.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 shutdown_connections 2024-09-10T21:29:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f94e40b52e0 0x7f94e40b77a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f94fc109150 0x7f94fc19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f94fc1087d0 0x7f94fc199880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 --2- 172.21.15.33:0/3250093574 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94fc102dc0 0x7f94fc199340 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 >> 172.21.15.33:0/3250093574 conn(0x7f94fc0fe830 msgr2=0x7f94fc100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 shutdown_connections 2024-09-10T21:29:35.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:35.304+0000 7f9501d4e640 1 -- 172.21.15.33:0/3250093574 wait complete. 2024-09-10T21:29:35.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:35 smithi033 ceph-mon[30860]: pgmap v1114: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-09-10T21:29:35.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:35 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3250093574' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T21:29:36.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:35 smithi104 ceph-mon[35325]: pgmap v1114: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-09-10T21:29:36.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:35 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3250093574' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T21:29:36.125 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:29:36.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:36.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:29:36.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:35 smithi174 ceph-mon[35635]: pgmap v1114: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 51 KiB/s wr, 14 op/s 2024-09-10T21:29:36.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:35 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3250093574' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T21:29:36.428 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi033.front.sepia.ceph.com 2024-09-10T21:29:36.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:36.428 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:29:36.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:36.492 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-09-10T21:29:36.545 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:29:36.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:36.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:29:36.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:29:36.618 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-09-10T21:29:36.719 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T21:29:36.729 INFO:tasks.cephadm:Teardown begin 2024-09-10T21:29:36.729 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:29:36.729 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:29:37.339 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:37.808 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:37 smithi033 ceph-mon[30860]: pgmap v1115: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-10T21:29:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 -- 172.21.15.33:0/926957122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c20103ca0 msgr2=0x7f9c20104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 --2- 172.21.15.33:0/926957122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c20103ca0 0x7f9c20104080 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009ca0 tx=0x7f9c1002a8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 -- 172.21.15.33:0/926957122 shutdown_connections 2024-09-10T21:29:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 --2- 172.21.15.33:0/926957122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c2010d100 0x7f9c2010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 --2- 172.21.15.33:0/926957122 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c201045c0 0x7f9c2010cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 --2- 172.21.15.33:0/926957122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c20103ca0 0x7f9c20104080 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 -- 172.21.15.33:0/926957122 >> 172.21.15.33:0/926957122 conn(0x7f9c200fd700 msgr2=0x7f9c200ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 -- 172.21.15.33:0/926957122 shutdown_connections 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.889+0000 7f9c27754640 1 -- 172.21.15.33:0/926957122 wait complete. 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 Processor -- start 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 -- start start 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c20103ca0 0x7f9c20199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c201045c0 0x7f9c201997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c2010d100 0x7f9c2019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9c20112d70 con 0x7f9c201045c0 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9c20112bf0 con 0x7f9c20103ca0 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c27754640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9c20112ef0 con 0x7f9c2010d100 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c201045c0 0x7f9c201997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c201045c0 0x7f9c201997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54922/0 (socket says 172.21.15.33:54922) 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 -- 172.21.15.33:0/2783479819 learned_addr learned my addr 172.21.15.33:0/2783479819 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c254f0640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c20103ca0 0x7f9c20199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c25cf1640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c2010d100 0x7f9c2019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 -- 172.21.15.33:0/2783479819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c2010d100 msgr2=0x7f9c2019eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c2010d100 0x7f9c2019eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 -- 172.21.15.33:0/2783479819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c20103ca0 msgr2=0x7f9c20199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c20103ca0 0x7f9c20199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.890+0000 7f9c24cef640 1 -- 172.21.15.33:0/2783479819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2019f310 con 0x7f9c201045c0 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.891+0000 7f9c24cef640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c201045c0 0x7f9c201997a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9c0 tx=0x7f9c1400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.891+0000 7f9c1e7fc640 1 -- 172.21.15.33:0/2783479819 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400ccc0 con 0x7f9c201045c0 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.891+0000 7f9c1e7fc640 1 -- 172.21.15.33:0/2783479819 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c140035f0 con 0x7f9c201045c0 2024-09-10T21:29:37.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.891+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2019f600 con 0x7f9c201045c0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.891+0000 7f9c1e7fc640 1 -- 172.21.15.33:0/2783479819 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1400ce60 con 0x7f9c201045c0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.891+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c201a5cc0 con 0x7f9c201045c0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.892+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c20104080 con 0x7f9c201045c0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.893+0000 7f9c1e7fc640 1 -- 172.21.15.33:0/2783479819 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9c14004000 con 0x7f9c201045c0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.894+0000 7f9c1e7fc640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c080b52e0 0x7f9c080b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.894+0000 7f9c254f0640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c080b52e0 0x7f9c080b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.894+0000 7f9c1e7fc640 1 -- 172.21.15.33:0/2783479819 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9c140e5e70 con 0x7f9c201045c0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.895+0000 7f9c254f0640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c080b52e0 0x7f9c080b77a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009430 tx=0x7f9c10009360 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:37.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:37.898+0000 7f9c1e7fc640 1 -- 172.21.15.33:0/2783479819 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9c140acd30 con 0x7f9c201045c0 2024-09-10T21:29:37.943 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:37 smithi174 ceph-mon[35635]: pgmap v1115: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-10T21:29:38.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:37 smithi104 ceph-mon[35325]: pgmap v1115: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 51 KiB/s wr, 13 op/s 2024-09-10T21:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.124+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c201a6090 con 0x7f9c201045c0 2024-09-10T21:29:38.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.124+0000 7f9c1e7fc640 1 -- 172.21.15.33:0/2783479819 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+12004 (secure 0 0 0) 0x7f9c140c5f00 con 0x7f9c201045c0 2024-09-10T21:29:38.128 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:29:38.128 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":101,"btime":"2024-09-10T21:29:09:611977+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":101,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:29:09.211896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:29:38.128 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 101 2024-09-10T21:29:38.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.126+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c080b52e0 msgr2=0x7f9c080b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.126+0000 7f9c27754640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c080b52e0 0x7f9c080b77a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9c10009430 tx=0x7f9c10009360 comp rx=0 tx=0).stop 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.126+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c201045c0 msgr2=0x7f9c201997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.126+0000 7f9c27754640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c201045c0 0x7f9c201997a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400e9c0 tx=0x7f9c1400ee80 comp rx=0 tx=0).stop 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 shutdown_connections 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9c080b52e0 0x7f9c080b77a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9c2010d100 0x7f9c2019eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c201045c0 0x7f9c201997a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 --2- 172.21.15.33:0/2783479819 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9c20103ca0 0x7f9c20199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 >> 172.21.15.33:0/2783479819 conn(0x7f9c200fd700 msgr2=0x7f9c200fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 shutdown_connections 2024-09-10T21:29:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:38.127+0000 7f9c27754640 1 -- 172.21.15.33:0/2783479819 wait complete. 2024-09-10T21:29:38.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:38 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2783479819' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:29:38.969 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:38 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2783479819' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:29:39.051 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T21:29:39.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:38 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2783479819' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:29:39.319 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:39.707 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:39 smithi033 ceph-mon[30860]: pgmap v1116: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-10T21:29:39.961 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:39 smithi104 ceph-mon[35325]: pgmap v1116: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 -- 172.21.15.33:0/2917689387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954101c90 msgr2=0x7fa954102070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 --2- 172.21.15.33:0/2917689387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954101c90 0x7fa954102070 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fa950009900 tx=0x7fa95002a410 comp rx=0 tx=0).stop 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 -- 172.21.15.33:0/2917689387 shutdown_connections 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 --2- 172.21.15.33:0/2917689387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954108020 0x7fa95410e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 --2- 172.21.15.33:0/2917689387 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9541076a0 0x7fa954107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 --2- 172.21.15.33:0/2917689387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954101c90 0x7fa954102070 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 -- 172.21.15.33:0/2917689387 >> 172.21.15.33:0/2917689387 conn(0x7fa9540fd700 msgr2=0x7fa9540ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:39.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 -- 172.21.15.33:0/2917689387 shutdown_connections 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.950+0000 7fa95b286640 1 -- 172.21.15.33:0/2917689387 wait complete. 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 Processor -- start 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 -- start start 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954101c90 0x7fa95419b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9541076a0 0x7fa95419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954108020 0x7fa9541a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111ca0 con 0x7fa954108020 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111b20 con 0x7fa9541076a0 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa95b286640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa954111e20 con 0x7fa954101c90 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954108020 0x7fa9541a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954108020 0x7fa9541a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60886/0 (socket says 172.21.15.33:60886) 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959022640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954101c90 0x7fa95419b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 -- 172.21.15.33:0/1660242743 learned_addr learned my addr 172.21.15.33:0/1660242743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa958821640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9541076a0 0x7fa95419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 -- 172.21.15.33:0/1660242743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954101c90 msgr2=0x7fa95419b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954101c90 0x7fa95419b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 -- 172.21.15.33:0/1660242743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9541076a0 msgr2=0x7fa95419ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9541076a0 0x7fa95419ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959823640 1 -- 172.21.15.33:0/1660242743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9541a15b0 con 0x7fa954108020 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa959022640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954101c90 0x7fa95419b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.951+0000 7fa958821640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9541076a0 0x7fa95419ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.952+0000 7fa959823640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954108020 0x7fa9541a0de0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009fd0 tx=0x7fa94c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.952+0000 7fa94a7fc640 1 -- 172.21.15.33:0/1660242743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c017070 con 0x7fa954108020 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.952+0000 7fa94a7fc640 1 -- 172.21.15.33:0/1660242743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa94c003e20 con 0x7fa954108020 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.952+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9541a18a0 con 0x7fa954108020 2024-09-10T21:29:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.952+0000 7fa94a7fc640 1 -- 172.21.15.33:0/1660242743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94c010460 con 0x7fa954108020 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.952+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9541a7f60 con 0x7fa954108020 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.953+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa954102070 con 0x7fa954108020 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.954+0000 7fa94a7fc640 1 -- 172.21.15.33:0/1660242743 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7fa94c003970 con 0x7fa954108020 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.955+0000 7fa94a7fc640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa9400b52e0 0x7fa9400b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.955+0000 7fa959022640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa9400b52e0 0x7fa9400b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.955+0000 7fa94a7fc640 1 -- 172.21.15.33:0/1660242743 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7fa94c0e5940 con 0x7fa954108020 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.956+0000 7fa959022640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa9400b52e0 0x7fa9400b77a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa950031440 tx=0x7fa950005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:39.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:39.959+0000 7fa94a7fc640 1 -- 172.21.15.33:0/1660242743 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7fa94c0ac7c0 con 0x7fa954108020 2024-09-10T21:29:40.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:39 smithi174 ceph-mon[35635]: pgmap v1116: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 52 KiB/s wr, 14 op/s 2024-09-10T21:29:40.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.180+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9541a8330 con 0x7fa954108020 2024-09-10T21:29:40.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.181+0000 7fa94a7fc640 1 -- 172.21.15.33:0/1660242743 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+12004 (secure 0 0 0) 0x7fa94c0ac0c0 con 0x7fa954108020 2024-09-10T21:29:40.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:29:40.184 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":101,"btime":"2024-09-10T21:29:09:611977+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14637,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6835/758676630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6834","nonce":758676630},{"type":"v1","addr":"172.21.15.33:6835","nonce":758676630}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14649,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6837/1550204890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6836","nonce":1550204890},{"type":"v1","addr":"172.21.15.33:6837","nonce":1550204890}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":101,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T21:02:01.445404+0000","modified":"2024-09-10T21:29:09.211896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24566,"mds_1":24547,"mds_2":24539,"mds_3":24532,"mds_4":24526},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14697":{"gid":14697,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6839/295723844","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6838","nonce":295723844},{"type":"v1","addr":"172.21.15.33:6839","nonce":295723844}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14709":{"gid":14709,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.33:6841/1476321443","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6840","nonce":1476321443},{"type":"v1","addr":"172.21.15.33:6841","nonce":1476321443}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6833/2049405106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6832","nonce":2049405106},{"type":"v1","addr":"172.21.15.104:6833","nonce":2049405106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24505":{"gid":24505,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/34665907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":34665907},{"type":"v1","addr":"172.21.15.174:6833","nonce":34665907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24518":{"gid":24518,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.104:6835/1051931570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6834","nonce":1051931570},{"type":"v1","addr":"172.21.15.104:6835","nonce":1051931570}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24526":{"gid":24526,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6835/982304250","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":982304250},{"type":"v1","addr":"172.21.15.174:6835","nonce":982304250}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24532":{"gid":24532,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":10,"addr":"172.21.15.174:6837/1592763439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1592763439},{"type":"v1","addr":"172.21.15.174:6837","nonce":1592763439}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24539":{"gid":24539,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.104:6837/2894143081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6836","nonce":2894143081},{"type":"v1","addr":"172.21.15.104:6837","nonce":2894143081}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24547":{"gid":24547,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.174:6839/3620064873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":3620064873},{"type":"v1","addr":"172.21.15.174:6839","nonce":3620064873}]},"join_fscid":-1,"export_targets":[0,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24566":{"gid":24566,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6839/2034440949","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6838","nonce":2034440949},{"type":"v1","addr":"172.21.15.104:6839","nonce":2034440949}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24566,"qdb_cluster":[24566,24547,24532,24539,24526]},"id":1}]} 2024-09-10T21:29:40.185 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 101 2024-09-10T21:29:40.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa9400b52e0 msgr2=0x7fa9400b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa9400b52e0 0x7fa9400b77a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa950031440 tx=0x7fa950005b50 comp rx=0 tx=0).stop 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954108020 msgr2=0x7fa9541a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954108020 0x7fa9541a0de0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009fd0 tx=0x7fa94c00b260 comp rx=0 tx=0).stop 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 shutdown_connections 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7fa9400b52e0 0x7fa9400b77a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa954108020 0x7fa9541a0de0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9541076a0 0x7fa95419ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 --2- 172.21.15.33:0/1660242743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa954101c90 0x7fa95419b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 >> 172.21.15.33:0/1660242743 conn(0x7fa9540fd700 msgr2=0x7fa954100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 shutdown_connections 2024-09-10T21:29:40.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:40.183+0000 7fa95b286640 1 -- 172.21.15.33:0/1660242743 wait complete. 2024-09-10T21:29:40.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:40 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/1660242743' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:29:40.974 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:40 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/1660242743' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:29:40.979 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T21:29:41.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:40 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/1660242743' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-10T21:29:41.245 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 -- 172.21.15.33:0/906436792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0310102dc0 msgr2=0x7f03101031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 --2- 172.21.15.33:0/906436792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0310102dc0 0x7f03101031a0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f030c009900 tx=0x7f030c02a410 comp rx=0 tx=0).stop 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 -- 172.21.15.33:0/906436792 shutdown_connections 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 --2- 172.21.15.33:0/906436792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0310109150 0x7f031010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 --2- 172.21.15.33:0/906436792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f03101087d0 0x7f0310108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 --2- 172.21.15.33:0/906436792 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0310102dc0 0x7f03101031a0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 -- 172.21.15.33:0/906436792 >> 172.21.15.33:0/906436792 conn(0x7f03100fe830 msgr2=0x7f0310100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 -- 172.21.15.33:0/906436792 shutdown_connections 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.720+0000 7f031766e640 1 -- 172.21.15.33:0/906436792 wait complete. 2024-09-10T21:29:41.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 Processor -- start 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 -- start start 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0310102dc0 0x7f03100754a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03101087d0 0x7f03100759e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0310109150 0x7f0310075f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f0310112e10 con 0x7f03101087d0 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f0310112c90 con 0x7f0310109150 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f031766e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0310112f90 con 0x7f0310102dc0 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.721+0000 7f0314c09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03101087d0 0x7f03100759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03101087d0 0x7f03100759e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60898/0 (socket says 172.21.15.33:60898) 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 -- 172.21.15.33:0/3773948414 learned_addr learned my addr 172.21.15.33:0/3773948414 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0315c0b640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0310109150 0x7f0310075f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f031540a640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0310102dc0 0x7f03100754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 -- 172.21.15.33:0/3773948414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0310102dc0 msgr2=0x7f03100754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0310102dc0 0x7f03100754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 -- 172.21.15.33:0/3773948414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0310109150 msgr2=0x7f0310075f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0310109150 0x7f0310075f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 -- 172.21.15.33:0/3773948414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03101a76e0 con 0x7f03101087d0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f0314c09640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03101087d0 0x7f03100759e0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f030000e9f0 tx=0x7f030000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f03067fc640 1 -- 172.21.15.33:0/3773948414 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000cd80 con 0x7f03101087d0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f03067fc640 1 -- 172.21.15.33:0/3773948414 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03000108e0 con 0x7f03101087d0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03101a79d0 con 0x7f03101087d0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f03067fc640 1 -- 172.21.15.33:0/3773948414 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300018980 con 0x7f03101087d0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.722+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03101a7f60 con 0x7f03101087d0 2024-09-10T21:29:41.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.723+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03101031a0 con 0x7f03101087d0 2024-09-10T21:29:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.725+0000 7f03067fc640 1 -- 172.21.15.33:0/3773948414 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f0300010430 con 0x7f03101087d0 2024-09-10T21:29:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.725+0000 7f03067fc640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02fc0b5360 0x7f02fc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.726+0000 7f031540a640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02fc0b5360 0x7f02fc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.726+0000 7f03067fc640 1 -- 172.21.15.33:0/3773948414 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f03000e5f90 con 0x7f03101087d0 2024-09-10T21:29:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.726+0000 7f031540a640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02fc0b5360 0x7f02fc0b7820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f030c031440 tx=0x7f030c0095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:41.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.729+0000 7f03067fc640 1 -- 172.21.15.33:0/3773948414 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f03000ace10 con 0x7f03101087d0 2024-09-10T21:29:41.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.917+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03101a8330 con 0x7f03101087d0 2024-09-10T21:29:41.921 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:41 smithi033 ceph-mon[30860]: pgmap v1117: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-09-10T21:29:41.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.919+0000 7f03067fc640 1 -- 172.21.15.33:0/3773948414 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+22763 (secure 0 0 0) 0x7f03000ac710 con 0x7f03101087d0 2024-09-10T21:29:41.922 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-09-10T21:29:41.922 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":77,"fsid":"a07b33bc-6fb6-11ef-bcea-c7b262605968","created":"2024-09-10T20:53:48.210364+0000","modified":"2024-09-10T21:02:41.855317+0000","last_up_change":"2024-09-10T21:01:11.170660+0000","last_in_change":"2024-09-10T21:00:48.406331+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T20:57:24.470902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:01:57.440910+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:01:59.340474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"343a51e3-d26a-4f11-9714-d6464a31c878","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6803","nonce":3008534631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6805","nonce":3008534631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6809","nonce":3008534631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3008534631},{"type":"v1","addr":"172.21.15.33:6807","nonce":3008534631}]},"public_addr":"172.21.15.33:6803/3008534631","cluster_addr":"172.21.15.33:6805/3008534631","heartbeat_back_addr":"172.21.15.33:6809/3008534631","heartbeat_front_addr":"172.21.15.33:6807/3008534631","state":["exists","up"]},{"osd":1,"uuid":"a51a4510-302a-4343-8cce-ee4b0c562880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6801","nonce":2112783762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6803","nonce":2112783762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6807","nonce":2112783762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2112783762},{"type":"v1","addr":"172.21.15.104:6805","nonce":2112783762}]},"public_addr":"172.21.15.104:6801/2112783762","cluster_addr":"172.21.15.104:6803/2112783762","heartbeat_back_addr":"172.21.15.104:6807/2112783762","heartbeat_front_addr":"172.21.15.104:6805/2112783762","state":["exists","up"]},{"osd":2,"uuid":"ae175f96-8529-4db0-83bd-7ee10b49dddf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":17,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6801","nonce":4030111255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6803","nonce":4030111255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6807","nonce":4030111255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":4030111255},{"type":"v1","addr":"172.21.15.174:6805","nonce":4030111255}]},"public_addr":"172.21.15.174:6801/4030111255","cluster_addr":"172.21.15.174:6803/4030111255","heartbeat_back_addr":"172.21.15.174:6807/4030111255","heartbeat_front_addr":"172.21.15.174:6805/4030111255","state":["exists","up"]},{"osd":3,"uuid":"87f711da-d77c-452d-9a29-3418de2ba617","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6811","nonce":2053264902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6813","nonce":2053264902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6817","nonce":2053264902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":2053264902},{"type":"v1","addr":"172.21.15.33:6815","nonce":2053264902}]},"public_addr":"172.21.15.33:6811/2053264902","cluster_addr":"172.21.15.33:6813/2053264902","heartbeat_back_addr":"172.21.15.33:6817/2053264902","heartbeat_front_addr":"172.21.15.33:6815/2053264902","state":["exists","up"]},{"osd":4,"uuid":"02b1eb40-cf26-4cee-9c4b-c5a353a8537a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6809","nonce":3963281579}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6811","nonce":3963281579}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6815","nonce":3963281579}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":3963281579},{"type":"v1","addr":"172.21.15.104:6813","nonce":3963281579}]},"public_addr":"172.21.15.104:6809/3963281579","cluster_addr":"172.21.15.104:6811/3963281579","heartbeat_back_addr":"172.21.15.104:6815/3963281579","heartbeat_front_addr":"172.21.15.104:6813/3963281579","state":["exists","up"]},{"osd":5,"uuid":"e7d7361f-8c71-472e-84a7-bedf288dc9bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6809","nonce":1209422453}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6811","nonce":1209422453}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6815","nonce":1209422453}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1209422453},{"type":"v1","addr":"172.21.15.174:6813","nonce":1209422453}]},"public_addr":"172.21.15.174:6809/1209422453","cluster_addr":"172.21.15.174:6811/1209422453","heartbeat_back_addr":"172.21.15.174:6815/1209422453","heartbeat_front_addr":"172.21.15.174:6813/1209422453","state":["exists","up"]},{"osd":6,"uuid":"03e0706e-6a19-42c7-97c4-54dbe8877a69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":41,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6819","nonce":3359699940}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6821","nonce":3359699940}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6825","nonce":3359699940}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":3359699940},{"type":"v1","addr":"172.21.15.33:6823","nonce":3359699940}]},"public_addr":"172.21.15.33:6819/3359699940","cluster_addr":"172.21.15.33:6821/3359699940","heartbeat_back_addr":"172.21.15.33:6825/3359699940","heartbeat_front_addr":"172.21.15.33:6823/3359699940","state":["exists","up"]},{"osd":7,"uuid":"5f9abb9d-783e-4ded-93ea-4604eb6df315","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6817","nonce":947117996}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6819","nonce":947117996}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6823","nonce":947117996}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":947117996},{"type":"v1","addr":"172.21.15.104:6821","nonce":947117996}]},"public_addr":"172.21.15.104:6817/947117996","cluster_addr":"172.21.15.104:6819/947117996","heartbeat_back_addr":"172.21.15.104:6823/947117996","heartbeat_front_addr":"172.21.15.104:6821/947117996","state":["exists","up"]},{"osd":8,"uuid":"f077a3ba-8e8e-4d9f-9f6c-0959dbe11e69","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6817","nonce":758997036}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6819","nonce":758997036}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6823","nonce":758997036}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":758997036},{"type":"v1","addr":"172.21.15.174:6821","nonce":758997036}]},"public_addr":"172.21.15.174:6817/758997036","cluster_addr":"172.21.15.174:6819/758997036","heartbeat_back_addr":"172.21.15.174:6823/758997036","heartbeat_front_addr":"172.21.15.174:6821/758997036","state":["exists","up"]},{"osd":9,"uuid":"dbae656c-994b-4a32-a22d-de19664248e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":58,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6827","nonce":3501032223}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6829","nonce":3501032223}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6832","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6833","nonce":3501032223}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6830","nonce":3501032223},{"type":"v1","addr":"172.21.15.33:6831","nonce":3501032223}]},"public_addr":"172.21.15.33:6827/3501032223","cluster_addr":"172.21.15.33:6829/3501032223","heartbeat_back_addr":"172.21.15.33:6833/3501032223","heartbeat_front_addr":"172.21.15.33:6831/3501032223","state":["exists","up"]},{"osd":10,"uuid":"f020be25-441a-4f54-8077-3d0ea98e1181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6825","nonce":575213460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6827","nonce":575213460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6830","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6831","nonce":575213460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6828","nonce":575213460},{"type":"v1","addr":"172.21.15.104:6829","nonce":575213460}]},"public_addr":"172.21.15.104:6825/575213460","cluster_addr":"172.21.15.104:6827/575213460","heartbeat_back_addr":"172.21.15.104:6831/575213460","heartbeat_front_addr":"172.21.15.104:6829/575213460","state":["exists","up"]},{"osd":11,"uuid":"9b6009d5-69cd-4c9c-9387-bc1fdceb041d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6825","nonce":117399142}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6827","nonce":117399142}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6831","nonce":117399142}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":117399142},{"type":"v1","addr":"172.21.15.174:6829","nonce":117399142}]},"public_addr":"172.21.15.174:6825/117399142","cluster_addr":"172.21.15.174:6827/117399142","heartbeat_back_addr":"172.21.15.174:6831/117399142","heartbeat_front_addr":"172.21.15.174:6829/117399142","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:31.093026+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:56:55.913628+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:21.184920+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:57:46.374473+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:10.768698+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:58:36.356152+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:01.784934+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:28.086383+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T20:59:53.316563+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:19.619028+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:00:44.415811+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:01:09.876592+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.2d","mappings":[{"from":3,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:0/2096552792":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/2468168068":"2024-09-11T20:54:39.992584+0000","172.21.15.33:0/2596082174":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2930837239":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6801/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2648657406":"2024-09-11T20:54:39.992584+0000","172.21.15.33:6800/1151081438":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/4287278386":"2024-09-11T20:54:14.879196+0000","172.21.15.33:0/2461327124":"2024-09-11T20:54:14.879196+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02fc0b5360 msgr2=0x7f02fc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02fc0b5360 0x7f02fc0b7820 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f030c031440 tx=0x7f030c0095f0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03101087d0 msgr2=0x7f03100759e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03101087d0 0x7f03100759e0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f030000e9f0 tx=0x7f030000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 shutdown_connections 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f02fc0b5360 0x7f02fc0b7820 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0310109150 0x7f0310075f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03101087d0 0x7f03100759e0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 --2- 172.21.15.33:0/3773948414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0310102dc0 0x7f03100754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 >> 172.21.15.33:0/3773948414 conn(0x7f03100fe830 msgr2=0x7f03101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 shutdown_connections 2024-09-10T21:29:41.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:41.921+0000 7f031766e640 1 -- 172.21.15.33:0/3773948414 wait complete. 2024-09-10T21:29:42.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:41 smithi104 ceph-mon[35325]: pgmap v1117: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-09-10T21:29:42.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:41 smithi174 ceph-mon[35635]: pgmap v1117: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 52 KiB/s wr, 15 op/s 2024-09-10T21:29:42.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-10T21:29:43.015 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:43.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:42 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3773948414' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:29:43.136 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:42 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3773948414' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:29:43.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:42 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3773948414' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T21:29:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 -- 172.21.15.33:0/1135659742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c101cd0 msgr2=0x7f9b8c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 --2- 172.21.15.33:0/1135659742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c101cd0 0x7f9b8c1020b0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009900 tx=0x7f9b7c02a410 comp rx=0 tx=0).stop 2024-09-10T21:29:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 -- 172.21.15.33:0/1135659742 shutdown_connections 2024-09-10T21:29:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 --2- 172.21.15.33:0/1135659742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b8c108060 0x7f9b8c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 --2- 172.21.15.33:0/1135659742 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8c1076e0 0x7f9b8c107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 --2- 172.21.15.33:0/1135659742 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c101cd0 0x7f9b8c1020b0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 -- 172.21.15.33:0/1135659742 >> 172.21.15.33:0/1135659742 conn(0x7f9b8c0fd720 msgr2=0x7f9b8c0ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 -- 172.21.15.33:0/1135659742 shutdown_connections 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.528+0000 7f9b9239c640 1 -- 172.21.15.33:0/1135659742 wait complete. 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 Processor -- start 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 -- start start 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b8c101cd0 0x7f9b8c198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c1076e0 0x7f9b8c198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8c108060 0x7f9b8c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c111d70 con 0x7f9b8c1076e0 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c111bf0 con 0x7f9b8c108060 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b9239c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c111ef0 con 0x7f9b8c101cd0 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c1076e0 0x7f9b8c198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b8b7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c1076e0 0x7f9b8c198660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60926/0 (socket says 172.21.15.33:60926) 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.529+0000 7f9b8b7fe640 1 -- 172.21.15.33:0/3129722081 learned_addr learned my addr 172.21.15.33:0/3129722081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b8bfff640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b8c101cd0 0x7f9b8c198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b90939640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8c108060 0x7f9b8c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b8b7fe640 1 -- 172.21.15.33:0/3129722081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b8c101cd0 msgr2=0x7f9b8c198120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b8b7fe640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b8c101cd0 0x7f9b8c198120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b8b7fe640 1 -- 172.21.15.33:0/3129722081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8c108060 msgr2=0x7f9b8c19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b8b7fe640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8c108060 0x7f9b8c19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b8b7fe640 1 -- 172.21.15.33:0/3129722081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c19e1d0 con 0x7f9b8c1076e0 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b90939640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8c108060 0x7f9b8c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b8b7fe640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c1076e0 0x7f9b8c198660 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d950 tx=0x7f9b8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b897fa640 1 -- 172.21.15.33:0/3129722081 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80017030 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c19e4c0 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b897fa640 1 -- 172.21.15.33:0/3129722081 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b80004020 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b897fa640 1 -- 172.21.15.33:0/3129722081 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b800047a0 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.530+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c1a4d50 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.531+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c1020b0 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.533+0000 7f9b897fa640 1 -- 172.21.15.33:0/3129722081 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f9b80009a30 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.534+0000 7f9b897fa640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9b740b52e0 0x7f9b740b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.534+0000 7f9b8bfff640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9b740b52e0 0x7f9b740b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.534+0000 7f9b897fa640 1 -- 172.21.15.33:0/3129722081 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8240+0+0 (secure 0 0 0) 0x7f9b800e5dd0 con 0x7f9b8c1076e0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.534+0000 7f9b8bfff640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9b740b52e0 0x7f9b740b77a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c031400 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.537+0000 7f9b897fa640 1 -- 172.21.15.33:0/3129722081 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f9b800acc50 con 0x7f9b8c1076e0 2024-09-10T21:29:43.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.762+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9b8c199250 con 0x7f9b8c1076e0 2024-09-10T21:29:43.764 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-mon[30860]: pgmap v1118: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-09-10T21:29:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.764+0000 7f9b897fa640 1 -- 172.21.15.33:0/3129722081 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) ==== 161+0+0 (secure 0 0 0) 0x7f9b800ac550 con 0x7f9b8c1076e0 2024-09-10T21:29:43.766 INFO:teuthology.orchestra.run.smithi033.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:29:43.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9b740b52e0 msgr2=0x7f9b740b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9b740b52e0 0x7f9b740b77a0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c000c00 tx=0x7f9b7c031400 comp rx=0 tx=0).stop 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c1076e0 msgr2=0x7f9b8c198660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c1076e0 0x7f9b8c198660 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f9b8000d950 tx=0x7f9b8000de10 comp rx=0 tx=0).stop 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 shutdown_connections 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f9b740b52e0 0x7f9b740b77a0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8c108060 0x7f9b8c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b8c1076e0 0x7f9b8c198660 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 --2- 172.21.15.33:0/3129722081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b8c101cd0 0x7f9b8c198120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 >> 172.21.15.33:0/3129722081 conn(0x7f9b8c0fd720 msgr2=0x7f9b8c0fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 shutdown_connections 2024-09-10T21:29:43.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:43.767+0000 7f9b9239c640 1 -- 172.21.15.33:0/3129722081 wait complete. 2024-09-10T21:29:44.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-mon[35325]: pgmap v1118: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-09-10T21:29:44.061 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h[65360]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.061 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h[65360]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.061 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-mds[65364]: -- 172.21.15.104:0/477303532 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55916f08dc20 con 0x55916f06b400 2024-09-10T21:29:44.061 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h[65360]: starting mds.h at 2024-09-10T21:29:44.061 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.062 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.062 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-mds[65661]: -- 172.21.15.104:0/322430513 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f804665860 con 0x55f804643000 2024-09-10T21:29:44.062 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: starting mds.b at 2024-09-10T21:29:44.062 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e[65058]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.062 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e[65058]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.062 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-mds[65062]: -- 172.21.15.104:0/1648760871 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2a8d19860 con 0x55b2a8cf7000 2024-09-10T21:29:44.062 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e[65058]: starting mds.e at 2024-09-10T21:29:44.063 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k[64747]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.063 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k[64747]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.063 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-mds[64751]: -- 172.21.15.104:0/865412189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564d42191860 con 0x564d4216e800 2024-09-10T21:29:44.063 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:29:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k[64747]: starting mds.k at 2024-09-10T21:29:44.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-mon[35635]: pgmap v1118: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 53 KiB/s wr, 13 op/s 2024-09-10T21:29:44.142 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i[66698]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.142 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i[66698]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.142 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-mds[66702]: -- 172.21.15.174:0/872521258 <== mon.2 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561be121f860 con 0x561be11fd400 2024-09-10T21:29:44.143 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i[66698]: starting mds.i at 2024-09-10T21:29:44.143 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c[66398]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.143 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c[66398]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.143 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-mds[66402]: -- 172.21.15.174:0/4143170518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5639de5a5860 con 0x5639de582800 2024-09-10T21:29:44.143 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c[66398]: starting mds.c at 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f[66095]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f[66095]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-mds[66099]: -- 172.21.15.174:0/2686199169 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560508173860 con 0x560508150800 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f[66095]: starting mds.f at 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l[65793]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l[65793]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-mds[65797]: -- 172.21.15.174:0/38845998 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561b19261c20 con 0x561b1923f000 2024-09-10T21:29:44.144 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:29:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l[65793]: starting mds.l at 2024-09-10T21:29:44.149 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a[30856]: 2024-09-10T21:29:43.762+0000 7f3885e53640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:29:44.149 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a[74391]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.149 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a[74391]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.149 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-mds[74395]: -- 172.21.15.33:0/3255112730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559173891860 con 0x55917386e800 2024-09-10T21:29:44.149 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a[74391]: starting mds.a at 2024-09-10T21:29:44.150 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g[74004]: ignoring --setuser ceph since I am not root 2024-09-10T21:29:44.150 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g[74004]: ignoring --setgroup ceph since I am not root 2024-09-10T21:29:44.150 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-mds[74020]: -- 172.21.15.33:0/109009873 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fd25055c20 con 0x55fd25033000 2024-09-10T21:29:44.150 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:29:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g[74004]: starting mds.g at 2024-09-10T21:29:44.585 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:29:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:44.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:44 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3129722081' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T21:29:44.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:44 smithi033 ceph-mon[30860]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T21:29:44.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:44 smithi033 ceph-mon[30860]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:29:44.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:44 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/3129722081' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T21:29:44.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:44 smithi033 ceph-mon[30860]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T21:29:44.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:44 smithi033 ceph-mon[30860]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T21:29:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:44 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3129722081' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T21:29:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:44 smithi104 ceph-mon[35325]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T21:29:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:44 smithi104 ceph-mon[35325]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:29:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:44 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/3129722081' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T21:29:45.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:44 smithi104 ceph-mon[35325]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T21:29:45.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:44 smithi104 ceph-mon[35325]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T21:29:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:44 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3129722081' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-10T21:29:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:44 smithi174 ceph-mon[35635]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T21:29:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:44 smithi174 ceph-mon[35635]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:29:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:44 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/3129722081' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T21:29:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:44 smithi174 ceph-mon[35635]: osdmap e78: 12 total, 12 up, 12 in 2024-09-10T21:29:45.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:44 smithi174 ceph-mon[35635]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-10T21:29:45.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 -- 172.21.15.33:0/2691549264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 msgr2=0x7f74c010e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 --2- 172.21.15.33:0/2691549264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c010e820 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f74b8009ca0 tx=0x7f74b802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 -- 172.21.15.33:0/2691549264 shutdown_connections 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 --2- 172.21.15.33:0/2691549264 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c010e820 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 --2- 172.21.15.33:0/2691549264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c01076a0 0x7f74c0107ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 --2- 172.21.15.33:0/2691549264 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c0101c90 0x7f74c0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 -- 172.21.15.33:0/2691549264 >> 172.21.15.33:0/2691549264 conn(0x7f74c00fd700 msgr2=0x7f74c00ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 -- 172.21.15.33:0/2691549264 shutdown_connections 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 -- 172.21.15.33:0/2691549264 wait complete. 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.353+0000 7f74c713e640 1 Processor -- start 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c713e640 1 -- start start 2024-09-10T21:29:45.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c713e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0101c90 0x7f74c0199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c713e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01076a0 0x7f74c01997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c713e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c019eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c713e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112e00 con 0x7f74c0108020 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c713e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112c80 con 0x7f74c01076a0 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c713e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74c0112f80 con 0x7f74c0101c90 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c019eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c019eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60990/0 (socket says 172.21.15.33:60990) 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 -- 172.21.15.33:0/2684462025 learned_addr learned my addr 172.21.15.33:0/2684462025 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74bffff640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01076a0 0x7f74c01997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c4eda640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0101c90 0x7f74c0199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 -- 172.21.15.33:0/2684462025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0101c90 msgr2=0x7f74c0199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0101c90 0x7f74c0199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 -- 172.21.15.33:0/2684462025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01076a0 msgr2=0x7f74c01997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01076a0 0x7f74c01997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 -- 172.21.15.33:0/2684462025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74c019f320 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74bffff640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01076a0 0x7f74c01997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.354+0000 7f74c56db640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c019eb50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005cc0 tx=0x7f74b802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.355+0000 7f74bdffb640 1 -- 172.21.15.33:0/2684462025 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b8003bf0 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.355+0000 7f74bdffb640 1 -- 172.21.15.33:0/2684462025 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b8003d90 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.355+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74c019f5b0 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.355+0000 7f74bdffb640 1 -- 172.21.15.33:0/2684462025 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b802fbc0 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.355+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74c01a5cd0 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.356+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74c0102070 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.357+0000 7f74bdffb640 1 -- 172.21.15.33:0/2684462025 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f74b8004610 con 0x7f74c0108020 2024-09-10T21:29:45.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.358+0000 7f74bdffb640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74a80b52e0 0x7f74a80b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.358+0000 7f74c4eda640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74a80b52e0 0x7f74a80b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.358+0000 7f74bdffb640 1 -- 172.21.15.33:0/2684462025 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8670+0+0 (secure 0 0 0) 0x7f74b8107510 con 0x7f74c0108020 2024-09-10T21:29:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.359+0000 7f74c4eda640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74a80b52e0 0x7f74a80b77a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f74b00058c0 tx=0x7f74b000a4c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:45.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.361+0000 7f74bdffb640 1 -- 172.21.15.33:0/2684462025 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f74b80ce1c0 con 0x7f74c0108020 2024-09-10T21:29:45.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.587+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f74c002e5c0 con 0x7f74c0108020 2024-09-10T21:29:45.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.777+0000 7f74bdffb640 1 -- 172.21.15.33:0/2684462025 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) ==== 104+0+0 (secure 0 0 0) 0x7f74b80098b0 con 0x7f74c0108020 2024-09-10T21:29:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74a80b52e0 msgr2=0x7f74a80b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74a80b52e0 0x7f74a80b77a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f74b00058c0 tx=0x7f74b000a4c0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 msgr2=0x7f74c019eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c019eb50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f74b8005cc0 tx=0x7f74b802f380 comp rx=0 tx=0).stop 2024-09-10T21:29:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 shutdown_connections 2024-09-10T21:29:45.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f74a80b52e0 0x7f74a80b77a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74c0108020 0x7f74c019eb50 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74c01076a0 0x7f74c01997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 --2- 172.21.15.33:0/2684462025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74c0101c90 0x7f74c0199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 >> 172.21.15.33:0/2684462025 conn(0x7f74c00fd700 msgr2=0x7f74c00ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 shutdown_connections 2024-09-10T21:29:45.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:45.780+0000 7f74c713e640 1 -- 172.21.15.33:0/2684462025 wait complete. 2024-09-10T21:29:45.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: pgmap v1120: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.33:6839/1811546231,v1:172.21.15.33:6841/1811546231] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.33:6838/3634096377,v1:172.21.15.33:6840/3634096377] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6833/709566045,v1:172.21.15.104:6836/709566045] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6832/2945569319,v1:172.21.15.104:6837/2945569319] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6834/215620443,v1:172.21.15.104:6838/215620443] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.104:6835/652106490,v1:172.21.15.104:6839/652106490] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6832/3648593753,v1:172.21.15.174:6834/3648593753] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6833/550406729,v1:172.21.15.174:6835/550406729] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6836/2351685392,v1:172.21.15.174:6838/2351685392] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: mds.? [v2:172.21.15.174:6837/1063174277,v1:172.21.15.174:6839/1063174277] up:boot 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T21:29:45.926 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T21:29:45.927 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:45 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2684462025' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:29:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: pgmap v1120: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-10T21:29:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.33:6839/1811546231,v1:172.21.15.33:6841/1811546231] up:boot 2024-09-10T21:29:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.33:6838/3634096377,v1:172.21.15.33:6840/3634096377] up:boot 2024-09-10T21:29:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6833/709566045,v1:172.21.15.104:6836/709566045] up:boot 2024-09-10T21:29:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6832/2945569319,v1:172.21.15.104:6837/2945569319] up:boot 2024-09-10T21:29:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6834/215620443,v1:172.21.15.104:6838/215620443] up:boot 2024-09-10T21:29:46.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.104:6835/652106490,v1:172.21.15.104:6839/652106490] up:boot 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6832/3648593753,v1:172.21.15.174:6834/3648593753] up:boot 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6833/550406729,v1:172.21.15.174:6835/550406729] up:boot 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6836/2351685392,v1:172.21.15.174:6838/2351685392] up:boot 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: mds.? [v2:172.21.15.174:6837/1063174277,v1:172.21.15.174:6839/1063174277] up:boot 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T21:29:46.062 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T21:29:46.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T21:29:46.063 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:45 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2684462025' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:29:46.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: pgmap v1120: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 14 op/s 2024-09-10T21:29:46.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.33:6839/1811546231,v1:172.21.15.33:6841/1811546231] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.33:6838/3634096377,v1:172.21.15.33:6840/3634096377] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6833/709566045,v1:172.21.15.104:6836/709566045] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6832/2945569319,v1:172.21.15.104:6837/2945569319] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6834/215620443,v1:172.21.15.104:6838/215620443] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.104:6835/652106490,v1:172.21.15.104:6839/652106490] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6832/3648593753,v1:172.21.15.174:6834/3648593753] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6833/550406729,v1:172.21.15.174:6835/550406729] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6836/2351685392,v1:172.21.15.174:6838/2351685392] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: mds.? [v2:172.21.15.174:6837/1063174277,v1:172.21.15.174:6839/1063174277] up:boot 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-10T21:29:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='mgr.14148 172.21.15.33:0/1257686108' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-10T21:29:46.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:45 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2684462025' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-10T21:29:46.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:29:46.866 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:46.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:46 smithi033 ceph-mon[30860]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T21:29:46.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:46 smithi033 ceph-mon[30860]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T21:29:46.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:46 smithi033 ceph-mon[30860]: Cluster is now healthy 2024-09-10T21:29:46.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:46 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2684462025' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T21:29:46.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:46 smithi033 ceph-mon[30860]: fsmap 12 up:standby 2024-09-10T21:29:47.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:46 smithi104 ceph-mon[35325]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T21:29:47.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:46 smithi104 ceph-mon[35325]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T21:29:47.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:46 smithi104 ceph-mon[35325]: Cluster is now healthy 2024-09-10T21:29:47.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:46 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2684462025' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T21:29:47.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:46 smithi104 ceph-mon[35325]: fsmap 12 up:standby 2024-09-10T21:29:47.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:46 smithi174 ceph-mon[35635]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T21:29:47.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:46 smithi174 ceph-mon[35635]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T21:29:47.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:46 smithi174 ceph-mon[35635]: Cluster is now healthy 2024-09-10T21:29:47.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:46 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2684462025' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-10T21:29:47.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:46 smithi174 ceph-mon[35635]: fsmap 12 up:standby 2024-09-10T21:29:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 -- 172.21.15.33:0/2937027701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 msgr2=0x7f3e041020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 --2- 172.21.15.33:0/2937027701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e041020b0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3df4009900 tx=0x7f3df402a410 comp rx=0 tx=0).stop 2024-09-10T21:29:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 -- 172.21.15.33:0/2937027701 shutdown_connections 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 --2- 172.21.15.33:0/2937027701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e04108060 0x7f3e0410e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 --2- 172.21.15.33:0/2937027701 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e041076e0 0x7f3e04107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 --2- 172.21.15.33:0/2937027701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e041020b0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 -- 172.21.15.33:0/2937027701 >> 172.21.15.33:0/2937027701 conn(0x7f3e040fd720 msgr2=0x7f3e040ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 -- 172.21.15.33:0/2937027701 shutdown_connections 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.390+0000 7f3e08b39640 1 -- 172.21.15.33:0/2937027701 wait complete. 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.391+0000 7f3e08b39640 1 Processor -- start 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.391+0000 7f3e08b39640 1 -- start start 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e08b39640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e0419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e08b39640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e041076e0 0x7f3e0419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e08b39640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e04108060 0x7f3e0419ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:47.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e08b39640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04111d20 con 0x7f3e04101cd0 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e08b39640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04111ba0 con 0x7f3e041076e0 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e08b39640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3e04111ea0 con 0x7f3e04108060 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e0419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e0419a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32778/0 (socket says 172.21.15.33:32778) 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 -- 172.21.15.33:0/4291067834 learned_addr learned my addr 172.21.15.33:0/4291067834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e02d9d640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e04108060 0x7f3e0419ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3df9d9b640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e041076e0 0x7f3e0419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 -- 172.21.15.33:0/4291067834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e04108060 msgr2=0x7f3e0419ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e04108060 0x7f3e0419ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 -- 172.21.15.33:0/4291067834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e041076e0 msgr2=0x7f3e0419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e041076e0 0x7f3e0419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 -- 172.21.15.33:0/4291067834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e041a5450 con 0x7f3e04101cd0 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e02d9d640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e04108060 0x7f3e0419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3df9d9b640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e041076e0 0x7f3e0419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e0259c640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e0419a070 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005940 tx=0x7f3df40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3dfbfff640 1 -- 172.21.15.33:0/4291067834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df403f030 con 0x7f3e04101cd0 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3dfbfff640 1 -- 172.21.15.33:0/4291067834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3df4003cb0 con 0x7f3e04101cd0 2024-09-10T21:29:47.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e041a56e0 con 0x7f3e04101cd0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.392+0000 7f3dfbfff640 1 -- 172.21.15.33:0/4291067834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df4035070 con 0x7f3e04101cd0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.393+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e041a5bc0 con 0x7f3e04101cd0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.393+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e041020b0 con 0x7f3e04101cd0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.395+0000 7f3dfbfff640 1 -- 172.21.15.33:0/4291067834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f3df403bb50 con 0x7f3e04101cd0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.395+0000 7f3dfbfff640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3df00b52e0 0x7f3df00b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.396+0000 7f3df9d9b640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3df00b52e0 0x7f3df00b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.396+0000 7f3dfbfff640 1 -- 172.21.15.33:0/4291067834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8670+0+0 (secure 0 0 0) 0x7f3df4106c30 con 0x7f3e04101cd0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.396+0000 7f3df9d9b640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3df00b52e0 0x7f3df00b77a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0058c0 tx=0x7f3dec005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:47.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.398+0000 7f3dfbfff640 1 -- 172.21.15.33:0/4291067834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f3df40cd940 con 0x7f3e04101cd0 2024-09-10T21:29:47.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.585+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3e0402e5c0 con 0x7f3e04101cd0 2024-09-10T21:29:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.785+0000 7f3dfbfff640 1 -- 172.21.15.33:0/4291067834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) ==== 181+0+0 (secure 0 0 0) 0x7f3df40cd240 con 0x7f3e04101cd0 2024-09-10T21:29:47.787 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:29:47.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3df00b52e0 msgr2=0x7f3df00b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3df00b52e0 0x7f3df00b77a0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f3dec0058c0 tx=0x7f3dec005850 comp rx=0 tx=0).stop 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 msgr2=0x7f3e0419a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e0419a070 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3df4005940 tx=0x7f3df40042f0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 shutdown_connections 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f3df00b52e0 0x7f3df00b77a0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3e04108060 0x7f3e0419ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3e041076e0 0x7f3e0419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 --2- 172.21.15.33:0/4291067834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3e04101cd0 0x7f3e0419a070 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 >> 172.21.15.33:0/4291067834 conn(0x7f3e040fd720 msgr2=0x7f3e040ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 shutdown_connections 2024-09-10T21:29:47.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:47.789+0000 7f3e08b39640 1 -- 172.21.15.33:0/4291067834 wait complete. 2024-09-10T21:29:47.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:47 smithi033 ceph-mon[30860]: pgmap v1121: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T21:29:47.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:47 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4291067834' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T21:29:48.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:47 smithi104 ceph-mon[35325]: pgmap v1121: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T21:29:48.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:47 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4291067834' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T21:29:48.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:47 smithi174 ceph-mon[35635]: pgmap v1121: 97 pgs: 97 active+clean; 975 MiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-09-10T21:29:48.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:47 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4291067834' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T21:29:48.613 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30 shell --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:29:48.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:48 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/4291067834' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T21:29:48.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:48 smithi033 ceph-mon[30860]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T21:29:48.957 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T21:29:49.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:48 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/4291067834' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T21:29:49.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:48 smithi104 ceph-mon[35325]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T21:29:49.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:48 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/4291067834' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T21:29:49.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:48 smithi174 ceph-mon[35635]: osdmap e79: 12 total, 12 up, 12 in 2024-09-10T21:29:49.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 -- 172.21.15.33:0/4279303261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff0101cd0 msgr2=0x7f5ff01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 --2- 172.21.15.33:0/4279303261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff0101cd0 0x7f5ff01020b0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0009900 tx=0x7f5fe002a410 comp rx=0 tx=0).stop 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 -- 172.21.15.33:0/4279303261 shutdown_connections 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 --2- 172.21.15.33:0/4279303261 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ff0108060 0x7f5ff010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 --2- 172.21.15.33:0/4279303261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ff01076e0 0x7f5ff0107b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 --2- 172.21.15.33:0/4279303261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff0101cd0 0x7f5ff01020b0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 -- 172.21.15.33:0/4279303261 >> 172.21.15.33:0/4279303261 conn(0x7f5ff00fd720 msgr2=0x7f5ff00ffb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 -- 172.21.15.33:0/4279303261 shutdown_connections 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 -- 172.21.15.33:0/4279303261 wait complete. 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.450+0000 7f5ff549f640 1 Processor -- start 2024-09-10T21:29:49.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5ff549f640 1 -- start start 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5ff549f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ff0101cd0 0x7f5ff019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5ff549f640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff01076e0 0x7f5ff019a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5ff549f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ff0108060 0x7f5ff01a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5ff549f640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff0111ce0 con 0x7f5ff01076e0 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5ff549f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff0111b60 con 0x7f5ff0101cd0 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5ff549f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff0111e60 con 0x7f5ff0108060 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff01076e0 0x7f5ff019a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff01076e0 0x7f5ff019a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36400/0 (socket says 172.21.15.33:36400) 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 -- 172.21.15.33:0/2118498685 learned_addr learned my addr 172.21.15.33:0/2118498685 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fef7fe640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ff0108060 0x7f5ff01a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5feeffd640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ff0101cd0 0x7f5ff019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 -- 172.21.15.33:0/2118498685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ff0108060 msgr2=0x7f5ff01a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ff0108060 0x7f5ff01a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 -- 172.21.15.33:0/2118498685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ff0101cd0 msgr2=0x7f5ff019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ff0101cd0 0x7f5ff019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 -- 172.21.15.33:0/2118498685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff01a55d0 con 0x7f5ff01076e0 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5feeffd640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ff0101cd0 0x7f5ff019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.451+0000 7f5fe67fc640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff01076e0 0x7f5ff019a550 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800c920 tx=0x7f5fd800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.452+0000 7f5fecff9640 1 -- 172.21.15.33:0/2118498685 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8007c70 con 0x7f5ff01076e0 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.452+0000 7f5fecff9640 1 -- 172.21.15.33:0/2118498685 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fd8007e10 con 0x7f5ff01076e0 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.452+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff01a57a0 con 0x7f5ff01076e0 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.452+0000 7f5fecff9640 1 -- 172.21.15.33:0/2118498685 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd8018820 con 0x7f5ff01076e0 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.452+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff01a5cd0 con 0x7f5ff01076e0 2024-09-10T21:29:49.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.453+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff01020b0 con 0x7f5ff01076e0 2024-09-10T21:29:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.454+0000 7f5fecff9640 1 -- 172.21.15.33:0/2118498685 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) ==== 153250+0+0 (secure 0 0 0) 0x7f5fd800f450 con 0x7f5ff01076e0 2024-09-10T21:29:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.455+0000 7f5fecff9640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fdc0b5360 0x7f5fdc0b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.455+0000 7f5feeffd640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fdc0b5360 0x7f5fdc0b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.455+0000 7f5fecff9640 1 -- 172.21.15.33:0/2118498685 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8116+0+0 (secure 0 0 0) 0x7f5fd80e5c00 con 0x7f5ff01076e0 2024-09-10T21:29:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.456+0000 7f5feeffd640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fdc0b5360 0x7f5fdc0b7820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0031440 tx=0x7f5fe00095f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:49.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.459+0000 7f5fecff9640 1 -- 172.21.15.33:0/2118498685 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197130 (secure 0 0 0) 0x7f5fd8014030 con 0x7f5ff01076e0 2024-09-10T21:29:49.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.646+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5ff01098b0 con 0x7f5ff01076e0 2024-09-10T21:29:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.798+0000 7f5fecff9640 1 -- 172.21.15.33:0/2118498685 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) ==== 169+0+0 (secure 0 0 0) 0x7f5fd80acae0 con 0x7f5ff01076e0 2024-09-10T21:29:49.800 INFO:teuthology.orchestra.run.smithi033.stderr:pool 'cephfs_data' removed 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fdc0b5360 msgr2=0x7f5fdc0b7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fdc0b5360 0x7f5fdc0b7820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0031440 tx=0x7f5fe00095f0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff01076e0 msgr2=0x7f5ff019a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff01076e0 0x7f5ff019a550 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f5fd800c920 tx=0x7f5fd800cde0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 shutdown_connections 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:6800/3102318098,v1:172.21.15.33:6801/3102318098] conn(0x7f5fdc0b5360 0x7f5fdc0b7820 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ff0108060 0x7f5ff01a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ff01076e0 0x7f5ff019a550 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 --2- 172.21.15.33:0/2118498685 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5ff0101cd0 0x7f5ff019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.801+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 >> 172.21.15.33:0/2118498685 conn(0x7f5ff00fd720 msgr2=0x7f5ff00ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:49.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.802+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 shutdown_connections 2024-09-10T21:29:49.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-09-10T21:29:49.802+0000 7f5ff549f640 1 -- 172.21.15.33:0/2118498685 wait complete. 2024-09-10T21:29:49.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:49 smithi033 ceph-mon[30860]: pgmap v1123: 65 pgs: 65 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:29:49.925 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:49 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2118498685' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T21:29:50.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:49 smithi104 ceph-mon[35325]: pgmap v1123: 65 pgs: 65 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:29:50.061 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:49 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2118498685' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T21:29:50.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:49 smithi174 ceph-mon[35635]: pgmap v1123: 65 pgs: 65 active+clean; 577 KiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-10T21:29:50.142 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:49 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2118498685' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-10T21:29:50.532 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:29:50.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:29:50.610 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:29:50.645 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T21:29:50.645 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T21:29:50.661 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T21:29:50.676 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T21:29:50.700 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T21:29:50.700 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T21:29:50.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a 2024-09-10T21:29:50.804 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:50 smithi033 systemd[1]: Stopping Ceph mon.a for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:29:50.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:50 smithi174 ceph-mon[35635]: from='client.? 172.21.15.33:0/2118498685' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T21:29:50.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:50 smithi174 ceph-mon[35635]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T21:29:51.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:50 smithi104 ceph-mon[35325]: from='client.? 172.21.15.33:0/2118498685' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T21:29:51.060 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:50 smithi104 ceph-mon[35325]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T21:29:51.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:50 smithi033 ceph-mon[30860]: from='client.? 172.21.15.33:0/2118498685' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T21:29:51.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:50 smithi033 ceph-mon[30860]: osdmap e80: 12 total, 12 up, 12 in 2024-09-10T21:29:51.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a[30856]: 2024-09-10T21:29:51.048+0000 7f388b65e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:29:51.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a[30856]: 2024-09-10T21:29:51.048+0000 7f388b65e640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T21:29:51.518 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:51 smithi033 podman[132599]: 2024-09-10 21:29:51.212198107 +0000 UTC m=+0.309420077 container died 9658efd159e1a861bb6475bb112ffea3de2ab5612ab4332c5bca9173064496d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T21:29:52.175 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:51 smithi033 podman[132599]: 2024-09-10 21:29:51.916470889 +0000 UTC m=+1.013692858 container cleanup 9658efd159e1a861bb6475bb112ffea3de2ab5612ab4332c5bca9173064496d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T21:29:52.312 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:51 smithi033 bash[132599]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a 2024-09-10T21:29:52.427 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T21:29:52.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:52 smithi033 podman[132611]: 2024-09-10 21:29:52.18534461 +0000 UTC m=+0.975246327 container remove 9658efd159e1a861bb6475bb112ffea3de2ab5612ab4332c5bca9173064496d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T21:29:52.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:52 smithi033 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T21:29:52.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:52 smithi033 systemd[1]: Stopped Ceph mon.a for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:29:52.451 INFO:journalctl@ceph.mon.a.smithi033.stdout:Sep 10 21:29:52 smithi033 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.a.service: Consumed 28.930s CPU time. 2024-09-10T21:29:52.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:29:52.487 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T21:29:52.487 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-10T21:29:52.487 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.b 2024-09-10T21:29:52.560 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:52 smithi104 systemd[1]: Stopping Ceph mon.b for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:29:53.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-b[35321]: 2024-09-10T21:29:52.845+0000 7fe5ab782640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:29:53.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-b[35321]: 2024-09-10T21:29:52.845+0000 7fe5ab782640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T21:29:53.311 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:53 smithi104 podman[74593]: 2024-09-10 21:29:53.050136097 +0000 UTC m=+0.326474740 container died a799043be3baa5b8a17bf9b2ad4d6faa3de92c514c00580917f65fc820fba47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-10T21:29:53.733 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:53 smithi104 podman[74593]: 2024-09-10 21:29:53.480260599 +0000 UTC m=+0.756599317 container cleanup a799043be3baa5b8a17bf9b2ad4d6faa3de92c514c00580917f65fc820fba47f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T21:29:53.734 INFO:journalctl@ceph.mon.b.smithi104.stdout:Sep 10 21:29:53 smithi104 bash[74593]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-b 2024-09-10T21:29:53.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T21:29:53.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:29:53.912 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-10T21:29:53.912 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-10T21:29:53.912 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.c 2024-09-10T21:29:54.287 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:53 smithi174 systemd[1]: Stopping Ceph mon.c for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:29:54.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c[35631]: 2024-09-10T21:29:54.284+0000 7ffa7fd26640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:29:54.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c[35631]: 2024-09-10T21:29:54.284+0000 7ffa7fd26640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T21:29:54.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:54 smithi174 podman[74351]: 2024-09-10 21:29:54.407308871 +0000 UTC m=+0.248767574 container died 9dea2b2e5d3467a1ab497c246ebfdcf9adc88afe813bb26a5f598588b010337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T21:29:55.138 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:54 smithi174 podman[74351]: 2024-09-10 21:29:54.826668724 +0000 UTC m=+0.668127424 container cleanup 9dea2b2e5d3467a1ab497c246ebfdcf9adc88afe813bb26a5f598588b010337e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:29:55.138 INFO:journalctl@ceph.mon.c.smithi174.stdout:Sep 10 21:29:54 smithi174 bash[74351]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mon-c 2024-09-10T21:29:55.263 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T21:29:55.312 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:29:55.312 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-10T21:29:55.312 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T21:29:55.312 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.x 2024-09-10T21:29:55.640 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:29:55 smithi033 systemd[1]: Stopping Ceph mgr.x for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:29:55.925 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:29:55 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x[31089]: 2024-09-10T21:29:55.637+0000 7f0e6dff3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:29:55.925 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:29:55 smithi033 podman[132720]: 2024-09-10 21:29:55.669205966 +0000 UTC m=+0.165558573 container died 72b7c416be32aed30b5a1ef8e4f7874183cfb96fa503e87f07e9930a9563a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T21:29:56.425 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:29:56 smithi033 podman[132720]: 2024-09-10 21:29:56.189724161 +0000 UTC m=+0.686076791 container cleanup 72b7c416be32aed30b5a1ef8e4f7874183cfb96fa503e87f07e9930a9563a924 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f219c476cb4f26fc5719c661c0323719a45cb30, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-09-10T21:29:56.425 INFO:journalctl@ceph.mgr.x.smithi033.stdout:Sep 10 21:29:56 smithi033 bash[132720]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-x 2024-09-10T21:29:56.581 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T21:29:56.617 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:29:56.617 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T21:29:56.617 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T21:29:56.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.y 2024-09-10T21:29:56.966 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 21:29:56 smithi104 systemd[1]: Stopping Ceph mgr.y for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:29:57.271 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 21:29:56 smithi104 podman[74718]: 2024-09-10 21:29:56.96444578 +0000 UTC m=+0.148871232 container died 8307e8746dc883a4069f4965e081475bd670124128b25b54c58bdec20851d554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T21:29:57.560 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 21:29:57 smithi104 podman[74718]: 2024-09-10 21:29:57.385867556 +0000 UTC m=+0.570293020 container cleanup 8307e8746dc883a4069f4965e081475bd670124128b25b54c58bdec20851d554 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS) 2024-09-10T21:29:57.561 INFO:journalctl@ceph.mgr.y.smithi104.stdout:Sep 10 21:29:57 smithi104 bash[74718]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-y 2024-09-10T21:29:57.857 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T21:29:57.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:29:57.892 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T21:29:57.893 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-10T21:29:57.893 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.z 2024-09-10T21:29:58.229 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 21:29:57 smithi174 systemd[1]: Stopping Ceph mgr.z for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:29:58.542 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 21:29:58 smithi174 podman[74472]: 2024-09-10 21:29:58.228029051 +0000 UTC m=+0.137582871 container died 2d5a591f66032b20dda64bf5dba9384128f84f74abab8d43395f818469a0b1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:29:58.892 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 21:29:58 smithi174 podman[74472]: 2024-09-10 21:29:58.640652303 +0000 UTC m=+0.550206111 container cleanup 2d5a591f66032b20dda64bf5dba9384128f84f74abab8d43395f818469a0b1eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T21:29:58.892 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Sep 10 21:29:58 smithi174 bash[74472]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mgr-z 2024-09-10T21:29:59.028 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-10T21:29:59.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:29:59.062 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-10T21:29:59.062 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T21:29:59.062 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.0 2024-09-10T21:29:59.380 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:29:59 smithi033 systemd[1]: Stopping Ceph osd.0 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:29:59.675 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:29:59 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0[47340]: 2024-09-10T21:29:59.378+0000 7f87e26c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:29:59.675 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:29:59 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0[47340]: 2024-09-10T21:29:59.378+0000 7f87e26c6640 -1 osd.0 80 *** Got signal Terminated *** 2024-09-10T21:29:59.675 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:29:59 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0[47340]: 2024-09-10T21:29:59.378+0000 7f87e26c6640 -1 osd.0 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:05.425 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:05 smithi033 podman[132840]: 2024-09-10 21:30:05.071992925 +0000 UTC m=+5.814469572 container died b45e15d32b9b9b87a6843eb6449d76a0ee8c62506cc3772b8d5693710f219af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T21:30:05.772 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:05 smithi033 podman[132840]: 2024-09-10 21:30:05.603625879 +0000 UTC m=+6.346102524 container cleanup b45e15d32b9b9b87a6843eb6449d76a0ee8c62506cc3772b8d5693710f219af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T21:30:05.773 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:05 smithi033 bash[132840]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0 2024-09-10T21:30:06.175 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:05 smithi033 podman[132856]: 2024-09-10 21:30:05.900002857 +0000 UTC m=+0.821719625 container remove b45e15d32b9b9b87a6843eb6449d76a0ee8c62506cc3772b8d5693710f219af3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T21:30:06.586 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:06 smithi033 podman[132946]: 2024-09-10 21:30:06.214186719 +0000 UTC m=+0.016470464 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:30:06.586 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:06 smithi033 podman[132946]: 2024-09-10 21:30:06.314844205 +0000 UTC m=+0.117127949 container create cb05a4d38efa8fb1c4bf0f77723c94be3c3f40fc7b3f896bfe41c6e673137d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T21:30:06.586 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:06 smithi033 podman[132946]: 2024-09-10 21:30:06.584691192 +0000 UTC m=+0.386974950 container init cb05a4d38efa8fb1c4bf0f77723c94be3c3f40fc7b3f896bfe41c6e673137d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T21:30:06.925 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:06 smithi033 podman[132946]: 2024-09-10 21:30:06.588517962 +0000 UTC m=+0.390801713 container start cb05a4d38efa8fb1c4bf0f77723c94be3c3f40fc7b3f896bfe41c6e673137d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T21:30:06.983 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:06 smithi033 podman[132946]: 2024-09-10 21:30:06.659401817 +0000 UTC m=+0.461685570 container attach cb05a4d38efa8fb1c4bf0f77723c94be3c3f40fc7b3f896bfe41c6e673137d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T21:30:06.983 INFO:journalctl@ceph.osd.0.smithi033.stdout:Sep 10 21:30:06 smithi033 podman[132946]: 2024-09-10 21:30:06.832463961 +0000 UTC m=+0.634747713 container died cb05a4d38efa8fb1c4bf0f77723c94be3c3f40fc7b3f896bfe41c6e673137d25 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T21:30:07.496 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T21:30:07.539 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:30:07.539 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T21:30:07.539 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T21:30:07.540 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.1 2024-09-10T21:30:07.875 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:07 smithi104 systemd[1]: Stopping Ceph osd.1 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:30:08.310 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1[39777]: 2024-09-10T21:30:07.872+0000 7f506b052640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:30:08.310 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1[39777]: 2024-09-10T21:30:07.872+0000 7f506b052640 -1 osd.1 80 *** Got signal Terminated *** 2024-09-10T21:30:08.311 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1[39777]: 2024-09-10T21:30:07.872+0000 7f506b052640 -1 osd.1 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:13.810 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:13 smithi104 podman[74845]: 2024-09-10 21:30:13.518710875 +0000 UTC m=+5.781704219 container died b9e81727d44961a7d645d9138cf38713f525861f19053065d6285cdd746b4cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T21:30:14.310 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:14 smithi104 podman[74845]: 2024-09-10 21:30:14.157296265 +0000 UTC m=+6.420289608 container cleanup b9e81727d44961a7d645d9138cf38713f525861f19053065d6285cdd746b4cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T21:30:14.311 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:14 smithi104 bash[74845]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1 2024-09-10T21:30:14.810 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:14 smithi104 podman[74863]: 2024-09-10 21:30:14.392229921 +0000 UTC m=+0.873122636 container remove b9e81727d44961a7d645d9138cf38713f525861f19053065d6285cdd746b4cdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T21:30:15.164 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:14 smithi104 podman[74964]: 2024-09-10 21:30:14.770824174 +0000 UTC m=+0.019512344 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:30:15.164 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:14 smithi104 podman[74964]: 2024-09-10 21:30:14.893119149 +0000 UTC m=+0.141807307 container create 04f61ff6eac03f41636798428bd4c3f3901e048c5c8e6b8416205c461ced4f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T21:30:15.165 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:15 smithi104 podman[74964]: 2024-09-10 21:30:15.162786269 +0000 UTC m=+0.411474426 container init 04f61ff6eac03f41636798428bd4c3f3901e048c5c8e6b8416205c461ced4f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-10T21:30:15.536 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:15 smithi104 podman[74964]: 2024-09-10 21:30:15.167449685 +0000 UTC m=+0.416137845 container start 04f61ff6eac03f41636798428bd4c3f3901e048c5c8e6b8416205c461ced4f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T21:30:15.536 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:15 smithi104 podman[74964]: 2024-09-10 21:30:15.296036143 +0000 UTC m=+0.544724315 container attach 04f61ff6eac03f41636798428bd4c3f3901e048c5c8e6b8416205c461ced4f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T21:30:15.810 INFO:journalctl@ceph.osd.1.smithi104.stdout:Sep 10 21:30:15 smithi104 podman[74964]: 2024-09-10 21:30:15.535119373 +0000 UTC m=+0.783807538 container died 04f61ff6eac03f41636798428bd4c3f3901e048c5c8e6b8416205c461ced4f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T21:30:16.172 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T21:30:16.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:30:16.213 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T21:30:16.213 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T21:30:16.213 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.2 2024-09-10T21:30:16.556 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:16 smithi174 systemd[1]: Stopping Ceph osd.2 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:30:16.892 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2[40634]: 2024-09-10T21:30:16.553+0000 7f48960fe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:30:16.892 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2[40634]: 2024-09-10T21:30:16.553+0000 7f48960fe640 -1 osd.2 80 *** Got signal Terminated *** 2024-09-10T21:30:16.892 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2[40634]: 2024-09-10T21:30:16.553+0000 7f48960fe640 -1 osd.2 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:22.287 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:21 smithi174 podman[74599]: 2024-09-10 21:30:21.953976586 +0000 UTC m=+5.528852376 container died c4340f3b9e8cf29ea9334781150a67b02bf09f8434d0dc65a5f1a5672abc4292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:30:22.642 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:22 smithi174 podman[74599]: 2024-09-10 21:30:22.386752024 +0000 UTC m=+5.961627813 container cleanup c4340f3b9e8cf29ea9334781150a67b02bf09f8434d0dc65a5f1a5672abc4292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T21:30:22.642 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:22 smithi174 bash[74599]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2 2024-09-10T21:30:23.145 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:22 smithi174 podman[74628]: 2024-09-10 21:30:22.671521379 +0000 UTC m=+0.696850481 container remove c4340f3b9e8cf29ea9334781150a67b02bf09f8434d0dc65a5f1a5672abc4292 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:30:23.435 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:23 smithi174 podman[74713]: 2024-09-10 21:30:23.054500037 +0000 UTC m=+0.015603988 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:30:23.435 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:23 smithi174 podman[74713]: 2024-09-10 21:30:23.197421421 +0000 UTC m=+0.158525371 container create 7672c0831b6422279b7a8f5260b667b723fef02cf42a1a36a8b961029616ae40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:30:23.827 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:23 smithi174 podman[74713]: 2024-09-10 21:30:23.433566606 +0000 UTC m=+0.394670569 container init 7672c0831b6422279b7a8f5260b667b723fef02cf42a1a36a8b961029616ae40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T21:30:23.827 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:23 smithi174 podman[74713]: 2024-09-10 21:30:23.437219912 +0000 UTC m=+0.398323863 container start 7672c0831b6422279b7a8f5260b667b723fef02cf42a1a36a8b961029616ae40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T21:30:23.827 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:23 smithi174 podman[74713]: 2024-09-10 21:30:23.550071257 +0000 UTC m=+0.511175222 container attach 7672c0831b6422279b7a8f5260b667b723fef02cf42a1a36a8b961029616ae40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:30:24.142 INFO:journalctl@ceph.osd.2.smithi174.stdout:Sep 10 21:30:23 smithi174 podman[74713]: 2024-09-10 21:30:23.825724199 +0000 UTC m=+0.786828150 container died 7672c0831b6422279b7a8f5260b667b723fef02cf42a1a36a8b961029616ae40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T21:30:24.547 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T21:30:24.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:30:24.589 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T21:30:24.589 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T21:30:24.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.3 2024-09-10T21:30:24.925 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:24 smithi033 systemd[1]: Stopping Ceph osd.3 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:30:25.425 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:24 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T21:30:24.924+0000 7f850e379640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:30:25.425 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:24 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T21:30:24.924+0000 7f850e379640 -1 osd.3 80 *** Got signal Terminated *** 2024-09-10T21:30:25.425 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:24 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T21:30:24.924+0000 7f850e379640 -1 osd.3 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:25.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:25 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:25.115+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:26.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:25.895+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:26.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:26 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:26.072+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:27.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:26.921+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:27.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:27 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:27.074+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:27.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:27.189+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:28.103 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:27 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T21:30:27.736+0000 7f850a98e640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.986165+0000 front 2024-09-10T21:30:03.986200+0000 (oldest deadline 2024-09-10T21:30:26.886366+0000) 2024-09-10T21:30:28.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:27.914+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:28.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:28.134+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:28.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:28 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:28.101+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:28.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:28 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:28.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:29.146 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:28 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T21:30:28.780+0000 7f850a98e640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.986165+0000 front 2024-09-10T21:30:03.986200+0000 (oldest deadline 2024-09-10T21:30:26.886366+0000) 2024-09-10T21:30:29.176 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:28.906+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:29.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:29 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:29.144+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:29.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:29 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:29.141+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:29.642 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:29.175+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:30.163 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:29 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3[53776]: 2024-09-10T21:30:29.752+0000 7f850a98e640 -1 osd.3 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.986165+0000 front 2024-09-10T21:30:03.986200+0000 (oldest deadline 2024-09-10T21:30:26.886366+0000) 2024-09-10T21:30:30.203 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:29.900+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:30.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:30 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:30.161+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:30.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:30 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:30.169+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:30.642 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:30.201+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:30.885 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:30 smithi033 podman[133060]: 2024-09-10 21:30:30.576291184 +0000 UTC m=+5.784851075 container died 4a332492049f75bfaccf5d6371f46e9c40f473fb328ca9ef31d62ae9a0007fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:30:30.885 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:30 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:30.657+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:31.175 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:31 smithi033 podman[133060]: 2024-09-10 21:30:31.074704 +0000 UTC m=+6.283263891 container cleanup 4a332492049f75bfaccf5d6371f46e9c40f473fb328ca9ef31d62ae9a0007fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T21:30:31.175 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:31 smithi033 bash[133060]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3 2024-09-10T21:30:31.180 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:30.913+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:31.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:31 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:31.191+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:31.633 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:31 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:31.204+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:31.633 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:31 smithi033 podman[133077]: 2024-09-10 21:30:31.401826296 +0000 UTC m=+0.824767046 container remove 4a332492049f75bfaccf5d6371f46e9c40f473fb328ca9ef31d62ae9a0007fc8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T21:30:31.641 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:31.179+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:31.919 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:31 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:31.631+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:31.919 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:31 smithi033 podman[133165]: 2024-09-10 21:30:31.661845133 +0000 UTC m=+0.016886118 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:30:31.919 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:31 smithi033 podman[133165]: 2024-09-10 21:30:31.785974434 +0000 UTC m=+0.141015404 container create bb33b03398f864de0f42074e251a9d327b14e293fa62fff5730ecb12d01c478c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T21:30:32.175 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:32 smithi033 podman[133165]: 2024-09-10 21:30:32.056033901 +0000 UTC m=+0.411074881 container init bb33b03398f864de0f42074e251a9d327b14e293fa62fff5730ecb12d01c478c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T21:30:32.175 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:32 smithi033 podman[133165]: 2024-09-10 21:30:32.059548234 +0000 UTC m=+0.414589214 container start bb33b03398f864de0f42074e251a9d327b14e293fa62fff5730ecb12d01c478c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T21:30:32.175 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:32 smithi033 podman[133165]: 2024-09-10 21:30:32.114141541 +0000 UTC m=+0.469182512 container attach bb33b03398f864de0f42074e251a9d327b14e293fa62fff5730ecb12d01c478c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:30:32.221 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:31.961+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:32.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:32 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:32.224+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:32.607 INFO:journalctl@ceph.osd.3.smithi033.stdout:Sep 10 21:30:32 smithi033 podman[133165]: 2024-09-10 21:30:32.252244667 +0000 UTC m=+0.607285638 container died bb33b03398f864de0f42074e251a9d327b14e293fa62fff5730ecb12d01c478c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T21:30:32.607 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:32 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:32.221+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:32.642 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:32.219+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:32.919 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:32 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:32.605+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:32.944 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T21:30:32.990 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:30:32.991 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T21:30:32.991 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T21:30:32.991 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.4 2024-09-10T21:30:33.310 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:33 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:33.237+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:33.311 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:33 smithi104 systemd[1]: Stopping Ceph osd.4 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:30:33.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:32.969+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:33.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:33.191+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:33.650 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:33 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:33.199+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:33.810 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:33 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4[46670]: 2024-09-10T21:30:33.317+0000 7f8a42424640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:30:33.810 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:33 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4[46670]: 2024-09-10T21:30:33.317+0000 7f8a42424640 -1 osd.4 80 *** Got signal Terminated *** 2024-09-10T21:30:33.810 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:33 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4[46670]: 2024-09-10T21:30:33.317+0000 7f8a42424640 -1 osd.4 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:33.925 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:33 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:33.648+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:34.219 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:33.923+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:34.478 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:34.217+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:34.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:34 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:34.274+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:34.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:34 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:34.230+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:34.891 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:34.477+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:35.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:34 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:34.711+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:35.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:34 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:34.696+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:35.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:35.213+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:34.968+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:34.968+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:35.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:35 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:35.282+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:35.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:35 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:35.224+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:35.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:35 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:35.224+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:35.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:35.524+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:36.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:35 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:35.694+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:36.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:35 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:35.744+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:36.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:35 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:35.744+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:36.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:36 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:36.209+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:36.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:35.966+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:36.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:35.966+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:36.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:36 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:36.303+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:36.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:36 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:36.204+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:36.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:36 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:36.204+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:36.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:36 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:36.504+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:37.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:36 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:36.680+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:37.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:36 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:36.757+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:37.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:36 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:36.757+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:37.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:36 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:36.996+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:37.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:36 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:36.996+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:37.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:37 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:37.193+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:37.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:37 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:37.193+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:37.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:37 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:37.263+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:37.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:37 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:37.263+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:37.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:37 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:37.242+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:37.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:37 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:37.242+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:37.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:37 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:37.488+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:38.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:37 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:37.639+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:38.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:37 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:37.788+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:38.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:37 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:37.788+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:38.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:37 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:37.969+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:38.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:37 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:37.969+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:38.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:38 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:38.168+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:38.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:38 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:38.168+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:38.560 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:38 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4[46670]: 2024-09-10T21:30:38.300+0000 7f8a3ea39640 -1 osd.4 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.779322+0000 front 2024-09-10T21:30:11.779348+0000 (oldest deadline 2024-09-10T21:30:38.180177+0000) 2024-09-10T21:30:38.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:38 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:38.300+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:38.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:38 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:38.300+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:38.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:38 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:38.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:38.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:38 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:38.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:38.893 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:38 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:38.446+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:38.937 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:38 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:38.650+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:39.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:38 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:38.755+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:39.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:38 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:38.755+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:39.258 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:38 smithi104 podman[75070]: 2024-09-10 21:30:38.93577314 +0000 UTC m=+5.744864004 container died e2045cce0b834c931464cb7b3b03c9574aa32800feb87264f2672d1a36a31201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T21:30:39.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:38 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:38.924+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:39.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:38 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:38.924+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:39.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:39 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:39.153+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:39.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:39 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:39.153+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:39.560 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:39 smithi104 podman[75070]: 2024-09-10 21:30:39.351438404 +0000 UTC m=+6.160529275 container cleanup e2045cce0b834c931464cb7b3b03c9574aa32800feb87264f2672d1a36a31201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T21:30:39.561 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:39 smithi104 bash[75070]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4 2024-09-10T21:30:39.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:39 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:39.339+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:39.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:39 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:39.339+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:39.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:39 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:39.282+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:39.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:39 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:39.282+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:39.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:39 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:39.494+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:39.913 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:39 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:39.664+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:39.913 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:39 smithi104 podman[75085]: 2024-09-10 21:30:39.603082739 +0000 UTC m=+0.661620188 container remove e2045cce0b834c931464cb7b3b03c9574aa32800feb87264f2672d1a36a31201 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T21:30:40.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:39 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:39.771+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:40.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:39 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:39.771+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:40.224 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:39 smithi104 podman[75171]: 2024-09-10 21:30:39.813864922 +0000 UTC m=+0.015680088 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:30:40.225 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:39 smithi104 podman[75171]: 2024-09-10 21:30:39.945653836 +0000 UTC m=+0.147468991 container create 5d45e878a08a96e01098d554182cd74bcf8df50357ba00cfe30129a639b09709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T21:30:40.225 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:40 smithi104 podman[75171]: 2024-09-10 21:30:40.223109034 +0000 UTC m=+0.424924188 container init 5d45e878a08a96e01098d554182cd74bcf8df50357ba00cfe30129a639b09709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T21:30:40.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:39 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:39.935+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:40.661 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:39 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:39.935+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:40.662 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:40 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:40.110+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:40.662 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:40 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:40.110+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:40.663 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:40 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:40.309+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:40.663 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:40 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:40.309+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:40.663 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:40 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:40.273+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:40.663 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:40 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:40.273+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:40.663 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:40 smithi104 podman[75171]: 2024-09-10 21:30:40.226523541 +0000 UTC m=+0.428338706 container start 5d45e878a08a96e01098d554182cd74bcf8df50357ba00cfe30129a639b09709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T21:30:40.663 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:40 smithi104 podman[75171]: 2024-09-10 21:30:40.323187595 +0000 UTC m=+0.525002767 container attach 5d45e878a08a96e01098d554182cd74bcf8df50357ba00cfe30129a639b09709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T21:30:40.664 INFO:journalctl@ceph.osd.4.smithi104.stdout:Sep 10 21:30:40 smithi104 podman[75171]: 2024-09-10 21:30:40.414422904 +0000 UTC m=+0.616238070 container died 5d45e878a08a96e01098d554182cd74bcf8df50357ba00cfe30129a639b09709 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T21:30:40.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:40 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:40.456+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:41.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:40 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:40.671+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:41.090 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T21:30:41.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:30:41.136 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T21:30:41.137 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T21:30:41.137 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.5 2024-09-10T21:30:41.164 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:41.141+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:41.164 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:41.141+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:41.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:40 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:40.892+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:41.165 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:40 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:40.892+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:41.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:40 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:40.753+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:41.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:40 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:40.753+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:41.467 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:41 smithi174 systemd[1]: Stopping Ceph osd.5 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:30:41.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:41 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:41.262+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:41.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:41 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:41.262+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:41.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:41 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:41.241+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:41.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:41 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:41.241+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:41.850 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:41.466+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:41.851 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:41.464+0000 7f2b73a8f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:30:41.851 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:41.464+0000 7f2b73a8f640 -1 osd.5 80 *** Got signal Terminated *** 2024-09-10T21:30:41.851 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:41.464+0000 7f2b73a8f640 -1 osd.5 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:42.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:41 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:41.694+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:42.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:41.848+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:42.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:41 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:41.848+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:42.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:41 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:41.797+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:42.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:41 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:41.797+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:42.493 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:42 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:42.154+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:42.493 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:42 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:42.154+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:42.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:42 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:42.267+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:42.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:42 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:42.267+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:42.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:42.234+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:42.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:42.234+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:42.807 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:42 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:42.491+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:43.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:42 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:42.737+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:43.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:42 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:42.804+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:43.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:42 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:42.804+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:43.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:42.785+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:43.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:42 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:42.785+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:43.335 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:43.298+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:43.335 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:43.298+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:43.335 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:43.298+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:43.532 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:43.174+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:43.532 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:43.174+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:43.533 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:43.174+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.273153+0000 front 2024-09-10T21:30:20.273124+0000 (oldest deadline 2024-09-10T21:30:42.573432+0000) 2024-09-10T21:30:43.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:43.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:43.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:43.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:43.811 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:43.530+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:44.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:43.705+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:44.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:43 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:43.705+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:44.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:43.809+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:44.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:43 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:43.809+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:44.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:43.801+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:44.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:43 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:43.801+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:44.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:44 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:44.336+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:44.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:44 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:44.336+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:44.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:44 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:44.336+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:44.567 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:44 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:44.216+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:44.567 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:44 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:44.216+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:44.567 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:44 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:44.216+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.273153+0000 front 2024-09-10T21:30:20.273124+0000 (oldest deadline 2024-09-10T21:30:42.573432+0000) 2024-09-10T21:30:44.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:44 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:44.296+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:44.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:44 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:44.296+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:44.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:44 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:44.565+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:44.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:44 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:44.780+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:44.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:44 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:44.780+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:45.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:44 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:44.672+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:45.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:44 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:44.672+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:45.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:44 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:44.793+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:45.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:44 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:44.793+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:45.584 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:45 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:45.231+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:45.584 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:45 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:45.231+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:45.584 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:45 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:45.231+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.273153+0000 front 2024-09-10T21:30:20.273124+0000 (oldest deadline 2024-09-10T21:30:42.573432+0000) 2024-09-10T21:30:45.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:45 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:45.308+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:45.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:45 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:45.308+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:45.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:45 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:45.308+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:45.716 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:45 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:45.360+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:45.716 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:45 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:45.360+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:45.716 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:45 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:45.360+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:45.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:45 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:45.581+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:45.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:45 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:45.779+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:45.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:45 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:45.779+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:46.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:45 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:45.713+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:46.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:45 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:45.713+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:46.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:45 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:45.807+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:46.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:45 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:45.807+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:46.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:45 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:45.807+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:46.588 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:46 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:46.242+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:02.170431+0000 front 2024-09-10T21:30:02.170269+0000 (oldest deadline 2024-09-10T21:30:27.470405+0000) 2024-09-10T21:30:46.589 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:46 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:46.242+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.571564+0000 front 2024-09-10T21:30:11.571597+0000 (oldest deadline 2024-09-10T21:30:36.871575+0000) 2024-09-10T21:30:46.589 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:46 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5[47430]: 2024-09-10T21:30:46.242+0000 7f2b700a4640 -1 osd.5 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.273153+0000 front 2024-09-10T21:30:20.273124+0000 (oldest deadline 2024-09-10T21:30:42.573432+0000) 2024-09-10T21:30:46.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:46.288+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:46.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:46.288+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:46.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:46.288+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:46.736 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:46 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:46.396+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:46.737 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:46 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:46.396+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:46.737 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:46 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:46.396+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:46.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:46 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:46.585+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:46.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:46 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:46.745+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:46.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:46 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:46.745+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:46.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:46 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:46.745+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:47.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:46 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:46.734+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:47.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:46 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:46.734+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:47.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:46.848+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:47.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:46.848+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:47.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:46.848+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:47.392 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:47 smithi174 podman[74817]: 2024-09-10 21:30:47.099843217 +0000 UTC m=+5.762622213 container died 85ad21d70817889c869f1362a6bdd4af6225177667a2ef94c1ef861ef8044fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:30:47.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:47 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:47.305+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:47.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:47 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:47.305+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:47.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:47 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:47.305+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:47.724 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:47 smithi174 podman[74817]: 2024-09-10 21:30:47.523871283 +0000 UTC m=+6.186650279 container cleanup 85ad21d70817889c869f1362a6bdd4af6225177667a2ef94c1ef861ef8044fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T21:30:47.725 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:47 smithi174 bash[74817]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5 2024-09-10T21:30:47.725 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:47 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:47.588+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:47.725 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:47 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:47.588+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:47.759 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:47 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:47.354+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:47.759 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:47 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:47.354+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:47.759 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:47 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:47.354+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:48.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:47 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:47.756+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:48.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:47 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:47.756+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:48.142 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:47 smithi174 podman[74832]: 2024-09-10 21:30:47.775485897 +0000 UTC m=+0.673595429 container remove 85ad21d70817889c869f1362a6bdd4af6225177667a2ef94c1ef861ef8044fb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T21:30:48.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:47 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:47.722+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:48.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:47 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:47.722+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:48.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:47 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:47.722+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:48.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:47 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:47.805+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:48.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:47 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:47.805+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:48.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:47 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:47.805+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:48.471 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:48 smithi174 podman[74915]: 2024-09-10 21:30:48.081057947 +0000 UTC m=+0.016837234 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:30:48.472 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:48 smithi174 podman[74915]: 2024-09-10 21:30:48.209810295 +0000 UTC m=+0.145589563 container create 91bea5177dc44011bb6f03ab5f6400d069ab7598e7323141af7ea44d5cde36eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:30:48.472 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:48 smithi174 podman[74915]: 2024-09-10 21:30:48.470485553 +0000 UTC m=+0.406264820 container init 91bea5177dc44011bb6f03ab5f6400d069ab7598e7323141af7ea44d5cde36eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T21:30:48.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:48 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:48.323+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:48.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:48 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:48.323+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:48.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:48 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:48.323+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:48.720 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:48 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:48.310+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:48.721 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:48 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:48.310+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:48.721 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:48 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:48.310+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:48.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:48 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:48.630+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:48.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:48 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:48.630+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:48.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:48 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:48.686+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:48.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:48 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:48.686+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:48.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:48 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:48.686+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:48.893 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:48 smithi174 podman[74915]: 2024-09-10 21:30:48.473950145 +0000 UTC m=+0.409729414 container start 91bea5177dc44011bb6f03ab5f6400d069ab7598e7323141af7ea44d5cde36eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T21:30:48.893 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:48 smithi174 podman[74915]: 2024-09-10 21:30:48.553842187 +0000 UTC m=+0.489621471 container attach 91bea5177dc44011bb6f03ab5f6400d069ab7598e7323141af7ea44d5cde36eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:30:48.893 INFO:journalctl@ceph.osd.5.smithi174.stdout:Sep 10 21:30:48 smithi174 podman[74915]: 2024-09-10 21:30:48.665306142 +0000 UTC m=+0.601085412 container died 91bea5177dc44011bb6f03ab5f6400d069ab7598e7323141af7ea44d5cde36eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:30:49.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:48 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:48.718+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:49.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:48 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:48.718+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:49.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:48 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:48.787+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:49.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:48 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:48.787+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:49.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:48 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:48.787+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:49.325 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T21:30:49.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:30:49.404 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T21:30:49.404 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T21:30:49.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.6 2024-09-10T21:30:49.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:49 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:49.281+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:49.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:49 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:49.281+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:49.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:49 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:49.281+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:49.675 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:49 smithi033 systemd[1]: Stopping Ceph osd.6 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:30:49.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:49.288+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:49.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:49.288+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:49.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:49.288+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:50.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:49 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:49.750+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:50.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:49 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:49.750+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:50.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:49 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:49.648+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:50.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:49 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:49.648+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:50.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:49 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:49.648+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:50.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:49 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:49.664+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:50.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:49 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:49.664+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:50.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:49.720+0000 7fa017b25640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:30:50.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:49.720+0000 7fa017b25640 -1 osd.6 80 *** Got signal Terminated *** 2024-09-10T21:30:50.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:49.720+0000 7fa017b25640 -1 osd.6 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:50.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:49.790+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:50.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:49.790+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:50.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:49.790+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:50.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:50 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:50.281+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:50.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:50 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:50.281+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:50.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:50 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:50.281+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:50.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:50 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:50.238+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:50.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:50 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:50.238+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:50.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:50 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:50.238+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:51.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:50 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:50.790+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:51.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:50 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:50.790+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:51.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:50 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:50.703+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:51.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:50 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:50.703+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:51.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:50 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:50.688+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:51.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:50 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:50.688+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:51.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:50 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:50.688+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:51.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:50 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:50.784+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:51.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:50 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:50.784+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:51.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:50 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:50.784+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:51.551 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:51.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:51.552 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:51.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:51.552 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:51.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:51.552 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:51.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:51.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:51 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:51.285+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:51.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:51 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:51.285+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:51.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:51 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:51.285+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:51.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:51 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:51.285+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:52.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:51 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:51.767+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:52.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:51 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:51.767+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:52.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:51 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:51.767+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:52.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:51 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:51.677+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:52.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:51 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:51.677+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:52.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:51 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:51.677+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:52.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:51 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:51.731+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:52.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:51 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:51.731+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:52.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:51 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:51.731+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:52.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:51.819+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:52.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:51.819+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:52.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:51 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:51.819+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:52.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:52.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:52.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:52.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:52.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:52.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:52.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:52.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:52.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:52 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:52.258+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:52.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:52 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:52.258+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:52.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:52 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:52.258+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:52.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:52 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:52.258+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:53.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:52.742+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:53.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:52.742+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:53.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:52 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:52.742+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:53.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:52 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:52.693+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:53.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:52 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:52.693+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:53.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:52 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:52.693+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:53.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:52 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:52.680+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:53.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:52 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:52.680+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:53.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:52 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:52.680+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:53.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:52 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:52.779+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:53.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:52 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:52.779+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:53.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:52 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:52.779+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:53.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:53 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:53.300+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:53.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:53 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:53.300+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:53.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:53 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:53.300+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:53.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:53 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:53.300+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:53.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:53 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:53.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:53.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:53 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:53.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:53.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:53 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:53.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:53.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:53 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:53.256+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:54.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:53 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:53.693+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:54.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:53 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:53.693+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:54.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:53 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:53.693+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:54.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:53 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:53.648+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:54.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:53 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:53.648+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:54.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:53 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:53.648+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:54.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:53 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:53.677+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:54.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:53 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:53.677+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:54.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:53 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:53.677+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:54.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:53 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:53.751+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:01.039440+0000 front 2024-09-10T21:30:01.039405+0000 (oldest deadline 2024-09-10T21:30:30.440323+0000) 2024-09-10T21:30:54.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:53 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:53.751+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:10.440824+0000 front 2024-09-10T21:30:10.440869+0000 (oldest deadline 2024-09-10T21:30:35.141017+0000) 2024-09-10T21:30:54.175 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:53 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6[60382]: 2024-09-10T21:30:53.751+0000 7fa010132640 -1 osd.6 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.542908+0000 front 2024-09-10T21:30:21.542954+0000 (oldest deadline 2024-09-10T21:30:45.643142+0000) 2024-09-10T21:30:54.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:54 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:54.246+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:54.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:54 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:54.246+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:54.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:54 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:54.246+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:54.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:54 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:54.246+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:54.708 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:54.329+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:54.708 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:54.329+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:54.708 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:54.329+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:54.708 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:54.329+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:54.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:54.617+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:54.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:54.617+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:54.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:54.617+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:54.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:54.628+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:54.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:54.628+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:54.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:54 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:54.628+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:55.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:54.706+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:55.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:54.706+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:55.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:54.706+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:55.456 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:55 smithi033 podman[133278]: 2024-09-10 21:30:55.202819551 +0000 UTC m=+5.593792224 container died 4b6d04befc55cf2de2d56449f4d995cbab8a938eb2043893c4e043d202fa41f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T21:30:55.457 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:55 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:55.294+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:55.457 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:55 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:55.294+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:55.457 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:55 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:55.294+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:55.457 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:55 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:55.294+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:55.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:55 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:55.292+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:55.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:55 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:55.292+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:55.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:55 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:55.292+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:55.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:55 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:55.292+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:55.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:55 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:55.636+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:55.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:55 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:55.636+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:55.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:55 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:55.636+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:55.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:55 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:55.619+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:55.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:55 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:55.619+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:55.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:55 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:55.619+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:55.925 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:55 smithi033 podman[133278]: 2024-09-10 21:30:55.687259294 +0000 UTC m=+6.078231964 container cleanup 4b6d04befc55cf2de2d56449f4d995cbab8a938eb2043893c4e043d202fa41f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T21:30:55.925 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:55 smithi033 bash[133278]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6 2024-09-10T21:30:56.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:55 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:55.719+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:56.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:55 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:55.719+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:56.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:55 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:55.719+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:56.302 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:55 smithi033 podman[133294]: 2024-09-10 21:30:55.947096272 +0000 UTC m=+0.746087254 container remove 4b6d04befc55cf2de2d56449f4d995cbab8a938eb2043893c4e043d202fa41f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-09-10T21:30:56.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:56 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:56.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:56.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:56 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:56.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:56.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:56 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:56.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:56.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:56 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:56.276+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:56.602 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:56 smithi033 podman[133383]: 2024-09-10 21:30:56.203815032 +0000 UTC m=+0.016376470 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:30:56.602 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:56 smithi033 podman[133383]: 2024-09-10 21:30:56.331285704 +0000 UTC m=+0.143847133 container create dad994e1042dfee64ab0a2fb40b06115a4c2ba24a4c072b9977cb7d7d6c01367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T21:30:56.602 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:56 smithi033 podman[133383]: 2024-09-10 21:30:56.6007708 +0000 UTC m=+0.413332238 container init dad994e1042dfee64ab0a2fb40b06115a4c2ba24a4c072b9977cb7d7d6c01367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T21:30:56.603 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:56 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:56.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:56.603 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:56 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:56.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:56.603 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:56 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:56.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:56.603 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:56 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:56.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:56.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:56 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:56.608+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:56.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:56 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:56.608+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:56.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:56 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:56.608+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:56.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:56 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:56.640+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:56.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:56 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:56.640+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:56.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:56 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:56.640+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:56.925 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:56 smithi033 podman[133383]: 2024-09-10 21:30:56.604671615 +0000 UTC m=+0.417233044 container start dad994e1042dfee64ab0a2fb40b06115a4c2ba24a4c072b9977cb7d7d6c01367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T21:30:56.925 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:56 smithi033 podman[133383]: 2024-09-10 21:30:56.684057479 +0000 UTC m=+0.496618907 container attach dad994e1042dfee64ab0a2fb40b06115a4c2ba24a4c072b9977cb7d7d6c01367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:30:56.925 INFO:journalctl@ceph.osd.6.smithi033.stdout:Sep 10 21:30:56 smithi033 podman[133383]: 2024-09-10 21:30:56.816197526 +0000 UTC m=+0.628758954 container died dad994e1042dfee64ab0a2fb40b06115a4c2ba24a4c072b9977cb7d7d6c01367 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T21:30:57.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:56 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:56.689+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:57.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:56 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:56.689+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:57.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:56 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:56.689+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:57.422 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:57 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:57.320+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:57.422 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:57 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:57.320+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:57.422 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:57 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:57.320+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:57.423 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:57 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:57.320+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:57.450 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T21:30:57.494 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:30:57.494 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T21:30:57.495 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T21:30:57.495 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.7 2024-09-10T21:30:57.560 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:57 smithi104 systemd[1]: Stopping Ceph osd.7 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:30:57.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:57.303+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:57.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:57.303+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:57.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:57.303+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:57.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:57.303+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:57.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:57 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:57.599+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:57.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:57 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:57.599+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:57.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:57 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:57.599+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:57.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:57 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:57.613+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:57.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:57 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:57.613+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:57.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:57 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:57.613+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:58.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:57.677+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:58.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:57.677+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:58.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:57.677+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:58.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:57.812+0000 7f57cd6ab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:30:58.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:57.812+0000 7f57cd6ab640 -1 osd.7 80 *** Got signal Terminated *** 2024-09-10T21:30:58.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:57.812+0000 7f57cd6ab640 -1 osd.7 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:30:58.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:58 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:58.273+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:58.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:58 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:58.273+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:58.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:58 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:58.273+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:58.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:58 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:58.273+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:58.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:58 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:58.368+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:58.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:58 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:58.368+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:58.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:58 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:58.368+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:58.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:58 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:58.368+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:58.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:58 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:58.555+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:58.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:58 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:58.555+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:58.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:58 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:58.555+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:30:58.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:58 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:58.617+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:58.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:58 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:58.617+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:58.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:58 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:58.617+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:59.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:58 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:58.632+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:30:59.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:58 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:58.632+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:30:59.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:58 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:58.632+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:30:59.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:59 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:59.288+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:30:59.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:59 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:59.288+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:30:59.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:59 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:59.288+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:30:59.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:30:59 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:30:59.288+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:30:59.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:59 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:59.322+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:30:59.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:59 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:59.322+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:30:59.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:59 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:59.322+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:30:59.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:30:59 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:30:59.322+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:30:59.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:59 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:59.590+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:30:59.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:59 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:59.590+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:30:59.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:30:59 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:30:59.590+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:30:59.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:59 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:59.509+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:30:59.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:59 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:59.509+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:30:59.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:30:59 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:30:59.509+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:00.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:59 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:59.619+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:31:00.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:59 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:59.619+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:31:00.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:30:59 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:30:59.619+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:31:00.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:00 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:00.298+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:00.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:00 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:00.298+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:00.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:00 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:00.298+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:00.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:00 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:00.298+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:00.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:00 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:00.280+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:00.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:00 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:00.280+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:00.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:00 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:00.280+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:00.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:00 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:00.280+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:00.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:00 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:00.549+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:00.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:00 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:00.549+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:00.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:00 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:00.549+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:00.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:00 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:00.560+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:00.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:00 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:00.560+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:00.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:00 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:00.560+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:00.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:00 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:00.560+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:01.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:00 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:00.646+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:31:01.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:00 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:00.646+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:31:01.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:00 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:00.646+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:31:01.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:01.257+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:01.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:01.257+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:01.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:01.257+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:01.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:01.257+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:01.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:01 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:01.270+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:01.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:01 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:01.270+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:01.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:01 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:01.270+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:01.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:01 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:01.270+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:01.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:01 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:01.270+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:01.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:01.512+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:01.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:01.512+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:01.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:01.512+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:01.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:01.512+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:01.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:01.604+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:01.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:01.604+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:01.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:01.604+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:01.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:01 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:01.604+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:02.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:01.660+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:31:02.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:01.660+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:31:02.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:01.660+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:31:02.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:02.256+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:02.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:02.256+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:02.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:02.256+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:02.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:02.256+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:02.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:02.256+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:02.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:02 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:02.250+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:02.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:02 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:02.250+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:02.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:02 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:02.250+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:02.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:02 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:02.250+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:02.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:02 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:02.250+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:02.554+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:02.554+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:02.554+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:02.554+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:02.635+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:02.635+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:02.635+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:02.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:02 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:02.635+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:03.060 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:02.674+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:12.074180+0000 front 2024-09-10T21:30:12.074206+0000 (oldest deadline 2024-09-10T21:30:34.374328+0000) 2024-09-10T21:31:03.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:02.674+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:17.875573+0000 front 2024-09-10T21:30:17.875448+0000 (oldest deadline 2024-09-10T21:30:43.175523+0000) 2024-09-10T21:31:03.061 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:02 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7[53450]: 2024-09-10T21:31:02.674+0000 7f57c5cb8640 -1 osd.7 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.078052+0000 front 2024-09-10T21:30:27.078077+0000 (oldest deadline 2024-09-10T21:30:51.178157+0000) 2024-09-10T21:31:03.497 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:03 smithi104 podman[75285]: 2024-09-10 21:31:03.226647542 +0000 UTC m=+5.533887860 container died 4e85383485254f9ca36181bcf05065e6e9f7ebc14cfda14e90d3870e5067dd31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux ) 2024-09-10T21:31:03.497 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:03 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:03.214+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:03.497 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:03 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:03.214+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:03.497 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:03 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:03.214+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:03.497 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:03 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:03.214+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:03.497 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:03 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:03.214+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:03.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:03 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:03.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:03.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:03 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:03.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:03.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:03 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:03.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:03.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:03 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:03.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:03.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:03 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:03.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:03.810 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:03 smithi104 podman[75285]: 2024-09-10 21:31:03.603285642 +0000 UTC m=+5.910525962 container cleanup 4e85383485254f9ca36181bcf05065e6e9f7ebc14cfda14e90d3870e5067dd31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2) 2024-09-10T21:31:03.810 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:03 smithi104 bash[75285]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7 2024-09-10T21:31:03.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:03.551+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:03.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:03.551+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:03.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:03.551+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:03.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:03.551+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:03.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:03.601+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:03.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:03.601+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:03.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:03.601+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:03.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:03 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:03.601+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:04.206 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:03 smithi104 podman[75300]: 2024-09-10 21:31:03.862982931 +0000 UTC m=+0.636107638 container remove 4e85383485254f9ca36181bcf05065e6e9f7ebc14cfda14e90d3870e5067dd31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T21:31:04.476 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:04 smithi104 podman[75386]: 2024-09-10 21:31:04.205400607 +0000 UTC m=+0.110582022 container create f4bd7e0642bb2cb782f4b99f93b2fcaa5cb04d17b2050452c9004cb08ba25468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T21:31:04.476 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:04 smithi104 podman[75386]: 2024-09-10 21:31:04.110996374 +0000 UTC m=+0.016177814 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:31:04.476 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:04 smithi104 podman[75386]: 2024-09-10 21:31:04.474769963 +0000 UTC m=+0.379951395 container init f4bd7e0642bb2cb782f4b99f93b2fcaa5cb04d17b2050452c9004cb08ba25468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T21:31:04.477 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:04 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:04.206+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:04.477 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:04 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:04.206+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:04.477 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:04 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:04.206+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:04.477 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:04 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:04.206+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:04.477 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:04 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:04.206+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:04.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:04 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:04.254+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:04.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:04 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:04.254+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:04.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:04 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:04.254+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:04.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:04 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:04.254+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:04.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:04 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:04.254+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:04.811 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:04 smithi104 podman[75386]: 2024-09-10 21:31:04.478224682 +0000 UTC m=+0.383406097 container start f4bd7e0642bb2cb782f4b99f93b2fcaa5cb04d17b2050452c9004cb08ba25468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T21:31:04.811 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:04 smithi104 podman[75386]: 2024-09-10 21:31:04.566227182 +0000 UTC m=+0.471408613 container attach f4bd7e0642bb2cb782f4b99f93b2fcaa5cb04d17b2050452c9004cb08ba25468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T21:31:04.811 INFO:journalctl@ceph.osd.7.smithi104.stdout:Sep 10 21:31:04 smithi104 podman[75386]: 2024-09-10 21:31:04.667183413 +0000 UTC m=+0.572364830 container died f4bd7e0642bb2cb782f4b99f93b2fcaa5cb04d17b2050452c9004cb08ba25468 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T21:31:04.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:04.583+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:04.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:04.583+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:04.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:04.583+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:04.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:04.583+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:04.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:04.572+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:04.572+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:04.572+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:04 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:04.572+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:05.304 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:05 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:05.163+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:05.304 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:05 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:05.163+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:05.304 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:05 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:05.163+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:05.304 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:05 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:05.163+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:05.304 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:05 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:05.163+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:05.321 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T21:31:05.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:05.360 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T21:31:05.360 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-10T21:31:05.361 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.8 2024-09-10T21:31:05.642 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 systemd[1]: Stopping Ceph osd.8 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:05.642 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:05.556+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:05.644 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:05.556+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:05.644 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:05.556+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:05.644 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:05.556+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:05.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:05.539+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:05.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:05.539+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:05.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:05.539+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:05.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:05.539+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:05.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:05 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:05.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:05.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:05 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:05.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:05.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:05 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:05.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:05.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:05 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:05.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:05.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:05 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:05.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:06.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:05.662+0000 7f5b0dbae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:06.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:05.662+0000 7f5b0dbae640 -1 osd.8 80 *** Got signal Terminated *** 2024-09-10T21:31:06.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:05 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:05.662+0000 7f5b0dbae640 -1 osd.8 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:31:06.527 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:06 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:06.202+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:06.527 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:06 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:06.202+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:06.527 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:06 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:06.202+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:06.527 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:06 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:06.202+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:06.527 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:06 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:06.202+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:06.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:06 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:06.192+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:06.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:06 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:06.192+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:06.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:06 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:06.192+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:06.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:06 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:06.192+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:06.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:06 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:06.192+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:06.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:06.533+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:06.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:06.533+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:06.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:06.533+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:06.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:06.533+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:06.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:06.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:06.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:06.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:06.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:06.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:06.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:06 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:06.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:07.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:07.242+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:07.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:07.242+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:07.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:07.242+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:07.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:07.242+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:07.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:07 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:07.242+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:07.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:07 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:07.177+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:07.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:07 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:07.177+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:07.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:07 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:07.177+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:07.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:07 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:07.177+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:07.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:07 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:07.177+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:07.558+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:07.558+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:07.558+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:07.558+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:07.526+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:07.526+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:07.526+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:07.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:07 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:07.526+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:08.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:08 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:08.166+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:08.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:08 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:08.166+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:08.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:08 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:08.166+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:08.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:08 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:08.166+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:08.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:08 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:08.166+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:08.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:08 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:08.209+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:08.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:08 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:08.209+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:08.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:08 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:08.209+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:08.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:08 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:08.209+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:08.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:08 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:08.209+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:08.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:08.592+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:08.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:08.592+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:08.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:08.592+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:08.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:08.592+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:08.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:08.557+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:08.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:08.557+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:08.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:08.557+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:08.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:08.557+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:08.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:08 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:08.557+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:09.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:09 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:09.172+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:09.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:09 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:09.172+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:09.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:09 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:09.172+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:09.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:09 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:09.172+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:09.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:09 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:09.172+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:09.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:09 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:09.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:09.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:09 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:09.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:09.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:09 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:09.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:09.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:09 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:09.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:09.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:09 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:09.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:09.893 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:09.590+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:09.915 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:09.590+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:09.915 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:09.590+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:09.915 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:09.590+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:09.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:09.576+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:09.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:09.576+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:09.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:09.576+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:09.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:09.576+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:09.916 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:09.576+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:10.479 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:10 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:10.217+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:10.479 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:10 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:10.217+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:10.479 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:10 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:10.217+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:10.480 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:10 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:10.217+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:10.480 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:10 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:10.218+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:10.480 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:10 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:10.218+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:10.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:10 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:10.173+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:10.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:10 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:10.173+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:10.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:10 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:10.173+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:10.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:10 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:10.173+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:10.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:10 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:10.173+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:10.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:10.557+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.188789+0000 front 2024-09-10T21:30:09.188619+0000 (oldest deadline 2024-09-10T21:30:33.888652+0000) 2024-09-10T21:31:10.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:10.557+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:21.490217+0000 front 2024-09-10T21:30:21.490274+0000 (oldest deadline 2024-09-10T21:30:47.390438+0000) 2024-09-10T21:31:10.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:10.557+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:27.891742+0000 front 2024-09-10T21:30:27.891486+0000 (oldest deadline 2024-09-10T21:30:50.791645+0000) 2024-09-10T21:31:10.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:10.557+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.793375+0000 front 2024-09-10T21:30:37.793449+0000 (oldest deadline 2024-09-10T21:31:01.194004+0000) 2024-09-10T21:31:10.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8[54326]: 2024-09-10T21:31:10.557+0000 7f5b0a1c3640 -1 osd.8 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:45.895048+0000 front 2024-09-10T21:30:45.895073+0000 (oldest deadline 2024-09-10T21:31:09.995231+0000) 2024-09-10T21:31:10.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:10.542+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:10.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:10.542+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:10.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:10.542+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:10.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:10.542+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:10.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:10 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:10.542+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:11.562 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:11.165+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:11.562 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:11.165+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:11.562 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:11.165+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:11.562 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:11.165+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:11.562 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:11 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:11.165+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:11.620 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:11 smithi174 podman[75018]: 2024-09-10 21:31:11.31003077 +0000 UTC m=+5.764472539 container died 601c9c1d11995723df060f31080363267307955598e0cd3e20933ed6753974bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:31:11.620 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:11 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:11.502+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:11.620 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:11 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:11.502+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:11.620 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:11 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:11.502+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:11.620 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:11 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:11.502+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:11.620 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:11 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:11.502+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:11.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:11 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:11.195+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:11.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:11 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:11.195+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:11.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:11 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:11.195+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:11.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:11 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:11.195+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:11.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:11 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:11.195+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:11.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:11 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:11.195+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:11.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:11 smithi174 podman[75018]: 2024-09-10 21:31:11.751174344 +0000 UTC m=+6.205616112 container cleanup 601c9c1d11995723df060f31080363267307955598e0cd3e20933ed6753974bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731) 2024-09-10T21:31:11.892 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:11 smithi174 bash[75018]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8 2024-09-10T21:31:12.392 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:12 smithi174 podman[75033]: 2024-09-10 21:31:12.00250849 +0000 UTC m=+0.687260428 container remove 601c9c1d11995723df060f31080363267307955598e0cd3e20933ed6753974bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:31:12.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:12 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:12.169+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:12.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:12 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:12.169+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:12.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:12 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:12.169+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:12.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:12 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:12.169+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:12.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:12 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:12.169+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:12.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:12 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:12.169+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:12.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:12 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:12.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:12.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:12 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:12.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:12.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:12 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:12.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:12.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:12 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:12.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:12.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:12 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:12.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:12.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:12 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:12.227+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:12.732 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:12 smithi174 podman[75124]: 2024-09-10 21:31:12.339917427 +0000 UTC m=+0.016125439 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:31:12.732 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:12 smithi174 podman[75124]: 2024-09-10 21:31:12.462116703 +0000 UTC m=+0.138324702 container create 72a64bae96dbfce3a99c1e53d09cbe3675b6f6ffa62d31762ab384107c0046e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T21:31:12.732 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:12 smithi174 podman[75124]: 2024-09-10 21:31:12.731062415 +0000 UTC m=+0.407270415 container init 72a64bae96dbfce3a99c1e53d09cbe3675b6f6ffa62d31762ab384107c0046e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-10T21:31:12.733 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:12 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:12.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:12.733 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:12 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:12.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:12.733 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:12 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:12.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:12.733 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:12 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:12.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:12.733 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:12 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:12.495+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:13.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:12 smithi174 podman[75124]: 2024-09-10 21:31:12.734649978 +0000 UTC m=+0.410857979 container start 72a64bae96dbfce3a99c1e53d09cbe3675b6f6ffa62d31762ab384107c0046e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:31:13.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:12 smithi174 podman[75124]: 2024-09-10 21:31:12.81448768 +0000 UTC m=+0.490695681 container attach 72a64bae96dbfce3a99c1e53d09cbe3675b6f6ffa62d31762ab384107c0046e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2) 2024-09-10T21:31:13.142 INFO:journalctl@ceph.osd.8.smithi174.stdout:Sep 10 21:31:12 smithi174 podman[75124]: 2024-09-10 21:31:12.935875822 +0000 UTC m=+0.612083823 container died 72a64bae96dbfce3a99c1e53d09cbe3675b6f6ffa62d31762ab384107c0046e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T21:31:13.462 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:13 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:13.158+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:13.462 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:13 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:13.158+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:13.462 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:13 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:13.158+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:13.462 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:13 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:13.158+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:13.462 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:13 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:13.158+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:13.462 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:13 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:13.158+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:13.641 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:13 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:13.467+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:13.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:13 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:13.467+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:13.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:13 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:13.467+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:13.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:13 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:13.467+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:13.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:13 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:13.467+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:13.658 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.8.service' 2024-09-10T21:31:13.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:13 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:13.267+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:13.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:13 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:13.267+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:13.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:13 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:13.267+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:13.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:13 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:13.267+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:13.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:13 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:13.267+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:13.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:13 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:13.267+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:13.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:13.702 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-10T21:31:13.702 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-10T21:31:13.702 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.9 2024-09-10T21:31:14.033 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:13 smithi033 systemd[1]: Stopping Ceph osd.9 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:14.034 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.031+0000 7f78de69d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:14.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.031+0000 7f78de69d640 -1 osd.9 80 *** Got signal Terminated *** 2024-09-10T21:31:14.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.031+0000 7f78de69d640 -1 osd.9 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:31:14.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:14.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:14.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:14.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:14.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:14.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:14 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:14.279+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:14.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:14 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:14.160+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:14.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:14 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:14.160+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:14.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:14 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:14.160+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:14.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:14 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:14.160+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:14.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:14 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:14.160+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:14.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:14 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:14.160+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:14.859 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:14 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:14.470+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:14.859 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:14 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:14.470+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:14.859 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:14 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:14.470+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:14.859 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:14 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:14.470+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:14.859 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:14 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:14.470+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:15.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:15 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:15.199+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:15.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:15 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:15.199+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:15.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:15 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:15.199+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:15.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:15 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:15.199+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:15.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:15 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:15.199+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:15.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:15 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:15.199+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:15.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:15 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:15.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:15.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:15 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:15.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:15.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:15 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:15.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:15.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:15 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:15.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:15.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:15 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:15.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:15.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:15 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:15.309+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:15.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:15 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:15.469+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:15.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:15 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:15.469+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:15.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:15 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:15.469+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:15.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:15 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:15.469+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:15.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:15 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:15.469+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:16.311 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:16.156+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:16.311 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:16.156+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:16.311 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:16.156+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:16.311 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:16.156+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:16.311 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:16.156+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:16.311 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:16.156+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:16.311 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:16 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:16.156+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:16.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:16 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:16.261+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:16.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:16 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:16.261+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:16.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:16 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:16.261+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:16.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:16 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:16.261+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:16.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:16 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:16.261+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:16.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:16 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:16.261+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:16.676 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:16 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:16.261+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:50.083811+0000 front 2024-09-10T21:30:50.083840+0000 (oldest deadline 2024-09-10T21:31:15.383989+0000) 2024-09-10T21:31:16.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:16.439+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:16.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:16.439+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:16.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:16.439+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:16.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:16.439+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:16.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:16 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:16.439+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:17.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:17 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:17.190+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:17.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:17 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:17.190+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:17.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:17 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:17.190+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:17.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:17 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:17.190+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:17.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:17 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:17.190+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:17.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:17 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:17.190+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:17.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:17 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:17.190+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:17.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:17.276+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:17.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:17.276+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:17.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:17.276+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:17.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:17.276+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:17.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:17.276+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:17.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:17.276+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:17.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:17 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:17.276+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:50.083811+0000 front 2024-09-10T21:30:50.083840+0000 (oldest deadline 2024-09-10T21:31:15.383989+0000) 2024-09-10T21:31:17.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:17 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:17.414+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:17.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:17 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:17.414+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:17.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:17 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:17.414+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:17.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:17 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:17.414+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:17.892 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:17 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:17.414+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:18.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:18 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:18.148+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:18.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:18 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:18.148+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:18.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:18 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:18.148+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:18.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:18 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:18.148+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:18.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:18 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:18.148+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:18.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:18 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:18.148+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:18.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:18 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:18.148+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:18.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:18 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:18.376+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:18.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:18 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:18.376+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:18.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:18 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:18.376+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:18.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:18 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:18.376+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:18.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:18 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:18.376+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:18.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:18.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:29:59.776280+0000 front 2024-09-10T21:29:59.776244+0000 (oldest deadline 2024-09-10T21:30:25.076392+0000) 2024-09-10T21:31:18.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:18.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:09.777708+0000 front 2024-09-10T21:30:09.777921+0000 (oldest deadline 2024-09-10T21:30:35.077724+0000) 2024-09-10T21:31:18.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:18.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:19.079917+0000 front 2024-09-10T21:30:19.079952+0000 (oldest deadline 2024-09-10T21:30:44.979906+0000) 2024-09-10T21:31:18.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:18.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:24.980183+0000 front 2024-09-10T21:30:24.980255+0000 (oldest deadline 2024-09-10T21:30:50.880394+0000) 2024-09-10T21:31:18.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:18.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:35.382689+0000 front 2024-09-10T21:30:35.382746+0000 (oldest deadline 2024-09-10T21:31:00.682728+0000) 2024-09-10T21:31:18.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:18.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:40.683225+0000 front 2024-09-10T21:30:40.683289+0000 (oldest deadline 2024-09-10T21:31:10.083538+0000) 2024-09-10T21:31:18.675 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:18 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9[67368]: 2024-09-10T21:31:18.266+0000 7f78dacb2640 -1 osd.9 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:50.083811+0000 front 2024-09-10T21:30:50.083840+0000 (oldest deadline 2024-09-10T21:31:15.383989+0000) 2024-09-10T21:31:19.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:19 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:19.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:19.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:19 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:19.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:19.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:19 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:19.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:19.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:19 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:19.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:19.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:19 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:19.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:19.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:19 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:19.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:19.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:19 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:19.149+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:19.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:19 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:19.362+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:19.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:19 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:19.362+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:19.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:19 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:19.362+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:19.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:19 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:19.362+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:19.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:19 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:19.362+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:19.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:19 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:19.362+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:19.891 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:19 smithi033 podman[133510]: 2024-09-10 21:31:19.620668988 +0000 UTC m=+5.696942920 container died 9715be0d846f397eb1fbcdc78f85f72928c31b2cb168d458d6042f195f00c093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True) 2024-09-10T21:31:20.175 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:20 smithi033 podman[133510]: 2024-09-10 21:31:20.06511899 +0000 UTC m=+6.141392921 container cleanup 9715be0d846f397eb1fbcdc78f85f72928c31b2cb168d458d6042f195f00c093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T21:31:20.175 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:20 smithi033 bash[133510]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9 2024-09-10T21:31:20.546 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:20 smithi033 podman[133526]: 2024-09-10 21:31:20.283183397 +0000 UTC m=+0.655707270 container remove 9715be0d846f397eb1fbcdc78f85f72928c31b2cb168d458d6042f195f00c093 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T21:31:20.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:20 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:20.119+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:20.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:20 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:20.119+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:20.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:20 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:20.119+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:20.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:20 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:20.119+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:20.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:20 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:20.119+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:20.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:20 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:20.119+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:20.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:20 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:20.119+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:20.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:20 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:20.319+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:20.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:20 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:20.319+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:20.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:20 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:20.320+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:20.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:20 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:20.320+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:20.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:20 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:20.320+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:20.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:20 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:20.320+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:20.818 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:20 smithi033 podman[133615]: 2024-09-10 21:31:20.66147749 +0000 UTC m=+0.016168280 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:31:21.079 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:20 smithi033 podman[133615]: 2024-09-10 21:31:20.817526853 +0000 UTC m=+0.172217693 container create 8623d1cae8cb75323bf3d3de7eab12c1039867cdb507f571faf6363df6988580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T21:31:21.079 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:21 smithi033 podman[133615]: 2024-09-10 21:31:21.078289793 +0000 UTC m=+0.432980571 container init 8623d1cae8cb75323bf3d3de7eab12c1039867cdb507f571faf6363df6988580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T21:31:21.425 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:21 smithi033 podman[133615]: 2024-09-10 21:31:21.081787304 +0000 UTC m=+0.436478087 container start 8623d1cae8cb75323bf3d3de7eab12c1039867cdb507f571faf6363df6988580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:31:21.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:21 smithi033 podman[133615]: 2024-09-10 21:31:21.203294728 +0000 UTC m=+0.557985521 container attach 8623d1cae8cb75323bf3d3de7eab12c1039867cdb507f571faf6363df6988580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:31:21.426 INFO:journalctl@ceph.osd.9.smithi033.stdout:Sep 10 21:31:21 smithi033 podman[133615]: 2024-09-10 21:31:21.276854004 +0000 UTC m=+0.631544800 container died 8623d1cae8cb75323bf3d3de7eab12c1039867cdb507f571faf6363df6988580 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T21:31:21.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:21 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:21.147+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:21.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:21 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:21.147+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:21.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:21 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:21.147+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:21.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:21 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:21.147+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:21.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:21 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:21.147+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:21.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:21 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:21.147+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:21.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:21 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:21.147+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:21.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:21.308+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:21.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:21.308+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:21.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:21.308+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:21.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:21.308+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:21.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:21.308+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:21.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:21.308+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:21.911 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.9.service' 2024-09-10T21:31:21.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:21.953 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-10T21:31:21.954 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-10T21:31:21.954 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.10 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 systemd[1]: Stopping Ceph osd.10 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:22.274 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.193+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:22.275 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.271+0000 7fde9e778640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:22.275 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.271+0000 7fde9e778640 -1 osd.10 80 *** Got signal Terminated *** 2024-09-10T21:31:22.275 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:22 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:22.271+0000 7fde9e778640 -1 osd.10 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:31:22.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:22.293+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:22.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:22.293+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:22.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:22.293+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:22.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:22.293+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:22.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:22.293+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:22.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:22 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:22.293+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:23.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:23 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:23.145+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:23.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:23 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:23.145+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:23.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:23 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:23.145+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:23.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:23 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:23.145+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:23.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:23 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:23.145+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:23.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:23 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:23.145+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:23.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:23 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:23.145+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:23.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:23 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:23.262+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:23.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:23 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:23.262+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:23.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:23 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:23.262+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:23.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:23 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:23.262+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:23.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:23 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:23.262+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:23.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:23 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:23.262+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:24.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:24 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:24.101+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:24.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:24 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:24.101+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:24.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:24 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:24.101+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:24.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:24 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:24.101+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:24.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:24 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:24.101+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:24.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:24 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:24.101+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:24.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:24 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:24.101+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:24.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:24 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:24.236+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:24.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:24 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:24.236+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:24.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:24 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:24.236+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:24.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:24 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:24.236+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:24.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:24 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:24.236+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:24.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:24 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:24.236+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:24.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:24 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:24.236+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:25.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:25 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:25.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:25.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:25 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:25.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:25.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:25 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:25.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:25.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:25 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:25.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:25.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:25 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:25.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:25.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:25 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:25.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:25.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:25 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:25.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:25.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:25.187+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:25.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:25.187+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:25.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:25.187+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:25.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:25.187+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:25.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:25.187+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:25.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:25.187+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:25.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:25 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:25.187+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:26.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:26 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:26.132+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:26.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:26 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:26.132+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:26.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:26 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:26.132+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:26.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:26 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:26.132+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:26.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:26 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:26.132+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:26.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:26 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:26.132+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:26.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:26 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:26.132+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:26.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:26.152+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:26.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:26.152+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:26.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:26.152+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:26.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:26.152+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:26.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:26.152+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:26.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:26.152+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:26.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:26 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:26.152+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:27.560 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:27.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:03.741915+0000 front 2024-09-10T21:30:03.741751+0000 (oldest deadline 2024-09-10T21:30:26.641897+0000) 2024-09-10T21:31:27.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:27.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.243727+0000 front 2024-09-10T21:30:11.243753+0000 (oldest deadline 2024-09-10T21:30:37.143968+0000) 2024-09-10T21:31:27.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:27.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:18.845188+0000 front 2024-09-10T21:30:18.845347+0000 (oldest deadline 2024-09-10T21:30:42.345202+0000) 2024-09-10T21:31:27.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:27.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6814 osd.3 since back 2024-09-10T21:30:28.747106+0000 front 2024-09-10T21:30:28.747169+0000 (oldest deadline 2024-09-10T21:30:51.047104+0000) 2024-09-10T21:31:27.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:27.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:37.448373+0000 front 2024-09-10T21:30:37.448337+0000 (oldest deadline 2024-09-10T21:31:01.548722+0000) 2024-09-10T21:31:27.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:27.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:46.150169+0000 front 2024-09-10T21:30:46.150151+0000 (oldest deadline 2024-09-10T21:31:11.350671+0000) 2024-09-10T21:31:27.561 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10[60029]: 2024-09-10T21:31:27.111+0000 7fde96d85640 -1 osd.10 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:52.451883+0000 front 2024-09-10T21:30:52.451971+0000 (oldest deadline 2024-09-10T21:31:15.951843+0000) 2024-09-10T21:31:27.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:27.143+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:27.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:27.143+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:27.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:27.143+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:27.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:27.143+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:27.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:27.143+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:27.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:27.143+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:27.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:27 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:27.143+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:28.099 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:27 smithi104 podman[75500]: 2024-09-10 21:31:27.822476846 +0000 UTC m=+5.670102363 container died 91dc5f956bae34f1dd1bb7a740435ac07d268039558302a5380a044b5f91640c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T21:31:28.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:28.140+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:28.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:28.140+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:28.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:28.140+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:28.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:28.140+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:28.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:28.140+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:28.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:28.140+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:28.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:28 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:28.140+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:28.473 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:28 smithi104 podman[75500]: 2024-09-10 21:31:28.196007818 +0000 UTC m=+6.043633337 container cleanup 91dc5f956bae34f1dd1bb7a740435ac07d268039558302a5380a044b5f91640c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T21:31:28.473 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:28 smithi104 bash[75500]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10 2024-09-10T21:31:28.782 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:28 smithi104 podman[75516]: 2024-09-10 21:31:28.472156662 +0000 UTC m=+0.651284096 container remove 91dc5f956bae34f1dd1bb7a740435ac07d268039558302a5380a044b5f91640c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T21:31:29.060 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:28 smithi104 podman[75602]: 2024-09-10 21:31:28.683725583 +0000 UTC m=+0.015856490 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:31:29.060 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:28 smithi104 podman[75602]: 2024-09-10 21:31:28.781146697 +0000 UTC m=+0.113277635 container create 62c8b9671a167dbd0c257a33ca5627d85975a75fa6ef1421cb0faa09b880ff2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T21:31:29.061 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:28 smithi104 podman[75602]: 2024-09-10 21:31:28.983468175 +0000 UTC m=+0.315599072 container init 62c8b9671a167dbd0c257a33ca5627d85975a75fa6ef1421cb0faa09b880ff2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T21:31:29.061 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:28 smithi104 podman[75602]: 2024-09-10 21:31:28.98675836 +0000 UTC m=+0.318889257 container start 62c8b9671a167dbd0c257a33ca5627d85975a75fa6ef1421cb0faa09b880ff2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:31:29.473 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:29 smithi104 podman[75602]: 2024-09-10 21:31:29.066473489 +0000 UTC m=+0.398604389 container attach 62c8b9671a167dbd0c257a33ca5627d85975a75fa6ef1421cb0faa09b880ff2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T21:31:29.473 INFO:journalctl@ceph.osd.10.smithi104.stdout:Sep 10 21:31:29 smithi104 podman[75602]: 2024-09-10 21:31:29.173898594 +0000 UTC m=+0.506029490 container died 62c8b9671a167dbd0c257a33ca5627d85975a75fa6ef1421cb0faa09b880ff2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T21:31:29.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:29.151+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:29.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:29.151+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:29.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:29.151+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:29.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:29.151+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:29.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:29.151+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:29.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:29.151+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:29.642 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:29.151+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:29.746 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.10.service' 2024-09-10T21:31:29.789 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:29.789 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-10T21:31:29.789 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-10T21:31:29.789 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.11 2024-09-10T21:31:30.126 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:29 smithi174 systemd[1]: Stopping Ceph osd.11 for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:30.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.124+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:30.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.124+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:30.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.124+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:30.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.124+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:30.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.124+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:30.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.124+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:30.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.124+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:30.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.128+0000 7f9d155cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:30.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.128+0000 7f9d155cc640 -1 osd.11 80 *** Got signal Terminated *** 2024-09-10T21:31:30.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:30 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:30.128+0000 7f9d155cc640 -1 osd.11 80 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T21:31:31.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:31.127+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:31.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:31.127+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:31.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:31.127+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:31.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:31.127+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:31.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:31.127+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:31.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:31.127+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:31.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:31 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:31.127+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:32.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:32.101+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:32.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:32.101+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:32.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:32.101+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:32.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:32.101+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:32.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:32.101+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:32.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:32.101+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:32.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:32 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:32.101+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:33.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.076+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:33.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.076+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:33.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.076+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:33.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.076+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:33.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.076+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:33.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.077+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:33.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.077+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:33.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:33 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:33.077+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-09-10T21:31:07.559772+0000 front 2024-09-10T21:31:07.559795+0000 (oldest deadline 2024-09-10T21:31:32.859797+0000) 2024-09-10T21:31:34.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:34.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:34.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:34.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:34.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:34.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:34.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:34.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:34 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:34.094+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-09-10T21:31:07.559772+0000 front 2024-09-10T21:31:07.559795+0000 (oldest deadline 2024-09-10T21:31:32.859797+0000) 2024-09-10T21:31:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-09-10T21:30:04.148938+0000 front 2024-09-10T21:30:04.148805+0000 (oldest deadline 2024-09-10T21:30:25.848827+0000) 2024-09-10T21:31:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6804 osd.1 since back 2024-09-10T21:30:11.051046+0000 front 2024-09-10T21:30:11.050960+0000 (oldest deadline 2024-09-10T21:30:34.551071+0000) 2024-09-10T21:31:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-09-10T21:30:20.452025+0000 front 2024-09-10T21:30:20.452052+0000 (oldest deadline 2024-09-10T21:30:46.352230+0000) 2024-09-10T21:31:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6812 osd.4 since back 2024-09-10T21:30:36.656112+0000 front 2024-09-10T21:30:36.656227+0000 (oldest deadline 2024-09-10T21:31:00.156141+0000) 2024-09-10T21:31:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-09-10T21:30:44.857040+0000 front 2024-09-10T21:30:44.857074+0000 (oldest deadline 2024-09-10T21:31:08.357013+0000) 2024-09-10T21:31:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.33:6822 osd.6 since back 2024-09-10T21:30:54.658645+0000 front 2024-09-10T21:30:54.658594+0000 (oldest deadline 2024-09-10T21:31:18.758414+0000) 2024-09-10T21:31:35.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.104:6820 osd.7 since back 2024-09-10T21:30:58.758987+0000 front 2024-09-10T21:30:58.759105+0000 (oldest deadline 2024-09-10T21:31:23.458876+0000) 2024-09-10T21:31:35.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11[61039]: 2024-09-10T21:31:35.115+0000 7f9d11be1640 -1 osd.11 80 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-09-10T21:31:07.559772+0000 front 2024-09-10T21:31:07.559795+0000 (oldest deadline 2024-09-10T21:31:32.859797+0000) 2024-09-10T21:31:35.797 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 podman[75241]: 2024-09-10 21:31:35.533145504 +0000 UTC m=+5.549015642 container died d8d215bfd1d4f619fd8044139b074f0a77f32536cf0e47cbf60bc49c9b6dffd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:31:36.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 podman[75241]: 2024-09-10 21:31:35.894859614 +0000 UTC m=+5.910729765 container cleanup d8d215bfd1d4f619fd8044139b074f0a77f32536cf0e47cbf60bc49c9b6dffd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-10T21:31:36.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:35 smithi174 bash[75241]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11 2024-09-10T21:31:36.544 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:36 smithi174 podman[75258]: 2024-09-10 21:31:36.235656323 +0000 UTC m=+0.699656692 container remove d8d215bfd1d4f619fd8044139b074f0a77f32536cf0e47cbf60bc49c9b6dffd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:31:36.795 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:36 smithi174 podman[75347]: 2024-09-10 21:31:36.44601296 +0000 UTC m=+0.015841754 image pull 589dd18248e494ba411dc838aa4ae0e5e569213f08fcd7a2cf22cd3ef7e3c480 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e 2024-09-10T21:31:36.796 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:36 smithi174 podman[75347]: 2024-09-10 21:31:36.555251516 +0000 UTC m=+0.125080359 container create 6983321a3949ac8cf79aa50472f39f7959242ff804034bf68579ea3b8e822403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True) 2024-09-10T21:31:36.796 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:36 smithi174 podman[75347]: 2024-09-10 21:31:36.791020229 +0000 UTC m=+0.360849035 container init 6983321a3949ac8cf79aa50472f39f7959242ff804034bf68579ea3b8e822403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:31:37.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:36 smithi174 podman[75347]: 2024-09-10 21:31:36.794483235 +0000 UTC m=+0.364312023 container start 6983321a3949ac8cf79aa50472f39f7959242ff804034bf68579ea3b8e822403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T21:31:37.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:36 smithi174 podman[75347]: 2024-09-10 21:31:36.907610811 +0000 UTC m=+0.477439610 container attach 6983321a3949ac8cf79aa50472f39f7959242ff804034bf68579ea3b8e822403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T21:31:37.142 INFO:journalctl@ceph.osd.11.smithi174.stdout:Sep 10 21:31:36 smithi174 podman[75347]: 2024-09-10 21:31:36.983936305 +0000 UTC m=+0.553765095 container died 6983321a3949ac8cf79aa50472f39f7959242ff804034bf68579ea3b8e822403 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-osd-11-deactivate, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:31:37.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@osd.11.service' 2024-09-10T21:31:37.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:37.603 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-10T21:31:37.604 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-10T21:31:37.604 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.a 2024-09-10T21:31:37.919 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:31:37 smithi033 systemd[1]: Stopping Ceph mds.a for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:38.175 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:31:37 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a[74391]: 2024-09-10T21:31:37.916+0000 7f367294e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:38.175 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:31:37 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a[74391]: 2024-09-10T21:31:37.916+0000 7f367294e640 -1 mds.a *** got signal Terminated *** 2024-09-10T21:31:40.136 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:31:39 smithi033 podman[133716]: 2024-09-10 21:31:39.852276493 +0000 UTC m=+2.039931263 container died be3eb55e3891b24a515e796b87b173a076dc1cdb4494108c48f6aa50079c5f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T21:31:40.425 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:31:40 smithi033 podman[133716]: 2024-09-10 21:31:40.325179917 +0000 UTC m=+2.512834685 container cleanup be3eb55e3891b24a515e796b87b173a076dc1cdb4494108c48f6aa50079c5f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, GIT_CLEAN=True) 2024-09-10T21:31:40.425 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:31:40 smithi033 bash[133716]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a 2024-09-10T21:31:40.758 INFO:journalctl@ceph.mds.a.smithi033.stdout:Sep 10 21:31:40 smithi033 podman[133728]: 2024-09-10 21:31:40.618537151 +0000 UTC m=+0.761499487 container remove be3eb55e3891b24a515e796b87b173a076dc1cdb4494108c48f6aa50079c5f71 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2) 2024-09-10T21:31:40.763 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.a.service' 2024-09-10T21:31:40.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:40.802 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-10T21:31:40.802 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-10T21:31:40.803 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.d 2024-09-10T21:31:41.173 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:31:40 smithi033 systemd[1]: Stopping Ceph mds.d for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:41.425 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:31:41 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d[73203]: 2024-09-10T21:31:41.170+0000 7f0650b19640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:41.425 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:31:41 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d[73203]: 2024-09-10T21:31:41.170+0000 7f0650b19640 -1 mds.d *** got signal Terminated *** 2024-09-10T21:31:45.115 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:31:44 smithi033 podman[133847]: 2024-09-10 21:31:44.841077898 +0000 UTC m=+3.782019699 container died b73e89ef5e364920ffa8f6266d3afe84c823279dc5086e48e31b4deb005c7e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T21:31:45.425 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:31:45 smithi033 podman[133847]: 2024-09-10 21:31:45.321204353 +0000 UTC m=+4.262146152 container cleanup b73e89ef5e364920ffa8f6266d3afe84c823279dc5086e48e31b4deb005c7e5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:31:45.425 INFO:journalctl@ceph.mds.d.smithi033.stdout:Sep 10 21:31:45 smithi033 bash[133847]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-d 2024-09-10T21:31:45.769 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.d.service' 2024-09-10T21:31:45.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:45.805 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-10T21:31:45.806 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-10T21:31:45.806 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.g 2024-09-10T21:31:46.152 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:31:45 smithi033 systemd[1]: Stopping Ceph mds.g for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:46.425 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:31:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g[74004]: 2024-09-10T21:31:46.149+0000 7fcd5deed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:46.425 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:31:46 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g[74004]: 2024-09-10T21:31:46.149+0000 7fcd5deed640 -1 mds.g *** got signal Terminated *** 2024-09-10T21:31:48.127 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:31:47 smithi033 podman[133967]: 2024-09-10 21:31:47.853954792 +0000 UTC m=+1.810227379 container died dd8736cf14872c8885025478e38b596476becad1824d2bea9f1e68a9ff13179b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:31:48.425 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:31:48 smithi033 podman[133967]: 2024-09-10 21:31:48.224948459 +0000 UTC m=+2.181221028 container cleanup dd8736cf14872c8885025478e38b596476becad1824d2bea9f1e68a9ff13179b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4) 2024-09-10T21:31:48.425 INFO:journalctl@ceph.mds.g.smithi033.stdout:Sep 10 21:31:48 smithi033 bash[133967]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-g 2024-09-10T21:31:48.662 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.g.service' 2024-09-10T21:31:48.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:48.696 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-10T21:31:48.697 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-10T21:31:48.697 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.j 2024-09-10T21:31:49.175 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:31:48 smithi033 systemd[1]: Stopping Ceph mds.j for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:49.175 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:31:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j[72794]: 2024-09-10T21:31:49.011+0000 7f35d4a08640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:49.175 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:31:49 smithi033 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j[72794]: 2024-09-10T21:31:49.011+0000 7f35d4a08640 -1 mds.j *** got signal Terminated *** 2024-09-10T21:31:53.306 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:31:53 smithi033 podman[134087]: 2024-09-10 21:31:53.027463556 +0000 UTC m=+4.108860638 container died 99e5230000fcd4c2ff0c69b5a7b639e52cedfbc32f4ff7f08cf02f6640dee5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T21:31:53.664 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:31:53 smithi033 podman[134087]: 2024-09-10 21:31:53.403649367 +0000 UTC m=+4.485046441 container cleanup 99e5230000fcd4c2ff0c69b5a7b639e52cedfbc32f4ff7f08cf02f6640dee5d6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T21:31:53.664 INFO:journalctl@ceph.mds.j.smithi033.stdout:Sep 10 21:31:53 smithi033 bash[134087]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-j 2024-09-10T21:31:53.832 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.j.service' 2024-09-10T21:31:53.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:53.867 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-10T21:31:53.867 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-10T21:31:53.867 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.b 2024-09-10T21:31:54.164 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:31:53 smithi104 systemd[1]: Stopping Ceph mds.b for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:54.415 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:31:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: 2024-09-10T21:31:54.161+0000 7f71da15a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:54.415 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:31:54 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b[65657]: 2024-09-10T21:31:54.161+0000 7f71da15a640 -1 mds.b *** got signal Terminated *** 2024-09-10T21:31:56.310 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:31:55 smithi104 podman[75707]: 2024-09-10 21:31:55.99434747 +0000 UTC m=+1.935134453 container died da5a9bb67f8f19856f42fcb39ba90695a2d436a59fb773ce68f070e03d1283c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T21:31:56.589 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:31:56 smithi104 podman[75707]: 2024-09-10 21:31:56.311828499 +0000 UTC m=+2.252615465 container cleanup da5a9bb67f8f19856f42fcb39ba90695a2d436a59fb773ce68f070e03d1283c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T21:31:56.589 INFO:journalctl@ceph.mds.b.smithi104.stdout:Sep 10 21:31:56 smithi104 bash[75707]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-b 2024-09-10T21:31:56.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.b.service' 2024-09-10T21:31:56.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:31:56.779 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-10T21:31:56.779 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-10T21:31:56.779 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.e 2024-09-10T21:31:57.118 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:31:56 smithi104 systemd[1]: Stopping Ceph mds.e for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:31:57.560 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:31:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e[65058]: 2024-09-10T21:31:57.114+0000 7fead5837640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:31:57.561 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:31:57 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e[65058]: 2024-09-10T21:31:57.114+0000 7fead5837640 -1 mds.e *** got signal Terminated *** 2024-09-10T21:32:00.203 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:31:59 smithi104 podman[75841]: 2024-09-10 21:31:59.882320264 +0000 UTC m=+2.879635876 container died 97bde8c1f2eec9c03ca2a15b0bbd8b073ab78ed610e1acbcdcb7c60584741a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-10T21:32:00.535 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:32:00 smithi104 podman[75841]: 2024-09-10 21:32:00.298835704 +0000 UTC m=+3.296151317 container cleanup 97bde8c1f2eec9c03ca2a15b0bbd8b073ab78ed610e1acbcdcb7c60584741a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T21:32:00.535 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:32:00 smithi104 bash[75841]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e 2024-09-10T21:32:00.785 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:32:00 smithi104 podman[75855]: 2024-09-10 21:32:00.533299194 +0000 UTC m=+0.641008456 container remove 97bde8c1f2eec9c03ca2a15b0bbd8b073ab78ed610e1acbcdcb7c60584741a44 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-e, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T21:32:00.785 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:32:00 smithi104 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.e.service: Deactivated successfully. 2024-09-10T21:32:00.785 INFO:journalctl@ceph.mds.e.smithi104.stdout:Sep 10 21:32:00 smithi104 systemd[1]: Stopped Ceph mds.e for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:32:00.787 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.e.service' 2024-09-10T21:32:00.821 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:32:00.821 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-10T21:32:00.821 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-10T21:32:00.822 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.h 2024-09-10T21:32:01.060 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:00 smithi104 systemd[1]: Stopping Ceph mds.h for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:32:01.560 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h[65360]: 2024-09-10T21:32:01.176+0000 7f5dea87e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:32:01.561 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:01 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h[65360]: 2024-09-10T21:32:01.176+0000 7f5dea87e640 -1 mds.h *** got signal Terminated *** 2024-09-10T21:32:04.212 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:03 smithi104 podman[75962]: 2024-09-10 21:32:03.878259294 +0000 UTC m=+2.840988077 container died 4f038929f650d89ab48b569171516a53f8739e98d70d61e144676d0d78ee6566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:32:04.504 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:04 smithi104 podman[75962]: 2024-09-10 21:32:04.310192253 +0000 UTC m=+3.272921024 container cleanup 4f038929f650d89ab48b569171516a53f8739e98d70d61e144676d0d78ee6566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T21:32:04.505 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:04 smithi104 bash[75962]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h 2024-09-10T21:32:04.744 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.h.service' 2024-09-10T21:32:04.765 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:04 smithi104 podman[75974]: 2024-09-10 21:32:04.503128852 +0000 UTC m=+0.619817670 container remove 4f038929f650d89ab48b569171516a53f8739e98d70d61e144676d0d78ee6566 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T21:32:04.766 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:04 smithi104 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.h.service: Deactivated successfully. 2024-09-10T21:32:04.766 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:04 smithi104 systemd[1]: Stopped Ceph mds.h for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:32:04.766 INFO:journalctl@ceph.mds.h.smithi104.stdout:Sep 10 21:32:04 smithi104 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.h.service: Consumed 12min 13.485s CPU time. 2024-09-10T21:32:04.777 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:32:04.777 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-10T21:32:04.777 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-10T21:32:04.777 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.k 2024-09-10T21:32:05.060 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:32:04 smithi104 systemd[1]: Stopping Ceph mds.k for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:32:05.430 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:32:05 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k[64747]: 2024-09-10T21:32:05.122+0000 7fc7f5872640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:32:05.430 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:32:05 smithi104 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k[64747]: 2024-09-10T21:32:05.122+0000 7fc7f5872640 -1 mds.k *** got signal Terminated *** 2024-09-10T21:32:08.310 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:32:07 smithi104 podman[76081]: 2024-09-10 21:32:07.879233748 +0000 UTC m=+2.880245782 container died f6c1b5f4684a11348d988c44ef7f27db88fd54049321e4a172fb477bf10b122d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T21:32:08.774 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:32:08 smithi104 podman[76081]: 2024-09-10 21:32:08.514127015 +0000 UTC m=+3.515139058 container cleanup f6c1b5f4684a11348d988c44ef7f27db88fd54049321e4a172fb477bf10b122d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T21:32:08.774 INFO:journalctl@ceph.mds.k.smithi104.stdout:Sep 10 21:32:08 smithi104 bash[76081]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-k 2024-09-10T21:32:08.937 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.k.service' 2024-09-10T21:32:08.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:32:08.969 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-10T21:32:08.970 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-10T21:32:08.970 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.c 2024-09-10T21:32:09.269 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:32:09 smithi174 systemd[1]: Stopping Ceph mds.c for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:32:09.540 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:32:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c[66398]: 2024-09-10T21:32:09.267+0000 7fe753906640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:32:09.540 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:32:09 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c[66398]: 2024-09-10T21:32:09.267+0000 7fe753906640 -1 mds.c *** got signal Terminated *** 2024-09-10T21:32:12.179 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:32:11 smithi174 podman[75451]: 2024-09-10 21:32:11.911959032 +0000 UTC m=+2.747398578 container died b261a3a6c7b86220561791759b778a3b396a66dbc090e1fac4f8962cd0c54884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T21:32:12.562 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:32:12 smithi174 podman[75451]: 2024-09-10 21:32:12.276235716 +0000 UTC m=+3.111675251 container cleanup b261a3a6c7b86220561791759b778a3b396a66dbc090e1fac4f8962cd0c54884 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:32:12.562 INFO:journalctl@ceph.mds.c.smithi174.stdout:Sep 10 21:32:12 smithi174 bash[75451]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-c 2024-09-10T21:32:12.712 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.c.service' 2024-09-10T21:32:12.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:32:12.748 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-10T21:32:12.748 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-10T21:32:12.748 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.f 2024-09-10T21:32:13.141 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:32:12 smithi174 systemd[1]: Stopping Ceph mds.f for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:32:13.142 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:32:13 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f[66095]: 2024-09-10T21:32:13.071+0000 7f7e9a368640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:32:13.142 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:32:13 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f[66095]: 2024-09-10T21:32:13.071+0000 7f7e9a368640 -1 mds.f *** got signal Terminated *** 2024-09-10T21:32:16.186 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:32:15 smithi174 podman[75585]: 2024-09-10 21:32:15.895932184 +0000 UTC m=+2.933582390 container died 121a532764b9dcbc0cecd931b57ac216a95df69195bbfa15031edd6ba606dbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T21:32:16.549 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:32:16 smithi174 podman[75585]: 2024-09-10 21:32:16.28852635 +0000 UTC m=+3.326176548 container cleanup 121a532764b9dcbc0cecd931b57ac216a95df69195bbfa15031edd6ba606dbbb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T21:32:16.549 INFO:journalctl@ceph.mds.f.smithi174.stdout:Sep 10 21:32:16 smithi174 bash[75585]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-f 2024-09-10T21:32:16.676 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.f.service' 2024-09-10T21:32:16.711 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:32:16.711 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-10T21:32:16.711 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-10T21:32:16.711 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.i 2024-09-10T21:32:16.891 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:16 smithi174 systemd[1]: Stopping Ceph mds.i for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:32:17.392 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:17 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i[66698]: 2024-09-10T21:32:17.032+0000 7fb44b5c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:32:17.392 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:17 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i[66698]: 2024-09-10T21:32:17.032+0000 7fb44b5c0640 -1 mds.i *** got signal Terminated *** 2024-09-10T21:32:20.185 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:19 smithi174 podman[75704]: 2024-09-10 21:32:19.894845215 +0000 UTC m=+2.968040597 container died d48521b7c06ec6a08e9dcc8ae56327ac1b8421a9060862537289141e0d440ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T21:32:20.443 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:20 smithi174 podman[75704]: 2024-09-10 21:32:20.24983796 +0000 UTC m=+3.323033329 container cleanup d48521b7c06ec6a08e9dcc8ae56327ac1b8421a9060862537289141e0d440ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T21:32:20.444 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:20 smithi174 bash[75704]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i 2024-09-10T21:32:20.673 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.i.service' 2024-09-10T21:32:20.694 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:20 smithi174 podman[75717]: 2024-09-10 21:32:20.442722991 +0000 UTC m=+0.543716119 container remove d48521b7c06ec6a08e9dcc8ae56327ac1b8421a9060862537289141e0d440ede (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T21:32:20.694 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:20 smithi174 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-10T21:32:20.694 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:20 smithi174 systemd[1]: Stopped Ceph mds.i for a07b33bc-6fb6-11ef-bcea-c7b262605968. 2024-09-10T21:32:20.694 INFO:journalctl@ceph.mds.i.smithi174.stdout:Sep 10 21:32:20 smithi174 systemd[1]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.i.service: Consumed 12min 28.758s CPU time. 2024-09-10T21:32:20.705 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:32:20.706 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-10T21:32:20.706 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-10T21:32:20.706 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.l 2024-09-10T21:32:21.023 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:32:20 smithi174 systemd[1]: Stopping Ceph mds.l for a07b33bc-6fb6-11ef-bcea-c7b262605968... 2024-09-10T21:32:21.391 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:32:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l[65793]: 2024-09-10T21:32:21.019+0000 7ff7b9161640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T21:32:21.392 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:32:21 smithi174 ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l[65793]: 2024-09-10T21:32:21.019+0000 7ff7b9161640 -1 mds.l *** got signal Terminated *** 2024-09-10T21:32:24.142 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:32:23 smithi174 podman[75827]: 2024-09-10 21:32:23.886472827 +0000 UTC m=+2.970014151 container died 6a4a62d2e96a4ae8db5bd85f55330ec43a10096cc2baf9b7230abb8f152f2f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l, CEPH_POINT_RELEASE=, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T21:32:24.530 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:32:24 smithi174 podman[75827]: 2024-09-10 21:32:24.262245106 +0000 UTC m=+3.345786436 container cleanup 6a4a62d2e96a4ae8db5bd85f55330ec43a10096cc2baf9b7230abb8f152f2f43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:15494b9bca13f9d7088cd5c92cad00398197153ea02321fd69a7c31aa379887e, name=ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-hemanth-testing-2024-09-10-0723-8f219c4, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T21:32:24.531 INFO:journalctl@ceph.mds.l.smithi174.stdout:Sep 10 21:32:24 smithi174 bash[75827]: ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968-mds-l 2024-09-10T21:32:24.688 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a07b33bc-6fb6-11ef-bcea-c7b262605968@mds.l.service' 2024-09-10T21:32:24.720 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T21:32:24.720 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-10T21:32:24.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T21:32:24.935 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T21:32:27.552 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T21:32:27.755 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T21:32:30.617 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T21:32:30.819 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T21:32:33.410 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:32:33.444 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:32:33.477 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T21:32:33.506 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T21:32:33.509 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi033/crash 2024-09-10T21:32:33.510 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash -- . 2024-09-10T21:32:33.559 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T21:32:33.559 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-09-10T21:32:33.560 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi104/crash 2024-09-10T21:32:33.561 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash -- . 2024-09-10T21:32:33.599 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T21:32:33.600 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-09-10T21:32:33.602 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi174/crash 2024-09-10T21:32:33.602 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash -- . 2024-09-10T21:32:33.652 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T21:32:33.652 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-09-10T21:32:33.653 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T21:32:33.653 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T21:32:33.738 INFO:tasks.cephadm:Compressing logs... 2024-09-10T21:32:33.738 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:32:33.741 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:32:33.744 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:32:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory --verbose 2024-09-10T21:32:33.780 INFO:teuthology.orchestra.run.smithi033.stderr: -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T21:32:33.780 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-10T21:32:33.781 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T21:32:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T21:32:33.784 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T21:32:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-10T21:32:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T21:32:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T21:32:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-10T21:32:33.785 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log: /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T21:32:33.786 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T21:32:33.788 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.5% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T21:32:33.800 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T21:32:33.800 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T21:32:33.800 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-10T21:32:33.801 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T21:32:33.801 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T21:32:33.801 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-10T21:32:33.801 INFO:teuthology.orchestra.run.smithi174.stderr:gzip/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log: -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T21:32:33.801 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-10T21:32:33.802 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-10T21:32:33.803 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T21:32:33.804 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T21:32:33.804 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-10T21:32:33.804 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T21:32:33.805 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T21:32:33.805 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T21:32:33.805 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-10T21:32:33.805 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-10T21:32:33.805 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T21:32:33.805 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-10T21:32:33.806 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T21:32:33.862 INFO:teuthology.orchestra.run.smithi033.stderr: 95.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T21:32:33.873 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.f.log: /var/log/ceph/cephadm.log: 83.6% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T21:32:33.992 INFO:teuthology.orchestra.run.smithi174.stderr: 91.0% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T21:32:34.018 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 93.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T21:32:34.030 INFO:teuthology.orchestra.run.smithi033.stderr: 83.2% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T21:32:34.070 INFO:teuthology.orchestra.run.smithi104.stderr: 91.0% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T21:32:34.076 INFO:teuthology.orchestra.run.smithi174.stderr: 93.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-10T21:32:34.094 INFO:teuthology.orchestra.run.smithi033.stderr: 91.1% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T21:32:34.120 INFO:teuthology.orchestra.run.smithi104.stderr: 87.5% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T21:32:34.137 INFO:teuthology.orchestra.run.smithi033.stderr: 90.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T21:32:34.174 INFO:teuthology.orchestra.run.smithi174.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T21:32:34.320 INFO:teuthology.orchestra.run.smithi104.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T21:32:34.355 INFO:teuthology.orchestra.run.smithi104.stderr: 83.7% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T21:32:34.361 INFO:teuthology.orchestra.run.smithi104.stderr: 90.5% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T21:32:34.466 INFO:teuthology.orchestra.run.smithi174.stderr: 87.5% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T21:32:34.466 INFO:teuthology.orchestra.run.smithi174.stderr: 90.5% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T21:32:34.693 INFO:teuthology.orchestra.run.smithi033.stderr: 95.1% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-10T21:32:34.812 INFO:teuthology.orchestra.run.smithi033.stderr: 95.1% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-10T21:32:37.078 INFO:teuthology.orchestra.run.smithi033.stderr: 90.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T21:32:40.811 INFO:teuthology.orchestra.run.smithi033.stderr: 90.1% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T21:33:01.987 INFO:teuthology.orchestra.run.smithi174.stderr: 91.6% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T21:33:20.647 INFO:teuthology.orchestra.run.smithi033.stderr: 95.1% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-10T21:33:25.048 INFO:teuthology.orchestra.run.smithi033.stderr: 95.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-10T21:34:34.896 INFO:teuthology.orchestra.run.smithi033.stderr: 91.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T21:34:55.803 INFO:teuthology.orchestra.run.smithi104.stderr: 93.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T21:35:30.759 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T21:35:40.021 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-10T21:35:58.552 INFO:teuthology.orchestra.run.smithi033.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T21:35:58.554 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-09-10T21:35:58.554 INFO:teuthology.orchestra.run.smithi033.stderr:real 3m24.798s 2024-09-10T21:35:58.554 INFO:teuthology.orchestra.run.smithi033.stderr:user 4m23.307s 2024-09-10T21:35:58.554 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m8.878s 2024-09-10T21:39:28.926 INFO:teuthology.orchestra.run.smithi104.stderr: 95.2% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-10T21:39:48.578 INFO:teuthology.orchestra.run.smithi104.stderr: 95.4% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-10T21:41:29.280 INFO:teuthology.orchestra.run.smithi174.stderr: 95.2% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-10T21:45:13.508 INFO:teuthology.orchestra.run.smithi104.stderr: 91.0% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T21:45:49.386 INFO:teuthology.orchestra.run.smithi104.stderr: 91.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T21:47:55.794 INFO:teuthology.orchestra.run.smithi104.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-10T21:51:16.827 INFO:teuthology.orchestra.run.smithi104.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T21:52:23.952 INFO:teuthology.orchestra.run.smithi174.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-10T21:54:57.219 INFO:teuthology.orchestra.run.smithi174.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T21:56:10.491 INFO:teuthology.orchestra.run.smithi174.stderr: 90.7% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T21:56:54.842 INFO:teuthology.orchestra.run.smithi174.stderr: 90.9% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-10T22:09:25.738 INFO:teuthology.orchestra.run.smithi104.stderr: 93.1% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-10T22:10:47.931 INFO:teuthology.orchestra.run.smithi104.stderr: 93.3% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-10T22:10:47.977 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-09-10T22:10:47.978 INFO:teuthology.orchestra.run.smithi104.stderr:real 38m14.219s 2024-09-10T22:10:47.978 INFO:teuthology.orchestra.run.smithi104.stderr:user 12m58.607s 2024-09-10T22:10:47.978 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m37.988s 2024-09-10T22:12:45.838 INFO:teuthology.orchestra.run.smithi174.stderr: 93.2% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-10T22:15:18.337 INFO:teuthology.orchestra.run.smithi174.stderr: 93.1% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-10T22:15:55.963 INFO:teuthology.orchestra.run.smithi174.stderr: 93.0% -- replaced with /var/log/ceph/a07b33bc-6fb6-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-10T22:15:55.999 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-09-10T22:15:55.999 INFO:teuthology.orchestra.run.smithi174.stderr:real 43m22.240s 2024-09-10T22:15:55.999 INFO:teuthology.orchestra.run.smithi174.stderr:user 15m8.848s 2024-09-10T22:15:55.999 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m42.719s 2024-09-10T22:15:55.999 INFO:tasks.cephadm:Archiving logs... 2024-09-10T22:15:56.001 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi033/log 2024-09-10T22:15:56.002 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:16:39.983 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi104/log 2024-09-10T22:16:39.984 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:18:40.059 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi174/log 2024-09-10T22:18:40.060 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:21:00.433 INFO:tasks.cephadm:Removing cluster... 2024-09-10T22:21:00.433 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 --force 2024-09-10T22:21:00.648 INFO:teuthology.orchestra.run.smithi033.stdout:Deleting cluster with fsid: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T22:21:01.282 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 --force 2024-09-10T22:21:02.175 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T22:21:03.100 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a07b33bc-6fb6-11ef-bcea-c7b262605968 --force 2024-09-10T22:21:03.754 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: a07b33bc-6fb6-11ef-bcea-c7b262605968 2024-09-10T22:21:04.821 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T22:21:04.821 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T22:21:04.842 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T22:21:04.863 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T22:21:04.883 INFO:tasks.cephadm:Teardown complete 2024-09-10T22:21:04.883 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T22:21:04.896 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T22:21:04.896 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:21:04.899 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:21:04.906 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:21:05.064 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T22:21:05.074 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T22:21:05.074 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:21:05.077 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:21:05.080 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:21:05.093 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:21:05.094 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:21:05.095 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-09-10T22:21:05.098 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:21:05.098 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-09-10T22:21:05.098 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv01.front.sepia.ceph.com 3 8 377 226 -233us[ -190us] +/- 35ms 2024-09-10T22:21:05.098 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 +1034us[+1034us] +/- 55ms 2024-09-10T22:21:05.098 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 100 -838us[ -838us] +/- 65ms 2024-09-10T22:21:05.099 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:21:05.159 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:21:05.159 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-09-10T22:21:05.159 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 -231us[ -238us] +/- 38ms 2024-09-10T22:21:05.159 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 +1069us[+1069us] +/- 55ms 2024-09-10T22:21:05.159 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 100 -809us[ -819us] +/- 65ms 2024-09-10T22:21:05.159 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:21:05.165 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T22:21:05.165 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-09-10T22:21:05.165 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv01.front.sepia.ceph.com 3 7 377 30 -221us[ -201us] +/- 38ms 2024-09-10T22:21:05.165 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 37 +1029us[+1049us] +/- 55ms 2024-09-10T22:21:05.165 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 35 -830us[ -810us] +/- 66ms 2024-09-10T22:21:05.165 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T22:21:05.177 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T22:21:05.187 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T22:21:05.187 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T22:21:05.238 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:21:05.429 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:21:05.576 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T22:21:05.727 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:21:05.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:21:05.762 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:21:05.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:21:05.795 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-10T22:21:05.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:21:05.824 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T22:21:05.838 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T22:21:05.849 INFO:teuthology.task.internal:Duration was 5643.956569 seconds 2024-09-10T22:21:05.849 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T22:21:05.863 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T22:21:05.864 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:21:05.866 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:21:05.869 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:21:05.933 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:21:05.934 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:21:05.940 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T22:21:06.241 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T22:21:06.241 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-09-10T22:21:06.241 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:21:06.265 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-09-10T22:21:06.265 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:21:06.295 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-09-10T22:21:06.295 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:21:06.321 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T22:21:06.321 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:21:06.324 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:21:06.337 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:21:06.389 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T22:21:06.389 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:21:06.394 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:21:06.408 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:21:08.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T22:21:08.473 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T22:21:08.473 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:21:08.514 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:21:08.547 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:21:08.578 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T22:21:08.589 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:21:08.592 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:21:08.594 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:21:08.621 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-09-10T22:21:08.649 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-09-10T22:21:08.664 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-09-10T22:21:08.688 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:21:08.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:21:08.703 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:21:08.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:21:08.720 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:21:08.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:21:08.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T22:21:08.748 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T22:21:08.749 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi033 2024-09-10T22:21:08.749 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:21:08.829 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi104 2024-09-10T22:21:08.830 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:21:08.871 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899646/remote/smithi174 2024-09-10T22:21:08.871 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:21:08.914 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T22:21:08.914 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:21:08.917 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:21:08.920 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:21:08.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T22:21:08.976 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T22:21:08.977 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T22:21:08.987 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T22:21:08.987 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:21:08.990 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:21:08.992 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:21:09.003 INFO:teuthology.orchestra.run.smithi033.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:21 /home/ubuntu/cephtest 2024-09-10T22:21:09.007 INFO:teuthology.orchestra.run.smithi104.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:21 /home/ubuntu/cephtest 2024-09-10T22:21:09.021 INFO:teuthology.orchestra.run.smithi174.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:21 /home/ubuntu/cephtest 2024-09-10T22:21:09.023 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T22:21:09.034 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T22:21:09.063 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/no 6-workunit/suites/dbench}} duration: 5643.9565687179565 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T22:21:09.063 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:21:09.170 INFO:teuthology.run:pass